> In our testing, raters preferred the reports generated by Gemini Deep Research powered by 2.5 Pro over other leading deep research providers by more than a 2-to-1 margin.

Are these raters experts in the field the report was written on? Did they rate the reports on factuality, broadness, and insights?

These sort of tests (and RLHF in general) are the reason that LLMs often respond with "Great question, you are exactly right to wonder..." or "Interesting insight, I agree that...". I do not want this obsequious behavior, I want "correct answers"[0]. We need some better benchmarks when it comes to human preference.

[0]: I know there is no objective correct answers for some questions.

Has anyone tested googles functionality vs ChatGPT? I have lightly played around with it but felt that generally ChatGPTs implementation was a little more educated sounding and felt like it took whatever necessary persona well.

Just did a test last week and OpenAIs research was way better. Found 10x more sources and did an overall pretty great job

The task was to lookup information about a late distant family member who had been a prominent employee in a certain foreign government about 100 years ago

Gemini barely scratched the surface and pretty much gave up

ChatGPT on the other hand, kept building up on its research, connecting the dots and leveraging each bit of acquired information to try to find more

I stumbled across the feature a few hours ago. I had asked Gemini why there's a hole in the middle of the city of Azusa, topologically speaking. It had given me a useless tautological response: because they never annexed it. Then it offered to create a research report and I agreed. Five minutes later I got a notification on my mobile that the report was ready. It had 120 sources including assessor's maps, historical maps, court cases, and narrative articles. The text that went along with it was too verbose and still contained paragraphs of vague stuff, but it had key information linking the Mexican land grants, the founding of the city, and other events of history. Very impressive.