Is it just me? Am I using this wrong or am I asking questions that are too hard?

Here’s an example of a hallucination that happened while explaining away another hallucination I called it out on. I rarely have experiences other than these.

@mwichary

Hallucinations remain common, and the strongest mitigation is a combination of search use & "extended thinking". Even then, models may over-privilege low quality sources.

May I ask which version of ChatGPT this is from (paid or free; Thinking mode enabled or not)? Here's the result with 5.2 + Thinking mode, in case it's useful for comparison:

https://chatgpt.com/share/694cb369-b3d4-800b-8abc-e29d565910d8

@eloquence Thanks! It is useful. This was on free and I sometimes wonder how that affects things.

@eloquence Sorry, is “search use” going to Google and/or veryfing by other means?

@mwichary

OpenAI operates its own crawlers and also licenses search results (Bing, as part of their longstanding relationship with MSFT); it's undisclosed what the exact "mix" is that comprises responses.

@mwichary

In terms of free vs. paid, the free plan is heavily restricted. In the response I shared it spent 75 seconds in "thinking" mode. For more comprehensive reports they have a "Deep research" feature that can run for 5-10 minutes.

That increase does tend to improve the quality of responses, better attribution of claims to sources, etc. It does not obviate the need to verify, of course.