我们连续两天向 AI 提出相同的 210 个购买问题,84% 的答案发生了变化。
We asked AI the same 210 buying questions 2 days in a row,84% of answers changed

原始链接: https://supercite.ai/blog/ai-answers-change-every-day

AI 推荐每天都不稳定,因为每次回答都基于新抓取的少量页面。例如,Less Annoying CRM 一天排名第一,第二天因引擎读取了不同来源而消失,HubSpot 则上升。品牌重叠高于页面重叠,尤其在领先品牌中,因为顶级品牌出现在许多榜单上。Gemini 变化最大(95% 的回答),AI Overviews 变化最小(67%);ChatGPT 最常保留其第一推荐。类别也不同:会计软件稳定,而 AI 可见度工具和跑鞋则不稳定。实用建议:不要依赖单张截图;跟踪品牌在重复提问和不同引擎中出现的频率;通过进入更多被引用的“最佳 X”榜单提高几率;监测被引用的来源;依据一两周的趋势而非单次结果判断。局限:仅两天、自动化品牌匹配、存在噪声。

一位作者在两天内测试了 ChatGPT、Gemini、Google AI Overviews 和 Google AI Mode 对 210 个购买相关问题的回答,发现 84% 的回答品牌发生变化;Gemini 为 95%。作者将此归因于每次引用了新的网页:第一天页面中有不到一半重复,因此品牌提及发生变化。大品牌因为出现在更多页面而持续存在,小品牌则波动。 评论者就这是否令人惊讶展开争论。有人说,宽泛的提示和 LLM 的随机性自然会导致变化,而可见性和 SEO 影响哪些来源被读取。也有人认为温度可能无法保证确定性。批评者还质疑实验方法,指出时间与随机性相混淆,并建议进行重复的同步查询和统计列表比较方法。总体而言,AI 推荐输出似乎不稳定,并且高度依赖每次检索到的来源。
相关文章

原文

On day one, Less Annoying CRM was the first pick, called out for its "zero-frills simplicity". AI Mode had read one page to answer: Zapier's list of the best small business CRMs.

On day two, Less Annoying CRM wasn't mentioned at all. HubSpot was first. This time AI Mode had read two different pages: a guide on Slack's blog and HubSpot's own product page.

Nothing about the CRMs changed overnight. What changed was the pages the engine happened to read. Swap the pages and you swap the answer.

That's the pattern behind the numbers. An AI answer isn't a fixed ranking. It's a summary of a few pages, picked fresh each time.

Your position today predicts tomorrow

Not every brand moves the same amount. The higher a brand ranked in an answer, the more likely it was to be named again the next day.

Brands that show up in a lot of answers in their category kept 73% of their spots. Brands that show up less kept 51%. The difference makes sense: a leader is on most of the pages AI reads, so whichever pages it picks, the leader is on them. A smaller brand is on a few. When the engine picks other pages, the smaller brand drops out.

If you're not a leader, this is the part that matters. Your visibility depends on which pages the engine reads that day. The way to make it stable is to be on more of those pages.

Each engine moves differently

ChatGPTGeminiAI OverviewsAI Mode
Answers that changed88%95%67%84%
Brands gone the next day23%33%24%28%
Same #1 brand both days71%57%66%58%
Same cited pages40%41%47%39%

Gemini changed the most. On 95% of questions, its list of brands was different the next day, and it dropped a third of the brands it had named. Google AI Overviews changed the least, but even there two out of three answers moved.

ChatGPT was the most loyal to its first pick: the same #1 brand 71% of the time. That fits what we saw yesterday. ChatGPT often starts with a shortlist from memory before it reads anything, so its top picks depend less on the pages it finds that day.

The pages change faster than the brands

Only 42% of the pages cited on day one were cited again on day two. Brands were more stable than that (61% overlap), mostly because the same leaders appear on many different pages.

This connects the two studies. Yesterday we found that 90% of recommended brands were on a list the engine cited. Today we see that the set of cited lists rotates every day. Put together:

  • Being on one list gets you into some answers.
  • Being on many lists gets you into most of them, because whichever lists the engine reads, you're on them.

Some categories move more than others

Most stableSame brandsSame #1
Accounting software79%93%
VPNs75%75%
Travel insurance69%53%
Least stableSame brandsSame #1
AI search visibility tools48%52%
Mattresses51%56%
Running shoes53%33%

Accounting software barely moved. QuickBooks, Xero, Wave and FreshBooks come up in most answers on every engine, so there's little room for change.

The least stable category was our own: AI search visibility tools. That makes sense. It's a new category with many young companies and no settled set of lists. Running shoes had the most unstable #1: the top pick changed on two out of three questions.

What this means for you

Don't trust a single check. A screenshot of ChatGPT recommending you, or ignoring you, is one sample. 84% of answers changed in a day. You need to ask the same questions many times and look at how often you come up.

Measure share, not yes or no. A single answer named about 6 brands. Across eight answers to the same question (four engines, two days), 14 different brands came up. 30% of them were named only once. Only about 2 brands per question were named in all eight. The useful number is how often you're named across many answers, and whether that number is going up.

If you're not the leader, get on more pages. Smaller brands dropped out half the time. The fix is the same one from yesterday's study: find the lists AI reads for your buyers' questions and get onto the ones that leave you out. The more of them name you, the less it matters which ones the engine reads.

Check the sources, not just the answer. The pages rotate even faster than the brands. A list that was cited last week may not be cited today, and a new one may have appeared. Watch which pages are being read, because that's where the next answer comes from.

Give it time before you judge a change. If you get onto a new list and your brand shows up the next day, that's a good sign but not proof. Watch the trend over a week or two.

Doing this by hand means asking the same questions on four engines every day and writing down every brand and every page. Nobody keeps that up for long. That's why we built Supercite.

Method and limitations

  • What we asked. 210 buying questions across 21 categories, such as "best CRM for small business" and "cheapest email marketing service for 10,000 subscribers". We asked each one on ChatGPT, Gemini, Google AI Overviews and Google AI Mode, as a US user in English, on 17 and 18 November 2026. That gave 839 answers we could compare across both days.
  • Brands. We matched brand names in each answer automatically. We removed names that are platforms rather than recommendations, such as YouTube, Reddit, Gmail and the AI engines themselves. Some noise remains.
  • "Changed" means the set of brands named was not exactly the same. Brand overlap is the number of brands named on both days divided by the number named on either day.
  • Two days, one sample a day. These are early numbers. Day-to-day change includes both real change and normal randomness in how the engines answer. We'll update this post with the full week.
  • The example was checked by hand against the full answer text on both days.

FAQ

Why do AI answers change from day to day?

The engines search the web each time they answer and read a handful of pages. Which pages they pick varies from run to run. Different pages name different brands, so the answer changes.

How often should I check whether AI recommends my brand?

Often enough to see a trend: at least weekly, ideally daily. Any single check can be off, because 84% of answers changed in a day.

Is the #1 recommendation stable?

More than the rest. A brand ranked first was named again the next day 90% of the time. But it was ranked first again only 63% of the time.

Which AI engine is the most consistent?

Google AI Overviews changed the least (67% of answers changed). Gemini changed the most (95%). ChatGPT kept the same top pick most often (71%).

How do I make my brand show up more consistently?

Be on more of the pages AI reads. Find the "best X" lists cited for your buyers' questions and get onto the ones that leave you out. Brands on many lists show up no matter which ones the engine reads that day.

联系我们 contact @ memedata.com