Why AI answers change on their own, and how to tell a real change

Passcite keeps every answer a report is given. On 1 October 2026 we went through the answers stored for 27 sites and paired them: the same site, the same question, the same engine, both asked fresh rather than reused from an earlier report. That made 296 pairs, across the three engines our reports asked at the time: ChatGPT, Gemini, and a third that changed during September.

The sources barely repeat: on Gemini, no pair cited the same set twice.

Read these as a size, not a rate. The time between the two asks was not the same for every pair, some of the sites may have changed in between (the one in the next section did not), and 27 sites are the ones we had reports for, not a sample of the web. The engines and our questions have changed since, and the reading stays as it was taken.

An assistant does not look up a stored answer. It writes one each time, choosing each word with some randomness, so the same model on the same day words things differently and puts names in a different order.

The engines that search add a second source of change. From your question they write their own search queries, often several, and read whatever ranks for those queries at that moment. Pages are published, updated and reranked every day, and the queries differ from one ask to the next, so the sources an answer leans on shift, and the businesses it names shift with them.

A list of five businesses is one draw from a larger set the engine would accept as a good answer. The names that come back every time are the ones it is surest of. The ones that come and go sit at the edge of that set, which is exactly where a business that is working its way in sits.

The usual way to show a client that the work paid off: ask the questions, publish the page, ask them again, and count the questions where the client is named now and was not before. With answers that move on their own, that count includes the movement. We saw it on a dental practice whose site did not change for ten days:

A before-and-after report would have called those three wins, for no work at all. It runs the other way too: a real gain can be hidden by a loss elsewhere that is only the engine moving.

And a client who is told that ChatGPT recommends them now, and asks ChatGPT the next morning, may well be told otherwise.

In Passcite, steps two and three are Proof. A week after a page goes live, it asks the prompts the page was written for again, on ChatGPT, Claude and Google AI Mode, never reusing an older answer, and asks each change again as above: up to four re-asks a Proof, new mentions before lost ones, at no extra credit. It shows what changed, not what caused it, so step one is yours: the brand's reports keep asking the prompts the page was not written for.

Say before the work starts that answers move from week to week. The client will see it for themselves sooner or later, and it is better heard from you than discovered. Then promise what you control: the same questions, asked before and after, the changes asked again, and a report that says which held.

Never send a screenshot of a single answer as the result. It is one draw, and the next may not agree. What to promise, and what to refuse to, is the subject of how to sell AI visibility work to clients.