Why Does the AI Give a Different Answer Every Time I Ask?

Because two separate things are varying, and only one of them is harmless. The writing step picks words probabilistically, so the same answer comes out phrased differently each time — that’s expected. But in a system that looks things up, the evidence can vary too: a slightly reworded question runs a slightly different search, retrieves different passages, and produces an answer built on different material. Rephrasing is noise. Changing facts means the answer was never resting on solid ground, and that’s worth knowing.

Here’s how to tell which one you’re looking at.

Variation one: the writing is generated, not retrieved

A language model doesn’t store answers and read them back. It produces text one piece at a time, each time choosing among the plausible continuations. There’s usually more than one good next word, and which one gets picked involves an element of chance.

So identical questions yield answers that differ in phrasing, ordering, length, how many examples get included, and whether it opens with a summary or builds to one. The underlying content is the same; the prose is freshly written. This is a feature — it’s the same property that lets it rewrite something for you in a different tone — and it is not a sign of unreliability.

Builders can turn this down. There are settings that make output much more repeatable, and systems that need stable outputs use them. That’s a knob on their side of the wall, not something you control from the chat box.

Variation two: the retrieval changed

This is the one that matters, and it’s specific to systems that look things up. Recall the pipeline in How does RAG work?: your question becomes a search, a handful of passages come back, and the answer gets written from those passages. Change what comes back and you change the answer’s substance, not just its wording.

What makes the retrieval move:

You reworded the question. “Vacation policy” and “annual leave” search differently. Even adding a word can shift which passages score highest.
The system rewrote your question differently. Before searching, most assistants condense your question into a query — and that rewrite is itself generated text, so it varies. Two runs of the same question can send two different queries.
You’re deeper into the conversation. Earlier messages feed into the rewrite. The same sentence asked as message one and as message nine produces different searches.
Ties get broken arbitrarily. When several near-identical passages score almost the same — five versions of the same policy, say — which ones make the cut is close to a coin flip. Then one run reads the current version and another reads the superseded one.

That last case is how you get two confident, sourced, contradictory answers to the same question. Neither is a hallucination. They’re faithful reports of different documents.

Variation three: the library moved

If the assistant searched the web, the web changed. Pages get updated, rankings shift, new results appear. A question asked twice an hour apart can genuinely retrieve two different sets of pages — see How do AI chatbots search the web?.

For internal document systems, the equivalent is re-indexing: a file was added, edited, or removed between your two attempts. If old content keeps resurfacing, that’s the lag described in Why is my AI assistant giving outdated answers?.

Which differences are fine and which aren’t

Sort what changed:

Harmless:

Different wording, structure, or length.
Different examples illustrating the same point.
More or less hedging.
A different order to the same list of items.

Worth a second look:

Different numbers. Any number that moves between runs is a number nobody should act on.
Different dates, names, or identifiers.
A yes flipping to a no, or a “you can” becoming a “you can’t.”
Different citations pointing at pages that say different things.
One run says “the documents don’t cover this” and the next answers confidently. The first run was probably telling the truth.

The rule of thumb: if the paraphrase changed, ignore it. If the claim changed, don’t trust either version until you’ve checked a source.

Turn it into a habit instead of an annoyance

The most useful thing about this behaviour is that it’s a free reliability test, and it’s the cheapest one available to a non-technical user.

Ask twice, deliberately. Once as you naturally would, once rephrased with different vocabulary. Then compare the substance.

Both answers agree and cite the same source → reasonably solid.
Both agree but cite nothing → they may agree because the model believes something, not because it’s true.
They disagree → you’ve found a genuine weak spot. Now go open the sources.

This pairs with the checks in How to tell if an AI answer is accurate, and it exploits exactly the property people find most frustrating. A system that always gave you the identical answer would feel more trustworthy while giving you no way to probe it.

Why “just make it deterministic” isn’t the fix

It’s tempting to want one fixed answer per question. But the variation isn’t the underlying problem — it’s a symptom of it. If a reworded question retrieves different documents, pinning the wording just means you never find out. The instability was always there; you’d merely have stopped seeing it.

The real fixes are on the builder’s side and all involve the library rather than the randomness: remove duplicate and superseded documents, keep one authoritative version of each policy, and make the search less sensitive to phrasing. Those are the same remedies that reduce wrong answers generally.

The takeaway

Different phrasing every time is how these systems write, and it means nothing. Different facts every time means the answer depended on which passages happened to be retrieved, which is a real signal about how shaky the ground is. So don’t treat the inconsistency as a glitch to suppress — use it. Ask twice, in different words, and pay attention only to what changed in substance. It’s the quickest audit a beginner can run, and it costs one extra question.