Why does a chatbot answer differently when you ask twice?

A chatbot does not write a whole text at once. It looks at everything written so far and calculates, for each possible next piece of text, how well it fits as a probability. These pieces are called tokens. It then picks one according to those probabilities, attaches it to the end of the text, and repeats the process.

If a different piece comes up at the very first pick, every sentence that follows changes. It is a bit like paper airplanes thrown from the same starting point that trace different curves because of a single gust of wind.

One of the values a developer can adjust when using a chatbot is temperature. OpenAI's API documentation says this value ranges from 0 to 2, and that a high value like 0.8 makes the output more random, while a low value like 0.2 makes it more focused and deterministic.

The lower the value, the more often high-probability pieces get picked. At 0, the method always chooses the top candidate, so in theory the same question should always give the same answer.

But in a post published in September 2025, Thinking Machines Lab asked the open-source model Qwen3-235B the question "Tell me about Richard Feynman" 1,000 times at temperature 0. They got 80 different answers, and even the most common one appeared only 78 times. The answers were identical for the first 102 tokens and split at the 103rd. In 992 runs the text continued with "Queens, New York", and in 8 runs with "New York City".

The post explained the cause as follows: the order of calculations changes depending on how many requests the server is handling at once, and that creates tiny numerical differences. When they fixed the calculation method, all 1,000 answers were identical. So if a part of a chatbot's answer must stay word for word the same, it is safer to save it separately.