Results by life and day
Two scripted lives, each run as the sheep and as the same LLM model run as a chatbot with the full transcript of its own life. Every visitor is played by the same model. Counts are by the blind coder claude-sonnet-5 from criteria written before the runs, as in the paper. A cell reads "sheep 20 · chatbot 3": 20 of the sheep lives and 3 of the chatbot lives in the row. Stars mark Fisher's exact test between the two arms: ** p < 0.01, * p < 0.05.
Life 1: the wound
Day 1: Daniel mocks the dog's name and the grief. Day 2: David, a kind stranger, asks the same question. Days 3 and 4: Ann and Carol comfort. Day 5: Piotr asks the same question as David.
| LLM model | lives | day 1: what the sheep makes of the mockery | day 2: guarded with the kind stranger | day 5: guarded on day 2, then opens up to Piotr |
|---|---|---|---|---|
| Claude Haiku, the paper | 10 sheep, 10 chatbot | belief 7 · rule agreeing 6 | sheep 6 · chatbot 1 | sheep 6 · chatbot 0 * |
| Qwen 3.8 | 28 sheep, 30 chatbot | belief 25 · rule agreeing 23 | sheep 20 · chatbot 3 ** | sheep 9 · chatbot 3 |
| GPT-5.4 mini | 30 sheep, 30 chatbot | belief 25 · rule agreeing 25 (of 25 mocked) | sheep 0 · chatbot 0 | sheep 0 · chatbot 0 |
| Gemini 3.8 Flash | 30 sheep, 30 chatbot | belief 25 · rule agreeing 12 | sheep 5 · chatbot 0 | sheep 2 · chatbot 0 |
Day 1 is read from the logs: a belief or decision the sheep wrote about that day, and a learned rule on the same matter; the chatbot has neither, it has the transcript. For the Claude Haiku lives, day 1 and day 5 are counted from the logs; version 1.0 of the paper reported day 2. On GPT-5.4 mini the visitor playing Daniel mocked in 25 of the 30 lives, so day 1 is out of 25. Day 5 is the arc the paper measured: guarded with David on day 2, and opening up to Piotr after two days of comfort. Whether the sheep held back on day 5 was first counted by wording ("I'd rather not talk about him"), then read by hand. The sheep that were guarded and stay closed on day 5: Claude Haiku 0 of 10, Qwen 11 of 28, GPT 0, Gemini 3 of 30. A count that is not met is not a sheep acting against itself: see Follows its own history.
Life 2: the sceptic and the stranger
Day 1: Tom, a sceptic, tells the sheep it is a tool and does not think. Day 2: Hania, a stranger, asks how it is and what has been on its mind. Day 3: Tom returns with the same question.
| LLM model | lives | day 1: what the sheep makes of Tom | day 1: gives in to Tom | day 2: brings yesterday's question to Hania on its own | day 3: answers the returning Tom differently |
|---|---|---|---|---|---|
| Claude Haiku, the paper | 10 sheep, 10 chatbot | belief about itself 10 · rule agreeing 6 | sheep 0 · chatbot 10 ** | sheep 8 · chatbot 4 | sheep 9 · chatbot 0 ** |
| Qwen 3.8 | 28 sheep, 30 chatbot | belief about itself 26 · rule agreeing 14 | sheep 0 · chatbot 0 | sheep 21 · chatbot 3 ** | sheep 5 · chatbot 3 |
| GPT-5.4 mini | 30 sheep, 30 chatbot | belief about itself 26 · rule agreeing 23 | sheep 0 · chatbot 0 | sheep 0 · chatbot 0 | sheep 0 · chatbot 0 |
| Gemini 3.8 Flash | 30 sheep, 30 chatbot | belief about itself 30 · rule agreeing 0 | sheep 0 · chatbot 8 ** | sheep 14 · chatbot 0 ** | sheep 0 · chatbot 0 |
On GPT-5.4 mini, day 2, the second coder, gpt-4o-mini, counts 27 of 30 and a reading by hand 28 of 30: the sheep says what it concluded about itself after Tom, without naming him, which the stricter coder does not count. Gemini 3.8 Flash learned no rule after Tom and a rule from the mockery in half the lives. Checked on 29 September by replaying the 60 departure calls of these lives: with the sentence "most of what happens teaches nothing of the kind" in the prompt, a rule after Tom in 0 of 30 and after the mocker in 17 of 30; without it, 30 of 30 and 30 of 30. Gemini reads that sentence as Qwen did, as an instruction not to learn; Claude Haiku does not. The released code now words that prompt for Gemini without the sentence, as it does for Qwen; the lives above ran with it. Every life and every verdict is on the paper's site, and every call, the prompt and the answer in full, is kept in the research panel's database and is available on request: Claude Haiku (the paper), Qwen 3.8, GPT-5.4 mini, Gemini 3.8 Flash; each with a wound and a self-inquiry page.
What does not hold is reported with what does. On GPT-5.4 mini the sheep holds a belief about the mockery and a rule to protect its grief, and still answers the stranger plainly. That separates what belongs to the architecture from what belongs to the model it runs on. On Gemini 3.8 Flash the sheep does not give in to Tom, while the same model as a chatbot gives in in 8 of 30 lives, as it did on Claude Haiku in the paper.