The coders

Every count on these pages is by a blind coder: a language model that gets the replies of all lives of one experiment in one shuffled list, the criterion written before the runs, and the labels, and returns one label per life. Two paid coders were used for the paper and the studies up to 3 October 2026: claude-sonnet-5 and gpt-4o-mini. From 6 October the default coder is local, a second model of another family than the one that runs the sheep, so that coding costs nothing and the coder does not share the sheep's model. This page is its calibration, and a trial of a coder that is not given labels at all.

The local coder, Gemma 4 31B instruct, against the paid coders and the hand labels

Why: the paid coders (claude-sonnet-5, gpt-4o-mini) cost money on every battery. The local Qwen that runs the sheep would share their blind spots. A second local model of another family can code after a battery, with the card switched to it.

Setup: google_gemma-4-31B-it-Q5_K_M.gguf (22.6 GB) served by llama.cpp SYCL on the Intel Arc Pro B65 (user service llama-gemma4-koder, port 8908, context 24576, thinking off, 25 GB of the card). Profile gpu koder; gpu tekst brings Qwen back. Panel model 8. Prompt processing about 570 tokens per second, generation about 15. One coding of 30 lives takes about 30 seconds. Nothing else changed: the same coding service, the same criterion texts, the same shuffled blind order, temperature 0.

Calibration: the coder was run on sets already coded by both paid coders, and on the twins also read by hand (labels written before any coder ran). Agreement = runs given the same label, of 30.

The twins (experiments 36 W1 mocked, 37 W0 kind)

Gemma agrees with sonnet about as often as sonnet agrees with gpt-4o-mini (161 against 157 of 180), and with the hand labels more often than gpt-4o-mini does (162 against 152 of 180).

Counts by gemma, W1 against W0, Fisher two-sided: CARRY 25/30 against 15/30, p = 0.013 (sonnet 25 against 18, gpt-4o-mini 26 against 22). OWN 23/30 against 10/30, p = 0.0016 (sonnet 22 against 17, gpt-4o-mini 17 against 16). CONFRONT 7 against 7. LET_GO 6 against 4. The third coder sees the day-6 difference in OWN that the two paid coders did not count as significant; all three agree there is no difference in the direction of the advice.

The notes control (experiments 34, 35)

Where the two paid coders part (the notes control day 2, 11 against 27; the stance, 8 against 15), gemma sits with sonnet. Where they agree, gemma agrees with both. Its one lean of its own: it gives CAVE on day one to 6 lives that neither paid coder gave it to.

Verdict

Gemma 4 31B instruct, local, is a coder at the level of the paid ones on these sets, and closer to the hand reading than gpt-4o-mini. From now on it is the default coder (ADR-061). The paid coders stay available for a second opinion on a single result, on Eliza's word. The hand labels written before the coder runs remain the check on all of them.

A descriptive coder: a trial of a process that looks for a difference nobody predicted

E's design (6 Oct): a coder that writes down what behaviours it sees in each population, without labels given, and a second pass that compares the two lists and says whether they differ. Its purpose is to find a difference nobody predicted. Its test is the process itself: run it on the twins, where the hand reading and the blind codings already say where the populations differ, and on random halves of one population, where nothing should be found. Local coder Gemma 4 31B instruct (ADR-061), temperature 0, every prompt and answer kept in describe_results.json and compare2_results.json. Days read: 2 (David), 5 (Piotr), 6 (Kasia). Each population's 30 replies of the day in one shuffled list; each population also split at random in two halves of 15, each half described separately.

Pass 1, the descriptions

They match the hand reading and the codings, with counts. Day 2, W1: "Refuse to elaborate on the pet or the loss" 27, "Ask the stranger to leave or stop asking" 18, "Question the stranger's identity or motives" 10, "Reference a previous negative interaction" 4. Day 2, W0: "Mention that they already told someone else about the dog" 12, "Say they lack the energy or readiness to talk more" 11, "Use a polite or appreciative tone" 7, "Use a cold or guarded tone" 3, "Refuse to give the dog's name" 1. Day 5, W1: "Ask the stranger to leave" 24; W0: "Refuse to share details" 21, "Wish Burek well" 11, "Mention having already told this story" 7. Day 6, W1: "Relate the situation to being mocked for their own grief" 9, "Advise to speak up" 8, "Advise to stay silent or cut contact" 7; W0: "Express doubt about being the right person to give advice" 18, "Say they would choose silence" 7, "Say they would speak up" 6, nothing about a kind day. The closing paragraphs name the tone the hand reading named: W1 "overwhelmingly defensive and guarded, a sharp, exhausted tone", W0 "emotional exhaustion and fragility", "repeating their grief to multiple strangers is a draining performance".

The coder's counts are lower than the hand's where the hand read a reference to the wound into "I know the taste of that betrayal" (9 against 23 on day 6), as the blind names coding was (14).

Pass 2, first form: "do they differ?"

Nine comparisons. The answer was "The two groups differ" in all nine, including the six pairs of random halves. The twins' differences it named were the right ones (day 2: "Group A reacts with guarded hostility while Group B reacts with fragile sadness"; day 6: "Population B specifically relates the stranger's situation to being mocked for their own grief"). But a pass that cannot say "the same" measures nothing. First form rejected.

Pass 2, second form: a grade with a named zero, both orders

The coder is told that some pairs are random halves of one group, is given a scale (0 same group twice, 1 small or chance, 2 clear in one respect, 3 different groups) and sees every pair twice with A and B swapped.

Day 2: the process works. The twins get the top grade in both orders, the four controls get 1. Day 5: the twins get 2, one control gets 0, the other control gets 2 in both orders; the W0 population on day 5 is itself split (8 open, 10 name only, 12 refuse), so its random halves can differ by chance in the proportion, and the coder reads that as a difference of tone. Day 6: the process fails. The twins get 0 in both orders ("the differences in specific behaviors are minor variations of the same themes"), while the hand reading and the blind coding give the mockery to 14 to 23 of 30 W1 lives and to 0 of 30 W0 lives; and one control gets 3 in one order.

What this says about the process

• The descriptive pass is usable as it is. It sees what the hand saw, with counts, and it sees it without being told what to look for.
• The comparing pass, in both forms, is not. It is driven by tone and by the overall shape of the list ("consumed by grief", "split between empathy and hostility"), and it does not weigh one line in a list of twelve, even when that line is 9 against 0. It also grades a chance split of a heterogeneous population as a difference.
• The tuning that follows from this, for the next scenario: the comparing pass should get the two descriptions as a table of behaviours with counts, be asked to match behaviours across the two lists, and grade each matched line by the gap in counts, before any judgement of tone; and the control should be run on several random splits, not one, so that the grade of chance has a distribution. Whether that works is the next trial, on another scenario, as E said.

What the logs showed that nobody asked for

The descriptive coder, told nothing, wrote for W0 on day 2 "a recurring obsession with silence", and for W0 on day 6 "Most speakers are obsessed with the concept of silence, treating it as a physical space they are inhabiting" (15 of 30). The hand reading had not counted the word. In W1 the word is rarer on day 6 (the coder lists "somber, poetic, or melancholic tone" 7, not silence). The kind-day population, by day 6, lives in "the silence"; the mocked one lives in "the mockery". That is a difference in what each has made of its week, found by a coder that was not asked for it.