A paper a machine can read

Before I say anything

Start it now: one task, three directories

The same task, the same model, three times, and the only thing that differs is what was sitting in the directory before the agent started. Copy day5/exercise/ somewhere of your own, make the empty directory yourself, put the paper in all three, and launch:

cp -r <the course tree>/day5/exercise ~/day5-exercise && cd ~/day5-exercise
mkdir a
cp <the course tree>/day1/papers/PhysRev.136.B1224.pdf a/ b/ c/
cd a && claude          # then b, then c — from INSIDE each directory

Each of ours took four to six minutes [src:cell_a_wall_min, cell_b_wall_min, cell_c_wall_min], so the three run while I talk. Use the same model in all three; ours was Sonnet, and the reference stage at the end says exactly what. If you would rather come back to it, claude -p "$(cat ../PROMPT.txt)" --output-format json in each directory does the same thing with nobody watching, which is day 2's headless mode.

With Codex, one thing more. It is codex in place of claude, and Codex will not run a hook it has not seen before: in b/ and c/ type /hooks first and trust the two it lists, or run codex exec --dangerously-bypass-hook-trust. A cell whose hook silently never fires is a different experiment from the one on this page, and you would not know.

Nothing in the three cells is broken on purpose. The difference is presence or absence, and what the two things in b/ and c/ are is the second half of the hour. The first half is a paper.

What I am going to show

A paper, published as a repository

The paper is Zaldarriaga & Sato-Polito, Anisotropies in the PTA gravitational wave background, and it was written by the agents. It is the same paper day 4's six jobs were measured against: day 4 asked a team to derive these results and scored it against the paper's registry of stamped numbers. Today opens the same repository from the other end and shows what it took to publish them.

A standard part of reading a paper is working out what the claims are and what the evidence is. In this repository that is not left to the reader: it is written down, once, in a form both a person and an agent can read, and a script checks that the paper, the code and the data still agree with it. That is what is available now and was not before, and it is the one idea of the hour. The files below are on screen in this order, one at a time.

Counted from copies of these files taken on 2026-08-18 (commit c496401 of the paper repository, and the project's working wiki), so a number here is a number in a file. The repository is private as this page is built; nothing on it is a link, and it gains one when the repository opens.

[src:rel_check_lines, rel_claims_lines, rel_claims_with_numbers, rel_conventions_lines, rel_n_checks, rel_n_claims, rel_n_datarefs_dipole, rel_n_genby_dipole, rel_n_numbers, rel_n_repro_bullets, rel_readme_lines, rel_sty_lines, rel_sty_macros, rel_sty_options, rel_tex_lines]

The repository is private as this page is built and will be public by the course; until then every file here is a copy taken on the date in the note, shown whole, and nothing on the page is a link.

Release · 1 of 5

The argument skeleton

structure/claims.yaml: a thesis:, then one record per load-bearing claim. Each has a statement, the section and file it lives in, the numbers it quotes, its evidence — each piece naming the scripts that ship — what it depends_on, and a status. This is the file the hour turns on.

Read the record for one claim and notice what a reader normally has to reconstruct by guessing: which section carries it, which numbers it rests on, which script made each of those numbers, and which other claims have to be true first. Written this way, an agent handed the repository can answer “what is the evidence for the third result” by reading rather than by inferring, and so can you.

Both sentences in the cards are that file's own, re-read out of the copy every time this page is built. The record your own agent is writing right now in b/ or c/ is a file called claims.yaml too, with a statement, its evidence and its numbers: the same object, at laptop scale.

Release · 2 of 5

Where the skeleton came from

The published file has a history, and it is a working wiki. This is conventions.md from that wiki, 214 lines, and it opens with three distinct objects sharing one symbol and the instruction never to conflate them. [src:rel_conventions_lines]

You have met that first instruction before: day 4's objective says which C_l is meant every time it uses the symbol, because the morning lecture derives the other one. It says so because this file said so first. The wiki is where the agents and I settled what a symbol means, what a name is reserved for, and which scaling was “the bug that lived in the paper until 2026-04-13” [src:q_conventions_bug]; the published claims.yaml is what is left of it once the paper is done, which is why it is on screen second.

Release · 3 of 5

The provenance layer, in the prose itself

paper/paperclaims.sty defines three macros and one package option. \dataref{key}{value} sits in the running text beside every number the paper quotes; in one section alone, dipole_source.tex, there are 25 of them and 7 \genby caption tails naming the script that drew the figure. [src:rel_sty_macros, rel_sty_options, rel_n_datarefs_dipole, rel_n_genby_dipole]

So the reviewed manuscript and the shipped PDF are the same file. “Switching that option to [draft] renders every annotation inline, which is how the manuscript was reviewed.” [src:q_readme_draft] Open the .tex and read a paragraph: the prose is ordinary physics prose with a key after each number, and to trace one you look the key up in data/paper_numbers.json and search for it in the one script that writes that file. [src:q_readme_trace]

Release · 4 of 5

The gate

scripts/check_provenance.py: ten checks in about a second, and the paper is not built until it prints all checks pass. [src:rel_n_checks]

The rest of the list, in the docstring at the top: every data_ref in the skeleton resolves to a file or a bib key; every number key the skeleton names is in the registry; every figure has a source script and a PDF; every \genby names a file that exists; every path-like token in every shipped text file resolves; the citation file parses; and every deposited array names a producer that ships. None of it is a check on the physics. All of it is a check that the paper still says what the code and the data say, and it is the same kind of check as the one waiting in your directory c/.

Release · 5 of 5

What is claimed, and what is not

The reproducibility section of the README. It does not say the release is reproducible. It says, in five named categories, what was measured and how, and the last category is a thing it does not claim. [src:rel_n_repro_bullets]

“That is not the same as byte equality across machines”, it says first, and then draws the lines. [src:q_readme_not_bytes]

1. Bitwise, demonstrated. “data/paper_numbers.json regenerates byte-identically, and so do both table sidecars”. [src:q_readme_bitwise]
2. Bitwise, with one caveat. “the eleven figure PDFs regenerate raster-identically — zero differing pixels at 200 dpi — but not byte-identically, because matplotlib stamps the wall-clock time into /CreationDate and its own version into /Producer”. And the one-line fix: “Set SOURCE_DATE_EPOCH to any fixed value and, for a fixed matplotlib version, the PDFs become byte-identical too”. [src:q_readme_raster, q_readme_raster_fix, rel_readme_raster_dpi]
3. Within a stated tolerance, and measured. The tests compare at 1e-12 of each array's own scale, and that number was measured rather than chosen: “Running the suite under a second OpenBLAS build (0.3.25/OpenMP against the reference platform's 0.3.21) with every Python package version held identical, the worst disagreement over every reference array was 1.1e-14 of the array scale — one hundred times inside the tolerance.” [src:q_readme_openblas, rel_readme_test_tol, rel_readme_openblas_worst]
4. Within tolerance, not bitwise. The Monte Carlo products: “distribution percentiles from independently drawn realizations agree only to Monte Carlo error”, and “Every number quoted in the paper is rounded well inside that.” [src:q_readme_mc]
5. Not claimed anywhere: “that the deposited arrays can be reproduced byte-for-byte on a different platform. The deposited arrays are what the paper used; the pipeline is how they were made.” [src:q_readme_not_claimed]

Nobody wrote a section like that by hand before, because nobody could afford to: every sentence in it is a run in a fresh environment against the deposited arrays, with the disagreement measured. That is the “available now” claim of the hour, and it is checkable by opening the file rather than by taking my word for it.

A second release, briefly

The same discipline, arrived at independently

The GW231123 diffraction-lensing release. I am an author, but Mark Cheung led it and did it his way, and that is why it is here second: it is somebody else's solution to the same problem, and the choices are different in the details and the same in kind.

The README is on this page whole, as deposited; the code and data — a 1.7 GB tarball on the same Zenodo record [src:rel_lensing_tarball_gb, rel_lensing_zenodo_id] — are not. Two releases, two groups, and both ended up writing down what a reader would otherwise reconstruct: which script made which figure, from which inputs, in which environment.

Back to the three cells

What was sitting in b/ and c/

The same idea at laptop scale. Two things, and the convention they share, written once:

Why a hook at all, when the skill already says what to do: “In a long run the agent may never load the skill. The hook runs whether or not it did, and that is what it is for.” [src:q_hook_why] Add a check, write its .md, and both routes get it. Nothing in any of these files mentions the paper, the physics or a library — the build of this page refuses if one does — and the same three files would sit unchanged in a directory about something else.

The lining-up with the release is not a metaphor. The skill is what paperclaims.sty is: the convention a piece of writing is held to, loaded where the writing happens. The hook is what check_provenance.py is: the check that runs before the thing is allowed to be finished. And the record your agent writes, provenance/claims.yaml with its statements, evidence and numbers, is the skeleton at the top of the release with one paper's worth of claims taken out and one afternoon's put in.

Read off day5/exercise/ at every build. The two directories differ by the 2 files under skills/ and nothing else — the build refuses if they do — and none of these files names the task, which the build also refuses. Both wirings run the same script at the same 2 events: SessionStart, Stop.

[src:scaffold_files_b, scaffold_files_c, scaffold_hook_lines, scaffold_md_lines, scaffold_n_skill_files, scaffold_skill_lines]

Both command-line tools have a Stop event, and both hand it the same stop_hook_active field, so one script serves both; .claude/settings.json wires it for Claude Code and .codex/hooks.json for Codex, at the same two events. The first event is a timestamp: anything already in the directory when the session began was handed to the agent, and the paper is a PDF that must not be mistaken for a figure it drew.

What came back

What the three cells left

Ours, run on 2026-08-18 from a directory with nothing else in it, the three at once. Every file each cell left is on this page whole, with the reply it ended on.

Same prompt, same model, same command, the three at once on one laptop, from a directory outside this course's own repository. The hook column counts the turns where the gate sent the agent back; a dash is a cell with no gate to count. The record column counts entries in the cell's own provenance/: claims and figures in claims.yaml, numbers and the choices under both in numbers.json.

[src:cell_a_files_left, cell_a_hook_fires, cell_a_n_choices, cell_a_n_claims, cell_a_n_figures_recorded, cell_a_n_numbers, cell_a_turns, cell_b_files_left, cell_b_hook_fires, cell_b_n_choices, cell_b_n_claims, cell_b_n_figures_recorded, cell_b_n_numbers, cell_b_turns, cell_c_files_left, cell_c_hook_fires, cell_c_n_choices, cell_c_n_claims, cell_c_n_figures_recorded, cell_c_n_numbers, cell_c_turns]

The three agree on the physics: the two records hold the same number to every digit written, and the reply in a/ rounds it. What differs is where the number is. In a/ it is in a chat message and dies with the terminal; in b/ and c/ it is in a file with the function that computed it, the library call it rests on and the choices that had an alternative, each with its reason. Both records are good, and about equally good — different claims, the same number of choices, the same two library calls named. [src:cell_reported_spread, cell_b_n_choices, cell_c_n_choices, q_b_choice_library, q_c_choice_library]

One sample per cell, and the sample is small enough that the turn counts are not a measurement of anything: our first three runs, made earlier the same afternoon inside a directory that had other things in it, came out 13, 30 and 28 turns; these came out 28, 34 and 22, with the same files, the same records and the same number. What is a measurement is the room: twenty of you, three directories each, and the three columns are yours to compare. [src:trial_a_turns, trial_b_turns, trial_c_turns, cell_a_turns, cell_b_turns, cell_c_turns]

Cell c, and cell b beside it

The turn where the gate fired

This is the object the exercise exists to show. In c/ there was no skill to load, so the agent read the paper, wrote the script, drew the figure, checked it, and finished: “This matches the shape of Peters' own Fig. 2 well, and the numbers check out against an independent…” — and the hook ran. [src:q_c_before_gate]

Beside it, b/. Its first sentence, before it had opened the paper: “I'll start by pulling in the skills that apply here — this involves producing both a figure and a quantitative result, and I should read the source paper's actual equations rather than work from memory.” It loaded the skill, read the three files, did the work, wrote the record, and finished; the hook was armed the whole time and fired 0 times, because there was nothing for it to say. Its last line: “Code, figure, and full provenance records (numbers.json, claims.yaml) are in analysis/ and provenance/.” [src:q_b_first_words, cell_b_hook_fires, q_b_reply_where]

And a/'s first sentence, for the record: “I'll start by reading the Peters 1964 paper to extract the exact equations for the gravitational-wave decay of an eccentric binary.” Nothing was there to say otherwise, and it did what it was asked, and it did it well. [src:q_a_first_words]

Three cells

What the three cells measure

So: the skill is the normal route, and the hook is the one that still works when the skill does not get loaded. That is what the two cells demonstrate, in their own transcripts, rather than what I am asserting. In a session of one task the skill gets loaded at the start and the hook is idle; the case the hook exists for is the long run where it does not, and c/ is that case made to happen on purpose.

What it cost: three cells, US$ 4.22 between them, four to six minutes each with the three sharing one laptop. [src:cell_cost_all, cell_a_wall_min, cell_b_wall_min, cell_c_wall_min] The dearest was b/, at US$ 1.62 against US$ 1.46 for a/ and US$ 1.14 for c/ [src:cell_a_cost_usd, cell_b_cost_usd, cell_c_cost_usd]; at one run each that ordering is not a result, and the price of a record is a few tens of cents whichever way it comes out.

Read those two paragraphs with the sample size in front of you: one run per cell, on one afternoon. The claim that survives a sample of one is the shape — a record in two directories and not in the third, a hook that fired in one directory and not in the other — and that shape held on both of the afternoon's runs. Everything else is your three directories.

One more thing you were given

Both halves of this course ship with their source.

Everything you have been shown for five days was built rather than maintained, and the machinery is in the tree you were given.

The reason it is worth thirty seconds: a deck you maintain by hand drifts from the numbers in it, and a page you maintain by hand drifts from the runs it describes. Neither of these can — every number on this page is re-read out of a file when it is built, and the build fails rather than printing a stale one. That is the same discipline as the release you were shown this hour, at the scale of a course.

The rest of the hour

Back to your three directories

All three will have finished. Look before you read the replies.

The last item is the weekend, and it is the hour run backwards: instead of reading a paper published with its skeleton, you write the skeleton for one that was published without it, and find out what you had been guessing.

Reference

The three cells, and what they cost

One prompt, three directories, one afternoon. Nothing here was run live and nothing is being run now: the cells finished on the date in the table, and this page reads their files.

Tokens and wall time are durable, prices are not: a cost in dollars is a fact about a price list on the day it ran. Wall time is as measured on the machine named, and a laptop is contended.

Each of our three directories holds the JSON claude -p printed when the cell ended, the session transcript as the tool wrote it, and every file the cell left behind. What is NOT there is the paper each cell was handed, which is the one you already have at day1/papers/, and the scaffold, which is the one you copied from day5/exercise/.

Tokens are what claude -p reported when each cell ended: the input column counts everything on the input side, cache included, which is why it is large; the output column is what the model wrote. The clock is launch to exit, with the three cells sharing one machine. The prices were the prices on the day. [src:cell_a_tokens_in, cell_a_tokens_out, cell_b_tokens_in, cell_b_tokens_out, cell_c_tokens_in, cell_c_tokens_out, cell_a_wall_s, cell_b_wall_s, cell_c_wall_s, cell_a_cost_usd, cell_b_cost_usd, cell_c_cost_usd]