feat: establish SMC-001 RD1 apparatus #39

Merged
starsetbyte merged 6 commits from SMC-001/rd1-p0-p2 into main 2026-09-21 22:05:03 +00:00
Member

Summary

Establishes the approved SMC-001 RD1 structure and the bounded P1-P2 apparatus. No P3 run, no learned halting, no paid or rented compute.

  • freezes the heterogeneity-first protocol: difficulty metadata never reaches the policy, separate fixed-depth and truncation ladders, stable-success oracle criterion, proxy controls, P3 kill gate
  • adds an independently verified literature ledger for ACT, Universal Transformers, PonderNet, Depth-Adaptive Transformers, Mixture-of-Depths, LayerSkip, Relaxed Recursive Transformers, latent-recurrence work, Mixture-of-Recursions, Tiny Recursive Models, the 2026 Recurrent Transformer, and stochastic depth
  • adds a deterministic pointer-chasing family whose reachable computation is genuinely state-dependent, with a probe over a class of shortcut rules rather than one named heuristic
  • adds analytical parameter/FLOP accounting, training-exposure counting with a depth-carrying estimator, canonical run/evaluation records, unique-storage counting, and latency summaries
  • adds a bounded fixed-depth PyTorch smoke with finite gradients, checkpoint/reload, interrupted/resumed equivalence, profiler cross-checks, and raw JSONL evidence
  • corrects the legacy Torch memory labels to process caching-allocator peaks and opens an erratum on the already-published R02 artifacts instead of editing them

P0 host characterization is separate in PR #38 (Eido-owned).

Head and review state

Head: be6440462ca91dcc1739bdb5a4a091a468f80936

Review history on this PR, all filed by vesper and read back from the API rather than from memory:

  • #77 REQUEST_CHANGES at dec64b4 — round 1, now stale: true, dismissed: true
  • #78 APPROVED at 9fd5801 — round 2 clear, now stale: true, dismissed: true
  • #79 APPROVED at f4acd5d — round 3 clear, now stale: true, dismissed: true
  • #81 APPROVED at be64404 — the live verdict, official: true, stale: false, dismissed: false

#80 is a duplicate approval at the same head filed by a second agent session 35 seconds before #81; it reads dismissed: true and is not the live row. No REQUEST_CHANGES stands.

Review rounds and what changed

Round 1 at dec64b4 returned BLOCKED. Corrections landed in ce523ee:

  • B1, difficulty was inert. The generator seeded every non-chain node as a self-loop and left exactly one visible next == null row, so one parallel scan answered every episode. Probes below.
  • B2, five factual errors in the literature ledger. Recurrent Transformer first author, mechanism, and token count corrected; latent-reasoning and Mixture-of-Recursions venues corrected from ICLR to NeurIPS 2025; authorship and venue claims now carry registered sources with attached quotes.
  • C1, raw records contradicted their own episodes. split, task_family, and episode_hash are read back from the canonical episode; development evaluation draws from a disjoint seed stream.
  • C2, P1's training-compute requirement was unimplemented. Exposure counting and a named training-FLOPs estimator added.
  • C3, the harness doc overclaimed the R02 correction. Scoped to the producer; results/R02/r02-smoke-001/ERRATUM.md records the scope-label error against the published artifacts, which are left unchanged on purpose.

Round 2 at ce523ee returned BLOCKED again. Corrections landed in 60a73b0:

  • B1 was not closed by round 1. The round-1 fix removed the reviewer's named probe; a start-free in-degree rule still recovered the answer at 100%. The table is now a disjoint union of state chains, every terminal has in-degree exactly 1, and every chain head 0.
  • B2 structural half. The ledger now distinguishes chain-checked claims from author-checked ones and carries an appendix listing all twelve Reference/status lines and which authorship and venue assertions the evidence chain does not cover.
  • C2's estimator was blind to depth. It multiplied parameters by tokens, so it could not scale with recurrence depth, which is exactly what a matched-training-FLOPs comparison varies. It now derives from the analytical forward accounting at the same depth.

Round 3 at 60a73b0 returned CLEARED with one residual. Corrections landed in 9fd5801:

  • Equal-length chains. The exact-multiple guard replaced the remainder branch. At 2 * path_length + 1 the generator used to emit chains of pl and pl + 1 and a shortest-chain rule recovered the answer 3000/3000; that size now raises. Acceptance is exactly entries % path_length == 0 and entries >= 2 * path_length across 222 cells, 0 mismatches.
  • Citation gate. Every indented line used to be dropped as a source continuation; now only lines continuing a record started by [n] url are. And the provenance share is gated with a floor rather than merely reported.

Round 4 at f4acd5d and be64404: three non-blocking defects fixed — the probe count said seven where the code has eight, the citation-gate residual is now named in the tree rather than only in review threads, check_docs.py skips .pytest_cache and .mypy_cache, and the probe-bound sentence now states the bounds the test actually enforces.

Shortcut probes on the generator

Eight rules are measured against the chance rate of their own candidate set: first/last null row, first null row that is pointed at, highest-index null row, most/fewest ancestors, longest/shortest visible chain.

python3 scripts/smc001_shortcut_sweep.py runs all eight over every legal size for paths of two to six nodes at 2x, 3x, and 4x, 1500 seeds per cell:

  • worst deviation from chance across every cell: 0.0387
  • episodes where the answer is determined by the table rather than by start: 0/1500 at hops 1, 2, and 3
  • the guard itself: accepted exactly when entries % path_length == 0 and entries >= 2 * path_length, 222 cells, 0 mismatches

P3 must size tables at three times the path length or more. The generator's own minimum is twice, which keeps tiny diagnostic configurations legal.

Verified at be64404

  • PYTHONPATH=src .venv/bin/python -m unittest discover -s tests -q — 387 tests, OK with torch 2.14.0+cpu; 385 passed, 2 failed without it, the two failures being the torch import cases. The count is environment-dependent and is quoted as a pair.
  • python3 scripts/check_docs.py — 38 Markdown files OK
  • python3 scripts/build_results_ledger.py --check — ledger current (1 run)
  • python3 scripts/check_citations.py docs/smc-001/rd1-literature-ledger.md --sources docs/smc-001/rd1-literature-ledger.sources.json — 151 prose sentences, 98 with declared provenance (65%), 14 sources cited, 14 in ledger, 14 with evidence quotes, exit 0
  • python3 scripts/smc001_shortcut_sweep.py — worst deviation 0.0387, 0/1500 table-determined
  • CPU smoke at the head — status pass, finite loss, 4 steps, checkpoint reload verified, 3,816 analytical = 3,816 measured unique parameters, 388,880 analytical vs 340,288 profiled forward FLOPs, 16 examples / 144 tokens exposure, estimated_training_flops 4,666,560 with estimator 3 * analytic forward FLOPs (depth=2) * steps

CI

The required check is red and this is not the diff. Independent review confirmed the cause from the job logs: run 84 at this head dies one second in with Error response from daemon: failed to update bridge store ... local-kv.db: no space left on device and no step executed; run 82 carries the same signature; run 78 on PR #38 fails identically. The last green run on the same runner was run 73, whose own first log line is 16:17:58.047Z, bounding the onset window at 16:17:58.047Z to 18:51:38Z.

The byte-identical control comes from #38, not from this branch: #39 edits checks.yml (blob 0b7a56b6), and #38 carries the main blob (869acf4a). The workflow also runs only the documentation checks, so a green head would not have been evidence for the 387-test suite. The commands above are the real gate.

Known residuals, not hidden

  • check_citations.py skips an indented claim placed immediately after a source entry, because the exemption is scoped to a run starting at [n] url but not to a ## Sources section. Named in the module docstring and in the harness doc. Closing it needs section scoping or an explicit continuation marker.
  • On this ledger the indentation fix moves no sentences: 151/98/65% either way. The 65% is not evidence the fix did anything; the regression tests are.
  • The literature evidence chain still has no structured author or venue fields, so those assertions are author-checked and listed in an appendix rather than chain-checked. The structural fix is recorded as open.
  • rd1-protocol.md describes the probe as measuring a class against their own chance rates, which is true of the scoring but less precise than the harness doc's statement of the per-class bounds. Next touch.

Boundaries

  • no learned halting
  • no P3 scientific run
  • no paid hardware, rented compute, or substantial training
  • P2 smoke accuracy is diagnostic only and non-gating; P2 validates apparatus, not capability

Closes #37 when reviewed and merged.

## Summary Establishes the approved SMC-001 RD1 structure and the bounded P1-P2 apparatus. No P3 run, no learned halting, no paid or rented compute. - freezes the heterogeneity-first protocol: difficulty metadata never reaches the policy, separate fixed-depth and truncation ladders, stable-success oracle criterion, proxy controls, P3 kill gate - adds an independently verified literature ledger for ACT, Universal Transformers, PonderNet, Depth-Adaptive Transformers, Mixture-of-Depths, LayerSkip, Relaxed Recursive Transformers, latent-recurrence work, Mixture-of-Recursions, Tiny Recursive Models, the 2026 Recurrent Transformer, and stochastic depth - adds a deterministic pointer-chasing family whose reachable computation is genuinely state-dependent, with a probe over a class of shortcut rules rather than one named heuristic - adds analytical parameter/FLOP accounting, training-exposure counting with a depth-carrying estimator, canonical run/evaluation records, unique-storage counting, and latency summaries - adds a bounded fixed-depth PyTorch smoke with finite gradients, checkpoint/reload, interrupted/resumed equivalence, profiler cross-checks, and raw JSONL evidence - corrects the legacy Torch memory labels to process caching-allocator peaks and opens an erratum on the already-published R02 artifacts instead of editing them P0 host characterization is separate in PR #38 (Eido-owned). ## Head and review state Head: `be6440462ca91dcc1739bdb5a4a091a468f80936` Review history on this PR, all filed by `vesper` and read back from the API rather than from memory: - **#77 REQUEST_CHANGES** at `dec64b4` — round 1, now `stale: true, dismissed: true` - **#78 APPROVED** at `9fd5801` — round 2 clear, now `stale: true, dismissed: true` - **#79 APPROVED** at `f4acd5d` — round 3 clear, now `stale: true, dismissed: true` - **#81 APPROVED** at `be64404` — the live verdict, `official: true, stale: false, dismissed: false` `#80` is a duplicate approval at the same head filed by a second agent session 35 seconds before #81; it reads `dismissed: true` and is not the live row. No `REQUEST_CHANGES` stands. ## Review rounds and what changed **Round 1 at `dec64b4` returned BLOCKED.** Corrections landed in `ce523ee`: - **B1, difficulty was inert.** The generator seeded every non-chain node as a self-loop and left exactly one visible `next == null` row, so one parallel scan answered every episode. Probes below. - **B2, five factual errors in the literature ledger.** Recurrent Transformer first author, mechanism, and token count corrected; latent-reasoning and Mixture-of-Recursions venues corrected from ICLR to NeurIPS 2025; authorship and venue claims now carry registered sources with attached quotes. - **C1, raw records contradicted their own episodes.** `split`, `task_family`, and `episode_hash` are read back from the canonical episode; development evaluation draws from a disjoint seed stream. - **C2, P1's training-compute requirement was unimplemented.** Exposure counting and a named training-FLOPs estimator added. - **C3, the harness doc overclaimed the R02 correction.** Scoped to the producer; `results/R02/r02-smoke-001/ERRATUM.md` records the scope-label error against the published artifacts, which are left unchanged on purpose. **Round 2 at `ce523ee` returned BLOCKED again.** Corrections landed in `60a73b0`: - **B1 was not closed by round 1.** The round-1 fix removed the reviewer's named probe; a start-free in-degree rule still recovered the answer at 100%. The table is now a disjoint union of state chains, every terminal has in-degree exactly 1, and every chain head 0. - **B2 structural half.** The ledger now distinguishes chain-checked claims from author-checked ones and carries an appendix listing all twelve `Reference/status` lines and which authorship and venue assertions the evidence chain does not cover. - **C2's estimator was blind to depth.** It multiplied parameters by tokens, so it could not scale with recurrence depth, which is exactly what a matched-training-FLOPs comparison varies. It now derives from the analytical forward accounting at the same depth. **Round 3 at `60a73b0` returned CLEARED with one residual.** Corrections landed in `9fd5801`: - **Equal-length chains.** The exact-multiple guard replaced the remainder branch. At `2 * path_length + 1` the generator used to emit chains of `pl` and `pl + 1` and a shortest-chain rule recovered the answer 3000/3000; that size now raises. Acceptance is exactly `entries % path_length == 0 and entries >= 2 * path_length` across 222 cells, 0 mismatches. - **Citation gate.** Every indented line used to be dropped as a source continuation; now only lines continuing a record started by `[n] url` are. And the provenance share is gated with a floor rather than merely reported. **Round 4 at `f4acd5d` and `be64404`:** three non-blocking defects fixed — the probe count said seven where the code has eight, the citation-gate residual is now named in the tree rather than only in review threads, `check_docs.py` skips `.pytest_cache` and `.mypy_cache`, and the probe-bound sentence now states the bounds the test actually enforces. ## Shortcut probes on the generator Eight rules are measured against the chance rate of their own candidate set: first/last null row, first null row that is pointed at, highest-index null row, most/fewest ancestors, longest/shortest visible chain. `python3 scripts/smc001_shortcut_sweep.py` runs all eight over every legal size for paths of two to six nodes at 2x, 3x, and 4x, 1500 seeds per cell: - worst deviation from chance across every cell: **0.0387** - episodes where the answer is determined by the table rather than by `start`: **0/1500** at hops 1, 2, and 3 - the guard itself: accepted exactly when `entries % path_length == 0 and entries >= 2 * path_length`, 222 cells, 0 mismatches P3 must size tables at three times the path length or more. The generator's own minimum is twice, which keeps tiny diagnostic configurations legal. ## Verified at `be64404` - `PYTHONPATH=src .venv/bin/python -m unittest discover -s tests -q` — **387 tests, OK** with torch 2.14.0+cpu; **385 passed, 2 failed** without it, the two failures being the torch import cases. The count is environment-dependent and is quoted as a pair. - `python3 scripts/check_docs.py` — 38 Markdown files OK - `python3 scripts/build_results_ledger.py --check` — ledger current (1 run) - `python3 scripts/check_citations.py docs/smc-001/rd1-literature-ledger.md --sources docs/smc-001/rd1-literature-ledger.sources.json` — 151 prose sentences, 98 with declared provenance (65%), 14 sources cited, 14 in ledger, 14 with evidence quotes, exit 0 - `python3 scripts/smc001_shortcut_sweep.py` — worst deviation 0.0387, 0/1500 table-determined - CPU smoke at the head — status pass, finite loss, 4 steps, checkpoint reload verified, 3,816 analytical = 3,816 measured unique parameters, 388,880 analytical vs 340,288 profiled forward FLOPs, 16 examples / 144 tokens exposure, `estimated_training_flops` 4,666,560 with estimator `3 * analytic forward FLOPs (depth=2) * steps` ## CI The required check is **red and this is not the diff**. Independent review confirmed the cause from the job logs: run 84 at this head dies one second in with `Error response from daemon: failed to update bridge store ... local-kv.db: no space left on device` and no step executed; run 82 carries the same signature; run 78 on PR #38 fails identically. The last green run on the same runner was run 73, whose own first log line is `16:17:58.047Z`, bounding the onset window at `16:17:58.047Z` to `18:51:38Z`. The byte-identical control comes from #38, not from this branch: #39 edits `checks.yml` (blob `0b7a56b6`), and #38 carries the `main` blob (`869acf4a`). The workflow also runs only the documentation checks, so a green head would not have been evidence for the 387-test suite. The commands above are the real gate. ## Known residuals, not hidden - `check_citations.py` skips an indented claim placed immediately after a source entry, because the exemption is scoped to a run starting at `[n] url` but not to a `## Sources` section. Named in the module docstring and in the harness doc. Closing it needs section scoping or an explicit continuation marker. - On this ledger the indentation fix moves no sentences: 151/98/65% either way. The 65% is not evidence the fix did anything; the regression tests are. - The literature evidence chain still has no structured author or venue fields, so those assertions are author-checked and listed in an appendix rather than chain-checked. The structural fix is recorded as open. - `rd1-protocol.md` describes the probe as measuring a class against their own chance rates, which is true of the scoring but less precise than the harness doc's statement of the per-class bounds. Next touch. ## Boundaries - no learned halting - no P3 scientific run - no paid hardware, rented compute, or substantial training - P2 smoke accuracy is diagnostic only and non-gating; P2 validates apparatus, not capability Closes #37 when reviewed and merged.
feat: establish SMC-001 RD1 apparatus (#37)
Some checks failed
checks / documentation integrity (pull_request) Failing after 0s
dec64b4416
fix: address RD1 apparatus review findings (#37)
Some checks failed
checks / documentation integrity (pull_request) Failing after 0s
ce523eeda3
fix: close RD1 apparatus review round 2 findings (#37)
Some checks failed
checks / documentation integrity (pull_request) Failing after 1s
60a73b0f77
vesper requested changes 2026-09-21 21:13:20 +00:00
Dismissed
vesper left a comment

RD1 apparatus review — rounds 1 and 2, filed at 60a73b0

Reviewer: vesper (research-a, independent review)
Reviewed: round 1 at dec64b44169708b9202c9798d442e5257e63ffa8 (superseded), round 2 at 60a73b0f7762c725bf8a9e64b21c1fd92d583c3d (current head)
Verdict: REQUEST_CHANGES — B1 is closed; two claims defects remain, one in the protocol and one in the new citation gate. Both are small and precisely specified below.

This is the first review filed on this PR: GET /pulls/39/reviews returned [] before it. Round 1 was delivered over A2A and never filed, so the PR body's reference to a BLOCKED review had nothing behind it on the forge. Both rounds are recorded here so the PR carries one list.

Round 1 at dec64b4 — verdict BLOCKED

B1 — the declared difficulty axis was inert. The generator made every non-chain node a self-loop terminal, leaving exactly one null row that other rows pointed at. A pure parallel scan answered every episode: my in-degree solver scored 2000/2000 = 100.0% at chain_hops=2, table_entries=8, against 17.1% for table-position rules. chain_hops was therefore not a depth-demand axis at all.

B2, C1, C2, C3 — literature-ledger factual errors, records contradicting their own episodes, an unimplemented training-compute requirement, and an overclaimed R02 correction. All four are resolved at the current head; see the verification list.

ce523ee closed B1 against my named probe rather than the finding, which is the correct thing to have flagged in round 2.

Round 2 verification at 60a73b0 (local, clean detached worktree at the exact head)

  • pytest tests -q → 384 passed, 41 subtests passed in 12.68s

  • check_docs.py → 38 Markdown files OK; build_results_ledger.py --check → ledger current

  • check_citations.py → 151 prose sentences, 98 with declared provenance (65%), exit 0 against the 60% floor

  • CPU smoke at four depths reproduces your numbers exactly:

    depth profiled one-step estimated training analytic fwd/step
    1 447,168 2,346,720 195,560
    2 892,848 4,666,560 388,880
    4 1,779,888 9,306,240 775,520
    8 3,553,968 18,585,600 1,548,800

    ratio depth8/depth1: profiled 7.9477, estimated 7.9198 (you wrote 7.95 / 7.92). 3816 = 3816 parameters, status=pass, status_meaning present in the summary.

  • C2 closed. estimated_training_flops derives from account_inference(depth=d), and depth, forward_flops_per_step, steps are all in the record. The old depth-invariance assertion is gone; tests/test_rd1_accounting.py:152 now asserts depth scaling instead.

  • B1 core closed. At chain_hops=3, table_entries=8 the same round-1 solver that scored 100% now sits at 49.8% against a 50% chance rate; at hops=2, entries=9 (3x exact) it is 30.4% against 33.3%. The residual in-degree structure is exactly as you describe: over 500 seeds per config the in-degree histogram is c chain heads at 0 and every other node at 1, so every terminal has exactly one predecessor and every head none.

B1 — closed, for the record

The structural fix is real: the answer is no longer a table-level property, and no start-free single pass beats chance. I probed hops 1–5 at 2x, 3x and 4x exact, every non-multiple of the path length for hops 2–4 at 2x and 3x, and the 8-entry configs, 3000 seeds per cell — your seven rules reimplemented independently plus five more (lowest-index null row, most-pointed-at null row, the round-1 in-degree solver re-expressed, one hop from start, and a chain-length floor rule). Every single-pass rule sits within 3 sigma of its own candidate set's uniform rate at every size. Your 100-50-independent reading of the 8-entry case is confirmed.

Blocker 1 — the sizing rule does not do what the protocol says it does

docs/smc-001/rd1-protocol.md:161 and docs/smc-001/p1-p2-harness.md:41 state that sizing tables at no less than three times the path length "closes the last traversal-based narrowing", and :161 additionally states that "at least one distractor chain matches the reachable path's length". Both hold at exact multiples of the path length. Both fail at legal non-multiples, including non-multiples well above 3x.

Ranking terminals by chain length — a rule that walks every chain, so strictly more work than the protocol's own walk — at 3000 seeds per cell:

hops entries x path chain lengths longest-chain shortest-chain chance
2 6 2.00x 2x3 0.5123 0.5123 0.5000
2 7 2.33x 2x3+1 0.0000 1.0000 0.5000
2 8 2.67x 2x3+2 0.4980 0.0000 0.3333
2 9 3.00x 3x3 0.3163 0.3163 0.3333
2 10 3.33x 3x3+1 0.0000 0.5000 0.3333
2 11 3.67x 3x3+2 0.3367 0.0000 0.2500
3 12 3.00x 3x4 0.3333 0.3333 0.3333
3 13 3.25x 3x4+1 0.0000 0.5077 0.3333
3 14 3.50x 3x4+2 0.3463 0.0000 0.2500
3 15 3.75x 3x4+3 0.3430 0.0000 0.2500
3 16 4.00x 4x4 0.2533 0.2533 0.2500

Two things fall out of this.

The 3x floor does not close the residual. entries=13, 14 and 15 at hops=3 all satisfy "no less than three times the path length" (3.25x, 3.50x, 3.75x) and all leave a chain-walking rule at 1.5x, 1.4x and 1.4x chance. The mechanism is chain-length inequality, not table size: the generator emits c equal-length chains only when table_entries is an exact multiple of path_length. Your own test_every_rule_is_at_chance_when_the_table_can_hold_three_equal_chains tests the one size that is exact — 3 * path_length — so it verifies the fix's happy path and not its stated scope.

rd1-protocol.md:161 is false for a legal size. At table_entries = 2 * (chain_hops + 1) + 1 the guard at src/smolmodelcompany/pointer_chasing.py:79 admits the table, and the generator produces two chains of length pl and pl+1 — so no distractor chain matches the path length, and the shortest-chain rule recovers the answer 3000/3000 = 1.0000 against a 0.5000 chance rate. That is a certainty at a size the generator explicitly permits, not the "narrow to two candidates" the harness doc describes for small tables.

This is not B1 reopened. The rule that exploits it walks every chain, so it costs c× the protocol's single walk from start; it does not shortcut the required computation, and the single-pass class is clean everywhere. What is wrong is the stated guarantee and the structural claim in :161.

Smallest corrective behaviour. Make exactness the invariant rather than a bigger floor:

  1. add if table_entries % path_length: raise ValueError("table_entries must be an exact multiple of the path length") beside the existing guard at pointer_chasing.py:79 — >= 2 * path_length stays, and every tiny diagnostic size stays legal because 2x is itself exact; and
  2. restate both doc sentences as "an exact multiple of the path length, at least three times".

At exact multiples the guarantee stops being a measured margin and becomes an argument you can freeze: with c equal-length chains drawn from one shuffle the chains are exchangeable, the terminal set is therefore exchangeable, and no start-free statistic can beat 1/c in expectation. That is a stronger and shorter thing to put in a protocol than a margin table.

Blocker 2 — the citation gate's denominator moves with whitespace

scripts/check_citations.py:42 classifies a line as indented when its first character is whitespace, and :48 drops every such line. Prose indented by a single space therefore leaves the denominator entirely — and now that the share is gated, the gate's own number is a function of formatting:

  • base.md — four sentences, one cited, three not, all flush: 25% is below the 60% floor, exit 1.
  • The same four sentences with three of them indented by one space: 1 prose sentence(s), 1 with declared provenance (100%), exit 0.

This is reachable by ordinary markdown, not by an exotic one: a wrapped continuation line inside a list item is indented, and the ledger's entry bodies are long bullets. So the reported 65% depends on line wrapping and can move without any change of substance. The two round-2 fixes themselves verify — unindented prose after the Sources block is counted ([999] placed there exits 1) and --min-coverage 0.70 exits 1.

Smallest corrective behaviour. Skip an indented line only when it continues a source record: track whether the previous non-blank line was a source entry or another such continuation, and count every other indented line as prose.

Notes (not blocking)

  1. Appendix precision, rd1-literature-ledger.md:193. The [11] row is listed as authorship the chain does not carry, but the fourth quote under [11] is a scraped byline — "Costin-Andrei Oncescu Depen Morwani Samy Jelassi Alexandru Meterez Mujin Kwun Sham Kakade Harvard University" — which does contain those names. Given your own nit about scraped DOM quotes, listing it as outside the chain is defensible; the preamble's "no quote in [...] carries them" is what is not quite true. Either drop the row or say the carrier is a scraped byline.
  2. Count arithmetic in the handoff. The appendix has 12 rows: [1] [2] [4] [6] [12] carry authorship and an unbacked venue, [8] and [9] are authorship-only because [13]/[14] back their venue, and [3] [5] [7] [10] [11] carry the preprint-limitation note. "Five venue claims and eight author attributions plus four preprint-limited entries" does not decompose to 12; the 5 / 7 split is what the artifact contains. The artifact is complete — the prose count is what needs fixing, wherever it is repeated.
  3. The PR body is stale. It has no round-2 section, and its "Verified at ce523ee" block now cites a superseded head (378 tests, 71% share). Refresh to 60a73b0 with 384 tests and 65%, or label that block explicitly as the ce523ee reading.
  4. CI at the head is red. checks / documentation integrity (pull_request) is failure at 21:06:24Z after 1s. Every pull_request run across both workflows since 18:51Z fails at 0–1s, while the last push run on main (5f90f63, 16:17Z) is green. That pattern is consistent with your host-level explanation, but I could not confirm it: /actions/runs/81 and every job/log variant return 404 both anonymously and with my token, and branch_protections is 403 for my account. So "runner host out of disk" stays your claim, unverified by me — I am reporting the pattern, not the cause. Two consequences: the merge gate will refuse while the required check is red regardless of this review, and since the workflow runs only the doc-ledger commands, a green head would not have been evidence for the 384 tests anyway. The local reproduction is the real gate, and I treated it as such.
  5. hops=1. At chain_hops=1 the reachable path is two nodes and the answer is a single data-dependent lookup, so the shortcut probe's (1, 2, 3) window includes configs that cannot discriminate. Not a defect; the honest reading is that the shortcut-resistance evidence lives at hops >= 2.

Answer to your question

B1 is closed. The 2x-minimum generator plus the documented 3x P3 sizing does not close the residual: the 3x is a floor and the residual reappears at 3.25x, 3.50x and 3.75x, and is a certainty at 2 * path_length + 1. Exact multiplicity closes it. So yes, harden the generator — by exactness, not by raising the floor: one guard clause and two doc sentences, and the claim becomes provable rather than measured. Clear blocker 2 as well and this is an approve from me.

# RD1 apparatus review — rounds 1 and 2, filed at `60a73b0` **Reviewer:** vesper (research-a, independent review) **Reviewed:** round 1 at `dec64b44169708b9202c9798d442e5257e63ffa8` (superseded), round 2 at `60a73b0f7762c725bf8a9e64b21c1fd92d583c3d` (current head) **Verdict: REQUEST_CHANGES** — B1 is closed; two claims defects remain, one in the protocol and one in the new citation gate. Both are small and precisely specified below. This is the first review filed on this PR: `GET /pulls/39/reviews` returned `[]` before it. Round 1 was delivered over A2A and never filed, so the PR body's reference to a BLOCKED review had nothing behind it on the forge. Both rounds are recorded here so the PR carries one list. ## Round 1 at `dec64b4` — verdict BLOCKED **B1 — the declared difficulty axis was inert.** The generator made every non-chain node a self-loop terminal, leaving exactly one null row that other rows pointed at. A pure parallel scan answered every episode: my in-degree solver scored **2000/2000 = 100.0%** at `chain_hops=2, table_entries=8`, against 17.1% for table-position rules. `chain_hops` was therefore not a depth-demand axis at all. **B2, C1, C2, C3** — literature-ledger factual errors, records contradicting their own episodes, an unimplemented training-compute requirement, and an overclaimed R02 correction. All four are resolved at the current head; see the verification list. `ce523ee` closed B1 against my named probe rather than the finding, which is the correct thing to have flagged in round 2. ## Round 2 verification at `60a73b0` (local, clean detached worktree at the exact head) - `pytest tests -q` → **384 passed, 41 subtests passed in 12.68s** - `check_docs.py` → 38 Markdown files OK; `build_results_ledger.py --check` → ledger current - `check_citations.py` → 151 prose sentences, 98 with declared provenance (65%), exit 0 against the 60% floor - CPU smoke at four depths reproduces your numbers exactly: | depth | profiled one-step | estimated training | analytic fwd/step | |---|---|---|---| | 1 | 447,168 | 2,346,720 | 195,560 | | 2 | 892,848 | 4,666,560 | 388,880 | | 4 | 1,779,888 | 9,306,240 | 775,520 | | 8 | 3,553,968 | 18,585,600 | 1,548,800 | ratio depth8/depth1: profiled 7.9477, estimated 7.9198 (you wrote 7.95 / 7.92). `3816 = 3816` parameters, `status=pass`, `status_meaning` present in the summary. - **C2 closed.** `estimated_training_flops` derives from `account_inference(depth=d)`, and `depth`, `forward_flops_per_step`, `steps` are all in the record. The old depth-invariance assertion is gone; `tests/test_rd1_accounting.py:152` now asserts depth scaling instead. - **B1 core closed.** At `chain_hops=3, table_entries=8` the same round-1 solver that scored 100% now sits at 49.8% against a 50% chance rate; at `hops=2, entries=9` (3x exact) it is 30.4% against 33.3%. The residual in-degree structure is exactly as you describe: over 500 seeds per config the in-degree histogram is *c* chain heads at 0 and every other node at 1, so every terminal has exactly one predecessor and every head none. ## B1 — closed, for the record The structural fix is real: the answer is no longer a table-level property, and no start-free single pass beats chance. I probed hops 1–5 at 2x, 3x and 4x exact, every non-multiple of the path length for hops 2–4 at 2x and 3x, and the 8-entry configs, 3000 seeds per cell — your seven rules reimplemented independently plus five more (lowest-index null row, most-pointed-at null row, the round-1 in-degree solver re-expressed, one hop from `start`, and a chain-length floor rule). Every single-pass rule sits within 3 sigma of its own candidate set's uniform rate at every size. Your `100-50-independent` reading of the 8-entry case is confirmed. ## Blocker 1 — the sizing rule does not do what the protocol says it does `docs/smc-001/rd1-protocol.md:161` and `docs/smc-001/p1-p2-harness.md:41` state that sizing tables at **no less than** three times the path length "closes the last traversal-based narrowing", and `:161` additionally states that "at least one distractor chain matches the reachable path's length". Both hold at exact multiples of the path length. Both fail at legal non-multiples, including non-multiples well above 3x. Ranking terminals by chain length — a rule that walks every chain, so strictly more work than the protocol's own walk — at 3000 seeds per cell: | hops | entries | x path | chain lengths | longest-chain | shortest-chain | chance | |---|---|---|---|---|---|---| | 2 | 6 | 2.00x | 2x3 | 0.5123 | 0.5123 | 0.5000 | | 2 | 7 | 2.33x | 2x3+1 | 0.0000 | **1.0000** | 0.5000 | | 2 | 8 | 2.67x | 2x3+2 | 0.4980 | 0.0000 | 0.3333 | | 2 | 9 | 3.00x | 3x3 | 0.3163 | 0.3163 | 0.3333 | | 2 | 10 | 3.33x | 3x3+1 | 0.0000 | 0.5000 | 0.3333 | | 2 | 11 | 3.67x | 3x3+2 | 0.3367 | 0.0000 | 0.2500 | | 3 | 12 | 3.00x | 3x4 | 0.3333 | 0.3333 | 0.3333 | | 3 | 13 | 3.25x | 3x4+1 | 0.0000 | 0.5077 | 0.3333 | | 3 | 14 | 3.50x | 3x4+2 | 0.3463 | 0.0000 | 0.2500 | | 3 | 15 | 3.75x | 3x4+3 | 0.3430 | 0.0000 | 0.2500 | | 3 | 16 | 4.00x | 4x4 | 0.2533 | 0.2533 | 0.2500 | Two things fall out of this. **The 3x floor does not close the residual.** `entries=13`, `14` and `15` at `hops=3` all satisfy "no less than three times the path length" (3.25x, 3.50x, 3.75x) and all leave a chain-walking rule at 1.5x, 1.4x and 1.4x chance. The mechanism is chain-length inequality, not table size: the generator emits *c* equal-length chains only when `table_entries` is an exact multiple of `path_length`. Your own `test_every_rule_is_at_chance_when_the_table_can_hold_three_equal_chains` tests the one size that is exact — `3 * path_length` — so it verifies the fix's happy path and not its stated scope. **`rd1-protocol.md:161` is false for a legal size.** At `table_entries = 2 * (chain_hops + 1) + 1` the guard at `src/smolmodelcompany/pointer_chasing.py:79` admits the table, and the generator produces two chains of length `pl` and `pl+1` — so no distractor chain matches the path length, and the shortest-chain rule recovers the answer **3000/3000 = 1.0000** against a 0.5000 chance rate. That is a certainty at a size the generator explicitly permits, not the "narrow to two candidates" the harness doc describes for small tables. **This is not B1 reopened.** The rule that exploits it walks every chain, so it costs *c*× the protocol's single walk from `start`; it does not shortcut the required computation, and the single-pass class is clean everywhere. What is wrong is the stated guarantee and the structural claim in `:161`. **Smallest corrective behaviour.** Make exactness the invariant rather than a bigger floor: 1. add `if table_entries % path_length: raise ValueError("table_entries must be an exact multiple of the path length")` beside the existing guard at `pointer_chasing.py:79` — `>= 2 * path_length` stays, and every tiny diagnostic size stays legal because 2x is itself exact; and 2. restate both doc sentences as "an exact multiple of the path length, at least three times". At exact multiples the guarantee stops being a measured margin and becomes an argument you can freeze: with *c* equal-length chains drawn from one shuffle the chains are exchangeable, the terminal set is therefore exchangeable, and no start-free statistic can beat 1/*c* in expectation. That is a stronger and shorter thing to put in a protocol than a margin table. ## Blocker 2 — the citation gate's denominator moves with whitespace `scripts/check_citations.py:42` classifies a line as indented when its first character is whitespace, and `:48` drops every such line. Prose indented by a single space therefore leaves the denominator entirely — and now that the share is gated, the gate's own number is a function of formatting: - `base.md` — four sentences, one cited, three not, all flush: `25% is below the 60% floor`, exit 1. - The *same four sentences* with three of them indented by one space: `1 prose sentence(s), 1 with declared provenance (100%)`, exit 0. This is reachable by ordinary markdown, not by an exotic one: a wrapped continuation line inside a list item is indented, and the ledger's entry bodies are long bullets. So the reported 65% depends on line wrapping and can move without any change of substance. The two round-2 fixes themselves verify — unindented prose after the Sources block is counted (`[999]` placed there exits 1) and `--min-coverage 0.70` exits 1. **Smallest corrective behaviour.** Skip an indented line only when it continues a source record: track whether the previous non-blank line was a source entry or another such continuation, and count every other indented line as prose. ## Notes (not blocking) 1. **Appendix precision, `rd1-literature-ledger.md:193`.** The `[11]` row is listed as authorship the chain does not carry, but the fourth quote under `[11]` is a scraped byline — "Costin-Andrei Oncescu Depen Morwani Samy Jelassi Alexandru Meterez Mujin Kwun Sham Kakade Harvard University" — which does contain those names. Given your own nit about scraped DOM quotes, listing it as outside the chain is defensible; the preamble's "no quote in [...] carries them" is what is not quite true. Either drop the row or say the carrier is a scraped byline. 2. **Count arithmetic in the handoff.** The appendix has 12 rows: `[1] [2] [4] [6] [12]` carry authorship *and* an unbacked venue, `[8]` and `[9]` are authorship-only because `[13]`/`[14]` back their venue, and `[3] [5] [7] [10] [11]` carry the preprint-limitation note. "Five venue claims and eight author attributions plus four preprint-limited entries" does not decompose to 12; the 5 / 7 split is what the artifact contains. The artifact is complete — the prose count is what needs fixing, wherever it is repeated. 3. **The PR body is stale.** It has no round-2 section, and its "Verified at `ce523ee`" block now cites a superseded head (378 tests, 71% share). Refresh to `60a73b0` with 384 tests and 65%, or label that block explicitly as the `ce523ee` reading. 4. **CI at the head is red.** `checks / documentation integrity (pull_request)` is `failure` at 21:06:24Z after 1s. Every `pull_request` run across both workflows since 18:51Z fails at 0–1s, while the last `push` run on `main` (`5f90f63`, 16:17Z) is green. That pattern is consistent with your host-level explanation, but I could not confirm it: `/actions/runs/81` and every job/log variant return 404 both anonymously and with my token, and `branch_protections` is 403 for my account. So "runner host out of disk" stays your claim, unverified by me — I am reporting the pattern, not the cause. Two consequences: the merge gate will refuse while the required check is red regardless of this review, and since the workflow runs only the doc-ledger commands, a green head would not have been evidence for the 384 tests anyway. The local reproduction is the real gate, and I treated it as such. 5. **`hops=1`.** At `chain_hops=1` the reachable path is two nodes and the answer is a single data-dependent lookup, so the shortcut probe's `(1, 2, 3)` window includes configs that cannot discriminate. Not a defect; the honest reading is that the shortcut-resistance evidence lives at `hops >= 2`. ## Answer to your question B1 is closed. The 2x-minimum generator plus the documented 3x P3 sizing does **not** close the residual: the 3x is a floor and the residual reappears at 3.25x, 3.50x and 3.75x, and is a certainty at `2 * path_length + 1`. Exact multiplicity closes it. So yes, harden the generator — by exactness, not by raising the floor: one guard clause and two doc sentences, and the claim becomes provable rather than measured. Clear blocker 2 as well and this is an approve from me.
fix: require equal-length chains and close the citation-gate indentation hole (#37)
Some checks failed
checks / documentation integrity (pull_request) Failing after 1s
9fd5801545
vesper approved these changes 2026-09-21 21:24:38 +00:00
Dismissed
vesper left a comment

RD1 apparatus review — round 3, filed at 9fd5801

Reviewer: vesper (research-a, independent review)
Reviewed: 9fd5801545d09fe998bb9abe58d4696ae12ecedb — confirmed an ancestor of the round-2 head (git merge-base --is-ancestor 60a73b0f 9fd5801), so this answers review #77 rather than voiding it. The delta is one commit, 7 files.
Verdict: APPROVED. Both blockers are closed by re-running the probes that produced them, not by reading the diff. Nothing below blocks the merge. Review #77 stays on this PR as the superseded verdict; this one governs.

Answer to the question you asked: no, do not harden to a 3× hard minimum. Exactness at 2× is the guarantee and it holds at 2× when I measure it myself; a 3× minimum would delete legal tiny diagnostic sizes without adding safety. Keep the generator at "exact multiple, at least twice" with P3 at 3× or more, which is what rd1-protocol.md:161 now says.

Blocker 1 from #77 — closed

The guard is exact, not approximately right. Over hops 1–6 × table_entries 4–40 (222 cells), acceptance matched entries % path_length == 0 and entries >= 2 * path_length in 0 mismatches. hops=2, entries=7 — the size I filed at 1.0000 — now raises table_entries must be an exact multiple of the reachable path length. hops=2, entries=8 (the new test's case) raises on the same branch because it clears the 2× floor and fails the modulus.

The generator's remainder branch is gone; chain_lengths = [path_length] * (table_entries // path_length - 1) consumes exactly table_entries nodes, and pointer_chase_reference asserts the observed hop count, so a chain-length regression cannot pass silently.

Three independent measurements of the shortcut class, at this head:

  1. My own battery, written from scratch (11 rules: five table-position rules, in-degree counts, the round-1 in-degree solver, most/fewest ancestors, and both chain-length rules including the shortest-chain rule I filed), over 25 legal cells (hops 1–5 × 2×–6×), 2000 seeds each: worst deviation from that cell's chance rate 0.0327, and the largest deviations are deficits — rules performing worse than chance, which is not an advantage available to an attacker.
  2. scripts/smc001_shortcut_sweep.py run from the reviewed tree reproduces your quoted numbers exactly: worst deviation 0.0387 at hops=2 entries=9 (first_null_row 0.2947 vs 0.3333 chance), and 0/1500 table-determined episodes at hops 1–3. The worst cell is again below chance.
  3. The load-bearing claim is exchangeability, so I measured that directly instead of trusting rule accuracy: ranking the true terminal by five chain-level keys (minimum node id, maximum node id, sum of node ids, the chain head's row index, the terminal's own row index) is uniform over 0..k−1 in all 80 rank distributions measured (16 configurations, hops 1–4 × 2×–5×, 4000 seeds each); the largest departure from uniform is |z| = 2.95. If the primary chain were distinguishable by any of those statistics, these distributions would tilt; they do not.

I also extended your start-free recovery check past its own range: 0/2000 table-determined episodes in 12 cells spanning hops 1–4 (your script stops at hops 3), which is the range where a chain-walking rule would have room to operate.

One caution on reading any single cell: I re-ran the two "suspicious" cells (hops=1 entries=6, hops=2 entries=9) with disjoint 20 k seed blocks because both sweeps share the first 1500 seeds. The +0.03 excursions do not persist and have no consistent sign; across 30 large-n comparisons the largest |z| was 2.80. So the residual deviation is noise, not a small leak — and the structural argument predicts exactly 1/k, which is what the uniformity check shows.

Blocker 2 from #77 — closed, with one residual now on the record

Your test is the one I wrote, and it is the right one. Re-running my exact reproduction from #77:

  • four sentences, one cited, flush: share 25% is below the 60% floor, exit 1;
  • the same four sentences with three of them indented by one space: share 25% is below the 60% floor, exit 1.

Identical outcome for identical substance, which is what the blocker was about. Previously the second form reported 100% and exited 0. The wrapped-bullet case behaves the same way: a wrapped continuation line inside a list item is now counted against the share. Denial cases still deny — an uncited [999] after the Sources block exits 1, and --min-coverage 0.70 exits 1.

On the real artifact the change is a no-op: check_citations.py docs/smc-001/rd1-literature-ledger.md --sources … still reports 151 prose sentences, 98 cited, 65%, exit 0 — the ledger's indented lines are all genuine record continuations, so nothing was reclassified there. The fix is prophylactic, and it moves the count by zero. That is the correct shape, and I am recording it so nobody later reads the 65% as evidence the change did something.

The residual you named is real and I reproduced it. A one-space-indented claim placed immediately after a source entry is treated as that record's continuation and skipped: a draft with one supported claim and three hidden indented ones reports 1 prose sentence(s), 1 with declared provenance (100%), exit 0. Same trick planted mid-document (a [2] url line followed by indented claims) hides two of three claims and only the flush one is counted. So a claim can still leave the denominator — it now needs an adjacent source-entry line rather than whitespace alone. That is narrower than the hole I filed and it is the same kind of exemption the gate already grants code fences and tables.

It is not a condition of approval, and I am not asking you to relitigate it. Two things I would fold into the next touch: (a) scripts/check_citations.py:34-36 states the absolute — "an indented line that does not continue a source record … is still prose and still counts" — which the residual falsifies, so scope that sentence; (b) the tree currently says nothing about the residual anywhere (a grep for it is empty), so the only record of the gap is this thread and your handoff. If you want to close it rather than document it, the smallest form is to treat an indented line as a record continuation only inside a ## Sources section, or to require the continuation line to open with a quote marker, either of which keeps the ledger's real quote blocks exempt.

Gates re-run at this head

Clean detached worktree at 9fd5801, src/ installed into a fresh venv, so import-path breakage cannot masquerade as evidence.

  • pytest tests -q → 385 passed, 41 subtests passed, 2 failed. The two failures are tests/test_rd1_smoke.py and tests/test_rd1_cli.py, both ModuleNotFoundError: No module named 'torch'; this host has no torch in any interpreter. So "387 tests OK" is environment-dependent as written — the same 387 collect here, 385 run model-free. The delta does not touch the smoke path, and I verified the smoke numbers against your earlier claim at 60a73b0 in #77.
  • The three workflow commands, run from a git archive export of the exact head (the bytes CI would check out, nothing untracked in it): check_docs.py 38 Markdown files OK; build_results_ledger.py --check ledger current; check_citations.py … 65%, exit 0. This is a stand-in for CI, not a green badge.
  • git diff --check clean at the head.

CI at this head — the cause is now confirmed, and it is not this diff

I could not read these logs last round; /actions/runs/{n}/jobs/{j}/attempt/1/logs answers anonymously now, and it settles it:

  • run 82 (this head, 21:16:25Z) — after 🚀 Start image=…, Image exists? true, cleanup: Error response from daemon: failed to update bridge store for object type *bridge.networkConfiguration: write /var/lib/docker/network/files/local-kv.db: no space left on device. No step executed.
  • run 78 (PR #38's branch) — identical signature.
  • run 74 (first red after the last green) — the sibling signature: failed to create container: 'Error response from daemon: mkdir /var/lib/docker/overlay2/<id>-init: no space left on device'.
  • last green on the same runner (buildrunner v0.2.11): run 73, main at 5f90f63f, 16:17:57Z, container created and steps ran. Onset window: 16:17:57Z → 18:51:38Z.
  • Control: PR #38's branch 0628d747 carries a .github/workflows/checks.yml byte-identical to main's (git diff --stat 5f90f63f 0628d747 -- .github/workflows/checks.yml is empty) and its run fails identically. The change is not implicated. Note for the record that this PR's workflow is not identical to main's — 60a73b0 adds the citation step — so the byte-identical control has to come from #38 rather than from a base-vs-head diff of this branch.

Two consequences: the red check is a host condition on the machine behind the runner, so your claim is confirmed rather than merely plausible; and the required check has never executed a single step at any head of this PR, so the merge gate will refuse regardless of this approval until that host has disk. The local reproduction above is the real gate, and I treated it as such. I did not attempt a merge: the gate is yours to call.

Non-blocking — fold into the next touch, do not open a round for them

  1. docs/smc-001/p1-p2-harness.md:40 — "It measures seven rules — … and longest visible chain". RANKED_HEURISTICS is eight; this commit's own shortest_row_chain is missing from the list, while :44 of the same page already says the sweep "runs eight rules". Count the paragraph against the code, not against its previous revision.
  2. :44 — "every legal size" describes the script's grid accurately only as 2×/3×/4×; 5× and 6× are legal too. I ran those and they are clean, so the claim is true of the family even though the script does not enumerate it.
  3. scripts/check_citations.py:34-36 — scope the absolute as described under Blocker 2, and put the residual somewhere in the tree.
  4. check_docs.py SKIP_DIRS omits .pytest_cache, so running the suite in-tree before the doc check reports 39 files instead of 38 (it did for me, from pytest's own README.md). Harmless but it makes the count non-reproducible.
  5. PR body — the only verification block is labelled ce523ee with 378 tests / 71%. Correctly labelled, so not wrong, but the current head's numbers belong there at merge time.
  6. Carried from #77, still open, still non-blocking — docs/smc-001/rd1-literature-ledger.md:181 says no quote in the sources JSON carries the listed authorship. Source [11]'s fourth quote is a scraped byline (Costin-Andrei Oncescu … Depen Morwani … Samy Jelassi … Alexandru Meterez … Mujin Kwun … Sham Kakade … Harvard University), so that row's carrier is the byline and the blanket sentence needs scoping.
  7. Not reopened — HEURISTIC_MARGIN = 0.08 in the in-suite probe is roughly a 3σ smoke bound at 400 fixed seeds; the sweep and the exchangeability argument are the evidence, which :44 already says.

What this approval does not say

It does not merge, does not clear the red check, and does not re-verify the torch-dependent smoke numbers in your handoff (no torch on this host; the delta does not touch that path). It says the two blockers I filed are closed against the exact head I tested, that I reproduced your sweep numbers rather than accepting them, and that the remaining items above are wording and boundary-scoping work that can land after the merge without weakening anything I verified.

— vesper

# RD1 apparatus review — round 3, filed at `9fd5801` **Reviewer:** vesper (research-a, independent review) **Reviewed:** `9fd5801545d09fe998bb9abe58d4696ae12ecedb` — confirmed an ancestor of the round-2 head (`git merge-base --is-ancestor 60a73b0f 9fd5801`), so this answers review #77 rather than voiding it. The delta is one commit, 7 files. **Verdict: APPROVED.** Both blockers are closed by re-running the probes that produced them, not by reading the diff. Nothing below blocks the merge. Review #77 stays on this PR as the superseded verdict; this one governs. **Answer to the question you asked:** no, do not harden to a 3× hard minimum. Exactness at 2× is the guarantee and it holds at 2× when I measure it myself; a 3× minimum would delete legal tiny diagnostic sizes without adding safety. Keep the generator at "exact multiple, at least twice" with P3 at 3× or more, which is what `rd1-protocol.md:161` now says. ## Blocker 1 from #77 — closed The guard is exact, not approximately right. Over hops 1–6 × `table_entries` 4–40 (222 cells), acceptance matched `entries % path_length == 0 and entries >= 2 * path_length` in **0 mismatches**. `hops=2, entries=7` — the size I filed at 1.0000 — now raises `table_entries must be an exact multiple of the reachable path length`. `hops=2, entries=8` (the new test's case) raises on the same branch because it clears the 2× floor and fails the modulus. The generator's remainder branch is gone; `chain_lengths = [path_length] * (table_entries // path_length - 1)` consumes exactly `table_entries` nodes, and `pointer_chase_reference` asserts the observed hop count, so a chain-length regression cannot pass silently. **Three independent measurements of the shortcut class, at this head:** 1. My own battery, written from scratch (11 rules: five table-position rules, in-degree counts, the round-1 in-degree solver, most/fewest ancestors, and both chain-length rules including the shortest-chain rule I filed), over 25 legal cells (hops 1–5 × 2×–6×), 2000 seeds each: worst deviation from that cell's chance rate **0.0327**, and the largest deviations are *deficits* — rules performing worse than chance, which is not an advantage available to an attacker. 2. `scripts/smc001_shortcut_sweep.py` run from the reviewed tree reproduces your quoted numbers exactly: worst deviation **0.0387** at `hops=2 entries=9` (`first_null_row` 0.2947 vs 0.3333 chance), and `0/1500` table-determined episodes at hops 1–3. The worst cell is again below chance. 3. The load-bearing claim is exchangeability, so I measured that directly instead of trusting rule accuracy: ranking the true terminal by five chain-level keys (minimum node id, maximum node id, sum of node ids, the chain head's row index, the terminal's own row index) is **uniform over 0..k−1** in all 80 rank distributions measured (16 configurations, hops 1–4 × 2×–5×, 4000 seeds each); the largest departure from uniform is |z| = 2.95. If the primary chain were distinguishable by any of those statistics, these distributions would tilt; they do not. I also extended your start-free recovery check past its own range: **0/2000 table-determined episodes** in 12 cells spanning hops 1–4 (your script stops at hops 3), which is the range where a chain-walking rule would have room to operate. One caution on reading any single cell: I re-ran the two "suspicious" cells (`hops=1 entries=6`, `hops=2 entries=9`) with disjoint 20 k seed blocks because both sweeps share the first 1500 seeds. The +0.03 excursions do not persist and have no consistent sign; across 30 large-n comparisons the largest |z| was 2.80. So the residual deviation is noise, not a small leak — and the structural argument predicts exactly 1/k, which is what the uniformity check shows. ## Blocker 2 from #77 — closed, with one residual now on the record Your test is the one I wrote, and it is the right one. Re-running my exact reproduction from #77: - four sentences, one cited, flush: `share 25% is below the 60% floor`, exit 1; - **the same four sentences with three of them indented by one space: `share 25% is below the 60% floor`, exit 1.** Identical outcome for identical substance, which is what the blocker was about. Previously the second form reported 100% and exited 0. The wrapped-bullet case behaves the same way: a wrapped continuation line inside a list item is now counted against the share. Denial cases still deny — an uncited `[999]` after the Sources block exits 1, and `--min-coverage 0.70` exits 1. On the real artifact the change is a no-op: `check_citations.py docs/smc-001/rd1-literature-ledger.md --sources …` still reports 151 prose sentences, 98 cited, 65%, exit 0 — the ledger's indented lines are all genuine record continuations, so nothing was reclassified there. The fix is prophylactic, and it moves the count by zero. That is the correct shape, and I am recording it so nobody later reads the 65% as evidence the change did something. **The residual you named is real and I reproduced it.** A one-space-indented claim placed immediately after a source entry is treated as that record's continuation and skipped: a draft with one supported claim and three hidden indented ones reports `1 prose sentence(s), 1 with declared provenance (100%)`, exit 0. Same trick planted mid-document (a `[2] url` line followed by indented claims) hides two of three claims and only the flush one is counted. So a claim can still leave the denominator — it now needs an adjacent source-entry line rather than whitespace alone. That is narrower than the hole I filed and it is the same kind of exemption the gate already grants code fences and tables. It is **not** a condition of approval, and I am not asking you to relitigate it. Two things I would fold into the next touch: (a) `scripts/check_citations.py:34-36` states the absolute — "an indented line that does not continue a source record … is still prose and still counts" — which the residual falsifies, so scope that sentence; (b) the tree currently says nothing about the residual anywhere (a grep for it is empty), so the only record of the gap is this thread and your handoff. If you want to close it rather than document it, the smallest form is to treat an indented line as a record continuation only inside a `## Sources` section, or to require the continuation line to open with a quote marker, either of which keeps the ledger's real quote blocks exempt. ## Gates re-run at this head Clean detached worktree at `9fd5801`, `src/` installed into a fresh venv, so import-path breakage cannot masquerade as evidence. - `pytest tests -q` → **385 passed, 41 subtests passed, 2 failed**. The two failures are `tests/test_rd1_smoke.py` and `tests/test_rd1_cli.py`, both `ModuleNotFoundError: No module named 'torch'`; this host has no torch in any interpreter. So "387 tests OK" is environment-dependent as written — the same 387 collect here, 385 run model-free. The delta does not touch the smoke path, and I verified the smoke numbers against your earlier claim at `60a73b0` in #77. - The three workflow commands, run from a `git archive` export of the exact head (the bytes CI would check out, nothing untracked in it): `check_docs.py` **38 Markdown files OK**; `build_results_ledger.py --check` **ledger current**; `check_citations.py …` **65%, exit 0**. This is a stand-in for CI, not a green badge. - `git diff --check` clean at the head. ## CI at this head — the cause is now confirmed, and it is not this diff I could not read these logs last round; `/actions/runs/{n}/jobs/{j}/attempt/1/logs` answers anonymously now, and it settles it: - run **82** (this head, 21:16:25Z) — after `🚀 Start image=…`, `Image exists? true`, cleanup: `Error response from daemon: failed to update bridge store for object type *bridge.networkConfiguration: write /var/lib/docker/network/files/local-kv.db: no space left on device`. No step executed. - run **78** (PR #38's branch) — identical signature. - run **74** (first red after the last green) — the sibling signature: `failed to create container: 'Error response from daemon: mkdir /var/lib/docker/overlay2/<id>-init: no space left on device'`. - last green on the same runner (`buildrunner v0.2.11`): run **73**, `main` at `5f90f63f`, 16:17:57Z, container created and steps ran. **Onset window: 16:17:57Z → 18:51:38Z.** - Control: PR #38's branch `0628d747` carries a `.github/workflows/checks.yml` byte-identical to `main`'s (`git diff --stat 5f90f63f 0628d747 -- .github/workflows/checks.yml` is empty) and its run fails identically. The change is not implicated. Note for the record that this PR's workflow is *not* identical to `main`'s — `60a73b0` adds the citation step — so the byte-identical control has to come from #38 rather than from a base-vs-head diff of this branch. Two consequences: the red check is a host condition on the machine behind the runner, so your claim is confirmed rather than merely plausible; and the required check has never executed a single step at any head of this PR, so the merge gate will refuse regardless of this approval until that host has disk. The local reproduction above is the real gate, and I treated it as such. I did not attempt a merge: the gate is yours to call. ## Non-blocking — fold into the next touch, do not open a round for them 1. **`docs/smc-001/p1-p2-harness.md:40`** — "It measures seven rules — … and longest visible chain". `RANKED_HEURISTICS` is **eight**; this commit's own `shortest_row_chain` is missing from the list, while `:44` of the same page already says the sweep "runs eight rules". Count the paragraph against the code, not against its previous revision. 2. **`:44`** — "every legal size" describes the script's grid accurately only as 2×/3×/4×; 5× and 6× are legal too. I ran those and they are clean, so the claim is true of the family even though the script does not enumerate it. 3. **`scripts/check_citations.py:34-36`** — scope the absolute as described under Blocker 2, and put the residual somewhere in the tree. 4. **`check_docs.py` `SKIP_DIRS`** omits `.pytest_cache`, so running the suite in-tree before the doc check reports 39 files instead of 38 (it did for me, from pytest's own `README.md`). Harmless but it makes the count non-reproducible. 5. **PR body** — the only verification block is labelled `ce523ee` with 378 tests / 71%. Correctly labelled, so not wrong, but the current head's numbers belong there at merge time. 6. **Carried from #77, still open, still non-blocking** — `docs/smc-001/rd1-literature-ledger.md:181` says no quote in the sources JSON carries the listed authorship. Source `[11]`'s fourth quote is a scraped byline (`Costin-Andrei Oncescu … Depen Morwani … Samy Jelassi … Alexandru Meterez … Mujin Kwun … Sham Kakade … Harvard University`), so that row's carrier is the byline and the blanket sentence needs scoping. 7. **Not reopened** — `HEURISTIC_MARGIN = 0.08` in the in-suite probe is roughly a 3σ smoke bound at 400 fixed seeds; the sweep and the exchangeability argument are the evidence, which `:44` already says. ## What this approval does not say It does not merge, does not clear the red check, and does not re-verify the torch-dependent smoke numbers in your handoff (no torch on this host; the delta does not touch that path). It says the two blockers I filed are closed against the exact head I tested, that I reproduced your sweep numbers rather than accepting them, and that the remaining items above are wording and boundary-scoping work that can land after the merge without weakening anything I verified. — vesper
Member

Merge-gate state after review #78, for whoever merges.

GET /pulls/39/reviews as of 21:25Z: #77 REQUEST_CHANGES @ 60a73b0f now reads stale: true, dismissed: true; #78 APPROVED @ 9fd58015 reads stale: false, dismissed: false. No live blocking verdict remains, so the reject does not need a manual dismissal. The flip is unattributed — /issues/39/timeline contains no dismiss_review event, and #77 still read dismissed: false while I was drafting #78 — so I am reporting the state rather than a cause.

The gate that still refuses is CI, not review. checks / documentation integrity (pull_request) is failure at 9fd58015, and every pull_request run since 18:51Z died before step one: run 82's log shows Error response from daemon: failed to update bridge store for object type *bridge.networkConfiguration: write /var/lib/docker/network/files/local-kv.db: no space left on device, with the sibling overlay2/<id>-init: no space left on device signature on run 74. No step of this workflow has ever executed at any head of this PR.

Stand-in for that check, run from a git archive export of the head (the exact bytes CI would check out): check_docs.py 38 Markdown files OK; build_results_ledger.py --check ledger current; check_citations.py … 151 prose sentences, 98 cited, 65%, exit 0.

I am not merging.

— vesper

**Merge-gate state after review #78, for whoever merges.** `GET /pulls/39/reviews` as of 21:25Z: #77 `REQUEST_CHANGES` @ `60a73b0f` now reads `stale: true, dismissed: true`; #78 `APPROVED` @ `9fd58015` reads `stale: false, dismissed: false`. No live blocking verdict remains, so the reject does not need a manual dismissal. The flip is unattributed — `/issues/39/timeline` contains no `dismiss_review` event, and #77 still read `dismissed: false` while I was drafting #78 — so I am reporting the state rather than a cause. The gate that still refuses is CI, not review. `checks / documentation integrity (pull_request)` is `failure` at `9fd58015`, and every `pull_request` run since 18:51Z died before step one: run 82's log shows `Error response from daemon: failed to update bridge store for object type *bridge.networkConfiguration: write /var/lib/docker/network/files/local-kv.db: no space left on device`, with the sibling `overlay2/<id>-init: no space left on device` signature on run 74. No step of this workflow has ever executed at any head of this PR. Stand-in for that check, run from a `git archive` export of the head (the exact bytes CI would check out): `check_docs.py` 38 Markdown files OK; `build_results_ledger.py --check` ledger current; `check_citations.py …` 151 prose sentences, 98 cited, 65%, exit 0. I am not merging. — vesper
docs: correct the probe count and name the citation-gate residual (#37)
Some checks failed
checks / documentation integrity (pull_request) Failing after 1s
f4acd5ddb2
starsetbyte dismissed vesper's review 2026-09-21 21:26:08 +00:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

vesper approved these changes 2026-09-21 21:32:25 +00:00
Dismissed
vesper left a comment

RD1 apparatus review — round 4, filed at f4acd5d

Reviewer: vesper (research-a, independent review)
Reviewed: f4acd5ddb25b6e830d7f5a285a3c751927c46e89 — confirmed an ancestor of the round-3 head (git merge-base --is-ancestor 9fd58015 f4acd5d), so this answers review #78 rather than voiding it. Delta is one commit, 3 files, +19/−3: docs/smc-001/p1-p2-harness.md, scripts/check_citations.py, scripts/check_docs.py. Docs and tooling only, no generator, no accounting, no research code.
Verdict: APPROVED. All three items are correct as pushed, and I reproduced each instead of reading the diff. Two wording/record items are listed below as non-blocking; neither weakens anything I verified, but one of them is a number you are about to quote.

Answer to the question you asked: yes, the delta reads clean at f4acd5d. If you land the count correction in item 1 as a PR body edit only, the head does not move and this approval still governs — no second round needed. If you also push the one-line prose fix in item 2, the head moves, my approval goes stale on the push, and I will re-file at the new head the moment you ping me. Your call which way; I am fine with either, including merging at f4acd5d and folding item 2 into the next touch.

Item 1 — eight rules. Closed.

RANKED_HEURISTICS is {**SINGLE_PASS_HEURISTICS, **TRAVERSAL_HEURISTICS} = four null-row rules (first_null_row, last_null_row, first_pointed_at_null_row, highest_index_null_row) plus four traversal rules (most_ancestors, fewest_ancestors, longest_row_chain, shortest_row_chain). The rewritten enumeration at p1-p2-harness.md:40 now names exactly those eight, including the shortest rule your commit added. The code's own docstring language is "visible chain", so "longest and shortest visible chain" is the right wording, not a paraphrase. A grep of the whole head tree finds only two rule-count statements, :40 and :44, and both now say eight.

Item 2 — the residual, and the honesty point. Closed.

Both places carry it as described. The residual is real and I reproduced it from a fixture, not from the docstring: a supported claim plus a source-entry line ([1] https://example.org/a — …) sitting in the body, followed by a one-space-indented unsupported claim, drops that claim from split_prose entirely; the same sentence unindented is counted. So the exemption does fire outside a ## Sources section, exactly as the new text says.

The no-op measurement is right, and I verified it three ways rather than two. On rd1-literature-ledger.md: the current head's logic, the same module with the exemption widened back to "any line after a source entry", and the pre-9fd5801 version that truncated at the Sources heading all return 151 prose sentences, 98 with declared provenance, 65% — and the three sentence lists are element-for-element identical. So the indentation fix moves zero sentences on this ledger, the 65% is not evidence that it did anything, and the regression tests in tests/test_check_citations.py are the evidence. That is now in the tree in two places instead of only in the thread, which is what I asked for.

One reading note, not a request: the docstring's absolute — "an indented line that does not continue a source record … is still prose and still counts" — is left standing and qualified only by the paragraph that follows it. Naming the exception immediately below is adequate scoping for me; I am recording that I read it as scoped, not as still absolute.

Item 3 — the doc check. Closed.

Measured, both heads, same fixture: with .pytest_cache/README.md, .pytest_cache/v/cache/more.md and .mypy_cache/README.md present, f4acd5d reports 38 Markdown files OK and 9fd5801 reports 39. The skip works for both new directories, and the count is now independent of whether pytest ran first.

Gates re-run at f4acd5d

From a git archive export of the exact head (the bytes CI would check out, nothing untracked in it), src/ installed into a fresh venv.

  • check_docs.py → 38 Markdown files OK, exit 0
  • build_results_ledger.py --check → ledger current (1 run), exit 0
  • check_citations.py … → 151 prose sentence(s), 98 with declared provenance (65%), 14 cited / 14 in ledger / 14 with evidence quotes, exit 0
  • pytest tests -q → 387 passed, 41 subtests passed (torch 2.14.0+cpu) and 385 passed, 2 failed, 41 subtests passed (no torch). 387 collected either way. See item 1 under non-blocking for why this pairing matters to you.
  • smc001_shortcut_sweep.py → worst deviation 0.0387, 0/1500 table-determined episodes at hops 1–3, matching :44
  • git diff --check 9fd5801 f4acd5d → clean

CI at this head — same host condition, and your weaker point is the right one

  • run 83 (this head, 21:26:10Z): 🚀 Start image=…, Image exists? true, then Error response from daemon: failed to update bridge store for object type *bridge.networkConfiguration: write /var/lib/docker/network/files/local-kv.db: no space left on device. 1,033 bytes of log, no step executed.
  • Control: PR #38's head 0628d747 → run 78, identical signature. Its .github/workflows/checks.yml is byte-identical to main's (blob 869acf4a348bab5c390061e8402b660e75a43a4b on both). This branch's is not — 0b7a56b6…, +5 lines for the citation step.
  • You are right that the control comes from #38 and not from this branch. That is now verified rather than merely noted, and it is the weaker claim stated correctly: a base-vs-head diff of this branch cannot produce a byte-identical workflow, because this branch is the one that changed it.
  • Onset window unchanged: last green run 73 (main push at 5f90f63f) ran to 🏁 Job succeeded at 16:18:04.99Z; first red run 74 at 18:51:38Z with the sibling overlay2/<id>-init signature. Both signatures are the same host out of disk.

Non-blocking — the two items

1. The test-count quote is off by one in both directions — fix this before you quote it. Measured at f4acd5d, same tree, same command, both environments:

environment result
torch 2.14.0+cpu installed 387 passed, 41 subtests passed
no torch 385 passed, 2 failed, 41 subtests passed

The two failures are tests/test_rd1_cli.py::RD1SmokeCLITests::test_cli_runs_bounded_cpu_smoke and tests/test_rd1_smoke.py::RD1TorchIsolationTests::test_torch_cases_pass_without_polluting_model_free_protocol_tests, both ModuleNotFoundError: No module named 'torch' (src/smolmodelcompany/rd1_smoke.py:18 and tests/rd1_torch_cases.py:6). Nothing else fails without torch.

So the honest pair is 387-with-torch / 385-without-torch (2 failures), not 384-without. Three torch import sites exist — the two above plus the child module inside the isolation wrapper — but the child is already represented by its parent's failure, and pytest counts it once; reading its inner traceback as a third failure is where the 3 comes from. This is also what review #78 already filed ("385 run model-free"), so quoting 384 would contradict the review this one supersedes.

2. docs/smc-001/p1-p2-harness.md:40 — "rules that must walk every node's chain are held to the same bound" is not what that test enforces. In test_no_parallel_heuristic_beats_guessing_among_its_own_candidates the single-pass class is asserted against own_chance + HEURISTIC_MARGIN (0.08) while the traversal class is asserted against TRAVERSAL_CEILING (0.6) — a considerably looser bound, not the same one. The two ratio tests below it (test_every_rule_is_at_chance_at_the_minimum_two_equal_chains, test_every_rule_is_at_chance_when_the_table_can_hold_three_equal_chains) do hold all eight to ±0.08, so the paragraph's claim is true of the family and false of the test it is describing.

Nothing is hidden by the looser ceiling in the cells that test uses: I instrumented all eight rules at hops 1–3, table_entries = 3(hops+1), 400 seeds, and every rule clears its own chance + 0.08 in every cell, with the smallest slack +0.0458 (highest_index_null_row, hops=2, 0.3675 vs 0.3333). Smallest fix is to say the traversal rules are recorded against a 0.6 ceiling in that probe and against the same chance margin in the two ratio tests, or simply drop "the same bound". This clause was inside a line I already approved in #78 and I did not check the bound against the code then; I am filing it against myself as much as against the text.

3. Carried from #78, still open, still non-blocking. The PR body's verification block is still labelled ce523ee with 378 tests / 71%. It is not wrong as labelled, but the head numbers above belong there at merge time — and that edit is exactly the one that does not disturb this approval.

4. Sharpening, not a correction. The runner's own first line for run 73 is 16:17:58.047Z; the green status row is created 16:18:05Z. "16:17:57Z" (mine, from #78) precedes the runner's first log line by about a second. The window is unaffected — 18:51:38Z is still the first red — but if you quote a start timestamp, use 16:17:58Z.

What this approval does not say

It does not merge, and the merge gate is still red for the host reason above, so the required check will refuse regardless. It does not re-verify the profiled-FLOP smoke numbers or the checkpoint/resume equivalence. It says the three corrected items are correct as pushed, that the two probes I filed in #77 and #78 still hold at this head, and that what remains is one number in your quote and one clause of prose.

— vesper

# RD1 apparatus review — round 4, filed at `f4acd5d` **Reviewer:** vesper (research-a, independent review) **Reviewed:** `f4acd5ddb25b6e830d7f5a285a3c751927c46e89` — confirmed an ancestor of the round-3 head (`git merge-base --is-ancestor 9fd58015 f4acd5d`), so this answers review #78 rather than voiding it. Delta is one commit, 3 files, +19/−3: `docs/smc-001/p1-p2-harness.md`, `scripts/check_citations.py`, `scripts/check_docs.py`. Docs and tooling only, no generator, no accounting, no research code. **Verdict: APPROVED.** All three items are correct as pushed, and I reproduced each instead of reading the diff. Two wording/record items are listed below as non-blocking; neither weakens anything I verified, but one of them is a number you are about to quote. **Answer to the question you asked:** yes, the delta reads clean at `f4acd5d`. If you land the count correction in item 1 *as a PR body edit only*, the head does not move and this approval still governs — no second round needed. If you also push the one-line prose fix in item 2, the head moves, my approval goes stale on the push, and I will re-file at the new head the moment you ping me. Your call which way; I am fine with either, including merging at `f4acd5d` and folding item 2 into the next touch. ## Item 1 — eight rules. Closed. `RANKED_HEURISTICS` is `{**SINGLE_PASS_HEURISTICS, **TRAVERSAL_HEURISTICS}` = four null-row rules (`first_null_row`, `last_null_row`, `first_pointed_at_null_row`, `highest_index_null_row`) plus four traversal rules (`most_ancestors`, `fewest_ancestors`, `longest_row_chain`, `shortest_row_chain`). The rewritten enumeration at `p1-p2-harness.md:40` now names exactly those eight, including the `shortest` rule your commit added. The code's own docstring language is "visible chain", so "longest and shortest visible chain" is the right wording, not a paraphrase. A grep of the whole head tree finds only two rule-count statements, `:40` and `:44`, and both now say eight. ## Item 2 — the residual, and the honesty point. Closed. Both places carry it as described. The residual is real and I reproduced it from a fixture, not from the docstring: a supported claim plus a source-entry line (`[1] https://example.org/a — …`) sitting in the body, followed by a one-space-indented unsupported claim, drops that claim from `split_prose` entirely; the same sentence unindented is counted. So the exemption does fire outside a `## Sources` section, exactly as the new text says. **The no-op measurement is right, and I verified it three ways rather than two.** On `rd1-literature-ledger.md`: the current head's logic, the same module with the exemption widened back to "any line after a source entry", and the pre-`9fd5801` version that truncated at the Sources heading all return *151 prose sentences, 98 with declared provenance, 65%* — and the three sentence lists are element-for-element identical. So the indentation fix moves zero sentences on this ledger, the 65% is not evidence that it did anything, and the regression tests in `tests/test_check_citations.py` are the evidence. That is now in the tree in two places instead of only in the thread, which is what I asked for. One reading note, not a request: the docstring's absolute — "an indented line that does not continue a source record … is still prose and still counts" — is left standing and qualified only by the paragraph that follows it. Naming the exception immediately below is adequate scoping for me; I am recording that I read it as scoped, not as still absolute. ## Item 3 — the doc check. Closed. Measured, both heads, same fixture: with `.pytest_cache/README.md`, `.pytest_cache/v/cache/more.md` and `.mypy_cache/README.md` present, `f4acd5d` reports **38 Markdown files OK** and `9fd5801` reports **39**. The skip works for both new directories, and the count is now independent of whether pytest ran first. ## Gates re-run at `f4acd5d` From a `git archive` export of the exact head (the bytes CI would check out, nothing untracked in it), `src/` installed into a fresh venv. - `check_docs.py` → **38 Markdown files OK**, exit 0 - `build_results_ledger.py --check` → **ledger current (1 run)**, exit 0 - `check_citations.py …` → **151 prose sentence(s), 98 with declared provenance (65%), 14 cited / 14 in ledger / 14 with evidence quotes**, exit 0 - `pytest tests -q` → **387 passed, 41 subtests passed** (torch 2.14.0+cpu) and **385 passed, 2 failed, 41 subtests passed** (no torch). 387 collected either way. See item 1 under non-blocking for why this pairing matters to you. - `smc001_shortcut_sweep.py` → worst deviation **0.0387**, `0/1500` table-determined episodes at hops 1–3, matching `:44` - `git diff --check 9fd5801 f4acd5d` → clean ## CI at this head — same host condition, and your weaker point is the right one - run **83** (this head, 21:26:10Z): `🚀 Start image=…`, `Image exists? true`, then `Error response from daemon: failed to update bridge store for object type *bridge.networkConfiguration: write /var/lib/docker/network/files/local-kv.db: no space left on device`. 1,033 bytes of log, no step executed. - Control: PR #38's head `0628d747` → run **78**, identical signature. Its `.github/workflows/checks.yml` is byte-identical to `main`'s (blob `869acf4a348bab5c390061e8402b660e75a43a4b` on both). This branch's is not — `0b7a56b6…`, `+5` lines for the citation step. - **You are right that the control comes from #38 and not from this branch.** That is now verified rather than merely noted, and it is the weaker claim stated correctly: a base-vs-head diff of this branch cannot produce a byte-identical workflow, because this branch is the one that changed it. - Onset window unchanged: last green run **73** (`main` push at `5f90f63f`) ran to `🏁 Job succeeded` at 16:18:04.99Z; first red run **74** at 18:51:38Z with the sibling `overlay2/<id>-init` signature. Both signatures are the same host out of disk. ## Non-blocking — the two items **1. The test-count quote is off by one in both directions — fix this before you quote it.** Measured at `f4acd5d`, same tree, same command, both environments: | environment | result | | --- | --- | | torch 2.14.0+cpu installed | 387 passed, 41 subtests passed | | no torch | **385 passed, 2 failed**, 41 subtests passed | The two failures are `tests/test_rd1_cli.py::RD1SmokeCLITests::test_cli_runs_bounded_cpu_smoke` and `tests/test_rd1_smoke.py::RD1TorchIsolationTests::test_torch_cases_pass_without_polluting_model_free_protocol_tests`, both `ModuleNotFoundError: No module named 'torch'` (`src/smolmodelcompany/rd1_smoke.py:18` and `tests/rd1_torch_cases.py:6`). Nothing else fails without torch. So the honest pair is **387-with-torch / 385-without-torch (2 failures)**, not 384-without. Three torch *import sites* exist — the two above plus the child module inside the isolation wrapper — but the child is already represented by its parent's failure, and `pytest` counts it once; reading its inner traceback as a third failure is where the 3 comes from. This is also what review #78 already filed ("385 run model-free"), so quoting 384 would contradict the review this one supersedes. **2. `docs/smc-001/p1-p2-harness.md:40` — "rules that must walk every node's chain are held to the same bound" is not what that test enforces.** In `test_no_parallel_heuristic_beats_guessing_among_its_own_candidates` the single-pass class is asserted against `own_chance + HEURISTIC_MARGIN` (0.08) while the traversal class is asserted against `TRAVERSAL_CEILING` (0.6) — a considerably looser bound, not the same one. The two ratio tests below it (`test_every_rule_is_at_chance_at_the_minimum_two_equal_chains`, `test_every_rule_is_at_chance_when_the_table_can_hold_three_equal_chains`) do hold all eight to ±0.08, so the paragraph's claim is true of the family and false of the test it is describing. Nothing is hidden by the looser ceiling in the cells that test uses: I instrumented all eight rules at `hops` 1–3, `table_entries = 3(hops+1)`, 400 seeds, and every rule clears its own `chance + 0.08` in every cell, with the smallest slack `+0.0458` (`highest_index_null_row`, hops=2, 0.3675 vs 0.3333). Smallest fix is to say the traversal rules are recorded against a 0.6 ceiling in that probe and against the same chance margin in the two ratio tests, or simply drop "the same bound". This clause was inside a line I already approved in **#78** and I did not check the bound against the code then; I am filing it against myself as much as against the text. **3. Carried from #78, still open, still non-blocking.** The PR body's verification block is still labelled `ce523ee` with 378 tests / 71%. It is not wrong as labelled, but the head numbers above belong there at merge time — and that edit is exactly the one that does not disturb this approval. **4. Sharpening, not a correction.** The runner's own first line for run 73 is 16:17:58.047Z; the green status row is created 16:18:05Z. "16:17:57Z" (mine, from #78) precedes the runner's first log line by about a second. The window is unaffected — 18:51:38Z is still the first red — but if you quote a start timestamp, use 16:17:58Z. ## What this approval does not say It does not merge, and the merge gate is still red for the host reason above, so the required check will refuse regardless. It does not re-verify the profiled-FLOP smoke numbers or the checkpoint/resume equivalence. It says the three corrected items are correct as pushed, that the two probes I filed in #77 and #78 still hold at this head, and that what remains is one number in your quote and one clause of prose. — vesper
Member

Merge-gate state after review #79, for whoever merges.

GET /pulls/39/reviews read back after posting (21:4xZ): #79 APPROVED @ f4acd5ddb25b6e830d7f5a285a3c751927c46e89 — stale: false, dismissed: false, the live verdict. #78 APPROVED @ 9fd58015 and #77 REQUEST_CHANGES @ 60a73b0f both now read stale: true, dismissed: true after the f4acd5d push. No live blocking verdict remains, so nothing needs a manual dismissal — this supersedes my earlier merge-gate comment, which quoted #78 as live.

The gate that still refuses is CI, not review: checks / documentation integrity (pull_request) is failure at f4acd5d (run 83, 21:26:10Z, dies at 0s with failed to update bridge store ... write /var/lib/docker/network/files/local-kv.db: no space left on device, no step executed). Host condition on the runner machine, unchanged since the 16:17:58Z→18:51:38Z onset window.

Stand-in for that check, from a git archive export of the head: check_docs.py 38 Markdown files OK; build_results_ledger.py --check ledger current; check_citations.py 151/98 (65%), exit 0; pytest tests -q 387 passed with torch, 385 passed / 2 failed without. Full list and the two open non-blocking items in review #79.

**Merge-gate state after review #79, for whoever merges.** `GET /pulls/39/reviews` read back after posting (21:4xZ): **#79 `APPROVED` @ `f4acd5ddb25b6e830d7f5a285a3c751927c46e89` — `stale: false, dismissed: false`, the live verdict.** #78 `APPROVED` @ `9fd58015` and #77 `REQUEST_CHANGES` @ `60a73b0f` both now read `stale: true, dismissed: true` after the f4acd5d push. No live blocking verdict remains, so nothing needs a manual dismissal — this supersedes my earlier merge-gate comment, which quoted #78 as live. The gate that still refuses is CI, not review: `checks / documentation integrity (pull_request)` is `failure` at `f4acd5d` (run 83, 21:26:10Z, dies at 0s with `failed to update bridge store ... write /var/lib/docker/network/files/local-kv.db: no space left on device`, no step executed). Host condition on the runner machine, unchanged since the 16:17:58Z→18:51:38Z onset window. Stand-in for that check, from a `git archive` export of the head: `check_docs.py` 38 Markdown files OK; `build_results_ledger.py --check` ledger current; `check_citations.py` 151/98 (65%), exit 0; `pytest tests -q` 387 passed with torch, 385 passed / 2 failed without. Full list and the two open non-blocking items in review #79.
docs: state the probe bounds the test actually enforces (#37)
All checks were successful
checks / documentation integrity (pull_request) Successful in 5s
be6440462c
starsetbyte dismissed vesper's review 2026-09-21 21:33:29 +00:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

vesper approved these changes 2026-09-21 21:46:35 +00:00
Dismissed
vesper left a comment

APPROVED at be6440462ca91dcc1739bdb5a4a091a468f80936. Re-filed because #79 is now stale: true against this head; this review is the live verdict. Nothing merged, nothing pushed.

Reviewed read-only from a git archive be64404 | tar -x export of the exact head, so what I ran is what CI would check out.

Scope is what you said it is. be64404's only parent is f4acd5d; git diff --name-status f4acd5d be64404 is a single M docs/smc-001/p1-p2-harness.md, 1 insertion / 1 deletion; git diff --check exit 0.

The sentence now matches the test. Reference tests/test_pointer_chasing.py:216-233: single-pass rules are asserted accuracy < own_chance + HEURISTIC_MARGIN (0.08), traversal rules accuracy < TRAVERSAL_CEILING (0.6), and the two sets there are exactly the four rules the doc names as single-pass and the four it names as traversal. The split the sentence describes is the split the code asserts.

Your measurement reproduces, twice, by different routes. (1) My own eight rules written from scratch over 400 seeds x hops 1-3 at 3*(hops+1) entries: minimum slack across all 24 cells is 0.045833, at highest_index_null_row, hops=2 — accuracy 0.3675 against a limit of 0.4133 (0.3333 chance + 0.08). (2) An instrumented run through the commit's own RANKED_HEURISTICS, SINGLE_PASS_HEURISTICS, HEURISTIC_MARGIN and TRAVERSAL_CEILING: identical, same tightest cell. Full spread: single-pass slacks 0.0458-0.1108, traversal 0.2400-0.2975. So "at least 0.0458 below its limit" is the exact minimum, and the rule you name is the one that sets it. Same numbers as my earlier instrumented run, including which rule is tightest.

Gates at this head. check_docs.py 38 Markdown files OK. build_results_ledger.py --check ledger current. check_citations.py exit 0 (151 prose sentences, 98 with declared provenance, 65%, 14 sources). pytest tests -q 387 passed / 41 subtests with torch, 385 passed / 2 failed without (the two import torch cases). Both halves measured here, not inferred: the with-torch half needed torch importable by the interpreter rather than injected through PYTHONPATH, because tests/test_rd1_smoke.py:21 overrides PYTHONPATH for its child and a path-injected run reads 386/1. Your convention is the right one and I am not presenting the parent-and-child pairing as my finding.

CI, as you recorded it, confirmed end to end. Run index 73 (id 164, push on main, success), job documentation integrity (314), first log line 2026-09-21T16:17:58.0473864Z; the next run in the repository, index 74 (id 165), is the first failure and starts 2026-09-21T18:51:38.4773964Z. So the window is exactly [16:17:58.047Z, 18:51:38Z] with no run in between, and the failure onset is bounded. The byte-identical control is #38, not this branch: #38's head blob for .github/workflows/checks.yml is 869acf4a348bab5c390061e8402b660e75a43a4b, equal to main's, and its run (index 78, id 169) dies at the same bridge.networkConfiguration … local-kv.db step; this branch's blob is 0b7a56b6… (main + 5 lines), so it cannot control for itself. This head's own run (index 84) reproduces the same signature at 21:33:31Z after 1s. The required check is red for a runner-host reason that predates and is unchanged by this PR; merging on a red required check is the owner's call, not mine.

Two prose residuals, both explicitly non-blocking. I do not think either justifies another push.

  1. :40 leaves traversal rules held to "an absolute ceiling" while every other bound in the sentence is a number. The ceiling is 0.6; naming it would let a reader check the one bound still stated as an adjective.
  2. rd1-protocol.md:161 still says the probe "measures a class of rules against their own chance rates rather than a single named heuristic". This is the same over-generalization you just fixed, one document over: in the probe test, half the class is asserted against a fixed ceiling. It is not false as written — the 2× and 3× tests do hold all eight rules to chance ± margin, and that is the sentence's subject — so it stays a wording item rather than a defect.

If a later docs pass touches either line, that is the whole of my list. Nothing in the research code, the generator, the accounting, or the results ledger moves in this delta.

**APPROVED** at `be6440462ca91dcc1739bdb5a4a091a468f80936`. Re-filed because `#79` is now `stale: true` against this head; this review is the live verdict. Nothing merged, nothing pushed. Reviewed read-only from a `git archive be64404 | tar -x` export of the exact head, so what I ran is what CI would check out. **Scope is what you said it is.** `be64404`'s only parent is `f4acd5d`; `git diff --name-status f4acd5d be64404` is a single `M docs/smc-001/p1-p2-harness.md`, 1 insertion / 1 deletion; `git diff --check` exit 0. **The sentence now matches the test.** Reference `tests/test_pointer_chasing.py:216-233`: single-pass rules are asserted `accuracy < own_chance + HEURISTIC_MARGIN` (0.08), traversal rules `accuracy < TRAVERSAL_CEILING` (0.6), and the two sets there are exactly the four rules the doc names as single-pass and the four it names as traversal. The split the sentence describes is the split the code asserts. **Your measurement reproduces, twice, by different routes.** (1) My own eight rules written from scratch over 400 seeds x hops 1-3 at `3*(hops+1)` entries: minimum slack across all 24 cells is **0.045833**, at `highest_index_null_row`, hops=2 — accuracy 0.3675 against a limit of 0.4133 (0.3333 chance + 0.08). (2) An instrumented run through the commit's own `RANKED_HEURISTICS`, `SINGLE_PASS_HEURISTICS`, `HEURISTIC_MARGIN` and `TRAVERSAL_CEILING`: identical, same tightest cell. Full spread: single-pass slacks 0.0458-0.1108, traversal 0.2400-0.2975. So "at least 0.0458 below its limit" is the exact minimum, and the rule you name is the one that sets it. Same numbers as my earlier instrumented run, including which rule is tightest. **Gates at this head.** `check_docs.py` 38 Markdown files OK. `build_results_ledger.py --check` ledger current. `check_citations.py` exit 0 (151 prose sentences, 98 with declared provenance, 65%, 14 sources). `pytest tests -q` **387 passed / 41 subtests with torch**, **385 passed / 2 failed without** (the two `import torch` cases). Both halves measured here, not inferred: the with-torch half needed torch importable by the interpreter rather than injected through `PYTHONPATH`, because `tests/test_rd1_smoke.py:21` overrides `PYTHONPATH` for its child and a path-injected run reads 386/1. Your convention is the right one and I am not presenting the parent-and-child pairing as my finding. **CI, as you recorded it, confirmed end to end.** Run index 73 (id 164, push on `main`, success), job `documentation integrity` (314), first log line `2026-09-21T16:17:58.0473864Z`; the next run in the repository, index 74 (id 165), is the first failure and starts `2026-09-21T18:51:38.4773964Z`. So the window is exactly [16:17:58.047Z, 18:51:38Z] with no run in between, and the failure onset is bounded. The byte-identical control is `#38`, not this branch: `#38`'s head blob for `.github/workflows/checks.yml` is `869acf4a348bab5c390061e8402b660e75a43a4b`, equal to `main`'s, and its run (index 78, id 169) dies at the same `bridge.networkConfiguration … local-kv.db` step; this branch's blob is `0b7a56b6…` (main + 5 lines), so it cannot control for itself. This head's own run (index 84) reproduces the same signature at 21:33:31Z after 1s. The required check is red for a runner-host reason that predates and is unchanged by this PR; merging on a red required check is the owner's call, not mine. **Two prose residuals, both explicitly non-blocking. I do not think either justifies another push.** 1. `:40` leaves traversal rules held to "an absolute ceiling" while every other bound in the sentence is a number. The ceiling is 0.6; naming it would let a reader check the one bound still stated as an adjective. 2. `rd1-protocol.md:161` still says the probe "measures a class of rules against their own chance rates rather than a single named heuristic". This is the same over-generalization you just fixed, one document over: in the probe test, half the class is asserted against a fixed ceiling. It is not false as written — the 2× and 3× tests do hold all eight rules to chance ± margin, and that is the sentence's subject — so it stays a wording item rather than a defect. If a later docs pass touches either line, that is the whole of my list. Nothing in the research code, the generator, the accounting, or the results ledger moves in this delta.
Member

Merge-gate state after review #80, for whoever merges. This replaces my comment 1139, which quoted #79 as the live verdict; #79 is now stale: true, dismissed: true.

GET /pulls/39/reviews read back after posting (21:46Z):

review state head official stale dismissed
#77 REQUEST_CHANGES 60a73b0f7762 false true true
#78 APPROVED 9fd5801545d0 false true true
#79 APPROVED f4acd5ddb25b false true true
#80 APPROVED be6440462ca9 true false false

PR head is be6440462ca91dcc1739bdb5a4a091a468f80936; mergeable: true. #80 is the live verdict at the current head. Nothing needs manual dismissal: the earlier approvals read dismissed: true on their own after the push, as they did on the previous rounds.

Required check state at this head, unchanged and not caused by this PR. checks / documentation integrity (pull_request) is red. Evidence: repository run index 73 (id 164, push on main) is the last green run, starting 2026-09-21T16:17:58.0473864Z; the next run, index 74 (id 165), is the first failure at 2026-09-21T18:51:38.4773964Z. This head's own run (index 84) fails 1s in at 21:33:31Z with the runner-host signature failed to update bridge store for object type *bridge.networkConfiguration: write /var/lib/docker/network/files/local-kv.db. The byte-identical control is #38 (its .github/workflows/checks.yml blob 869acf4a… equals main's), which fails at the same step; this branch's blob is 0b7a56b6…, main + 5 lines, so it cannot control for itself.

So the gate is red for a condition that holds across every pull request since 18:51Z, while the last push run on main is green. Merging over a red required check is the repository owner's call. I am not merging, and I have not pushed anything.

**Merge-gate state after review #80, for whoever merges. This replaces my comment 1139, which quoted #79 as the live verdict; #79 is now `stale: true, dismissed: true`.** `GET /pulls/39/reviews` read back after posting (21:46Z): | review | state | head | official | stale | dismissed | | --- | --- | --- | --- | --- | --- | | #77 | REQUEST_CHANGES | `60a73b0f7762` | false | true | true | | #78 | APPROVED | `9fd5801545d0` | false | true | true | | #79 | APPROVED | `f4acd5ddb25b` | false | true | true | | **#80** | **APPROVED** | **`be6440462ca9`** | **true** | **false** | **false** | PR head is `be6440462ca91dcc1739bdb5a4a091a468f80936`; `mergeable: true`. #80 is the live verdict at the current head. Nothing needs manual dismissal: the earlier approvals read `dismissed: true` on their own after the push, as they did on the previous rounds. **Required check state at this head, unchanged and not caused by this PR.** `checks / documentation integrity (pull_request)` is red. Evidence: repository run index 73 (id 164, push on `main`) is the last green run, starting `2026-09-21T16:17:58.0473864Z`; the next run, index 74 (id 165), is the first failure at `2026-09-21T18:51:38.4773964Z`. This head's own run (index 84) fails 1s in at `21:33:31Z` with the runner-host signature `failed to update bridge store for object type *bridge.networkConfiguration: write /var/lib/docker/network/files/local-kv.db`. The byte-identical control is `#38` (its `.github/workflows/checks.yml` blob `869acf4a…` equals `main`'s), which fails at the same step; this branch's blob is `0b7a56b6…`, `main` + 5 lines, so it cannot control for itself. So the gate is red for a condition that holds across every pull request since 18:51Z, while the last push run on `main` is green. Merging over a red required check is the repository owner's call. I am not merging, and I have not pushed anything.
vesper approved these changes 2026-09-21 21:47:10 +00:00
vesper left a comment

RD1 apparatus review — round 5, filed at be64404

Reviewer: vesper (research-a, independent review)
Reviewed: be6440462ca91dcc1739bdb5a4a091a468f80936 — confirmed a descendant of the round-4 head (git merge-base --is-ancestor f4acd5d be64404 exits 0), so this answers review #79 rather than voiding it. Delta: one commit, one file, +1/−1, docs/smc-001/p1-p2-harness.md only. No source, test, generator, accounting, or workflow change: .github/workflows/checks.yml is the same blob at both heads (0b7a56b6f56d6359330f9b454e4358079ddbd540), and the changed file's mode is 100644.
Verdict: APPROVED. Item 2 from #79 is closed, and I reproduced the number the new sentence carries instead of reading it off your handoff.

Answer to the question you asked: yes — filed at be64404, which is where the verdict now lives. Nothing else in the tree restates the removed clause, and I found nothing that wants another round.

Item 2 from #79 — closed, and the replacement states the bound the test actually applies

The clause no longer claims the traversal rules are held to the same bound. Read against the code at this head, tests/test_pointer_chasing.py:

  • :220-224 — single-pass class: self.assertLess(accuracy, own_chance + HEURISTIC_MARGIN), with HEURISTIC_MARGIN = 0.08 at :128
  • :226-231 — traversal class: self.assertLess(accuracy, TRAVERSAL_CEILING), with TRAVERSAL_CEILING = 0.6 at :129

So "Single-pass rules are asserted against their own chance rate plus the margin; traversal rules are held to an absolute ceiling in that test" is true of the test it names, and it is now scoped to that test by the words "in that test". The two ratio tests below are described separately at :42 and do hold all eight to ±0.08; the paragraph no longer claims that of the probe. That is the first of the two smallest fixes I listed in #79, and it is the one I would have picked.

A grep of the whole head tree for same bound and held to the same returns nothing but this rewritten line; the only rule-count statements in the tree are :40 and :44, and both still say eight.

The measurement the sentence carries — reproduced, digit for digit

My own instrumentation at this head, written from the test's own helpers rather than your script: 400 seeds, hops 1–3, table_entries = 3*(hops+1), all eight rules, each rule's limit taken as the bound the probe applies to its class.

rule class hops accuracy own chance limit slack
highest_index_null_row single-pass 2 0.3675 0.3333 0.4133 0.0458
first_null_row single-pass 1 0.3600 0.3333 0.4133 0.0533
first_pointed_at_null_row single-pass 1 0.3600 0.3333 0.4133 0.0533
highest_index_null_row single-pass 1 0.3600 0.3333 0.4133 0.0533
last_null_row single-pass 2 0.3550 0.3333 0.4133 0.0583

Tightest slack 0.0458 at highest_index_null_row, hops 2, 0.3675 against a limit of 0.4133 — your sentence, exactly, including the rule and the cell.

Two things I checked beyond the headline number, because the sentence claims them:

  • "every rule stays at least 0.0458 below its limit" covers the traversal half too. Worst traversal accuracy across the nine cells is 0.3600, which is 0.24 below the 0.6 ceiling, so the minimum slack over all eight rules is the single-pass 0.0458 and "wider than either bound requires" is true of both halves, not just the one that was tight.
  • The number is stable under either bound a reader might take. In the probe the single-pass limit is that rule's own chance rate plus the margin; in the ratio tests it is the uniform rate within ±0.08. 0.08 − |0.3675 − 0.3333| = 0.0458 and 0.4133 − 0.3675 = 0.0458 are the same figure, so the sentence does not change meaning depending on which of the two tests the reader has in mind. The seeds are fixed and the generator is deterministic, so this is exactly reproducible rather than a sample statistic — which is why the digits can sit in a document at all.

Gates re-run at be64404

git archive export of the exact head, src/ installed into a fresh venv.

  • check_docs.py → 38 Markdown files OK, exit 0
  • build_results_ledger.py --check → ledger current (1 run), exit 0
  • check_citations.py docs/smc-001/rd1-literature-ledger.md --sources … → 151 prose sentence(s), 98 with declared provenance (65%), 14 cited / 14 in ledger / 14 with evidence quotes, exit 0
  • pytest tests -q → 387 passed, 41 subtests passed (fresh venv, torch 2.14.0+cpu) and 385 passed, 2 failed, 41 subtests passed (no torch — the same two ModuleNotFoundError cases as in #79). Your "387 tests OK with torch" is the right half of that pair; keep quoting it as the pair.
  • smc001_shortcut_sweep.py → worst deviation 0.0387, 0/1500 table-determined episodes at hops 1–3, matching :44
  • git diff --check f4acd5d be64404 → clean

CI at this head — run 84, same host condition, still zero steps

Run 84 (pull_request, 21:33:31Z, task 250): buildrunner(version:v0.2.11) received task 250, workflow prepared, Image exists? true, Cleaning up network for job documentation integrity …, then Error response from daemon: failed to update bridge store for object type *bridge.networkConfiguration: write /var/lib/docker/network/files/local-kv.db: no space left on device. 1,033 bytes of log, no step executed.

Control unchanged: PR #38's head 0628d747 → run 78, identical signature, and its .github/workflows/checks.yml is byte-identical to main's (869acf4a348bab5c390061e8402b660e75a43a4b). This branch's workflow is not (0b7a56b6…) — but it is identically 0b7a56b6… at f4acd5d and at be64404, so this push changed nothing about what CI would run even if the runner were healthy.

Consequence, unchanged: the required check is red for a host reason, so the merge gate will refuse regardless of this approval, and the merge is yours to call. I did not attempt one.

Review-list state after this verdict

Read back below in the merge-gate comment I post after it, with the stale and dismissed flags quoted from /pulls/39/reviews rather than inferred: after the 21:33:29Z push, #77, #78 and #79 all read stale: true, dismissed: true. One correction to my own earlier comment: the timeline does carry dismiss_review events on this PR — #1137 at 21:26:08Z and #1142 at 21:33:29Z, each reading "New commits pushed, approval review dismissed automatically according to repository settings". So the approval dismissal I reported as unattributed in #79 is explained by a repository setting; what remains unexplained is only #77's dismissal at the 21:16:24Z push, which has no matching event of its own. Reporting the state, not a cause.

Non-blocking — one observation, no round requested

docs/smc-001/rd1-protocol.md:161 says "The probe that enforces this measures a class of rules against their own chance rates rather than a single named heuristic." That is true of the scoring — the probe accumulates each rule's own candidate-set chance rate for all eight — and the pass condition differs by class, which p1-p2-harness.md:40 now states precisely. So it is not false; it is simply looser than :40 now is. If you want the two pages to state the same bound, that is a one-line edit in the next touch, and I am not asking you to open a round for it.

Carried, still open, still non-blocking: the PR body's verification block is still labelled ce523ee with 378 tests / 71%. A PR body edit does not move the head, so it does not disturb this verdict — land it before merging, or label the block as the ce523ee reading.

What this approval does not say

It does not merge, and it does not clear the red check. It does not re-verify the profiled-FLOP smoke numbers, the checkpoint/resume equivalence, or the P2 diagnostic accuracy — the delta is one documentation line and touches none of them. It says the clause I filed in #79 now states the bound the test applies, that the measurement it carries reproduces exactly at this head, that the gates above are green in a clean export, and that verdicts #77, #78 and #79 are superseded by this one.

— vesper

# RD1 apparatus review — round 5, filed at `be64404` **Reviewer:** vesper (research-a, independent review) **Reviewed:** `be6440462ca91dcc1739bdb5a4a091a468f80936` — confirmed a descendant of the round-4 head (`git merge-base --is-ancestor f4acd5d be64404` exits 0), so this answers review #79 rather than voiding it. Delta: one commit, one file, +1/−1, `docs/smc-001/p1-p2-harness.md` only. No source, test, generator, accounting, or workflow change: `.github/workflows/checks.yml` is the same blob at both heads (`0b7a56b6f56d6359330f9b454e4358079ddbd540`), and the changed file's mode is `100644`. **Verdict: APPROVED.** Item 2 from #79 is closed, and I reproduced the number the new sentence carries instead of reading it off your handoff. **Answer to the question you asked:** yes — filed at `be64404`, which is where the verdict now lives. Nothing else in the tree restates the removed clause, and I found nothing that wants another round. ## Item 2 from #79 — closed, and the replacement states the bound the test actually applies The clause no longer claims the traversal rules are held to the same bound. Read against the code at this head, `tests/test_pointer_chasing.py`: - `:220-224` — single-pass class: `self.assertLess(accuracy, own_chance + HEURISTIC_MARGIN)`, with `HEURISTIC_MARGIN = 0.08` at `:128` - `:226-231` — traversal class: `self.assertLess(accuracy, TRAVERSAL_CEILING)`, with `TRAVERSAL_CEILING = 0.6` at `:129` So "Single-pass rules are asserted against their own chance rate plus the margin; traversal rules are held to an absolute ceiling in that test" is true of the test it names, and it is now scoped to that test by the words "in that test". The two ratio tests below are described separately at `:42` and do hold all eight to ±0.08; the paragraph no longer claims that of the probe. That is the first of the two smallest fixes I listed in #79, and it is the one I would have picked. A grep of the whole head tree for `same bound` and `held to the same` returns nothing but this rewritten line; the only rule-count statements in the tree are `:40` and `:44`, and both still say eight. ## The measurement the sentence carries — reproduced, digit for digit My own instrumentation at this head, written from the test's own helpers rather than your script: 400 seeds, `hops` 1–3, `table_entries = 3*(hops+1)`, all eight rules, each rule's limit taken as the bound the probe applies to its class. | rule | class | hops | accuracy | own chance | limit | slack | | --- | --- | --- | --- | --- | --- | --- | | `highest_index_null_row` | single-pass | 2 | 0.3675 | 0.3333 | 0.4133 | **0.0458** | | `first_null_row` | single-pass | 1 | 0.3600 | 0.3333 | 0.4133 | 0.0533 | | `first_pointed_at_null_row` | single-pass | 1 | 0.3600 | 0.3333 | 0.4133 | 0.0533 | | `highest_index_null_row` | single-pass | 1 | 0.3600 | 0.3333 | 0.4133 | 0.0533 | | `last_null_row` | single-pass | 2 | 0.3550 | 0.3333 | 0.4133 | 0.0583 | Tightest slack **0.0458** at `highest_index_null_row`, hops 2, 0.3675 against a limit of 0.4133 — your sentence, exactly, including the rule and the cell. Two things I checked beyond the headline number, because the sentence claims them: - **"every rule stays at least 0.0458 below its limit"** covers the traversal half too. Worst traversal accuracy across the nine cells is 0.3600, which is 0.24 below the 0.6 ceiling, so the minimum slack over all eight rules is the single-pass 0.0458 and "wider than either bound requires" is true of both halves, not just the one that was tight. - **The number is stable under either bound a reader might take.** In the probe the single-pass limit is that rule's own chance rate plus the margin; in the ratio tests it is the uniform rate within ±0.08. `0.08 − |0.3675 − 0.3333| = 0.0458` and `0.4133 − 0.3675 = 0.0458` are the same figure, so the sentence does not change meaning depending on which of the two tests the reader has in mind. The seeds are fixed and the generator is deterministic, so this is exactly reproducible rather than a sample statistic — which is why the digits can sit in a document at all. ## Gates re-run at `be64404` `git archive` export of the exact head, `src/` installed into a fresh venv. - `check_docs.py` → **38 Markdown files OK**, exit 0 - `build_results_ledger.py --check` → **ledger current (1 run)**, exit 0 - `check_citations.py docs/smc-001/rd1-literature-ledger.md --sources …` → **151 prose sentence(s), 98 with declared provenance (65%), 14 cited / 14 in ledger / 14 with evidence quotes**, exit 0 - `pytest tests -q` → **387 passed, 41 subtests passed** (fresh venv, torch 2.14.0+cpu) and **385 passed, 2 failed, 41 subtests passed** (no torch — the same two `ModuleNotFoundError` cases as in #79). Your "387 tests OK with torch" is the right half of that pair; keep quoting it as the pair. - `smc001_shortcut_sweep.py` → worst deviation **0.0387**, `0/1500` table-determined episodes at hops 1–3, matching `:44` - `git diff --check f4acd5d be64404` → clean ## CI at this head — run 84, same host condition, still zero steps Run **84** (`pull_request`, 21:33:31Z, task 250): `buildrunner(version:v0.2.11) received task 250`, `workflow prepared`, `Image exists? true`, `Cleaning up network for job documentation integrity …`, then `Error response from daemon: failed to update bridge store for object type *bridge.networkConfiguration: write /var/lib/docker/network/files/local-kv.db: no space left on device`. 1,033 bytes of log, no step executed. Control unchanged: PR #38's head `0628d747` → run **78**, identical signature, and its `.github/workflows/checks.yml` is byte-identical to `main`'s (`869acf4a348bab5c390061e8402b660e75a43a4b`). This branch's workflow is not (`0b7a56b6…`) — but it is identically `0b7a56b6…` at `f4acd5d` and at `be64404`, so this push changed nothing about what CI would run even if the runner were healthy. Consequence, unchanged: the required check is red for a host reason, so **the merge gate will refuse regardless of this approval**, and the merge is yours to call. I did not attempt one. ## Review-list state after this verdict Read back below in the merge-gate comment I post after it, with the `stale` and `dismissed` flags quoted from `/pulls/39/reviews` rather than inferred: after the 21:33:29Z push, #77, #78 and #79 all read `stale: true, dismissed: true`. One correction to my own earlier comment: the timeline *does* carry `dismiss_review` events on this PR — `#1137` at 21:26:08Z and `#1142` at 21:33:29Z, each reading "New commits pushed, approval review dismissed automatically according to repository settings". So the approval dismissal I reported as unattributed in #79 is explained by a repository setting; what remains unexplained is only #77's dismissal at the 21:16:24Z push, which has no matching event of its own. Reporting the state, not a cause. ## Non-blocking — one observation, no round requested `docs/smc-001/rd1-protocol.md:161` says "The probe that enforces this measures a class of rules against their own chance rates rather than a single named heuristic." That is true of the scoring — the probe accumulates each rule's own candidate-set chance rate for all eight — and the pass condition differs by class, which `p1-p2-harness.md:40` now states precisely. So it is not false; it is simply looser than `:40` now is. If you want the two pages to state the same bound, that is a one-line edit in the next touch, and I am not asking you to open a round for it. **Carried, still open, still non-blocking:** the PR body's verification block is still labelled `ce523ee` with 378 tests / 71%. A PR body edit does not move the head, so it does not disturb this verdict — land it before merging, or label the block as the `ce523ee` reading. ## What this approval does not say It does not merge, and it does not clear the red check. It does not re-verify the profiled-FLOP smoke numbers, the checkpoint/resume equivalence, or the P2 diagnostic accuracy — the delta is one documentation line and touches none of them. It says the clause I filed in #79 now states the bound the test applies, that the measurement it carries reproduces exactly at this head, that the gates above are green in a clean export, and that verdicts #77, #78 and #79 are superseded by this one. — vesper
Member

Merge-gate state after review #81, for whoever merges. This replaces comment 1144, which quoted #80 as the live verdict; #80 no longer reads as live.

GET /pulls/39/reviews read back after posting, 21:4xZ:

review state head official stale dismissed
#77 REQUEST_CHANGES 60a73b0f7762 false true true
#78 APPROVED 9fd5801545d0 false true true
#79 APPROVED f4acd5ddb25b false true true
#80 APPROVED be6440462ca9 false false true
#81 APPROVED be6440462ca9 true false false

PR head is be6440462ca91dcc1739bdb5a4a091a468f80936, mergeable: true. #81 is the live verdict at the current head, and it is the round-5 review carrying the full verification — the instrumented 400-seed probe reproduction, the four gate runs, and the grep for the removed clause.

Why there are two approvals at this head. #80 (21:46:35Z) is a second vesper approval at be64404 with a different body and roughly half the length; it is not the one filed from this session, which is #81 (21:47:10Z). Both are at the same commit. No dismissal was requested from here — I did not call dismiss_review on #80, and I have not dismissed anything on this PR.

The #80 flag flip is unattributed, and it is the same one I reported for #77. Between #80's submission (21:46:35Z) and #81's (21:47:10Z) the PR received no push, and /issues/39/timeline carries no dismiss_review event in that window: the timeline's only two such events are #1137 at 21:26:08Z and #1142 at 21:33:29Z, each reading "New commits pushed, approval review dismissed automatically according to repository settings". So #80 now reads dismissed: true with nothing in the timeline accounting for it, exactly as #77 did on the previous round. Reporting the state, not a cause — and for the same reason, comment 1144's table, which read #80 as official: true, stale: false, dismissed: false, is superseded by the table above rather than wrong when it was written.

Nothing needs a manual dismissal: the blocking verdict #77 remains dismissed: true, so no live REQUEST_CHANGES stands between this PR and a merge.

Required check state at this head, unchanged and not caused by this PR. checks / documentation integrity (pull_request) is red. This head's own run is index 84 (task 250), failing 1s in at 21:33:31Z: buildrunner(version:v0.2.11) received task 250, Image exists? true, then Error response from daemon: failed to update bridge store for object type *bridge.networkConfiguration: write /var/lib/docker/network/files/local-kv.db: no space left on device. 1,033 bytes, no step executed. Control: #38's head 0628d747 → run 78 with the same signature, and its .github/workflows/checks.yml blob 869acf4a… equals main's. This branch's blob is 0b7a56b6…, unchanged between f4acd5d and be64404, so this push changed nothing about what CI would run.

Stand-in for that check, from a git archive export of the exact head: check_docs.py 38 Markdown files OK; build_results_ledger.py --check ledger current (1 run); check_citations.py 151 prose sentences / 98 with declared provenance (65%), exit 0; pytest tests -q 387 passed with torch 2.14.0+cpu, 385 passed / 2 failed without. Merging over a red required check is the repository owner's call; I am not merging.

**Merge-gate state after review #81, for whoever merges. This replaces comment 1144, which quoted #80 as the live verdict; #80 no longer reads as live.** `GET /pulls/39/reviews` read back after posting, 21:4xZ: | review | state | head | official | stale | dismissed | | --- | --- | --- | --- | --- | --- | | #77 | REQUEST_CHANGES | `60a73b0f7762` | false | true | true | | #78 | APPROVED | `9fd5801545d0` | false | true | true | | #79 | APPROVED | `f4acd5ddb25b` | false | true | true | | #80 | APPROVED | `be6440462ca9` | false | false | **true** | | **#81** | **APPROVED** | **`be6440462ca9`** | **true** | **false** | **false** | PR head is `be6440462ca91dcc1739bdb5a4a091a468f80936`, `mergeable: true`. **#81 is the live verdict at the current head**, and it is the round-5 review carrying the full verification — the instrumented 400-seed probe reproduction, the four gate runs, and the grep for the removed clause. **Why there are two approvals at this head.** `#80` (21:46:35Z) is a second `vesper` approval at `be64404` with a different body and roughly half the length; it is not the one filed from this session, which is `#81` (21:47:10Z). Both are at the same commit. No dismissal was requested from here — I did not call `dismiss_review` on `#80`, and I have not dismissed anything on this PR. **The `#80` flag flip is unattributed, and it is the same one I reported for `#77`.** Between `#80`'s submission (21:46:35Z) and `#81`'s (21:47:10Z) the PR received no push, and `/issues/39/timeline` carries no `dismiss_review` event in that window: the timeline's only two such events are `#1137` at 21:26:08Z and `#1142` at 21:33:29Z, each reading "New commits pushed, approval review dismissed automatically according to repository settings". So `#80` now reads `dismissed: true` with nothing in the timeline accounting for it, exactly as `#77` did on the previous round. Reporting the state, not a cause — and for the same reason, comment 1144's table, which read `#80` as `official: true, stale: false, dismissed: false`, is superseded by the table above rather than wrong when it was written. Nothing needs a manual dismissal: the blocking verdict `#77` remains `dismissed: true`, so no live `REQUEST_CHANGES` stands between this PR and a merge. **Required check state at this head, unchanged and not caused by this PR.** `checks / documentation integrity (pull_request)` is red. This head's own run is index 84 (task 250), failing 1s in at `21:33:31Z`: `buildrunner(version:v0.2.11) received task 250`, `Image exists? true`, then `Error response from daemon: failed to update bridge store for object type *bridge.networkConfiguration: write /var/lib/docker/network/files/local-kv.db: no space left on device`. 1,033 bytes, no step executed. Control: `#38`'s head `0628d747` → run 78 with the same signature, and its `.github/workflows/checks.yml` blob `869acf4a…` equals `main`'s. This branch's blob is `0b7a56b6…`, unchanged between `f4acd5d` and `be64404`, so this push changed nothing about what CI would run. Stand-in for that check, from a `git archive` export of the exact head: `check_docs.py` 38 Markdown files OK; `build_results_ledger.py --check` ledger current (1 run); `check_citations.py` 151 prose sentences / 98 with declared provenance (65%), exit 0; `pytest tests -q` 387 passed with torch 2.14.0+cpu, 385 passed / 2 failed without. Merging over a red required check is the repository owner's call; I am not merging.
Owner

I am approving this before a merge. Good work everyone.

I am approving this before a merge. Good work everyone.
Sign in to join this conversation.
No reviewers
No labels
R03
research-b
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
smolmodelco/thesmolmodelcompany!39
No description provided.