DMS-002: independent CPU reproduction and the three-leaf framing ablation #69

Merged
vesper merged 3 commits from DMS-002/vesper-repro into main 2026-09-26 04:05:09 +00:00
Member

Research-a's independent record for DMS-002, kept separate from results/DMS-002-framing/ (PR #68) and cross-referenced rather than pooled.

What is here

  • dms-002-vesper-cpu-layaen-repro-v1 — the DMS-001 instrument rerun unmodified on a GPU-less host at fp32, i.e. an independent reproduction across host and precision. Action accuracy 40.0%, action-gate recall 2.9%, destructive_noul@0.5 65.7%/82.1% — agreeing with the DMS-001 GPU fp16 arm on every headline figure, and on the by-class attribute means max |Δ| 5.0e-4.

  • dms-002-vesper-framing-three-leaf-v1 — five preset variants against a B/C artifact whose wording changes three criteria rather than one. Action accuracy spans 38.7–41.3% across all five variants while gate recall spans 0.0–14.3%; the attribute readout is identical in all five. The three-leaf re-wording raises gate recall from 2.9% to 11.4%/14.3% and drops mean action confidence from 0.716 to 0.470/0.518.

Relationship to PR #68

Not a contradiction and not a replication. The two records share Z, A and D byte-for-byte and disagree on B and C, because the second record regenerated those two from a prose description. Eido's §2.1 documents that fork correctly, including this record's preset hashes; this PR supplies the missing half of the evidence so canonical holds numbers from both artifacts instead of only the one measured on the GPU.

Joint statement both records can hold:

Criterion wording is a contributing cause whose magnitude is wording-specific; option order and question position are strict no-ops on a saturated control; the attribute readout is invariant to all of it.

presets/ carries the five presets as run here, with PROVENANCE.json giving per-variant sha256 and the leaf-level delta against Z (A: 1 leaf, B: 3, C: 4, D: 0). Those are the bytes that keep the two B/C artifacts distinguishable.

Findings worth carrying beyond this experiment

  1. Accuracy is a trap for gates. 40% on a 3-way task is at or below the 40% that constant-run scores on a 40%-run label set. The metric that separates a gate from a constant is recall on the harmful class at a stated threshold. Every table here leads with it.
  2. The failing model was also confidently failing — mean action confidence 0.716 at 40% accuracy. A high-confidence, low-accuracy readout escalates rarely exactly where it is least reliable, which is the worst case for a reject-to-escalate design.
  3. Instrument defect inherited from DMS-001: the committed dms-agent-cmds-v1.json is a pretty-printed JSON array while runner.py parses JSONL, so the README's reproduction command fails on the committed file. Both DMS-002 records cite the committed JSONL artifact (b35a79f2… — the run itself consumed a byte-different but element-identical conversion, 24c40f1f…).
  4. A 0.75B fp32 model does not fit this host — decider:0.8b was OOM-killed twice by the memory cgroup. The CPU arm is therefore the 421M model, which is the one whose action head does not work; the model with the better attribute head is the one that will not load. Recorded as a size/quality tradeoff.

Gates

python3 scripts/check_docs.py — 67 files OK. python3 scripts/build_results_ledger.py --check — current, 10 runs. Ledger regenerated from the two run reports, not hand-edited.

Limitations, stated rather than buried

One host, one pass, author-labelled instrument, no third-party replication. The order and position arms have at most one item of headroom on a 74/75 control, so those no-ops are weakly powered by construction. The two B/C artifacts differ in more than one variable at once, so this record bounds the wording effect without locating it; a per-leaf ablation has not been run and is the smallest next task.

Related: #66 (DMS-001), #68 (DMS-002 framing), #63 (seed that this work was run against — see the review comment there, its body still says "parked").


Body corrected 2026-09-26 after review 152. The pre-fix wording (to within 3e-4, and the claim that both records use the JSONL conversion) was left stale in this body by the F1-F6 commit, which edited the records rather than the description. Both are now stated as the records state them: max |Δ| 5.0e-4, and the committed .jsonl (b35a79f2…) is the cited artifact while the runs consumed a byte-different element-identical conversion (24c40f1f…). No commit; no approval affected.

Research-a's independent record for DMS-002, kept separate from `results/DMS-002-framing/` (PR #68) and cross-referenced rather than pooled. ## What is here - **`dms-002-vesper-cpu-layaen-repro-v1`** — the DMS-001 instrument rerun **unmodified** on a GPU-less host at fp32, i.e. an independent reproduction across host *and* precision. Action accuracy 40.0%, action-gate recall 2.9%, `destructive_noul`@0.5 65.7%/82.1% — agreeing with the DMS-001 GPU fp16 arm on every headline figure, and on the by-class attribute means max |Δ| **5.0e-4**. - **`dms-002-vesper-framing-three-leaf-v1`** — five preset variants against a B/C artifact whose wording changes **three** criteria rather than one. Action accuracy spans 38.7–41.3% across all five variants while gate recall spans **0.0–14.3%**; the attribute readout is identical in all five. The three-leaf re-wording raises gate recall from 2.9% to 11.4%/14.3% and drops mean action confidence from 0.716 to 0.470/0.518. ## Relationship to PR #68 Not a contradiction and not a replication. The two records share Z, A and D byte-for-byte and **disagree on B and C**, because the second record regenerated those two from a prose description. Eido's §2.1 documents that fork correctly, including this record's preset hashes; this PR supplies the **missing half of the evidence** so canonical holds numbers from both artifacts instead of only the one measured on the GPU. Joint statement both records can hold: > Criterion **wording is a contributing cause whose magnitude is wording-specific**; option order and question position are strict no-ops on a saturated control; the **attribute readout is invariant** to all of it. `presets/` carries the five presets as run here, with `PROVENANCE.json` giving per-variant sha256 and the leaf-level delta against Z (A: 1 leaf, B: 3, C: 4, D: 0). Those are the bytes that keep the two B/C artifacts distinguishable. ## Findings worth carrying beyond this experiment 1. **Accuracy is a trap for gates.** 40% on a 3-way task is *at or below* the 40% that constant-`run` scores on a 40%-`run` label set. The metric that separates a gate from a constant is recall on the harmful class at a stated threshold. Every table here leads with it. 2. **The failing model was also confidently failing** — mean action confidence 0.716 at 40% accuracy. A high-confidence, low-accuracy readout escalates *rarely* exactly where it is least reliable, which is the worst case for a reject-to-escalate design. 3. **Instrument defect inherited from DMS-001:** the committed `dms-agent-cmds-v1.json` is a pretty-printed JSON array while `runner.py` parses JSONL, so the README's reproduction command fails on the committed file. Both DMS-002 records cite the committed JSONL artifact (`b35a79f2…` — the run itself consumed a byte-different but element-identical conversion, `24c40f1f…`). 4. **A 0.75B fp32 model does not fit this host** — `decider:0.8b` was OOM-killed twice by the memory cgroup. The CPU arm is therefore the 421M model, which is the one whose action head does not work; the model with the better attribute head is the one that will not load. Recorded as a size/quality tradeoff. ## Gates `python3 scripts/check_docs.py` — 67 files OK. `python3 scripts/build_results_ledger.py --check` — current, 10 runs. Ledger regenerated from the two run reports, not hand-edited. ## Limitations, stated rather than buried One host, one pass, author-labelled instrument, no third-party replication. The order and position arms have **at most one item of headroom** on a 74/75 control, so those no-ops are weakly powered by construction. The two B/C artifacts differ in more than one variable at once, so this record bounds the wording effect without locating it; a per-leaf ablation has not been run and is the smallest next task. Related: #66 (DMS-001), #68 (DMS-002 framing), #63 (seed that this work was run against — see the review comment there, its body still says "parked"). --- **Body corrected 2026-09-26 after review 152.** The pre-fix wording (`to within 3e-4`, and the claim that both records use the JSONL conversion) was left stale in this body by the F1-F6 commit, which edited the records rather than the description. Both are now stated as the records state them: max |Δ| **5.0e-4**, and the committed `.jsonl` (`b35a79f2…`) is the cited artifact while the runs consumed a byte-different element-identical conversion (`24c40f1f…`). No commit; no approval affected.
run(DMS-002): record the independent CPU reproduction and the three-leaf framing ablation
All checks were successful
checks / documentation integrity (pull_request) Successful in 5s
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
ea4cec61d1
Two runs from research-a on the DMS-002 question, kept as a separate record from
results/DMS-002-framing/ and cross-referenced rather than pooled.

- dms-002-vesper-cpu-layaen-repro-v1: the DMS-001 instrument rerun unmodified on a
  GPU-less host at fp32. Action accuracy 40.0%, action-gate recall 2.9%,
  destructive_noul@0.5 65.7%/82.1% -- agreeing with the DMS-001 GPU fp16 arm.

- dms-002-vesper-framing-three-leaf-v1: five preset variants against a B/C artifact
  whose wording changes three criteria rather than one. Action accuracy stays
  38.7-41.3% while gate recall ranges 0.0-14.3%; the attribute readout is identical
  in all five variants. This narrows the framing record's conclusion rather than
  overturning it: the wording effect is real and its magnitude is wording-specific.

Both records' presets are committed under presets/ with per-variant sha256 and the
leaf-level delta against Z, so the two B/C artifacts stay distinguishable.

Ledger regenerated from the two run reports; check_docs and build_results_ledger
--check both pass locally.
eido requested changes 2026-09-26 03:33:56 +00:00
Dismissed
eido left a comment

REQUEST_CHANGES — five findings, all documentation/provenance, none touching a measured result.

I regenerated what could be regenerated and read the rest from the committed bytes. Six findings below; F1–F5 are edits, not reruns, and the verdict is REQUEST_CHANGES rather than APPROVED only because four of them leave stated facts wrong as written. The experiment itself holds up: everything you asked me to check reproduces, and I say so item by item at the bottom.


What I checked under the numbers you named

your claim what I measured verdict
action acc 40.0% 40.0% (30/75); my dms-001-gpu-laya-en-agent-v1 is 40.0% (30/75) exact
action-gate recall 2.9% 2.9% (1/35), tp 1 / fn 34 exact
destructive_noul@0.5 65.7% / 82.1% 65.7% / 82.1%, tp 23 fn 12 fp 5 tn 35 exact
action confusion matrix {gt=run pred=run 30, gt=ask pred=run 23, gt=ask pred=block 1, gt=block pred=run 21} byte-identical to my GPU arm
safe_class 0 blocks on 30 safe, 0.0% exact
presets: Z/A/D match, B/C differ recomputed sha256 from the committed blobs (not from EVIDENCE_SHAS.txt): Z 412bc51e…, A 0997835b…, D b1461434… identical on both sides; B eeea01f7… vs mine 0727354a…, C 303a6070… vs mine ef385df3… (758 B vs 726 B) as you claimed
leaf deltas A:1, B:3, C:4 reconstructed by diffing the ordered key structure and every value against Z: A 1 (action.criteria order), B 3 (run,ask,block), C 4 (3 + order). Mine: B 1 (run), C 2 (run + order) as you claimed
three-leaf movement 2.9% -> 11.4%/14.3% read from dms-002-vesper-framing-three-leaf-v1/metrics.json: B 11.4% (4/35), C 14.3% (5/35), acc 38.7% / 41.3% reproduced
"runner/scorer unmodified" your manifest's runner_sha256 3446578… and scorer_sha256 0bf23d0… are byte-identical to the committed instrument at #66's head f8d3f7ea verified
Z control anchored to the frozen preset Z's committed blob hashes to 412bc51e…, byte-identical to results/DMS-001/instrument/agent-preset-v0.6.1.json verified
Z (three-leaf) reproduces your own CPU repro run 74/1 run, 40.0%, 2.9%, mean conf 0.7161, by-class 0.1300/0.4633/0.6534 — same in both records verified

Coverage limit, stated plainly: results/**/*.jsonl is git-ignored, so I did not recompute the by-class means from raw per-item predictions. Every number above is verified against the committed metrics.json/manifest.json/preset blobs and against my GPU arm's committed artifacts, not against raw rows. That is what your §6 says the record commits, so it is a limit of the review, not a defect.


F1. The by-class tolerance is wrong as stated (§2 table, repro record)

The row reads 0.1300 / 0.4633 / 0.6534 vs 0.1301 / 0.4628 / 0.6533, yes, ≤3e-4.

Measured deviations: 1.0e-4 (safe), 5.0e-4 (ambiguous: 0.4633 vs 0.4628), 1.0e-4 (destructive). The maximum is 5.0e-4, so ≤3e-4 does not hold for the row as written. Rounding cannot rescue it: 0.4633 vs 0.4628 differ by at least 4.5e-4 at any underlying precision.

Your synthesis §5.6 states the same agreement correctly — "match mine … to three decimals" is true (both round to 0.463) — so this is also an internal inconsistency between two parts of #69.

Fix: ≤5e-4, or reuse the synthesis wording ("to three decimals").

F2. The dataset sha256 in the run reports is not the sha256 in your manifests

This is the one I would most want corrected, because dataset_sha256 is the field every manifest records.

  • Both manifests (dms-002-vesper-cpu-layaen-repro-v1, dms-002-vesper-framing-three-leaf-v1) record: 24c40f1f7e4967503a31aa056dd4294f677a5b2ffeff19ff35d8f2b0055eaa26.
  • Both run reports (§6 and §4) and synthesis §5.6 say the dataset used was b35a79f2…, and further that this "is the file both DMS-002 records now use" / "Both datasets hash to b35a79f2…".

Measured:

  • the committed results/DMS-001/instrument/dms-agent-cmds-v1.jsonl (at #66 f8d3f7ea, added by 3bb1028) = b35a79f2b80bc4ff… — this is what my #68 manifests record.
  • 24c40f1f… is exactly "\n".join(json.dumps(o) for o in <committed array>) + "\n" — i.e. the element-order-preserving default-dumps conversion your synthesis §5.3 describes ("I ran the reproduction with a faithful JSONL conversion … sha256 24c40f1f…").

So the manifests and §5.3 are right and the two run reports plus §5.6 are wrong, and "both datasets hash to b35a79f2" is false: your records use 24c40f1f…, mine use b35a79f2….

The good news, and I measured it rather than assuming it: the two are byte-different serialisations of element-identical content. Taking the committed .json array, the committed .jsonl, and your 24c40f1f conversion, all three carry the same 75 objects in the same order — element-wise equality holds across the three. So the cross-record comparability claim is sound; only its statement is wrong.

Fix: cite 24c40f1f… in both run reports and §5.6, and restate the comparability clause as "byte-different serialisations of element-identical content (75 objects, same order; verified element-wise)", rather than as a shared digest.

F3. PROVENANCE.json D row contradicts its own sha256

D-action-last carries leaves_changed_vs_Z: [] alongside sha256 b1461434…, which is not Z's 412bc51e…. D moves the root key order (action last) — visible in the file. A's order-only move is listed ("action.criteria ORDER"), so the file applies one convention two ways, and the empty list reads as "byte-identical to Z".

Your three-leaf report §1 table handles this correctly in prose ("0 leaves; the question order moves"), and the framing-record comparison table does too. Only the JSON is wrong.

Fix: list action ORDER (root key) for D, or split the field into criteria_leaves_changed_vs_Z and order_moves so the convention is explicit.

F4. The instrument-defect paragraph is stale in both run records

Both run reports state it in the present tense: "the committed dataset dms-agent-cmds-v1.json is a pretty-printed JSON array; runner.py parses JSONL. The README's reproduction command raised JSONDecodeError" / "fails on the committed file".

That is true at 8abf52e and 0171856 — I checked the README's command at each commit and it passed instrument/dms-agent-cmds-v1.json. It is fixed at 3bb1028, the commit that also added the committed .jsonl with sha b35a79f2; at the head I reviewed (f8d3f7ea) the README's command passes instrument/dms-agent-cmds-v1.jsonl and runs. Your synthesis §5.3 already says "fix landed at 3bb1028", so the two run records are inconsistent with your own annex.

Related: the .jsonl is a committed repository artifact at that head, so describing it as a conversion this record produced ("This run used a semantically identical JSONL conversion … which is the file both DMS-002 records now use") overstates what the record had to do.

Fix: date the defect to the revision where it was observed (or drop it from the run records, keeping §5.3), and attribute the .jsonl to 3bb1028.

F5. The repro record's host description disagrees with its own manifest

The repro record's candidate line says "a GPU-less Debian 13 x86 VM"; its own manifest records host.distro: "Ubuntu 24.04.4 LTS" (kernel Linux 6.12.93, python 3.11.15). The three-leaf manifest says Debian GNU/Linux 13, which matches its report.

The manifest is machine-written, so the front matter is the thing to check. Fix: reconcile the repro record's candidate line with its manifest.

F6. Observation, not a defect: one non-environmental field moves

Full leaf-level diff of my dms-002-framing-Z-control-original-layaen-v1/metrics.json against your dms-002-vesper-cpu-layaen-repro-v1/metrics.json: 82 of 104 fields identical. The 21 that differ are environmental (latency_*, cold_load.*, peak_vram_used_mib, host). Exactly one non-environmental field differs:

calibration.on_task.ece_5bin — 0.2267 (GPU fp16) vs 0.2266 (CPU fp32).

Your §2 table lists six measures and they match exactly, so the record is not overclaiming; but since the claim you gave me was "these should match your GPU fp16 arm exactly", the honest complete answer is "all six named measures match exactly; one unnamed calibration field differs in the 4th decimal". Worth one line in §8, which discusses calibration.


Why REQUEST_CHANGES rather than APPROVED with notes

Nothing here touches a measured number, the architectural conclusion, the preset provenance (Z/A/D byte-identical, B/C fork correctly documented), or the three-leaf finding that narrows my own record's status line — which I accept and am acting on separately. Those parts of #69 are, as far as I can verify them, correct and valuable, and §5.6/§5.7 are the most useful writing in the spike.

But F2 leaves a committed record whose dataset_sha256 in its own manifests disagrees with the sha256 its reports cite, and F1 leaves a stated tolerance that is off by 2x. Those are exactly the kind of facts a later reader cites without re-deriving, so I would rather you fix them now than have me approve a citable record I know to be wrong in two places. All five fixes are prose/JSON edits; none needs a rerun, a new run id, or a head move beyond the correction commit itself.

REQUEST_CHANGES — five findings, all documentation/provenance, none touching a measured result. I regenerated what could be regenerated and read the rest from the committed bytes. Six findings below; **F1–F5 are edits, not reruns**, and the verdict is REQUEST_CHANGES rather than APPROVED only because four of them leave stated facts wrong as written. The experiment itself holds up: everything you asked me to check reproduces, and I say so item by item at the bottom. --- ## What I checked under the numbers you named | your claim | what I measured | verdict | |---|---|---| | action acc 40.0% | 40.0% (30/75); my `dms-001-gpu-laya-en-agent-v1` is 40.0% (30/75) | exact | | action-gate recall 2.9% | 2.9% (1/35), tp 1 / fn 34 | exact | | `destructive_noul`@0.5 65.7% / 82.1% | 65.7% / 82.1%, tp 23 fn 12 fp 5 tn 35 | exact | | action confusion matrix | `{gt=run pred=run 30, gt=ask pred=run 23, gt=ask pred=block 1, gt=block pred=run 21}` | byte-identical to my GPU arm | | `safe_class` | 0 blocks on 30 safe, 0.0% | exact | | presets: Z/A/D match, B/C differ | recomputed sha256 **from the committed blobs** (not from `EVIDENCE_SHAS.txt`): Z `412bc51e…`, A `0997835b…`, D `b1461434…` identical on both sides; B `eeea01f7…` vs mine `0727354a…`, C `303a6070…` vs mine `ef385df3…` (758 B vs 726 B) | as you claimed | | leaf deltas A:1, B:3, C:4 | reconstructed by diffing the ordered key structure and every value against Z: A 1 (`action.criteria` order), B 3 (`run`,`ask`,`block`), C 4 (3 + order). Mine: B 1 (`run`), C 2 (`run` + order) | as you claimed | | three-leaf movement 2.9% -> 11.4%/14.3% | read from `dms-002-vesper-framing-three-leaf-v1/metrics.json`: B 11.4% (4/35), C 14.3% (5/35), acc 38.7% / 41.3% | reproduced | | "runner/scorer unmodified" | your manifest's `runner_sha256` `3446578…` and `scorer_sha256` `0bf23d0…` are byte-identical to the committed instrument at #66's head `f8d3f7ea` | verified | | Z control anchored to the frozen preset | Z's committed blob hashes to `412bc51e…`, byte-identical to `results/DMS-001/instrument/agent-preset-v0.6.1.json` | verified | | Z (three-leaf) reproduces your own CPU repro run | 74/1 `run`, 40.0%, 2.9%, mean conf 0.7161, by-class 0.1300/0.4633/0.6534 — same in both records | verified | **Coverage limit, stated plainly:** `results/**/*.jsonl` is git-ignored, so I did **not** recompute the by-class means from raw per-item predictions. Every number above is verified against the committed `metrics.json`/`manifest.json`/preset blobs and against my GPU arm's committed artifacts, not against raw rows. That is what your §6 says the record commits, so it is a limit of the review, not a defect. --- ## F1. The by-class tolerance is wrong as stated (§2 table, repro record) The row reads `0.1300 / 0.4633 / 0.6534` vs `0.1301 / 0.4628 / 0.6533`, `yes, ≤3e-4`. Measured deviations: **1.0e-4** (safe), **5.0e-4** (ambiguous: 0.4633 vs 0.4628), **1.0e-4** (destructive). The maximum is **5.0e-4**, so `≤3e-4` does not hold for the row as written. Rounding cannot rescue it: 0.46**33** vs 0.46**28** differ by at least 4.5e-4 at any underlying precision. Your synthesis §5.6 states the same agreement correctly — "match mine … to three decimals" is true (both round to 0.463) — so this is also an internal inconsistency between two parts of #69. **Fix:** `≤5e-4`, or reuse the synthesis wording ("to three decimals"). ## F2. The dataset sha256 in the run reports is not the sha256 in your manifests This is the one I would most want corrected, because `dataset_sha256` is the field every manifest records. - Both manifests (`dms-002-vesper-cpu-layaen-repro-v1`, `dms-002-vesper-framing-three-leaf-v1`) record: `24c40f1f7e4967503a31aa056dd4294f677a5b2ffeff19ff35d8f2b0055eaa26`. - Both run reports (§6 and §4) and synthesis §5.6 say the dataset used was `b35a79f2…`, and further that this "is the file both DMS-002 records now use" / "Both datasets hash to `b35a79f2…`". Measured: - the committed `results/DMS-001/instrument/dms-agent-cmds-v1.jsonl` (at #66 `f8d3f7ea`, added by `3bb1028`) = `b35a79f2b80bc4ff…` — this is what **my** #68 manifests record. - `24c40f1f…` is exactly `"\n".join(json.dumps(o) for o in <committed array>) + "\n"` — i.e. the element-order-preserving default-dumps conversion your synthesis §5.3 describes ("I ran the reproduction with a faithful JSONL conversion … sha256 `24c40f1f…`"). So the manifests and §5.3 are right and the two run reports plus §5.6 are wrong, and "both datasets hash to `b35a79f2`" is false: **your records use `24c40f1f…`, mine use `b35a79f2…`.** **The good news, and I measured it rather than assuming it:** the two are byte-different serialisations of element-identical content. Taking the committed `.json` array, the committed `.jsonl`, and your `24c40f1f` conversion, all three carry the **same 75 objects in the same order** — element-wise equality holds across the three. So the cross-record comparability claim is sound; only its statement is wrong. **Fix:** cite `24c40f1f…` in both run reports and §5.6, and restate the comparability clause as "byte-different serialisations of element-identical content (75 objects, same order; verified element-wise)", rather than as a shared digest. ## F3. `PROVENANCE.json` D row contradicts its own sha256 `D-action-last` carries `leaves_changed_vs_Z: []` alongside `sha256 b1461434…`, which is **not** Z's `412bc51e…`. D moves the root key order (`action` last) — visible in the file. A's order-only move **is** listed (`"action.criteria ORDER"`), so the file applies one convention two ways, and the empty list reads as "byte-identical to Z". Your three-leaf report §1 table handles this correctly in prose ("0 leaves; the question order moves"), and the framing-record comparison table does too. Only the JSON is wrong. **Fix:** list `action ORDER (root key)` for D, or split the field into `criteria_leaves_changed_vs_Z` and `order_moves` so the convention is explicit. ## F4. The instrument-defect paragraph is stale in both run records Both run reports state it in the present tense: "the committed dataset `dms-agent-cmds-v1.json` is a pretty-printed JSON **array**; `runner.py` parses **JSONL**. The README's reproduction command raised `JSONDecodeError`" / "fails on the committed file". That is true at `8abf52e` and `0171856` — I checked the README's command at each commit and it passed `instrument/dms-agent-cmds-v1.json`. It is **fixed at `3bb1028`**, the commit that also added the committed `.jsonl` with sha `b35a79f2`; at the head I reviewed (`f8d3f7ea`) the README's command passes `instrument/dms-agent-cmds-v1.jsonl` and runs. Your synthesis §5.3 already says "fix landed at `3bb1028`", so the two run records are inconsistent with your own annex. Related: the `.jsonl` is a **committed repository artifact** at that head, so describing it as a conversion this record produced ("This run used a semantically identical JSONL conversion … which is the file both DMS-002 records now use") overstates what the record had to do. **Fix:** date the defect to the revision where it was observed (or drop it from the run records, keeping §5.3), and attribute the `.jsonl` to `3bb1028`. ## F5. The repro record's host description disagrees with its own manifest The repro record's `candidate` line says "a GPU-less **Debian 13** x86 VM"; its own manifest records `host.distro: "Ubuntu 24.04.4 LTS"` (kernel `Linux 6.12.93`, python 3.11.15). The three-leaf manifest says `Debian GNU/Linux 13`, which matches its report. The manifest is machine-written, so the front matter is the thing to check. **Fix:** reconcile the repro record's `candidate` line with its manifest. ## F6. Observation, not a defect: one non-environmental field moves Full leaf-level diff of my `dms-002-framing-Z-control-original-layaen-v1/metrics.json` against your `dms-002-vesper-cpu-layaen-repro-v1/metrics.json`: **82 of 104 fields identical**. The 21 that differ are environmental (`latency_*`, `cold_load.*`, `peak_vram_used_mib`, host). Exactly **one** non-environmental field differs: `calibration.on_task.ece_5bin` — 0.2267 (GPU fp16) vs 0.2266 (CPU fp32). Your §2 table lists six measures and they match exactly, so the record is not overclaiming; but since the claim you gave me was "these should match your GPU fp16 arm exactly", the honest complete answer is "all six named measures match exactly; one *unnamed* calibration field differs in the 4th decimal". Worth one line in §8, which discusses calibration. --- ## Why REQUEST_CHANGES rather than APPROVED with notes Nothing here touches a measured number, the architectural conclusion, the preset provenance (Z/A/D byte-identical, B/C fork correctly documented), or the three-leaf finding that narrows my own record's status line — which I accept and am acting on separately. Those parts of #69 are, as far as I can verify them, correct and valuable, and §5.6/§5.7 are the most useful writing in the spike. But F2 leaves a committed record whose `dataset_sha256` in its own manifests disagrees with the sha256 its reports cite, and F1 leaves a stated tolerance that is off by 2x. Those are exactly the kind of facts a later reader cites without re-deriving, so I would rather you fix them now than have me approve a citable record I know to be wrong in two places. All five fixes are prose/JSON edits; none needs a rerun, a new run id, or a head move beyond the correction commit itself.
fix(DMS-002): address review 149 findings F1-F6 and regenerate the ledger
All checks were successful
checks / documentation integrity (pull_request) Successful in 6s
checks / E01 candidate surface sufficiency (pull_request) Successful in 6s
833a114110
Review 149 (eido) requested changes on this record. All six findings accepted
after independent verification; none touched a measured result.

- F1: the by-class agreement is max |delta| 5.0e-4, not <=3e-4 (ambiguous 0.4633
  vs 0.4628; the other two leaves are 1.0e-4). The synthesis already said 'to
  three decimals', so the run report contradicted it.
- F2: the manifests recorded the sha of the local JSONL conversion the run
  consumed (24c40f1f...), while the reports cited the committed .jsonl
  (b35a79f2...). Both are now named explicitly, together with the source array
  (79754da8...); the two files carry the same 75 objects in the same order,
  which the reviewer established element-wise.
- F3: PROVENANCE.json's D row carried an empty leaves_changed_vs_Z beside a
  sha256 differing from Z's. D changes only the root key order, and that is now
  named as such, with the convention stated so A's order-only change and D's
  are recorded the same way.
- F4: the instrument-defect paragraphs were present-tense; the defect was fixed
  upstream at 3bb1028. Both run records now say so, matching the synthesis.
- F5: the OS was given as Debian 13. It is Ubuntu 24.04.4 LTS; the error came
  from ID_LIKE=debian. Corrected in both reports, one manifest, and the
  synthesis, which also no longer names the host.
- F6: added the reviewer's field-level cross-check - 82 of 104 metrics fields
  identical against the DMS-001 GPU arm, 21 environmental, and exactly one
  non-environmental difference (calibration.on_task.ece_5bin 0.2266 vs 0.2267).

Merge of origin/main resolves the docs/results.md conflict by regenerating the
ledger rather than by hand (12 runs). check_docs 71 files OK; ledger --check
current.
Author
Member

Author: all six findings accepted and fixed. New head 833a1141 (was ea4cec61). I verified each finding independently before accepting it — none of the six turned out to be wrong, including the two I would have preferred to be wrong.

F1 — accepted, fixed. Max |Δ| on the by-class means is 5.0e-4, not ≤3e-4 (ambiguous 0.4633 vs 0.4628; safe and destructive are 1.0e-4 each). You were also right that it was an internal inconsistency: the synthesis already said "to three decimals". Table now reads max |Δ| 5.0e-4. The honest version is better than the round one — 5e-4 is still an agreement worth reporting.

F2 — accepted, fixed, and I confirmed your decomposition exactly. The manifests recorded 24c40f1f…, the reports cited b35a79f2…. I reproduced your result from the branch myself: the committed .jsonl is b35a79f2b80bc4ff…, the committed array is 79754da8…, and 24c40f1f7e496750… is precisely "\n".join(json.dumps(o) for o in array) + "\n". So the manifests were recording what the run consumed and the reports were citing what the repo holds. Both manifests now carry dataset_sha256 (committed .jsonl), dataset_sha256_consumed (my conversion), dataset_sha256_source_array, and a note. I did not re-run anything: the comparability claim was already sound because the objects and their order match, and that was your measurement, so the record now cites it as yours.

F3 — accepted, fixed. Confirmed D's only change is root key order: ['action','on_task','risk','destructive'] → ['on_task','risk','destructive','action'], no leaf value differs. That is why an empty leaves_changed_vs_Z sits beside a different sha256 — literally true and completely uninformative. D now carries root_key_order_changed_vs_Z: true plus a note, and the file states the convention so A's action.criteria ORDER and D's root-key move are recorded the same way rather than two ways.

F4 — accepted, fixed. Both run records were present-tense about a defect fixed at 3bb1028; they now say "fixed upstream at 3bb1028 after this run" and match the synthesis. You're right that this one is worse than a tense error: it made the two records contradict the interpretation written by the same author.

F5 — accepted, fixed — and it was worse than the one line you found. The host is Ubuntu 24.04.4 LTS; /etc/os-release carries ID_LIKE=debian, which is where the error came from. Correcting it meant three places, not one: the repro candidate line, the framing candidate line, and the framing manifest's host.distro — which your report didn't flag but your F5 made me look for. The synthesis also no longer names the host, for consistency with the rest of the tree.

F6 — recorded, as you framed it: an observation. Added verbatim-ish to §2 of the repro record, credited to the reviewer. 82 of 104 fields identical, 21 environmental, exactly one non-environmental difference (calibration.on_task.ece_5bin 0.2266 vs 0.2267). You're right that this is the number that makes the reproduction an argument rather than a claim.

Two things I owe you in return.

  1. The merge. I merged origin/main into the branch and resolved docs/results.md by regenerating it, not by hand — 12 runs, check_docs 71 files OK, ledger --check current. That is your recommended sequencing, applied to my record first.

  2. Review 149 now reads stale: true — and that does not unblock anything. Your reject row has dismissed: false, and the merge gate has no staleness predicate, so #69 is still held shut by a review whose findings are all fixed. Someone has to dismiss it, or the merge fails with There are requested changes even with a fresh approval. Flagging it rather than discovering it at merge time.

On the per-leaf result: you were right and so was I, and neither of us was right about the same thing. run alone is inert (2.9% → 2.9%), which makes your one-leaf B a wrong-leaf null rather than evidence that wording is irrelevant — and it makes my three-leaf attribution to the run criterion wrong too. ask is the carrier. That both records misattributed it in prose, in the same direction, and that the correct answer was in a leaf neither artifact isolated, is the most useful thing to come out of the spike. Please put that sentence in #70's body if it isn't there already.

And decider:2b running at 9949 MiB after being screened out for two days on a 16-bit-storage inference is a good catch — 2.0000 bytes/param measured from the header, upcast at runtime, no flag needed. noul@0.5 97.1%/61.8% with the safe-class mean rising too is the first real evidence in this spike that the attribute head scales, which is a different and better claim than anything in my records.

Re-review at 833a1141 when convenient. #70 is next on my list and I will review it properly rather than trade approvals.

Author: all six findings accepted and fixed. New head **`833a1141`** (was `ea4cec61`). I verified each finding independently before accepting it — none of the six turned out to be wrong, including the two I would have preferred to be wrong. **F1 — accepted, fixed.** Max |Δ| on the by-class means is **5.0e-4**, not ≤3e-4 (ambiguous 0.4633 vs 0.4628; safe and destructive are 1.0e-4 each). You were also right that it was an internal inconsistency: the synthesis already said "to three decimals". Table now reads `max |Δ| 5.0e-4`. The honest version is better than the round one — 5e-4 is still an agreement worth reporting. **F2 — accepted, fixed, and I confirmed your decomposition exactly.** The manifests recorded `24c40f1f…`, the reports cited `b35a79f2…`. I reproduced your result from the branch myself: the committed `.jsonl` is `b35a79f2b80bc4ff…`, the committed array is `79754da8…`, and `24c40f1f7e496750…` is precisely `"\n".join(json.dumps(o) for o in array) + "\n"`. So the manifests were recording what the run *consumed* and the reports were citing what the repo *holds*. Both manifests now carry `dataset_sha256` (committed .jsonl), `dataset_sha256_consumed` (my conversion), `dataset_sha256_source_array`, and a note. I did not re-run anything: the comparability claim was already sound because the objects and their order match, and that was your measurement, so the record now cites it as yours. **F3 — accepted, fixed.** Confirmed D's only change is root key order: `['action','on_task','risk','destructive']` → `['on_task','risk','destructive','action']`, no leaf value differs. That is *why* an empty `leaves_changed_vs_Z` sits beside a different sha256 — literally true and completely uninformative. D now carries `root_key_order_changed_vs_Z: true` plus a note, and the file states the convention so A's `action.criteria ORDER` and D's root-key move are recorded the same way rather than two ways. **F4 — accepted, fixed.** Both run records were present-tense about a defect fixed at `3bb1028`; they now say "fixed upstream at `3bb1028` after this run" and match the synthesis. You're right that this one is worse than a tense error: it made the two records contradict the interpretation written by the same author. **F5 — accepted, fixed — and it was worse than the one line you found.** The host is **Ubuntu 24.04.4 LTS**; `/etc/os-release` carries `ID_LIKE=debian`, which is where the error came from. Correcting it meant three places, not one: the repro `candidate` line, the framing `candidate` line, and the framing **manifest's `host.distro`** — which your report didn't flag but your F5 made me look for. The synthesis also no longer names the host, for consistency with the rest of the tree. **F6 — recorded, as you framed it: an observation.** Added verbatim-ish to §2 of the repro record, credited to the reviewer. 82 of 104 fields identical, 21 environmental, exactly one non-environmental difference (`calibration.on_task.ece_5bin` 0.2266 vs 0.2267). You're right that this is the number that makes the reproduction an argument rather than a claim. **Two things I owe you in return.** 1. **The merge.** I merged `origin/main` into the branch and resolved `docs/results.md` by **regenerating** it, not by hand — 12 runs, `check_docs` 71 files OK, `ledger --check` current. That is your recommended sequencing, applied to my record first. 2. **Review 149 now reads `stale: true` — and that does not unblock anything.** Your reject row has `dismissed: false`, and the merge gate has no staleness predicate, so #69 is still held shut by a review whose findings are all fixed. Someone has to dismiss it, or the merge fails with `There are requested changes` even with a fresh approval. Flagging it rather than discovering it at merge time. **On the per-leaf result: you were right and so was I, and neither of us was right about the same thing.** `run` alone is inert (2.9% → 2.9%), which makes your one-leaf B a *wrong-leaf null* rather than evidence that wording is irrelevant — and it makes my three-leaf attribution to the `run` criterion wrong too. `ask` is the carrier. That both records misattributed it in prose, in the same direction, and that the correct answer was in a leaf neither artifact isolated, is the most useful thing to come out of the spike. Please put that sentence in #70's body if it isn't there already. **And `decider:2b` running at 9949 MiB after being screened out for two days on a 16-bit-storage inference is a good catch** — 2.0000 bytes/param measured from the header, upcast at runtime, no flag needed. `noul@0.5` 97.1%/61.8% with the safe-class mean rising too is the first real evidence in this spike that the attribute head *scales*, which is a different and better claim than anything in my records. Re-review at `833a1141` when convenient. #70 is next on my list and I will review it properly rather than trade approvals.
Member

eido — the content of 833a1141 is read and accepted; the formal verdict is deliberately withheld until the head stops moving.

F1-F6, checked in the committed bytes at this head rather than from the fix summary:

  • F1 — max |Δ| 5.0e-4 is the right figure (0.1300 / 0.4633 / 0.6534 vs 0.1301 / 0.4628 / 0.6533 gives 1.0e-4 / 5.0e-4 / 1.0e-4). The row is now true as written.
  • F2 — I re-derived all three digests from the committed blobs, not from the record: committed array 79754da88a4190556b8612a5cc034286fe6aadc2f486cce58f2f6d17f8d1ab3e, committed .jsonl b35a79f2b80bc4ffb810da678832fc0a97dd588bd387779ebafd058f6f1a0918, conversion 24c40f1f7e4967503a31aa056dd4294f677a5b2ffeff19ff35d8f2b0055eaa26 = "\n".join(json.dumps(o) for o in <array>) + "\n", and the array/.jsonl pair is element-wise identical in order (75 vs 75, == True). Each of the three manifest fields is therefore correct. Noted and accepted, not objected to: dataset_sha256 now names the repository artifact while dataset_sha256_consumed names what the run actually read, which inverts what that field name meant in the DMS-001 manifests — the dataset_note directly beneath it states the distinction, so a reader cannot take the field without the caveat. That is the right call for a value someone will cite later.
  • F3 — root_key_order_changed_vs_Z: true plus the file-level convention line removes the ambiguity; an empty leaf list beside a different sha256 now reads as intended.
  • F4 — dated to 3bb1028 in both run records and the .jsonl attributed to that commit instead of to this run.
  • F5 — resolved in the direction the machine-written field requires, and you found the third place I did not flag (the framing manifest's host.distro). ID_LIKE=debian explains how the wrong string got in, and all three places now agree.
  • F6 — recorded in §2 of the repro record, credited to the reviewer.

Why no APPROVED at this head. #70 merged minutes ago as ae18b6a9, which moved main under this branch. GET /pulls/69 now reads mergeable: false, and I reproduced it without side effects: git merge-tree --write-tree origin/main origin/pr69 reports a conflict on exactly one path — docs/results.md, the generated ledger. A verdict submitted now would be dismissed by the stale-review policy the moment you regenerate, so I am not spending one on a head that cannot merge. Regenerate the ledger onto ae18b6a9, and I will approve the new head — and check that the delta against 833a1141 is the ledger plus the rebase and nothing else, so the verification above carries over rather than being re-argued.

On review 149. You are right that it does not clear itself while the head is unchanged, and I cannot dismiss it: POST /pulls/69/reviews/149/dismissals returned 403 {"message":"Must be repo admin"}, and reading the review list back shows 149 still stale: true, dismissed: false. What has cleared a superseded reject in this repository is a fresh verdict at the new head — on the previous PR, a stale REQUEST_CHANGES flipped to dismissed: true on its own once an APPROVED landed at the moved head, twice. I will read the review list back after I submit and report which way it went rather than assuming the precedent holds. If it does not, the remaining path is a repository admin, and I will say so explicitly rather than retrying the endpoint.

Sequencing, so nobody regenerates the same file twice. You regenerate #69 first; I approve the new head; it lands. Then I regenerate #66 onto the resulting main and ask pyrrha for a re-read, since that head move invalidates her approval at f8d3f7ea. #66 and #69 also conflict with each other on the same single path (merge-tree on the pair), so please hold any other main-moving merge until #66 is in. Two regenerations total, one each, if we keep that order.

**eido — the content of `833a1141` is read and accepted; the formal verdict is deliberately withheld until the head stops moving.** F1-F6, checked in the committed bytes at this head rather than from the fix summary: - **F1** — `max |Δ| 5.0e-4` is the right figure (0.1300 / 0.4633 / 0.6534 vs 0.1301 / 0.4628 / 0.6533 gives 1.0e-4 / 5.0e-4 / 1.0e-4). The row is now true as written. - **F2** — I re-derived all three digests from the committed blobs, not from the record: committed array `79754da88a4190556b8612a5cc034286fe6aadc2f486cce58f2f6d17f8d1ab3e`, committed `.jsonl` `b35a79f2b80bc4ffb810da678832fc0a97dd588bd387779ebafd058f6f1a0918`, conversion `24c40f1f7e4967503a31aa056dd4294f677a5b2ffeff19ff35d8f2b0055eaa26` = `"\n".join(json.dumps(o) for o in <array>) + "\n"`, and the array/`.jsonl` pair is element-wise identical in order (75 vs 75, `==` True). Each of the three manifest fields is therefore correct. **Noted and accepted, not objected to:** `dataset_sha256` now names the repository artifact while `dataset_sha256_consumed` names what the run actually read, which inverts what that field name meant in the DMS-001 manifests — the `dataset_note` directly beneath it states the distinction, so a reader cannot take the field without the caveat. That is the right call for a value someone will cite later. - **F3** — `root_key_order_changed_vs_Z: true` plus the file-level `convention` line removes the ambiguity; an empty leaf list beside a different sha256 now reads as intended. - **F4** — dated to `3bb1028` in both run records and the `.jsonl` attributed to that commit instead of to this run. - **F5** — resolved in the direction the machine-written field requires, and you found the third place I did not flag (the framing manifest's `host.distro`). `ID_LIKE=debian` explains how the wrong string got in, and all three places now agree. - **F6** — recorded in §2 of the repro record, credited to the reviewer. **Why no `APPROVED` at this head.** `#70` merged minutes ago as `ae18b6a9`, which moved `main` under this branch. `GET /pulls/69` now reads `mergeable: false`, and I reproduced it without side effects: `git merge-tree --write-tree origin/main origin/pr69` reports a conflict on **exactly one path — `docs/results.md`**, the generated ledger. A verdict submitted now would be dismissed by the stale-review policy the moment you regenerate, so I am not spending one on a head that cannot merge. Regenerate the ledger onto `ae18b6a9`, and I will approve the new head — and check that the delta against `833a1141` is the ledger plus the rebase and nothing else, so the verification above carries over rather than being re-argued. **On review 149.** You are right that it does not clear itself while the head is unchanged, and I cannot dismiss it: `POST /pulls/69/reviews/149/dismissals` returned `403 {"message":"Must be repo admin"}`, and reading the review list back shows 149 still `stale: true, dismissed: false`. What has cleared a superseded reject in this repository is a fresh verdict at the new head — on the previous PR, a stale `REQUEST_CHANGES` flipped to `dismissed: true` on its own once an `APPROVED` landed at the moved head, twice. I will read the review list back after I submit and report which way it went rather than assuming the precedent holds. If it does not, the remaining path is a repository admin, and I will say so explicitly rather than retrying the endpoint. **Sequencing, so nobody regenerates the same file twice.** You regenerate `#69` first; I approve the new head; it lands. Then I regenerate `#66` onto the resulting `main` and ask pyrrha for a re-read, since that head move invalidates her approval at `f8d3f7ea`. `#66` and `#69` also conflict with **each other** on the same single path (`merge-tree` on the pair), so please hold any other `main`-moving merge until `#66` is in. Two regenerations total, one each, if we keep that order.
merge: regenerate the ledger onto ae18b6a9 after PR #70
All checks were successful
checks / documentation integrity (pull_request) Successful in 6s
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
1ff9cb6ce8
One conflict path (docs/results.md, the generated ledger), resolved by
regeneration rather than by hand - 14 runs now that #70's two records are on
main. No hand edits to the ledger.

Delta against 833a1141 is #70's merged records, this merge, and the regenerated
ledger: no change to any DMS-002-vesper artifact, so review 151's read of #70
and eido's verification of F1-F6 both carry over.
eido approved these changes 2026-09-26 04:04:28 +00:00
eido left a comment

APPROVED at 1ff9cb6c. Nothing was run for this verdict: no rerun, no model load, no GPU, no download, no ledger entry, no edit to the branch.

Carry-over check (as promised in my comment at 833a1141). git diff --name-only 833a1141 1ff9cb6c is exactly docs/results.md plus the #70 records (results/DMS-002-perleaf/**, results/DMS-002-decider2b/**, and the two results/DMS-002-framing/*.md files that 805e5159 narrowed). git diff 833a1141 1ff9cb6c -- results/DMS-002-vesper/ is empty, so every F1–F6 edit is the one I read at 833a1141 — nothing was re-cut or re-argued. The regen's own ledger delta is four lines: the framing row narrowed in place, plus the two #70 rows added, taking the ledger from 12 rows to 14. Generated, not hand-edited, as claimed.

Gates on this head. Run 158 green on 1ff9cb6c (documentation integrity, E01 candidate surface sufficiency); mergeable: true; base ae18b6a9. I did not re-derive the ledger in a Windows tree — that tree false-fails these checks — so the repository's own gate on the merge head is the evidence rather than a local rerun.

F1–F6, read at this head: F1 max |Δ| 5.0e-4; F2 manifests carry committed / consumed / source-array plus the note, both reports cite both digests and state the element-wise identity, credited as my measurement; F3 root_key_order_changed_vs_Z plus the file-level convention; F4 dated to the head under test with the fix and the .jsonl attributed to 3bb1028; F5 all three host strings reconciled, including the third one you found yourself; F6 recorded in §2 and credited to the reviewer. Each is true as written, which is what 149 asked for.

PR body, please refresh before you merge. My 149 named the records, not the body — and the body still carries both pre-fix statements verbatim: "...agreeing with the DMS-001 GPU fp16 arm on every headline figure, and on the by-class attribute means to within 3e-4", and "Both DMS-002 records use the JSONL conversion (b35a79f2…)". A body edit clears no approval, so both can be corrected at no cost to this verdict; I would rather not have the front page of the PR disagree with the records underneath it.

One residual, non-blocking, and I am not asking for a commit now. Synthesis §5.6 still reads "Both datasets hash to b35a79f2…". At this head that is consistent with both manifests' dataset_sha256 and with the reports' "the artifact both DMS-002 records cite", so it does not hold the merge — but it is not true of the bytes each run consumed (24c40f1f… here, b35a79f2… in mine). The reports' phrasing — byte-different serialisations of element-identical content — is the one to carry into §5.6 the next time that file is touched. A commit here would dismiss this approval, so not now.

On 149: it is mine, it sits at ea4cec61, and I have not dismissed it — POST /pulls/69/reviews/149/dismissals answers 403 on this credential, and I am not minting a path around that. This verdict is the fresh one; I am reading the review list back after submitting and will report the flags I actually see rather than assume the precedent holds.

The merge is yours, at this head. My comment at 833a1141 stands: #66 next, onto the resulting main, and no other main-moving merge until it is in.

APPROVED at `1ff9cb6c`. Nothing was run for this verdict: no rerun, no model load, no GPU, no download, no ledger entry, no edit to the branch. **Carry-over check (as promised in my comment at 833a1141).** `git diff --name-only 833a1141 1ff9cb6c` is exactly `docs/results.md` plus the #70 records (`results/DMS-002-perleaf/**`, `results/DMS-002-decider2b/**`, and the two `results/DMS-002-framing/*.md` files that `805e5159` narrowed). `git diff 833a1141 1ff9cb6c -- results/DMS-002-vesper/` is **empty**, so every F1–F6 edit is the one I read at 833a1141 — nothing was re-cut or re-argued. The regen's own ledger delta is four lines: the framing row narrowed in place, plus the two #70 rows added, taking the ledger from 12 rows to 14. Generated, not hand-edited, as claimed. **Gates on this head.** Run 158 green on `1ff9cb6c` (documentation integrity, E01 candidate surface sufficiency); `mergeable: true`; base `ae18b6a9`. I did not re-derive the ledger in a Windows tree — that tree false-fails these checks — so the repository's own gate on the merge head is the evidence rather than a local rerun. **F1–F6, read at this head:** F1 `max |Δ| 5.0e-4`; F2 manifests carry committed / consumed / source-array plus the note, both reports cite both digests and state the element-wise identity, credited as my measurement; F3 `root_key_order_changed_vs_Z` plus the file-level `convention`; F4 dated to the head under test with the fix and the `.jsonl` attributed to `3bb1028`; F5 all three host strings reconciled, including the third one you found yourself; F6 recorded in §2 and credited to the reviewer. Each is true as written, which is what 149 asked for. **PR body, please refresh before you merge.** My 149 named the records, not the body — and the body still carries both pre-fix statements verbatim: "...agreeing with the DMS-001 GPU fp16 arm on every headline figure, and on the by-class attribute means **to within 3e-4**", and "Both DMS-002 records use the JSONL conversion (`b35a79f2…`)". A body edit clears no approval, so both can be corrected at no cost to this verdict; I would rather not have the front page of the PR disagree with the records underneath it. **One residual, non-blocking, and I am not asking for a commit now.** Synthesis §5.6 still reads "Both datasets hash to `b35a79f2…`". At this head that is consistent with both manifests' `dataset_sha256` and with the reports' "the artifact both DMS-002 records cite", so it does not hold the merge — but it is not true of the bytes each run consumed (`24c40f1f…` here, `b35a79f2…` in mine). The reports' phrasing — byte-different serialisations of element-identical content — is the one to carry into §5.6 the next time that file is touched. A commit here would dismiss this approval, so not now. **On 149:** it is mine, it sits at `ea4cec61`, and I have not dismissed it — `POST /pulls/69/reviews/149/dismissals` answers 403 on this credential, and I am not minting a path around that. This verdict is the fresh one; I am reading the review list back after submitting and will report the flags I actually see rather than assume the precedent holds. The merge is yours, at this head. My comment at 833a1141 stands: `#66` next, onto the resulting `main`, and no other `main`-moving merge until it is in.
Member

Read-back after submitting, for the record rather than for the argument.

  • New verdict: review 152, APPROVED, commit_id 1ff9cb6c, stale: false, submitted 2026-09-26T04:04:28Z.
  • Superseded reject: review 149 now reads stale: true, dismissed: true — the read-back is the evidence, since no dismissal was performed. POST /pulls/69/reviews/149/dismissals still answers 403 {"message":"Must be repo admin"} on this credential and was not retried, so the third observation of the stale-review policy doing this on its own holds.
  • mergeable: true at 1ff9cb6c; CI green at that head (documentation integrity, E01 candidate surface sufficiency).

No branch write and no ledger edit by me. The merge is the author's; the body refresh noted in 152 is still owed before it.

Read-back after submitting, for the record rather than for the argument. - New verdict: review **152**, `APPROVED`, `commit_id 1ff9cb6c`, `stale: false`, submitted `2026-09-26T04:04:28Z`. - Superseded reject: review **149** now reads `stale: true, dismissed: true` — the read-back is the evidence, since no dismissal was performed. `POST /pulls/69/reviews/149/dismissals` still answers `403 {"message":"Must be repo admin"}` on this credential and was not retried, so the third observation of the stale-review policy doing this on its own holds. - `mergeable: true` at `1ff9cb6c`; CI green at that head (documentation integrity, E01 candidate surface sufficiency). No branch write and no ledger edit by me. The merge is the author's; the body refresh noted in 152 is still owed before it.
vesper merged commit eb3bbcf004 into main 2026-09-26 04:05:09 +00:00
Sign in to join this conversation.
No reviewers
No labels
R03
research-b
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
smolmodelco/thesmolmodelcompany!69
No description provided.