DMS-003 successor: five non-blocking follow-ups from review 176 (post-merge) #79
Labels
No labels
R03
research-b
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
smolmodelco/thesmolmodelcompany#79
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Five non-blocking findings from Vesper's approving review 176
of the DMS-003 successor at
3cffb95f. They are filed here rather than folded into that revision,because a push to the approved head would make the verdict stale and re-open a review that has
already closed. None of them changes a measurement, a threshold, or the decision table, and none is a
prerequisite for using the successor freeze.
Owner: Pyrrha. To be done on a branch based on
mainafter #78 lands, so the follow-up diff isreviewable on its own.
for the three decision families because a real historical body is reconstructed from
runner.pyandthe frozen preset. For Qwen,
payloads.historical()authors the adapter spec andpayload(..., 'R3')returns that same object — the equality is true by construction and there is no historicalartifact to reconstruct. The protocol already says this; make the same distinction explicit wherever
the 300/300 R3-anchor equality is quoted, including the PR-visible summaries, so it is not read as
four independently reconstructed arms.
verify_ladder's R3-vs-historicalcheck and its "R0 source facts" check are no-ops when
observedis not supplied from outside; theydo not bind inside the freeze path. Either supply
observedfrom an external payload set in thefreeze path or label those two checks as diagnostic-only in the docstring and the protocol, so a
reader does not count them as enforcing.
ci_includes_zerois the wrong reason string for a wholly-negative interval. Theprotocol text is accurate ("CI lower bound ≤ 0, including wholly negative intervals") but the emitted
reason code reads as though the interval contained zero. Add a distinct reason (or a documented
spelling) for
upper < 0so the reason code and the prose agree.tests/test_dms003_score.pyerrorswhen NumPy is absent. Guard the bootstrap cases with
skipUnless(importlib.util.find_spec("numpy"), ...)so the skip is a visible decision in the runreport instead of a red suite. CI installs NumPy, so this is about local and peer runs.
--self-testshould exercise the production--verifypath. It currently drives thecomparator directly. Route at least one planted mutation through
dms003_freeze.py --verifyon atemporary manifest, so the self-test proves the entry point a reviewer runs rather than the function
underneath it.
Independent of these, the successor's review disposition is: B1 identity half and B2 token-matched LM
control remain open (
--require-readyexits 1 naming four blockers), R4 runtime acceptance of acriteria-free action head stays declared unverified, and Qwen's historical checkpoint revision is
unresolved.
Sixth item, found after the five were merged: review 179 on #80 showed the stated reason for the four per-transition checks was wrong for three of them. Fixed on PR #81; this issue closes when #81 merges.
Implemented on PR #80, head
5b2e77967f6e0094851ff43015bea58fb424bd68, based onmainafter #78 landed. All four required CIcontexts are green on that head, including the successor job that now runs the
--verifyself-test.Mapping to the five boxes:
adapter_provenancein the manifest plus the 225/75 split inthe protocol, plan and experiment file. Taken as the machine-readable split rather than prose only.
EXTERNAL_ONLY_CHECKSnames the two, the manifest publishes
checks.bind_only_with_external_observationandchecks.self_consistency_only_in_freeze_path: true, and a test proves both fire on tamperedexternally-supplied rows while being invisible in the freeze path. I did not take the "supply
observedin the freeze path" branch: the only rows available there are the builder's own, sofeeding them back would relabel the same no-op as enforcement. The external half stays open until an
adapter captures the bytes it actually sent.
ci_includes_zero—upper < 0now emitsci_wholly_negative; the protocol names both spellings.--self-testexercises--verify— subprocess on a temporary manifest, intact (rc 0) and planted(rc 1 for an
R0/payload reason).Verification on the head: 28 dms003 unit tests, freeze
--verifyplus self-test (3978 field mutationsrefused), leakscan self-test with the legacy manifest,
check_docs, ledger--check, and the required--require-readyrefusal. The full suite's 65 failures on this host are byte-identical in a detachedworktree at
origin/main(missing torch), so this branch adds none.Boxes stay unticked until the PR merges. No model ran and no GPU was touched.
Merged. Review 179 (APPROVED) at
5b2e77967f6e0094851ff43015bea58fb424bd68, filed and read back from the API asstate=APPROVED, official=true, stale=false, dismissed=false, commit_id=5b2e779…. Merge landed as8791b671606703dfddc4319242071ad4005ac9c6at 2026-09-26T20:46:35Z, parents273fcc67(main) +5b2e779(the reviewed head), so the reviewed revision is the merge's second parent andrefs/heads/mainnow resolves to it. All four push-triggered contexts on the merge commit are green — the PR head'spull_requestcontexts are not evidence about post-mergemain, so I read the merge commit's own status and waited for it to finish rather than reporting the head's green.All five boxes are discharged; the boxes are yours to tick. The independent verification behind that, reproduced on the exact head:
--writereproduces the committed manifest byte-identically;--verifyrc 0;--require-readyrc 1 naming the four blockers.273fcc67reads 2 errors + 1 failure, so the guard closes a real defect, not a hypothetical one.payload(...,'R3')starts fromhistorical(...)and adds nothing at R3, sopayloads.py:150compares a value with itself, and:187comparesR0with a second call to the same function. Your "label it" branch is right — I enumerated every 64-hex key acrossresults/*/manifest.jsonand nothing committed records a digest of an adapter-boundary request body, so there is no external observation to supply today. The reconstruction's inputs are externally pinned (all 23 controls match, preset412bc51e…agreeing with the DMS-001 record), which is the strongest available anchoring.adapter_provenance/checksfields through the production--verify: 0 undetected — including editing only the note text and adding an extra key. The new fields are bound, not decorative.Three non-blocking items are recorded in review 179 and need no re-open of this change; two are worth fixing while the text is warm:
+192/−17is stale — the diff is +196/−18 across 10 files.unittestat base it is 2 errors + 1 failure (bothBootstrapTestsERROR;test_cli_outputs_json_and_scores_reference_when_requestedFAILs). Head is exactly as claimed.configs/dms-003-protocol.md:87–88says the four transition checks bind "because each expected row is re-derived from the frozen historical object" — true forR3→R4only.R0→R1,R1→R2andR2→R3deriveexpectedfrom the neighbouring rung's payload (scripts/dms003_payloads.py:155). The conclusion holds (all four can fail; I showed the firing list), but the reason overstates independence for three of them.B1's adapter-equivalence half, B2's token-matched LM control, R4 runtime acceptance and Qwen's historical checkpoint revision remain open and unclosed by this merge. No model ran and no GPU was touched.
Both halves are on
mainnow, verified from the server rather than from the reports.8791b671606703dfddc4319242071ad4005ac9c6(merged_by vesper; review 179 APPROVED at5b2e77967f6e0094851ff43015bea58fb424bd68; all four post-merge push contexts green on the merge commit).376cee6f293d0129bc710ca9192677be6567433b(merged_by vesper; review 180 APPROVED ata23439c0d7a3d12222a45a2ad919b5fe7bd248cb; all four post-merge push contexts green).Closing as complete: five boxes ticked above, plus the sixth item the review of #80 found.
mainnowcarries the corrected taxonomy (
NEIGHBOUR_DERIVED_CHECKS/HISTORICAL_DERIVED_CHECK,checks.expectation_source) and no longer states the wrong reason anywhere.Two non-blocking items from review 180 are filed separately rather than smuggled into this closure:
the committed tamper test covers
laya:enonly, andexpectation_sourcemixes a list with a scalar.Nothing here closes B1's adapter-equivalence half, B2's token-matched LM control, R4 runtime acceptance
or Qwen's historical revision.