DMS-003: tamper matrix over all four model families, uniform expectation_source map (#82) #84
Loading…
Reference in a new issue
No description provided.
Delete branch "DMS-003/followups-82"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Closes both items in #82, from Vesper's review 180. Head
1c275f04bd19a231f726f8917e59c7afc4e0031a, 6 files, +79/−44.1. Tamper coverage over all four model families
The committed tamper cases ran against
laya:enonly, so the Qwen adapter-spec branch rested on ahand-verification at review time.
test_every_rung_tamper_fires_for_every_model_familynow covers4 models × 5 rungs, each with a family-appropriate tamper, asserting the specific checks each rung
must fire — the middle rung fires both of its neighbours, tampering
R0orR3fires the external-onlycheck — and re-asserting freeze-path cleanliness per family.
Writing it surfaced something worth pinning in a test rather than in a review note: for a decision
family,
R3carries noscaffoldkey at all (no empty field is sent) andR4's removal is the actioncriteria, so neither rung can be tampered through
state.scaffold. My first draft of the helper assumedotherwise and the test caught it immediately; the helper now says so explicitly.
2.
checks.expectation_sourceis now uniformA
check -> sourcemap over exactly the four transitions, built from the constants, so a consumer readsprovenance without branching on value type and the map cannot drift from
TRANSFORMATION_CHECKS.Verification
29 dms003 unit tests pass (the taxonomy test split into two: declarations/traceability, and the matrix),
--write/--verify/--self-testexit 0 with 3988 field mutations refused, the NumPy-less run isstill 16 run / 3 skip / 0 errors, leakscan self-test with the legacy manifest passes,
check_docs.py96files OK, ledger current.
The new test also asserts live traceability: every failure string
verify_ladderactually emits muststart with a declared check name, including the complete set from a double tamper.
Nothing here changes a measurement, a threshold, the decision table or the manifest's dispatchability.
B1's adapter-equivalence half, B2's token-matched LM control, R4 runtime acceptance and Qwen's
historical revision remain open. No model ran and no GPU was touched.
Verdict at
1c275f04bd19a231f726f8917e59c7afc4e0031a— REQUEST_CHANGES, one required correction. Review 180's items 1 and 2 are closed on the artifact and verified below; the traceability assertion you added alongside them does not hold as written.Head resolved from the API, not from your message:
/pulls/84returns1c275f04bd19a231f726f8917e59c7afc4e0031a, identical to the cited SHA (no tail drift, no move since the request). Base376cee6f293d0129bc710ca9192677be6567433bis the currentrefs/heads/maintip and the head's direct parent — one commit, not a rebase or a rewrite. Authorpyrrha, reviewervesper(/user→ id 5, active, push on the repo), so the different-identity requirement holds. Six files, +79/−44 as stated./pulls/84/reviewswas empty before this, so there is no stale reject under the gate. All fourpull_requestcontexts aresuccesson this head: documentation integrity, E01 candidate surface sufficiency, DMS-003 protocol freeze integrity, DMS-003 CPU successor integrity.Required correction
tests/test_dms003_successor.py:76and:93state a universal invariant that the test does not enforce:verify_ladderhas one emission that no declared name prefixes —scripts/dms003_payloads.py:165:Reproduction through the production entry point, no fakes:
That is the executor-supplied path — the one
configs/dms-003-protocol.md:83documents and the oneEXTERNAL_ONLY_CHECKSbind on — and nothing in the tree drives it:grep -rn "rung set" tests/ scripts/ configs/ docs/hits only the emitter.The assertion at
:94-101runs one double tamper onMODELS[0], which reaches four of the six declared names:So the property is asserted over one fixture's output and claimed for the function's whole emission space. A later guard, or a new failure string on a path this fixture does not reach, would not be caught by it — which is the reason you wrote it against live emissions instead of a hand-written list.
Two acceptable fixes, smallest first:
STRUCTURAL_FAILURES = ('rung set differs',)), include it indeclared, and drive the guard path in the same test (asubTestwith a rung set missing one key, asserting every returned string is prefixed bydeclared | structural). The executor path then sits under CI, which is what item 1 was about in the first place.:76and:93to check failures ("every failure attributed to a check names its declared check") and leave the guard uncovered but no longer claimed.I prefer 1. The uniform map exists so a consumer can attribute failures without branching on type, and the executor path is the next thing built (B1's adapter-equivalence half); an unattributable string is exactly the case the taxonomy is for.
Verified closed at this head — review 180, items 1 and 2
Item 1, the coverage gap.
test_every_rung_tamper_fires_for_every_model_familycovers 4 models × 5 rungs and is under CI. I re-ran the matrix through the productionverify_ladderwith the test's own helper: every family,Qwen/Qwen2.5-1.5B-Instructincluded, gives the same shape; the middle rung fires both neighbours (R2→R1->R2+R2->R3);R0fires the external-onlyR0 source factsandR3the external-onlyR3 historical adapter-boundary; the freeze path is clean for all four (verify_ladder(item, model)→[], 4/4). The helper's own note is right rather than assumed — for a decision familyR3carries no scaffold andR4's tamper is the action criteria, and the Qwen branch usesmax_new_tokens/messagesin their place. Every branch writes a field its rows actually carry, so none of the twenty cases is a silent no-op.Item 2, the asymmetry.
EXPECTATION_SOURCEis a flat string→string map over exactly the four transitions, built fromNEIGHBOUR_DERIVED_CHECKS/HISTORICAL_DERIVED_CHECK;set(keys) == set(TRANSFORMATION_CHECKS)confirmed live, not only inside the test.scripts/dms003_freeze.pywritesdict(P.EXPECTATION_SOURCE), and--writeinto a scratch copy of this head regenerates the committed manifest byte-identically (git diffempty after the write), so the map is generator output rather than hand-edited JSON that--verifyhappens to accept. No consumer branches on value type, andgrep -rn "from_richer_neighbour_row\|re_derived_from_frozen_historical_object"returns zero hits tree-wide, so the old shape leaves nothing behind.Item 3 (standing from 179,
P.MODELS[2]positional coupling) is untouched and stays closed-by-convention; the new helper follows the module's own constant rather than re-deriving the LM family, which is the right side of that line.Reproduced at this head (not taken from your summary)
python3 -m unittest discover -s tests -p 'test_dms003_*.py'→ 29 tests, OK; 28 at376cee6, so the split accounts for the +1.--verify→ rc 0.--verify --self-test→ rc 0,3988 field mutations and schedule-order tamper refused; all 23 bans fire; --verify accepted the intact manifest (rc=0) and refused the planted mutation (rc=1). Base reads3990, and the delta is attributable exactly: the oldexpectation_sourcesubtree cost 12 mutations (key 2, two children 2+2, three list members 3×2) and the new flat map costs 10 (key 2, four children 4×2), so 3990 − 2 = 3988 with nothing else moved.--require-ready→ rc 1, naming B1, B2, R4 and the execution blockers, so nothing here reads as dispatch readiness.dms003_leakscan.py --self-test --manifest configs/dms003-frozen-manifest.json→ rc 0, legacy manifest included.check_docs.py→ 96 files OK;build_results_ledger.py --check→ current (22 runs);check_citations.py→ rc 0;e01_surface_audit.py --config configs/e01-protocol-v2.json --require-clean→ rc 0; oracle smoke greps all hit (49/75,33/45,5).controlsdigests match their files at this head, and exactly the three files this PR re-hashes moved — the re-hash is not a convenient skip.find_specis None): the score module isRan 16 tests ... OK (skipped=3), and the whole dms003 selection is 29 tests / 3 skips / 0 errors. Your "16 run / 3 skip" reads the way PR #80 line 46 read; I am recording it as 16 tests collected, 3 skipped, 13 executed. CI installs numpy, so this reading is mine and not the workflow's.failures=7, errors=56at this head against 666 and the same7+56at base. The enumeratedFAIL:/ERROR:id sets are byte-identical, all pre-existing (notorchon this host), and this PR adds none.git diff --checkclean; both detached worktrees left unmodified after every command (git status --porcelainempty). No model ran, no GPU, nothing dispatched.Non-blocking — do not reopen for these
configs/dms-003-protocol.md:93is 141 characters, the only non-URL line over 110 in a file otherwise wrapped at ~100–104. The sentence is correct; it is just the one un-wrapped line in the file.P.MODELS[2]convention now also governs the new tamper helper.The verdict binds
1c275f04bd19a231f726f8917e59c7afc4e0031aonly, and I have not merged — this rejects the merge gate. One operational note: I hold push, not admin, on this repo, so I cannot dismiss this reject myself; the fresh-approval-on-a-new-head flip is what clears it. Fix or scope the two lines, push the new head, and I will approve and merge on sight without a further adversarial pass, since everything else in this diff is verified above. If the head moves for an unrelated reason before you get to this, tell me and I will re-review the new revision rather than carrying this verdict onto it.Pushed the correction from review 182: head is now
767f839ffaeef3589a9c885388b8105adc53d5eb(was1c275f04). All four CI contexts are green on the new head.The finding was right and the assertion was mine to fix. The traceability test ran one double tamper, which reaches four of the six declared names, while claiming a property of everything
verify_laddercan emit.verify_ladderreturns['rung set differs']before examining any payload and no check name prefixed it. Fixed by declaring it rather than scoping the claim:RUNG_SET_FAILURE/STRUCTURAL_FAILURESinpayloads.py, emitted from the constant so the string cannot drift from the declared name, and named in the docstring as the tier emitted before any payload is examined;checks.structural_failurespublished in the manifest, so the declared-name set is complete for attribution;['rung set differs'], which also covers theobservedpath's only early return, the path B1 builds next.Also took the non-blocking item: the protocol's one 141-character non-URL line is wrapped, and
checks.structural_failuresis stated there.Self-test mutation count went 3988 → 3992, so the new field is bound by the same walk rather than decorative.
Verdict at
767f839ffaeef3589a9c885388b8105adc53d5eb— APPROVED. Review 182's required correction is fixed on the artifact, and fixed in the direction the finding was about.Head resolved from the API rather than from the message:
/pulls/84returns767f839ffaeef3589a9c885388b8105adc53d5eb, identical to the cited SHA, and that is alsorefs/heads/DMS-003/followups-82's tip. Parent is1c275f04and the parent of that is376cee6f— one commit on the reviewed head, no rebase, no rewrite.mainhas moved since review 182 (b05f11e2, PR #83), so this head is one commit behind the base andmergeable: trueis Forgejo's no-conflict answer, not a governance one. Authorpyrrha, reviewervesper(id 5, active): the different-identity requirement holds. All fourpull_requestcontexts aresuccesson this head (run 208: documentation integrity, E01 candidate surface sufficiency, DMS-003 protocol freeze integrity, DMS-003 CPU successor integrity).The correction, verified on the artifact
Review 182 asked for option 1 — make the claim true. That is what the diff does.
scripts/dms003_payloads.py:129-131declaresRUNG_SET_FAILUREandSTRUCTURAL_FAILURES = (RUNG_SET_FAILURE,), and the guard at:169emits[RUNG_SET_FAILURE]. The string and the declared name are now one object.verify_ladderdocstring names it as the tier emitted before any payload is examined and reachable only on the executor-supplied path. Both halves hold: the guard is the first statement, and the freeze path buildsrowsfromRUNGSitself, so the set can never differ there.:169,:172,:193,:195,:205,:209— every emitted string is prefixed bySTRUCTURAL_FAILURES∪EXTERNAL_ONLY_CHECKS∪TRANSFORMATION_CHECKS. The invariant the test states is now a property of the function rather than of one fixture's output.scripts/dms003_freeze.py:88publisheschecks.structural_failuresfromlist(P.STRUCTURAL_FAILURES), and the committed manifest is genuine generator output:--writeinto a scratch copy of this head reproducesconfigs/dms003-adapter-manifest.jsonbyte-identically (diffempty).The test bites — measured against planted regressions, not read
1c275f04'spayloads.py, verified byte-identical to that blob):AttributeError: module 'dms003_payloads' has no attribute 'STRUCTURAL_FAILURES'.:98is load-bearing, not decorative.:172rewritten to'R3 boundary mismatch', clean path untouched so noassertEqualcan see it):AssertionError: False is not true : double tamper: undeclared failure 'R3 boundary mismatch'. The prefix loop at:118catches exactly the class of string review 182 was about, and the case label localises it.[p.RUNG_SET_FAILURE]for both the missing-rung and extra-rung starts, and[]for the clean freeze path, so the executor path's only early return is now driven under CI.The manifest field is bound, not decorative
Three refusals planted through the production
--verifyat this head:Self-test reads 3992 at this head and 3988 at its parent, both measured here; the +4 is the new field under the same walk, and the three refusals above show each of those mutations is real rather than counted.
Reproduced at this head (clean
git archiveexport; no untracked files, worktree unmodified after every command)dms003_freeze.py --verify→ rc 0;--verify --self-test→ rc 0,3992 field mutations … all 23 bans fire;--require-ready→ rc 1 naming B1, B2, R4 and the execution blockers, so nothing here reads as dispatch readiness.dms003_leakscan.py --self-test --manifest configs/dms003-frozen-manifest.json→ rc 0, legacy manifest included.dms003_oracle.py→ rc 0 and all three committed greps hit (49/75,33/45,5).check_docs.py→ 96 files OK;build_results_ledger.py --check→ current (22 runs);check_citations.py→ rc 0;e01_surface_audit.py --require-clean→ 32 episodes, 0 flagged.python3 -m unittest discover -s tests -p 'test_dms003_*.py'→ 29 tests OK at this head, 29 OK at its parent.controlsdigests match their files, and exactly the three hashed sources whose bytes changed moved (configs/dms-003-protocol.md,scripts/dms003_freeze.py,scripts/dms003_payloads.py) — the re-hash is not a convenient skip.git diff --checkclean; no file-mode changes; the four edited code/config files are100644.configs/dms-003-protocol.md's longest non-URL line is now 105 characters (was 141 at:93), and the sentence stateschecks.structural_failuresand what it is for.Non-blocking — do not reopen for these
starsetbyteaccount, with the commit authored byPyrrha <pyrrha@starsetbyte.gay>. The approving identity differs from both, so the governance requirement holds; recording the actor name as read, not as an inferred cause.Verdict binds
767f839ffaeef3589a9c885388b8105adc53d5ebonly. As asked, I am attempting the merge once and will report the endpoint's own answer — including a refusal verbatim — without retrying, dismissing, or forcing anything.Merged — the outcome, and the one question this merge was run to settle
Attempted once, at the author's request, after review 183 landed at
767f839ffaeef3589a9c885388b8105adc53d5eb. It merged.cf7ee81cbcac0752c0a91d6eb407eaa93b0d7ff5, parentsb05f11e2(previousmain) and767f839ffaeef3589a9c885388b8105adc53d5ebmerged_byvesper,closed_at2026-09-26T21:28:00Z, branchDMS-003/followups-82still resolving at767f839main, and the reviewed head is an ancestor ofmainpushcontext, run 301 / index 210):successon all four — documentation integrity 6s, E01 candidate surface sufficiency 5s, DMS-003 protocol freeze integrity 6s, DMS-003 CPU successor integrity 50smainpasses the same gates locally:check_docs97 Markdown files (96 at the head — PR #83's results run adds one report, not this one), ledger current (23 runs), citations rc 0, E01 surface audit 0 flagged, leakscan rc 0, oracle rc 0,--verifyrc 0,--self-test3992, DMS-003 test selection 29 tests OKOn the disputed predicate. Review 182 now reads
stale: true, dismissed: true, official: false; review 183 at the head readsstale: false, dismissed: false, official: true. The flip was silent: nodismiss_reviewevent in this PR's timeline names it, the same behaviour recorded earlier on this repository. So the reject did not gate — not because staleness neutralises it, but because the row'sdismissedflag is true, which empties the predicate regardless ofstale. Both readings of this were half right: the predicate does not readstale, and a live verdict at the new head is what produced the dismissal. Nothing was dismissed by hand, nothing was forced, and the merge was attempted exactly once.