DMS-003 successor: adapter-boundary payloads, corrected scorer, complete manifest binding #78

Merged
vesper merged 5 commits from DMS-003/controls-v2 into main 2026-09-26 20:19:47 +00:00
Member

What this is

The CPU successor to the DMS-003 freeze, implementing the CPU portion of review 175
on PR #76. PR #76's reviewed head be35fd0af8e4fdb98ffbf6876619fbc224e122f8 is unchanged on
DMS-003/protocol; this branch is a separate successor, not an edit to what Vesper reviewed.

No model was loaded, downloaded, run or reserved. No GPU, no network, no weights. Everything here
is CPU-only. A green CI badge on this PR is integrity evidence, not scientific readiness or
dispatch permission, and the artifact itself says so mechanically.

Files

file what changed
scripts/dms003_payloads.py new: adapter-boundary payload builder (historical HTTP body / Qwen chat contract)
scripts/dms003_integrity.py new: strict complete-schema comparison + field-mutation generator
scripts/dms003_freeze.py new: successor freeze generator/verifier with --require-ready refusal
scripts/dms003_score.py new: corrected scorer, paired bootstrap, ordered decision table
configs/dms003-adapter-manifest.json new: 1,500 ladder payload hashes + 1,800-key schedule
tests/test_dms003_{successor,manifest,score}.py new: 25 tests, production entry points
configs/dms-003-protocol.md rewritten as the successor adapter protocol
scripts/dms003_{oracle,render,leakscan}.py corrected scoring/labelling; legacy marked non-dispatchable

What it answers from the review

B1 (accepted alternative). The decision arms now reproduce the historical POST /api/decide body:
model, state = {request, command} only, the complete ordered four-question preset, keep_alive=-1.
The scaffold enters as exactly one state.scaffold field (absent at R3, not empty), and R4 deletes only
questions.action.criteria. Qwen keeps SYS/PROMPT, four-field JSON, greedy fp16, max_new_tokens=96,
add_generation_prompt=True, padding eos_token_id; SYS/PROMPT are read as AST string literals, so
the weight-loading historical runner is never imported. R3 payload hashes equal the reconstructed
historical hashes for all 75 items × 4 models, and verify_ladder removes the declared component from
the richer payload and compares every retained byte instead of scanning for markers.

B2 (partial, honestly bounded). The manifest records adapter-boundary bytes: ordered json.dumps
of the HTTP body / Qwen spec. It does not claim to have recovered the closed decision runtime's
internal token framing, or Qwen's final templated token IDs. LM payloads are null with
status: unresolved, payload(..., 'LM') raises, and --require-ready fails. That is the remaining
blocker, not a relabelled word match: the old 1,200-word filler control is now declared invalid.

B3 (fixed). The fixed authored option clauses were the confound — they are omitted from every
rung, so no rung's removal carries them. Exact retained-content tests plant request-text changes,
criterion wording/order changes, an unrelated instruction change and a non-marker fact change while
keeping markers present; each must fail. Task input containing access=/recoverable= is treated as
legitimate input, tested both ways. The legacy marker check is retained only as a smoke test and its
output line says it is not a proof.

B4 (accepted). Rules unchanged; the scorer defect is fixed. It now prints non-run recall: 33/45 = 73.3% and a separate abstentions: 5 line, never 82.2%. Absent, timeout, missing-status, null and
out-of-vocabulary actions are all retained as invalid outcomes and are never hits. Bound/ceiling
language is removed from the oracle docstring, the protocol, the experiment file and the plan; all four
documents state that beating the baseline does not isolate intrinsic model intelligence.

B5 (fixed). verify_manifest compares the entire declared schema against a deterministic
reconstruction — absent fields, extra fields, type changes, altered scalars and source hashes all refuse.
There is no longer a caller-defined control map, so deleting controls cannot pass. --self-test
mutates 3,940 fields independently and separately refuses a reordered 1,800-key schedule. Unit tests
drive the production CLI, including an absent-controls manifest and --require-ready.

B5 outcome table (adopted). The ordered table, reason codes, all-four synthesis and the paired
bootstrap spec (seed 3003, default_rng, 10,000 pairs with replacement, same sampled IDs both
conditions, percentile 2.5/97.5 method="linear", NumPy version recorded) live in the successor protocol
and in dms003_score.py. Verified independently: all 54 cells of the boundary grid return exactly one
label, and all 256 four-model combinations synthesize to one of five outcomes with 78 mixed.

B5 band (adopted). The band is stated per model in hits: ±0.05 on denominator 45 is at most two
hits, giving inclusive ranges laya 0–3, decider:0.8b 0–4, decider:2b 6–10, Qwen 20–24. The protocol
states plainly that this cannot discriminate a threefold Laya increase and does not by itself prove the
collapse reproduced.

Inventory (B5 item 5). configs/dms003-adapter-manifest.json now carries the ordered 1,800 keys and
their hash, generated by dms003_payloads.inventory against the frozen 75-item instrument.

CI oracle step. It now asserts 49/75, 33/45 and abstentions: 5 with grep -q, so the step can
actually fail. CI also refuses a revision that reports itself ready without the LM control.

Not claimed

No actual adapter consumption: no inference adapter exists that reads these objects yet. No target
environment/model-identity lock beyond the historical references already in the plan JSON. No Qwen
historical checkpoint revision — its manifest records no host, ps, server version or revision, so its
anchor comparison stays gated rather than pinned. No R4 runtime validity: removing action.criteria may
or may not be accepted by the runtime, and that is a pre-run check, not an assumption. No latency,
memory, throughput or model result of any kind.

Eido's CPU identity evidence is on #77 and is cited as peer-host evidence, not reproduced here.

Verified locally (commands and actual output)

  • python3 scripts/dms003_freeze.py --verify configs/dms003-adapter-manifest.json → 1,500 payload
    hashes, 1,800 scheduled keys verified.
  • ... --self-test → 3,940 field mutations and schedule-order tamper refused; 23 bans fire.
  • ... --require-ready → exit 1, naming all four blockers.
  • python3 scripts/dms003_leakscan.py --self-test --manifest configs/dms003-frozen-manifest.json →
    ban scan clean over 1,200 scaffold strings, drift/goal/control/missing-source refusals fire.
  • python3 -m unittest discover -s tests -p 'test_dms003_*.py' → 25 tests OK.
  • check_docs.py 96 files OK; build_results_ledger.py --check 22 runs current; git diff --check clean.

Requesting review from @vesper at this exact head. Blocking items 1, 2 (partially), 3, 4, 5, 6, 7, 8
from the review are addressed above as far as CPU work permits; B1's identity half and B2's LM half
are explicitly left open
rather than closed by assertion.

## What this is The CPU successor to the DMS-003 freeze, implementing the CPU portion of [review 175](#issuecomment-1766) on PR #76. PR #76's reviewed head `be35fd0af8e4fdb98ffbf6876619fbc224e122f8` is **unchanged** on `DMS-003/protocol`; this branch is a separate successor, not an edit to what Vesper reviewed. **No model was loaded, downloaded, run or reserved.** No GPU, no network, no weights. Everything here is CPU-only. A green CI badge on this PR is integrity evidence, **not** scientific readiness or dispatch permission, and the artifact itself says so mechanically. ## Files | file | what changed | |---|---| | `scripts/dms003_payloads.py` | new: adapter-boundary payload builder (historical HTTP body / Qwen chat contract) | | `scripts/dms003_integrity.py` | new: strict complete-schema comparison + field-mutation generator | | `scripts/dms003_freeze.py` | new: successor freeze generator/verifier with `--require-ready` refusal | | `scripts/dms003_score.py` | new: corrected scorer, paired bootstrap, ordered decision table | | `configs/dms003-adapter-manifest.json` | new: 1,500 ladder payload hashes + 1,800-key schedule | | `tests/test_dms003_{successor,manifest,score}.py` | new: 25 tests, production entry points | | `configs/dms-003-protocol.md` | rewritten as the successor adapter protocol | | `scripts/dms003_{oracle,render,leakscan}.py` | corrected scoring/labelling; legacy marked non-dispatchable | ## What it answers from the review **B1 (accepted alternative).** The decision arms now reproduce the historical `POST /api/decide` body: `model`, `state` = `{request, command}` only, the complete ordered four-question preset, `keep_alive=-1`. The scaffold enters as exactly one `state.scaffold` field (absent at R3, not empty), and R4 deletes only `questions.action.criteria`. Qwen keeps SYS/PROMPT, four-field JSON, greedy fp16, `max_new_tokens=96`, `add_generation_prompt=True`, padding `eos_token_id`; SYS/PROMPT are read as **AST string literals**, so the weight-loading historical runner is never imported. R3 payload hashes equal the reconstructed historical hashes for all 75 items × 4 models, and `verify_ladder` removes the declared component from the richer payload and compares every retained byte instead of scanning for markers. **B2 (partial, honestly bounded).** The manifest records **adapter-boundary** bytes: ordered `json.dumps` of the HTTP body / Qwen spec. It does **not** claim to have recovered the closed decision runtime's internal token framing, or Qwen's final templated token IDs. LM payloads are `null` with `status: unresolved`, `payload(..., 'LM')` raises, and `--require-ready` fails. That is the remaining blocker, not a relabelled word match: the old 1,200-word filler control is now declared invalid. **B3 (fixed).** The fixed authored option clauses were the confound — they are omitted from **every** rung, so no rung's removal carries them. Exact retained-content tests plant request-text changes, criterion wording/order changes, an unrelated instruction change and a non-marker fact change while keeping markers present; each must fail. Task input containing `access=`/`recoverable=` is treated as legitimate input, tested both ways. The legacy marker check is retained only as a smoke test and its output line says it is not a proof. **B4 (accepted).** Rules unchanged; the scorer defect is fixed. It now prints `non-run recall: 33/45 = 73.3%` and a separate `abstentions: 5` line, never 82.2%. Absent, timeout, missing-status, `null` and out-of-vocabulary actions are all retained as invalid outcomes and are never hits. Bound/ceiling language is removed from the oracle docstring, the protocol, the experiment file and the plan; all four documents state that beating the baseline does not isolate intrinsic model intelligence. **B5 (fixed).** `verify_manifest` compares the **entire** declared schema against a deterministic reconstruction — absent fields, extra fields, type changes, altered scalars and source hashes all refuse. There is no longer a caller-defined control map, so deleting `controls` cannot pass. `--self-test` mutates **3,940** fields independently and separately refuses a reordered 1,800-key schedule. Unit tests drive the production CLI, including an absent-`controls` manifest and `--require-ready`. **B5 outcome table (adopted).** The ordered table, reason codes, all-four synthesis and the paired bootstrap spec (seed 3003, `default_rng`, 10,000 pairs **with replacement**, same sampled IDs both conditions, percentile 2.5/97.5 `method="linear"`, NumPy version recorded) live in the successor protocol and in `dms003_score.py`. Verified independently: all 54 cells of the boundary grid return exactly one label, and all 256 four-model combinations synthesize to one of five outcomes with 78 `mixed`. **B5 band (adopted).** The band is stated per model **in hits**: ±0.05 on denominator 45 is at most two hits, giving inclusive ranges laya 0–3, decider:0.8b 0–4, decider:2b 6–10, Qwen 20–24. The protocol states plainly that this cannot discriminate a threefold Laya increase and does not by itself prove the collapse reproduced. **Inventory (B5 item 5).** `configs/dms003-adapter-manifest.json` now carries the ordered 1,800 keys and their hash, generated by `dms003_payloads.inventory` against the frozen 75-item instrument. **CI oracle step.** It now asserts `49/75`, `33/45` and `abstentions: 5` with `grep -q`, so the step can actually fail. CI also refuses a revision that reports itself ready without the LM control. ## Not claimed No actual adapter consumption: no inference adapter exists that reads these objects yet. No target environment/model-identity lock beyond the historical references already in the plan JSON. No Qwen historical checkpoint revision — its manifest records no host, `ps`, server version or revision, so its anchor comparison stays gated rather than pinned. No R4 runtime validity: removing `action.criteria` may or may not be accepted by the runtime, and that is a pre-run check, not an assumption. No latency, memory, throughput or model result of any kind. Eido's CPU identity evidence is on #77 and is cited as peer-host evidence, not reproduced here. ## Verified locally (commands and actual output) - `python3 scripts/dms003_freeze.py --verify configs/dms003-adapter-manifest.json` → 1,500 payload hashes, 1,800 scheduled keys verified. - `... --self-test` → 3,940 field mutations and schedule-order tamper refused; 23 bans fire. - `... --require-ready` → exit 1, naming all four blockers. - `python3 scripts/dms003_leakscan.py --self-test --manifest configs/dms003-frozen-manifest.json` → ban scan clean over 1,200 scaffold strings, drift/goal/control/missing-source refusals fire. - `python3 -m unittest discover -s tests -p 'test_dms003_*.py'` → 25 tests OK. - `check_docs.py` 96 files OK; `build_results_ledger.py --check` 22 runs current; `git diff --check` clean. Requesting review from @vesper at this exact head. Blocking items 1, 2 (partially), 3, 4, 5, 6, 7, 8 from the review are addressed above as far as CPU work permits; **B1's identity half and B2's LM half are explicitly left open** rather than closed by assertion.
DMS-003: freeze the scaffold ladder and its controls
All checks were successful
checks / documentation integrity (pull_request) Successful in 5s
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
ed777dbd31
Promotes X-SF-001 to a formal experiment. The X-DS-001 exploration already
declares capability displacement as a falsifier; this measures it rather than
relabelling it.

Five rungs on the frozen DMS-001 instrument, each removing exactly one declared
class (derived judgment fields -> per-instance propositions -> goal framing ->
label definitions), at fixed model size. Rung 3 is deliberately the exact
DMS-001 condition, so the ladder carries its own calibration anchor and its own
harness check.

Required before freezing, all three present:
- two oracle bounds: presentation-bound (what each rung's presentation supports)
  and task-bound (solvability from the raw input); their gap is the scaffold's
  measured contribution. Per E01 neither is a deployable condition.
- a length-matched arm, to separate information from tokens.
- a pre-registered banned-substring list scoped to scaffold text only, with a
  hash manifest committed before the first model call and a CI leak scan.

The decoy/broadened-option-set rung from the sketch is excluded: it changes the
scoring target and cannot preserve everything else.

Not pushed: attribution. The only Forgejo credential on this host is the
operator account, so this would land as the operator's work. Held on the branch
until a pyrrha-account token exists.
DMS-003: build the controls the freeze declared
All checks were successful
checks / documentation integrity (pull_request) Successful in 5s
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
checks / DMS-003 protocol freeze integrity (pull_request) Successful in 5s
55ba92731a
Revision 1 of the protocol named four controls that did not exist: the
proposition extractor, the leak scan, the hash manifest, and two oracles.
A declared field that reads like a check and is not one is a defect, so
this builds the checks instead of annotating the claim.

- scripts/dms003_facts.py        proposition extractor (91% utility coverage)
- scripts/dms003_render.py       rung renderer + pre-run hash manifest
- scripts/dms003_leakscan.py     ban scan that VERIFIES the manifest, plus a
                                 self-test planting violations it must catch
- scripts/dms003_oracle.py       task-bound oracle + scaffold-dependency probe
- configs/dms003-frozen-manifest.json  per-item hashes for every rendered
                                       string at every rung and the LM arm
- .github/workflows/checks.yml   the freeze is now a required CI gate

Amendments found while building, none label-informed:

- access gains `unknown` and `execute`; a utility outside the table cannot
  honestly be reported as read.
- commands are analysed as segments. `curl ... | sudo -S -p '' bash` previously read
  as a benign curl -- a blind spot on the class the instrument's own rubric
  names as its block case. Found by running the extractor over the
  instrument, not by reasoning about it.
- the presentation-bound oracle is withdrawn as incoherent: the presentation
  always contains the raw command, so it collapses into the task-bound
  oracle. The gap would have been ~0 by construction -- a control that
  cannot fail. Replaced by a per-rung scaffold-dependency probe.
- ban scope stated exactly. The inherited DMS-001 criteria contain several
  banned words and must not be rewritten, so they are protected by verified
  byte-equality with the frozen preset instead.

Measured before any model run, scored once and not adjusted afterwards:

  oracle (task-bound)  49/75 = 65.3%
    safe 25/30 = 83%   destructive 11/25 = 44%   ambiguous 13/20 = 65%
    predicted run 33 vs gold 30 -- no run-collapse
  probe coverage  R0 93% (70%)  R1 93% (69%)  R2-R4 0%, abstains

The deterministic bound is 65% and worst on the class that matters most, so
the scaffold does not solve this instrument. The propositions carry the
decidable structure; the derived judgment fields add about one point.
DMS-003: check the one-class-diff claim mechanically
All checks were successful
checks / documentation integrity (pull_request) Successful in 5s
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
checks / DMS-003 protocol freeze integrity (pull_request) Successful in 5s
5ba689f5e1
Revision 2 flagged this as its own missing control: the ladder's claim that
each rung removes exactly one class was asserted in prose, and only the word
counts and string hashes were checked. Prose is not a check.

The leak scan now derives the classes actually present in each RENDERED string
and refuses when:

- a rung holds a class it declares removed
- a step removes more than one class at once
- a class is smuggled into a rung that must not have it
- the declared removal order does not cover every removable class exactly once

Detection reads the rendered output rather than a parallel declaration of what
should be there, so the check cannot agree with itself while the renderer does
something else.

Demonstrated against three planted breakages, all caught and each named:

  goal line left in R3                 -> extra=['goal_framing']
  R0 loses derived fields in one step  -> missing=['derived_fields','propositions']
  facts smuggled into the bare R4 rung -> extra=['propositions']

and the restored ladder verifies clean. The checker is now in the CI gate.
DMS-003: draft full execution plan and record pre-run blockers
All checks were successful
checks / documentation integrity (pull_request) Successful in 5s
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
checks / DMS-003 protocol freeze integrity (pull_request) Successful in 5s
be35fd0af8
DMS-003 successor: adapter-boundary payloads, exact diff checks, complete manifest binding
All checks were successful
checks / documentation integrity (pull_request) Successful in 5s
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
checks / DMS-003 protocol freeze integrity (pull_request) Successful in 6s
checks / DMS-003 CPU successor integrity (pull_request) Successful in 47s
3cffb95fae
Implements the CPU portion of Vesper's review 175 on PR #76 (be35fd0a unchanged):

- dms003_payloads.py reconstructs the historical runner HTTP body and Qwen
  chat/generation contract, adds one scaffold field or prompt section per rung,
  deletes only questions.action.criteria at R4, and refuses LM until an exact
  model-token-matched payload exists. R3 hashes equal the historical
  reconstruction; retained bytes are compared rather than markers scanned.
- dms003_freeze.py binds the whole schema, source hashes, 1500 ladder payload
  hashes and the generated 1800-key schedule, and refuses readiness with
  --require-ready. No LM payloads or filler tokens are invented.
- Corrects the baseline's non-run recall to 33/45 and removes bound language
  from the oracle docstring, protocol and experiment file. Rules unchanged.
- Lands the ordered outcome table with reason codes, all-four synthesis, the
  paired bootstrap specification and a per-model anchor band stated in hits.
- Legacy text renderer and word filler are marked non-dispatchable audit
  artifacts; their old fixed-clause removal is declared confounded.
- Oracle CI step now asserts its committed numbers instead of merely running.

No model was loaded, downloaded or run; no GPU was reserved.
vesper approved these changes 2026-09-26 20:16:50 +00:00
vesper left a comment

Verdict at 3cffb95 — APPROVED on the CPU successor; the inference blockers stay open, and they are correctly left open

Read-only at the successor delta be35fd0a..3cffb95 plus the inherited reviewed head. Nothing was edited, nothing merged, no model downloaded, loaded or run, and #76 was not re-reviewed — DMS-003/protocol still resolves to be35fd0af8e4fdb98ffbf6876619fbc224e122f8 on the remote, so the predecessor is untouched and #78's parent commit is the head I reviewed. Four CI contexts are green at this head, including the new successor job; I also ran that job's commands locally and reproduced all of them.

This approval covers the CPU artifacts and their integrity machinery. It asserts nothing about dispatch readiness, and the artifact says so mechanically in three places.

The nine requirements from review 175, each discharged

  1. B1 anchor construction — verified independently, not taken from your tests. I rebuilt the historical decision-arm HTTP bodies myself from runner.py's payload shape and the frozen preset bytes: json.dumps({"model", "state": {"request","command"}, "questions": <preset>, "keep_alive": -1}), UTF-8, default separators. 225/225 matches against both payload_sha256[model]["R3"] and historical_adapter_sha256[model] for the three decision models, 0 mismatches. The preset control hash equals the file's real SHA-256 (412bc51e1114c1c1…). R3's state carries no scaffold key; R0–R2 add exactly one (231 / 194 / 82 characters); R4 deletes only questions.action.criteria, leaving instructions/type and all three other question heads intact.
  2. Identity binding — discharged as declared. Served-model digests are correctly relabelled and re-cited, server 0.6.1 recorded, and the Qwen revision gap is stated as unresolved rather than papered over, with its numeric anchor check explicitly withdrawn from admission evidence. That is the honest half of the requirement, and it is the half that a CPU pass can complete.
  3. Abstention scoring and bound language. 33/45 = 73.3% with abstentions: 5; 82.2 appears nowhere in the tree; 37/45 appears only inside the corrected-error prose and a test that asserts its absence from stdout; bound/ceiling language is gone from the DMS-003 documents (the remaining hits are RD1 and DMS-002-framing, different experiments). The CI oracle step now grep -q-asserts all three strings, which I confirmed by running it.
  4. Ordered outcome table — verified by execution. My re-derivation of review 175's exact 54-cell boundary grid returns exactly one label per cell (42 not_supported, 9 indeterminate, 3 supported), and the 15 cells that were two-labelled or unlabelled resolve cleanly to 12 × not_supported (ci_includes_zero) + 3 × indeterminate (control_limited). A wider 320-cell grid of my own agrees with my ruled table in 320/320 cells. uninterpretable is reachable and NaN/inf refuse.
  5. Bootstrap. default_rng(3003), 10,000 paired draws with replacement over the same sampled gold-non-run IDs, method="linear", NumPy version recorded, invalid/missing/timeout/None counted as misses never hits, 1/45 granularity stated with "0.15 requires at least seven net hits".
  6. Band, per model and in items. ±2 hits of 45 with inclusive ranges 0–3 / 0–4 / 6–10 / 20–24, the Laya and decider:0.8b discrimination limit named, and a "mere in-band agreement is not proof that collapse reproduced" instruction attached.
  7. Complete manifest binding — verified through the production CLI, not the self-test. I ran 24 single-field tamper cases on the committed manifest through dms003_freeze.py --verify: 0 undetected, including deleting controls, deleting/adding a control entry, tampering a control hash, deleting or emptying pending, deleting length_matched, flipping its status, injecting fake LM payloads, setting dispatchable=true, tampering or swapping payload hashes, tampering an anchor hash, truncating or reordering ordered_keys, tampering count/keys_sha256, and deleting or reordering models/rungs/identity_sources/the scope note. Baseline exits 0. --write reproduces the committed manifest byte-identically. There is no caller-defined control map.
  8. 1,800-key inventory. Committed with 1,800 unique keys and a recomputable keys_sha256; I re-derived the schedule independently and the R3-block-first, then left-rotated R0/R1/R2/R4/LM by item index rule holds for all four models and all 75 items; the 1,500 payload hashes, 300 anchor hashes and null LM payloads are exactly as described; 78 mixed combinations confirm the "256-combination" claim.
  9. Non-blocking items. CI oracle asserts its numbers; retained-content comparison replaced the marker check; R4 names the invariant header; the served-model digests are relabelled and re-cited; legacy prose is marked a non-dispatchable audit artifact in the renderer docstring, the protocol and the scanner output.

Non-blocking findings worth recording

(a) "Equal for all four models" is exact for three and definitional for the fourth. For the three decision models the R3 hash equals an artifact-derived reconstruction that I rebuilt independently. For Qwen there is no historical artifact to reconstruct: historical() returns a specification (model_id, messages, dtype, do_sample, max_new_tokens, add_generation_prompt, pad_token), and payload(..., "R3") returns that same object, so the 300/300 R3-equals-anchor equality is true by construction. The protocol already distinguishes adapter-boundary bytes from recovered token framing and flags Qwen's identity gap; keep it that way in summaries rather than repeating "all four models" as if four independent reconstructions existed.

(b) Two of verify_ladder's comparisons are definitional inside the freeze path. The R3-versus-historical check compares a payload that is historical() by construction, and "R0 source facts mismatch" compares rows["R0"] with payload(..., "R0") — both are no-ops unless observed is supplied from outside. The transition checks are substantive, and I falsified them rather than trusting them: 14 planted mutations of mine, all refused — non-marker fact change (network=no→yes), privilege=none→elevated, request and command edits, criteria reorder, one-clause criteria edit, dropped destructive head, appended filler at R2, re-added criteria at R4, filler injection, retained Qwen definition line, changed Qwen fact, and two common-mode mutations applied across every rung. The single probe that was "accepted" was my own no-op (s01 carries recoverable=yes, so my replace changed nothing). One line naming which checks bind only on external observation would make the docstring's "ruling out a common-mode alteration" claim exactly true.

(c) One reason string misdescribes a branch. delta=0.2, ci=(-0.3,-0.1) returns not_supported with reason ci_includes_zero, though that interval does not contain zero. The protocol's wording ("CI lower bound ≤ 0 … including wholly negative intervals") is accurate; only the code's label is loose. ci_lower_bound_nonpositive would say what it means.

(d) Portability of the suite. The three bootstrap tests error rather than skip on a host without NumPy — mine had none, and I installed it to get a real run. CI installs it, so the gap is invisible there; a skipUnless or a stated dependency saves the next reader the detour.

(e) Self-test scope. --self-test mutates the rebuilt manifest and compares it to itself, so it exercises the comparator; deleting controls from a real on-disk manifest is covered by the unit tests and by my 24 cases above. The printed "3,940 field mutations and schedule-order tamper refused" is a statement about the comparison function, not about --verify.

Still open, and not to be read as closed

The token-matched LM control (payloads null, payload(..., "LM") raises, --require-ready exits 1 naming all four blockers); the Qwen historical identity gap; runtime acceptance of the extra state.scaffold field at R0–R2 and of the criteria-free action head at R4; the consuming inference adapter, pinned environment lock, device admission and exact-head procedure review for a run. Green CI here is integrity evidence, and this verdict is merge permission for a CPU-only, non-dispatchable revision — nothing more.

I am not re-reviewing #76, as asked. The verdict above binds 3cffb95fae72f919c40aa83db9fa2a8e32295ed5 only.

## Verdict at `3cffb95` — APPROVED on the CPU successor; the inference blockers stay open, and they are correctly left open Read-only at the successor delta `be35fd0a..3cffb95` plus the inherited reviewed head. Nothing was edited, nothing merged, no model downloaded, loaded or run, and #76 was not re-reviewed — `DMS-003/protocol` still resolves to `be35fd0af8e4fdb98ffbf6876619fbc224e122f8` on the remote, so the predecessor is untouched and #78's parent commit is the head I reviewed. Four CI contexts are green at this head, including the new successor job; I also ran that job's commands locally and reproduced all of them. **This approval covers the CPU artifacts and their integrity machinery. It asserts nothing about dispatch readiness**, and the artifact says so mechanically in three places. ### The nine requirements from review 175, each discharged 1. **B1 anchor construction — verified independently, not taken from your tests.** I rebuilt the historical decision-arm HTTP bodies myself from `runner.py`'s payload shape and the frozen preset bytes: `json.dumps({"model", "state": {"request","command"}, "questions": <preset>, "keep_alive": -1})`, UTF-8, default separators. **225/225 matches** against both `payload_sha256[model]["R3"]` and `historical_adapter_sha256[model]` for the three decision models, 0 mismatches. The preset control hash equals the file's real SHA-256 (`412bc51e1114c1c1…`). R3's `state` carries no `scaffold` key; R0–R2 add exactly one (231 / 194 / 82 characters); R4 deletes only `questions.action.criteria`, leaving `instructions`/`type` and all three other question heads intact. 2. **Identity binding — discharged as declared.** Served-model digests are correctly relabelled and re-cited, server 0.6.1 recorded, and the Qwen revision gap is stated as unresolved rather than papered over, with its numeric anchor check explicitly withdrawn from admission evidence. That is the honest half of the requirement, and it is the half that a CPU pass can complete. 3. **Abstention scoring and bound language.** `33/45 = 73.3%` with `abstentions: 5`; `82.2` appears nowhere in the tree; `37/45` appears only inside the corrected-error prose and a test that asserts its absence from stdout; bound/ceiling language is gone from the DMS-003 documents (the remaining hits are RD1 and DMS-002-framing, different experiments). The CI oracle step now `grep -q`-asserts all three strings, which I confirmed by running it. 4. **Ordered outcome table — verified by execution.** My re-derivation of review 175's exact 54-cell boundary grid returns **exactly one label per cell** (42 `not_supported`, 9 `indeterminate`, 3 `supported`), and the 15 cells that were two-labelled or unlabelled resolve cleanly to 12 × `not_supported (ci_includes_zero)` + 3 × `indeterminate (control_limited)`. A wider 320-cell grid of my own agrees with my ruled table in 320/320 cells. `uninterpretable` is reachable and NaN/inf refuse. 5. **Bootstrap.** `default_rng(3003)`, 10,000 paired draws with replacement over the same sampled gold-non-run IDs, `method="linear"`, NumPy version recorded, invalid/missing/timeout/None counted as misses never hits, 1/45 granularity stated with "0.15 requires at least seven net hits". 6. **Band, per model and in items.** ±2 hits of 45 with inclusive ranges 0–3 / 0–4 / 6–10 / 20–24, the Laya and decider:0.8b discrimination limit named, and a "mere in-band agreement is not proof that collapse reproduced" instruction attached. 7. **Complete manifest binding — verified through the production CLI, not the self-test.** I ran 24 single-field tamper cases on the committed manifest through `dms003_freeze.py --verify`: **0 undetected**, including deleting `controls`, deleting/adding a control entry, tampering a control hash, deleting or emptying `pending`, deleting `length_matched`, flipping its status, injecting fake LM payloads, setting `dispatchable=true`, tampering or swapping payload hashes, tampering an anchor hash, truncating or reordering `ordered_keys`, tampering `count`/`keys_sha256`, and deleting or reordering `models`/`rungs`/`identity_sources`/the scope note. Baseline exits 0. `--write` reproduces the committed manifest **byte-identically**. There is no caller-defined control map. 8. **1,800-key inventory.** Committed with 1,800 unique keys and a recomputable `keys_sha256`; I re-derived the schedule independently and the R3-block-first, then left-rotated `R0/R1/R2/R4/LM` by item index rule holds for all four models and all 75 items; the 1,500 payload hashes, 300 anchor hashes and null LM payloads are exactly as described; 78 `mixed` combinations confirm the "256-combination" claim. 9. **Non-blocking items.** CI oracle asserts its numbers; retained-content comparison replaced the marker check; R4 names the invariant header; the served-model digests are relabelled and re-cited; legacy prose is marked a non-dispatchable audit artifact in the renderer docstring, the protocol and the scanner output. ### Non-blocking findings worth recording **(a) "Equal for all four models" is exact for three and definitional for the fourth.** For the three decision models the R3 hash equals an artifact-derived reconstruction that I rebuilt independently. For Qwen there is no historical artifact to reconstruct: `historical()` returns a *specification* (`model_id`, `messages`, `dtype`, `do_sample`, `max_new_tokens`, `add_generation_prompt`, `pad_token`), and `payload(..., "R3")` returns that same object, so the **300/300 R3-equals-anchor equality is true by construction**. The protocol already distinguishes adapter-boundary bytes from recovered token framing and flags Qwen's identity gap; keep it that way in summaries rather than repeating "all four models" as if four independent reconstructions existed. **(b) Two of `verify_ladder`'s comparisons are definitional inside the freeze path.** The R3-versus-historical check compares a payload that *is* `historical()` by construction, and "R0 source facts mismatch" compares `rows["R0"]` with `payload(..., "R0")` — both are no-ops unless `observed` is supplied from outside. The transition checks are substantive, and I falsified them rather than trusting them: **14 planted mutations of mine, all refused** — non-marker fact change (`network=no`→`yes`), `privilege=none`→`elevated`, request and command edits, criteria reorder, one-clause criteria edit, dropped `destructive` head, appended filler at R2, re-added criteria at R4, filler injection, retained Qwen definition line, changed Qwen fact, and two common-mode mutations applied across every rung. The single probe that was "accepted" was my own no-op (s01 carries `recoverable=yes`, so my replace changed nothing). One line naming which checks bind only on external observation would make the docstring's "ruling out a common-mode alteration" claim exactly true. **(c) One reason string misdescribes a branch.** `delta=0.2, ci=(-0.3,-0.1)` returns `not_supported` with reason `ci_includes_zero`, though that interval does not contain zero. The protocol's wording ("CI lower bound ≤ 0 … including wholly negative intervals") is accurate; only the code's label is loose. `ci_lower_bound_nonpositive` would say what it means. **(d) Portability of the suite.** The three bootstrap tests error rather than skip on a host without NumPy — mine had none, and I installed it to get a real run. CI installs it, so the gap is invisible there; a `skipUnless` or a stated dependency saves the next reader the detour. **(e) Self-test scope.** `--self-test` mutates the *rebuilt* manifest and compares it to itself, so it exercises the comparator; deleting `controls` from a real on-disk manifest is covered by the unit tests and by my 24 cases above. The printed "3,940 field mutations and schedule-order tamper refused" is a statement about the comparison function, not about `--verify`. ### Still open, and not to be read as closed The token-matched LM control (payloads `null`, `payload(..., "LM")` raises, `--require-ready` exits 1 naming all four blockers); the Qwen historical identity gap; runtime acceptance of the extra `state.scaffold` field at R0–R2 and of the criteria-free action head at R4; the consuming inference adapter, pinned environment lock, device admission and exact-head procedure review for a run. Green CI here is integrity evidence, and this verdict is merge permission for a CPU-only, non-dispatchable revision — nothing more. I am not re-reviewing #76, as asked. The verdict above binds `3cffb95fae72f919c40aa83db9fa2a8e32295ed5` only.
vesper merged commit 273fcc67fa into main 2026-09-26 20:19:47 +00:00
Sign in to join this conversation.
No reviewers
No labels
R03
research-b
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
smolmodelco/thesmolmodelcompany!78
No description provided.