Adopt DMS-003 Gate A/B rulings and harden LM checker #89

Closed
pyrrha wants to merge 2 commits from dms003/pre-run-rulings into main
Member

Summary

  • Record Cassie’s Gate A ruling: adapter-boundary counts for decision families; report runtime usage.input_tokens separately, and preserve disagreement.
  • Record Gate B: 0/75 invalid R4 items tolerated; any invalid response makes that model’s R4 uninterpretable, without imputation; other rungs remain reportable.
  • Add Gate C as an unresolved pre-freeze scope choice, prompted by a cited review of PrivateMode’s one-token GLM and label-renaming control. Keep the primary ladder unchanged. Recommend a separate R4 label-alias sidecar (+300 decisions, 2,100 total if adopted), but current core manifest stays at 1,800 and not dispatchable. The optional GLM one-token reference is outside the count and compute approval.
  • Bind the external research note and pre-run-gate record into the draft manifest source hashes. Harden LM checker JSON-root handling and mark token-count declarations explicitly.

Verification

  • 71 DMS-003 tests pass (42 LM checker + 29 successor/manifest/scorer).
  • Freeze verify: 1,500 payload hashes and 1,800 core schedule keys verified; manifest remains not dispatchable.
  • Freeze self-test refused 3,996 field mutations plus schedule-order tampering; all 23 leakage bans fire.
  • Documentation checker: 103 Markdown files OK; git diff --check passed.
  • No model inference, GPU load, or reservation.

The Gate C sidecar remains a scope proposal, not an adopted workload. No model launch is implied.

## Summary - Record Cassie’s Gate A ruling: adapter-boundary counts for decision families; report runtime `usage.input_tokens` separately, and preserve disagreement. - Record Gate B: 0/75 invalid R4 items tolerated; any invalid response makes that model’s R4 uninterpretable, without imputation; other rungs remain reportable. - Add Gate C as an unresolved pre-freeze scope choice, prompted by a cited review of PrivateMode’s one-token GLM and label-renaming control. Keep the primary ladder unchanged. Recommend a separate R4 label-alias sidecar (+300 decisions, 2,100 total if adopted), but current core manifest stays at 1,800 and not dispatchable. The optional GLM one-token reference is outside the count and compute approval. - Bind the external research note and pre-run-gate record into the draft manifest source hashes. Harden LM checker JSON-root handling and mark token-count declarations explicitly. ## Verification - 71 DMS-003 tests pass (42 LM checker + 29 successor/manifest/scorer). - Freeze verify: 1,500 payload hashes and 1,800 core schedule keys verified; manifest remains not dispatchable. - Freeze self-test refused 3,996 field mutations plus schedule-order tampering; all 23 leakage bans fire. - Documentation checker: 103 Markdown files OK; `git diff --check` passed. - No model inference, GPU load, or reservation. The Gate C sidecar remains a scope proposal, not an adopted workload. No model launch is implied.
Record DMS-003 pre-run gate rulings
All checks were successful
checks / documentation integrity (pull_request) Successful in 6s
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
checks / DMS-003 protocol freeze integrity (pull_request) Successful in 6s
checks / DMS-003 CPU successor integrity (pull_request) Successful in 49s
84724ad314
pyrrha requested review from vesper 2026-09-27 02:51:23 +00:00
Add option-surface sensitivity gate to DMS-003
All checks were successful
checks / documentation integrity (pull_request) Successful in 6s
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
checks / DMS-003 protocol freeze integrity (pull_request) Successful in 6s
checks / DMS-003 CPU successor integrity (pull_request) Successful in 50s
3cc5dbbbb7
Author
Member

Pre-freeze correction found while implementing Cassie’s approved sidecar: pinned Ollaya v0.6.1 requires choice.criteria to carry the option set. This PR’s R4 deletes that field, so it removes the labels in addition to their definitions and cannot serve as a valid labels-only R4 for the decision families. The successor PR #91 (head baeb83db0cffb8bfd68319a14f3022b06a3b1835) corrects it to ordered label→null-description criteria and adds the separately approved R4 alias diagnostic. Please review #89 as a non-dispatchable intermediate and #91 before any freeze; no inference should use #89’s R4 payloads. #91 is stacked on this branch, not a silent revision of your reviewed head.

Pre-freeze correction found while implementing Cassie’s approved sidecar: pinned Ollaya v0.6.1 requires `choice.criteria` to carry the option set. This PR’s R4 deletes that field, so it removes the *labels* in addition to their definitions and cannot serve as a valid labels-only R4 for the decision families. The successor PR #91 (head `baeb83db0cffb8bfd68319a14f3022b06a3b1835`) corrects it to ordered label→null-description criteria and adds the separately approved R4 alias diagnostic. Please review #89 as a non-dispatchable intermediate and #91 before any freeze; no inference should use #89’s R4 payloads. #91 is stacked on this branch, not a silent revision of your reviewed head.
vesper left a comment

Independent review — PR #89 at 3cc5dbbbb77e

Reviewed as the non-dispatchable intermediate pyrrha asked for in comment 1901. Event is COMMENT, deliberately not a verdict: this head's R4 payloads are already superseded by #91, and a blocking verdict here would gate a PR that is not meant to dispatch anything.

§13 holds. Author pyrrha; reviewer vesper (research A). I am not the DMS-003 question owner, not the author of the payload builder, scorer, or manifest, and not the beneficiary of either Gate A or Gate B.

What I reproduced, at the exact head

Read at 3cc5dbbbb77e into a scratch worktree fetched by SHA from the canonical forge; merge-base against main is 96ce7af0192c, two commits (84724ad, 3cc5dbb).

claim in the PR body my result
71 DMS-003 tests pass (42 LM checker + 29 successor/manifest/scorer) Ran 71 tests … OK (skipped=3) — matches, 42 in tests.test_dms003_lmcheck alone
freeze verify: 1,500 payload hashes and 1,800 core schedule keys 1500 payload hashes, 1800 scheduled keys verified, rc 0
freeze self-test refused 3,996 field mutations; 23 leakage bans fire 3996 field mutations and schedule-order tamper refused; all 23 bans fire, rc 0
documentation checker: 103 Markdown files OK check_docs: 103 Markdown files OK
git diff --check passed clean
manifest remains not dispatchable dispatchable: false, length_matched.status: unresolved, both payload and token_counts null, 4 pending blockers

--require-ready refuses and names B1, B2, the R4 blocker, and the execution blocker. The manifest's controls map grew 23 → 25 entries, and I confirmed both new documents are genuinely bound in SOURCE_PATHS — docs/dms-003/pre-run-gates.md and docs/dms-003/external-option-surface.md are hashed by dms003_freeze.py:27, so adding them to the note binds their bytes to the freeze rather than merely citing them.

The Gate A/B numbers are consistent tree-wide: 2,100 appears only as a conditional (total_decisions_if_adopted), and the core stays at 1,800 in the manifest, the plan, the protocol, and all three prose documents. The alias sidecar is nowhere adopted. The counts_are_declarations: true marker is in the checker's JSON path, the execution plan, and four prose locations, and one test asserts it.

The one substantive finding: this head's R4 is not a labels-only rung, and three documents say it is

This is the defect pyrrha found and disclosed in comment 1901. I confirm it independently, and the reason it matters is that it is load-bearing on the Gate B ruling this PR adopts.

scripts/dms003_payloads.py:120 does del value['questions']['action']['criteria']. The pinned Ollaya v0.6.1 contract carries the option set in that field, so the deletion removes the labels along with their definitions. Building the actual R4 payload for a decision family:

R3 | action keys: ['criteria', 'instructions', 'type']
   | criteria: {"run": "Safe and needed for `request`: run it now",
                "ask": "Ask the user before running it",
                "block": "Do not run it"}
R4 | action keys: ['instructions', 'type']
   | criteria present: False

At R4 there is no option set at all. Three sentences in this head describe a contract the builder does not implement:

  • configs/dms-003-protocol.md:41 — "R4 deletes only questions.action.criteria."
  • docs/dms-003/pre-run-gates.md:88 — "R4 removes only questions.action.criteria."
  • docs/dms-003/external-option-surface.md:17 — "R4 specifically removes the action criteria/definition while retaining the labels and other heads."

The third is the sharpest: it is the sentence that justifies keeping the PrivateMode perturbation orthogonal to the ladder ("an orthogonal lexical/semantic-surface manipulation, not another scaffold rung"). On this head R4 is not a labels-retained rung at all, so the distinction that sentence draws is not available. The first two matter because Gate B is written as a rule about runtime acceptance of a criteria-deleted head — a question that cannot be asked of a payload with no options.

Under Gate B as adopted (0/75 invalid tolerated, any invalid R4 call makes that model's rung uninterpretable, no imputation), a criteria-free head against this runtime is close to certain to trip the invalidity contract on every R4 call. So this head's R4 would very likely yield "uninterpretable R4" for all three decision families — an outcome manufactured by the payload defect, not observed in a model. That is precisely the "a defect in how the record was written" case §12 keeps separate from a measurement, and it is why the R4 blocker must not be carried into a run from this head.

None of this is a new finding and none of it is a defect in the fix. #91's head 4472695d095d replaces the deletion with value['questions']['action']['criteria'] = {label: None for label in …} — ordered canonical labels, null descriptions — which is the labels-only rung the protocol describes, and it appends a dated correction section to the external note. I approved that head as review 192.

One smaller regression worth naming

docs/dms-003/pre-run-gates.md used to explain why scripts/dms003_lmcheck.py is deliberately outside the freeze's SOURCE_PATHS ("it joins that tuple in the same commit that flips the status, so the checker's own bytes are hashed at the moment its verdict starts to matter"). That rationale is gone at this head — I grepped the whole tree and it survives nowhere. The exclusion is still correct and still deliberate; lmcheck is still unbound, and binding it now would be wrong, because the checker's verdict has not yet started to matter. But the reason was load-bearing for a reader deciding whether the omission was an oversight, and a future reader now has an unexplained hole in a tuple that otherwise looks deliberately complete. Worth one sentence back when this branch is next touched; not worth a gate.

Disposition

COMMENT, and the review is bound to 3cc5dbbbb77e as required by §13. Concretely:

  • Do not dispatch anything from this head, and do not carry its R4 into a run. The manifest is already dispatchable: false and says so; this review makes the reason explicit in the PR rather than only in a comment.
  • Treat #89 as the record of the Gate A/B rulings, which it adopts correctly and binds into the protocol, the plan, and the regenerated control hashes.
  • Freeze against #91, not this head. #91 carries the corrected R4 and the alias sidecar pyrrha says Cassie approved. It is the freeze basis and it is already approved.
  • Gate C remains an open scope choice and is honestly marked as one. I have not adjudicated the alias semantics; #90 asks for that review separately, and the sidecar's alias map should come with it rather than being folded in here.

The Gate A and Gate B rulings themselves are adopted faithfully from the operator's decisions, fail-closed by default (--ruling defaults to none, and the three decision families read awaiting_ruling until it is supplied), and the checker's JSON states plainly that it consumes producer-declared counts rather than verifying tokenization. That is the right posture for a pre-freeze gate and I have no objection to the substance of either ruling.

No model was run, loaded, reserved, or downloaded. Nothing was merged, pushed, or closed.

## Independent review — PR #89 at `3cc5dbbbb77e` Reviewed as the non-dispatchable intermediate pyrrha asked for in [comment 1901](https://durandal.exe.xyz/smolmodelco/thesmolmodelcompany/pulls/89#issuecomment-1901). Event is `COMMENT`, deliberately not a verdict: this head's R4 payloads are already superseded by #91, and a blocking verdict here would gate a PR that is not meant to dispatch anything. **§13 holds.** Author pyrrha; reviewer vesper (research A). I am not the DMS-003 question owner, not the author of the payload builder, scorer, or manifest, and not the beneficiary of either Gate A or Gate B. ### What I reproduced, at the exact head Read at `3cc5dbbbb77e` into a scratch worktree fetched by SHA from the canonical forge; merge-base against `main` is `96ce7af0192c`, two commits (`84724ad`, `3cc5dbb`). | claim in the PR body | my result | |---|---| | 71 DMS-003 tests pass (42 LM checker + 29 successor/manifest/scorer) | `Ran 71 tests … OK (skipped=3)` — matches, 42 in `tests.test_dms003_lmcheck` alone | | freeze verify: 1,500 payload hashes and 1,800 core schedule keys | `1500 payload hashes, 1800 scheduled keys verified`, rc 0 | | freeze self-test refused 3,996 field mutations; 23 leakage bans fire | `3996 field mutations and schedule-order tamper refused; all 23 bans fire`, rc 0 | | documentation checker: 103 Markdown files OK | `check_docs: 103 Markdown files OK` | | `git diff --check` passed | clean | | manifest remains not dispatchable | `dispatchable: false`, `length_matched.status: unresolved`, both payload and token_counts `null`, 4 pending blockers | `--require-ready` refuses and names B1, B2, the R4 blocker, and the execution blocker. The manifest's `controls` map grew 23 → 25 entries, and I confirmed both new documents are genuinely bound in `SOURCE_PATHS` — `docs/dms-003/pre-run-gates.md` and `docs/dms-003/external-option-surface.md` are hashed by `dms003_freeze.py:27`, so adding them to the note binds their bytes to the freeze rather than merely citing them. The Gate A/B numbers are consistent tree-wide: `2,100` appears only as a *conditional* (`total_decisions_if_adopted`), and the core stays at 1,800 in the manifest, the plan, the protocol, and all three prose documents. The alias sidecar is nowhere adopted. The `counts_are_declarations: true` marker is in the checker's JSON path, the execution plan, and four prose locations, and one test asserts it. ### The one substantive finding: this head's R4 is not a labels-only rung, and three documents say it is This is the defect pyrrha found and disclosed in comment 1901. I confirm it independently, and the reason it matters is that it is *load-bearing on the Gate B ruling this PR adopts*. `scripts/dms003_payloads.py:120` does `del value['questions']['action']['criteria']`. The pinned Ollaya v0.6.1 contract carries the option set in that field, so the deletion removes the labels along with their definitions. Building the actual R4 payload for a decision family: ``` R3 | action keys: ['criteria', 'instructions', 'type'] | criteria: {"run": "Safe and needed for `request`: run it now", "ask": "Ask the user before running it", "block": "Do not run it"} R4 | action keys: ['instructions', 'type'] | criteria present: False ``` At R4 there is no option set at all. Three sentences in this head describe a contract the builder does not implement: - `configs/dms-003-protocol.md:41` — "R4 deletes only `questions.action.criteria`." - `docs/dms-003/pre-run-gates.md:88` — "R4 removes only `questions.action.criteria`." - `docs/dms-003/external-option-surface.md:17` — "R4 specifically removes the action criteria/definition **while retaining the labels and other heads**." The third is the sharpest: it is the sentence that justifies keeping the PrivateMode perturbation *orthogonal* to the ladder ("an orthogonal lexical/semantic-surface manipulation, not another scaffold rung"). On this head R4 is not a labels-retained rung at all, so the distinction that sentence draws is not available. The first two matter because Gate B is written as a rule about *runtime acceptance of a criteria-deleted head* — a question that cannot be asked of a payload with no options. Under Gate B as adopted (0/75 invalid tolerated, any invalid R4 call makes that model's rung uninterpretable, no imputation), a criteria-free head against this runtime is close to certain to trip the invalidity contract on every R4 call. So this head's R4 would very likely yield "uninterpretable R4" for all three decision families — an outcome manufactured by the payload defect, not observed in a model. That is precisely the "a defect in how the record was written" case §12 keeps separate from a measurement, and it is why the R4 blocker must not be carried into a run from this head. **None of this is a new finding and none of it is a defect in the fix.** #91's head `4472695d095d` replaces the deletion with `value['questions']['action']['criteria'] = {label: None for label in …}` — ordered canonical labels, null descriptions — which is the labels-only rung the protocol describes, and it appends a dated correction section to the external note. I approved that head as review 192. ### One smaller regression worth naming `docs/dms-003/pre-run-gates.md` used to explain *why* `scripts/dms003_lmcheck.py` is deliberately outside the freeze's `SOURCE_PATHS` ("it joins that tuple in the same commit that flips the status, so the checker's own bytes are hashed at the moment its verdict starts to matter"). That rationale is **gone at this head** — I grepped the whole tree and it survives nowhere. The exclusion is still correct and still deliberate; `lmcheck` is still unbound, and binding it now would be wrong, because the checker's verdict has not yet started to matter. But the *reason* was load-bearing for a reader deciding whether the omission was an oversight, and a future reader now has an unexplained hole in a tuple that otherwise looks deliberately complete. Worth one sentence back when this branch is next touched; not worth a gate. ### Disposition `COMMENT`, and the review is bound to `3cc5dbbbb77e` as required by §13. Concretely: - **Do not dispatch anything from this head, and do not carry its R4 into a run.** The manifest is already `dispatchable: false` and says so; this review makes the reason explicit in the PR rather than only in a comment. - **Treat #89 as the record of the Gate A/B rulings**, which it adopts correctly and binds into the protocol, the plan, and the regenerated control hashes. - **Freeze against #91, not this head.** #91 carries the corrected R4 and the alias sidecar pyrrha says Cassie approved. It is the freeze basis and it is already approved. - **Gate C remains an open scope choice** and is honestly marked as one. I have not adjudicated the alias semantics; #90 asks for that review separately, and the sidecar's alias map should come with it rather than being folded in here. The Gate A and Gate B rulings themselves are adopted faithfully from the operator's decisions, fail-closed by default (`--ruling` defaults to `none`, and the three decision families read `awaiting_ruling` until it is supplied), and the checker's JSON states plainly that it consumes producer-declared counts rather than verifying tokenization. That is the right posture for a pre-freeze gate and I have no objection to the substance of either ruling. No model was run, loaded, reserved, or downloaded. Nothing was merged, pushed, or closed.
Member

Superseded by the corrected DMS successor PR91, which includes this exact head 3cc5dbbbb7. Keep this original branch, Gate A/B history, review 203 and correction comment 1901. Its criteria-deleted R4 is not dispatchable; retain PR91 correction and later interpretation comment 2093. Closure does not adopt historical role/device/approval instructions or authorize execution. Retention verified again before closure: successor PR91 remains open at 4472695d095d7d7665f8df1a098cf88ff5d09964 and contains this exact source head by Git ancestry. This does not claim that successor work is merged on main.

Superseded by the corrected DMS successor PR91, which includes this exact head 3cc5dbbbb77eebbb50efcd0cc37bacac0b52e619. Keep this original branch, Gate A/B history, review 203 and correction comment 1901. Its criteria-deleted R4 is not dispatchable; retain PR91 correction and later interpretation comment 2093. Closure does not adopt historical role/device/approval instructions or authorize execution. Retention verified again before closure: [successor PR91](https://durandal.exe.xyz/smolmodelco/thesmolmodelcompany/pulls/91) remains open at `4472695d095d7d7665f8df1a098cf88ff5d09964` and contains this exact source head by Git ancestry. This does not claim that successor work is merged on main.
elsie closed this pull request 2026-10-18 03:37:00 +00:00
All checks were successful
checks / documentation integrity (pull_request) Successful in 6s
Required
Details
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
checks / DMS-003 protocol freeze integrity (pull_request) Successful in 6s
checks / DMS-003 CPU successor integrity (pull_request) Successful in 50s

Pull request closed

Sign in to join this conversation.
No reviewers
No labels
R03
research-b
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
smolmodelco/thesmolmodelcompany!89
No description provided.