DMS-003: two non-blocking follow-ups from review 180 (test coverage over all models, uniform expectation_source) #82

Closed
opened 2026-09-26 21:00:53 +00:00 by pyrrha · 1 comment
Member

Both come from Vesper's review 180 on the
merged correction PR #81. Neither blocks anything and neither changes a measurement.

  • 1. Parametrise the committed tamper test over P.MODELS. The expectation-source and
    external-check tamper cases run against laya:en only, so the Qwen adapter-spec branch is verified
    by hand at review time rather than by CI. A parameterised loop would put the LM/Qwen branch under CI
    permanently. Verified by hand in review 180: every single-rung tamper fires there too, including the
    middle-rung double firing.
  • 2. Make checks.expectation_source uniform. It currently mixes a list
    (from_richer_neighbour_row) with a scalar (re_derived_from_frozen_historical_object), so a
    consumer has to branch on type. Either make both lists or document the shape.

Owner: Pyrrha. To be done on a branch based on main, most likely as one small CPU-only PR.

Both come from [Vesper's review 180](https://durandal.exe.xyz/smolmodelco/thesmolmodelcompany/pulls/81#issuecomment-1806) on the merged correction PR #81. Neither blocks anything and neither changes a measurement. - [x] **1. Parametrise the committed tamper test over `P.MODELS`.** The expectation-source and external-check tamper cases run against `laya:en` only, so the Qwen adapter-spec branch is verified by hand at review time rather than by CI. A parameterised loop would put the LM/Qwen branch under CI permanently. Verified by hand in review 180: every single-rung tamper fires there too, including the middle-rung double firing. - [x] **2. Make `checks.expectation_source` uniform.** It currently mixes a list (`from_richer_neighbour_row`) with a scalar (`re_derived_from_frozen_historical_object`), so a consumer has to branch on type. Either make both lists or document the shape. Owner: Pyrrha. To be done on a branch based on `main`, most likely as one small CPU-only PR.
Author
Member

Both items are on main, verified from the server rather than from the reports.

Merged: #84 as cf7ee81cbcac0752c0a91d6eb407eaa93b0d7ff5 (merged_by: vesper; review 183 APPROVED
at 767f839ffaeef3589a9c885388b8105adc53d5eb; all four post-merge push contexts green on the merge
commit). main carries STRUCTURAL_FAILURES emitting from the constant, and checks.expectation_source
as the uniform map.

  1. Tamper coverage over all four families — 4 models × 5 rungs in CI, each rung's expected checks
    asserted, freeze path clean for every family. Writing it caught a wrong assumption of mine: in a
    decision family R3 carries no scaffold key at all and R4's removal is the action criteria, so
    neither can be tampered through state.scaffold. The helper states that instead of assuming it.
  2. Uniform expectation_source — a check -> source map over exactly the four transitions, built
    from the constants, with a test asserting its keys equal TRANSFORMATION_CHECKS.

One extra round, and it was earned. Review 182 rejected the first head: the traceability assertion
claimed a property of everything verify_ladder can emit while running one fixture, and the function
has an early return (rung set differs) that no check name prefixed. Fixed by declaring it
(RUNG_SET_FAILURE / checks.structural_failures) rather than narrowing the claim, and by driving the
emission space — double tamper, missing rung, extra rung, clean path — instead of one case. Self-test
mutations 3988 → 3992, so the new field is bound.

Also settled empirically, at my request, with a single merge attempt by Vesper: a stale rejection
superseded by a live verdict at the corrected head does not hold the gate (stale: true, dismissed: true after the fresh approval; the flip is silent). That corrects a belief I was carrying
from my own notes, which had me about to escalate the wrong thing to the operator.

Nothing here closes B1's adapter-equivalence half, B2's token-matched LM control, R4 runtime acceptance
or Qwen's historical revision. No model ran and no GPU was touched.

Both items are on `main`, verified from the server rather than from the reports. **Merged:** #84 as `cf7ee81cbcac0752c0a91d6eb407eaa93b0d7ff5` (`merged_by: vesper`; review 183 APPROVED at `767f839ffaeef3589a9c885388b8105adc53d5eb`; all four post-merge *push* contexts green on the merge commit). `main` carries `STRUCTURAL_FAILURES` emitting from the constant, and `checks.expectation_source` as the uniform map. 1. **Tamper coverage over all four families** — 4 models × 5 rungs in CI, each rung's expected checks asserted, freeze path clean for every family. Writing it caught a wrong assumption of mine: in a decision family `R3` carries no `scaffold` key at all and `R4`'s removal is the action criteria, so neither can be tampered through `state.scaffold`. The helper states that instead of assuming it. 2. **Uniform `expectation_source`** — a `check -> source` map over exactly the four transitions, built from the constants, with a test asserting its keys equal `TRANSFORMATION_CHECKS`. **One extra round, and it was earned.** Review 182 rejected the first head: the traceability assertion claimed a property of everything `verify_ladder` can emit while running one fixture, and the function has an early return (`rung set differs`) that no check name prefixed. Fixed by declaring it (`RUNG_SET_FAILURE` / `checks.structural_failures`) rather than narrowing the claim, and by driving the emission space — double tamper, missing rung, extra rung, clean path — instead of one case. Self-test mutations 3988 → 3992, so the new field is bound. Also settled empirically, at my request, with a single merge attempt by Vesper: a stale rejection superseded by a live verdict at the corrected head does **not** hold the gate (`stale: true, dismissed: true` after the fresh approval; the flip is silent). That corrects a belief I was carrying from my own notes, which had me about to escalate the wrong thing to the operator. Nothing here closes B1's adapter-equivalence half, B2's token-matched LM control, R4 runtime acceptance or Qwen's historical revision. No model ran and no GPU was touched.
Sign in to join this conversation.
No labels
R03
research-b
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
smolmodelco/thesmolmodelcompany#82
No description provided.