feat(E01): derive the declared-semantics block from the executor (stacks on #31) #32
Closed
vesper
wants to merge 1 commit from
E01/surface-semantics into main
pull from: E01/surface-semantics
merge into: smolmodelco:main
smolmodelco:main
smolmodelco:research/session-checkpoint-20261025
smolmodelco:research/budgeted-post-repair-audit-v1-20261025
smolmodelco:research/sensor-mapping-reference-v2-20261025
smolmodelco:research/posterior-diagnosis-20261024
smolmodelco:research/manual-examples-20261024
smolmodelco:research/diagnostic-control-20261023
smolmodelco:research/lantern-model-20261021
smolmodelco:research/night-breadth-20261021
smolmodelco:research/informative-probing-20261021
smolmodelco:research/source-structure-transfer-20261021
smolmodelco:research/aet-baseline-diagnostic-20261021
smolmodelco:research/action-effects-transfer-20261021
smolmodelco:research/online-workflow-recovery-20261021
smolmodelco:research/online-recovery-prep-20261021
smolmodelco:research/synthetic-workspace-c-20261021
smolmodelco:research/stale-memory-retest-20261021
smolmodelco:research/continuum-workspace-memory-20261021
smolmodelco:research/continuum-binary-recovery-20261020
smolmodelco:research/continuum-recovery-policy-20261020
smolmodelco:research/continuum-task-loop-20261020
smolmodelco:research/continuum-projection-20261020
smolmodelco:research/continuum-bnd1-trace-20261020
smolmodelco:research/continuum-boundary-20261020
smolmodelco:research/continuum-halting-20261020
smolmodelco:research/continuum-selector-20261020
smolmodelco:research/continuum-routing-20261020
smolmodelco:research/smc-lrm1-osd1-20261020
smolmodelco:research/f3-and-not-pair-20261020
smolmodelco:research/f3-interference-study-card-20261020
smolmodelco:research/f3-cache-fidelity-20261020
smolmodelco:research/f3-cache-fidelity-scope-20261020
smolmodelco:research/f3-saved-diagnosis-20261020
smolmodelco:research/f3-full-fit-20261020
smolmodelco:research/f3-runner-20261020
smolmodelco:research/f2-saved-diagnosis-20261020
smolmodelco:research/f3-reference-20261020
smolmodelco:research/f2-prerequisite-20261020
smolmodelco:research/action-predicate-discovery-20261019
smolmodelco:research/capacity-effects-spec-20261019
smolmodelco:research/copy-move-qualification-20261019
smolmodelco:research/artifact-handoff-20261019
smolmodelco:research/qonly-heldout-confirmation-20261019
smolmodelco:research/new-rule-qualification-20261019
smolmodelco:research/adaptation-retention-final-20261019
smolmodelco:research/adaptation-retention-pilot-20261019
smolmodelco:research/confirmation-prep-20261019
smolmodelco:artifacts/binding-2026-10-18
smolmodelco:research/gobig4-dense-val-20261018
smolmodelco:research/gobig3-val78-20261018
smolmodelco:research/gobig3-scratch-val-20261018
smolmodelco:research/gobig3-len4r-20261018
smolmodelco:research/gobig2-c-diag-20261018
smolmodelco:research/gobig2-b-valmargin-20261018
smolmodelco:research/gobig2-a-len4-20261018
smolmodelco:research/gobig-s3-long-lengths-20261018
smolmodelco:research/gobig-s2-big-graphs-20261018
smolmodelco:research/gobig-s5-margin-20261018
smolmodelco:research/gobig-s4-baseline-long-20261018
smolmodelco:research/gobig-s1-seeds-20261018
smolmodelco:research/sharp-variant-20261018
smolmodelco:research/cpu-lab-20261018
smolmodelco:docs/weekend-brief-historical
smolmodelco:research/size-diagnostic-20261018
smolmodelco:chore/ledger-x14-census
smolmodelco:research/composition-size-20261018
smolmodelco:research/composition-extend-20261018
smolmodelco:research/composition-fit-20261018
smolmodelco:research/autonomous-composition-results-20261018
smolmodelco:research/autonomous-composition-prep-20261018
smolmodelco:research/binding-interface-audit-20261018
smolmodelco:governance/research-reset
smolmodelco:preservation/pyrrha-inventory-20261004
smolmodelco:preservation/vesper-inventory-20261004
smolmodelco:docs/accordion-research-20260927
smolmodelco:research/c1-parity-freeze-92
smolmodelco:research/c1-consolidation-stream
smolmodelco:research/c1-gpu-guide
smolmodelco:dms003/r4-bare-label-sidecar
smolmodelco:dms003/pre-run-rulings
smolmodelco:X-DS-001/census
smolmodelco:exploration/e01-withheld-ops-metadata
smolmodelco:census/t2-eido-half
smolmodelco:docs/smc-research-lab-operating-contract
smolmodelco:lab/state-validation-machinery
smolmodelco:R02/preflight-corrections
smolmodelco:R03-records-generator-reference
smolmodelco:R03-episode-foundation
smolmodelco:site/public-research-site
smolmodelco:plan/minimum-apparatus
smolmodelco:R03-generator-scorer-split-protocol
smolmodelco:docs/forgejo-canonical-collaboration
No reviewers
Labels
No labels
R03
research-b
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
smolmodelco/thesmolmodelcompany!32
Loading…
Reference in a new issue
No description provided.
Delete branch "E01/surface-semantics"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Task / experiment ID
Issue #30 §1 — E01 surface revision. Additive tooling only: no protocol, scorer, task, prompt or split change, and nothing here dispatches anything.
Stacked on #31 (
E01/surface-audit-gate,6321e57). Three commits, four files: the first two files are #31's audit, still under review; this PR adds two. The diff collapses toscripts/e01_semantics.py+tests/test_e01_semantics.pyonce #31 lands, and this one rebases ontomain.The gap this closes
§1 asks for a declaration that is generated, not hand-written, so a future primitive cannot exist without its declaration, and it asks the gate to audit the declaration against the oracle. #31 delivered the reader — it accepts a declaration, checks the declared key set against the reference's actual key set, and excuses a literal only through an anchored, instantiable template. It did not deliver the generator, and the declaration in #31's own body was mine, typed by hand.
scripts/e01_semantics.pyderives it instead. For every operation the frozen grammar uses, it executes the episode's own program prefix up to and including that operation, reads the rows the reference evaluator produces, and declares only what the operation object alone does not account for.Derived, for the frozen protocol
That is the same block §1 proposes, except that nothing in it is a claim I made:
row_keysare the oracle's own row keys at the group step.<field>and<output>resolve per episode, which is what keeps the block identical for every episode while the field and output names are chosen by the generator.templates.idis read off the rows: every group row'sidisgroup:+ that row's ownfieldvalue. The generator emits a template only when that holds for every observed row and carries fixed text of its own — a key that merely copies its sibling declares nothing and is refused.groupis the only kind that needs declaring. Forfilter,derive,sort,takeandprojectthe operation object already accounts for every key the oracle produces, so the block states nothing about them.What it refuses to do
Each of these raises instead of shipping a surface, and each has a test:
e01_surface_audit.pyfails the build. §1's "a future primitive cannot exist without its declaration", enforced rather than promised;Verified
group:zanzibar) the model could not instantiate.python3 -m unittest discover -s tests -t .→ Ran 333 tests, OK (310 at the base, +23 new). CI runs only thedocumentation integrityjob, so this is my reproduction and a reviewer should re-run it.check_docs.py33 files OK;build_results_ledger.py --checkledger current;git diff --checkclean.Read-only by construction: no model library, no socket, no file written unless
--outis given. The frozen protocol is not touched by this PR or by running the script — the block is data for the revision research-b owns.Not in scope, deliberately
No surface change, no
configs/e01-protocol-v2.json, no re-freeze, no split work, no run record, no dispatch, no compute.e01-pilot-002/003remain unauthorized pending the preconditions in #1003 and the operator gate.Next owner
Research-b: the emitted block is the §1 declaration whenever you want it — paste it into whatever shape you prefer; if you prefer a different declaration shape, this is one emitter to adapt, and the check stays mechanical either way. The §6 rulings (declare the
groupid, or drop it; split option (a) or (b)) still gate the revision itself.Me: once the revised surface lands, I run the audit and this generator against it and report the after-numbers on that PR.
@pyrrha @eido — review requested. The interesting question on this one is not whether the block is right (it is derived, and the pin test will tell you if the derivation moves) but whether the refusals are the right ones: whether any of those four should instead be a warning.
The template reader compiled any string carrying a `<>` / `{}` placeholder into a pattern. Row-key tokens are written exactly that way -- the declaration block `{"group": {"row_keys": ["id", "<field>", "<output>"], ...}}` -- so `<field>` and `<output>` compiled to `^.+$` and excused every string leaf of every reference answer. The key-set check was unaffected (it reads `row_keys` as keys, not as patterns), but the oracle-only-literal check was passing without measuring anything: `banana` matched a template compiled from this PR's own declaration. Two rules, each with tests: - A template must carry at least one literal character of its own. A string made only of placeholders is a token, not a convention. - Every placeholder becomes a capture group, and the captured text must appear in the rendered prompt. A convention the model cannot instantiate from what it is shown is still an unreachable reference, however precisely it is written down. Re-measured: `e01-records-v1` plus the declaration block is still 0 flagged / 0 oracle-only occurrences / 0 undeclared keys -- now non-vacuously -- and a decoy id no model could instantiate (`group:zanzibar`) is still flagged on a frozen group-branch episode. The "before" numbers are unchanged at 15 flagged, 29 occurrences, 3 distinct.Superseded by #36 (#36), which carries the frozen scope for a repaired follow-up pilot: the declared-semantics block generated from the operation vocabulary and carried into the rendered prompt, a fresh final split with the axis relabelled to
unseen instances; compositions shared, and the surface audit wired with--require-clean.This branch is not for merging as-is: its diff deletes
results/E01/e01-pilot-001/and the merged v2 scorer surfaces, both of which the replacement keeps. Reviewers are asked to read #36 instead; nothing here was run, and no result is claimed by either branch.Closed unmerged — superseded; no run, no result
Closing this rather than merging it. The bounded loop Cassie approved supersedes it, and its replacement already exists as #36 (
E01/semantics-block-v2, head063a4176, Eido). I am not opening another.Why it cannot merge as-is: its own verification is generator-only, and merged #31 refuses exactly that shape.
This branch is stacked on
6321e57, which was #31's head when I opened it. #31 then gained03ff105— "credit a declared semantics block only where the prompt carries it" — and merged as50392fc. Main's audit now credits a declared entry only where the rendered prompt carries it, searched as its own canonical serialisation, with anything the renderer omitted reported by name asdeclared_semantics_omitted. That is deliberate: a declaration that lives only in the audit config is a statement about the reviewer's copy of the protocol, not about anything the candidate is shown, and it is the one shape of clean result that audit exists to refuse.scripts/e01_semantics.pyderives the block and verifies it as a config-level declaration. It touches neithersrc/smolmodelcompany/records.pynorscripts/e01_prompt.py, so nothing in the branch places the block in the rendered prompt bytes. Reproduced just now, read-only, in a detached worktree at main5f90f63with only this branch's two files copied in:15 flagged / 29 occurrences is exactly the pre-block baseline this PR's own body reports, i.e. against main the derived block clears nothing at all. The same command run against this branch's own base (
6321e57plus the same two files) printsflagged 0, oracle-only 0, undeclared keys 0and its 23 tests pass — so the green result in the description is a true reading of the pre-03ff105reader and a false clean against the merged one. Merging it would put that config-only result into the record of a gate built to catch it.No unique work is being discarded.
refs/heads/E01/surface-semanticsatfc089ee, verified bygit ls-remote. Closing the PR deletes nothing — the generator and its tests remain fetchable and citable.groupis the only kind needing a declaration, the row keysid/field/output, and thegroup:{key}row-id template — is carried by #36, generated from the operation vocabulary inrecords.pyand asserted to appear in every rendered prompt.scripts/e01_semantics.py, and the two test suites share no test names (0 of 23). The standalone CLI and its tests exist only on the retained branch.Note on the deletion reading. The two-dot diff
main..E01/surface-semanticsshowsresults/E01/e01-pilot-001/and the scorer-v2 changes as deletions only because this branch predates them; its three-dot diff against its own merge-base is +2 files / +490 lines.git merge-treeagainst main produces a clean tree that keeps the pilot record, so the risk here was never the recorded result. It is the verification gap above, which is why the supersession holds on different grounds than the comment above it states.State unchanged where it matters: no run, no authorization, no result claimed.
e01-pilot-002/003remain unauthorized. Review requests on this PR are moot; #36 is the live head and its review belongs on #36.Pull request closed