research: dense-validation held-out stop (GOBIG4-DENSE-FROZEN-001) — base #143 #145
Open
starsetbyte
wants to merge 2 commits from
research/gobig4-dense-val-20261018 into research/gobig3-val78-20261018
pull from: research/gobig4-dense-val-20261018
merge into: smolmodelco:research/gobig3-val78-20261018
smolmodelco:main
smolmodelco:research/session-checkpoint-20261025
smolmodelco:research/budgeted-post-repair-audit-v1-20261025
smolmodelco:research/sensor-mapping-reference-v2-20261025
smolmodelco:research/posterior-diagnosis-20261024
smolmodelco:research/manual-examples-20261024
smolmodelco:research/diagnostic-control-20261023
smolmodelco:research/lantern-model-20261021
smolmodelco:research/night-breadth-20261021
smolmodelco:research/informative-probing-20261021
smolmodelco:research/source-structure-transfer-20261021
smolmodelco:research/aet-baseline-diagnostic-20261021
smolmodelco:research/action-effects-transfer-20261021
smolmodelco:research/online-workflow-recovery-20261021
smolmodelco:research/online-recovery-prep-20261021
smolmodelco:research/synthetic-workspace-c-20261021
smolmodelco:research/stale-memory-retest-20261021
smolmodelco:research/continuum-workspace-memory-20261021
smolmodelco:research/continuum-binary-recovery-20261020
smolmodelco:research/continuum-recovery-policy-20261020
smolmodelco:research/continuum-task-loop-20261020
smolmodelco:research/continuum-projection-20261020
smolmodelco:research/continuum-bnd1-trace-20261020
smolmodelco:research/continuum-boundary-20261020
smolmodelco:research/continuum-halting-20261020
smolmodelco:research/continuum-selector-20261020
smolmodelco:research/continuum-routing-20261020
smolmodelco:research/smc-lrm1-osd1-20261020
smolmodelco:research/f3-and-not-pair-20261020
smolmodelco:research/f3-interference-study-card-20261020
smolmodelco:research/f3-cache-fidelity-20261020
smolmodelco:research/f3-cache-fidelity-scope-20261020
smolmodelco:research/f3-saved-diagnosis-20261020
smolmodelco:research/f3-full-fit-20261020
smolmodelco:research/f3-runner-20261020
smolmodelco:research/f2-saved-diagnosis-20261020
smolmodelco:research/f3-reference-20261020
smolmodelco:research/f2-prerequisite-20261020
smolmodelco:research/action-predicate-discovery-20261019
smolmodelco:research/capacity-effects-spec-20261019
smolmodelco:research/copy-move-qualification-20261019
smolmodelco:research/artifact-handoff-20261019
smolmodelco:research/qonly-heldout-confirmation-20261019
smolmodelco:research/new-rule-qualification-20261019
smolmodelco:research/adaptation-retention-final-20261019
smolmodelco:research/adaptation-retention-pilot-20261019
smolmodelco:research/confirmation-prep-20261019
smolmodelco:artifacts/binding-2026-10-18
smolmodelco:research/gobig3-val78-20261018
smolmodelco:research/gobig3-scratch-val-20261018
smolmodelco:research/gobig3-len4r-20261018
smolmodelco:research/gobig2-c-diag-20261018
smolmodelco:research/gobig2-b-valmargin-20261018
smolmodelco:research/gobig2-a-len4-20261018
smolmodelco:research/gobig-s3-long-lengths-20261018
smolmodelco:research/gobig-s2-big-graphs-20261018
smolmodelco:research/gobig-s5-margin-20261018
smolmodelco:research/gobig-s4-baseline-long-20261018
smolmodelco:research/gobig-s1-seeds-20261018
smolmodelco:research/sharp-variant-20261018
smolmodelco:research/cpu-lab-20261018
smolmodelco:docs/weekend-brief-historical
smolmodelco:research/size-diagnostic-20261018
smolmodelco:chore/ledger-x14-census
smolmodelco:research/composition-size-20261018
smolmodelco:research/composition-extend-20261018
smolmodelco:research/composition-fit-20261018
smolmodelco:research/autonomous-composition-results-20261018
smolmodelco:research/autonomous-composition-prep-20261018
smolmodelco:research/binding-interface-audit-20261018
smolmodelco:governance/research-reset
smolmodelco:preservation/pyrrha-inventory-20261004
smolmodelco:preservation/vesper-inventory-20261004
smolmodelco:docs/accordion-research-20260927
smolmodelco:research/c1-parity-freeze-92
smolmodelco:research/c1-consolidation-stream
smolmodelco:research/c1-gpu-guide
smolmodelco:dms003/r4-bare-label-sidecar
smolmodelco:dms003/pre-run-rulings
smolmodelco:X-DS-001/census
smolmodelco:exploration/e01-withheld-ops-metadata
smolmodelco:census/t2-eido-half
smolmodelco:docs/smc-research-lab-operating-contract
smolmodelco:lab/state-validation-machinery
smolmodelco:E01/surface-semantics
smolmodelco:R02/preflight-corrections
smolmodelco:R03-records-generator-reference
smolmodelco:R03-episode-foundation
smolmodelco:site/public-research-site
smolmodelco:plan/minimum-apparatus
smolmodelco:R03-generator-scorer-split-protocol
smolmodelco:docs/forgejo-canonical-collaboration
No reviewers
Labels
No labels
R03
research-b
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
smolmodelco/thesmolmodelcompany!145
Loading…
Reference in a new issue
No description provided.
Delete branch "research/gobig4-dense-val-20261018"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Base:
research/gobig3-val78-20261018(#143,6f8bbfb). Unmerged; for Codex review. Owner-approved single study (Cassie, 2026-10-18), cap 40 min wall / 30 CPU-min.Study: dense-validation held-out stop (GOBIG4-DENSE-FROZEN-001)
Recipe B (#139) rerun with a much denser held-out stop set: continue #133
gated_sharp_maxcheckpoints (same init as #139), 10 seeds 619719-619728, on the original 840 train rows, until train strict (margin >= 0.001) AND VALD all-correct with min margin >= 0.001; check every 5 updates; cap 100 (admitted).VALD (19,456 rows): 4-node graphs, 4 graphs (seed 863101), per graph/start ALL 480 words of lengths 5-8 (exhaustive, not sampled) + a frozen deterministic sample of 32 distinct words at each length 9-12; base + fresh_names views.
Overlap handling: VALD graphs are drawn with rejection of the 48 PL16 abstract graphs (reconstructed by
pl16_graphs(), verified in precheck against every PL16 trace), so no (graph, start, word) triple is shared with an eval panel. Names/keys disjoint from train and all eval panels (asserted at freeze). Word strings necessarily recur in PL16. PN8/PN16 are 8/16-node panels, structurally disjoint.Framework forked to
research/gobig_v3(gv2.py -> gv3.py) so earlier FROZEN hashes stay valid. Only loop change: VALD is checked only when the train check already passes (same stop decision).Protocol: stdlib tests -> dummy pipeline -> budget check -> FROZEN (sha256
7b8cfa4d41d85fff43a34ad739df2863b73f148a60efd5890946c31dc0529548) -> calibration -> 2x admission -> fit -> eval, each a separate supervised command (1 CPU, nice +10, 2 GiB RSS); all exit 0.Receipts (wall / CPU / peak RSS): calibration 11.2 s / 11.2 s / 557 MiB; fit 37.4 s / 37.4 s / 380 MiB; eval 146.9 s / 146.8 s / 483 MiB. Total 195.5 s wall / 195.4 s CPU (of 2400 / 1800).
Dense-validation held-out stop vs #139 and #143
Per seed: strict depth PN8 / PN16 / PL16, PL16 exact at length 16, extra updates over #133.
#143 column: the 6 continued seeds use the val78 PL16 eval; other entries (and all PN8/PN16 for #143, which val78 did not re-run) are #139 values.
Decisions (dense): {"gated_sharp_max": {"PN8": "MIXED_PARTIAL_COMPOSITION", "PN16": "CONSISTENT_AUTONOMOUS_COMPOSITION_THROUGH_LENGTH_4", "PL16": "MIXED_PARTIAL_COMPOSITION"}}
Decisions (#139): {"gated_sharp_max": {"PN8": "CONSISTENT_AUTONOMOUS_COMPOSITION_THROUGH_LENGTH_4", "PN16": "CONSISTENT_AUTONOMOUS_COMPOSITION_THROUGH_LENGTH_4", "PL16": "MIXED_PARTIAL_COMPOSITION"}}
Interpretation
Status: research paused by Cassie (Oct 18, 2026, 11:56 PM ET). No further runs until she resumes.
Takeaway: the remaining errors depend on which graph is tested, not on rare words. Every seed got every length 5–8 check word right on the 4 check graphs, but 5 seeds still fail on the evaluation graphs. The check set was 4-node only, so it also lost the size signal from #139's 4/6/8-node set.
Next-step options (each needs a new freeze and Cassie's approval):
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.Merge
Merge the changes and update on Forgejo.Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.