research: gobig_v2 B — held-out-length margin stop (frozen, run) #139

Open
starsetbyte wants to merge 2 commits from research/gobig2-b-valmargin-20261018 into research/gobig2-a-len4-20261018
Owner

gobig_v2 study B (valmargin) — held-out-length margin stop

Continues the #133 gated_sharp_max checkpoints (seeds 619719-619728) on the original 840 training rows (lengths 0-2) until BOTH train strict (margin >= 0.001) AND a held-out validation panel VAL56 (N=4/6/8, 4 graphs each, 8 random words of length 5 and 6 per start, base+fresh views, 2150 rows, new seeds, names disjoint from train/eval) is all-correct with min margin >= 0.001. Cap 100. Eval on PN8, PN16, PL16 (identical to #133/#136/#137).

  • Freeze GOBIG2-VALMARGIN-FROZEN-001 ea6c9bd8…defc. Pre-freeze checks PASS. B's eval budget was raised to 520/440 before freezing (after A's calibration); A is counted at its charged cost; all-study reservation 1129 s wall / 934 s CPU of 2400/1800.
  • Receipts: calibration 9.01 s / 8.98 s / 476 MiB; fit 11.37 s / 11.36 s / 314 MiB; eval 144.2 s / 144.2 s / 482 MiB.
  • Fit: every seed passed after 5-30 extra updates (val margin before: <= 0.0005, not strict; after: 0.017-2.89).
  • Results (strict depth per seed 619719..28):
    • PN8: 4 x10, all 480 graphs exact -> CONSISTENT_..._THROUGH_LENGTH_4 (was #133: 3,2,3,2,2,2,3,2,1,3, NO_RELIABLE)
    • PN16: 4 x10 -> CONSISTENT (was #136: 4,2,3,2,3,2,3,2,1,3)
    • PL16 (lengths 0-16): 5,5,6,16,5,8,5,15,16,16 -> MIXED; exact mean at L8/12/16 0.999/0.991/0.987, worst seed L16 0.969 (was #137: 0.878/0.776/0.703, worst 0.342)
  • Interpretation: the #133 checkpoints were under-trained (stopped at the first train margin >= 0.001); a few more updates on the same short data, selected by a held-out-length check, give exact composition through length 4 at 8 and 16 nodes and near-exact through length 16.

Base: #133 branch (stacked on gobig_v2 code from the A PR). Do not merge.

## gobig_v2 study B (valmargin) — held-out-length margin stop Continues the #133 gated_sharp_max checkpoints (seeds 619719-619728) on the original 840 training rows (lengths 0-2) until BOTH train strict (margin >= 0.001) AND a held-out validation panel VAL56 (N=4/6/8, 4 graphs each, 8 random words of length 5 and 6 per start, base+fresh views, 2150 rows, new seeds, names disjoint from train/eval) is all-correct with min margin >= 0.001. Cap 100. Eval on PN8, PN16, PL16 (identical to #133/#136/#137). - Freeze GOBIG2-VALMARGIN-FROZEN-001 `ea6c9bd8…defc`. Pre-freeze checks PASS. B's eval budget was raised to 520/440 before freezing (after A's calibration); A is counted at its charged cost; all-study reservation 1129 s wall / 934 s CPU of 2400/1800. - Receipts: calibration 9.01 s / 8.98 s / 476 MiB; fit 11.37 s / 11.36 s / 314 MiB; eval 144.2 s / 144.2 s / 482 MiB. - Fit: every seed passed after 5-30 extra updates (val margin before: <= 0.0005, not strict; after: 0.017-2.89). - Results (strict depth per seed 619719..28): - PN8: 4 x10, all 480 graphs exact -> CONSISTENT_..._THROUGH_LENGTH_4 (was #133: 3,2,3,2,2,2,3,2,1,3, NO_RELIABLE) - PN16: 4 x10 -> CONSISTENT (was #136: 4,2,3,2,3,2,3,2,1,3) - PL16 (lengths 0-16): 5,5,6,16,5,8,5,15,16,16 -> MIXED; exact mean at L8/12/16 0.999/0.991/0.987, worst seed L16 0.969 (was #137: 0.878/0.776/0.703, worst 0.342) - Interpretation: the #133 checkpoints were under-trained (stopped at the first train margin >= 0.001); a few more updates on the same short data, selected by a held-out-length check, give exact composition through length 4 at 8 and 16 nodes and near-exact through length 16. Base: #133 branch (stacked on gobig_v2 code from the A PR). Do not merge.
gobig_v2/valmargin: fit (5-30 extra updates, all pass held-out L5-6) + eval
All checks were successful
checks / select relevant integrity checks (pull_request) Successful in 7s
checks / E01 candidate surface sufficiency (pull_request) Has been skipped
checks / DMS-003 protocol freeze integrity (pull_request) Has been skipped
checks / DMS-003 CPU successor integrity (pull_request) Has been skipped
checks / resource accounting and workflow consistency (pull_request) Has been skipped
checks / documentation integrity (pull_request) Successful in 7s
69e81f6c05
All checks were successful
checks / select relevant integrity checks (pull_request) Successful in 7s
checks / E01 candidate surface sufficiency (pull_request) Has been skipped
checks / DMS-003 protocol freeze integrity (pull_request) Has been skipped
checks / DMS-003 CPU successor integrity (pull_request) Has been skipped
checks / resource accounting and workflow consistency (pull_request) Has been skipped
checks / documentation integrity (pull_request) Successful in 7s
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin research/gobig2-b-valmargin-20261018:research/gobig2-b-valmargin-20261018
git switch research/gobig2-b-valmargin-20261018

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch research/gobig2-a-len4-20261018
git merge --no-ff research/gobig2-b-valmargin-20261018
git switch research/gobig2-b-valmargin-20261018
git rebase research/gobig2-a-len4-20261018
git switch research/gobig2-a-len4-20261018
git merge --ff-only research/gobig2-b-valmargin-20261018
git switch research/gobig2-b-valmargin-20261018
git rebase research/gobig2-a-len4-20261018
git switch research/gobig2-a-len4-20261018
git merge --no-ff research/gobig2-b-valmargin-20261018
git switch research/gobig2-a-len4-20261018
git merge --squash research/gobig2-b-valmargin-20261018
git switch research/gobig2-a-len4-20261018
git merge --ff-only research/gobig2-b-valmargin-20261018
git switch research/gobig2-a-len4-20261018
git merge research/gobig2-b-valmargin-20261018
git push origin research/gobig2-a-len4-20261018
Sign in to join this conversation.
No reviewers
No labels
R03
research-b
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
smolmodelco/thesmolmodelcompany!139
No description provided.