research: gobig round 3 (1) — lengths 0-4 training rerun (frozen, run) #141

Open
starsetbyte wants to merge 2 commits from research/gobig3-len4r-20261018 into research/gobig2-c-diag-20261018
Owner

gobig_v2 round 3, study 1 (len4r) — rerun of A (#138) with an eval budget that admits it

Round-3 code changes in this PR (all pre-freeze, dummy pipeline for all five keys passed): round-3 study definitions in studies.py; check_budget.py totals each approval round separately (round 3 cap 2400 s wall / 1800 s CPU, reserved 2290 / 1800); gv2.val_rows takes a key (VAL56 unchanged — hash re-checked against valmargin/FROZEN.json in precheck — plus new VAL78). Review check: no review comments/reviews existed on #125-#140.

Design identical to A: gated_sharp_max trained on all words of length 0-4 (sizes 4-6, 4 graphs/size, 1860 rows), 10 seeds, stop at train strict margin >= 0.001; eval PN8 / PN16 / PL16 (same panels as #133/#136/#137/#139).

  • Freeze GOBIG3-LEN4R-FROZEN-001 00b5b44d…e988.
  • Admission: eval proj 205.7 s (admitted under 500/430); fit cap 100 (200/150 not admitted under 340/300).
  • Receipts: calibration 9.11 s / 9.08 s / 450 MiB; fit 55.3 s / 55.3 s / 309 MiB; eval 142.4 s / 142.3 s / 464 MiB. All exit 0.
  • Fit: all 10 pass, 40-60 updates.
  • Strict depth (619719..28): PN8 4 x10, PN16 4 x10 (both CONSISTENT through length 4); PL16 13,16,6,8,4,4,16,6,4,3 (MIXED). PL16 exact mean L8/12/16 0.985/0.957/0.921, worst seed L16 0.760.
  • vs B (#139, short data + held-out L5-6 stop): same on PN8/PN16; B is better on PL16 (0.999/0.991/0.987, worst 0.969, min depth 5). Training on longer words alone does not give B's long-length robustness.

Base: C branch (#140). Do not merge.

## gobig_v2 round 3, study 1 (len4r) — rerun of A (#138) with an eval budget that admits it Round-3 code changes in this PR (all pre-freeze, dummy pipeline for all five keys passed): round-3 study definitions in `studies.py`; `check_budget.py` totals each approval round separately (round 3 cap 2400 s wall / 1800 s CPU, reserved 2290 / 1800); `gv2.val_rows` takes a key (VAL56 unchanged — hash re-checked against valmargin/FROZEN.json in precheck — plus new VAL78). Review check: no review comments/reviews existed on #125-#140. Design identical to A: gated_sharp_max trained on all words of length 0-4 (sizes 4-6, 4 graphs/size, 1860 rows), 10 seeds, stop at train strict margin >= 0.001; eval PN8 / PN16 / PL16 (same panels as #133/#136/#137/#139). - Freeze GOBIG3-LEN4R-FROZEN-001 `00b5b44d…e988`. - Admission: eval proj 205.7 s (admitted under 500/430); fit cap 100 (200/150 not admitted under 340/300). - Receipts: calibration 9.11 s / 9.08 s / 450 MiB; fit 55.3 s / 55.3 s / 309 MiB; eval 142.4 s / 142.3 s / 464 MiB. All exit 0. - Fit: all 10 pass, 40-60 updates. - Strict depth (619719..28): PN8 4 x10, PN16 4 x10 (both CONSISTENT through length 4); PL16 13,16,6,8,4,4,16,6,4,3 (MIXED). PL16 exact mean L8/12/16 0.985/0.957/0.921, worst seed L16 0.760. - vs B (#139, short data + held-out L5-6 stop): same on PN8/PN16; B is better on PL16 (0.999/0.991/0.987, worst 0.969, min depth 5). Training on longer words alone does not give B's long-length robustness. Base: C branch (#140). Do not merge.
gobig_v2/len4r: fit + eval
All checks were successful
checks / select relevant integrity checks (pull_request) Successful in 8s
checks / E01 candidate surface sufficiency (pull_request) Has been skipped
checks / DMS-003 protocol freeze integrity (pull_request) Has been skipped
checks / DMS-003 CPU successor integrity (pull_request) Has been skipped
checks / resource accounting and workflow consistency (pull_request) Has been skipped
checks / documentation integrity (pull_request) Successful in 7s
a11d01d501
All checks were successful
checks / select relevant integrity checks (pull_request) Successful in 8s
checks / E01 candidate surface sufficiency (pull_request) Has been skipped
checks / DMS-003 protocol freeze integrity (pull_request) Has been skipped
checks / DMS-003 CPU successor integrity (pull_request) Has been skipped
checks / resource accounting and workflow consistency (pull_request) Has been skipped
checks / documentation integrity (pull_request) Successful in 7s
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin research/gobig3-len4r-20261018:research/gobig3-len4r-20261018
git switch research/gobig3-len4r-20261018

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch research/gobig2-c-diag-20261018
git merge --no-ff research/gobig3-len4r-20261018
git switch research/gobig3-len4r-20261018
git rebase research/gobig2-c-diag-20261018
git switch research/gobig2-c-diag-20261018
git merge --ff-only research/gobig3-len4r-20261018
git switch research/gobig3-len4r-20261018
git rebase research/gobig2-c-diag-20261018
git switch research/gobig2-c-diag-20261018
git merge --no-ff research/gobig3-len4r-20261018
git switch research/gobig2-c-diag-20261018
git merge --squash research/gobig3-len4r-20261018
git switch research/gobig2-c-diag-20261018
git merge --ff-only research/gobig3-len4r-20261018
git switch research/gobig2-c-diag-20261018
git merge research/gobig3-len4r-20261018
git push origin research/gobig2-c-diag-20261018
Sign in to join this conversation.
No reviewers
No labels
R03
research-b
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
smolmodelco/thesmolmodelcompany!141
No description provided.