research: gobig round 3 (2) — held-out stop from scratch, soft + mean-pool (frozen, run) #142

Open
starsetbyte wants to merge 2 commits from research/gobig3-scratch-val-20261018 into research/gobig3-len4r-20261018
Owner

gobig_v2 round 3, study 2 (scratch_val) — B's held-out stop rule from scratch for gated_soft and gated_sharp_mean

Fresh training (840 rows, lengths 0-2, sizes 4-6) of 10 seeds each, stopping only when train strict (margin >= 0.001) AND VAL56 (N=4/6/8, lengths 5-6, 2150 rows; same panel as #139) is all-correct with min margin >= 0.001; cap 150 (admitted). Eval PN8 and PL16 (PN16 omitted for budget).

  • Freeze GOBIG3-SCRATCH_VAL-FROZEN-001 ce631247…b730. Pre-freeze tests + dummy pipeline PASS.
  • Receipts: calibration 7.70 s / 7.69 s / 378 MiB; fit 139.9 s / 139.7 s / 314 MiB; eval 118.9 s / 118.9 s / 401 MiB. All exit 0.
  • Fit: gated_sharp_mean 9/10 met the rule in 40-65 updates (619723 failed at 150, as in #133). gated_soft 0/10: train strict held but VAL56 min margin stayed 0.0 for every seed (it contains 8-node graphs, where soft weights dilute ~1/N, #131); all ran the full 150.
  • Strict depth (619719..28):
    • gated_soft PN8 3,3,1,3,2,3,3,3,2,3 (L4 exact mean 0.384) NO_RELIABLE; PL16 16,16,5,16,16,16,16,16,10,16, L16 exact mean 0.999, min 0.990 — MIXED (the best long-length result so far, at N=4).
    • gated_sharp_mean PN8 4 x9 + 0 (619723) CHECK_FAILED; PL16 8,5,7,5,0,7,5,15,16,8.
  • Compare (#139 sharp_max + same rule from #133 checkpoints): PN8 4 x10, PL16 min depth 5, L16 0.987 mean.
  • Reading: longer training (150 updates) makes the soft arm an excellent length extrapolator on 4-node graphs but it still cannot handle size; mean pool matches max pool on size when it trains, but is slightly worse on long lengths and has a seed that does not train.

Base: #141 branch. Do not merge.

## gobig_v2 round 3, study 2 (scratch_val) — B's held-out stop rule from scratch for gated_soft and gated_sharp_mean Fresh training (840 rows, lengths 0-2, sizes 4-6) of 10 seeds each, stopping only when train strict (margin >= 0.001) AND VAL56 (N=4/6/8, lengths 5-6, 2150 rows; same panel as #139) is all-correct with min margin >= 0.001; cap 150 (admitted). Eval PN8 and PL16 (PN16 omitted for budget). - Freeze GOBIG3-SCRATCH_VAL-FROZEN-001 `ce631247…b730`. Pre-freeze tests + dummy pipeline PASS. - Receipts: calibration 7.70 s / 7.69 s / 378 MiB; fit 139.9 s / 139.7 s / 314 MiB; eval 118.9 s / 118.9 s / 401 MiB. All exit 0. - Fit: gated_sharp_mean 9/10 met the rule in 40-65 updates (619723 failed at 150, as in #133). gated_soft 0/10: train strict held but VAL56 min margin stayed 0.0 for every seed (it contains 8-node graphs, where soft weights dilute ~1/N, #131); all ran the full 150. - Strict depth (619719..28): - gated_soft PN8 3,3,1,3,2,3,3,3,2,3 (L4 exact mean 0.384) NO_RELIABLE; PL16 16,16,5,16,16,16,16,16,10,16, L16 exact mean 0.999, min 0.990 — MIXED (the best long-length result so far, at N=4). - gated_sharp_mean PN8 4 x9 + 0 (619723) CHECK_FAILED; PL16 8,5,7,5,0,7,5,15,16,8. - Compare (#139 sharp_max + same rule from #133 checkpoints): PN8 4 x10, PL16 min depth 5, L16 0.987 mean. - Reading: longer training (150 updates) makes the soft arm an excellent length extrapolator on 4-node graphs but it still cannot handle size; mean pool matches max pool on size when it trains, but is slightly worse on long lengths and has a seed that does not train. Base: #141 branch. Do not merge.
gobig_v2/scratch_val: fit + eval
All checks were successful
checks / select relevant integrity checks (pull_request) Successful in 8s
checks / E01 candidate surface sufficiency (pull_request) Has been skipped
checks / DMS-003 protocol freeze integrity (pull_request) Has been skipped
checks / DMS-003 CPU successor integrity (pull_request) Has been skipped
checks / resource accounting and workflow consistency (pull_request) Has been skipped
checks / documentation integrity (pull_request) Successful in 7s
9cbe21eb2d
All checks were successful
checks / select relevant integrity checks (pull_request) Successful in 8s
checks / E01 candidate surface sufficiency (pull_request) Has been skipped
checks / DMS-003 protocol freeze integrity (pull_request) Has been skipped
checks / DMS-003 CPU successor integrity (pull_request) Has been skipped
checks / resource accounting and workflow consistency (pull_request) Has been skipped
checks / documentation integrity (pull_request) Successful in 7s
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin research/gobig3-scratch-val-20261018:research/gobig3-scratch-val-20261018
git switch research/gobig3-scratch-val-20261018

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch research/gobig3-len4r-20261018
git merge --no-ff research/gobig3-scratch-val-20261018
git switch research/gobig3-scratch-val-20261018
git rebase research/gobig3-len4r-20261018
git switch research/gobig3-len4r-20261018
git merge --ff-only research/gobig3-scratch-val-20261018
git switch research/gobig3-scratch-val-20261018
git rebase research/gobig3-len4r-20261018
git switch research/gobig3-len4r-20261018
git merge --no-ff research/gobig3-scratch-val-20261018
git switch research/gobig3-len4r-20261018
git merge --squash research/gobig3-scratch-val-20261018
git switch research/gobig3-len4r-20261018
git merge --ff-only research/gobig3-scratch-val-20261018
git switch research/gobig3-len4r-20261018
git merge research/gobig3-scratch-val-20261018
git push origin research/gobig3-len4r-20261018
Sign in to join this conversation.
No reviewers
No labels
R03
research-b
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
smolmodelco/thesmolmodelcompany!142
No description provided.