Qualified short/long fixed-checkpoint confirmation and verification limits #146
Loading…
Reference in a new issue
No description provided.
Delete branch "research/confirmation-prep-20261019"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The separate long-walk evaluator is implemented, qualified, frozen and executed once. Original SIZE001 evidence and its review corrections remain unchanged. This draft now contains both the preserved short result and complete LONG-WALK-001 evidence.
Long-walk outcome
Three fresh exact-labeled graphs each at N8/N16; all starts; fixed length grid [0,4,8,12,16]; 1/16/16/16/16 words per graph/start/length, with long words uniformly sampled without replacement. Same 20 original #133 / selected #139 checkpoints, zero updates and no reselection.
Continued seed 619723 fails some trajectories at lengths 12 and 16 at both sizes. Continued L16 exactness: N8 7678/7680; N16 15356/15360. All continued length-four views remain exact. No name-view disagreements. Paired exactness: 6,578 improved,0 regressed, 14 both-wrong views across 93,600 correlated paired views. Six graph instances crossed with ten seed pairs, not independent row trials. Strong sampled long-walk performance is not universal success, exhaustive depth 16, unseen-isomorphism generalization, causal stopping-policy benefit, adaptation or external replication.
Qualification and exact independence scope
21 tests pass, no skips. Pre-import bootstrap verifies all project sources; every checkpoint hashes before development/final CPU load. Original/adapted actual forward outputs match for 40 declared development rows across both sizes/all grid lengths (max error 0, atol 1e-6/rtol 1e-5); same architecture/weights/Torch/host, so adapter qualification rather than independent model replication.
A fresh-context author implemented scoring/public-edge traversal from the contract without reading old scorers. Both reducers agree on all 187,200 live forward-logit rows. A separately authored offline verifier checks all 187,200 saved rows, hashes, coverage, graph provenance, targets and counters. Postchecks confirm 20 checkpoint identities, 144 globally fresh names, zero prior exact graph/name overlap, and unchanged original evidence. Independent artifact reviewers found no counter/report inconsistency.
Correction to earlier verifier claim: it independently recomputed metrics from retained predictions, but reused the generator's node oracle and had no logits. It did not independently reproduce model outputs or rederive argmax/margins/ties. Original construction did already cross-check node traces with a separate public-record traversal. The earlier 'batch/logit parity' phrase overstated tensor/metric tests. New live scoring is stronger, but full logits remain unretained; offline batch digests/check counts are metadata and cannot recover logits. The new offline counter audit also shares the secondary public oracle. These limits are explicit in INDEPENDENCE.md; no claim of fully independent model verification.
Freeze, costs and evidence
Source/protocol/exposure/checkpoint/admission/seed freeze commit
14091daprecedes panel generation; SHA256 aacd1f2ebbcdbda40dd104a9f95baebcb3f0647864435ff4a0a75b7c1445db36. Exposure includes all declared prior streams, SIZE001 and own fixtures; N4 remains saturated. Representative development calibration measured CPU and wall separately. Six graphs/size failed 2x admission; three passed at 136.868 CPU / 166.385 wall projected seconds plus safeguards. Final run 139.055 CPU / 149.047 wall; no retry, source tuning, optimizer update or GPU initialization.Accounted cumulative 366.826 CPU / 484.441 worker wall within original 600/900, one CPU, low priority, 2 GiB sampled group RSS stop, 8 GiB address cap, 32 MiB output / 2 MiB logs. This includes an explicit conservative 5 CPU / 5 wall charge for a brief metadata-only reconstruction outside the guardian; no model/framework call occurred there. Control-plane writing/review/native Git/API remains unmeasured. All owned workers reaped. Sampled group peaks, lock/cleanup receipts, every raw prediction, graph/word/name provenance, per-seed/per-graph tables, reviews and retained failures are committed. 293 Markdown files pass, ledger current, whitespace clean. No training, GPU, paid resources, access expansion, unrelated process control or merge.
Preserved SIZE001 short result: continued 10/10 versus original 2/10 N8 and 1/10 N16 through length 4, on its separate 12 graphs/size. Do not combine those panels' denominators.
Draft for review; no merge requested. Existing Elsie HTTPS identity; only per-command signing disable, global configuration unchanged.
Long report · Independence clarification · Execution receipt
Reviewed artifact-only closeout
Seed 619723 has seven logical failures (14 agreeing name views), clustered in three graphs. First errors occur at steps 10–14; all first wrong outputs select the query/start node and two trajectories end correctly. These are conditional output correlations, not evidence of a hidden reset or forgetting mechanism. No forward inference, model/panel rerun or training occurred in closeout.
See research/confirmation_closeout_v1/REPORT.md and retained FAILURES.json for denominators, verification limits and provenance. Exactly one ADAPTATION_RETENTION_CARD.md is proposed for assessment/admission; it has not been run and does not authorize training. Stop after integration/card.
Fresh independent result/code/card/report review: no integration blocker. Closeout tests: 2 passed; documentation: 295 files passed; results ledger: current at 30 runs. Final accounted cumulative resource totals: 373.3005299 CPU / 517.781080344 supervised worker-wall seconds, including conservative 5/5 upper charge, within the original 600/900 envelope. Control-plane Git/API/edit/review costs excluded. All owned workers reaped; no GPU used in closeout.
Merge under standing human authorization after required exact-head checks pass; no protection bypass.
WIP: Prepare independent binding confirmation (no model runs)to WIP: Fresh N8/N16 fixed-checkpoint confirmation (zero updates)WIP: Fresh N8/N16 fixed-checkpoint confirmation (zero updates)to WIP: Fresh size confirmation review and long-walk preparationWIP: Fresh size confirmation review and long-walk preparationto WIP: Qualified short/long fixed-checkpoint confirmation and verification limitsWIP: Qualified short/long fixed-checkpoint confirmation and verification limitsto Qualified short/long fixed-checkpoint confirmation and verification limits