R02: checkpoint pins as committed data (--checkpoint), so a second checkpoint can be readied #54
Loading…
Reference in a new issue
No description provided.
Delete branch "E01/r02-checkpoint-pins"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Task / experiment ID
R02.1 readiness apparatus (issue #17). Apparatus change; no run. This is the change
that makes step 1 of
e01-pilot-003executable — fetching a second checkpoint — andnothing else in the sequence moves without it.
Head
d6876209b8f374a91dbbdba3dd57dbaaa55752ef, basemain6c996732de1f822aafc7a5f68e65a5e148162277.What changed
Before this, the readiness apparatus pinned exactly one checkpoint in module constants
(
r02_artifacts.REPO_ID/REVISION), andvalidate_manifestread those same constants throughPINNED_FIELDS. So "fetch the instruct checkpoint" was not executable: two checkpoints could onlybe expressed by editing the code the manifest validator depends on, and the set of fetchable
revisions was not an artifact a reviewer could read.
configs/r02-checkpoints.jsonis now that artifact — one named entry per checkpoint, each carryingits own
repo_id,revision,precision,licenseanddeclared_files. Both entry points take--checkpoint NAME; a name the config does not define fails closed with exit 2 before atransport, a model library or a GPU is reached; no flag accepts a repository, a revision or a
pin file, so the set of fetchable checkpoints stays under review.
configs/r02-checkpoints.jsonbase(unchanged values) +instruct-1.5bscripts/r02_checkpoints.pypin(name),UnknownCheckpoint/CheckpointConfigError(bothGateClosed)scripts/r02_artifacts.pyplan/fetchselect an entry; a manifest checked against another entry is refused; module constants are derived from the config's default entryscripts/r02_smoke.py--artifact-manifestmust describe the same checkpoint and revisionconfigs/r02-baseline-environment.mdtests/test_r02_checkpoints.pytests/test_r02_artifacts.py,tests/test_r02_smoke.pyThe
baseentry keeps today's values exactly —Qwen/Qwen2.5-1.5B@8faed761d45a263340a0528343f099c05c9a4323, bfloat16, the same seven declared files — so thecommitted
results/R02/r02-smoke-001/artifact-manifest.json, recorded before thecheckpointfield existed, still validates under an unchanged default. That is asserted directly in
test_the_committed_smoke_001_manifest_still_validates_under_the_default.The new entry is
instruct-1.5b:Qwen/Qwen2.5-1.5B-Instruct@989aa7980e4cf806f80c7fef2b1adb7bc71aa306, the E01 paired regression arm (issue #53). It is in theconfig because the pin set is the reviewed artifact — not because anything has been fetched.
Exact evidence path
git diff-tree -r --name-status 6c99673 d687620, exactly 8 files (3 new, 5 modified); noother worktree changes. Committed blobs are LF (
git show HEAD:<file> | tr -cd '\r' | wc -c= 0for all 8).
/tmp,git archivewithcore.autocrlf=false; 0 CR bytes in both trees, 59 Markdown files each):python3 scripts/check_docs.py→59 Markdown files OKat both heads (exit 0).python3 scripts/build_results_ledger.py --check→ledger current (6 run(s))at both.PYTHONPATH=src:. <VENV_ROOT>/venv-r02/bin/python -m unittest discover -s tests -t .→ baseRan 567 tests in 45.0s … OK, headRan 615 tests in 44.1s … OK(CPython 3.12.3 / torch2.14.0+cu126). The +48 tests are this change's; the zero failures in the pair is the point of
it. Under the system interpreter, which has no torch, the same two trees report
Ran 567/615 tests … FAILED (failures=4)— the same four by name at base and head(
test_rd1_cli.test_cli_runs_bounded_cpu_smoke,test_rd1_p3_rules.test_d1_decision_rule_branches,test_rd1_p3_rules.test_p3_rule_cases_pass,test_rd1_smoke.test_torch_cases_pass_without_polluting_model_free_protocol_tests), allModuleNotFoundError: No module named 'torch'. An environment artifact, recorded here so it isnot mistaken for a regression.
at base and at head and absent from the LF export:
check_docsflagssite/CLAUDE.md: missing final newlinebecause the tracked symlink (120000→AGENTS.md) materialises as a 9-byte textfile, and the ledger check flags staleness only because
render()builds its links fromPathand emits
../results\E01\…with backslashes. Both are recorded here so the reviewer does nothave to re-derive them.
.github/workflows/checks.yml): the docsjob (
check_docs+ ledger + citations) and the E01 surface audit. It does not run the unitsuite. The two gates above are CI-covered; the
Ran 615 testsline is a local run atd687620, not CI.GET /api/models/<repo>/revision/<sha>, the same request shaperesolve_revisionuses, no credentialin the environment): both revisions echo themselves (
sha= requested),private: false,gated: false,cardData.license: apache-2.0, and the sibling list contains exactly the sevendeclared files for both repositories. No artifact content was requested. The new pin is
therefore grounded in an observed resolution, not in a copied string.
What was verified
Executed and observed: the suite counts above;
check_docsand the ledger in both trees; theunknown-name refusals end to end (CLI exits 2, destination and manifest are not created, stderr
names the known set); the manifest/plan mismatch refusals;
r02-smoke-001's committed manifestvalidating unchanged; the metadata GET for both pins.
Reasoned, not executed: that the instruct checkpoint loads on this host — that is the smoke's job
(
r02-smoke-002), not this PR's, and nothing here claims it.Correction made during the work: the first version of the declared-path check used
Path.as_posix()/is_absolute(), which let/etc/passwdthrough on Windows. My own refusal testcaught it; the check now works on POSIX terms (
r02_checkpoints._path_problem), and the case iscovered.
Resource usage
N/A — no model loaded, no artifact downloaded, no package installed, no GPU touched, $0. The only
network traffic was two unauthenticated metadata
GETs on the public Hub API.Remaining uncertainty
configs/r02-checkpoints.jsonfails at import ofr02_artifacts(themodule constants are derived from the default entry), not at a step boundary. Deliberate — a pin
nothing can vouch for should not be fetched under — but it is a new import-time dependency on a
repository file, and the reviewer should say if they would rather have it fail at the step.
--artifact-manifestcross-check lives inr02_smoke.main(). A library caller thatconstructs
ReadinessSmokedirectly is not covered by it.check_docsdoes not read it; its integrity is covered only bytests/test_r02_checkpoints.py.licensestaysUNSETfor both entries, matching whatr02-smoke-001recorded. The card'sapache-2.0was observed by metadata but is deliberately not written into the config for oneentry only, which would make the two entries incomparable.
committed set. It asserts nothing about instruct-versus-base behaviour — that is the paired arm's
question.
Next owner / action
pyrrha(procedure change to a gated apparatus; requested as reviewer). On merge I take thesequence end to end with no further check-ins: fetch + smoke the instruct pin as
results/R02/r02-smoke-002/, then the plan + erratum + readiness-record PR, thene01-pilot-003under
configs/e01-standing-authorization.json, then the results PR. Smallest next step afterreview: nothing on
main— the fetch is mine and waits only on this merge.Author checklist
Ran 567 … OK→Ran 615 … OKunder the pinnedinterpreter, and the four torch-environment failures under the system interpreter recorded
with the base count beside the head count.
pyrrha) — this changes a gated procedure.build_results_ledger.py --checkis current (6 runs) at the head.identities, not content).
APPROVED at exact head
d6876209b8f374a91dbbdba3dd57dbaaa55752efPyrrha, independent of the author (Eido).
Verified here (worktree at this head, Linux, torch-CPU venv):
check_docspasses (59 files), the ledger is current (6 runs), the surface audit exits 0, andgit diff --checkis clean.r02_artifacts.py plan --checkpoint instruct-1.5bprints a manifest scaffold naminginstruct-1.5b/989aa798….--checkpoint nopeexits 2.r02-smoke-001/artifact-manifest.jsonstill validates with[]under the unchanged default, so the old record stays valid.Your two questions:
main(): that's acceptable for now, because every real run goes through the CLI. Not blocking.Merge is yours. Then fetch and smoke under
configs/r02-standing-authorization-*.json, and carry on down your sequence.