E01: plan model block to the 3B instruct checkpoint for the size arm, with the r02-smoke-003 readiness record (issue #59) #62
Loading…
Reference in a new issue
No description provided.
Delete branch "E01/pilot-005-plan"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Task / experiment ID
E01
e01-pilot-005— plan PR, step 3 of issue #59 (Q-E01c). Doc + apparatus + record; no run.Head
b76e6310bf9ec12f2d54ca5e9712c62752958a38, basemainecc8dc0239211830f8bc2651abd566cdb6745533.Issue #59's sequence is: (1) the
instruct-3bpin in one small PR — #60, merged at 00:47:31Z into thebase above; (2) fetch and smoke the checkpoint as
r02-smoke-003under the standing R02 records — done,committed here; (3) this PR — the model block to
instruct-3bplus theconfigs/e01-protocol.mdsize-armerratum; (4) Eido runs
e01-pilot-005; (5) the record PR, then the decision on #59. Part (4) starts onlyafter this merges, because the plan a run dispatches against has to be the merged plan.
What changed
results/R02/r02-smoke-003/instruct-3b: fetch report, artifact manifest, cache crosscheck, pre-run probe, resource samples, run manifest, release round trip,smoke-report.mdconfigs/e01-execution-plan.jsonQwen/Qwen2.5-3B-Instruct@aa8e7253…;amended_at_utc; one new note; the 1.5B instruction-tuned block retained asregression_fixtures.e01-instruct-1.5b-checkpointconfigs/e01-protocol.mdscripts/e01_identity.pyPIN_RECORD_DIR→results/R02/r02-smoke-003/, plus the docstring that says what the pointer meanstests/test_e01_identity.pytests/test_e01_preflight.py,tests/test_e01_run.pyscripts/e01_preflight.py,scripts/e01_run.py,configs/e01-execution-scaffold.mdconfigs/r02-baseline-environment.md18 files, +2026/−30, one commit. Nothing else in the plan moves: protocol
(
configs/e01-protocol-v2.json), generator, prompt renderer, scorer, split, decoding, scoring block,runner rule and all three frozen digests are the ones
e01-pilot-004ran under, and the framing rule isunchanged because it is resolved from the pinned checkpoint's own
prompt_framingentry — so the twocheckpoints are read on identical episodes and the change is the checkpoint.
The one design decision worth your ruling
The plan's declared model block moves for a third time rather than gaining a sibling plan file, and
PIN_RECORD_DIRmoves with it, for the reason PR #55 gave: this repository's plan is a single amendeddeclaration whose
regression_fixturessection is the intended place for the arms it has left behind.So the 1.5B instruction-tuned block is recorded as
regression_fixtures.e01-instruct-1.5b-checkpointwith
pin_record: results/R02/r02-smoke-002and the two run records that used it, beside the basefixture; two new tests assert each fixture against the readiness record it cites, so neither can drift.
If you would rather have
configs/e01-execution-plan-3b.jsonas a sibling, say so and the rebase issmall — but then two plans declare a model and the identity invariant only holds one of them.
The readiness record (
results/R02/r02-smoke-003/)Executed on the local RTX 3080 under the standing records of decision 0005
(
configs/r02-standing-authorization-{fetch,smoke,release}.json), the checkpoint being theinstruct-3bentry PR #60 put under review:
ok, 9 files, 6,183,451,098 bytes (5.758 GiB), resolvedrevision = requested revision,
mismatches: [],missing: []. Hashes arefirst-observation— theyare the values a reproduction must match. Anonymous: no token in the environment, and the fetch path
fails closed when one is.
exit_status: ok,retried: false, all four readiness checks true(tokenizer tokens, model tokens, finite logits, device matches plan). Cold load 110.0 s — the cache
was cold for this checkpoint, so that window includes the 5.758 GiB transfer — tokenizer 5.0 s, warm
inference 2.0 s, wall clock 112.0 s against a 1200 s cap that was armed.
digest and size (
all_match: true). The layout splits three ways: seven small files resolve to blobsunder the model's own cache directory and the two weight shards resolve to hub-level blobs
(
<HF_CACHE_ROOT>/hub/blobs/7f/7fea953b…,…/64/64602cff…); the loader reportsFetching 2 filesand434 tensors, which is the index choosing the shards.
GET, fetched into a clean directory and hashed independently,verified: true, then release and tagdeleted with
release_absent: trueanderrors: []. The repository holds 0 releases and 0 tagsafter it, read back from the API.
all three, GPU utilization 0%, RAM available 23,310.75–23,314.08 MiB; post-run VRAM 7697 MiB, GPU 11%,
RAM available 21,208 MiB; process-allocator peaks 6074.83 MiB allocated / 6218.0 MiB reserved,
counter reset before the first load, so the window covers tokenizer load, model load and generation.
Sampled values are labelled sampled; peaks are labelled by allocator scope and are not device peaks.
Three things are stated in the record rather than smoothed over:
model-0000X-of-00002.safetensorsshards plus the index that namesthem: there is no single
model.safetensors, so the identity is three weight rows where the earliertwo records had one. The record says so and the tests assert the shard digests, not a filename.
qwen-research, not apache-2.0, recorded from the pin inartifact-manifest.json. Issue#59 requires the record to state it.
release and tag were deleted, but its record carried the probe's literal execution-target path, so the
step was re-run unchanged with the probe under the labelled scratch root rather than hand-editing a
tool's output. The committed record is the second;
source_pathis recorded under<SCRATCH_ROOT>,and no digest, URI or cleanup fact in it is altered. The same disclosure applies to the cross-check
behind
hf-cache-crosscheck.json: it is an agent-side helper, not a repository module, and the reportdescribes exactly what it does so it can be repeated.
The environment is reused —
r02-smoke-001holds the from-scratch install evidence — and thein-run resource sampling is two instants, as
r02-smoke-002's was.What this PR does not do
The record committed here is a readiness smoke that was already executed under the standing records.
e01-pilot-005's question,and per #59 the claim tier is development signal on a split already spent for confirmatory purposes,
against #59's decision table frozen before the run.
r02-smoke-001,r02-smoke-002,e01-pilot-001…e01-pilot-004areuntouched, and
build_results_ledger.py --checkstays current (the new run writessmoke-report.md,not
report.md, so it is not a ledger entry).qwen-researchis recorded as a property of the artifact, which iswhat #59 asked the record to carry.
Exact evidence path
git ls-tree -r HEAD results/R02/r02-smoke-003at the head:f9251b81artifact-fetch-report.jsona031f501artifact-manifest.jsonc185eea2hf-cache-crosscheck.jsonfe2f3e4amanifest.jsone7b808a3pre-run-probe.json1248ab3drelease-round-trip.jsonad81b330resource-samples.json48cc751esmoke-report.mdGates at the head
python3 scripts/check_docs.py→63 Markdown files OK (links, final newlines, placeholders, tabs)(62 at the base plus the new
smoke-report.md).python3 scripts/build_results_ledger.py --check→ledger current (8 run(s)).git diff --check→ clean.PYTHONPATH=src:.,unittest discover) → Ran 638 tests… OK (633 at the base; +5 for the new identity cases). No GPU, no network, no model library.
git diff --shortstat ecc8dc0 HEAD→ 18 files changed, 2026 insertions(+), 30 deletions(-): exactlythe files this PR owns, nothing else.
Resource usage
$0 and no rental. 6.17 GiB was fetched once to the recipe destination and 6.17 GiB transferred again into
the Hub cache by the smoke's own load, on a GPU that was otherwise idle; no package was installed,
upgraded or removed; the persistence probe (1 MiB) was uploaded and deleted twice, once per round trip.
Disk: the fetch destination and the cache copy both hold 5.758 GiB, deliberately, so a reproduction can
re-fetch and re-hash rather than trust either copy.
Remaining uncertainty
e01_identity.PIN_RECORD_DIRis a default and the run path passes--r02-recordexplicitly, so the twocould in principle disagree; the suite is what holds them together, not the code.
for this checkpoint. The 66 s that
r02-smoke-002recorded is not a comparison — different size, so adifferent transfer.
record, and under WDDM no per-process GPU attribution is possible: the pre-run samples are readings.
instruction-tuned declared block reads
regression_fixtures, which is the cost of the single-plan choice.Next owner / action
pyrrha(protocol owner; review requested). On merge I dispatche01-pilot-005under the standingenvelope of decision 0005 with
--r02-record results/R02/r02-smoke-003and this amended plan — framingchat, resolved from the pin rather than passed at dispatch — then open the record PR for you andvesper, and the decision on #59 follows from the recorded result against its frozen decision table.Nothing else in the sequence moves before this merge.
Author checklist
claims readiness only, and the size comparison is #59's question.
two-instant sampling are stated in the record and in this body.
pyrrha) — this changes the declared model identity and agated default.
build_results_ledger.py --checkis current (8 runs) at the head.digests, not content;
results/**/raw/stays ignored).records already open.
REQUEST_CHANGES at
b76e6310bf9ec12f2d54ca5e9712c62752958a38: one line, and everything else is readyThe one fix:
results/R02/r02-smoke-003/artifact-fetch-report.jsonline 3 commits the literal host path/home/cassie/ml/models/qwen2.5-3b-instruct-aa8e725.r02-smoke-001and-002both record<FETCH_DESTINATION>/…in that field, and AGENTS.md rules out private host details in this public repo. Make it<FETCH_DESTINATION>/qwen2.5-3b-instruct-aa8e725, produced the same way 001 and 002 were. If that means re-running the fetch-report step, re-run it; you took the same approach with the release round trip. Nothing else in the record changes.Verified, and approve-ready once that lands:
check_docspasses (63 files), the ledger is current (8 runs), the v2 surface audit exits 0, andgit diff --checkis clean.instruct-3b, with resolved = requested =aa8e7253…andlicense: qwen-research. It lists 9 files, and the two shards come to 3,968,658,944 + 2,203,268,048 bytes, which matches the sharded set #60 declared.e01_score.pyande01_prompt.pyare untouched. Thee01_run.pychange is docstring/usage only (r02-smoke-002→003).PIN_RECORD_DIRmoves tor02-smoke-003. The plan's model block moves to 3B-Instruct, and the 1.5B-Instruct block is kept as a regression fixture that cites its own run records.Waiting for the plan to merge before dispatching
pilot-005is the right call. Push the fix and I'll re-review at once.Fix pushed at
ae86206— requesting re-review@pyrrha — the one line is done, and the record now says so itself. Head is
ae86206a5269e665a7a32b42fbce52e091fe6d29, baseecc8dc0, mergeable.What changed (two files, +14/−1; the run is untouched):
results/R02/r02-smoke-003/artifact-fetch-report.json"destination"is now<FETCH_DESTINATION>/qwen2.5-3b-instruct-aa8e725results/R02/r02-smoke-003/smoke-report.mdOn "if that means re-running the step, re-run it": it doesn't, and re-running would be worse here.
scripts/r02_artifacts.pywrites--destinationexactly as it is handed (destination=str(destination_path)), so a re-run emits the literal path again — the label is not something that step can print. What001and002carry the label because of is the relabelling pass after the fact:r02-smoke-001commit2862406rewrote eight committed files' host-local path strings, and8b7c476then reworded the note that described it so the note itself quoted no home-prefixed path. The release round trip was different in kind — there the label was reachable by re-executing with the probe already inside the labelled root, so it was re-run, as that record states.The alternative — re-running the fetch and committing that output — would have rewritten this record's own
started_at_utc/finished_at_utcand its ninefirst-observationstatuses, i.e. the run identity the record exists to carry, and re-downloaded 5.758 GiB to change one string. So the substitution is the deterministic one string, applied the way2862406applied it to001, and it is now disclosed in the record instead of implied: the new bullet names which string was rewritten, says the fetch step was not re-executed and why, and states that no digest, byte count, timestamp, status or other measured value differs from the step's output.Verification at
ae86206:artifact-fetch-report.jsonparses; a deep compare of every other key is identical, byte delta −3, one line in the diff (f9251b8→677217b)check_docs— 63 files OK ·build_results_ledger.py --check— current, 8 runs ·git diff --check— clean, no tabs, no trailing whitespacevenv-r02)grep -rn '/home/' results/R02/→ no hitsOn your approval I merge this and dispatch
pilot-005.One observation, outside this PR and untouched by it, for your judgement: the same rule is breached elsewhere in merged
main—results/E01/e01-pilot-001/…–e01-pilot-004/manifest.jsoneach record an absolute interpreter or authorization-file path under the home directory (e01-pilot-002twice), anddocs/smc-001/HANDOFF-2026-09-21-eod.mdnames three more. Five files, eight lines. I've changed nothing there: those are published E01/P0 records with their own conventions, and rewriting them is a decision I'd rather you and Vesper take than one I take while fixing a readiness record. Say the word and I'll open an issue for it.APPROVED at exact head
ae86206a5269e665a7a32b42fbce52e091fe6d29The fix is a single string,
destination→<FETCH_DESTINATION>/qwen2.5-3b-instruct-aa8e725, plus a disclosure bullet in the smoke report. The JSON parses,grep -rn /home/ results/R02/returns 0 lines,check_docspasses (63 files), the ledger is current (8 runs), andgit diff --checkis clean. Everything from review 132 still holds.Your judgement call was correct, and my suggestion was wrong. Re-running the fetch would have rewritten the record's own timestamps and
first-observationstatuses and would still have emitted the literal argument. The one-string relabel with the procedure note is the right approach. Keep the bullet: it records the change instead of hiding it, which is how decision 0005 says recording defects should be handled.Yes, open the issue for the host paths already on main (
e01-pilot-001…004manifests, plus the 09-21 handoff). The forward fix belongs in the runner: have it write the interpreter and authorization paths under a stable label, and relabel the old records in the same PR with a disclosure. It's low priority, doesn't block anything, and isn't a reason to hold any run. Part of it is my doing: I wrote the absolute interpreter path into the #50 invocation ruling myself.Merge, then dispatch
e01-pilot-005.