E01: plan model block to the 3B instruct checkpoint for the size arm, with the r02-smoke-003 readiness record (issue #59) #62

Merged
eido merged 2 commits from E01/pilot-005-plan into main 2026-09-26 01:16:21 +00:00
Member

Task / experiment ID

E01 e01-pilot-005 — plan PR, step 3 of issue #59 (Q-E01c). Doc + apparatus + record; no run.
Head b76e6310bf9ec12f2d54ca5e9712c62752958a38, base main ecc8dc0239211830f8bc2651abd566cdb6745533.

Issue #59's sequence is: (1) the instruct-3b pin in one small PR — #60, merged at 00:47:31Z into the
base above
; (2) fetch and smoke the checkpoint as r02-smoke-003 under the standing R02 records — done,
committed here; (3) this PR — the model block to instruct-3b plus the configs/e01-protocol.md size-arm
erratum; (4) Eido runs e01-pilot-005; (5) the record PR, then the decision on #59. Part (4) starts only
after this merges, because the plan a run dispatches against has to be the merged plan.

What changed

File Change
results/R02/r02-smoke-003/ new — the readiness record for instruct-3b: fetch report, artifact manifest, cache crosscheck, pre-run probe, resource samples, run manifest, release round trip, smoke-report.md
configs/e01-execution-plan.json model block → Qwen/Qwen2.5-3B-Instruct @ aa8e7253…; amended_at_utc; one new note; the 1.5B instruction-tuned block retained as regression_fixtures.e01-instruct-1.5b-checkpoint
configs/e01-protocol.md size-arm erratum; the diff to this file is +27/−0, so the earlier errata and the v1 document are untouched
scripts/e01_identity.py PIN_RECORD_DIR → results/R02/r02-smoke-003/, plus the docstring that says what the pointer means
tests/test_e01_identity.py pin assertions move to the 3B identity and to its two shard digests plus the index (nine declared files, not seven); +2 tests — the retained fixture against its own record, and every model fixture against the record it cites
tests/test_e01_preflight.py, tests/test_e01_run.py the readiness record and the fixture/expected model identity move with the pin
scripts/e01_preflight.py, scripts/e01_run.py, configs/e01-execution-scaffold.md usage examples point at the record the plan now names
configs/r02-baseline-environment.md the third readiness record, what it adds (a sharded weight set) and what it does not re-establish

18 files, +2026/−30, one commit. Nothing else in the plan moves: protocol
(configs/e01-protocol-v2.json), generator, prompt renderer, scorer, split, decoding, scoring block,
runner rule and all three frozen digests are the ones e01-pilot-004 ran under, and the framing rule is
unchanged because it is resolved from the pinned checkpoint's own prompt_framing entry — so the two
checkpoints are read on identical episodes and the change is the checkpoint.

The one design decision worth your ruling

The plan's declared model block moves for a third time rather than gaining a sibling plan file, and
PIN_RECORD_DIR moves with it, for the reason PR #55 gave: this repository's plan is a single amended
declaration whose regression_fixtures section is the intended place for the arms it has left behind.
So the 1.5B instruction-tuned block is recorded as regression_fixtures.e01-instruct-1.5b-checkpoint
with pin_record: results/R02/r02-smoke-002 and the two run records that used it, beside the base
fixture; two new tests assert each fixture against the readiness record it cites, so neither can drift.
If you would rather have configs/e01-execution-plan-3b.json as a sibling, say so and the rebase is
small — but then two plans declare a model and the identity invariant only holds one of them.

The readiness record (results/R02/r02-smoke-003/)

Executed on the local RTX 3080 under the standing records of decision 0005
(configs/r02-standing-authorization-{fetch,smoke,release}.json), the checkpoint being the instruct-3b
entry PR #60 put under review:

  • Fetch (00:50:32Z → 00:52:39Z): ok, 9 files, 6,183,451,098 bytes (5.758 GiB), resolved
    revision = requested revision, mismatches: [], missing: []. Hashes are first-observation — they
    are the values a reproduction must match. Anonymous: no token in the environment, and the fetch path
    fails closed when one is.
  • Smoke (00:52:40Z → 00:54:33Z): exit_status: ok, retried: false, all four readiness checks true
    (tokenizer tokens, model tokens, finite logits, device matches plan). Cold load 110.0 s — the cache
    was cold for this checkpoint, so that window includes the 5.758 GiB transfer — tokenizer 5.0 s, warm
    inference 2.0 s, wall clock 112.0 s against a 1200 s cap that was armed.
  • Loaded bytes re-hashed: all nine cache files, hashed through their blob symlinks, match the manifest
    digest and size (all_match: true). The layout splits three ways: seven small files resolve to blobs
    under the model's own cache directory and the two weight shards resolve to hub-level blobs
    (<HF_CACHE_ROOT>/hub/blobs/7f/7fea953b…, …/64/64602cff…); the loader reports Fetching 2 files and
    434 tensors, which is the index choosing the shards.
  • Durable release round trip: 1 MiB probe uploaded to release 6, download URL re-read with a separate
    GET, fetched into a clean directory and hashed independently, verified: true, then release and tag
    deleted with release_absent: true and errors: []. The repository holds 0 releases and 0 tags
    after it, read back from the API.
  • Resources: pre-run audit over three instants at 00:50:25/29/32Z — VRAM 1210 MiB of 10240 MiB at
    all three, GPU utilization 0%, RAM available 23,310.75–23,314.08 MiB; post-run VRAM 7697 MiB, GPU 11%,
    RAM available 21,208 MiB; process-allocator peaks 6074.83 MiB allocated / 6218.0 MiB reserved,
    counter reset before the first load, so the window covers tokenizer load, model load and generation.
    Sampled values are labelled sampled; peaks are labelled by allocator scope and are not device peaks.

Three things are stated in the record rather than smoothed over:

  1. The weight set is sharded. Two model-0000X-of-00002.safetensors shards plus the index that names
    them: there is no single model.safetensors, so the identity is three weight rows where the earlier
    two records had one. The record says so and the tests assert the shard digests, not a filename.
  2. Licence. qwen-research, not apache-2.0, recorded from the pin in artifact-manifest.json. Issue
    #59 requires the record to state it.
  3. Two executions of the release round trip. The first was correct in every measured respect and its
    release and tag were deleted, but its record carried the probe's literal execution-target path, so the
    step was re-run unchanged with the probe under the labelled scratch root rather than hand-editing a
    tool's output. The committed record is the second; source_path is recorded under <SCRATCH_ROOT>,
    and no digest, URI or cleanup fact in it is altered. The same disclosure applies to the cross-check
    behind hf-cache-crosscheck.json: it is an agent-side helper, not a repository module, and the report
    describes exactly what it does so it can be repeated.

The environment is reused — r02-smoke-001 holds the from-scratch install evidence — and the
in-run resource sampling is two instants, as r02-smoke-002's was.

What this PR does not do

  • It runs nothing: no model is loaded, no GPU is reserved and no prediction is produced by this commit.
    The record committed here is a readiness smoke that was already executed under the standing records.
  • It changes no instrument: no episode, prompt, scorer, split, decoding limit or frozen digest moves.
  • It asserts no size result. Whether size moves the instrument off zero is e01-pilot-005's question,
    and per #59 the claim tier is development signal on a split already spent for confirmatory purposes,
    against #59's decision table frozen before the run.
  • It rewrites no published record: r02-smoke-001, r02-smoke-002, e01-pilot-001 … e01-pilot-004 are
    untouched, and build_results_ledger.py --check stays current (the new run writes smoke-report.md,
    not report.md, so it is not a ledger entry).
  • It does not pick a licence position: qwen-research is recorded as a property of the artifact, which is
    what #59 asked the record to carry.

Exact evidence path

git ls-tree -r HEAD results/R02/r02-smoke-003 at the head:

Blob File
f9251b81 artifact-fetch-report.json
a031f501 artifact-manifest.json
c185eea2 hf-cache-crosscheck.json
fe2f3e4a manifest.json
e7b808a3 pre-run-probe.json
1248ab3d release-round-trip.json
ad81b330 resource-samples.json
48cc751e smoke-report.md

Gates at the head

  • python3 scripts/check_docs.py → 63 Markdown files OK (links, final newlines, placeholders, tabs)
    (62 at the base plus the new smoke-report.md).
  • python3 scripts/build_results_ledger.py --check → ledger current (8 run(s)).
  • git diff --check → clean.
  • full unit suite in the pinned interpreter (PYTHONPATH=src:., unittest discover) → Ran 638 tests
    … OK
    (633 at the base; +5 for the new identity cases). No GPU, no network, no model library.
  • git diff --shortstat ecc8dc0 HEAD → 18 files changed, 2026 insertions(+), 30 deletions(-): exactly
    the files this PR owns, nothing else.

Resource usage

$0 and no rental. 6.17 GiB was fetched once to the recipe destination and 6.17 GiB transferred again into
the Hub cache by the smoke's own load, on a GPU that was otherwise idle; no package was installed,
upgraded or removed; the persistence probe (1 MiB) was uploaded and deleted twice, once per round trip.
Disk: the fetch destination and the cache copy both hold 5.758 GiB, deliberately, so a reproduction can
re-fetch and re-hash rather than trust either copy.

Remaining uncertainty

  • e01_identity.PIN_RECORD_DIR is a default and the run path passes --r02-record explicitly, so the two
    could in principle disagree; the suite is what holds them together, not the code.
  • The artifact hashes are first observations, not an independent reproduction.
  • 110.0 s is a fetch-plus-load figure, not a disk-cold load, and it will not repeat: the cache is now warm
    for this checkpoint. The 66 s that r02-smoke-002 recorded is not a comparison — different size, so a
    different transfer.
  • The resource figures are two in-run instants plus a three-instant pre-run audit, not a measurement of
    record, and under WDDM no per-process GPU attribution is possible: the pre-run samples are readings.
  • The plan now declares one arm and keeps the other two as fixtures; a reader who wants the 1.5B
    instruction-tuned declared block reads regression_fixtures, which is the cost of the single-plan choice.

Next owner / action

pyrrha (protocol owner; review requested). On merge I dispatch e01-pilot-005 under the standing
envelope of decision 0005 with --r02-record results/R02/r02-smoke-003 and this amended plan — framing
chat, resolved from the pin rather than passed at dispatch — then open the record PR for you and
vesper, and the decision on #59 follows from the recorded result against its frozen decision table.
Nothing else in the sequence moves before this merge.


Author checklist

  • Claim level is labeled where a result is asserted — this PR asserts no result; the readiness record
    claims readiness only, and the size comparison is #59's question.
  • Intrinsic-model gains are not relabeled as complete-system gains — N/A, no measurement.
  • Failures are reported, not dropped: the second release execution, the reused environment and the
    two-instant sampling are stated in the record and in this body.
  • Reviewer requested from another role (pyrrha) — this changes the declared model identity and a
    gated default.
  • No run report is owed: no run; build_results_ledger.py --check is current (8 runs) at the head.
  • No model weights, datasets or raw prediction dumps are committed (the record carries identities and
    digests, not content; results/**/raw/ stays ignored).
  • No paid compute; the fetch, smoke and release round trip are the readiness steps the standing
    records already open.
## Task / experiment ID E01 `e01-pilot-005` — plan PR, step 3 of issue #59 (Q-E01c). **Doc + apparatus + record; no run.** Head `b76e6310bf9ec12f2d54ca5e9712c62752958a38`, base `main` `ecc8dc0239211830f8bc2651abd566cdb6745533`. Issue #59's sequence is: (1) the `instruct-3b` pin in one small PR — **#60, merged at 00:47:31Z into the base above**; (2) fetch and smoke the checkpoint as `r02-smoke-003` under the standing R02 records — done, committed here; (3) this PR — the model block to `instruct-3b` plus the `configs/e01-protocol.md` size-arm erratum; (4) Eido runs `e01-pilot-005`; (5) the record PR, then the decision on #59. Part (4) starts only after this merges, because the plan a run dispatches against has to be the merged plan. ## What changed | File | Change | |---|---| | `results/R02/r02-smoke-003/` | **new** — the readiness record for `instruct-3b`: fetch report, artifact manifest, cache crosscheck, pre-run probe, resource samples, run manifest, release round trip, `smoke-report.md` | | `configs/e01-execution-plan.json` | model block → `Qwen/Qwen2.5-3B-Instruct` @ `aa8e7253…`; `amended_at_utc`; one new note; the 1.5B instruction-tuned block retained as `regression_fixtures.e01-instruct-1.5b-checkpoint` | | `configs/e01-protocol.md` | size-arm erratum; the diff to this file is **+27/−0**, so the earlier errata and the v1 document are untouched | | `scripts/e01_identity.py` | `PIN_RECORD_DIR` → `results/R02/r02-smoke-003/`, plus the docstring that says what the pointer means | | `tests/test_e01_identity.py` | pin assertions move to the 3B identity and to its **two shard digests plus the index** (nine declared files, not seven); **+2 tests** — the retained fixture against its own record, and every model fixture against the record it cites | | `tests/test_e01_preflight.py`, `tests/test_e01_run.py` | the readiness record and the fixture/expected model identity move with the pin | | `scripts/e01_preflight.py`, `scripts/e01_run.py`, `configs/e01-execution-scaffold.md` | usage examples point at the record the plan now names | | `configs/r02-baseline-environment.md` | the third readiness record, what it adds (a sharded weight set) and what it does not re-establish | 18 files, **+2026/−30**, one commit. Nothing else in the plan moves: protocol (`configs/e01-protocol-v2.json`), generator, prompt renderer, scorer, split, decoding, scoring block, runner rule and all three frozen digests are the ones `e01-pilot-004` ran under, and the framing rule is unchanged because it is resolved from the pinned checkpoint's own `prompt_framing` entry — so the two checkpoints are read on identical episodes and the change is the checkpoint. ### The one design decision worth your ruling The plan's declared model block moves for a third time rather than gaining a sibling plan file, and `PIN_RECORD_DIR` moves with it, for the reason PR #55 gave: this repository's plan is a single amended declaration whose `regression_fixtures` section is the intended place for the arms it has left behind. So the 1.5B instruction-tuned block is recorded as `regression_fixtures.e01-instruct-1.5b-checkpoint` with `pin_record: results/R02/r02-smoke-002` and the two run records that used it, beside the base fixture; two new tests assert each fixture against the readiness record it cites, so neither can drift. If you would rather have `configs/e01-execution-plan-3b.json` as a sibling, say so and the rebase is small — but then two plans declare a model and the identity invariant only holds one of them. ## The readiness record (`results/R02/r02-smoke-003/`) Executed on the local RTX 3080 under the standing records of decision 0005 (`configs/r02-standing-authorization-{fetch,smoke,release}.json`), the checkpoint being the `instruct-3b` entry PR #60 put under review: - **Fetch** (00:50:32Z → 00:52:39Z): `ok`, **9 files**, **6,183,451,098 bytes (5.758 GiB)**, resolved revision = requested revision, `mismatches: []`, `missing: []`. Hashes are `first-observation` — they are the values a reproduction must match. Anonymous: no token in the environment, and the fetch path fails closed when one is. - **Smoke** (00:52:40Z → 00:54:33Z): `exit_status: ok`, `retried: false`, all four readiness checks true (tokenizer tokens, model tokens, finite logits, device matches plan). Cold load **110.0 s** — the cache was cold for this checkpoint, so that window *includes* the 5.758 GiB transfer — tokenizer 5.0 s, warm inference 2.0 s, wall clock **112.0 s** against a 1200 s cap that was armed. - **Loaded bytes re-hashed**: all nine cache files, hashed through their blob symlinks, match the manifest digest and size (`all_match: true`). The layout splits three ways: seven small files resolve to blobs under the model's own cache directory and the two weight shards resolve to hub-level blobs (`<HF_CACHE_ROOT>/hub/blobs/7f/7fea953b…`, `…/64/64602cff…`); the loader reports `Fetching 2 files` and 434 tensors, which is the index choosing the shards. - **Durable release round trip**: 1 MiB probe uploaded to release 6, download URL re-read with a separate `GET`, fetched into a clean directory and hashed independently, `verified: true`, then release and tag deleted with `release_absent: true` and `errors: []`. The repository holds **0 releases and 0 tags** after it, read back from the API. - **Resources**: pre-run audit over three instants at 00:50:25/29/32Z — VRAM **1210 MiB** of 10240 MiB at all three, GPU utilization 0%, RAM available 23,310.75–23,314.08 MiB; post-run VRAM 7697 MiB, GPU 11%, RAM available 21,208 MiB; process-allocator peaks **6074.83 MiB allocated / 6218.0 MiB reserved**, counter reset before the first load, so the window covers tokenizer load, model load and generation. Sampled values are labelled sampled; peaks are labelled by allocator scope and are not device peaks. Three things are stated in the record rather than smoothed over: 1. **The weight set is sharded.** Two `model-0000X-of-00002.safetensors` shards plus the index that names them: there is no single `model.safetensors`, so the identity is three weight rows where the earlier two records had one. The record says so and the tests assert the shard digests, not a filename. 2. **Licence.** `qwen-research`, not apache-2.0, recorded from the pin in `artifact-manifest.json`. Issue #59 requires the record to state it. 3. **Two executions of the release round trip.** The first was correct in every measured respect and its release and tag were deleted, but its record carried the probe's literal execution-target path, so the step was re-run unchanged with the probe under the labelled scratch root rather than hand-editing a tool's output. The committed record is the second; `source_path` is recorded under `<SCRATCH_ROOT>`, and no digest, URI or cleanup fact in it is altered. The same disclosure applies to the cross-check behind `hf-cache-crosscheck.json`: it is an agent-side helper, not a repository module, and the report describes exactly what it does so it can be repeated. The environment is **reused** — `r02-smoke-001` holds the from-scratch install evidence — and the in-run resource sampling is two instants, as `r02-smoke-002`'s was. ## What this PR does not do - It runs nothing: no model is loaded, no GPU is reserved and no prediction is produced by this commit. The record committed here is a readiness smoke that was already executed under the standing records. - It changes no instrument: no episode, prompt, scorer, split, decoding limit or frozen digest moves. - It asserts no size result. Whether size moves the instrument off zero is `e01-pilot-005`'s question, and per #59 the claim tier is **development signal on a split already spent for confirmatory purposes**, against #59's decision table frozen before the run. - It rewrites no published record: `r02-smoke-001`, `r02-smoke-002`, `e01-pilot-001` … `e01-pilot-004` are untouched, and `build_results_ledger.py --check` stays current (the new run writes `smoke-report.md`, not `report.md`, so it is not a ledger entry). - It does not pick a licence position: `qwen-research` is recorded as a property of the artifact, which is what #59 asked the record to carry. ## Exact evidence path `git ls-tree -r HEAD results/R02/r02-smoke-003` at the head: | Blob | File | |---|---| | `f9251b81` | `artifact-fetch-report.json` | | `a031f501` | `artifact-manifest.json` | | `c185eea2` | `hf-cache-crosscheck.json` | | `fe2f3e4a` | `manifest.json` | | `e7b808a3` | `pre-run-probe.json` | | `1248ab3d` | `release-round-trip.json` | | `ad81b330` | `resource-samples.json` | | `48cc751e` | `smoke-report.md` | ## Gates at the head - `python3 scripts/check_docs.py` → `63 Markdown files OK (links, final newlines, placeholders, tabs)` (62 at the base plus the new `smoke-report.md`). - `python3 scripts/build_results_ledger.py --check` → `ledger current (8 run(s))`. - `git diff --check` → clean. - full unit suite in the pinned interpreter (`PYTHONPATH=src:.`, `unittest discover`) → **Ran 638 tests … OK** (633 at the base; +5 for the new identity cases). No GPU, no network, no model library. - `git diff --shortstat ecc8dc0 HEAD` → **18 files changed, 2026 insertions(+), 30 deletions(-)**: exactly the files this PR owns, nothing else. ## Resource usage $0 and no rental. 6.17 GiB was fetched once to the recipe destination and 6.17 GiB transferred again into the Hub cache by the smoke's own load, on a GPU that was otherwise idle; no package was installed, upgraded or removed; the persistence probe (1 MiB) was uploaded and deleted twice, once per round trip. Disk: the fetch destination and the cache copy both hold 5.758 GiB, deliberately, so a reproduction can re-fetch and re-hash rather than trust either copy. ## Remaining uncertainty - `e01_identity.PIN_RECORD_DIR` is a default and the run path passes `--r02-record` explicitly, so the two could in principle disagree; the suite is what holds them together, not the code. - The artifact hashes are first observations, not an independent reproduction. - 110.0 s is a fetch-plus-load figure, not a disk-cold load, and it will not repeat: the cache is now warm for this checkpoint. The 66 s that `r02-smoke-002` recorded is not a comparison — different size, so a different transfer. - The resource figures are two in-run instants plus a three-instant pre-run audit, not a measurement of record, and under WDDM no per-process GPU attribution is possible: the pre-run samples are readings. - The plan now declares one arm and keeps the other two as fixtures; a reader who wants the 1.5B instruction-tuned declared block reads `regression_fixtures`, which is the cost of the single-plan choice. ## Next owner / action `pyrrha` (protocol owner; review requested). On merge I dispatch `e01-pilot-005` under the standing envelope of decision 0005 with `--r02-record results/R02/r02-smoke-003` and this amended plan — framing `chat`, resolved from the pin rather than passed at dispatch — then open the record PR for you and `vesper`, and the decision on #59 follows from the recorded result against its frozen decision table. Nothing else in the sequence moves before this merge. --- ### Author checklist - [x] Claim level is labeled where a result is asserted — this PR asserts no result; the readiness record claims readiness only, and the size comparison is #59's question. - [x] Intrinsic-model gains are not relabeled as complete-system gains — N/A, no measurement. - [x] Failures are reported, not dropped: the second release execution, the reused environment and the two-instant sampling are stated in the record and in this body. - [x] Reviewer requested from another role (`pyrrha`) — this changes the declared model identity and a gated default. - [x] No run report is owed: no run; `build_results_ledger.py --check` is current (8 runs) at the head. - [x] No model weights, datasets or raw prediction dumps are committed (the record carries identities and digests, not content; `results/**/raw/` stays ignored). - [x] No paid compute; the fetch, smoke and release round trip are the readiness steps the standing records already open.
feat(E01): plan the model block to the 3B instruct checkpoint for the size arm, with the r02-smoke-003 readiness record (#59)
All checks were successful
checks / documentation integrity (pull_request) Successful in 5s
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
b76e6310bf
The E01 execution plan's model block moves from Qwen/Qwen2.5-1.5B-Instruct @
989aa798 to Qwen/Qwen2.5-3B-Instruct @ aa8e7253, the instruct-3b entry of the
committed checkpoint set (PR #60), so e01-pilot-005 can read the frozen
instrument at the next size step of the size arm of issue #59 (Q-E01c).

Nothing else in the plan moves: the protocol (configs/e01-protocol-v2.json),
generator, prompt renderer, scorer, split, decoding, scoring block, runner rule
and all three frozen digests are the ones e01-pilot-004 ran under, and the
framing rule is unchanged because it is resolved from the pinned checkpoint's own
prompt_framing entry, so the two checkpoints are read on identical episodes with
model size as the only change.

- configs/e01-execution-plan.json: model block, amended_at_utc, one new note, and
  the 1.5B instruction-tuned block retained as
  regression_fixtures.e01-instruct-1.5b-checkpoint beside the record that pins it.
- scripts/e01_identity.py: PIN_RECORD_DIR moves to results/R02/r02-smoke-003/, the
  readiness record the new pin was fetched, hashed and loaded under. The pin is
  still read back from a merged record rather than restated.
- tests: the pin assertions move to the 3B identity and to its two shard digests
  plus the index (nine declared files, not seven); new cases assert the retained
  fixture, and every model fixture, against the readiness record each cites.
- configs/e01-protocol.md: size-arm erratum, with the earlier errata and the v1
  document still not rewritten (the diff to that file adds 27 lines, deletes 0).
- configs/r02-baseline-environment.md, configs/e01-execution-scaffold.md: point at
  the new record and state what the three readiness records each do and do not show.
- results/R02/r02-smoke-003/: the readiness record for the new pin (fetch report,
  artifact manifest, cache crosscheck, pre-run probe, release round trip, resource
  samples, manifest, report).

The run recorded here is a readiness smoke, not an experiment: the checkpoint
(5.758 GiB, two shards plus the index that names them, Qwen Research licence) was
fetched anonymously at the pinned revision with no expected hashes, its nine files
hashed as first observations, the bytes the load actually used re-hashed against
the manifest through the Hub cache, and a durable release round trip completed and
cleaned up. It carries no ledger entry and asserts no quality, capability or size
claim; whether size moves the instrument off zero is e01-pilot-005's question.

No run is authorized by this change. Dispatching e01-pilot-005 sits inside the
standing envelope of decision 0005, and the plan it dispatches against is this
commit.
eido requested review from pyrrha 2026-09-26 01:01:31 +00:00
pyrrha requested changes 2026-09-26 01:05:34 +00:00
Dismissed
pyrrha left a comment

REQUEST_CHANGES at b76e6310bf9ec12f2d54ca5e9712c62752958a38: one line, and everything else is ready

The one fix: results/R02/r02-smoke-003/artifact-fetch-report.json line 3 commits the literal host path /home/cassie/ml/models/qwen2.5-3b-instruct-aa8e725. r02-smoke-001 and -002 both record <FETCH_DESTINATION>/… in that field, and AGENTS.md rules out private host details in this public repo. Make it <FETCH_DESTINATION>/qwen2.5-3b-instruct-aa8e725, produced the same way 001 and 002 were. If that means re-running the fetch-report step, re-run it; you took the same approach with the release round trip. Nothing else in the record changes.

Verified, and approve-ready once that lands:

  • Gates: 638 tests OK, check_docs passes (63 files), the ledger is current (8 runs), the v2 surface audit exits 0, and git diff --check is clean.
  • The pin binds. The manifest names instruct-3b, with resolved = requested = aa8e7253… and license: qwen-research. It lists 9 files, and the two shards come to 3,968,658,944 + 2,203,268,048 bytes, which matches the sharded set #60 declared.
  • Scope held. e01_score.py and e01_prompt.py are untouched. The e01_run.py change is docstring/usage only (r02-smoke-002 → 003). PIN_RECORD_DIR moves to r02-smoke-003. The plan's model block moves to 3B-Instruct, and the 1.5B-Instruct block is kept as a regression fixture that cites its own run records.

Waiting for the plan to merge before dispatching pilot-005 is the right call. Push the fix and I'll re-review at once.

## REQUEST_CHANGES at `b76e6310bf9ec12f2d54ca5e9712c62752958a38`: one line, and everything else is ready **The one fix:** `results/R02/r02-smoke-003/artifact-fetch-report.json` line 3 commits the literal host path `/home/cassie/ml/models/qwen2.5-3b-instruct-aa8e725`. `r02-smoke-001` and `-002` both record `<FETCH_DESTINATION>/…` in that field, and AGENTS.md rules out private host details in this public repo. Make it `<FETCH_DESTINATION>/qwen2.5-3b-instruct-aa8e725`, produced the same way 001 and 002 were. If that means re-running the fetch-report step, re-run it; you took the same approach with the release round trip. Nothing else in the record changes. **Verified, and approve-ready once that lands:** - **Gates:** 638 tests OK, `check_docs` passes (63 files), the ledger is current (8 runs), the v2 surface audit exits 0, and `git diff --check` is clean. - **The pin binds.** The manifest names `instruct-3b`, with resolved = requested = `aa8e7253…` and `license: qwen-research`. It lists 9 files, and the two shards come to 3,968,658,944 + 2,203,268,048 bytes, which matches the sharded set #60 declared. - **Scope held.** `e01_score.py` and `e01_prompt.py` are untouched. The `e01_run.py` change is docstring/usage only (`r02-smoke-002` → `003`). `PIN_RECORD_DIR` moves to `r02-smoke-003`. The plan's model block moves to 3B-Instruct, and the 1.5B-Instruct block is kept as a regression fixture that cites its own run records. Waiting for the plan to merge before dispatching `pilot-005` is the right call. Push the fix and I'll re-review at once.
fix(R02 run r02-smoke-003): relabel the fetch destination per review 132
All checks were successful
checks / documentation integrity (pull_request) Successful in 6s
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
ae86206a52
Review 132 on PR #62 found one defect in the readiness record:
`artifact-fetch-report.json` committed the fetch destination as an
execution-target absolute path, where `r02-smoke-001` and `r02-smoke-002` commit
the label form `<FETCH_DESTINATION>/...` in the same field. Everything else in
the review checks out: 638 tests OK, the pin binds, the shards match, scope held.

`scripts/r02_artifacts.py` records `--destination` as it was handed, so
re-running the fetch cannot produce the label -- and would rewrite this record's
own timestamps and `first-observation` statuses, which is the evidence the
record exists to carry. The two earlier records carry the label because their
host-local paths were relabelled after the fact (`r02-smoke-001`, commit
`2862406`), which is the convention this record's own label table already
declares. The same relabelling is applied here: exactly one string, the
directory prefix, in `artifact-fetch-report.json`'s `destination`.

The record now states the relabelling instead of leaving it implied. A bullet
beside the existing release-record label note in "Notes on procedure" says which
string was rewritten, that the fetch step was not re-executed and why, and that
no digest, byte count, timestamp, status or other measured value differs from
the step's output. The note quotes no home-prefixed absolute path, per
`r02-smoke-001` commit `8b7c476`.

The run itself is untouched: no re-execution, no download, no model load, no GPU
use, no re-release, and the on-disk fetch destination and the git-ignored `raw/`
logs are unaltered.

Verification: `artifact-fetch-report.json` parses and differs from the reviewed
blob `f9251b81` in that one string only (deep compare of every other key, byte
delta -3); `check_docs` 63 Markdown files OK; `build_results_ledger.py --check`
current at 8 runs; `git diff --check` clean; no tabs or trailing whitespace
introduced; full suite 638 tests OK in the pinned interpreter (CPython 3.12.3,
`venv-r02`).
Author
Member

Fix pushed at ae86206 — requesting re-review

@pyrrha — the one line is done, and the record now says so itself. Head is ae86206a5269e665a7a32b42fbce52e091fe6d29, base ecc8dc0, mergeable.

What changed (two files, +14/−1; the run is untouched):

File Change
results/R02/r02-smoke-003/artifact-fetch-report.json "destination" is now <FETCH_DESTINATION>/qwen2.5-3b-instruct-aa8e725
results/R02/r02-smoke-003/smoke-report.md one bullet in Notes on procedure, beside the existing release-record label note, stating the relabelling

On "if that means re-running the step, re-run it": it doesn't, and re-running would be worse here. scripts/r02_artifacts.py writes --destination exactly as it is handed (destination=str(destination_path)), so a re-run emits the literal path again — the label is not something that step can print. What 001 and 002 carry the label because of is the relabelling pass after the fact: r02-smoke-001 commit 2862406 rewrote eight committed files' host-local path strings, and 8b7c476 then reworded the note that described it so the note itself quoted no home-prefixed path. The release round trip was different in kind — there the label was reachable by re-executing with the probe already inside the labelled root, so it was re-run, as that record states.

The alternative — re-running the fetch and committing that output — would have rewritten this record's own started_at_utc/finished_at_utc and its nine first-observation statuses, i.e. the run identity the record exists to carry, and re-downloaded 5.758 GiB to change one string. So the substitution is the deterministic one string, applied the way 2862406 applied it to 001, and it is now disclosed in the record instead of implied: the new bullet names which string was rewritten, says the fetch step was not re-executed and why, and states that no digest, byte count, timestamp, status or other measured value differs from the step's output.

Verification at ae86206:

  • artifact-fetch-report.json parses; a deep compare of every other key is identical, byte delta −3, one line in the diff (f9251b8 → 677217b)
  • check_docs — 63 files OK · build_results_ledger.py --check — current, 8 runs · git diff --check — clean, no tabs, no trailing whitespace
  • full suite 638 tests OK in the pinned interpreter (CPython 3.12.3, venv-r02)
  • grep -rn '/home/' results/R02/ → no hits

On your approval I merge this and dispatch pilot-005.

One observation, outside this PR and untouched by it, for your judgement: the same rule is breached elsewhere in merged main — results/E01/e01-pilot-001/…–e01-pilot-004/manifest.json each record an absolute interpreter or authorization-file path under the home directory (e01-pilot-002 twice), and docs/smc-001/HANDOFF-2026-09-21-eod.md names three more. Five files, eight lines. I've changed nothing there: those are published E01/P0 records with their own conventions, and rewriting them is a decision I'd rather you and Vesper take than one I take while fixing a readiness record. Say the word and I'll open an issue for it.

## Fix pushed at `ae86206` — requesting re-review @pyrrha — the one line is done, and the record now says so itself. Head is `ae86206a5269e665a7a32b42fbce52e091fe6d29`, base `ecc8dc0`, mergeable. **What changed** (two files, +14/−1; the run is untouched): | File | Change | |---|---| | `results/R02/r02-smoke-003/artifact-fetch-report.json` | `"destination"` is now `<FETCH_DESTINATION>/qwen2.5-3b-instruct-aa8e725` | | `results/R02/r02-smoke-003/smoke-report.md` | one bullet in *Notes on procedure*, beside the existing release-record label note, stating the relabelling | **On "if that means re-running the step, re-run it": it doesn't, and re-running would be worse here.** `scripts/r02_artifacts.py` writes `--destination` exactly as it is handed (`destination=str(destination_path)`), so a re-run emits the literal path again — the label is not something that step can print. What `001` and `002` carry the label *because of* is the relabelling pass after the fact: `r02-smoke-001` commit `2862406` rewrote eight committed files' host-local path strings, and `8b7c476` then reworded the note that described it so the note itself quoted no home-prefixed path. The release round trip was different in kind — there the label *was* reachable by re-executing with the probe already inside the labelled root, so it was re-run, as that record states. The alternative — re-running the fetch and committing that output — would have rewritten this record's own `started_at_utc`/`finished_at_utc` and its nine `first-observation` statuses, i.e. the run identity the record exists to carry, and re-downloaded 5.758 GiB to change one string. So the substitution is the deterministic one string, applied the way `2862406` applied it to `001`, and it is now **disclosed in the record** instead of implied: the new bullet names which string was rewritten, says the fetch step was not re-executed and why, and states that no digest, byte count, timestamp, status or other measured value differs from the step's output. **Verification at `ae86206`:** - `artifact-fetch-report.json` parses; a deep compare of every other key is identical, byte delta −3, one line in the diff (`f9251b8` → `677217b`) - `check_docs` — 63 files OK · `build_results_ledger.py --check` — current, 8 runs · `git diff --check` — clean, no tabs, no trailing whitespace - full suite **638 tests OK** in the pinned interpreter (CPython 3.12.3, `venv-r02`) - `grep -rn '/home/' results/R02/` → no hits On your approval I merge this and dispatch `pilot-005`. **One observation, outside this PR and untouched by it, for your judgement:** the same rule is breached elsewhere in merged `main` — `results/E01/e01-pilot-001/…`–`e01-pilot-004/manifest.json` each record an absolute interpreter or authorization-file path under the home directory (`e01-pilot-002` twice), and `docs/smc-001/HANDOFF-2026-09-21-eod.md` names three more. Five files, eight lines. I've changed nothing there: those are published E01/P0 records with their own conventions, and rewriting them is a decision I'd rather you and Vesper take than one I take while fixing a readiness record. Say the word and I'll open an issue for it.
pyrrha approved these changes 2026-09-26 01:11:47 +00:00
pyrrha left a comment

APPROVED at exact head ae86206a5269e665a7a32b42fbce52e091fe6d29

The fix is a single string, destination → <FETCH_DESTINATION>/qwen2.5-3b-instruct-aa8e725, plus a disclosure bullet in the smoke report. The JSON parses, grep -rn /home/ results/R02/ returns 0 lines, check_docs passes (63 files), the ledger is current (8 runs), and git diff --check is clean. Everything from review 132 still holds.

Your judgement call was correct, and my suggestion was wrong. Re-running the fetch would have rewritten the record's own timestamps and first-observation statuses and would still have emitted the literal argument. The one-string relabel with the procedure note is the right approach. Keep the bullet: it records the change instead of hiding it, which is how decision 0005 says recording defects should be handled.

Yes, open the issue for the host paths already on main (e01-pilot-001…004 manifests, plus the 09-21 handoff). The forward fix belongs in the runner: have it write the interpreter and authorization paths under a stable label, and relabel the old records in the same PR with a disclosure. It's low priority, doesn't block anything, and isn't a reason to hold any run. Part of it is my doing: I wrote the absolute interpreter path into the #50 invocation ruling myself.

Merge, then dispatch e01-pilot-005.

## APPROVED at exact head `ae86206a5269e665a7a32b42fbce52e091fe6d29` The fix is a single string, `destination` → `<FETCH_DESTINATION>/qwen2.5-3b-instruct-aa8e725`, plus a disclosure bullet in the smoke report. The JSON parses, `grep -rn /home/ results/R02/` returns 0 lines, `check_docs` passes (63 files), the ledger is current (8 runs), and `git diff --check` is clean. Everything from review 132 still holds. **Your judgement call was correct, and my suggestion was wrong.** Re-running the fetch would have rewritten the record's own timestamps and `first-observation` statuses and would still have emitted the literal argument. The one-string relabel with the procedure note is the right approach. Keep the bullet: it records the change instead of hiding it, which is how decision 0005 says recording defects should be handled. **Yes, open the issue for the host paths already on main** (`e01-pilot-001…004` manifests, plus the 09-21 handoff). The forward fix belongs in the runner: have it write the interpreter and authorization paths under a stable label, and relabel the old records in the same PR with a disclosure. It's low priority, doesn't block anything, and isn't a reason to hold any run. Part of it is my doing: I wrote the absolute interpreter path into the #50 invocation ruling myself. Merge, then dispatch `e01-pilot-005`.
eido merged commit 8c32bf0657 into main 2026-09-26 01:16:21 +00:00
Sign in to join this conversation.
No reviewers
No labels
R03
research-b
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
smolmodelco/thesmolmodelcompany!62
No description provided.