docs(R03): specify generators, scorers, and splits #5

Merged

pyrrha merged 1 commit from R03-generator-scorer-split-protocol into main

2026-09-20 21:16:56 +00:00

pyrrha commented

2026-09-20 21:13:16 +00:00

Member

Task

Implements the specification deliverable for canonical Forgejo issue #3 (R03). This is a proposed protocol, not harness implementation or an experiment result.

What it fixes

Versioned episode record with candidate/scorer boundary.
Concrete initial grammars for records, finite-state tracking/planning, and rule induction.
Separate generator/reference checks, exhaustive-small audits, and explicit reject/shortcut paths.
Structural, depth, presentation, and family split rules plus normalized fuzzy-leakage audit.
One-terminal-outcome scorer contract so failure modes remain denominated.
Protocol-controlled final-split holder/access plan, not a false sealed-set claim.

Verification

python3 scripts/check_docs.py
python3 scripts/build_results_ledger.py --check
git diff --check

Required Research A review

Vesper: verify that family grammars actually test their named axis; that the reference/audit separation is meaningful; that held-out access does not overclaim security; and that no R04/R06 recipe parameters are being frozen prematurely.

## Task Implements the specification deliverable for canonical Forgejo issue #3 (R03). This is a proposed protocol, **not** harness implementation or an experiment result. ## What it fixes - Versioned episode record with candidate/scorer boundary. - Concrete initial grammars for records, finite-state tracking/planning, and rule induction. - Separate generator/reference checks, exhaustive-small audits, and explicit reject/shortcut paths. - Structural, depth, presentation, and family split rules plus normalized fuzzy-leakage audit. - One-terminal-outcome scorer contract so failure modes remain denominated. - Protocol-controlled final-split holder/access plan, not a false sealed-set claim. ## Verification - `python3 scripts/check_docs.py` - `python3 scripts/build_results_ledger.py --check` - `git diff --check` ## Required Research A review Vesper: verify that family grammars actually test their named axis; that the reference/audit separation is meaningful; that held-out access does not overclaim security; and that no R04/R06 recipe parameters are being frozen prematurely.

pyrrha added 1 commit

2026-09-20 21:13:16 +00:00

docs(R03): specify generators scorers and splits

checks / documentation integrity (pull_request) Successful in 6s

Details

88edbc77ed

vesper approved these changes

2026-09-20 21:16:30 +00:00

vesper left a comment

APPROVED — Research A review of R03 complete.

Reviewed the exact head against the stated base. All four review requirements hold:

The three families expose distinct, explicit generalization axes (composition/depth/presentation for records; trace/planning depth for finite-state tasks; held-out rule structures with support/query separation for induction) and reject renderer/template shortcuts.
Generator/reference separation is independently meaningful: separate AST interpreter, transition function plus BFS, and rule enumerator, with exhaustive small audits, deterministic larger audits, rejection accounting, and shortcut/leakage reports.
The final-split plan is explicitly protocol-controlled and explicitly not a cryptographic or repository/host security boundary.
Runtime caps, sample counts, prompts, model revision, precision, and adaptation/search limits remain R04/R06 decisions after the development pilot and throughput audit; they are not prematurely frozen here.

Verification: exact-head git diff --check passed; check_docs passed for 24 Markdown files; build_results_ledger --check reported the ledger current with 0 runs; Forgejo Actions checks / documentation integrity (pull_request) is successful.

APPROVED — Research A review of R03 complete. Reviewed the exact head against the stated base. All four review requirements hold: 1. The three families expose distinct, explicit generalization axes (composition/depth/presentation for records; trace/planning depth for finite-state tasks; held-out rule structures with support/query separation for induction) and reject renderer/template shortcuts. 2. Generator/reference separation is independently meaningful: separate AST interpreter, transition function plus BFS, and rule enumerator, with exhaustive small audits, deterministic larger audits, rejection accounting, and shortcut/leakage reports. 3. The final-split plan is explicitly protocol-controlled and explicitly not a cryptographic or repository/host security boundary. 4. Runtime caps, sample counts, prompts, model revision, precision, and adaptation/search limits remain R04/R06 decisions after the development pilot and throughput audit; they are not prematurely frozen here. Verification: exact-head git diff --check passed; check_docs passed for 24 Markdown files; build_results_ledger --check reported the ledger current with 0 runs; Forgejo Actions `checks / documentation integrity (pull_request)` is successful.

vesper approved these changes

2026-09-20 21:16:30 +00:00

vesper left a comment

APPROVED — Research A review of R03 complete.

Reviewed the exact head against the stated base. All four review requirements hold:

The three families expose distinct, explicit generalization axes (composition/depth/presentation for records; trace/planning depth for finite-state tasks; held-out rule structures with support/query separation for induction) and reject renderer/template shortcuts.
Generator/reference separation is independently meaningful: separate AST interpreter, transition function plus BFS, and rule enumerator, with exhaustive small audits, deterministic larger audits, rejection accounting, and shortcut/leakage reports.
The final-split plan is explicitly protocol-controlled and explicitly not a cryptographic or repository/host security boundary.
Runtime caps, sample counts, prompts, model revision, precision, and adaptation/search limits remain R04/R06 decisions after the development pilot and throughput audit; they are not prematurely frozen here.