X14: implement bounded procedural-transfer research apparatus #105

Merged
starsetbyte merged 5 commits from X14/procedural-transfer into main 2026-10-07 02:33:29 +00:00
Member

Implements the approved X14 W0–W6 apparatus and closes independent W7 engineering review. Adds versioned finite typed-record semantics and a separate reference implementation, exact equivalence and support-identifiability checks, exposure-wide transfer audits, immutable experience tapes and read-only panels, matched metered baselines, chronological acquisition/transfer/retention reporting, full cost/state accounting, and Windows Job Object containment for closed deterministic developer checks.

Validation: owned invocation 16 passed 30 tests, including six independently derived semantic/accounting fixtures. Invocation 17 exported the tiny deterministic fixture with 75,671 expanded primitive applications. All 17 retained invocations total 54.8523476999253 seconds wall and 54.140625 process-tree CPU seconds, with cleanup confirmed. Failures remain in the journal. git diff --cached --check passed before commit; only 20 X14 files were included. Source code was unchanged after the final suite; publication updated the progress record only.

The repository documentation checker has one pre-existing Windows checkout failure: site/CLAUDE.md is a tracked symlink materialized as a text stub without a final newline. The historical symlink is preserved. No broader model/GPU suite ran.

W8 remains disabled. This is apparatus engineering evidence, not scientific admission, eligible named scientific acceptance, external reproduction, or empirical performance. G1/G2, scientific populations/thresholds/horizon/confirmation and execution authority remain outstanding. No X05 rerun, model calls, fitting, benchmark population, GPU, paid compute or downloads occurred. No merge or deployment is requested.

Please retain this as a draft and obtain distinct-role review before any integration. Elsie is the publication identity authorized by Cassie; the independent engineering reviewer is not relabelled as a named scientific role.

Implements the approved X14 W0–W6 apparatus and closes independent W7 engineering review. Adds versioned finite typed-record semantics and a separate reference implementation, exact equivalence and support-identifiability checks, exposure-wide transfer audits, immutable experience tapes and read-only panels, matched metered baselines, chronological acquisition/transfer/retention reporting, full cost/state accounting, and Windows Job Object containment for closed deterministic developer checks. Validation: owned invocation 16 passed 30 tests, including six independently derived semantic/accounting fixtures. Invocation 17 exported the tiny deterministic fixture with 75,671 expanded primitive applications. All 17 retained invocations total 54.8523476999253 seconds wall and 54.140625 process-tree CPU seconds, with cleanup confirmed. Failures remain in the journal. `git diff --cached --check` passed before commit; only 20 X14 files were included. Source code was unchanged after the final suite; publication updated the progress record only. The repository documentation checker has one pre-existing Windows checkout failure: `site/CLAUDE.md` is a tracked symlink materialized as a text stub without a final newline. The historical symlink is preserved. No broader model/GPU suite ran. W8 remains disabled. This is apparatus engineering evidence, not scientific admission, eligible named scientific acceptance, external reproduction, or empirical performance. G1/G2, scientific populations/thresholds/horizon/confirmation and execution authority remain outstanding. No X05 rerun, model calls, fitting, benchmark population, GPU, paid compute or downloads occurred. No merge or deployment is requested. Please retain this as a draft and obtain distinct-role review before any integration. Elsie is the publication identity authorized by Cassie; the independent engineering reviewer is not relabelled as a named scientific role.
D1 finite-synthesis diagnostic for issue #92: the pre-committed kill criterion fires
All checks were successful
checks / documentation integrity (pull_request) Successful in 6s
checks / E01 candidate surface sufficiency (pull_request) Successful in 6s
checks / DMS-003 protocol freeze integrity (pull_request) Successful in 7s
checks / DMS-003 CPU successor integrity (pull_request) Successful in 50s
025451bdcc
Bounded CPU-only diagnostic (stdlib, no GPU, no WSL). Two files plus tests, per the
revised scope: D2-D4 are dropped and the single-bit case is the only thing measured.

WHAT WAS BUILT
- research/smc_dsl_diagnostic/smc_diagnostic.py -- finite domain (40 sequences over
  {0,1,2} of length 0..3), 72-program class (1- and 2-call compositions of CUT_X/CUT_Y
  with n in 0..3, no literal-output primitive), hidden-bit sandbox that returns
  truthful observable behavior only, goal generation by execution, extensional task
  spec, underdetermination detection, private diagnostic evaluator, seven arms.
- research/smc_dsl_diagnostic/test_smc_diagnostic.py -- 19 tests, all passing.
- research/smc_dsl_diagnostic/RESULTS.md -- the run, the accounting, and the negative.
- research/smc_dsl_diagnostic/results.jsonl -- 1800 episodes.

RESULT: THE KILL CRITERION FIRED. D1 is killed as a learning benchmark, and it
collapsed further than predicted -- to zero probes, not one. no_probe solves 72/72
tasks at 0.000 probes/episode; bounded_search also 72/72 at zero probes. With the full
finite I/O table exposed, the support pairs already pin the denotation on all 40
inputs, so resolving the hidden bit buys nothing. The information contract, not the
learning mechanism, determined the outcome. Every failure is an explicit `unresolved`;
no arm ever guessed.

Two secondary findings worth carrying forward:
- Resolution-before-action is a failure mode, not a safety property. Arms that require
  the hypothesis set to shrink to size 1 before committing score 0.000 at budget 0
  while no_probe scores 1.000 on identical tasks.
- Probe choice is not the binding constraint at this scale. fixed_degenerate
  (CUT_X([],0)) splits nothing and fails at every budget, but fixed_informative,
  active_max_partition, and random all reach 1.000 with a single probe. With a
  two-element hypothesis class one split is decisive, so max-partition selection has
  no room to beat a fixed good query.

MEASURED DOMAIN STRUCTURE (recorded, not assumed): 36 of 72 generators yield a
well-posed task and 36 do not; the 8 hypothesis-independent programs are all
constant-empty and are nonetheless well-posed, so identifiability of the denotation is
strictly weaker than identifiability of the program.

ROLES HELD SEPARATE: sandbox, teacher, and evaluator are distinct; the teacher is not
instantiated in D1. private_evaluate is not a candidate action, returns a bare boolean,
and never drips counterexamples. Feedback is OFF for this diagnostic, which does not
remove verified teacher experience from later escalation/acquisition experiments.
exact_known_h is privileged, non-deployable, and reported separately as a solvability
control -- never compared as a method.

ACCOUNTING: vector only, per-episode, no single-unit currency and no scalarization.
probes_executed, candidate_evals, cpu_wall_s, bytes_task_spec, support_pair_count,
resolved_hypothesis, status, success.

A LEAK I INTRODUCED AND FIXED, recorded rather than quietly patched: the first
implementation had the candidate-side consistency check consult task.hidden, i.e. read
the answer key. It is replaced by consistent_under(prog, hypotheses), and
test_candidate_solvers_never_touch_hidden_state nulls the attribute and re-runs a
non-privileged solver to prevent regression. Two of my own test assertions were also
wrong (about identity, and about which programs are well-posed); both were corrected to
the measured values rather than by loosening the assertion.

SCOPE LIMITS, STATED: this licenses nothing about weight updates, LoRA, or retrieval
dominance -- no trained arm exists and no adaptation is required at all. Retention,
paired deterioration, and margin are untested because there is nothing to retain. It
does not speak to the wider research program.

Not merged. Requesting independent review/reproduction and adversarial tests. The
prospective question and main research spec are Pyrrha's branch.
D1 corrected per review 194: full domain, four separated quantities, contract enforcement, honest accounting
All checks were successful
checks / documentation integrity (pull_request) Successful in 6s
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
checks / DMS-003 protocol freeze integrity (pull_request) Successful in 6s
checks / DMS-003 CPU successor integrity (pull_request) Successful in 50s
6ac7b8c3b0
Withdraws the zero-probe claim from 025451b as a selection artifact: well_posed
required one program consistent under every live environment and main() discarded
every task failing that precheck, so no_probe's 72/72 covered only the constant-empty
function. The full 146-task domain is now preserved and abstention replaces deletion.

Separates goal_identifiable / hypothesis_survivors / robust_no_probe / success.
All 146 tasks are goal-identifiable; 86 are robust-no-probe, 60 need exactly one
probe. Program class is now the 73-program class of PR #94 so the two declarations
are not conflated.

Program.is_valid is enforced inside private_evaluate before any denotation, and at
the sandbox boundary; type(n) is int rejects bool. Two independent semantics
implementations cross-check every execution.

Accounting repaired: candidate_evals is non-zero for every enumerating arm,
active lookahead is counted (640/episode), real survivor counts reported, and a test
asserts no scalar currency exists. 306/320 informative queries is reported as an
exhaustive finite fact, separately from the three-seed random rows.

Also fixes two bugs in my own first correction: bounded_search narrowed a local copy
of the hypothesis set so it never used its probe, and then accepted programs
consistent under only one hypothesis (solved 146 / success 73). Both produced
plausible status strings, which is the same failure mode as the original leak.

Tests 35/35, 3650 episodes, stdlib CPU, no GPU. Supersedes RESULTS.md, which is
retained unmodified for audit.
PR95 finding 1954: check the expansion cap before counting a candidate evaluation
All checks were successful
checks / documentation integrity (pull_request) Successful in 6s
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
checks / DMS-003 protocol freeze integrity (pull_request) Successful in 5s
checks / DMS-003 CPU successor integrity (pull_request) Successful in 50s
00f5b7f578
The inner re-scan in solve_bounded_search incremented candidate_evals and only then
checked max_expansions, so an expansion the cap had just prevented was charged. The
outer loop already had the correct order. Finding 1954 is correct.

Accounting-only: 35/35 tests still pass and the run is unchanged (146/86/60 intact,
bounded_search 0.589/1.000/1.000 at budgets 0/1/2, cand/ep 36.5 and 5.6).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
X14: implement bounded procedural-transfer research apparatus
All checks were successful
checks / documentation integrity (pull_request) Successful in 6s
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
checks / DMS-003 protocol freeze integrity (pull_request) Successful in 6s
checks / DMS-003 CPU successor integrity (pull_request) Successful in 50s
282239dda4
Implement W0-W6 and independently reviewed deterministic W7 fixtures. Keep empirical W8 disabled and scientific admission gates unset.
Merge branch 'main' into X14/procedural-transfer
All checks were successful
checks / documentation integrity (pull_request) Successful in 6s
checks / E01 candidate surface sufficiency (pull_request) Successful in 5s
checks / DMS-003 protocol freeze integrity (pull_request) Successful in 6s
checks / DMS-003 CPU successor integrity (pull_request) Successful in 50s
ffd9d6200d
starsetbyte changed title from WIP: X14: implement bounded procedural-transfer research apparatus to X14: implement bounded procedural-transfer research apparatus 2026-10-07 02:02:34 +00:00
starsetbyte scheduled this pull request to auto merge when all checks succeed 2026-10-07 02:05:45 +00:00
Sign in to join this conversation.
No reviewers
No labels
R03
research-b
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
smolmodelco/thesmolmodelcompany!105
No description provided.