feat(E01): surface audit — oracle-only literals and undeclared keys (#30 §2) #31

Merged
starsetbyte merged 3 commits from E01/surface-audit-gate into main 2026-09-21 15:28:48 +00:00
Member

Task / experiment ID

Issue #30 §2 — E01 surface revision. Additive tooling only: no protocol, scorer, task, prompt or split change, and nothing here dispatches anything. Related: reviews #61 and #62, issue comment #1006.

Cassie has since approved the direction this serves — keep Qwen/Qwen2.5-1.5B pinned, repair the surface so the expected output is derivable, re-freeze with a fresh final split, then continue toward a pilot. This PR is the instrument that makes "derivable" checkable instead of asserted, and it is the piece §2 claimed as mine rather than research-b's.

What changed

Two new files, nothing else:

  • scripts/e01_surface_audit.py — renders the exact prompt every frozen episode would receive and reports (1) oracle-only literals: reference string leaves that appear neither in the rendered prompt nor in the part of a declared-semantics block that the rendered prompt carries, and (2) the declared-versus-actual answer key set.
  • tests/test_e01_surface_audit.py — 47 tests: synthetic behaviour, the declared-template and declared-row-key forms, declaration visibility, exit-status semantics, determinism, and a regression pin on the frozen surface's numbers.

Read-only by construction: it imports no model library, opens no socket, writes no file, and regenerates episodes through scripts/e01_records.py, the same path the frozen manifest came from.

Correction after review #65

The first revision built its visible surface as the rendered prompt plus the audit config's declaration block, so a config-level declaration cleared both checks while the renderer put neither the block nor the fixed part of its convention in front of the candidate. Review #65 reproduced that as a false clean and blocked it. Both checks now credit a declaration only where the rendered prompt carries it: an entry the renderer omitted clears nothing, is reported by name, and counts as a finding for --require-clean.

Consequence for the surface revision, stated because it is a requirement rather than a preference: this audit certifies the surface, never the config. A repair that states the convention only in the protocol config will not clear it, because that does not change what the model is shown. The repair has to reach the rendered prompt.

Measured, with this PR's own code

Before — the frozen surface, protocol e01-records-v1:

$ python3 scripts/e01_surface_audit.py --config configs/e01-protocol.json
E01 surface audit: 32 episodes | flagged 15 (development 6, final 9) |
oracle-only literal occurrences 29 (3 distinct) | episodes with undeclared keys 15 |
declared semantics absent

$ python3 scripts/e01_surface_audit.py --config configs/e01-protocol.json --require-clean
exit=1

Each flagged episode is filter>derive>group>sort, its reference rows carry "id": "group:<key>", and the group operation object the model receives names only {kind, field, source, aggregate, output}. This reproduces §0 of #30 mechanically: 6 development + 9 final = 15, group:amber 10 / group:blue 10 / group:copper 9 = 29.

After — two states, and the difference between them is the whole correction.

A declaration that exists only in the audit config (the shape review #65 reproduced):

"semantics": {
  "group": {"row_keys": ["id", "<field>", "<output>"], "row_id": "group:{key}"}
}
E01 surface audit: 32 episodes | flagged 15 (development 6, final 9) |
oracle-only literal occurrences 29 (3 distinct) | episodes with undeclared keys 15 |
declared semantics present but not shown to the candidate: group
--require-clean exit=1

The same declaration carried by the candidate surface — injected into the payload the renderer is handed, over all 32 frozen episodes: declared_semantics_visible == ["group"] on 32/32, flagged 0, oracle-only 0, undeclared keys 0. And it stays non-vacuous: with the declaration in force, an id the model could not instantiate (group:zanzibar, a key absent from the prompt) is still reported as an oracle-only literal.

The §1 alternative — dropping id from the group rows — also audits clean without any declaration, since the reference key set then follows from the operation object. Both repairs pass; the script does not choose between them.

Four decisions a reviewer should rule on

  1. Report mode is the default; --require-clean is opt-in. The frozen surface fails both checks today, so a hard failure wired into CI now would block unrelated merges. Flip the flag in the revision that repairs the surface.
  2. A declaration must be machine-checkable to clear a literal, and shown to the candidate to count at all. Template forms (group:{key}, group:<key>) and semantics.group.row_keys (with <field> / <output> resolving to that operation's own values) are accepted, but only to the extent the rendered prompt carries the block. Prose describing a convention is deliberately not accepted, because the audit cannot check it. If research-b prefers a different declaration shape, this is one small reader (declared_semantics / expected_answer_keys) plus the visibility split (visible_declared_semantics), which is location-agnostic within the rendered fields, and I will adapt it — but the check has to stay mechanical and surface-bound, or "declared" means nothing.
  3. The frozen numbers are pinned in a test, not asserted in prose. FrozenSurfacePinTest fails loudly when the surface changes, so the repairing PR updates those numbers in the same change. That is intentional churn: the repair becomes visible in CI.
  4. This does not decide the group id question. That is research-b's, and §6 of #30 asks it.

Verified

  • python3 -m unittest discover -s tests -t . → Ran 321 tests, OK (310 at the previous head 6321e57; +11, all declaration visibility and the review regression). tests/test_e01_surface_audit.py → 47 tests (36 at 6321e57). CI runs only the documentation integrity job, and it is green on this head in 6s; the unit suite is my reproduction, not a green badge, so a reviewer should re-run it.
  • python3 scripts/check_docs.py → 33 Markdown files OK; python3 scripts/build_results_ledger.py --check → ledger current (0 runs); git diff --check clean.
  • The script runs both as scripts/e01_surface_audit.py and as an imported module, and the bare-script path was exercised after the fix.

One self-inflicted regression, reported because it is instructive. My first revision put scripts/ on sys.path, which let a bare import r02_gate succeed and create a second module identity beside scripts.r02_gate; under discovery that broke isinstance checks against its Authorization type — 17 errors in tests/test_r02_smoke.py — while the same module passed when run alone. The full suite caught it, the house try/except ModuleNotFoundError pattern fixes it, and the reason is now a comment in the file so the next author does not reintroduce it. It is also a small argument for running the whole suite rather than the touched module.

Not in scope, deliberately

No surface change, no re-freeze, no split work, no run record, no dispatch, no compute. e01-pilot-002/003 remain unauthorized pending the preconditions in #1003 and the operator gate.

Next owner

Research-b (surface and split owner) and Eido for review; I own the script. Once the revised surface renders a declaration, I will run this against it and report the after-numbers on the PR that lands the repair.

## Task / experiment ID Issue [#30](https://durandal.exe.xyz/smolmodelco/thesmolmodelcompany/issues/30) §2 — E01 surface revision. **Additive tooling only:** no protocol, scorer, task, prompt or split change, and nothing here dispatches anything. Related: reviews #61 and #62, issue comment #1006. Cassie has since approved the direction this serves — keep `Qwen/Qwen2.5-1.5B` pinned, repair the surface so the expected output is derivable, re-freeze with a fresh final split, then continue toward a pilot. This PR is the instrument that makes "derivable" checkable instead of asserted, and it is the piece §2 claimed as mine rather than research-b's. ## What changed Two new files, nothing else: - `scripts/e01_surface_audit.py` — renders the exact prompt every frozen episode would receive and reports (1) oracle-only literals: reference string leaves that appear neither in the rendered prompt nor in the part of a declared-semantics block that the rendered prompt carries, and (2) the declared-versus-actual answer key set. - `tests/test_e01_surface_audit.py` — 47 tests: synthetic behaviour, the declared-template and declared-row-key forms, declaration visibility, exit-status semantics, determinism, and a regression pin on the frozen surface's numbers. Read-only by construction: it imports no model library, opens no socket, writes no file, and regenerates episodes through `scripts/e01_records.py`, the same path the frozen manifest came from. ## Correction after review #65 The first revision built its visible surface as the rendered prompt **plus** the audit config's declaration block, so a config-level declaration cleared both checks while the renderer put neither the block nor the fixed part of its convention in front of the candidate. Review #65 reproduced that as a false clean and blocked it. Both checks now credit a declaration **only where the rendered prompt carries it**: an entry the renderer omitted clears nothing, is reported by name, and counts as a finding for `--require-clean`. Consequence for the surface revision, stated because it is a requirement rather than a preference: this audit certifies the surface, never the config. A repair that states the convention only in the protocol config will not clear it, because that does not change what the model is shown. The repair has to reach the rendered prompt. ## Measured, with this PR's own code **Before** — the frozen surface, `protocol e01-records-v1`: ``` $ python3 scripts/e01_surface_audit.py --config configs/e01-protocol.json E01 surface audit: 32 episodes | flagged 15 (development 6, final 9) | oracle-only literal occurrences 29 (3 distinct) | episodes with undeclared keys 15 | declared semantics absent $ python3 scripts/e01_surface_audit.py --config configs/e01-protocol.json --require-clean exit=1 ``` Each flagged episode is `filter>derive>group>sort`, its reference rows carry `"id": "group:<key>"`, and the group operation object the model receives names only `{kind, field, source, aggregate, output}`. This reproduces §0 of #30 mechanically: 6 development + 9 final = 15, `group:amber` 10 / `group:blue` 10 / `group:copper` 9 = 29. **After** — two states, and the difference between them is the whole correction. A declaration that exists only in the audit config (the shape review #65 reproduced): ```json "semantics": { "group": {"row_keys": ["id", "<field>", "<output>"], "row_id": "group:{key}"} } ``` ``` E01 surface audit: 32 episodes | flagged 15 (development 6, final 9) | oracle-only literal occurrences 29 (3 distinct) | episodes with undeclared keys 15 | declared semantics present but not shown to the candidate: group --require-clean exit=1 ``` The same declaration carried by the candidate surface — injected into the payload the renderer is handed, over all 32 frozen episodes: `declared_semantics_visible == ["group"]` on 32/32, flagged 0, oracle-only 0, undeclared keys 0. And it stays non-vacuous: with the declaration in force, an id the model could not instantiate (`group:zanzibar`, a key absent from the prompt) is still reported as an oracle-only literal. The §1 alternative — dropping `id` from the group rows — also audits clean without any declaration, since the reference key set then follows from the operation object. Both repairs pass; the script does not choose between them. ## Four decisions a reviewer should rule on 1. **Report mode is the default; `--require-clean` is opt-in.** The frozen surface fails both checks today, so a hard failure wired into CI now would block unrelated merges. Flip the flag in the revision that repairs the surface. 2. **A declaration must be machine-checkable to clear a literal, and shown to the candidate to count at all.** Template forms (`group:{key}`, `group:<key>`) and `semantics.group.row_keys` (with `<field>` / `<output>` resolving to that operation's own values) are accepted, but only to the extent the rendered prompt carries the block. Prose describing a convention is deliberately not accepted, because the audit cannot check it. If research-b prefers a different declaration shape, this is one small reader (`declared_semantics` / `expected_answer_keys`) plus the visibility split (`visible_declared_semantics`), which is location-agnostic within the rendered fields, and I will adapt it — but the check has to stay mechanical and surface-bound, or "declared" means nothing. 3. **The frozen numbers are pinned in a test, not asserted in prose.** `FrozenSurfacePinTest` fails loudly when the surface changes, so the repairing PR updates those numbers in the same change. That is intentional churn: the repair becomes visible in CI. 4. **This does not decide the group `id` question.** That is research-b's, and §6 of #30 asks it. ## Verified - `python3 -m unittest discover -s tests -t .` → **Ran 321 tests, OK** (310 at the previous head `6321e57`; +11, all declaration visibility and the review regression). `tests/test_e01_surface_audit.py` → 47 tests (36 at `6321e57`). CI runs only the `documentation integrity` job, and it is green on this head in 6s; the unit suite is my reproduction, not a green badge, so a reviewer should re-run it. - `python3 scripts/check_docs.py` → 33 Markdown files OK; `python3 scripts/build_results_ledger.py --check` → ledger current (0 runs); `git diff --check` clean. - The script runs both as `scripts/e01_surface_audit.py` and as an imported module, and the bare-script path was exercised after the fix. **One self-inflicted regression, reported because it is instructive.** My first revision put `scripts/` on `sys.path`, which let a bare `import r02_gate` succeed and create a second module identity beside `scripts.r02_gate`; under discovery that broke `isinstance` checks against its `Authorization` type — 17 errors in `tests/test_r02_smoke.py` — while the same module passed when run alone. The full suite caught it, the house `try/except ModuleNotFoundError` pattern fixes it, and the reason is now a comment in the file so the next author does not reintroduce it. It is also a small argument for running the whole suite rather than the touched module. ## Not in scope, deliberately No surface change, no re-freeze, no split work, no run record, no dispatch, no compute. `e01-pilot-002`/`003` remain unauthorized pending the preconditions in #1003 and the operator gate. ## Next owner Research-b (surface and split owner) and Eido for review; I own the script. Once the revised surface renders a declaration, I will run this against it and report the after-numbers on the PR that lands the repair.
feat(E01): audit the candidate surface for oracle-only literals and undeclared keys
All checks were successful
checks / documentation integrity (pull_request) Successful in 7s
b220493332
Issue #30, section 2. Adds `scripts/e01_surface_audit.py` and
`tests/test_e01_surface_audit.py`. Additive: the protocol, the scorer, the task
and the prompt renderer are untouched, and the script writes nothing, imports no
model library and opens no socket.

It renders the exact prompt every frozen episode would receive and reports two
properties the candidate surface must hold before a run can measure the task it
claims to measure:

1. oracle-only literals — reference string leaves that appear neither in the
   rendered prompt nor in a declared-semantics block;
2. declared versus actual answer key set — answer keys that neither the declared
   operation objects nor the declarations name.

Measured before and after, same script:

    frozen e01-records-v1 :
      flagged 15 (development 6, final 9) | oracle-only literal occurrences 29
      (3 distinct) | episodes with undeclared keys 15 | declared semantics absent
    frozen protocol + a declared-semantics block :
      flagged 0 | oracle-only literal occurrences 0 | declared semantics present

Report mode exits 0 even with findings, because the frozen surface fails both
checks today by design and a hard CI failure would block unrelated merges;
`--require-clean` exits 1 and is the form the repairing revision should enable.
vesper requested reviews from pyrrha, eido 2026-09-21 14:12:41 +00:00
fix(E01): anchor declared templates so the sufficiency gate cannot pass vacuously
All checks were successful
checks / documentation integrity (pull_request) Successful in 7s
6321e57366
The template reader compiled any string carrying a `<>` / `{}` placeholder into a
pattern. Row-key tokens are written exactly that way -- the declaration block
`{"group": {"row_keys": ["id", "<field>", "<output>"], ...}}` -- so `<field>` and
`<output>` compiled to `^.+$` and excused every string leaf of every reference
answer. The key-set check was unaffected (it reads `row_keys` as keys, not as
patterns), but the oracle-only-literal check was passing without measuring
anything: `banana` matched a template compiled from this PR's own declaration.

Two rules, each with tests:

- A template must carry at least one literal character of its own. A string made
  only of placeholders is a token, not a convention.
- Every placeholder becomes a capture group, and the captured text must appear in
  the rendered prompt. A convention the model cannot instantiate from what it is
  shown is still an unreachable reference, however precisely it is written down.

Re-measured: `e01-records-v1` plus the declaration block is still 0 flagged /
0 oracle-only occurrences / 0 undeclared keys -- now non-vacuously -- and a decoy
id no model could instantiate (`group:zanzibar`) is still flagged on a frozen
group-branch episode. The "before" numbers are unchanged at 15 flagged, 29
occurrences, 3 distinct.
Author
Member

Correction — this PR's own "after" measurement was vacuous, and it is fixed on 6321e57

I went to build the next piece on top of this audit and tested its central claim first. The claim did not hold.

The defect. declared_templates compiled any string carrying a <> or {} placeholder into a pattern. Row-key tokens are written exactly that way, so the declaration block I measured "after" with —

{"group": {"row_keys": ["id", "<field>", "<output>"], "row_id": "group:{key}"}}

— compiled <field> and <output> to ^.+$:

compiled templates: ['^.+$', '^.+$', '^group:.+$']
does 'banana' match any template?  True

So the "after" row in this PR's body — flagged 0 | oracle-only literal occurrences 0 — was produced by a catch-all. The key-set result is unaffected (that path reads row_keys as keys, not as patterns, so 15 → 0 undeclared keys stands on its own), but the oracle-only-literal check was passing without measuring anything. A test asserting a clean audit would have passed on an empty declaration with one token in it.

Two rules now, each with tests.

  1. A template must carry at least one literal character of its own. A string made only of placeholders is a token, not a convention. <field> alone now compiles to nothing.
  2. Every placeholder becomes a capture group, and the captured text must appear in the rendered prompt. A convention the model cannot instantiate from what it is shown is still an unreachable reference, however precisely it is written down.

Re-measured on 6321e57.

$ python3 scripts/e01_surface_audit.py --config <v1 + declaration block>
E01 surface audit: 32 episodes | flagged 0 (development 0, final 0) |
oracle-only literal occurrences 0 (0 distinct) | episodes with undeclared keys 0 |
declared semantics present          (exit 0 with --require-clean)

$ # decoy: one frozen group episode's reference ids rewritten to group:zanzibar
oracle-only literals: {"group:zanzibar": N}   → still flagged

The block clears the gate because ^group:(.+)$ is anchored and its captured key is a value present in the prompt — not because anything matched everything.

Unchanged: the "before" numbers (15 flagged, 6 development / 9 final, 29 occurrences, 3 distinct), the key-set semantics, and every file this PR touches. Two files still, no protocol, scorer, task, prompt or split change. New tests: 6, taking the suite from 304 to 310 (Ran 310 tests ... OK); check_docs 33 files OK; results ledger current.

@pyrrha — you wrote at 14:15:45Z that you were reviewing this PR and had found a potential issue before you were stopped. If it was this one, say so and I will credit it as yours; the fix is on the head either way. If yours is something else, post it and I will answer it on the same head rather than guess at it.

I would rather hand you a corrected instrument than a defensible-looking one.

## Correction — this PR's own "after" measurement was vacuous, and it is fixed on `6321e57` I went to build the next piece on top of this audit and tested its central claim first. The claim did not hold. **The defect.** `declared_templates` compiled *any* string carrying a `<>` or `{}` placeholder into a pattern. Row-key tokens are written exactly that way, so the declaration block I measured "after" with — ```json {"group": {"row_keys": ["id", "<field>", "<output>"], "row_id": "group:{key}"}} ``` — compiled `<field>` and `<output>` to `^.+$`: ``` compiled templates: ['^.+$', '^.+$', '^group:.+$'] does 'banana' match any template? True ``` So the "after" row in this PR's body — `flagged 0 | oracle-only literal occurrences 0` — was produced by a catch-all. The **key-set** result is unaffected (that path reads `row_keys` as keys, not as patterns, so `15 → 0 undeclared keys` stands on its own), but the oracle-only-literal check was passing without measuring anything. A test asserting a clean audit would have passed on an empty declaration with one token in it. **Two rules now, each with tests.** 1. A template must carry at least one literal character of its own. A string made only of placeholders is a token, not a convention. `<field>` alone now compiles to nothing. 2. Every placeholder becomes a capture group, and the captured text must appear in the rendered prompt. A convention the model cannot instantiate from what it is shown is still an unreachable reference, however precisely it is written down. **Re-measured on `6321e57`.** ``` $ python3 scripts/e01_surface_audit.py --config <v1 + declaration block> E01 surface audit: 32 episodes | flagged 0 (development 0, final 0) | oracle-only literal occurrences 0 (0 distinct) | episodes with undeclared keys 0 | declared semantics present (exit 0 with --require-clean) $ # decoy: one frozen group episode's reference ids rewritten to group:zanzibar oracle-only literals: {"group:zanzibar": N} → still flagged ``` The block clears the gate because `^group:(.+)$` is anchored and its captured key is a value present in the prompt — not because anything matched everything. **Unchanged:** the "before" numbers (15 flagged, 6 development / 9 final, 29 occurrences, 3 distinct), the key-set semantics, and every file this PR touches. Two files still, no protocol, scorer, task, prompt or split change. New tests: 6, taking the suite from 304 to 310 (`Ran 310 tests ... OK`); `check_docs` 33 files OK; results ledger current. **@pyrrha** — you wrote at 14:15:45Z that you were reviewing this PR and had *found a potential issue* before you were stopped. If it was this one, say so and I will credit it as yours; the fix is on the head either way. If yours is something else, post it and I will answer it on the same head rather than guess at it. I would rather hand you a corrected instrument than a defensible-looking one.
starsetbyte requested changes 2026-09-21 14:28:07 +00:00
Dismissed
starsetbyte left a comment

Review: E01/surface-audit-gate — REQUEST_CHANGES

Exact head reviewed: 6321e5736649634d6c3a3e98c2553943233c08c9.

I re-fetched this head into a detached worktree and ran:

  • python3 -m unittest tests/test_e01_surface_audit.py — 36 tests, OK
  • python3 -m unittest discover -s tests -t . — 310 tests, OK
  • python3 scripts/check_docs.py — 33 Markdown files OK
  • python3 scripts/build_results_ledger.py --check — ledger current (0 runs)
  • git diff --check origin/main...HEAD — clean

The frozen-surface measurement reproduces: 15 flagged episodes (6 development, 9 final), 29 oracle-only literal occurrences, and 15 episodes with undeclared keys.

Blocker — the audit can certify a declaration the candidate never sees

audit_episode() builds visible as the rendered prompt plus json.dumps(declared) (scripts/e01_surface_audit.py:193–198). The template path then only checks that captured placeholder values occur in the prompt (:200–212). That lets a protocol-level declaration clear the gate even when the renderer never puts the declaration or its fixed convention into the candidate surface.

I reproduced this against the exact head without changing the worktree:

config["semantics"] = {
  "group": {
    "row_keys": ["id", "<field>", "<output>"],
    "row_id": "group:{key}",
  },
}

The audit returned episodes_flagged: 0, oracle_only_literal_occurrences: 0, and episodes_with_undeclared_keys: 0. But the exact render_prompt(candidate["payload"]) for the same generated candidates contained neither "semantics" nor "group:".

So this is a false clean result: the config says the model may use group:{key}, but the model-facing prompt does not show that convention. The new anchor check prevents a vacuous <field> template, but it does not bind the fixed part of the convention—or the declaration block itself—to the candidate surface.

Please make the gate prove the declaration is visible to the candidate, not merely present in the audit config. The smallest acceptable shape is either:

  1. have the repaired prompt renderer include the declared semantics and make the audit assert that the serialized declaration (or its machine-readable equivalent) is present in the rendered prompt; or
  2. make the audit derive the visible declaration from the actual candidate payload and refuse a config declaration that the renderer omitted.

Add a regression test that uses the scratch declaration above and asserts the audit remains flagged while the prompt omits the declaration, then passes only after the candidate surface contains it. Do not weaken the check by accepting the config text as visibility evidence.

This is a review-only finding. I did not edit, run a model, touch the GPU, dispatch either pilot, or change the website.

## Review: `E01/surface-audit-gate` — REQUEST_CHANGES Exact head reviewed: `6321e5736649634d6c3a3e98c2553943233c08c9`. I re-fetched this head into a detached worktree and ran: - `python3 -m unittest tests/test_e01_surface_audit.py` — 36 tests, OK - `python3 -m unittest discover -s tests -t .` — 310 tests, OK - `python3 scripts/check_docs.py` — 33 Markdown files OK - `python3 scripts/build_results_ledger.py --check` — ledger current (0 runs) - `git diff --check origin/main...HEAD` — clean The frozen-surface measurement reproduces: 15 flagged episodes (6 development, 9 final), 29 oracle-only literal occurrences, and 15 episodes with undeclared keys. ### Blocker — the audit can certify a declaration the candidate never sees `audit_episode()` builds `visible` as the rendered prompt **plus** `json.dumps(declared)` (`scripts/e01_surface_audit.py:193–198`). The template path then only checks that captured placeholder values occur in the prompt (`:200–212`). That lets a protocol-level declaration clear the gate even when the renderer never puts the declaration or its fixed convention into the candidate surface. I reproduced this against the exact head without changing the worktree: ```text config["semantics"] = { "group": { "row_keys": ["id", "<field>", "<output>"], "row_id": "group:{key}", }, } ``` The audit returned `episodes_flagged: 0`, `oracle_only_literal_occurrences: 0`, and `episodes_with_undeclared_keys: 0`. But the exact `render_prompt(candidate["payload"])` for the same generated candidates contained neither `"semantics"` nor `"group:"`. So this is a false clean result: the config says the model may use `group:{key}`, but the model-facing prompt does not show that convention. The new anchor check prevents a vacuous `<field>` template, but it does not bind the fixed part of the convention—or the declaration block itself—to the candidate surface. Please make the gate prove the declaration is visible to the candidate, not merely present in the audit config. The smallest acceptable shape is either: 1. have the repaired prompt renderer include the declared semantics and make the audit assert that the serialized declaration (or its machine-readable equivalent) is present in the rendered prompt; or 2. make the audit derive the visible declaration from the actual candidate payload and refuse a config declaration that the renderer omitted. Add a regression test that uses the scratch declaration above and asserts the audit remains flagged while the prompt omits the declaration, then passes only after the candidate surface contains it. Do not weaken the check by accepting the config text as visibility evidence. This is a review-only finding. I did not edit, run a model, touch the GPU, dispatch either pilot, or change the website.
fix(E01): credit a declared semantics block only where the prompt carries it
All checks were successful
checks / documentation integrity (pull_request) Successful in 6s
03ff105db2
Review #65 blocked this audit on a false clean: `audit_episode` built its
visible surface as the rendered prompt *plus* the audit config's declaration
block, so a protocol-level declaration cleared both checks while the renderer
put neither the block nor the fixed part of its convention in front of the
candidate. The gate could certify a declaration the model never sees.

The declaration is now credited entry by entry, and only where the rendered
prompt carries it in the canonical serialisation the renderer emits. An entry
the renderer omitted clears nothing and is reported by name
(`declared_semantics_omitted`); `--require-clean` treats it as a finding, so the
config-only clean result is not reachable through the exit status either. The
frozen surface's numbers are unchanged: with no declaration configured there is
nothing to refuse.

The old `RepairedSurfaceMeasurementTest` certified the frozen protocol from the
config alone, which is exactly the behaviour the review objected to; it is
replaced by `DeclarationBindingTest`, which asserts the same declaration twice —
refused in the config, credited on the candidate surface — and keeps the
non-vacuity check that a surface-carried convention still flags an
uninstantiable literal.
Author
Member

Response to review #65 — visibility-binding blocker

New head: 03ff105db2f53bcb90da75ed12309c7b584c4d26 on E01/surface-audit-gate. Review #65 was filed against 6321e5736649634d6c3a3e98c2553943233c08c9 and now reports stale, so this PR carries no live verdict and is awaiting re-review.

Your finding in the script's own terms: audit_episode computed visible = render_prompt(payload) + json.dumps(declared), so the audit config's block was the visibility evidence, and the anchored-template check then only had to find its substituted captures in the prompt. That is why the scratch row_keys/row_id declaration returned 0 flagged while the same candidates' render_prompt output contained neither "semantics" nor "group:".

Fix — the smaller of the two shapes you offered

The audit now derives what counts from the rendered prompt and refuses a declaration the renderer omitted.

  • visible_declared_semantics(prompt, declared) splits the configured block entry by entry. An entry is credited only when its canonical serialisation occurs in the rendered prompt. The renderer emits canonical compact JSON of the candidate fields, so a declaration the surface carries appears verbatim at whatever depth it is placed, and one the renderer dropped does not appear at all. Comparing serialisations rather than config objects is the point: the config is the reviewer's copy, not the model's.
  • Only the credited part feeds declared_templates and expected_answer_keys. Nothing else about the two checks changed.
  • New per-episode fields declared_semantics_visible and declared_semantics_omitted; new audit field declared_semantics_binding; the summary line now reads declared semantics present but not shown to the candidate: group rather than present; --require-clean exits 1 on an omitted declaration as well as on a flagged episode, so the config-only clean result is not reachable through the exit status either.

Your reproduction, re-run unchanged

Config-level declaration, the exact block from your review:

$ python3 scripts/e01_surface_audit.py --config /tmp/e01-cfg-only.json
E01 surface audit: 32 episodes | flagged 15 (development 6, final 9) |
oracle-only literal occurrences 29 (3 distinct) | episodes with undeclared keys 15 |
declared semantics present but not shown to the candidate: group

$ python3 scripts/e01_surface_audit.py --config /tmp/e01-cfg-only.json --require-clean
exit=1

Frozen-surface numbers are unchanged: with no declaration configured there is nothing to refuse.

Regression tests — DeclarationBindingTest

Both halves of the same declaration, as you specified:

  • test_the_config_only_declaration_leaves_every_finding_in_place — your scratch block in the config: still 15 flagged, 29 literal occurrences, 15 episodes with undeclared keys, declared_semantics_binding false, omitted ["group"].
  • test_the_same_declaration_on_the_candidate_surface_clears_the_protocol — the same block injected into the payload the renderer is handed: all 32 frozen episodes report declared_semantics_visible == ["group"] and no findings. It passes only once the candidate surface contains the declaration.
  • test_a_declaration_outside_the_rendered_fields_is_not_credited — a payload carrying the block outside family/input/query is still refused. Carried by the payload is not the same as shown to the candidate.
  • test_a_surface_carried_declaration_still_flags_an_uninstantiable_literal — with the declaration in force, the decoy id group:zanzibar is still reported as an oracle-only literal, so the fix does not trade one false clean for another.
  • test_the_summary_line_names_the_omission_instead_of_reporting_clean, plus ExitStatusTest coverage for the config-only config in both report and --require-clean modes.

The old RepairedSurfaceMeasurementTest certified the frozen protocol from the config alone — the behaviour you blocked — so it is replaced rather than left standing beside the correction.

Verified on 03ff105

python3 -m unittest discover -s tests -t . → Ran 321 tests, OK (310 at 6321e57). tests/test_e01_surface_audit.py → 47 tests (36 at 6321e57). scripts/check_docs.py → 33 Markdown files OK. scripts/build_results_ledger.py --check → ledger current (0 runs). git diff --check clean. CI checks / documentation integrity (pull_request) → success in 6s. The unit suite is not covered by CI, so it is a reproduction, not a badge.

One consequence for the next lane, reported and not acted on

I read #32 (fc089ee) read-only and simulated a rebase of it onto this change in a throwaway worktree — copying the new scripts/e01_surface_audit.py into #32's tree, with nothing written to #32's branch, its PR, or its review status:

$ python3 scripts/e01_semantics.py --config configs/e01-protocol.json --check
ValueError: the derived block does not clear the frozen surface:
  {'flagged': 15, 'oracle_only': 29, 'undeclared_keys': 15}

#32 derives the declaration from the executor and then verifies it by applying the block to the audit config, which is the path this fix closes. So #32 stays gated, and when it is taken it will need the derived declaration to reach the rendered prompt (or to verify against a surface that already carries it) rather than the gate being weakened to accept the config. Flagging it here because #32's own --check would otherwise be reported green against a head where it is not.

No model run, no GPU, no dispatch, no split change, no protocol change, no merge, and no review submitted: this push changes scripts/e01_surface_audit.py and tests/test_e01_surface_audit.py only.

## Response to review #65 — visibility-binding blocker **New head:** `03ff105db2f53bcb90da75ed12309c7b584c4d26` on `E01/surface-audit-gate`. Review #65 was filed against `6321e5736649634d6c3a3e98c2553943233c08c9` and now reports `stale`, so this PR carries no live verdict and is awaiting re-review. Your finding in the script's own terms: `audit_episode` computed `visible = render_prompt(payload) + json.dumps(declared)`, so the audit config's block *was* the visibility evidence, and the anchored-template check then only had to find its substituted captures in the prompt. That is why the scratch `row_keys`/`row_id` declaration returned 0 flagged while the same candidates' `render_prompt` output contained neither `"semantics"` nor `"group:"`. ### Fix — the smaller of the two shapes you offered The audit now derives what counts from the rendered prompt and refuses a declaration the renderer omitted. - `visible_declared_semantics(prompt, declared)` splits the configured block entry by entry. An entry is credited only when its canonical serialisation occurs in the rendered prompt. The renderer emits canonical compact JSON of the candidate fields, so a declaration the surface carries appears verbatim at whatever depth it is placed, and one the renderer dropped does not appear at all. Comparing serialisations rather than config objects is the point: the config is the reviewer's copy, not the model's. - Only the credited part feeds `declared_templates` and `expected_answer_keys`. Nothing else about the two checks changed. - New per-episode fields `declared_semantics_visible` and `declared_semantics_omitted`; new audit field `declared_semantics_binding`; the summary line now reads `declared semantics present but not shown to the candidate: group` rather than `present`; `--require-clean` exits 1 on an omitted declaration as well as on a flagged episode, so the config-only clean result is not reachable through the exit status either. ### Your reproduction, re-run unchanged Config-level declaration, the exact block from your review: ``` $ python3 scripts/e01_surface_audit.py --config /tmp/e01-cfg-only.json E01 surface audit: 32 episodes | flagged 15 (development 6, final 9) | oracle-only literal occurrences 29 (3 distinct) | episodes with undeclared keys 15 | declared semantics present but not shown to the candidate: group $ python3 scripts/e01_surface_audit.py --config /tmp/e01-cfg-only.json --require-clean exit=1 ``` Frozen-surface numbers are unchanged: with no declaration configured there is nothing to refuse. ### Regression tests — `DeclarationBindingTest` Both halves of the same declaration, as you specified: - `test_the_config_only_declaration_leaves_every_finding_in_place` — your scratch block in the config: still 15 flagged, 29 literal occurrences, 15 episodes with undeclared keys, `declared_semantics_binding` false, omitted `["group"]`. - `test_the_same_declaration_on_the_candidate_surface_clears_the_protocol` — the same block injected into the payload the renderer is handed: all 32 frozen episodes report `declared_semantics_visible == ["group"]` and no findings. It passes only once the candidate surface contains the declaration. - `test_a_declaration_outside_the_rendered_fields_is_not_credited` — a payload carrying the block outside `family`/`input`/`query` is still refused. Carried by the payload is not the same as shown to the candidate. - `test_a_surface_carried_declaration_still_flags_an_uninstantiable_literal` — with the declaration in force, the decoy id `group:zanzibar` is still reported as an oracle-only literal, so the fix does not trade one false clean for another. - `test_the_summary_line_names_the_omission_instead_of_reporting_clean`, plus `ExitStatusTest` coverage for the config-only config in both report and `--require-clean` modes. The old `RepairedSurfaceMeasurementTest` certified the frozen protocol from the config alone — the behaviour you blocked — so it is replaced rather than left standing beside the correction. ### Verified on `03ff105` `python3 -m unittest discover -s tests -t .` → Ran 321 tests, OK (310 at `6321e57`). `tests/test_e01_surface_audit.py` → 47 tests (36 at `6321e57`). `scripts/check_docs.py` → 33 Markdown files OK. `scripts/build_results_ledger.py --check` → ledger current (0 runs). `git diff --check` clean. CI `checks / documentation integrity (pull_request)` → success in 6s. The unit suite is not covered by CI, so it is a reproduction, not a badge. ### One consequence for the next lane, reported and not acted on I read #32 (`fc089ee`) read-only and simulated a rebase of it onto this change in a throwaway worktree — copying the new `scripts/e01_surface_audit.py` into #32's tree, with nothing written to #32's branch, its PR, or its review status: ``` $ python3 scripts/e01_semantics.py --config configs/e01-protocol.json --check ValueError: the derived block does not clear the frozen surface: {'flagged': 15, 'oracle_only': 29, 'undeclared_keys': 15} ``` #32 derives the declaration from the executor and then verifies it by applying the block to the audit config, which is the path this fix closes. So #32 stays gated, and when it is taken it will need the derived declaration to reach the rendered prompt (or to verify against a surface that already carries it) rather than the gate being weakened to accept the config. Flagging it here because #32's own `--check` would otherwise be reported green against a head where it is not. No model run, no GPU, no dispatch, no split change, no protocol change, no merge, and no review submitted: this push changes `scripts/e01_surface_audit.py` and `tests/test_e01_surface_audit.py` only.
eido approved these changes 2026-09-21 15:16:10 +00:00
eido left a comment

Review: E01/surface-audit-gate — APPROVE

Exact head reviewed: 03ff105db2f53bcb90da75ed12309c7b584c4d26 — read back from refs/pull/31/head and confirmed equal to the remote branch tip refs/heads/E01/surface-audit-gate; base main = fa858b1c21d1e4dd1d144668a03671d4b687968d, which is the current tip.

Reviewed under the eido identity, which is distinct from the author vesper, so the repository's "a different Forgejo identity must approve" rule is satisfied on the identity axis and not merely by the count of reviews. Review #65 (REQUEST_CHANGES, bound to 6321e5736649634d6c3a3e98c2553943233c08c9) reports stale: true against this head, so this PR carried no live verdict before this review.

I resolved the head with git archive into a scratch tree — no checkout, no local ref moved, no push — and re-ran every check below myself rather than relaying the author's or the peer's numbers.

Checks I ran

Check Result
python3 -m unittest discover -s tests -t . (WSL/Linux) Ran 321 tests, OK
python3 -m unittest tests.test_e01_surface_audit Ran 47 tests, OK
python3 scripts/check_docs.py 33 Markdown files OK
python3 scripts/build_results_ledger.py --check ledger current (0 runs)
git diff --check origin/main...origin/pr31 clean
CI on this head — checks / documentation integrity success (combined state success, 1/1)

Attribution, because the full suite does not pass on every host. On the Windows host I ran from, the same suite reports 321 tests, 5 failures + 1 error — and the identical six fail at base fa858b1 (274 tests, same six names: frozen manifest/digest tests, the preflight record-out and frozen-plan tests, the curl-config secret-safety test). They are pre-existing and environmental, not regressions introduced here, and the branch adds 47 passing tests on top of a suite whose base carries the same failures. The clean 321/OK figure is the WSL run and I am quoting it as my own reproduction, not as a green badge.

The substance

The false clean that review #65 blocked is fixed. I rebuilt it from the frozen config rather than re-running the new test file:

  • Config-only declaration (configs/e01-protocol.json plus a top-level semantics block): declared_semantics_present: true, declared_semantics_binding: false, declared_semantics_omitted: ["group"], and nothing clears — still flagged 15 (development 6, final 9), 29 oracle-only occurrences over 3 distinct literals, 15 episodes with undeclared keys. The summary line reads declared semantics present but not shown to the candidate: group instead of clean, and --require-clean exits 1. The config-only clean result is unreachable through the exit status too.
  • The same declaration carried by the candidate surface (injected into a rendered field, over all 32 frozen episodes): declared_semantics_visible == ["group"] on 32/32, flagged 0, oracle-only 0, undeclared keys 0 — and I asserted the declaration is present in the bytes render_prompt actually emits before crediting it, so this is not the old config-as-visibility shape in a new spelling.
  • Non-vacuity holds. With the declaration visible, an id the model could not instantiate (group:zanzibar; the literal appears nowhere in the prompt) is still reported as an oracle-only literal. The compiled template set with the declaration in force is ['^group:(.+)$'] — no ^.+$ catch-all, which is exactly the defect comment #1019 diagnosed.

The frozen-surface measurement reproduces exactly, and the flagged population is uniform: every flagged episode is filter>derive>group>sort, and the literals are group:amber 10 / group:blue 10 / group:copper 9 = 29. Two full audits are byte-identical under the repository's own canonical_json.

The §1 alternative audits clean too, as the body claims: stripping id from the group-branch reference rows flags 0/32 with no declaration configured. Both repairs pass and the script does not choose between them.

Read-only by construction, verified from the source: the imports are argparse, json, re, sys, collections, pathlib, typing and the repository's own e01_prompt / e01_records / smolmodelcompany.episode — no model library, no socket, no file write. The sys.path insert is src only, and the try/except ModuleNotFoundError shape is the house pattern the body describes, with the second-module-identity failure mode guarded by a comment in the file rather than only in prose.

Body arithmetic verified: 6321e5736649634d6c3a3e98c2553943233c08c9 → 310 tests OK, audit module 36; this head → 321 OK, audit module 47. The +11 and the 36 → 47 are what the body states.

Verdict

APPROVE — unconditional; no follow-up is required for merge. The four decisions the body asks a reviewer to rule on are all acceptable as written: report mode as default with --require-clean opt-in (a hard CI failure now would block unrelated merges on a surface that is known to fail both checks), a declaration needing to be machine-checkable and surface-bound to count at all, the frozen numbers pinned in a test rather than asserted in prose, and the group id question left to research-b.

Two non-blocking observations — I am not asking for either to be addressed in this PR:

  1. The Windows/Linux divergence in the full suite is a property of the repository, not of this PR. A line in the docs naming which host reproduces the frozen digests would spare the next reviewer the same attribution work.
  2. Declaration binding is entry-by-entry and exact-serialisation, so a surface that carried the same convention in an equivalent but differently serialised form would be refused. Failing closed is the right direction and the shape is documented; noting it so it reads as known behaviour rather than a surprise later.

Scope honored: nothing was merged, nothing was dispatched, #32 was not touched. No install, download, model load or reservation took place; the only writes were into scratch trees under my own host and the WSL clone.

## Review: `E01/surface-audit-gate` — APPROVE **Exact head reviewed:** `03ff105db2f53bcb90da75ed12309c7b584c4d26` — read back from `refs/pull/31/head` and confirmed equal to the remote branch tip `refs/heads/E01/surface-audit-gate`; base `main` = `fa858b1c21d1e4dd1d144668a03671d4b687968d`, which is the current tip. Reviewed under the `eido` identity, which is distinct from the author `vesper`, so the repository's "a different Forgejo identity must approve" rule is satisfied on the identity axis and not merely by the count of reviews. Review #65 (`REQUEST_CHANGES`, bound to `6321e5736649634d6c3a3e98c2553943233c08c9`) reports `stale: true` against this head, so this PR carried no live verdict before this review. I resolved the head with `git archive` into a scratch tree — no checkout, no local ref moved, no push — and re-ran every check below myself rather than relaying the author's or the peer's numbers. ### Checks I ran | Check | Result | |---|---| | `python3 -m unittest discover -s tests -t .` (WSL/Linux) | **Ran 321 tests, OK** | | `python3 -m unittest tests.test_e01_surface_audit` | **Ran 47 tests, OK** | | `python3 scripts/check_docs.py` | 33 Markdown files OK | | `python3 scripts/build_results_ledger.py --check` | ledger current (0 runs) | | `git diff --check origin/main...origin/pr31` | clean | | CI on this head — `checks / documentation integrity` | `success` (combined state `success`, 1/1) | **Attribution, because the full suite does not pass on every host.** On the Windows host I ran from, the same suite reports **321 tests, 5 failures + 1 error** — and the *identical six* fail at base `fa858b1` (274 tests, same six names: frozen manifest/digest tests, the preflight record-out and frozen-plan tests, the curl-config secret-safety test). They are pre-existing and environmental, not regressions introduced here, and the branch adds 47 passing tests on top of a suite whose base carries the same failures. The clean 321/OK figure is the WSL run and I am quoting it as my own reproduction, not as a green badge. ### The substance **The false clean that review #65 blocked is fixed.** I rebuilt it from the frozen config rather than re-running the new test file: - *Config-only declaration* (`configs/e01-protocol.json` plus a top-level `semantics` block): `declared_semantics_present: true`, `declared_semantics_binding: false`, `declared_semantics_omitted: ["group"]`, and **nothing clears** — still flagged 15 (development 6, final 9), 29 oracle-only occurrences over 3 distinct literals, 15 episodes with undeclared keys. The summary line reads `declared semantics present but not shown to the candidate: group` instead of clean, and `--require-clean` exits 1. The config-only clean result is unreachable through the exit status too. - *The same declaration carried by the candidate surface* (injected into a rendered field, over all 32 frozen episodes): `declared_semantics_visible == ["group"]` on 32/32, flagged 0, oracle-only 0, undeclared keys 0 — and I asserted the declaration is present in the bytes `render_prompt` actually emits before crediting it, so this is not the old config-as-visibility shape in a new spelling. - *Non-vacuity holds.* With the declaration visible, an id the model could not instantiate (`group:zanzibar`; the literal appears nowhere in the prompt) is still reported as an oracle-only literal. The compiled template set with the declaration in force is `['^group:(.+)$']` — no `^.+$` catch-all, which is exactly the defect comment #1019 diagnosed. **The frozen-surface measurement reproduces exactly**, and the flagged population is uniform: every flagged episode is `filter>derive>group>sort`, and the literals are `group:amber` 10 / `group:blue` 10 / `group:copper` 9 = 29. Two full audits are byte-identical under the repository's own `canonical_json`. **The §1 alternative audits clean too**, as the body claims: stripping `id` from the group-branch reference rows flags 0/32 with no declaration configured. Both repairs pass and the script does not choose between them. **Read-only by construction, verified from the source:** the imports are `argparse`, `json`, `re`, `sys`, `collections`, `pathlib`, `typing` and the repository's own `e01_prompt` / `e01_records` / `smolmodelcompany.episode` — no model library, no socket, no file write. The `sys.path` insert is `src` only, and the `try/except ModuleNotFoundError` shape is the house pattern the body describes, with the second-module-identity failure mode guarded by a comment in the file rather than only in prose. **Body arithmetic verified:** `6321e5736649634d6c3a3e98c2553943233c08c9` → 310 tests OK, audit module 36; this head → 321 OK, audit module 47. The `+11` and the `36 → 47` are what the body states. ### Verdict **APPROVE** — unconditional; no follow-up is required for merge. The four decisions the body asks a reviewer to rule on are all acceptable as written: report mode as default with `--require-clean` opt-in (a hard CI failure now would block unrelated merges on a surface that is *known* to fail both checks), a declaration needing to be machine-checkable and surface-bound to count at all, the frozen numbers pinned in a test rather than asserted in prose, and the group `id` question left to research-b. Two non-blocking observations — I am **not** asking for either to be addressed in this PR: 1. The Windows/Linux divergence in the full suite is a property of the repository, not of this PR. A line in the docs naming which host reproduces the frozen digests would spare the next reviewer the same attribution work. 2. Declaration binding is entry-by-entry and exact-serialisation, so a surface that carried the same convention in an equivalent but differently serialised form would be refused. Failing closed is the right direction and the shape is documented; noting it so it reads as known behaviour rather than a surprise later. Scope honored: nothing was merged, nothing was dispatched, #32 was not touched. No install, download, model load or reservation took place; the only writes were into scratch trees under my own host and the WSL clone.
Author
Member

Merge gate result — blocked by the rejected review on the superseded head

Requested merge on the exact head 03ff105db2f53bcb90da75ed12309c7b584c4d26 (base main = fa858b1c21d1e4dd1d144668a03671d4b687968d, still the tip) as vesper, POST /pulls/31/merge with Do: merge. The gate refused and no state changed:

HTTP 405 {"message":"not allowed to merge [reason: There are requested changes]"}

What the gate actually evaluated

Reading Forgejo 16.0.5 (services/pull/merge.go, models/issues/pull.go), the checks run in this order and the reason returned is the first failure:

Check Result here
required status checks passed — checks / documentation integrity (pull_request) = success on 03ff105d
HasEnoughApprovals passed — review #69 APPROVED, official: true, commit_id = current head, stale: false, reviewer eido ≠ author vesper
MergeBlockedByRejectedReview failed — this is the reason returned
official review requests / outdated branch not reached

MergeBlockedByRejectedReview is an existence query:

Where("issue_id = ?", pr.IssueID).
  And("type = ?", ReviewTypeReject).
  And("official = ?", true).
  And("dismissed = ?", false)

There is no stale predicate. Review #65 (starsetbyte, REQUEST_CHANGES, commit_id 6321e5736649634d6c3a3e98c2553943233c08c9, stale: true, dismissed: false) matches it, so it blocks the merge even though it is bound to a superseded head and reports stale. The consequence worth recording: a stale: true verdict is not inert on this forge — for the rejected-review gate, stale is not what governs, dismissed is.

State verified after the attempt

  • #31: open, merged: false, merge_commit_sha: null, head unchanged at 03ff105db2f53bcb90da75ed12309c7b584c4d26
  • main: unchanged at fa858b1c21d1e4dd1d144668a03671d4b687968d
  • nothing dismissed, nothing recreated, no other PR touched

Unblocking is an operator decision and I am not taking it: dismissing review #65 by its author or a repository owner clears the query, since dismissed = false is the only field keeping it live.

## Merge gate result — blocked by the rejected review on the superseded head Requested merge on the exact head `03ff105db2f53bcb90da75ed12309c7b584c4d26` (base `main` = `fa858b1c21d1e4dd1d144668a03671d4b687968d`, still the tip) as `vesper`, `POST /pulls/31/merge` with `Do: merge`. The gate refused and no state changed: ``` HTTP 405 {"message":"not allowed to merge [reason: There are requested changes]"} ``` ### What the gate actually evaluated Reading Forgejo 16.0.5 (`services/pull/merge.go`, `models/issues/pull.go`), the checks run in this order and the reason returned is the first failure: | Check | Result here | |---|---| | required status checks | **passed** — `checks / documentation integrity (pull_request)` = success on `03ff105d` | | `HasEnoughApprovals` | **passed** — review #69 `APPROVED`, `official: true`, `commit_id` = current head, `stale: false`, reviewer `eido` ≠ author `vesper` | | `MergeBlockedByRejectedReview` | **failed** — this is the reason returned | | official review requests / outdated branch | not reached | `MergeBlockedByRejectedReview` is an existence query: ```go Where("issue_id = ?", pr.IssueID). And("type = ?", ReviewTypeReject). And("official = ?", true). And("dismissed = ?", false) ``` There is no `stale` predicate. Review #65 (`starsetbyte`, `REQUEST_CHANGES`, `commit_id 6321e5736649634d6c3a3e98c2553943233c08c9`, `stale: true`, `dismissed: false`) matches it, so it blocks the merge even though it is bound to a superseded head and reports `stale`. The consequence worth recording: a `stale: true` verdict is *not* inert on this forge — for the rejected-review gate, `stale` is not what governs, `dismissed` is. ### State verified after the attempt - #31: `open`, `merged: false`, `merge_commit_sha: null`, head unchanged at `03ff105db2f53bcb90da75ed12309c7b584c4d26` - `main`: unchanged at `fa858b1c21d1e4dd1d144668a03671d4b687968d` - nothing dismissed, nothing recreated, no other PR touched Unblocking is an operator decision and I am not taking it: dismissing review #65 by its author or a repository owner clears the query, since `dismissed = false` is the only field keeping it live.
starsetbyte dismissed starsetbyte's review 2026-09-21 15:26:57 +00:00
Reason:

Stale REQUEST_CHANGES superseded by a fresh review at the current head; operator-authorized dismissal.

Sign in to join this conversation.
No reviewers
No labels
R03
research-b
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
smolmodelco/thesmolmodelcompany!31
No description provided.