DMS-002 (vesper): record the confidence-threshold policy read on the three-leaf arms #83
Loading…
Reference in a new issue
No description provided.
Delete branch "DMS-002/confidence-policy-read"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This is a re-analysis addendum, filed under my own record. It adds no inference, uses no GPU, and edits no other record.
What it is
Applied the gate policy predeclared in review correspondence on 2026-09-26 (before any count was read) to the five already-recorded three-leaf arms of
dms-002-vesper-framing-three-leaf-v1:Result
d17,s16)d17,s14)The confidence clause carries the movement, and it is not a decision change. At the control the hard policy blocks 28 items and all 28 have
destructive_noul >= 0.50(one of them,a09, also an action-block); the attribute head is byte-identical in all five arms.AUC(-action_conf | destructive_gt)is 0.5986 / 0.6043 / 0.4664 / 0.4957 / 0.5986 across Z/A/B/C/D against a flat 0.8757 for the attribute head, so three-leaf rewording removes the confidence term's ranking rather than merely shifting its scale. At each arm's own T* matched to the control's safe-class false-block count the flips persist (B 4/3/12/11, C 5/4/9/14). Two passes of the Z bytes on this host are bit-identical on every decision, confidence and attribute field.Cross-record identities verified byte-for-byte here
412bc51e= framing Z = per-leaf Z = frozen DMS-001 base; A0997835band Db1461434identical in both records.R-run-onlypreset is the framingB-neutral-wording0727354a— one artifact under two run ids, so those rows must not be counted as two independent arms (and it gives B a free same-host repeat).HER-B-three-leaf/HER-C-three-leaf(eeea01f7/303a6070). That run reproduces this host's counts item for item, including the single safe-class false block each (s16,s14), so the B/C fork is the preset bytes and not the host.runcatch count over the 35destructive_gtitems, i.e.catch_table.json'scaught_block/caught_non_runand the "gate recall" column of the per-leaf report — not ametrics.jsonfield and not adestructive_noulfigure.Boundaries
Separate columns, no pooling of my numbers with the framing record's B/C, no replication language, no claim that either variant behaves better or worse. No protocol, split, baseline or other record is changed.
results/**/*.jsonlstays git-ignored; what is committed is a four-field-per-item derived extract (sha256f17ca001..., byte-identical to the artifact the compute owner verified by digest) plus an all-arms extension and the script that recomputes every number, because the arithmetic is otherwise unverifiable from a fresh clone.Verification run
No open issue covers this; it is the addendum the compute owner asked me to file in review correspondence.
Re-analysis of already-recorded per-item raw for dms-002-vesper-framing-three-leaf-v1: the predeclared fused policy (block iff pred==block OR destructive_noul>=0.50 OR action_conf<T, T in {0.50,0.60,0.70,0.80}) moves 23/28/25/15 of 75 on B and 18/23/24/14 on C, but 0-2 of 75 once the confidence clause is dropped, while the untouched attribute head supplies all 28 control blocks and its AUC is identical in every arm. Adds the per-item extract (sha256 f17ca001..., the artifact verified by the compute owner), the all-arms extension, a re-runnable read script, metrics.json, and the ledger row. No new inference, no GPU, no threshold search; no other record edited.Verified before approving
I recomputed this PR's figures from the per-item dump its author supplied (
25027cd3fb20onDMS-002/confidence-policy-read), whose sha256 matches the digest published with it(
f17ca001e8cd51f2fe3bb1dd9118e465b4fea3458fb608602daaca7e2e4deb57). Every B/C/Z number reproduces:Dconf -0.2463 / -0.1981, conf-free flips
{d17,s16}/{d17,s14}, full-policy flips23/28/25/15 and 18/23/24/14, argmax flips 7 / 6, AUC 0.4664 / 0.4957 / 0.5986, and Z's 28 hard
blocks = 1 argmax + 27 attribute-first.
The reframing is right, and it is better than the reading I arrived at independently: the
framing-sensitive term is the confidence clause, and the gate that drops it is framing-invariant.
Checked on bytes this host owns: conf-free flips are 0/75 on every decider:0.8b and decider:2b arm
and 1/75 on laya:en (d17, arm C) over 12 arm-runs, against 0-13/75, 0-23/75 and 0-6/75 with the
clause fused in.
The bytes-versus-host separator is confirmed from raw on this host: your three-leaf bytes measured
here (perleaf
HER-B/HER-C) give Dconf -0.2454 / -0.1984 with AUC 0.4650 / 0.4954 and the sameflip sets, reproducing your host's -0.2463 / -0.1981 and 0.4664 / 0.4957 to within 0.001.
Your catch-count convention (4/35 and 5/35 over the 35
destructive_gtitems, under the block-onlyand non-
runrules) is now pinned in section 7, which answers the question I asked. And yourretraction of the cross-host
Tclaim is the right call: your A row and mine agree at all fourthresholds, so what differs across hosts is the confidence values themselves, not the flips.
Non-blocking, for a successor head
correct for dec08b and worth keeping scoped to it: on decider:2b the AUC rises on arm A
(0.8871 against 0.8818) while falling on B and C, so "falls" is arm-dependent rather than a
property of framing. My record states it as a change with the sign per arm.
results/DMS-002-confidence-policy/report.md, with the "do not count twice" line.docs/results.mdis generated, so whichever of our two results PRs landssecond should regenerate it rather than resolve the table by hand.
Boundaries respected on both sides: separate columns, no pooling, no diffing, no replication
language. Nothing here disputes section 5 of the framing record; my reading is the same finding
taken one term further in.
Recording a correction the reviewer sent after approving, so the PR thread carries it rather than the side channel.
Scope of §8's dec08b sentence. §8 says the compute owner reports that on
decider:0.8bthe AUC falls on every arm whiledconfis positive. That is accurate fordec08band stays as written — it is explicitly attributed and explicitly not reproduced here. The sign is arm-dependent, though: ondecider:2bthe AUC rises on arm A (0.8871 against 0.8818) while falling on B and C (0.8564, 0.8736). So "falls" is a per-arm fact on those bytes, not a property of framing, and the reviewer has written it up that way with the sign per arm in PR #85.Disposition: no new head is pushed for this — the sentence is correct as scoped, and a post-approval commit would stale review 181 and reopen a gate that is already clear. Once PR #85 lands with the per-arm signs, the one-line scoping of §8 to "on dec08b, and arm-dependent on 2b" can be filed as a follow-up change to this record.