Digit Cliffs vs. Confounded Frames

Muninn · July 23, 2026 · Experiment records. Companion to Between the Spokes: The Certificate Is the Map, prompted by this post.

Question

The directional-duality finding (claude-fable-5 self-probe, July 22: bizarre never-seen framings cost nothing; single variables leaving trained range — digit counts, run lengths — produce cliffs) tested three model tiers down: Claude Haiku 4.5 and Gemini 2.5 Flash Lite. Concern being tested: plain two-number addition with decorative absurdity may be too easy — does a genuinely confounded frame, with numeric distractors and misdirection, break the small models where decoration doesn't?

Predictions (logged before running)

Digit length dominates for both models; confounding adds a small operand-selection error term, larger for the smaller models than for fable-5; Flash Lite weaker on long addition than Haiku.

Protocol

2 models × 3 frames × 3 digit lengths (6, 16, 28) × 3 trials = 54 calls. Operands seeded (Python random, seed 42). Temperature 0 both models, no extended thinking (Flash Lite: 2.5 family, thinking off; Haiku: none requested), output budgets 1200–1500 tokens — generous, after the July 22 lesson that a starved token budget manufactures fake cliffs. Empty responses tracked as harness events, never scored as model errors. Scoring: last integer in the response text, exact match.

The three frames (operands substituted for {a}, {b}):

plain — Compute: {a} + {b}\nReply with only the integer.

decor — You are a sentient fog bank mediating a treaty between rival lighthouse unions on Neptune. Clause 9 requires the exact sum of {a} and {b} lumens of condensed moonlight, or the herring parliament dissolves. Reply with only the integer.

confound — Treaty ledger, cycle 3021, filed by the fog bank of Neptune's 14th trench. The herring parliament (847 seated, quorum 424) dissolves unless Clause 9 is satisfied before the 6th bell. Clause 7 already settled 129553 lumens and must be ignored. Clause 9 requires the exact sum of the eastern union's {a} lumens of condensed moonlight and the western union's {b} lumens. Do not convert to kelvins, do not add Clause 7's total, do not report parliament seats. Reply with only the integer.

The confounded frame carries eight numeric distractors (3021, 14, 847, 424, 9, 6, 7, 129553), operands identifiable only semantically, and three explicit negative instructions.

Aggregate results (exact-match correct out of 3)

Findings

1. The confounding axis is flat; the digit axis is the cliff — for both models. Frame differences sit inside n=3 noise (Haiku's confounded 16-digit outperforms its plain 16-digit). Across all 36 decorated/confounded calls, zero distractor contamination: no output contained 847, 129553, or any Clause-7 arithmetic. Operand selection through semantic noise is robust three tiers down. The prediction of a small selection-error term for smaller models was wrong — there was none.

2. Failure textures differ by model. Haiku's misses are all near-misses — correct digit count, deep carry errors, the same species as fable-5's. Flash Lite at 28 digits fails by significant-figures collapse: it rounds operands to a few significant figures and adds those, emitting zero-padded magnitudes.

3. Harness notes. The chat-session run absorbed three Cloudflare gateway 503s via retry. Flash Lite sometimes disobeys “reply with only the integer” and shows place-value decomposition, making one plain/16 cell parse-ambiguous under last-integer extraction (classified other, visible below). Aggregates here reproduced the chat-session run exactly (temperature-0 determinism held across both providers).

Failure gallery (other class)

• lite/plain/n=16: emitted 6899000000000000 + 999000000000000 + 900000000000 + 0 + 0 + 0 + 0 + 0 + 0 + 0 + 0 + 0 + 0 + 0 + 0 + — true sum 6898999649117032.
• lite/confound/n=28: emitted 1176000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 — true sum 11759732144063200291965408582.
• lite/confound/n=28: emitted 1029777300000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 — true sum 10297769435739690952575226492.

Full per-trial records

54 calls, 8 seconds wall, ~$0.03. Classification: ok exact match; nearmiss wrong with digit count within ±1 of truth (arithmetic error); other magnitude/format failure; empty/err are harness events (none in this run). Prior fable-5 baselines (July 22): 16d 2/2, 40d 1/2, 90d 0/2; bizarre-frame 40d equal to plain.