While running extended coding workflows on Codex gpt-5.5, a puzzling failure signature kept showing up: hard reasoning prompts returned wrong final answers while turn.completed.usage.reasoning_output_tokens stopped on suspiciously quantized values—specifically 516, 1034, or 1552 tokens.
To measure whether this was anecdotal or systemic without disclosing private prompts or transcripts, codex-reasoning-bug-516-token aggregates usage metadata across 9,205 calls in 187 local session files and pairs it with codex-reasoning-fix-skill for five-shot diagnosis and remediation.
Measuring 516, 1034, and 1552 Clusters
Because base-instruction changes are scoped to session initialization, evaluating before/after impact by raw event timestamps misclassifies already-running sessions. The analyzer groups turns by session start time (--phase-basis session):
python .\scripts\analyze_reasoning_tokens.py `
--sessions "<local-codex-sessions-dir>" `
--cutoff "2026-07-06T19:37:12Z" `
--phase-basis session `
--out-dir "<local-output-dir>"Across historical gpt-5.5 calls prior to the maximum-reasoning intervention, 258 of 4,507 calls (5.73%) landed directly on exact 516, 1034, or 1552 token counts—nearly 88% of them at 516 alone. By contrast, gpt-5.3-codex-spark recorded 0 cluster hits across 2,159 calls.
Manual Audit: Progress Turns vs. Real Failures
Aggregate hit rates alone can be misleading: is a 516-token turn just a harmless status ping? To answer that, we drew a deterministic 10% stratified sample (40 hits: 26 before, 14 after) and classified the local window as no, probably, yes, or unclear needing deeper reasoning:
- Routine progress / completed work (
30/40,75.0%):0/30warranted deeper thinking. - High-risk / complex turns (
9/40,22.5%):100%of sampled hits in safety-critical operations (4/4), secret/compliance workflows (3/3), and brittle platform orchestration (2/2) fell short of the reasoning depth the task called for.
Removing only the trailing ## Intermediary updates section from base instructions helped on targeted benchmarks—turning a 0/5 failure (all wrong at 516 tokens) into 5/5 on a reproducible 5-shot benchmark—yet ordinary multi-turn agent metadata still showed persistent cluster hits (4.93%).
A Reproducible Five-Shot Diagnostic
To test any local setup deterministically without touching workspace files, codex-reasoning-fix-skill runs five ephemeral read-only probes on a combinatorics bag puzzle whose exact answer is 21:
import json, pathlib, sys
root = pathlib.Path(sys.argv[1])
for path in sorted(root.glob("run*.jsonl")):
events = [json.loads(line) for line in path.read_text(encoding="utf-8").splitlines() if line.strip()]
messages = [
e["item"]["text"]
for e in events
if e.get("type") == "item.completed" and e.get("item", {}).get("type") == "agent_message"
]
completed = [e for e in events if e.get("type") == "turn.completed"][-1]
print({
"run": path.stem,
"reasoning_output_tokens": completed["usage"].get("reasoning_output_tokens"),
"final_agent_message": messages[-1] if messages else "",
})An affected configuration trips on at least one run with a wrong number paired with 516, 1034, or 1552 reasoning tokens; a healthy override passes 5/5 with standalone 21.
Why models_cache.json Edits Don’t Stick
Patching models_cache.json directly is tempting, but codex exec refreshes the model cache automatically and silently overwrites local edits. The durable workaround extracts gpt-5.5’s base_instructions, strips the final ## Intermediary updates block into a dedicated model-instructions.md, and pins it at the top level of config.toml before any [section] header:
model_instructions_file = "C:\\Users\\<user>\\.codex\\model-instructions.md"
[profiles.default]
model_reasoning_effort = "high"Keeping telemetry pipelines anonymized—stripping prompts, tool arguments, and paths while retaining (model, phase, reasoning_output_tokens)—made it possible to accept external contributor reproductions cleanly while isolating the failure mechanism in public.