Is multi-turn reasoning broken?

Everybody assumed reasoning fashions would fail the apparent means. A mannequin commits to one thing in flip two, contradicts it in flip 9, and also you catch it. Clean, detectable, patchable. Grab a consistency checker, add some grounding, and transfer on.

A paper offered on the

Why your present verification stack is flying blind

The subject has spent two years constructing hallucination detection infrastructure: retrieval grounding, factual verification, G-Eval, and

Why single-turn benchmarks are hiding this from you

MMLU, HumanEval, SWE-bench, GPQA: all single-turn. A mannequin that scores 87% on SWE-bench Verified can nonetheless drift badly on a 12-turn constraint reasoning chain.

These are measuring totally different properties totally, and treating robust single-turn scores as a proxy for multi-turn

A ultimate thought

Here is the trustworthy image. 

The AI industry has spent a whole lot of vitality worrying concerning the failures it may see: hallucinations, contradictions, refusals. Satisfiable drift is the failure that appears tremendous on the best way out the door, will get deployed, and causes issues three weeks later when somebody lastly traces the dialog again to show 4.

Multi-turn reasoning is the place manufacturing AI is headed. Agents, copilots, long-horizon planning, autonomous workflows: all of it relies on a mannequin that may maintain a dedication throughout time. That seems to be a tougher downside than anybody budgeted for.

The benchmarking tradition rewards single-turn efficiency as a result of single-turn efficiency is simple to measure. Production cares about multi-turn reliability as a result of that’s the place issues really break. DRIFT-Bench places a reputation on the hole. 

The relaxation is as much as the groups constructing it…