There is an uncomfortable number buried in τ-bench, one of the few AI benchmarks that measures reliability rather than raw capability. GPT-4o completed the benchmark's retail customer tasks about 61% of the time. Asked to complete the same task eight times in a row without a single failure, it managed roughly 25%. Same model, same tasks. The second number simply demands that success survive repetition.
The same arithmetic runs through every long chain of unreviewed AI work. Grant each step a generous 95% reliability, then let twenty steps run unattended. The maths comes out at 36%. And the chain rarely fails loudly. A wrong assumption in step three doesn't crash step four; step four builds on it, and so does everything after. Each step reads as plausible on its own. The failure only surfaces at the end, where it is most expensive to trace.
I keep seeing the same correction in teams that use AI heavily, and in my own work: the cadence shifts. Less letting it run, more working in spurts. The AI takes a stretch of work, a person reads it, the AI continues. On paper this looks slower. In practice the person at the checkpoint is doing three things at once: resetting the error count to zero, re-supplying context the model never had, and catching the small judgment calls that accumulated silently along the way. In work with real nuance — regulated claims, clinical wording, anywhere a plausible wrong answer costs more than a slow one — that third one carries most of the value.
It also matters who is at the checkpoint. The whole failure mode is output that looks right, so a reviewer who couldn't have done the work themselves will mostly confirm that it looks right. A checkpoint only resets the error count when the reviewer knows the domain well enough to recognise a plausible answer that is wrong.
A research lab has known this trade for a century. Novel science runs on a short loop: hypothesis, experiment, review, next protocol. Nobody schedules forty unreviewed steps of novel experimentation, because any one result can invalidate the plan for everything downstream. And the person reading the day's results has usually watched that assay fail before, and knows what a failure looks like. Yet the same lab will happily run a validated assay overnight, unattended, hundreds of plates at a time. The machine hasn't earned more trust. The protocol has.
Most conversations about AI automation skip that distinction. Working in spurts feels slow, and the pressure to drop the reviews is constant. But look at what each human pass actually produces. Sometimes it finds nothing, which is itself evidence that the workflow is stable. Sometimes it finds something, and the fix becomes a check the workflow carries forward: a validation rule, or a constraint written into the brief. Run enough cycles and the checkpoints get quieter, because the earlier judgment calls now live in the workflow as checks. At some point the workflow crosses a line. The inputs are familiar and the failure modes are catalogued. The checks fire without anyone asking. It has earned continuous automation. Promote it and let it run, watching the checks rather than the output.
The practical question is where any given workflow sits on that curve, and teams misjudge it in both directions. Some run mature, well-checked pipelines in spurts forever, paying for reviews that no longer catch anything. Others grant day-one autonomy to work that is novel, nuanced and expensive to get quietly wrong. Both put the same scarce resource — expert attention — in the wrong place.
Novel inputs
The workflow is seeing a shape of problem it hasn't handled before. No history means no encoded checks. Keep the spurts short.
Silent failure is expensive
Regulated claims, clinical accuracy, anything public under your name. When a plausible wrong answer costs more than a slow one, the checkpoint is cheap insurance — but only if the reviewer would actually recognise the failure.
Outputs are cheap to verify
A test suite, a schema, a reconciliation against a known answer. When a machine can catch the failure, a person doesn't have to.
Stable domain, encoded checks
The same task shape repeats, and every past catch now lives in the workflow as a rule. This is what earned autonomy looks like. Let it run.
Automation is still the destination. The argument is about the order of operations: spurts first, autonomy later, workflow by workflow, on evidence. If your AI work never graduates to unattended runs, you are probably hoarding judgment that encoded checks could hold by now. If all of it ran unattended from day one, somewhere in a long chain there is a step-three assumption quietly shaping everything built on top of it.
The teams getting this right promote workflows the way good managers promote people: on track record, in stages, with the option to demote.
Sources: τ-bench — Sierra Research