Why Agent Pilots Don't Ship: Five Questions That Decide It

Using AI agents is no longer the hard part. Shipping them is.

Temporal's 2026 State of AI Agent Development survey asked 554 developers who already use agents how they work. 80.8% use them daily or more. Only 49.1% say agents are in production or core to how they ship. The enterprise picture is starker. In an MIT Technology Review Insights poll, sponsored by Atlan, 38% of organisations said they were mostly still piloting AI, and just 12% had agents in production.

The two surveys ask different people different questions, so the numbers don't combine into one figure. But they point the same way: most agent work stops at the pilot.

A pilot answers the easy question

A pilot usually proves one thing: the model can do the task. Production asks five more questions, and in the stalled pilots we see, that is where the answers are thin.

The five questions

• Who owns it after launch? One named person who answers for its results and its mistakes. "The AI team" is not an answer.
• What does "working" mean? One outcome, measured against a baseline from before the agent: time per case, error rate or cost per task. Without it, nobody can say whether the pilot succeeded.
• Does it have the context? The systems, definitions and rules a new hire would need on day one. A pilot run on clean demo data hasn't met your real edge cases yet.
• What can it touch and spend? Clear limits on access and cost: read-only first, human approval for writes, and a budget cap per task.
• What happens when it fails? A plan for the bad day, built before launch: retries that can't act twice, an alert when work stops, and a person to hand off to.

Score it honestly

Count the clear yeses. Five means you are ready to plan the launch. Three or four means close the gaps before you scale. Two or fewer means it is still a demo, however good it looks. It's a rule of thumb, not a benchmark, but it turns "the pilot is going well" into a short list someone can fix.

Notice that none of the five is about which model you chose. That is usually the question a team has already answered best.

How we approach it at Zylen

Zylen is an AI optimization partner. We work inside the AI systems companies already run and improve cost, security and data control one measured change at a time. Our ten-day AI Optimization Audit finds what is stopping your pilot from shipping and what to fix first.

Close the pilot gap: start the AI Optimization Audit →

Sources: Temporal, State of AI Agent Development 2026, as reported by Developer Tech; MIT Technology Review Insights, "Creating AI value with enterprise context", 28 Oct 2026 (sponsored by Atlan).