Your “AI Agent” Might Just Be a Script

The agent boom has created a familiar product trap: a workflow gets wrapped in model calls, a dashboard gets an “autonomous” label, and suddenly a deterministic system is being sold as a thinking colleague. For AI builders and buyers, the problem is not that agents are useless. It’s that the term is elastic enough to hide a lot of ordinary automation.

That matters because the gap between a real agent and a script is not cosmetic. It changes how you evaluate reliability, how much human oversight you need, how much token spend you can tolerate, and whether the system can survive novelty instead of only replaying happy-path demos.

The first test: does it actually revise its own next move?

A useful line in the source material is simple: if a system does not inspect the result of its own action to decide what to do next, it is a script, not an agent. That is the architectural difference that keeps coming up across the evidence. Common “agentic” products often use prompt chains, routers, retrieval assistants, or trigger-action automations where the rest of the path is already written in advance. The model may choose a branch, but it is still moving through a prescriptive flow. ^{1, 2}

That distinction is not academic. Gartner’s definition of agent washing is precisely the rebranding of chatbots, RPA, assistants, and other conventional automation as “agentic AI.” Gartner says only about 130 of the thousands of vendors claiming agentic AI are actually building real agents, and it warns that more than 40% of agentic AI projects could be canceled by the end of 2027. ^{3, 4}

"If the ‘agent’ only handles discrete tasks that are defined by the user, if it only works inside its own system, or if it’s only accessible through chat, it may in fact be an AI capability, but it’s not an agent — it’s an automation or it’s a chatbot."

— Sagi Eliyahu, Tonkean ⁵

That’s a useful screening question for builders too. If the system cannot handle an unanticipated outcome without a human or a fixed fallback, it may still be valuable software. But it is not autonomous in the way the label suggests.

The second test: can it survive more than a few steps?

Agent marketing often leans on demos with a handful of polished actions. The problem is that reliability decays fast in long loops. One source cites a 95% per-step success rate collapsing into only 60% over ten steps and 35% over twenty. Another says the same structural issue differently: AI agents are reliable for short bursts but struggle with end-to-end autonomy, with 95% per-step accuracy cascading into just 8% total reliability over 50 steps. ^{6, 7}

That is why many “agents” look impressive in clips but fragile in production. A model can do a short sequence of actions well, especially when the environment is clean and the task is narrow. But the more steps, the more opportunities for a small error to compound into a dead end, a loop, or a silent failure.

"Even if your underlying model has a 95% success rate at selecting and formatting a tool call, a ten-step agent loop only has a 60% chance of finishing successfully."

— Why AI agents in 2026 are mostly just expensive shell scripts ⁶

This is where agent washing gets expensive. A workflow that looks autonomous in a demo may actually be a brittle chain that only works when the world behaves exactly as expected. In production, the world rarely does.

When “more context” is just a nicer wrapper on the same bottleneck

Vendors also like to equate bigger context windows with stronger agency. The sources push back on that. Larger context windows expand the range of interaction, but they do not inherently improve intelligence or logic accuracy. Modern agentic workflows still need the model to inspect the relevant context, select tools, and decide what matters. If the system is just stuffing more history into the prompt without improving selection or verification, it is not becoming smarter. It is becoming more expensive. ^{8, 9}

The practical warning from the context-engineering source is especially relevant: do not equate expanded context space with increased problem-solving capability. ⁸

That matters for founders because context stuffing is one of the easiest ways to sell “agentic” value without solving the hard problem. You can build a product that looks more capable because it remembers more, handles more tokens, and chains more calls. But if it still cannot choose well, verify well, or recover from failure, the user is paying for bulk, not autonomy.

The real differentiator is orchestration, not rhetoric

Some of the strongest evidence in the source set points to architecture, not model branding, as the deciding factor. In one comparison, switching from a heavier Codex-style nested agent setup to Claude Code workflows reduced token usage by about 75%. The key point was not that one model was magically smarter, but that harness design changed behavior substantially. ¹⁰

That same idea shows up elsewhere in a more direct form: the most effective AI automation strategy is not better prompts, but rigid infrastructure — scoped permissions, codified knowledge docs, and deterministic verification loops. If a process can be solved with a simple script or static workflow, it should be. Reserve agentic complexity for tasks that genuinely require reasoning. ¹¹

"The choice of harness significantly changes how models behave, proving that architectural instruction is just as influential as raw model weights."

— 1 Minute Signal coverage of Theo - t3․gg ¹⁰

This is one of the central anti-hype lessons for builders. A lot of “agent” differentiation lives in orchestration layers: permissions, memory, retries, fallbacks, evaluation, and human review. If those pieces are missing, the label is doing more work than the product.

Human review is not a temporary compromise

A recurring pattern in the source material is that the systems which actually work in production are not fully autonomous. They are collaborative, human-steered, and constrained. IBM Research says 68% of production agents execute 10 or fewer steps before human intervention, 70% rely on prompting off-the-shelf models, and 74% depend primarily on human evaluation. ¹²

That should change how teams think about maturity. A product is not less real because a human remains in the loop. In many cases, the human is what makes the system safe, auditable, and useful. The opposite claim — that autonomy is the only marker of sophistication — is one of the most misleading forms of agent washing.

Dex Horthy’s “lights-off” software factory is a useful cautionary example here. The stated lesson from that experiment was not that AI-generated code is worthless, but that fully autonomous loops tend to fail on long-term maintainability because models do not understand existing codebases as well as they generate snippets. The source’s own conclusion is blunt: engineering gains come from human-steered leverage, not total autonomy. ¹³

"The primary failure mode of fully autonomous agentic loops is not correctness but long-term maintainability, as models often fail to read and understand existing code structure as well as they write new snippets."

— 1 Minute Signal coverage of The Pragmatic Engineer ¹³

For founders, this means a mature AI workflow may look less like “hands off” and more like “well supervised.” That is not a bug. It is often the product.

A practical way to spot agent washing

The most useful screening questions are boring, which is usually a sign they are right.

First, ask whether the system can revise its own next step after observing the result of its action. If not, it is likely a script, router, or workflow with an AI veneer. ^{1, 2}

Second, ask whether it can handle novel inputs or only pre-approved paths. Harvard’s corporate-governance guidance notes that if a workflow requires approval before external communication, it should not be marketed as autonomous execution. The more specific the claim, the easier it is to verify against real behavior, failure rates, and human involvement. ¹⁴

Third, ask what happens after step three, step ten, or step fifty. Reliability should be measured as a floor, not a demo ceiling. Trajectory-opaque evaluation is a problem precisely because it can hide safety and robustness failures behind a successful final output. ^{15, 16}

Fourth, ask what the system does when the environment breaks its assumptions. The production failures that matter are often not glamorous: missing dependencies, weird UI states, stale context, or a prompt that creates a loop instead of progress. That is why deterministic scaffolding, test suites, audit logs, and observability matter more than the agent label itself. ^{9, 17}

What to build instead

If you are building, the goal is not to avoid agents entirely. It is to stop confusing different layers of automation.

Use deterministic workflows when the task is well defined. Use agents when the system truly needs to re-evaluate state and choose among multiple paths. Use human review when the cost of a wrong step is high, or when the environment is too chaotic for the model to recover alone. And do not treat a language model wrapped around a script as a breakthrough just because it can narrate its own steps. ^{11, 18, 19}

The cleanest summary in the source set is also the most useful for product teams:

"Most problems in production do not require dynamic orchestration. They require reliable execution."

— Michał Nasternak ¹⁹

That is the real anti-agent-washing test. If your product’s value comes from reliable execution, call it that. If it can truly observe, decide, and revise across a meaningful loop, then you may have something closer to an agent. The market is full of systems that borrow the vocabulary of autonomy while depending on the safety rails of ordinary software. Don’t buy the vocabulary. Inspect the loop.