When Your AI Agent’s Prompt Lies About Its Tools

/ The failure

How does an AI agent build break with no error?

The request was ordinary: "create a new user from a CF7 submission, use the WP REST API." The agent behind Build with AI did everything right. It read the captured Contact Form 7 payload, read the POST /wp/v2/users contract, saw that password is required but absent from the form, and proposed a six-step plan — including a Code Glue snippet^¹ that generates the password before dispatch.

Then the trace log told a different story:

Trace — the model proposed 6 steps; the plan kept 4

// response_raw: the model's plan (abridged)
"plan": [
  { "id": "step_1", "ability": "create_webhook",   ... },
  { "id": "step_2", "ability": "assign_credential", ... },
  { "id": "step_3", "ability": "set_mapping",       ... },
  { "id": "step_4", "ability": "create_snippet",    ... },  // ← dropped
  { "id": "step_5", "ability": "assign_snippet",    ... },  // ← dropped
  { "id": "step_6", "ability": "test_dispatch",     ... }
]

// what the executor recorded for the same turn
"parsed_ok": true,
"plan_steps": 4

Two steps vanished between the model and the plan review — the two that injected the required password. Every surviving step ran green. The user got a finished build with checkmarks that could only ever answer 400 missing password on a live delivery. No exception, no log entry, nothing to bisect. That is the worst kind of bug an agent can have: the system did exactly what it was written to do.

/ Root cause

Where did the capability lie come from?

The dropped abilities belong to our Pro plugin, which registers them into the agent's ability catalog through a filter. On this site, Pro had quietly deactivated itself: its bootstrap requires a minimum version of the free plugin and bails out at plugins_loaded when the check fails. So far so good — defensive version gating working as designed.

The lie lived one layer up. The system prompt decides whether to tell the model "Pro Code Glue is available" with a capability check — and that check used class_exists on a Pro class. But Pro's Composer autoloader loads unconditionally, before the version gate. The class existed; the plugin behind it wasn't running:

PHP — the check that lied, and the fix

// BEFORE — true even when Pro bailed out at plugins_loaded:
// the autoloader is registered before the version gate, so the
// class exists while the ability filter never ran.
private function proIsActive(): bool {
    return class_exists( 'Pro\License\LicenseManager' )
        && ( new LicenseManager() )->isActive();
}

// AFTER — ask the executable catalog itself. create_snippet is
// registered exactly when Pro booted AND holds an active license,
// so the prompt can no longer promise what the executor lacks.
private function proIsActive(): bool {
    return isset( $this->registry->definitions()['create_snippet'] );
}

The result of the mismatch: the prompt advertised snippet abilities that were not in the catalog, the model (correctly!) planned with them, and the plan normalizer — which validates each step against the catalog — discarded the unknown steps on the way in.

The general shape of this bug is worth naming: capability drift. Any agent whose prompt is assembled from one code path and whose tools are registered by another will eventually see the two disagree — a plugin deactivates, a license expires, a feature flag flips, a module fails to boot. The only durable fix is to make the prompt a projection of the tool registry, never a parallel account of it.

If the executor can't run it, the prompt must not promise it. Capability claims are derived from the tool catalog — the same array the executor validates against — or they are lies waiting for a boot-order bug. — the rule we now build by

/ Silent drops

Why is silently dropping a step worse than failing?

Our plan normalizer was written defensively, and reasonably so: a language model can name a tool that never existed, so unknown abilities were skipped. The mistake was skipping them silently. A plan is not a list of independent conveniences — steps depend on each other. Drop the snippet that injects password and the surviving create_webhook + set_mapping steps aren't a smaller version of the build; they're a different, broken build wearing the same name.

Failing loudly would have surfaced this in the first test run. Instead the thinned plan executed cleanly and the defect moved downstream — to a live form submission, days later, on someone else's site. The fix keeps the defensive skip but makes it visible: the normalizer records every dropped ability name, and the turn shows a notice in chat before anyone runs the plan:

Chat notice — what the user now sees instead of nothing

Some proposed steps were removed because this site cannot run
them: create_snippet, assign_snippet. The remaining plan may be
incomplete. If these are Code Glue abilities, make sure Webhook
Actions Pro is installed, up to date and licensed.

The same notice catches genuinely hallucinated tool names — the case the silent skip was originally written for — which previously also vanished without a trace.

/ Read, don't recall

Should the agent read contracts or recall them?

The password bug had a prequel. Weeks earlier, the same build request produced a plan with no password handling at all — because the model recalled the wp/v2/users contract from training data and recalled it wrong. Models are confident about popular APIs precisely because they've seen thousands of near-copies, current or not.

The fix was to stop asking the model to remember and give it a way to look: a get_rest_route_schema read ability that performs an internal OPTIONS request on the site's own REST route and returns the self-declared argument list with required flags — about 1.3 KB for wp/v2/users. The system prompt makes the read mandatory before any internal build. Since then, every trace shows the same pattern: the model reads the contract, sees required: ["username", "email", "password"], and plans around reality.

The honest-refusal path is the part we're proudest of. With Pro genuinely absent, the prompt now says so truthfully — and the model does exactly what it's told:

Trace — the free-tier plan, after the fix (abridged)

"The WordPress REST API requires a 'password' field to create
a new user, but your Contact Form 7 submission does not provide
one. On the Free plugin tier, I cannot inject a generated or
static password. This means the user creation will likely fail
[...] you would need Webhook Actions Pro with Code Glue."

plan: create_webhook (disabled) → assign_credential → set_mapping
plan_steps: 3   — nothing dropped, nothing promised, nothing fake

Same model, same request. The only variable is whether the prompt tells the truth about what the site can do — and a truthful prompt turns out to be the difference between a broken build with green checkmarks and an honest plan with a clear upgrade path.