Learning about agents by building one with agents

Prompt is policy. Code is control flow.

I was tuning one of my agents recently. The problem: it kept skipping steps — either failing to perform the action I wanted, or performing it wrong. Prompt changes didn’t help. The fix was structural: enforce through code, not prose. Four patterns came out of the process.

A prompt is something the model interprets. toolConfig — or its equivalents, OpenAI’s tool_choice: "required", Anthropic’s tool_choice: {type: "any"} — is something the model has to obey. The distinction matters most when you have a behavior that must happen.

The Gemini field looks like this:

functionCallingConfig: {
  mode: "AUTO" | "ANY" | "NONE";
  allowedFunctionNames?: string[];
}

AUTO leaves the model free to choose whether to call a tool and which one. ANY forces it to call a tool, restricted to the names in allowedFunctionNames — the API will reject a non-function-call response. NONE disables tools for the turn. Most turns ride in AUTO; ANY is the precision instrument.

Tool choice (“should I call this function now?”) is control flow. You cannot drive control flow reliably with prose, no matter how loud. The model has its own tool-selection policy and a paragraph of instructions doesn’t override it.

If you’ve ever found yourself writing “you MUST call X” in all caps and watching the model not call X, this is the reason. The instruction is a wish. The API has a guarantee available; use it.

The practical version: when you can name in code the exact moment a tool must fire, force it. Everywhere else, leave the model in AUTO. Most turns flow freely; the few where a behavior is non-negotiable get hard-coded.

The router: deciding in code when to force

Forcing only works if something else decides when to force. That something is a small router — usually a cheap classifier call that runs before the main agent turn and returns a tiny struct:

classifyIntent(message) → {
  intent: "edit" | "question" | "unsupported" | ...
  target?: string
}

The classifier runs only on turns where forcing is even possible (e.g., when the user has write access) and decides which toolConfig the agent gets this turn. Its job is small and bounded: pick a label from a closed set.

You might be tempted to use a regex for this. Don’t. A brittle regex that fires on “move section to the end” will force an edit on a request the system can’t actually satisfy — turning a correct refusal into a wrong action. The few hundred milliseconds of a Flash-tier classifier call buy you a real categorical decision, including the category “I can’t tell, don’t force anything.”

This is the bridge between prompt and code: the prompt no longer needs to teach the model “is this an edit?” — code already decided, and the model receives the consequences of that decision as constraints on the tools it’s allowed to call.

Force the prerequisites before forcing the action

If your agent’s prompt doesn’t contain the data the action operates on, you can’t safely force the action — the model will fabricate the data to satisfy the constraint. A forced edit on a document body the model has never read produces an invented edit.

The pattern is to drive toolConfig as a phase machine:

1. Force retrieval. First force the retrieval tool so the model loads the real content.
2. Force the mutation. Once retrieval has happened, narrow the allowed tools to the mutation tools so the model must commit.
3. Back to AUTO. After the mutation, drop back so the model writes its natural confirmation.

This generalizes past edits. Any time an agent has to commit an irreversible action based on context that isn’t in the prompt — write to a database, send a message, place an order — the same shape applies. Read first, narrow second, free third.

Plain questions stay in AUTO throughout. Requests the system can’t satisfy also stay in AUTO — the model already refuses gracefully when honest, and forcing it would turn a correct refusal into a wrong action.

A wrong forced action is much more expensive than a wrong soft reply, so the classifier’s “I’m not sure” answer should always route to AUTO.

Observability has to grow when you move from soft to hard decisions

Two asymmetries make this non-optional once you start forcing tools.

The first is blast radius. A misfired soft instruction produces, at worst, a wrong text reply. A misfired forced tool call writes to the database, sends the message, places the order. The cost of a wrong decision goes up, which is exactly when you most need to see the decision being made.

The second is legibility. A model running on prose narrates itself — “I think you wanted me to edit, so…” — and you can read that trace after the fact. A model under forced tools doesn’t narrate; it goes straight to the call. The self-documentation disappears the moment you take control. You have to replace it deliberately.

The cheap version is a small routing record persisted on every turn: detected intent, which phases were forced, which tools were allowed, whether the action actually applied. A misfire becomes one glance — “intent=edit on a plain question” — instead of reverse-engineering tool-call arrays from logs.

Key takeaways

1. Prompt is policy. Code is control flow. If a behavior must happen, a prompt instruction is a wish; toolConfig is a guarantee.
2. A router decides in code; the model executes inside the decision. Put a small classifier in front of the agent loop so the prompt doesn’t have to do classification work.
3. Read before commit. Forcing a mutation on context the model hasn’t loaded produces fabrication. Force the prerequisite tool first, then narrow to the mutation tool.
4. Hard decisions need observability. Forced tools have larger blast radius and smaller legibility than prose; instrument the routing decision in the same change that introduces it.