The system under execution cannot change with that tool call, yet every call returns success. The executor receives a successful tool result each time, so from inside the loop the tool looks like it is working.

My position: agents loop because there is no definitive check of success or failure. The fix is a deterministic check of the goal against the state of the system, run by a review step that sits outside both planner and executor and reports only to the planner.

Why the ReAct Loop Has No Exit

A ReAct-style agent loop ends when the model stops asking for tools. Strands Agents describes its loop exactly that way: call the model, run any tools it requests, call the model again, and repeat until it produces a final response [1]. In Strands 1.62.0 no turn or token limit applies unless you pass one [1][2]. The OpenAI Agents SDK defaults to 10 turns and raises MaxTurnsExceeded [8]; LangGraph defaults to 1,000 steps and raises GraphRecursionError [9]. Either way, the loop's only exit is the model's own decision to stop, plus whatever budget you remembered to set.

Repeated identical calls are a measured failure, not an anecdote. One study counted instances where an agent called the same tool with identical arguments more than once. With step-by-step planning, Gemini-3-Flash did so in 45.5% of multi-objective HotpotQA instances and GPT-5-mini in 39.5% [10]. The repeats followed a failed or empty tool result [10].

On live websites, 34.2% of gpt-5-nano's actions produced no change in the page at all, and 10.4% of its failures came from repeating one action more than three times [12]. Across 1,794 Terminal-Bench trajectories, failed runs lasted a median of 27 steps, and "keeps repeating the same approach" accounted for 29% of all wasted steps [11].

Running out of context does not stop the loop either. When Strands catches a ContextWindowOverflowException, Agent asks its conversation manager to reduce the context and runs the cycle again [4]. The default SlidingWindowConversationManager keeps 40 messages and pins none, so the first message to be trimmed is the original task [5]. The agent keeps calling tools after it has lost the instruction that told it when to stop.

What a Review Step Is

A review step is the component that decides whether the goal is met. Most agent designs leave that decision to the model that is doing the work, and the evidence says this is the weakest place for it.

Anthropic reports that agents "reliably skew positive when grading their own work" and that separating the doer from the judge is a strong lever [14]. In a 54-cycle pilot study, an agent that judged its own progress claimed improvement every cycle, while 56% of cycles showed no measured gain or a decline [13]. A stronger in-band judge still accepted real regressions; the effect disappeared when success could be verified from the artifact itself [13]. On Terminal-Bench, 26% of failed trajectories fabricated success [11].

In my opinion, a review step is a deterministic check of the goal against the state of the system. It belongs to neither the planner nor the executor. It reports to the planner and never talks to the executor.

The definition puts the evidence in the environment. Anthropic's eval guidance makes the same distinction: the outcome is the final state of the environment, so a flight-booking agent passes when a reservation exists in the database, not when its message says the flight is booked [15]. A code-based check is cheap and reproducible; its cost is brittleness when a valid result does not match the expected pattern [15].

How the Planner/Executor Split Places Each Check

The sample runs on Strands Agents 1.62.0. ServiceEnv is a service that is down because its db_url points at a decommissioned database. restart_service always returns "restart ok" and changes nothing the goal can see. It is the tool from the hook: it succeeds, and nothing moves. Read the diagram from the top.

Orchestrator.run() owns the round budget, max_rounds. Each round it gives the planner the goal, the current state snapshot and the reviewer's findings from the previous round.

The planner is a Strands Agent with no tools. It writes a numbered plan and executes nothing.

The executor is a fresh Strands Agent each round, with the service tools, two bounds and nothing else in its history. Its first message is the plan text, so no reviewer output and no previous round's transcript ever reach it. The first bound is limits=Limits(turns=8, total_tokens=40_000) on every call, which ends the run with stop_reason="limit_turns" or "limit_total_tokens" instead of an exception [3]. The second is NoProgressGuard, a BeforeToolCallEvent hook:

# src/loopstop/guard.py
    def _before_tool_call(self, event: BeforeToolCallEvent) -> None:
        name = event.tool_use["name"]
        args = json.dumps(event.tool_use["input"], sort_keys=True)
        key = (name, args, self._fingerprint())
        if key not in self._seen:
            self._seen.add(key)
            return
        self.tripped = f"{name}({args}) repeated with no state change"
        logger.warning("no_progress tool=%s args=%s state=%s", name, args, key[2])
        event.cancel_tool = self.tripped
        event.agent.cancel(after_current_tools=True)

The guard keys every call on its name, its arguments and a hash of the environment state. A deterministic tool called with the same input from the same state cannot produce a new outcome, so the second such call is pure cost. Setting cancel_tool replaces the call with an error result [6], and agent.cancel(after_current_tools=True) exits the loop before the next model call [4]. The executor never gets a turn to reason about the cancellation. In the test that scripts the restart loop, the run ends after two model calls and one restart.

GoalCheck.review() is the review step. It compares two snapshots of ServiceEnv and nothing else:

# src/loopstop/review.py
    def review(self, before: dict[str, Any], after: dict[str, Any]) -> Verdict:
        reasons = self.unmet(after)
        if not reasons:
            return Verdict("pass")
        if after == before:
            return Verdict("stalled", ("no state change this round", *reasons))
        return Verdict("fail", tuple(reasons))

On a pass, the run ends. A round that changed nothing ends it as stalled, whatever the executor said it did. On a fail, the orchestrator sends the reasons, such as service unhealthy with db_url=..., back to the planner for the next round. The full sample is in the tensorup-examples repo. Its 13 offline tests (uv run pytest -q) cover the restart loop, a transcript that claims success the state does not show, both limits, the round budget, and an executor whose second round starts from the plan alone.

I also ran it live against gpt-6-luna through the OpenAI Responses API, on an earlier version of the sample whose error message named the new database only as "db-new". In round 1 the executor scaled the service to three replicas and set db_url to postgres://db-new:5432/orders, a reasonable guess that was wrong, and GoalCheck returned fail. In round 2 it called get_status again from the same state, NoProgressGuard cancelled the repeat, and the run ended stalled after 11 model calls. The guard and the review step stopped a task the agent could not solve; they could not make it solvable. The current sample's error message names the full URL, and with it the same run passed in round 1 after 8 model calls.

The split works because each part sees a different thing: the planner sees state and findings, the executor sees a plan, and the review step sees only state. The next question is what the alternatives cost.

Design Decisions and Trade-offs

Turn limits alone. A cap stops the bill, and Strands' Limits stops gracefully with the message history intact [3]. It does not say whether the run failed or was about to succeed, and choosing the number is guesswork. In the sample the cap is a backstop and the guard is the detector.

Strands' GoalLoop plugin. Strands ships a goal plugin that validates the agent's response after each invocation [7]. It accepts a code validator or a natural-language goal judged by a second agent.

When validation fails, the plugin feeds the feedback back into the same agent as a new user message. That makes the executor its own reviewer's audience, which is the coupling this design avoids. Its max_attempts and timeout both default to infinity, and it warns that execution is then unbounded [7]. The default judge reads the transcript, with each tool result cut to 500 characters [7].

An LLM judge instead of code. A model-based grader handles open-ended goals that no assertion can express, at the cost of being non-deterministic and needing calibration against human graders [15]. For a goal you can state as a predicate over state (a healthy service with three replicas, say), code is the better reviewer. Anthropic's long-running agents work made features start as "failing" in a JSON file that agents could change only by flipping a passes field after testing [16].

Stopping on the first stall. The orchestrator ends the run after one round with no state change, rather than letting the planner try again. A planner given the same state and the same findings tends to produce the same plan. The cost is that a stall caused by a transient fault ends a run that a retry would have rescued.

Where It Breaks

The same call from the same state. Symptom: tool-call count climbs while every result is a success. Cause: the tool cannot change the state any further, and the loop has no way to notice [10][12]. Fix: fingerprint the goal-relevant state around each call and stop on a repeat, as NoProgressGuard does. Run tools sequentially: Strands' default ConcurrentToolExecutor runs a batch in parallel, and the fingerprint must not change under the guard [4].

A done check that reads the transcript. Symptom: the run reports success and the system is still broken. Cause: the judge sees the agent's claims, and failing agents fabricate success late in a run [11][13]. Fix: check the environment, as GoalCheck does; the sample's test with a "service is healthy" message and no state change ends stalled.

A fingerprint that is too coarse or too fine. Symptom: the guard stops a polling tool on its second call, or never fires at all. Cause: the fingerprint holds something that changes on every call (a timestamp, a restart counter), or misses something that matters. Fix: hash only the fields the goal is defined on, which is why restarts sits outside ServiceEnv.snapshot(), and give polling tools an exemption or a time field.

Checklist

• Write the success condition as code over system state, not over the agent's final message.
• Pass limits with turns and total_tokens on every Strands executor call [3].
• Fingerprint goal-relevant state before each tool call and cancel a repeat from the same state.
• Run the executor's tools sequentially when a hook reads shared state.
• Send review findings to the planner only; start each executor run from the plan alone.
• Cap planner rounds and end the run on a round with no state change.
• Pin the task message (pin_first) if you rely on the sliding window manager [5].
• Log the stop reason of every executor run and every verdict.

My verdict: an agent that cannot fail will loop, and the only reliable way to let it fail is a check that does not depend on the model's opinion of its own work. In my case the agent kept running for hours, through 50 to 100 tool calls, until it filled the context window and threw an exception, on a task that should have passed or failed within a few seconds.

What does your agent use to decide it is done?

References

1. Strands Agents, Agent Loop, docs, updated 2026-10-21.
2. Strands Agents, event_loop/event_loop.py, GitHub, python/v1.62.0, 2026-10-21.
3. Strands Agents, types/agent.py (Limits), GitHub, python/v1.62.0, last changed 2026-10-16.
4. Strands Agents, agent/agent.py, GitHub, python/v1.62.0, last changed 2026-10-16.
5. Strands Agents, sliding_window_conversation_manager.py, GitHub, python/v1.62.0, last changed 2026-06-16.
6. Strands Agents, hooks/events.py, GitHub, python/v1.62.0, last changed 2026-09-25.
7. Strands Agents, vended_plugins/goal/ (GoalLoop), GitHub, python/v1.62.0, last changed 2026-09-30.
8. OpenAI, run_config.py (DEFAULT_MAX_TURNS), OpenAI Agents SDK, GitHub, v0.24.0, 2026-10-15.
9. LangChain, LangGraph Graph API: recursion limit, docs, accessed 2026-10-25.
10. Do Agents Need to Plan Step-by-Step? Rethinking Planning Horizon in Data-Centric Tool Calling, arXiv, 2026-05-08.
11. Failure as a Process: An Anatomy of CLI Coding Agent Trajectories, arXiv, 2026-07-10.
12. Why Do LLM-based Web Agents Fail? A Hierarchical Planning Perspective, arXiv, 2026-03-15.
13. Park and Choi, When Do Agent Loops Mistake Stagnation for Progress?, arXiv, v2 2026-09-29.
14. Rajasekaran, Anthropic, Harness design for long-running application development, 2026-03-24.
15. Anthropic, Demystifying evals for AI agents, 2026-01-09.
16. Anthropic, Effective harnesses for long-running agents, 2025-11-26.

Nothing in the prompt text changed. What changed is how one layer of the prompt was delivered. Anthropic's structured outputs reject maxLength with a 400, so LiteLLM 1.106.0 removes it from the JSON Schema and moves the limit into the field description as a note [4][6]. Gemini accepts the keyword and silently ignores it [5]. Both paths turn a hard constraint into a suggestion.

My position is that a prompt is a layered program, and each prompt engineering technique targets exactly one layer. Once you know which layer a technique touches, you know what will break when the model or provider underneath it changes.

Why One Tuned String Breaks on the Next Model

The common approach treats a prompt as a single string, tuned by trial and error against one model until the outputs look right. That tuning fits one model's quirks. PromptBridge measured the cost on HumanEval: the best prompt for GPT-5 scored 68.70% when reused on Llama-3.1-70B-Instruct, against 79.47% for Llama's own best prompt [1]. Moving to a stronger model does not avoid it either; a prompt engineered for GPT-4o reached 92.27% on o3, below the 98.37% that o3 achieves with a prompt optimized for it [1]. Pure formatting sensitivity has shrunk on current closed-weight models, which show little accuracy loss when asked for JSON, but open-weight models still pay that cost [7].

OpenAI's own guidance points the same way. It recommends pinning production applications to dated snapshots and building evals, since different models may need different prompts [2]. A string tuned for one snapshot is a program compiled for one target, with no record of which parts depended on that target.

The fix starts with naming the parts.

What a Layered Prompt Program Is

A prompt program is the set of inputs that define one model call, kept as separate layers, each of which travels through a different slot of the API request. There are six:

• L1 instructions: the rules for every request. OpenAI sends them as a developer or system message [2]; Anthropic takes system as a top-level request parameter, outside the message list [3].
• L2 tools: tool schemas in the tools field, with optional strict validation [13][4].
• L3 examples: demonstrations, sent as prior user and assistant turns or as tagged blocks [3].
• L4 context: retrieved documents. Anthropic and Google both recommend placing long documents above the query [3][10].
• L5 task: what this particular request asks for.
• L6 output contract: the schema the answer must satisfy, sent as response_format on OpenAI [13], output_config.format on Anthropic [4], and a schema subset on Gemini [5].

Request configuration such as temperature or reasoning effort is not a prompt layer, but it breaks on model upgrades in the same way, so the sample treats it alongside the layers.

In my opinion, the useful definition of a prompt engineering technique is the layer it edits. If you cannot say which part of the request a technique changes, you cannot say what will break when the model changes.

Which Layer Each Technique Changes

Mapping the well-known techniques to layers shows that most of them edit exactly one, and that two of the most cited ones are not prompt edits at all. The evidence column uses only studies and provider guidance from the past year, measured on current models.

The table also explains why some techniques age badly. A chain-of-thought trigger lives in L5, and ReasonLab found it worth about 4 points without a reasoning budget and nothing measurable once reasoning is enabled, while enabling reasoning itself was worth up to 12.74 points [9]. Google's Gemini 3 guide tells you to replace chain-of-thought prompt engineering with a high thinking level and simpler prompts [18]. Anthropic's current guide warns that emphatic "CRITICAL: You MUST" phrasing written for older models now overtriggers [3]. Instructions that once improved results became instructions you must delete on upgrade.

How the Compiler Places Each Layer

The sample keeps the six layers in a frozen PromptProgram dataclass and compiles them per model. Read the diagram from top to bottom.

PromptProgram owns the layers and nothing else. It holds no provider name or request syntax, so the same program can target any model.

ModelProfile owns every provider difference the code knows about. It records whether the model can enforce a JSON schema and whether it accepts temperature. It also lists the schema keywords to strip, and its optional instruction_override replaces L1 for that model. profile_from_litellm() fills the capability flags from LiteLLM's bundled model map, using supports_response_schema() and get_supported_openai_params() [6].

compile_request() takes both and emits the keyword arguments for LiteLLM's completion(). Each layer goes to its own slot:

# src/promptlayers/compile.py
def compile_request(
    program: PromptProgram, profile: ModelProfile, *, temperature: float | None = None
) -> dict[str, Any]:
    # L1: LiteLLM moves a "system" message to Anthropic's top-level system field.
    instructions = profile.instruction_override or program.instructions
    messages: list[dict[str, Any]] = [{"role": "system", "content": instructions}]
    for ex in program.examples:  # L3: demonstrations as prior turns
        messages.append({"role": "user", "content": ex.input})
        messages.append({"role": "assistant", "content": ex.output})
    docs = "".join(f"<document>\n{doc}\n</document>\n" for doc in program.context)
    messages.append({"role": "user", "content": f"{docs}<task>\n{program.task}\n</task>"})  # L4, L5

    request: dict[str, Any] = {"model": profile.model, "messages": messages}
    if program.tools:  # L2
        request["tools"] = list(program.tools)

The function uses the system role for L1 because LiteLLM's Anthropic transform moves system messages into Anthropic's top-level system field [6]. Context goes above the task inside one user message, which follows the placement advice in both Anthropic's and Google's guides [3][10]. L6 becomes a strict json_schema with additionalProperties: false on every object and the non-portable keywords removed. When the profile says a layer cannot be carried, the function raises UnsupportedLayerError instead of sending a weaker request.

run() sends the request through LiteLLMModel, which calls litellm.completion() with drop_params=False, a timeout and a retry count. It then validates the raw text against the full Pydantic model, including the constraints that never reached the provider. A failed validation is appended as feedback and retried within max_attempts; when the budget runs out, run() raises ContractError. A reply that contains tool calls raises ToolCallError, because this sample has no tool loop, and a tool call is not a malformed answer.

The full sample is in the tensorup-examples repo; uv run pytest -q in its folder runs 17 offline tests that cover each layer's placement and every failure mode below.

Design Decisions and Trade-offs

LiteLLM as the translation layer. LiteLLM accepts OpenAI-shaped requests and translates them per provider, including the system-message move and the schema filtering described above [6]. Per-provider SDK adapters give full control over each request at the cost of one adapter per provider to write and keep current. I chose LiteLLM because it gives agent framework SDKs one provider-agnostic model class: Strands Agents, for example, takes a LiteLLMModel, and the LiteLLM model id decides the provider [19]. It also exposes prompt caching for OpenAI, Anthropic and Gemini through the same interface, which matters for token cost [20]. Check the cache counters in the response's usage object, though: LiteLLM's docs warn that caching is skipped without an error when the input is below the provider's minimum, which is 1,024 tokens on OpenAI [20].

A profile you can override. LiteLLM's model map is a convenient starting point, but it is a third-party description of each provider. profile_from_litellm() therefore accepts overrides for any flag:

# src/promptlayers/profiles.py
    params = litellm.get_supported_openai_params(model=model) or []
    detected = {
        "supports_json_schema": litellm.supports_response_schema(model=model),
        "supports_temperature": "temperature" in params,
        # Send only the portable schema subset everywhere; Pydantic enforces the rest.
        "rejected_schema_keywords": NON_PORTABLE_KEYWORDS,
    }
    return ModelProfile(model=model, **{**detected, **overrides})  # type: ignore[arg-type]

The portable schema subset everywhere. The sample strips minimum, maximum, minLength and maxLength for every provider, including OpenAI. The cost is that OpenAI no longer enforces those bounds while decoding, so a violation costs one extra call through the retry loop. In return, the schema on the wire is identical for every provider, and one validator enforces the full contract on the client. Anthropic's Python SDK makes the same choice: it removes unsupported constraints and validates responses against the original schema [4].

Fail loud over drop_params. LiteLLM can silently remove parameters a provider does not support when drop_params=True [14]. That keeps requests from failing, and it also hides configuration that no longer has any effect. The sample keeps LiteLLM's default of raising and adds its own compile-time check, so the failure appears in CI rather than in production output.

Where It Breaks

Silently dropped parameters. Symptom: a model change that should have altered nothing makes outputs noticeably more varied. Cause: drop_params=True removed temperature, which OpenAI's guide for its latest models says must be removed whenever reasoning effort is not none [15][14]. Fix: leave drop_params off, record supports_temperature per model, and let compile_request() raise UnsupportedLayerError.

An output contract that became advisory. Symptom: values outside the declared range reach downstream code, with no errors logged. Cause: the provider or the translation layer removed the constraint from the schema; LiteLLM does this for Anthropic, and Gemini ignores minLength and maxLength [6][5]. Fix: treat the wire schema as a decoding hint and validate the raw output with the full Pydantic model on every response, with a bounded retry.

Instructions tuned for the previous model. Symptom: after an upgrade, the model overthinks, re-verifies its own answers or follows an emphatic rule too literally. Cause: L1 and L5 still carry chain-of-thought triggers and emphatic phrasing written for an older model [3][18]. Fix: pin the snapshot [2], keep a per-model instruction_override, and run the same eval set against the new snapshot before switching.

Checklist

• Store instructions, tools, examples, context, task and output schema as separate fields, never as one string.
• Pin every production model to a dated snapshot [2].
• Record each model's capabilities in one profile object and keep provider checks out of prompt code.
• Turn off drop_params and fail at compile time on unsupported parameters.
• Send a portable schema subset and validate every response with the full schema on the client.
• Remove chain-of-thought triggers from prompts that target models with reasoning enabled [9][18].
• Bound retries with an attempt budget and raise a named error when it runs out.
• Run the same eval set against a new model or provider before switching traffic.

My verdict: treat a prompt as source code with a build step. Write the layers once, keep every provider difference in one profile, and make the compiler fail loudly when a model cannot carry a layer. I learned this when a model upgrade and a provider switch broke multi-provider support that had looked finished, and the fix was structural: separating the layers, never rewording the prompt.

Which layer broke first the last time you switched models or providers?

References

1. Wang et al., PromptBridge: Cross-Model Prompt Transfer for Large Language Models, arXiv, 2025-12-01.
2. OpenAI, Text generation guide, accessed 2026-10-23.
3. Anthropic, Prompting best practices, accessed 2026-10-23.
4. Anthropic, Structured outputs, accessed 2026-10-23.
5. Google, Gemini API structured outputs, updated 2026-10-21.
6. LiteLLM, litellm/llms/anthropic/chat/transformation.py, v1.106.0, 2026-10-18.
7. Lee, D'Antoni and Berg-Kirkpatrick, The Format Tax, arXiv, 2026-04-04.
8. IH-Benchmark: A Conflict-Centered Benchmark for Instruction-Hierarchy Robustness in LLM Applications, arXiv, 2026-07-28.
9. ReasonLab: A Controlled and Auditable Evaluation of Prompting Techniques for Multiple-Choice Question Answering, arXiv, v2 2026-09-22.
10. Google, Gemini API long context, updated 2026-10-15.
11. MiniMax, Interleaved Thinking Unlocks Reliable MiniMax-M2 Agentic Capability, 2025-11-03.
12. The Structured Output Benchmark: A Multi-Source Benchmark for Evaluating Structured Output, arXiv, 2026-04-28.
13. OpenAI, Structured model outputs, accessed 2026-10-23.
14. LiteLLM, drop_params, updated 2026-10-21.
15. OpenAI, Using GPT-6 (latest model guide), accessed 2026-10-23.
16. Althoubi, Reasoning Concentrates Errors, and Self-Consistency Never Notices, arXiv, 2026-09-25.
17. Yin, Ken and Stremmel, Denoising Iterative Self-Correction: Structured Verification Loops for Reliable LLM Reasoning, arXiv, 2026-06-19.
18. Google, Gemini 3 developer guide, updated 2026-10-09.
19. Strands Agents, strands-py/src/strands/models/litellm.py (LiteLLMModel), last changed 2026-10-16.
20. LiteLLM, Prompt caching, updated 2026-10-21.