Your Agent Keeps Leaking Its Scratchpad Prompting will not fix it

The Thought / Action / Observation trace bleeds in front of the real answer, and no system prompt makes it stop. The leak is structural. Here is what actually does.

June 25, 2026 12 min read
Where the leak comes from Why a better prompt is limited Move the boundary into the structure Two minds instead of one Isolate the context The model layer settled this What to reach for, in order References

We shipped an agent that plans, calls tools, then replies. Most turns were clean. Every so often a user got this:

Thought: The user wants their account balance. I'll call get_balance with their id. Action: get_balance(user_id="4471") Observation: {"balance": 182.40, "currency": "USD"} Your balance is $182.40.

Only the last line was meant for them. The rest is the model's private bookkeeping, the tool call, and the raw tool result, pasted in front of the real answer.

A bright aerial view of a parcel-sortation facility - conveyor belts splitting a single stream into separate sorted lanes

The system prompt said not to do this. It said it several ways: do not reveal your reasoning, never print tool calls, output only the message for the user. Compliance was high. It was not total. At any real traffic level, a leak rate of even a fraction of a percent means some users see the internals every day, and the ones who see it are the ones most likely to screenshot it.

This looks like a prompting problem. It is a structural one. Here is where the leak comes from and what actually makes it stop.

Where the leak comes from

The scaffold that makes tool use work is the same scaffold that leaks. Most agent loops descend from ReAct (Yao et al., 2022), which interleaves reasoning traces with actions: a Thought, an Action, an Observation fed back from the environment, repeated until the model produces an answer. That format is why the agent can plan and recover from a failed tool call. It is also a rhythm. Once the model is a few cycles into Thought, Action, Observation, the most probable way to start the next turn is often another Thought, even on the turn where the next thing should have been a plain sentence to the user.

Underneath that sits the real issue: one text stream is carrying two audiences. The model's working notes and the user's answer go down the same channel, and the only thing dividing them is the model's moment-to-moment decision about where the private part ends. That decision is a generation. Generations move around with temperature, context length, and how unusual the current turn happens to be.

private notes tool call + result the user's answer
One channel, two audiences. The only divider is the model's moment-to-moment guess about where the private part ends.

Negative instructions are weak fences here. "Do not reveal your reasoning" asks for the suppression of a behavior rather than the production of a specific shape, and suppression is the first thing to degrade under distribution shift. A long context, a tool that errored in a new way, a user message that itself reads like reasoning, a plan that ran longer than the examples: any of these can tip the model back into leaking. The instruction is still in there competing. It is one voice among many. And nothing in the basic loop looks at the output before it ships. The first reviewer of a leak is the user.

Why a better prompt only buys you so much

The instinct is to escalate. More emphasis, more do-not examples, a sterner tone. This lowers the leak rate. It does not zero it, and it taxes every single call to chase a failure that shows up on a few.

State it flatly

You are trying to guarantee a structural property - that internal content never reaches the user - with a probabilistic mechanism, an instruction the model samples against each turn. No wording converts a probability into a guarantee. You can drive the failure rate down. You cannot drive it to zero, and from outside the model you cannot tell which turns will slip.

So stop asking the model to police the boundary, and put the boundary where the model does not get a vote.

Move the boundary into the structure

Give the model a typed output, not a blob of text

Instead of free text you hope ends with a clean answer, ask for an object with named fields and read only the field meant for the user.

{ "scratchpad": "User wants balance. Called get_balance(4471) -> 182.40 USD.", "reply": "Your balance is $182.40." }
reply

Your code returns this field to the user.

scratchpad

Discarded. The model can reason all it wants here - the field never leaves your server.

The boundary is no longer a behavior the model has to remember. It is a key your parser ignores. This is sturdier than it sounds, because on the major APIs the structure is enforced during decoding, not requested politely. OpenAI's structured outputs constrain generation against your JSON Schema by masking any token that would break the schema, so the model cannot emit a response that fails to parse.

100%
schema-following with constrained decoding, up from below 40% on an older model without it (OpenAI, 2024). "Reasoning leaked into the answer" stops being "the model ignored an instruction" and becomes the much rarer "the model wrote internal text inside the reply field specifically."

This is the production-standard move, not a novel one. LangChain's agent executor has done a lighter version for years: intermediate reasoning and tool observations accumulate in an agent_scratchpad that stays inside the prompt, and the executor parses out only the final answer to return. Same principle, enforced by parsing rather than by decoding.

Two caveats

Structured output does not make the model correct about what belongs in which field, only about the shape. And it adds a failure mode: the model can refuse, a refusal does not match your schema, so handle it as its own branch instead of letting it blow up the parser.

Or separate the channels at the protocol level

The same idea appears one layer down. OpenAI's Harmony format, used by the gpt-oss models, routes every assistant message to one of three channels. The application renders final and keeps the other two server-side. It is the typed-output contract baked into the message protocol rather than hand-rolled, which means tool calls and reasoning have a defined place to live that is not the place the user reads.

analysis

The chain of thought.

Server-side
commentary

Tool calls and preambles.

Server-side
final

The user-facing answer.

Rendered

Make extraction fail closed

Whatever the contract, the code that pulls the user-facing text has to fail closed. If the response does not parse, or the reply field is missing, do not fall back to shipping the raw model output. That fallback is the most common way leaks reach production. Someone wraps the parse in a try block and, on failure, returns the original string. Return a safe canned message, retry with a repair prompt, or raise an error. Anything except the unparsed stream.

Keep a dumb backstop on the way out

Structured outputs cover the common case. For the residue, run a cheap deterministic check on the text about to be sent, and reject or scrub anything shaped like internals: Thought, Action, or Observation prefixes, tool-call syntax, a JSON object that looks like a tool result. This is the output-rail pattern. NeMo Guardrails formalizes it as a stage between your application and the user that can block or rewrite a draft before delivery. A regex pass will miss subtle leaks. It reliably catches the blatant "Thought: ..." bleed, which is the one that embarrasses you, and it runs in microseconds.

The part most people underestimate: streaming

A typed output only parses once it is complete, but users expect tokens to appear as they generate. Stream the raw model output to the screen and you are back where you started, because the scratchpad streams too.

Decide the streaming strategy before you ship

Parse incrementally and stream only the reply field; or generate the reasoning in a separate non-streamed call and stream only the composed answer; or validate the stream in chunks against a sliding buffer, the way NeMo Guardrails does, holding back a small window so a violation straddling two chunks still gets caught. Naive implementations tend to regress here after everything else is fixed.

The heavier option: two minds instead of one

If reasoning and conversation are genuinely separate jobs, split them across two model roles rather than two fields. Google DeepMind's Talker-Reasoner architecture (Christakopoulou et al., 2024) does this, framed as Kahneman's two systems.

System 2 · slow

Reasoner

Runs the multi-step planning, calls the tools, forms beliefs about the world. No channel to the user.

beliefs
System 1 · fast

Talker

The only component that speaks to the user, working from the Reasoner's beliefs.

only the Talker has a line to the user · the Reasoner's internals cannot leak by construction

They validated it on a sleep-coaching agent and noted a benefit that matters in production: the Talker keeps the conversation moving without blocking on the slow reasoning step, which cuts perceived latency. MIRROR (2025) carries the same split into multi-turn chat with a Talker and a Thinker, where the Thinker reflects between turns and the reasoning persists across turns without being pasted into the visible dialogue. Their own phrase for the goal is avoiding history pollution, which is this exact concern under a different name.

The cost is real: two model calls, more orchestration, more state to keep coherent. For a plain tool-using assistant, the typed-output contract is enough and much cheaper. Reach for two minds when the reasoning is heavy and you want it off the user's critical path anyway.

The strongest version: isolate the context, not the paragraph

Everything up to here still runs reasoning and reply through one model in one context, then sorts them apart by field, channel, or filter. There is a cleaner cut: put the reasoning in a different context window entirely, so there is no shared stream for anything to leak across.

Claude Code does this with subagents. When the main agent hands a task to a subagent, that subagent runs in its own fresh context window with its own system prompt and its own restricted set of tools. The exploration, the tool calls, the intermediate results, the dead ends, all of it stays inside the subagent's window. The only thing that crosses back to the parent is the subagent's final message.

User-facing
Parent context
  • The clean conversation
  • Receives only the final message
prompt final only
Fresh window
Subagent context
  • Tool calls and raw results
  • Intermediate steps and dead ends
  • None of it crosses back

Reasoning and reply do not share a stream, so a leak has no boundary to cross.

The channel between the two is deliberately narrow. In the Agent SDK, the only input a subagent gets from the parent is the prompt string passed through the Agent tool, and the only thing that returns is the final result. There is no path for the messy middle to reach the user, because the messy middle was never in the user-facing context to begin with.

That reframes the problem rather than solving a harder version of it. The earlier layers all answer the same question: given that reasoning and reply share a stream, how do we separate them reliably after the fact? Context isolation deletes the premise. For keeping internals out of the user's view, this is the firmest guarantee on the list, since it is the only approach that never relies on correctly splitting two things that were generated together. The tradeoff is the one every multi-context design carries: you pay for extra model calls, and you have to decide what the hand-back summary contains, because too terse a summary drops information the parent still needed.

The model layer already settled this

Frontier reasoning models made the same call, for the same reason. OpenAI's o1 generates a long internal chain of thought and does not show users the raw trace. Users get a model-written summary instead. The gpt-oss guidance is blunter: do not display the raw chain of thought, route it through a summarizer that also strips anything unsafe before anyone sees it.

Carry one caveat with you

Those summaries are not guaranteed to faithfully represent the underlying reasoning, so a summary is a product surface, not an audit log. If you need the real trace to debug, log it server-side. Do not reconstruct it from what the user sees.

What to reach for, in order

1

A typed output contract

Use your provider's structured-output or tool-schema mode, and read only the reply field.

2

Make extraction fail closed

On a parse failure or missing field, never fall back to the raw stream. Canned message, repair retry, or raise.

3

A regex output rail, and the streaming strategy

Add the deterministic backstop and settle streaming in the same change, not later.

4

Treat prompting as the weakest layer

Spend it on one or two positive examples of the exact output shape - including a hard case where reasoning and answer sit close - not a longer list of prohibitions.

5

A two-model Talker / Reasoner split

Only when the reasoning is heavy enough to justify the orchestration. Take the latency win when you do.

6

Context isolation, if you already have it

On a framework with subagents, such as Claude Code, you get the strongest separation almost for free: run the heavy work in a subagent and relay only its final message.

“

The shift is small and it is the entire point: quit asking the model to remember not to leak, and build a system where leaking is not a move it can make.

References

The leak is a Day Two problem in miniature

It looks fixed in the demo and embarrasses you in production. We build agentic systems where the failure is structurally impossible, not prompted against - and instrument them so you see the edge cases first. Get a free expert review of your AI idea or roadmap, read by our Chief AI Officer.

Get your free expert report