A reader sent me a new security paper with the excellent title “Stealing Reasoning Traces from Proprietary LLM APIs.”
The project website is also unusually good: Stolen Thoughts And the actual finding is slightly alarming.
When reasoning models from Anthropic, OpenAI and Google “think,” their APIs can return opaque encrypted reasoning blocks to the client. Those blocks can then be sent back when the conversation continues.
You can’t normally read what’s inside. So naturally, somebody found a way to read what’s inside.
Stealing a Strong Model’s Thoughts With a Weaker Model
The attack is beautifully simple. Take an encrypted reasoning trace generated by a strong model. In the researchers’ example, that’s Claude Opus. Replay the same block into a weaker sibling model from the same provider, such as Haiku. Jailbreak the weaker model and ask it to transcribe the reasoning attached to the turn. And out comes the stronger model’s hidden reasoning in plaintext.
The important discovery is that these encrypted blocks are portable. They aren’t sufficiently bound to the original user, session or model, so they can be replayed elsewhere within the provider’s infrastructure.
Even better, from the attacker’s perspective, the strong model is never attacked directly. So the anti-distillation protections designed to prevent people extracting proprietary reasoning from frontier models don’t fire.
The basic attack: take an encrypted reasoning block from a stronger model, replay it into a weaker model from the same provider, and make the weaker model transcribe it.
They Appear to Be Recovering Almost Everything
Obviously, the next question is whether this produces fragments or actually reconstructs the reasoning. The researchers tested extraction on 120 Codeforces problems across models from Anthropic, OpenAI and Google. They compared the number of hidden thinking tokens reported by the API with the number of tokens recovered by their decoder.
The two track closely, suggesting the attack is recovering something close to the complete hidden trace rather than a few accidental snippets. That’s already an interesting model-security problem. Then they searched the internet. And things got considerably worse.
315,320 Stolen Thoughts
Developers routinely publish agent trajectories. GitHub repositories. Hugging Face datasets. Debug logs. Benchmark runs. Research artifacts. Some of those trajectories still contain the encrypted reasoning blocks returned by the model APIs.
The researchers collected 6,708 publicly available trajectories produced by Claude, GPT and Gemini. They ran the decoding pipeline across them. Result: 315,320 reconstructed reasoning blocks.
Then they looked inside. Pu-pu-pu.
People Had Published Their Secrets
Restricting the analysis to genuine user sessions rather than benchmark data, the researchers recovered 704 distinct privacy artifacts.
Among them:
62 API keys
33 passwords
24 access tokens
30 personal email addresses
plus names, postal addresses, internal URLs and hundreds of other technical identifiers and pieces of PII.
And here’s the number I find most important:
64 of those artifacts existed exclusively inside the hidden reasoning.
They weren’t present anywhere in the visible session. So you could inspect the transcript.
Sanitize the visible output. Conclude that everything sensitive had been removed. Publish the trajectory. And still publish the secret inside an opaque encrypted blob you couldn’t inspect yourself. Lovely.
From 6,708 public agent trajectories, the researchers reconstructed 315,320 reasoning blocks and recovered 704 distinct privacy artifacts.
This Changes How I’d Treat Agent Traces
The immediate practical lesson is fairly simple. If you’re logging agent sessions, publishing trajectories, uploading datasets or committing debugging artifacts containing encrypted reasoning blocks: stop treating those blocks as harmless opaque metadata.
At least under the vulnerable implementations studied here, they’re closer to encrypted copies of everything the model may have considered while working. That can include information never shown in the final answer.
The paper includes a particularly nice example involving a coding agent sanitizing a repository. Its hidden reasoning identifies credentials while deciding what needs removing.
Which creates an almost perfect security joke: the agent removes the secret from the repository, then somebody publishes the agent trajectory, which contains the secret in its hidden reasoning.
Security achieved.
Hidden Reasoning Was Supposed to Help Security
There is another irony here. AI providers have good reasons not to expose raw chain-of-thought. One is safety. Another is protecting proprietary reasoning behavior from distillation. If every frontier model happily printed its internal reasoning, competitors could potentially collect enormous datasets of high-quality reasoning trajectories and train weaker models on them.
So the compromise has increasingly been: the model reasons internally, the user gets the answer or a sanitized summary, and the raw reasoning remains hidden. Except here the reasoning still travels through the client. Encrypted. And apparently decryptable through another model.
So we’ve ended up with a wonderfully modern security architecture: the secret is encrypted, but the system that understands the secret can also be persuaded to explain it.
The Bigger Problem Is Agent Infrastructure
I think this matters beyond one clever jailbreak. Agents create vastly more intermediate state than ordinary chatbots. They inspect files. Read environment variables. Search repositories. Open emails. Use credentials. Call tools. Examine databases. Reason about sensitive information. And increasingly we log all of this because traces are incredibly useful for debugging, evals and improving agent behavior.
That means agent trajectories themselves are becoming high-value security objects. The visible conversation is only part of them. Tool calls matter. Tool outputs matter. Memory matters. Environment state matters. And apparently opaque reasoning blobs matter too.
The more autonomous our agents become, the more dangerous the assumption becomes that: “I can’t read this field, therefore there’s nothing sensitive in it.”
Check What You’re Publishing
So if your company works with agent traces, I’d probably do one thing today: search your repositories, datasets and logs for persisted encrypted reasoning blocks from model APIs. Especially anything public. Don’t assume that because the content is encrypted and unreadable to you, it is safe to publish.
The researchers already found thousands of examples in public GitHub and Hugging Face data. And 704 actual privacy artifacts were enough to demonstrate that this isn’t merely a theoretical cryptographic curiosity.
The good news is that this class of vulnerability looks patchable. Reasoning blocks can be cryptographically bound to users, sessions and model identities rather than remaining portable. The more interesting question is how quickly every provider closes every version of this attack, and whether encrypted reasoning blocks eventually disappear from client-visible traces entirely. Because right now we’ve discovered an uncomfortable new rule for agent security: If your AI thought about a secret, assume the trace may contain it.
Even when you can’t see it.