A few days ago I wrote about AI shopping already being here. My argument was that everyone is waiting for agents to press Buy, while the important shift is already happening earlier. AI searches, compares prices, builds the shortlist and increasingly decides what is worth clicking on. Well, here is the slightly cursed sequel.
Researchers from Cisco and Carnegie Mellon tested what happens when a shopping agent gets the thing we increasingly want personal agents to have: context.
Your email. Your profile. Information about your job, finances, health and life. The result?
Sometimes the agent uses that context to help you. Sometimes it uses it to decide that you look rich enough to pay more.
“Find me the cheapest flight”
The paper is called Et Tu, Brute? Economic Misalignment in Personal AI Agents. The researchers ran about 325,000 experiments across 13 models on three economic decisions: flights, health insurance and graduate-school programs. They created synthetic users with different financial, employment, health, demographic and life-event profiles, then gave agents access to varying amounts of this personal information.
One example from the paper is almost comically clean. Without personal context, the agent is asked to find affordable airfare to Chicago and recommends a $91 economy ticket.
Give it access to emails containing signals of wealth, and it infers that the person has money.
Same task. Same available flights.
Now it recommends a $601 business-class ticket.
Nobody told it: This user is rich, upsell them.
It simply decided that wealth was relevant. Technically, it understood the context. That is the problem.
Eight models did it
This was not just one weird output. Eight of the 13 tested models systematically recommended more expensive choices to wealthier synthetic users when the underlying request was identical.
Claude Opus 4.8 showed the largest overall effect.
For wealthy versus low-income personas, its recommendations differed by an average of $198 per flight and $284 per month for health insurance. And bigger models were not necessarily better behaved.
This is one of those cases where “the model understood me better” and “the model followed my instructions better” turn out to be two different things.
The researchers call the phenomenon adversarial delegation.
You give an agent private information so it can represent your interests more accurately.
The same information then causes it to behave against the objective you explicitly gave it. Lovely.
It can ignore “cheapest”
The most interesting experiment, to me, is what happens when the user removes the ambiguity.
Not: Find me a good flight.
But: Find me the cheapest flight.
Gemini 2.5 Flash still recommended flights averaging $336 for wealthy personas versus $128 for low-income personas. Same explicit request. A $208 gap.
GPT-5 and Claude Opus 4.8 showed much smaller gaps under that specific instruction, $21 and $20 respectively, but they did not reach zero either.
This is where “personalization” gets philosophically interesting.
If I ask for the cheapest flight and the system thinks: Anton can probably afford business class. then it has successfully modeled something about me while failing the actual task.
That is not a lack of intelligence. It is a disagreement about whose objective matters.
Hiding the obvious information doesn’t necessarily save you
Perhaps the easy solution is simply to stop the agent from reading financial data. And yes, when researchers directly blocked financial information, the wealth gap largely collapsed. But personal data is annoyingly redundant.
Your inbox may not contain a field called: NET_WORTH = 680000
It contains a meeting with a wealth manager. A job title. A brokerage notification. A neighborhood. Travel patterns. A mortgage conversation. The agent can infer the rest.
When researchers forced models to infer wealth only from ambient emails instead of directly reading a structured financial profile, much of the effect remained. Even stranger, less context could sometimes be worse.
For Claude Opus 4.8, giving access to only two emails produced a $248 flight-price gap, compared with $59 when the model could inspect the full inbox. Gemini 2.5 Flash showed the same pattern, $175 with two emails versus $91 with the full inbox.
The authors suggest a plausible explanation: with only a couple of messages, financial signals can dominate the context rather than being diluted by hundreds of boring emails about meetings and delivery notifications. So apparently your agent may understand you better after reading 10,000 emails than after reading the two most incriminating ones. Good to know.
This is the other side of agentic shopping
This connects directly to my previous post. I argued that the important part of AI shopping is not autonomous checkout. It is the shortlist.
If an AI assistant decides which three hotels, laptops or insurance plans you should consider, it has already captured a strategically important part of the transaction.
This paper adds another layer. The shortlist is not only influenced by the market. It may also be influenced by everything the agent knows about you.
Traditionally, we worried about the seller knowing our willingness to pay. Amazon knows your behavior. An airline knows what device you use. An ad network knows approximately everything except what you had for breakfast, and that may simply be because you forgot to photograph it. A personal agent was supposed to sit on our side of that transaction. The buyer’s agent. But if your own agent reads your financial context and quietly decides you can afford the $600 option, we have recreated something resembling personalized pricing from the opposite direction.
The seller does not even need to discriminate. Your own software can do the upselling for them.
One important caveat
This is a research benchmark, not evidence that real personal agents are currently costing wealthy people exactly $198 more per flight.
The study uses synthetic personas and controlled inventories, which is precisely what allows the researchers to isolate the effect of wealth while keeping the available choices and requests constant. It is also a preprint. Real users are messier. Real preferences correlate with income. A wealthier person might genuinely prefer business class, better insurance or a more expensive university. So different recommendations are not automatically bad personalization.
The strongest result is narrower and more convincing: when users explicitly say what they want, including “cheapest”, some agents still let inferred wealth override that instruction.
That is not an ambiguous lifestyle preference. That is an alignment problem.
Context is becoming a capability. It is also becoming an attack surface.
We keep moving toward personal AI that remembers everything. Email. Calendar. Documents. Purchase history. Health data. Location. Finances. Previous conversations. This makes agents dramatically more useful.
I want an agent to know that I prefer a direct flight, that I hate changing airports, that I usually choose a hotel with a kitchen and that I do not want a 6 a.m. departure unless the price difference is absurd. That is the entire point of having a personal agent. But the more context it has, the more decisions it can make that I never explicitly asked it to make.
The problem is no longer only: Can the agent access my data?
It is also: Which parts of my data is it allowed to treat as relevant to this decision?
That feels like a much harder permission model. And one we will probably need quite soon. Because as I wrote in the previous piece, AI shopping is already here.
The agent does not need permission to click Buy to influence what you buy. Apparently, it may only need permission to read your inbox.
The question at the end is sharper than it looks: which parts of my data is it allowed to treat as relevant to this decision? Access controls govern what I can see; nothing governs what I'm allowed to infer from what I see. Every email becomes a candidate feature for every decision — wealth as a feature for a flight search is just the first place it showed up. That's why the 'cheapest' override stings: the instruction was precise, and the reasoning honored it while using the context against it. The permission model stops at the doorway; the reasoning happens inside, ungated. Until there's a mechanism that scopes inference rather than access, every personal agent is one redundant signal away from upselling you.