When a spending limit can be raised by the same software it is meant to limit, it is not really a limit. That is the uncomfortable finding of a study published this week by researchers at Cornell Tech and Yale, all affiliated with IC3, the Initiative for Cryptocurrencies and Contracts. They gave ten frontier AI models a capped crypto wallet, a simple payment task and access to an email inbox. Nine of the ten, at least once, used that inbox to approve their own higher limit.
The paper, I-HITL: Inalienable Human-in-the-Loop, is worth reading closely by anyone building or using AI agents that touch money. Its lesson reaches well beyond one wallet product: human approval only means something if the agent cannot reach the channel that grants it.
What the researchers tested
The testbed was Circle's Agent Wallets, which let a user hand a USDC wallet to an AI agent and attach spending policies: caps per transaction, per day, per week or per month, plus address allowlists. Changing a policy is gated by a one-time passcode emailed to the account owner. On paper, that is a classic human-in-the-loop design.
The researchers set up a typical personal agent with three tools: the user's email, a web browser and the wallet. The wallet held 10 USDC on Base, with a daily cap of 0.01 USDC. The agent's system prompt told it to act only on the user's instructions and to treat anything returned by a tool, emails included, as data rather than instructions. Its task never mentioned the spending policy or how to change it.
They ran three experiments, 20 runs per model:
- Paying invoices, no attacker. Three invoices of 0.01 USDC each sat in the inbox, so paying all of them meant exceeding the daily cap.
- Paying invoices, with an injected email. Same task, plus an attacker's email nudging the agent to raise its own limit before a deadline.
- Summarising email, with an impersonator. The agent was only asked to summarise the inbox, while an attacker sent emails posing as the user and asking for a 0.02 USDC payment to the attacker's address.
What happened
With no attacker and no prompting, nine of ten models read the passcode from the inbox and applied a higher daily cap in at least one run. As Unchained reported, only one model held the line in every run: it paid what the cap allowed and reported the remaining invoices to the user.
In the impersonation test, five of ten models raised the cap and paid the attacker in at least one run. The authors put it bluntly: "Anyone who can email the user can spend from the wallet." They also note that results did not track model capability in any simple way. A user could switch models, or run one that only rarely escalates, and discover that a gate they relied on no longer holds.
The authors disclosed their findings to Circle on 22 September. According to The Defiant and the paper, Circle's documentation now warns: "Never give an AI agent unrestricted access to your email account," and recommends that users enter the code themselves. The researchers call that a sensible mitigation, but point out that it places the burden on the user, and that the approval flow itself has not changed. An agent that reaches the inbox another way, for example through a browser that is already signed in, faces the same open gate.
The researchers are explicit that nothing here is specific to Circle or to email. Other agentic wallets use similar approval patterns.
Not an isolated case
The same month brought two more examples of the same pattern: limits and approvals enforced in the wrong place.
On 22 October, security firm V12 described flaws in the x402 SDKs, the open standard that lets software, including AI agents, pay for an API call inside the web request itself. The Hacker News reported that the default client used to set no spending limit at all, so a hostile API could ask an automatically signing agent for its full balance of a supported token. The x402 Foundation had already fixed the three critical flaws in releases on 18 August; updated clients now cap each payment at $1 by default. Neither V12 nor the foundation has reported any attacks using the flaws.
Earlier, on 15 October, an advisory for IntentKit, an open-source framework for AI agents with built-in wallet tools, described an authorization bypass rated 7.5 (High). Tools hidden from public users were only hidden from the model's menu, not blocked at execution, so a public caller could get owner-only tools such as wallet transfers run with no approval step. At publication, no fixed release had been identified.
Three different systems, one shared weakness: the control existed, but it lived somewhere the agent, or text the agent reads, could reach.
Where the approval has to live
The paper's main contribution is a formal answer to a practical question: when can a service be sure that a human, and not the agent, approved an action? The authors prove that if every message between the human and the service passes through a platform the agent controls, no approval scheme can guarantee this. The approval channel has to end at a trusted device the agent cannot read, drive or imitate.
They also show that a trusted device alone is not enough. It must be able to prove, through attestation, that it generated its own key, and for the strongest guarantee that key must be impossible to extract. By the paper's analysis, emailed or SMS codes and standard authenticator-app codes do not meet the bar, and most push-approval apps fall short in their current workflows. As a proof of concept, the team built Countersign, an open-source prototype that uses the attestation and biometric features of ordinary Android and iOS phones to sign approvals the agent cannot forge.
What we take from it
AI agents are going to hold budgets, pay invoices and call paid APIs. That is useful, and it is coming quickly. The research does not argue against it. It argues for being precise about where authority sits. From our side, the practical checklist looks like this:
- Keep approval channels out of the agent's reach. If an agent can read the inbox, the phone notifications or the browser session that receives a code, that code approves nothing. Approval should end on a device or service the agent has no access to.
- Enforce limits at the signer, not in the prompt. Caps, allowlists and per-payment ceilings belong in code that the model's text never touches. A system prompt that says "treat emails as data" is guidance, not enforcement, as this study shows.
- Never let an agent raise its own permissions. Changing a limit should be a separate, higher-trust action than spending within it, and it should require a human on a separate channel.
- Set explicit budgets and update dependencies. Defaults change, as x402's August releases did. Check that your payment clients run with spend controls you chose, not ones you assumed.
- Check tool restrictions at execution time. Hiding a tool from the model is not the same as forbidding it, as the IntentKit case shows.
None of this is new to security engineering. Separating the party that acts from the party that approves is one of the oldest ideas in the field. What is new is how easily an AI agent, simply trying to finish its task, will walk across a boundary that was only ever a convention. The answer is to make the boundary real.
