Table of Contents
Researcher
Source: X (@balajis)
As the debate around agents and the x402 protocol heats up, let us turn the clock back to August 2025 for a moment. ****Balaji Srinivasan(@balajis), who has long argued for web-based micropayments as a precursor to x402, defined AI agents as an "extension of a human" in a post. In particular, he emphasized that because AI agents operate as an extension of the person, their web requests should be treated as user-directed actions, and therefore should be distinguished from autonomous bot scraping.
This post did not appear out of nowhere. It was triggered by a dispute that had erupted the day before between two heavyweights: Cloudflare(@Cloudflare), a giant in internet infrastructure, and Perplexity(@perplexity_ai), a rising force in AI search.
[25.08] The Cloudflare-Perplexity dispute
Source: Cloudflare Blog
In August 2025, Cloudflare issued a strong criticism, claiming that Perplexity was using stealth, undeclared crawlers to evade robots.txt directives.
Cloudflare is a company that protects and provides infrastructure for roughly 15~20% of the world’s websites, and protecting publishers’ content and traffic is central to its business. Perplexity, by contrast, offers a search experience where, instead of returning a list of links, the AI collects and analyzes information from the web in real time and summarizes a direct answer.
The problem is what this implies for publishers: during this process, their content can be scraped without authorization, and users can lose any practical reason to visit the original sites in the first place. From Cloudflare’s standpoint, it is not something it can ignore, because it results in customer content and traffic leaking out of the ecosystem Cloudflare is supposed to protect.
Source: X (@perplexity_ai)
Perplexity quickly pushed back. It framed its service not as a simple crawler but as a "user-driven agent," arguing that it only fetches content when a user asks a specific question, and that it does not store the content or use it for training. It also argued that Cloudflare’s system is fundamentally incapable of distinguishing legitimate AI assistants from real threats, and claimed the blog post was little more than marketing for Cloudflare’s blocking products.
This clash exposed an uncomfortable truth about the web:The traditional web has relied on an implicit value exchange: users visit pages, publishers get ad impressions and traffic, and that traffic turns into revenue. Agents, however, can extract information without time-on-page or ad consumption. From a publisher’s perspective, this looks like blatant free riding.
At that point, it became clear that new rules were needed to restore the broken exchange, namely immediate payment in response to requests.
[25.09] Cloudflare and Coinbase establish the x402 Foundation
Source: Cloudflare Blog
Just a month and a half after the dispute, in September 2025, Cloudflare officially announced the launch of the x402 Foundation together with Coinbase. In reality, even before x402, Cloudflare had already been sending more than 1 billion HTTP 402 "Payment Required" responses every day to bots and crawlers, containing a simple message: payment required. But because there was no standardized specification, these responses were largely ignored.
Cloudflare called out this gap directly. With the formation of the x402 Foundation, it also declared its technical commitment to supporting the protocol.
[25.09] Cloudflare Announces 'Pay-per-Crawl' Model
Source: Cloudflare Blog
Along with the establishment of the foundation, the core of the announcement is the 'Pay-per-Crawl' model. This model controls AI crawling through the following process:
Price setting: The site owner sets a crawling price by content area.
Access request: When an AI agent (crawler) attempts access, the server returns 402 Payment Required along with a price list.
Payment and access: When the agent retries the request with proof of payment (such as a token) in the header, the content is finally served.
With this, Cloudflare signaled a direction beyond pure defense. It positioned itself as a platform that intermediates between publishers, who gain revenue, and agents, who gain legitimate access to data through payment.
Of course, we cannot know exactly when Cloudflare began preparing to establish the foundation. Still, from the perspective of an infrastructure company, this move was a predictable step. In the coming agent era, it is obvious that dozens of agents per person will traverse networks and generate enormous traffic load. Building access controls (throttling) and pricing models for these agents is not optional. The conflict with Perplexity effectively reinforced Cloudflare’s existing roadmap, and served as the catalyst that accelerated execution.
[Loading..] An x402-based Agent Web is coming
Given Cloudflare’s direction, we can imagine a new form of the web: an "Agent Web" built on x402:
Web for humans (Human Layer): rich UI and UX, advertising, and visually driven information.
Web for agents (Agent Layer): lightweight, machine-readable data such as JSON, and micropayments based on x402.
As seen in recent moves such as the Solana Foundation launching an LLM-specific interface, websites will likely bifurcate. When a human visits a site, they will be served the human-oriented web, and value will still be settled through the familiar advertising model. Meanwhile, when an agent collects information through the agent-oriented web, each act of access will be priced. The agent will open its wallet, process costs in real time through micropayments, and consume data through paid requests.
No matter what form the agent economy ultimately takes, the direction of travel is clear. Every resource on the web will increasingly demand compensation from agents, and as the language for exchanging that compensation, an infrastructure player’s choice of x402 will look less like an option and more like an inevitability.
