Cloudflare Introduces AI Training Opt-Out Without Search Penalty — Cloudflare, Inc.

Event summary

Cloudflare launched 'Disallow AI Training', allowing websites to refuse AI training while remaining in search results.
Apple, Google, and Microsoft have been designated as 'Accountable' for meeting Cloudflare's AI crawling criteria.
Mixed-use crawlers (combining search and AI training) now make up 36.6% of verified crawler traffic on Cloudflare's network.
Cloudflare is replacing its 'Block AI Bots' switch with three independent controls for search, AI training, and AI agents.
By early 2027, Cloudflare aims to let site owners control how much of their content is included in AI summaries.

The big picture

Cloudflare's move addresses a growing tension between website owners and AI crawlers, where the traditional tradeoff of search visibility for AI training is no longer sustainable. With mixed-use crawlers dominating traffic, Cloudflare's new controls aim to restore balance, potentially setting a precedent for the broader AI and search ecosystem. The shift underscores the increasing need for transparency and control in AI content usage, as well as the strategic importance of governance in the evolving digital landscape.

What we're watching

Adoption Pace: How quickly websites will adopt Cloudflare's new AI training opt-out feature, given the tradeoff between AI training restrictions and search visibility.
Industry Standards: Whether Cloudflare's 'Accountable' designation and AI crawling criteria will become an industry standard, influencing other crawler operators.
Regulatory Impact: The potential for regulatory scrutiny or intervention as Cloudflare's policies reshape the dynamics between website owners and AI crawlers.