NEW YORK – Neil Vogel, CEO of People, Inc., a major U.S. digital and print publisher, publicly accused Google of being a "bad actor" this week, asserting the tech giant uses a unified web crawler to both index content for its search engine and train its artificial intelligence (AI) products without compensation. The accusation was made during the Fortune Brainstorm Tech conference.
Vogel, whose company operates over 40 brands including People, Food & Wine, and Better Homes & Gardens, stated that Google's single crawler means "they use the same crawler for their search, where they still send us traffic, as they do for their AI products, where they steal our content." He highlighted a significant decline in traffic from Google Search to People, Inc.'s properties, noting a drop from approximately 65% three years ago to the "high 20s" currently. Reports from AdExchanger previously indicated Google traffic accounted for as much as 90% of the publisher's open web traffic several years prior.
To gain leverage in the evolving AI landscape, People, Inc. has implemented web infrastructure company Cloudflare's solution to block AI crawlers that do not provide compensation for content usage. Vogel reported that this strategy has led to "large LLM providers" initiating discussions for potential content deals, though no agreements have been finalized. The company maintains an existing content deal with OpenAI, which Vogel described as a "good actor."
However, Vogel noted that Google's crawler presents a unique challenge, as blocking it would simultaneously prevent People, Inc.'s websites from being indexed in Google Search, thereby eliminating the remaining "20%-ish" of traffic the search engine still delivers. "They know this, and they're not splitting their crawler. So they are an intentional bad actor here," Vogel declared.
Janice Min, CEO of Ankler Media, echoed these sentiments, labeling major technology companies like Google and Meta as "content kleptomaniacs" and stating her company also blocks AI crawlers. Matthew Prince, CEO of Cloudflare, who was part of the same panel discussion, expressed a belief that the behavior of AI companies would change, potentially driven by new regulations. Prince also predicted that "by this time next year, Google will be paying content creators for crawling their content and taking it and putting it in AI models," citing internal disagreements within Google regarding its strategy.
