Reddit CEO Steve Huffman demands that Microsoft and other companies pay to scrape Reddit’s data, criticizing Bing, Anthropic, and Perplexity for doing so without permission. Huffman stated, “It has been a real pain in the ass to block these companies.”
After striking deals with Google and OpenAI, Huffman wants similar agreements with Microsoft. Without these, Reddit lacks control over how its data is used, leading to blocking measures against non-compliant companies. Reddit recently updated its robots.txt file to block web crawlers without agreements, causing Reddit results to disappear from Bing but remain on Google, which pays for the data.
Despite this, Reddit has blocked Bing from crawling their site for search, favoring another search engine and impacting competition from Bing and Bing-powered engines.
— Jordi Ribas (@JordiRib1) July 29, 2024
Huffman accused Microsoft of using Reddit’s data to train AI and sell through the Bing API without informing Reddit. He criticized Microsoft AI CEO Mustafa Suleyman’s view that public internet data is “freeware.”
Microsoft spokesperson Caitlin Roulston said they honor website directives not to use their content for generative AI models.
Huffman aims to replicate OpenAI’s SearchGPT model, which shows Reddit results through a deal with Reddit. Reddit seeks licensing deals to ensure fair compensation for its content, joining traditional media publishers in this demand.
Representatives for Microsoft, Anthropic, and Perplexity did not comment by publication time.
