Pieter Levels, the indie developer behind bootstrapped web products including Nomad List and Photo AI, says Meta is crawling his sites at unusual volume this week. He also says a Meta employee privately told him the company is building its own search index, so that AI generated web queries never reach Google. Both claims trace back to Levels alone, and Meta has not confirmed either one.
The scraping claim rests on operational evidence. Levels posted that the crawling was “heavy heavy heavy” across all of his properties, enough to trigger load average alerts on one of his servers. He noted the requests were hitting url2og, a screenshot tool he built for generating link previews, and guessed Meta might be capturing images and video, not just text. That would point toward training data for a multimodal system rather than a text search index specifically.
The search engine claim rests on something thinner: an unnamed Meta employee’s direct message, which Levels says he was given permission to share. In it, the employee described a project to build a Meta run web index so its AI’s search queries stop flowing through Google, where Google could otherwise see and reuse that activity. Levels flagged the claim as “allegedly” true in his own post. That hedge matters. The account is secondhand, unverifiable, and impossible to check against Meta’s internal roadmap.
AI Insiders has not independently verified the scraping pattern or the DM, and no other outlet has either. The employee’s identity is undisclosed, so the tip cannot be checked against a title, a team, or a track record. Meta did not respond to Levels’ posts and has made no public statement about a search product.
Heavy crawling has several explanations that do not require a new search engine. Large AI labs routinely scrape the open web to refresh pretraining and retrieval datasets, which alone can produce the kind of traffic spike Levels describes. Meta’s platforms also fetch Open Graph images and metadata by the billions every day simply to generate link previews inside Facebook, Instagram, and WhatsApp. A screenshot service like url2og sits directly in the path of that unrelated, mundane process. Neither explanation requires Meta to be building a search index at all.
The claim is not implausible on its own terms, though. The Information reported in 2024 that Meta was already crawling the web to reduce its dependence on Google and Microsoft’s Bing for grounding Meta AI’s answers about current events. An expanded index would extend a known project rather than start one from nothing, which makes the DM more credible than an average anonymous tip. It does not make the DM confirmed.
What the episode really demonstrates is a fact about the web, not a fact about Meta’s roadmap. Operators of high traffic sites have effectively become a sensor network for how large AI companies crawl, because their server alerts fire in real time, long before any company publishes a roadmap or a research paper. That is genuinely useful information. It is also only as reliable as one operator’s read of one week of his own logs.
Site owners curious whether they are seeing the same pattern should check their own logs for an unusual jump in Meta linked crawler traffic this week. Do not take Levels’ account on faith alone. Editors covering Meta should treat the search engine claim as a lead worth chasing through the company’s press office, not as a launch worth reporting as settled fact.
Pieter Levels (@levelsio), posted on X.
