From October 30, businesses using Facebook or Instagram authorize Meta to read their website and use it for AI: what to do
From October 30, 2026, anyone who uses Facebook, Instagram, or WhatsApp for business authorizes Meta to crawl their website, including with “test credentials,” and to use the content to train its AI. With no way to decline. We read the terms, the page listing Meta’s five crawlers, and 120 days of visits by Meta’s bots on a real site: what changes and what to do before the date.
In brief
- From October 30, 2026, anyone who uses Facebook, Instagram, or WhatsApp for business, with a Page, ads, or Business Suite, authorizes Meta to crawl and copy the websites “associated with” their products, including by creating “test credentials,” and to use the content to “train” its products. There is no way to decline: whoever does not accept has to stop using those tools.
- Meta has five crawlers. Two of them may ignore robots.txt. On the online motoring daily we run, over 120 days, Meta’s bots made 233,890 visits, almost 2,000 a day: the crawler behind Meta AI’s search is the fifth-busiest bot overall, ahead of each of OpenAI’s. Not a single human visit came from Meta AI.
- Before October 30 a brand can decide what to let Meta read: block its crawlers one by one in robots.txt, knowing that for two of them this is not enough, and close the business accounts it no longer needs.
Meta is updating the commercial terms for Facebook, Instagram, and WhatsApp. Almost nobody noticed: the notice appears only in Business Suite, and the clause that matters sits in the middle of a document many pages long. It was brought to light on October 15 by an investigation from Nieman Lab, Harvard’s journalism lab. The clause concerns every company with a Page or an ad account, not just publishers. We compared the new terms with the current ones and counted the visits of Meta’s crawlers on a real website over four months.
Index
What changes on October 30
Meta’s commercial terms apply to anyone who uses its products “for business or commercial purposes”: ads and marketing messages, selling products, catalogs, managing a Page or a business group, Ads Manager, Business Suite, measurement services. In practice, any company with a presence on Facebook or Instagram. From October 30, 2026 the text contains a new paragraph. Here it is in full, because the meaning lies in the individual words:
“You authorize us (or any entity acting on our behalf) to use automated software (such as web scrapers, bots, robots, spiders, crawlers, and user agents) and create test credentials to access, index, cache, analyze, collect, retrieve, or crawl the domains, web pages, or URLs (and any content available through them) that are associated with your products or services […] Meta may use any content from such domains, web pages, or URLs to provide, develop, improve, research, train, and maintain the integrity of, the Meta Company Products.”
In the terms in force until October 29 this paragraph does not exist. In the version that takes effect on the 30th it does, right after the section on licenses. And there is no box for saying no: Meta’s notice says that whoever does not agree has to stop all business use of its products. Keeping the Page means accepting.
The two words that weigh: “train” and “test credentials”
The first word is “train.” A company’s website, with its product pages, prices, guides, and articles, can be used to train Meta’s products. And Meta AI, as Nieman Lab points out, is one of those products under Meta’s own terms of service. Until now the license covered what you publish inside Facebook or Instagram; from the end of October it extends to what sits on the company’s site, outside the platforms.
The second is “test credentials.” Meta reserves the right to create itself an account in order to read the parts of a site reserved for registered users: a customer area, a price list behind a sign-up form, a subscription. Nieman Lab asked Meta whether this allows it to get past paywalls and received no answer. A spokesperson said the text consolidates clauses already scattered across other terms, that the way Meta crawls sites does not change, and that “content owners can still manage how their content is accessed by using robots.txt.”
The three legal scholars consulted by the same investigation, Matt Topic of Loevy & Loevy, James Grimmelmann of Cornell, and Benjamin Sobel of the University of Wisconsin, read the text differently: as written, the contractual permission could be invoked over the instructions in robots.txt, and a court might consider it enforceable. Sobel compares it to what Google does with search: whoever wants to appear in traditional results accepts appearing in AI answers too. With one difference: Google reads a site in order to show it; Meta asks to read the site as a condition for using another service, the Page or the ads.
Meta’s five crawlers, and the two that ignore robots.txt
Meta publishes a page for site owners listing its crawlers, updated in May 2026. There are five crawlers with five different purposes, and each can be blocked individually in robots.txt, with a delay of up to 24 hours:
- FacebookExternalHit reads the title, description, and image of a link shared on Facebook, Instagram, or Messenger. It may ignore robots.txt “when performing security or integrity checks.”
- Meta-WebIndexer crawls the web for Meta AI’s search: allowing it, Meta writes, “helps us cite your content and link to it in Meta AI responses.”
- Meta-ExternalAds collects content for advertising and business products.
- Meta-ExternalAgent serves “training AI foundation models” and the direct indexing of content.
- Meta-ExternalFetcher opens a single link when an AI user asks for it, for example to complete a task on a site. This one too “may bypass robots.txt rules.”
Meta says it respects robots.txt and does not recognize alternative formats such as “NoAI” tags. Keep in mind that these are two different documents: the crawler page says how Meta behaves today, the commercial terms say what Meta has the right to do. From October 30 the right is broader than the declared behavior.
How much it already reads: 120 days of data
On quotidianomotori.com, the online motoring daily we run, we log every visit by the bots of AI companies. Between June 18 and October 16, 2026, 120 days, search and AI bots made 3,982,906 visits. Meta’s accounted for 233,890, or 5.9%: the sixth vendor after Microsoft, Google, Apple, Amazon, and OpenAI, ahead of Anthropic.
Almost all of it is Meta-WebIndexer, the crawler for Meta AI’s search: 231,945 visits, just under 2,000 a day, the fifth-busiest bot overall behind Bingbot, Googlebot, Applebot, and Amazon’s bot. Meta-ExternalAgent, the training crawler, made 1,734; Meta-ExternalFetcher 211. By comparison, OpenAI’s OAI-SearchBot stopped at 117,907 and GPTBot at 2,087.
Then there is the other side: how many people reach the site from Meta AI. Over the 120 days, human visits arriving from AI assistants came to 1,337, of which 984 from ChatGPT. From Meta AI, zero. Meta reads the site almost 2,000 times a day and sends nobody back. Meanwhile Facebook is the source Google’s AI Overviews use most on motoring questions in Italian, present in 42% of answers. In other words: what companies publish inside Facebook gets cited by Google’s AI, while Meta’s AI reads companies’ websites and sends nobody back.
What a brand can do before October 30
The first thing is to decide what to let it read. The five crawlers have different names precisely for this: you can block Meta-ExternalAgent, the training one, and keep Meta-WebIndexer, if you want to appear in Meta AI’s answers once they start bringing visits. Or the other way around. What matters is that it is a choice, not an omission.
The second is to know where robots.txt does not reach. For FacebookExternalHit and Meta-ExternalFetcher, Meta says it may ignore it, and from October 30 the contract gives Meta a permission that a text file does not revoke. Members-only areas and paid content need real access control, not an instruction to bots: more than half of web traffic is already non-human, and the bots that show up under their own name are only part of it.
The third is to count your business accounts. The clause applies to the sites “associated with” the products or services of whoever uses Business Suite, ads, or a Page. A company with a Page abandoned years ago, opened for an event or a campaign long over, now has one more reason to close it. Whoever sells through Instagram, on the other hand, knows that from the end of October the price of that presence includes their website too.
What you cannot do is wait for things to clear up on their own. Meta declined to answer questions about the scope of the clause, and the legal scholars agree on one point: until a court says otherwise, a company should assume it is enforceable. The date is October 30.
Frequently asked questions
Yes. Meta’s commercial terms are the same in every country where its products are used for business purposes, and the clause on scrapers and test credentials appears in every language version, including the Italian one. Only the dispute rules differ: companies based in the United States are bound to arbitration.
Yes, the two are separate: the Page stays active and robots.txt can exclude Meta’s crawlers one by one, under names such as Meta-ExternalAgent or Meta-WebIndexer. Meta says it respects robots.txt, with two exceptions: FacebookExternalHit for security checks and Meta-ExternalFetcher for user requests. From October 30, though, the commercial terms contain a contractual permission that, according to the legal scholars consulted by Nieman Lab, could prevail over robots.txt.
On our data, no. Over 120 days the crawler behind Meta AI’s search visited the motoring site we run 231,945 times, almost 2,000 a day, and human visits arriving from Meta AI were zero, against 984 from ChatGPT. For now Meta reads without sending anyone back.
