Meta’s Oct. 30 Business Terms Let It Crawl Your Website. Read the Clause

From Oct. 30, any business that keeps using Facebook, Instagram or Meta’s business tools agrees to one long sentence. Meta’s updated Commercial Terms say: “You authorize us (or any entity acting on our behalf) to use automated software (such as web scrapers, bots, robots, spiders, crawlers, and user agents) and create test credentials to access, index, cache, analyze, collect, retrieve, or crawl” the websites “associated with your products or services.” The next sentence says Meta “may use any content from such domains, web pages, or URLs to provide, develop, improve, research, train” its products. Nieman Lab first reported the change on Oct. 15. There’s no opt-out: per the terms, “If you do not agree to the updated terms, you may stop all access or use of our Products for business or commercial purposes.”

That is a bigger ask than it sounds. The terms define business use to include “managing a Page,” “using ads or marketing messages” and “using any business product, such as Ads Manager or Meta Business Suite.” “Virtually every news organization has at least one Meta business account,” Nieman Lab noted. So we read the new text against the version it replaces, line by line.

Meta Commercial Terms before and after: March 2024 vs. Oct. 30

The Commercial Terms in force today carry an effective date of March 6, 2024. They contain no language about scrapers, crawlers or test credentials. Their licence covers content “you share, post, or upload on or in connection with our Meta Products,” and the permission you warrant you hold is permission “to display, distribute and deliver your content.”

The Oct. 30 version keeps that and widens it in three places, all in the section titled “Licenses & Rights You Provide Meta”:

• A crawling grant. The whole automated-software sentence quoted above is new to the Commercial Terms.
• New uses. Meta may now use your content “to provide, develop, improve, research, and maintain Meta Company Products,” and site content can be used to “train.” The 2024 licence has no research or training language.
• Derivative works. The warranty now covers “permission to display, distribute, create derivative works, and deliver your Content.” “Create derivative works” is the addition, and in copyright terms it is the phrase that covers making new things out of yours.

A Meta public affairs spokesperson, who wasn’t named, told Nieman Lab by email that the crawling language used to be spread across other documents, including its generative AI ad terms, and that the update pulls it into one place without changing how Meta crawls publishers. That may hold for those other documents. The Commercial Terms, which every business account accepts, said none of this before Oct. 30.

Meta’s commercial terms, phrase by phrase

“And create test credentials.” Cornell Law School professor James Grimmelmann told Nieman Lab this could let Meta “sign up as a fake user” to get past registration walls. Meta declined to tell the outlet whether test credentials would be used on paywalled content.

“Access, index, cache, analyze, collect, retrieve, or crawl.” Seven verbs. “Cache” and “collect” mean keeping copies, and the clause sets no time limit.

“(And any content available through them).” The parenthetical reaches past the pages themselves to whatever they serve: images, video, feeds, files.

“Or that are made available through your or users’ use of Meta Company Products.” This is the broadest phrase in the clause and the least discussed. It isn’t only sites you link. Our read: a URL becomes eligible once users share it on Facebook or Instagram, though whether that reaches a site whose owner never accepted the terms is a question no court has answered.

“Provide, develop, improve, research, train, and maintain the integrity of.” Six uses, applying to “the Meta Company Products, services, and features provided under these Commercial Terms and any other applicable supplemental terms.” That last phrase leaves the list open.

Two smaller additions are worth a look. A new “AI Features” section says “you must not alter or remove any watermark or content-authenticating metadata generated by AI in connection with Content.” And the US arbitration clause “does not cover any commercial claims relating to violations of your or our intellectual property rights, including, but not limited to, copyright infringement,” so a publisher could still take a copyright fight to court.

“For news publishers, this clearly represents permission to Meta to train AI on any content from their sites.” James Grimmelmann, Cornell Law School, to Nieman Lab

Robots.txt vs. a signed contract

The spokesperson told Nieman Lab that “content owners can still manage how their content is accessed by using robots.txt.” But robots.txt is a request posted on your server. The terms say that “by continuing any access or use of any Meta Company Products for business or commercial purposes after any notice of an update to these Commercial Terms, you agree to be bound by them.” The lawyers Nieman Lab spoke to worry Meta could argue that explicit authorization outranks a “Disallow” line.

“This is a remarkable provision to bury in [the] terms of service,” media and intellectual property attorney Matt Topic of Loevy & Loevy told the outlet. “It’s not clear whether a court would enforce this, but publishers should assume that it would be enforceable.” University of Wisconsin law professor Benjamin Sobel called the arrangement “tying,” an antitrust concept: to use the Meta products that require the terms, “you have to also acquiesce to the scraping and crawling permissions exacted in those terms.”

Courts haven’t been kind to publishers lately. A federal judge found Google uses publishers’ work and dismissed their AI Overviews case anyway, and the first AI-training appeal turned on a narrow footnote. In May, five book publishers sued Meta alleging it pirated books to train its Llama models, the Guardian reported. A signed licence makes a fair-use argument unnecessary.

What publishers and creators can do before Oct. 30

None of this cancels the contract. It builds a record and makes crawling you harder.

• Block the right Meta bots by name. Meta’s crawler documentation, updated May 21, 2026, lists Meta-ExternalAgent, which “crawls the web for use cases such as training foundation AI models”; Meta-ExternalAds, for “improving advertising and other business-related products and services”; and Meta-WebIndexer, which works “to improve Meta AI search result quality.” Each takes a “User-agent:” line followed by “Disallow: /” in robots.txt. Meta says to “allow up to 24 hours for changes to robots.txt to take effect.”
• Know which bots ignore robots.txt. By Meta’s own account, FacebookExternalHit, which builds link previews, “might bypass robots.txt when performing security or integrity checks,” and Meta-ExternalFetcher, which fetches links at a user’s request, “may bypass robots.txt rules.” Blocking FacebookExternalHit also breaks preview cards when readers share your stories.
• Enforce at the edge. A firewall or CDN rule refusing Meta’s user agents actually stops them. Our read: server logs showing you blocked Meta are stronger evidence than a text file Meta could argue it was licensed to ignore.
• Audit your accounts. Every Page, ad account and catalogue you run counts as business use, and each links back to a domain.

Individuals have separate settings for their own chats and posts, covered in our guide to stopping AI training on your data. More platform fights over content are in our internet culture coverage.

Stephen Beale, the Nieman Lab reader who first spotted the notice in his Business Suite dashboard, told the outlet he is seriously considering closing his Facebook and Instagram accounts before the deadline. Under the March 2024 terms he would have had no reason to. Under the new ones, leaving is the only exit on offer, and he has until Oct. 30 to use it.