My Reddit AMA on the Law of Web Scraping

On Friday, I did a Reddit AMA with the r/WebScrapingInsider subreddit about the law of web scraping. You can see the entire discussion here.

I posted an introduction to the AMA, which I am sharing here:

__

Hi everyone, and thanks for joining this Reddit AMA on web scraping! I thought it would be helpful to start with some introductory remarks.

First, and most importantly, a reminder about the ground rule. Nothing in this AMA constitutes legal advice. I will discuss some general legal principles, but I can’t opine how those principles apply to your circumstances. Further, many of the topics we’re going to discuss are contested, and everyone has different tolerances for legal risk. Before you make any decisions, you need to talk with a lawyer about how the law applies to your circumstance and what level of legal risk you’re willing to accept.

Also, as Ian mentioned, I am focused on US law. The rules outside the US could look quite different!

Some background about me: I am an OG Internet Law expert. I got my first email account in 1991, published my first Internet Law paper in 1993, started practicing Internet Law in 1994, and started teaching an Internet Law course in 1996—a course I still teach today, 30 years later. I have been a full-time law professor since 2002.

As an Internet Law expert who formed his intellectual identity in the 1990s, I still approach Internet Law as a generalist, which was the norm back then. I don’t focus on any one topic of Internet Law. Instead, I still track and pay attention to dozens of Internet Law topics. The scraping/trespass to chattels doctrines are just a subset of my overall portfolio of interests. You can see my academic writings at https://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=332758 Some of my other scholarly priorities include liability for user-generated content (UGC)/Section 230, online trademark law, and emoji law. My forthcoming book project will be about the Internet’s pro-social benefits and how legal regulation jeopardizes those benefits.

Despite my generalist orientation, I find the law of scraping especially interesting. I got my first client question about scraping probably around 1997. When I was in-house, scraping was an evergreen topic. I routinely test on scraping in my Internet Law course. For example, this year’s midterm was based on Vox Marketing Group v. Prodigy Promos, 556 F. Supp. 3d 1280 (2021).

* * *

Part of why I find the topic so fascinating is that I have complicated feelings about scraping from a policy standpoint.

On the one hand, I believe property owners should generally have the ability to decide how their property is used. Scraping can involve unwanted and unconsented uses of a property owner’s property, in ways that can be pernicious.

On the other hand, I believe in access to information. (My mom was a librarian). To the extent that scraping enables better insights, better knowledge, and stronger markets, it’s a social benefit.

Furthermore, the Internet succeeded in part because of its seamless, low-friction, low-transactions-cost navigation. To keep that architecture, browsing the Internet should be a permission-less experience. Anti-scraping doctrines threaten to convert web browsing into a permissioned experience, where there are checkpoints at every site’s perimeter and permission is needed for access. Those barriers threaten to undermine the Internet’s navigability and utility, in ways that will disadvantage all of us.

[Note: Trespass to chattels doctrines are just one way that services can impose permissioned access into web navigation. Copyright law is another. Yet another is the online age authentication mandates sweeping the globe. For more on my opposition to age authentication mandates, see https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5208739 An obvious point: if age authentication checkpoints become ubiquitous in the Internet, that will have major impacts on scraping because most content will live behind age authentication walls].

I have never fully written up my complicated thoughts about these topics, but this piece will give you a sense of my thinking. https://blog.ericgoldman.org/archives/2013/04/rethinking_onli.htm

Here’s how I frame some policy considerations of scraping for students:

On the plus side: Scraping can enable services that create significant social benefits, such as search engines, price comparison bots, and AI modelbuilding. Also, anti-scraping efforts are frequently anti-competitive, i.e., an intent to manufacture or maintain monopolies. In particular, anti-scraping tactics can represent an attempt to obtain IP-like rights, even if the service doesn’t qualify for IP protection. You can see some of these dynamics in the Register.com v. Verio case, where Register.com was using contracts and trespass to chattels to protect non-copyrightable material (WHOIS registrant contact info) to preserve its first-mover-advantage control over that contact information.

On the minus side, scraping can look like a DOS attack (intentionally or not). It can facilitate massive copying, including possibly copyright infringements. And scraping can be a massive invasion of consumer privacy.

So, from a policy standpoint, is scraping a good or bad thing? ¿Por qué no los dos?: Or, to put it into legalese, “it depends.”

* * *

As for the law of scraping, it’s a mess, especially the trespass to chattels doctrines (I summarize them as “TTC”). For about two decades, I taught the Intel v. Hamidi and Register.com v. Verio cases. Around 2024, I felt like those cases no longer reflected the current legal state, so I regretfully took them out of my Internet Law casebook. [For more on the casebook, see https://blog.ericgoldman.org/archives/2026/07/announcing-the-2026-edition-of-my-internet-law-casebook.htm] I have not yet found a good replacement case to add to the casebook, so my casebook coverage of the topic is now adrift. I now teach the X v. Bright Data ruling (the first one granting the motion to dismiss) and it’s not very satisfying. Because there is no good single case or set of cases to teach the topic, and because the law is so unsettled, I tell my students that the topic has become “unteachable.” You can imagine how much they like hearing that LOL.

I teach students to run through a four-step process to determine if/how trespass to chattels applies:

Step 1: does the plaintiff own the affected chattel? Usually, the chattel is the servers or other physical hardware that is used to run the server and connect it to the Internet. The trespass to chattels doctrines are a good illustration of the interplay between the law of physical space and activities taking place in a virtual environment. Students sometimes struggle to wrap their heads around the distinctions between tangible items (i.e., “chattel”) and intangible things, such as data or online accounts, which may be protected by other legal doctrines (copyright or other IP) but don’t legally qualify as the “chattel.”

Step 2: did the defendant “use” the chattel? Usually, sending or receiving packets from a server qualifies.

Created by ChatGPT Dec. 2025

Step 3: was the defendant’s use “unauthorized”? The law might presume that a server owner impliedly permits ordinary Internet traffic merely by making the choice to connect to the open Internet. How can the server owners remove this implied permission? This is one of the areas that is unteachable. There is no clear answer to how a server owner can structure this dialogue with Internet users. Some of the possible tools that a server owner MIGHT be able to use to remove any implied authorization to use their chattel:

• TOSes
• C&D letters
• Technical access controls like registration walls
• Technical access controls like IP address blocking
• Signals such as robot exclusion headers

But, none of these mechanisms are guaranteed to work. This is an area of doctrinal chaos. I’m sure we’ll spend some time discussing if and when those efforts are effective. In general, for each bullet, I have more questions than answers.

Step 4: did the defendant’s unauthorized use create legally recognizable harm? This is also an unteachable aspect because, again, I typically have more questions than answers about what harm will suffice. The court’s flip-flop in the X v. Bright Data case is a good example of how unpredictable this topic is.

* * *

For my students, I summarize the law of scraping with the following tagline: “Scraping is ubiquitous but not clearly legal.” This breaks my students’ brains. If it’s so ubiquitous, it must be clearly legal…right?

In order for a scraping campaign to be legal, we have to navigate ALL of the following doctrines:

• Trespass to chattels. I tell my students that they should always discuss all three of these doctrines as a troika:
  • Federal statutes, especially the Computer Fraud and Abuse Act (CFAA).
  • State statutes, such as the state computer crime laws. Here in California, we focus on Cal. Penal Code 502.
  • Common law trespass to chattels, such as the Hamidi and Bright Data cases.
• Copyright law (106). I spend some time explaining why ordinary web browsing might be copyright infringement because copyrighted material necessarily is downloaded into the browser’s RAM. The students really don’t like that discussion LOL. If web browsing is infringement, then scraping would be as well. In addition to the initial downloading, scrapers might encounter further 106 trouble with republishing some or all of the scraped material.
• Copyright law DMCA (1201/1202). I don’t cover this angle in class yet because it’s too underdeveloped and chaotic, but I’m sure we’ll spend some time on 1201 today. To give you a preview, the 1201 rulings to date break my brain, and I have more questions than answers.
• Many TOSes contain anti-scraping provisions, but it’s not clear if the TOSes are properly formed. In my Internet Law course, we spend a full 2 weeks discussing TOS formation. The law evolved a lot in the 2025 decisions in Chabolla and Godun in the 9th Circuit.

There may be other legal doctrines raised by scraping, such as trademarks and privacy rights.

So here’s why I say that scraping is not clearly legal. In order for a scraping campaign to be permissible, it must survive each and every one of the doctrines enumerated above. If the campaign fails any one of the doctrines, it’s not legal and the scraper can face significant consequences (including, in some cases, criminal liability). It is daunting to build a scraping campaign that can successfully navigate so many legal doctrines perfectly.

* * *

My apologies for such a long-winded introduction (although, I remind you that I’m a law professor, so loquaciousness isn’t really that surprising, is it?) The law of scraping is so complicated and multi-faceted that it’s easy for the discussion to go down rabbitholes or for conversants to talk past each other. I hope this intro provides some helpful structure and context for my more detailed comments.

Thanks again for engaging with the AMA. A blanket apology in advance for all of the times I answer “it depends,” “I don’t know,” or “that’s a good question.” And in all cases, don’t take my word as the final one—talk with your lawyer if you are making any decisions!