Meta’s settlement with 52 state attorneys general is a bad deal for all internet users, and especially for teens. That’s what we said the day the settlement was announced.

In this post, we go through the Settlement’s provisions in detail and explain why that is so, including how:

• The Settlement embeds age assurance technology and age-gates into Meta’s social media products and requires all users—minors and adults—to undergo a rights-threatening age estimation process
• The Settlement places severe restrictions on Teens that can largely only be modified by the Teens’ parents and only then in exchange for giving their parents a ton of information about their online community and usage;
• The Settlement seems to empower the attorneys general to enforce Meta’s content restriction on “age inappropriate content,” categories that Meta itself has had trouble administering without excluding information about sexuality, sexual and reproductive healthcare, and abortion medication;
• The Settlement actually requires Meta to collect, analyze, and retain more information about its Teen users—when the pressure should have been on Meta to diminish its surveillance capitalism. And the Settlement in no way limits the attorneys general from seeking the user information for their own law enforcement purposes.

Note: A chunk of the settlement addresses unauthorized users under age 13, and Meta’s obligations to comply with the Children’s Online Privacy Protection Act. Meta policy has banned users under 13 since the company opened to the public in 2006. Aside from the age assurance frameworks that support both those and the other parts of the Settlement, the under-13 provisions are not addressed in this post. Those provisions essentially require Meta to detect and delete all under-13 accounts.

Further note: All U.S. states are parties to the Settlement except Florida, New Mexico, and Texas. The Settlement includes D.C., American Samoa, Guam, Northern Mariana Islands, and Puerto Rico.

Age Gates Reinforced By Age Estimation Technology

In the Settlement, Meta agrees to age-gate Instagram and Facebook, thus making age gates a legal mandate. And further, Meta will now enforce these age gates with age assurance technology, ditching its previous practice where the person signing up for the services self-attests to their birthdate. This concession firmly embeds deeply flawed age estimation technology into the online experience of millions of people around the world. First and foremost, the age verification setup seriously threatens online anonymity and privacy for everyone, as we’ve said before. The technology also just adds a layer of creepiness into the use of any service. In the Settlement, Meta pledges to, within one year, apply one or more age assurance methods to each Instagram or Facebook user in the states and territories that joined the Settlement. [P. 10, §II.A.1]

1. Age Assurance Framework. Within one (1) year of the Effective Date, Meta will adopt an age assurance framework (“Age Assurance Framework”), wherein it will apply one or more age assurance methods developed by a third party and licensed to customers (“Commercially Available Age Assurance Methods”) or age assurance methods developed by Meta (“Proprietary Age Assurance Methods”) (collectively, “Age Assurance Methods”) to each Meta SMP user in the Settling States. For the purposes of this Section II.A, an age assurance method developed or acquired by Meta that uses the same or functionally identical technology and methodology to a Commercially Available Age Assurance Method shall be treated as a Commercially Available Age Assurance Method. The Age Assurance Framework must include Age Assurance Methods to evaluate whether a Meta SMP user is a Teen User or U13, as described in Section II.A.6. New users of Meta SMPs who have not yet had their age assessed by an Age Assurance Method pursuant to Meta’s Age Assurance Framework shall receive the Default Protections pursuant to Section II.A.10 of this Agreement.

Those methods might include commercially available products, as well as proprietary age estimation process Meta might have or develop. Meta also pledges to consider age signals from Google and Apple operating systems and app stores. [§II.A.5] Meta has previously advocated for age assurance requirements to fall on Google and Apple rather than on individual services.

This age assessment essentially dumps users into one of three age-range buckets: 18+, 13-17, and under-13. Users under 13 have long been barred from Meta products, but this Settlement creates new obligations to search for and detect users who may have said they were older.

For those estimated to be over-18, the Settlement guarantees no direct benefit to you: no privacy protections, no greater user controls for your own accounts, no dent in Meta’s surveillance capitalism.

Those estimated to be 13-17 years old will be limited to Teen User accounts.

Those estimated to be under-13 will lose their accounts altogether.

Those who open new accounts will have two weeks to submit to age estimation, and if they decline to do so, Meta is now required to treat them as a Teen User by default, even if they self-identify as being 18 and older. [P. 18, §II.A.10.b]

(b) Fourteen (14) days or more after creating a Meta SMP account, Meta SMP users who have not yet had their age assessed by an Age Assurance Method pursuant to the Age Assurance Framework shall be treated as Teen Users for the purposes of this Agreement regardless of their stated age, except that Meta SMP users with a stated age of 18 years old or older shall receive the protections described in Section II.A.10.a.ii.

What about people with existing accounts, who are well past that two-week period to submit to age estimation? Will they also be defaulted to Teen User status if they decline age estimation? It seems so—the AGs would likely not have accepted a settlement that did not require Meta to take action against existing teen users who choose to forgo the age assurance process. Perhaps Meta will use its existing store of information about its current users as a type of permitted proprietary age assurance process? Thus, perhaps, an adult user whose Facebook account is itself older than 18 will be assessed as being over-18? Or a user who is identified as the spouse of a user who has been age-assured? But Meta can only rely on a proprietary process if it meets the accuracy standards set out in the Settlement Agreement, and that seems to require a formal assessment of accuracy.

How accurate does the age assurance process need to be?

The Settlement sets maximum false-positive rates for both commercially available and proprietary age assurance methods [Pages 12-13, §II.A.6]. Within two years, each shall be no more than 10% for ages 16-17 and no more than 3% for ages 13-15. 

6. Age Assurance Standards.
(a) U18 False Positive Rate Thresholds.
(i) Any Commercially Available Age Assurance Methods deployed by Meta for any new and existing users in the Settling States shall
meet or fall below the following U18 False Positive Rates excluding method circumvention within one year of the Effective Date: 10% for minors aged 16-17 and 3% for minors aged 13-15.
(ii) Any Proprietary Age Assurance Methods deployed by Meta for any new and existing users in the Settling States shall meet or fall below the following U18 False Positive Rates excluding method circumvention within 1 and 2 years of the Effective Date: 
(A) Year 1: Within one year of the Effective Date: 14% for minors aged 16-17 and 7% for minors aged 13-15.
(B) Year 2: Within two years of the Effective Date, 10% for minors aged 16-17 and 5% for minors aged 13-15.

Notably, there is no limit indicated in the error rate for false negatives, when the process wrongly identifies an adult as being under 18. The Settlement generally shows little concern for those falsely placed in its Teen User category. The Settlement only requires Meta to maintain an appeal process for users whose age range is wrongly assessed. [P. 17, §II.A.9]

9. Appeals Process. Users claiming to have been mis-identified as minors must be offered a Clear and Conspicuous means to appeal the decision. Decisions on all user appeals must be made in a timely manner and communicated to the user along with a basis for the decision.

Meta must also employ measures to discourage age estimation circumvention, including placing limits on the number of attempts any user might make. [P. 16, §II.A.7] As part of this, Meta agrees to proactively monitor adult accounts to determine whether a user needs to undergo additional age estimation. [P. 16, §II.A.7.c] This is just one of the ways the Settlement embeds Meta’s active surveillance of its users for the next ten years (see below for more).

(c) Incorporating a proactive monitoring system that requires users to undergo an additional Age Assurance Method where a user is determined, including based on their conduct on Meta SMPs, to have likely circumvented the Age Assurance Method and is: (A) likely a Teen User after having been previously assessed as 18 or older; or (B) likely U13 after having been previously assessed as 13 or older. Users Meta determines are likely Teen Users after having been previously assessed as at least eighteen may choose not to undergo additional age assurance but then will be treated as Teen Users; and 

Any age assurance process Meta uses must be tested annually.

Data minimization

The Settlement does have data minimization requirements for the data collected during the age assurance process. [§II.A.8] But there are numerous holes. The Settlement requires that all information obtained and retained as part of the age assurance processes thereafter be “immediately enqueued for deletion, after which it shall be deleted after a reasonable period of time.” The “reasonable period of time” is not defined. And the Settlement defines a category of “Retainable Data” that may be retained for 90 days. This includes “metadata about the age assurance method used by the user information ... where required to ensure the ongoing integrity of age assurance systems, including but not limited to the ability to identify circumvention and related patterns over time, and only for as long as it is required for those purposes.” The Settlement requires that all data collected by Meta or its vendor be stored according to industry-standard data security measures,” a standard that unfortunately does not eliminate the risk of a data breach.

8. Data minimization and security.
(a) Except as set forth herein, all data collected by Meta from users of Meta SMPs in the Settling States for the sole purpose of conducting age assurance, all data maintained from known U13s, and all data collected by a vendor for use in a Commercially Available Age Assurance Method shall be held for the minimum period required to determine a user’s age status and thereafter immediately enqueued for deletion, after which it shall be deleted in a reasonable period of time. Meta may retain (1) U13 data only to the extent required for purposes of developing, training, testing, and measuring the performance of the U13 Age Model (“U13 Data”), provided that any U13 data that constitutes Personally Identifiable Information as defined by 16 C.F.R. Part 312.2 will be protected using Meta’s highest data privacy and security standards, and (2) metadata about the age assurance method used by the user information (“Retainable Data”) only where required to ensure the ongoing integrity of age assurance systems, including but not limited to the ability to identify circumvention and related patterns over time, and only for as long as it is required for those purposes. For the avoidance of doubt, U13 Data cannot be used for purposes such as ads targeting and delivery, marketing, or algorithmic optimization efforts. Any U13 Data and Retainable Data shall be maintained at the coarsest viable granularity and cannot be used for any other purpose unless legally required. Any Retainable Data that is no longer required for the purposes set forth herein shall be deleted within 90 days. The terms above shall not pertain to the user’s stated date of birth, stated age, nor the outcome of the Age Assurance Method (e.g., “teen or adult” classification).
(b) Data collected by Meta or a vendor, or transmitted by a vendor, must be collected and stored using industry-standard data security measures and as required by law, including encryption in transit and at rest.
(c) The Parties agree to discuss in good faith potential modifications to this provision as necessary to permit Meta to improve the efficacy of its Age Assurance Framework while preserving the principles of data minimization and security set forth in this Section.

Restrictions For Teens (And Anyone Who Opts Out of Age-Gating)

Teen User Accounts are subject to time, feature, and content restrictions. These will be applied to those estimated to be 13-17 as well as any new user who declines to submit to the age assurance process and to existing users who decline to submit to age estimation and for whom Meta cannot ascertain that they are older than 17.

Time restrictions

Teen Users will be subject to the following time restrictions (§II.B). These measures seem to assume that most of teen’s social media use is frivolous and unserious (not that those are in and of themselves bad; the right to play is among young people’s human rights), ignoring the fact that teens use social media for school and personal research, conducting activism campaigns, and other endeavors that might naturally not fit within these time limits, and that some teens may need to work at the restricted times to support their families and themselves:

• Night Access Mode – no access (except messaging) to Instagram and Facebook from Midnight to 6 AM, and no push notifications from 10 PM – 7 AM.
• School mode – no push notifications from 8 AM – 3 PM Monday-Friday from Aug 15-June 15.
• Daily cumulative time limit of 2 hours per day across Instagram and Facebook, resetting at midnight, excluding video and audio content at least 22 minutes long, absent artificial prolongation, defined by the Settlement as “longform content.”
• “Productive pauses and notices” designed to “reduce or prevent excessive, mindless, or unintended teen usage.” This means that a teen’s usage will be momentarily paused after 60 and 90 minutes of daily cumulative use with notices sent every 15 minutes of continuous use. According to the Settlement, these productive pauses and notices will look like this:

To be clear, the ability to set time limits, blackout times, and scheduled pauses are all useful features that should be available and easy to implement for users of all ages. Such tools would have allowed teens, and all users, the ability to design their own safe experience, customized to their own needs, online. Such user controls would have recognized that teens have human rights, agency, and autonomy.

But that’s not what these restrictions are. They are not tools that give the teen users control. Rather, they are imposed, top-down, on teens and anyone else who declines to submit to Meta’s age assurance process.

Feature restrictions (§II.C-D)

Within four months of the effective date of the Settlement, Meta must offer teens an option for a non-personalized feed, which is defined as a feed of chronologically ordered posts from friends and follows. Teens will also be able to disable autoplay as part of an “optional protective settings” package. Each of these settings must be “viewable within three user gestures and clearly labeled, easy to notice, viewable without scrolling, and discoverable in an intuitive location within” the service. 

Again, these would be useful user controls that should be offered to users of all ages. And while a Teen User potentially has control over these features, they cede that control to their parent once they enroll in Parental Supervision, part of the Parental Supervision Tradeoff discussed below.

Also, by default, teens will not see the number of likes or other reactions to their posts.

Teens will also not have access to what the Settlement calls “Cosmetic Procedure Filters,” that is, “any digital filter or augmented reality effect that distorts, sculpts, redefines, or idealizes a user’s face in a way that cannot be achieved without cosmetic surgery or extreme make-up techniques.”

X. “Cosmetic Procedure Filter” shall mean any digital filter or augmented reality (AR) effect that distorts, sculpts, redefines, or idealizes a user’s face in a way that cannot be achieved without cosmetic surgery or extreme makeup techniques. For the avoidance of doubt, “Cosmetic Procedure Filter” does not include: (1) Fantasy/Character Effects: Filters that change a user’s facial structure for the purpose of turning the user into a non-human or fictional character (e.g., an elf or cartoon) or an animal (e.g., a dragon or puppy); (2) Makeup/Smoothing Effects: Filters that smooth skin or alter appearance in ways that can be achieved by ordinary makeup techniques without altering the appearance of underlying physical facial structure or meaningfully changing skin tone; or (3) Parody and Exaggeration Effects: Effects that entertain users by distorting their appearance through parody, satire, or exaggerated forms (e.g., extreme visual distortions outside the scope of normal cosmetic procedures). To help operationalize this definition, the Settling States will provide Meta illustrative examples and guidance of AR effects that are Cosmetic Procedure Filters and AR effects that are not Cosmetic Procedure Filters in a letter to be sent within two (2) months of the Effective Date.

Meta has had rules about cosmetic effects directed at teens since 2019. But the Settlement will give the states a major role in helping Meta identify what features are and are not Cosmetic Procedure Filters. 

Content restrictions (P.1, §II.E, as defined by §I.C, E, F)

For content, Meta is basically pledging to continue its existing practices limiting Teen Users to age-appropriate content and accounts, to default Teen Users to age-appropriate experiences. This includes limiting access to accounts that “regularly share content that is inappropriate for teens” such as content from the following Meta community standards categories: Adult Nudity & Sexual Activity, Restricted Goods & Services, Suicide, Self-Harm or Eating Disorders.

C. “Age Appropriate Experiences” refers to content that is appropriate for Teen Users based on input from experts, parents, and teens. For the purposes of this Agreement, Age Appropriate Experiences shall mean content captured in Meta’s applicable Ages 13+ content setting, which is tied to policies inspired by movie ratings for ages 13+ and parent feedback.
D. “Age Assurance Methods” shall have the meaning set forth in Section II.
E. “Age Inappropriate Accounts” refers to accounts that regularly share content that is inappropriate for teens or that have account information that otherwise suggests the account is inappropriate for Teen Users. For the purposes of this Agreement, Age Inappropriate Accounts shall mean accounts that: (1) regularly share Age Inappropriate Content in the following categories: Adult Nudity & Sexual Activity, Restricted Goods & Services, Suicide, Self-Harm or Eating Disorders, or (2) have account names or profile photos or bios that suggest the account is otherwise inappropriate for minors, based on Meta’s policies for the following categories: Adult Nudity & Sexual Activity, Restricted Goods & Services, Suicide, Self-Harm or Eating Disorders.
F. “Age Inappropriate Content” refers to content that is generally perceived by U.S. parents, youth experts, and teens as not being appropriate for Teen Users. For the purposes of this Agreement, Age Inappropriate Content shall mean content prohibited by Meta’s Community Standards concerning bullying and harassment; nudity and sexual activity; child sexual exploitation, abuse, and nudity; sexually explicit language; suicide, self-harm and eating disorders; graphic violence and incitements to violence; gambling; and restricted substances or goods (including illegal drug use), as well as policies specifically focused on protections for Teen Users, including those regarding high-risk viral challenges and risky stunts.

The issue here is that some of these categories are problematic. For example, the Restricted Goods & Services standard has been used by Meta to justify removing information about abortion medication, as we detailed in our Stop Censoring Abortion campaign, and in our comment to the Meta Oversight Board. And under the Adult Nudity & Sexual Activity standard, Meta blocks teens from “real world art of visible genitalia ... where the nudity is the focus of the image” and has a history of applying the standard inconsistently, including with respect to representations of indigenous women, breast cancer awareness posts,  educational posts about ovulation, and posts about testicular and breast self-exams. And it has disproportionately applied the standard negatively to gay and lesbian content as compared to straight content.  

And even more worrisome, even though this is just Meta continuing its existing practices, the Settlement empowers the states to enforce its provisions. [P. 40, §IV.C.1.i; §VII.C] That means that over the next ten years, the duration of the Settlement, Meta will face the threat that a state attorney general will pursue legal action against it because it disagrees with how Meta interprets these categories of community standards, and pressures Meta to eliminate Teen User access to posts about sexuality and reproductive and sexual health. And Meta will now lack the hard-earned First Amendment defenses to make its own curatorial decisions. 

C. Notwithstanding anything in Sections VIII.A-B above, a Settling State may take any action, including but not limited to legal action to enforce compliance with the Agreement, without delay if the Settling State believes that a threat to the health or safety of the public requires immediate action.

The Parental Supervision Tradeoff 

All of these Teen User restrictions can be modified – but only if the Teen User enrolls in Parental Supervision that links their account to a parent’s or guardian’s account. Once their accounts are linked, parents can modify the Teen User settings to make them less restrictive (they need the teen’s permission if they want to make them more restrictive).

Parental Supervision comes with huge tradeoffs. There is a huge privacy tradeoff: in exchange for designating someone as their Parent, the Parent gets a lot of information about the Teen’s use: the usernames of all of the teen’s connections, reports on how much time the Teen User spends on a Meta service, the time spent watching longform content, usernames of all those messaging with the Teen User, and any evidence Meta has about suspected secondary accounts. The Supervising Parent also gets notices of the teen’s repeated searches related to suicide, self-harm and eating disorders. [P. 28, §II.G] And there are huge autonomy tradeoffs: once enrolled in Parental Supervision, the ability to control features like recommendations and autoplay, discussed above, transfer from the Teen user over to their parent.

Parental Supervision
1. Meta agrees to provide Supervising Parents with information concerning the amount of time their Teen User is spending on Meta SMPs, including separately for time on the Meta SMP, time using the Meta SMP’s messaging features, time viewing Longform Content (to the extent excluded from the calculation of daily limits pursuant to Section II.B.3.a.i), and the usernames of the Teen User’s social connections and individuals messaging the Teen User, and usernames of any user reported by a Teen User. In addition, Instagram will notify Supervising Parents should their Teen User engage in repeated searches for terms related to suicide, self-harm, or eating disorders, and Facebook will notify Supervising Parents should their Teen User engage in repeated searches for terms related to suicide, self-harm, or eating disorders.
2. For Teen Users, Supervising Parents will be notified on a daily basis when the Teen User communicates directly with each adult user for the first time with the Teen User and shall provide a link to the adult user’s profile to provide information such as their stated hometown or city or mutual connections, to the extent the information is publicly available.
3. Upon a Teen User enrolling Parental Supervision, Meta SMPs shall prompt the Supervising Parent to review their Teen User’s settings to consider making updates to support how their Teen User spends their time on Meta SMPs. Meta shall not preselect, recommend, or encourage settings to Supervising Parents that are less restrictive than the default settings in this Agreement.
4. Meta SMPs will periodically suggest checkups for Supervising Parents to do with their Teen Users to evaluate their settings and usage.
5. Meta SMPs agree to continue to take steps designed to encourage enrollment in their Parental Supervision tools.
6. If a Teen User is enrolled in Parental Supervision, Meta shall notify the Supervising Parent if the Teen User creates or explicitly links a new secondary Meta SMP account within their Accounts Center or has been linked to a Soft Matched Account through Meta SMPs, including a link to the secondary account’s profile. Meta SMPs shall automatically apply the Supervising Parent’s approved time management settings in Section II.B and content restrictions to all explicitly linked Meta SMP accounts and all Supervised Accounts for a Teen User belonging to the same Meta SMP.

Unfortunately, Meta doesn’t have a great way to learn whether there exists a genuine parent-Teen User relationship. But it will try to get it right. [§II.G.8] 

This may be ultimately workable for young people with healthy and safe relationships with their parent or guardian. But obviously it is not good at all for a Teen User lacking such a safe relationship. 

More Surveillance, Not Less

Moreover, rather than pushing Meta away from the collection, analysis, and retention of user data, the Settlement requires Meta to do more of it for the next ten years. Several provisions of the Settlement require Meta to carefully track its users' use of Instagram and Facebook for the purposes of determining whether its age assurance framework and Teen User restrictions are working as intended.  

For example: 

• Meta promises to review detected U13 users’ friend networks to identify other possible U13 users. [§II.A.6.(b).(i).(D)] 

• Meta will incorporate a “proactive monitoring system” to identify possible Teen Users among those who were initially determined to be older than 17 by its age assurance process. [§II.7.(c)] 

• Meta pledges to utilize and improve its existing “soft matching models” that track signals such as “device IDs, phone numbers, and email addresses” to identify duplicate accounts. [§II.B.6.(a)] 

• Meta will continue to monitor Teen Users’ activity to “regularly evaluate the prevalence of Teen User exposure to Age Inappropriate Content, and Teen User exposure to experiences that are not Age Appropriate Experiences,” [§II.E.3] and also to “regularly evaluate the prevalence of Teen User exposure to Harmful Experiences on Meta SMPs.” [§II.F.3] 

• Monitor Teen Users’ messaging to identify message threads with potentially suspicious accounts. [§II.F.4] 

• The Settlement also requires Meta to actively monitor use of its services so that it can supply data to the Independent Auditor that will now monitor its compliance with the Settlement. “The Independent Auditor is entitled to access the non-privileged information, personnel, systems, and records that are reasonably relevant and sufficient to evaluate Meta’s implementation of the Injunctive Relief Terms, including, but not limited to, access to raw data; aggregated data; information; internal documents and communications” plus information from its age assurance processes, data regarding Teen User responses to the Productive Pauses, data regarding its models for soft matching of secondary accounts, data regarding the prevalence of Teen User exposure to Age Inappropriate Content and Harmful Experiences, and to experiences that are not Age Appropriate Experiences. [§III.E] 

Moreover, one of the chief threats of Meta’s surveillance is the honeypot of data it creates that may be accessible to governments for law enforcement and other investigations. Nowhere in the Settlement do the 52 attorneys general pledge to not try to access all of the data the Settlement requires Meta to collect and retain. 

Meta Has To Pay The States And Pays To Establish Norms Beyond Meta

The Settlement also includes annual payments from Meta to the states, apparently proportionate to the size of each state’s teen user base. Over the ten-year life of the Settlement, these annual payments will total over $11 billion. And the states then get an additional $5 billion if Meta competitors adopt the same measures. 

This quirk of the Settlement incentivizes the States to pursue similar age assurance processes and at-least-as-restrictive teen user measures for Meta’s chief existing competitors for teen use, YouTube, TikTok, and Snap, and for any new service that may gain widespread teen use over the life of the Settlement Agreement. If the states are able to get Meta’s competitors to adopt the same measures, then the states will get the additional $5 billion in annual payments. That’s quite the incentive for the states to pursue litigation and regulatory measures against those companies. All of this will further entrench age assurance, age-gating, and the ceding of teen autonomy as the norm across online services. 

1. In the event the Contingent Monetary Payment Trigger has occurred in a Settling State, Meta shall be obligated to pay to such Settling State ten equal installments in the amount as set out in Exhibit B (each, a “Contingency Installment Payment”).

2. Following the date the Contingent Monetary Payment Trigger has occurred, the Contingency Installment Payments shall be made to the Settling State on January 15 in each subsequent calendar year of the Agreement Term as follows:

(a) At the next scheduled payment date, the Settling State shall be paid the Contingency Installment Payment for that payment date and all prior payment dates.
(b) For each of the remaining payment dates, the Settling State will be paid the Contingency Installment Payment for that payment date.

3. If a Settling State fails to achieve the Contingent Monetary Payment Trigger during the Agreement Term, the Contingency Installment Payments shall be permanently forfeited by such Settling State and retained by Meta. For the avoidance of doubt, no Settling State will have an obligation to repay the Contingency Payment

The Settlement is thus a bad deal for all users of Facebook and Instagram. It normalizes age gating and age assurance for millions of internet users. It denies teens the tools to create their own safe experiences online and places their social media experience firmly under the control of either Meta or their parents. And rather than addressing Meta’s collection, analysis, and retention of data about teens’ use of Instagram and Facebook, it binds Meta to continued surveillance, and does nothing to protect access to such data by the states. 

A federal appeals court just made it harder for online services, big and small, to get lawsuits over user speech dismissed early. In California v. Meta, a Ninth Circuit three-judge panel held that the lower court’s denial of Section 230 immunity to Meta is not immediately appealable. The misguided ruling has the potential to have widespread impact and to threaten the free speech of all internet users.

The ruling is bigger than a loss for Meta, which has the resources to defend itself against these lawsuits. The court’s ruling signals that all online services (and internet users) that host others’ speech—including those without Meta’s deep pockets—must bear the burden and expense of fighting lawsuits that Section 230 ultimately precludes. This will have real consequences, incentivizing online services to take down users’ speech in response to spurious legal threats, filter speech preemptively, or simply stop offering a place for people to speak online. So even though some may think that Meta is not a sympathetic company, the ruling should raise concerns for anyone who cares about an open and free internet.

Immunities from Suit Advance Important Public Interests

A little procedural background is necessary to understand the implications of the Ninth Circuit’s ruling.

Meta had moved to dismiss a group of social media addiction cases brought by state attorneys general, school districts, and local governments. Meta argued that Section 230(c)(1) immunity applies because the plaintiffs’ claims, framed as seeking to hold Meta liable for allegedly harmful platform features, really seek to hold the company liable for publishing decisions related to third-party content. Section 230 is one of the most important laws supporting online free speech, because its protections for online services enable them to distribute users’ speech at an unprecedented scale.

The district court ruled that Section 230 does not apply to certain features (and does apply to others) and so denied the motion to dismiss on the claims related to those features. Meta immediately appealed invoking appellate jurisdiction under 28 U.S.C. § 1291, but the question before the Ninth Circuit was whether the appeal was legally appropriate.

Under Section 1291, U.S. circuit courts generally only have jurisdiction to hear appeals of “final decisions” from the district courts. Final decisions are trial court orders ending a case, or come after a trial on the merits. Section 230 appellate cases often arise from a district court’s grant of a defendant platform’s motion to dismiss the plaintiff’s case based on Section 230. Typically, a district court’s denial of a defendant’s motion to dismiss is not a final order—it simply means that the case may continue to discovery and summary judgment or trial, after which time an appeal would be appropriate.

However, federal law allows for “interlocutory appeals,” which are appeals of orders that do not end a case but nonetheless are allowed because they involve important legal issues. For example, there is an exception to Section 1291 called the “collateral order doctrine”—at issue in this case—allowing for immediate appeal if, as the Ninth Circuit explained here, “holding a trial would imperil a substantial public interest.”

Inherent in the collateral order doctrine is the consideration of whether an immunity like Section 230 provides mere “immunity from liability” or a more robust “immunity from suit.”

An immunity from liability does not require an immediate appeal and so demands that Section 1291’s final order rule be followed. That’s because waiting until the end of a case before an appellate court can consider the trial court’s denial of immunity does not prejudice the defendant. The appellate court may overturn the trial court and grant the immunity, and thus the defendant’s right to be immune from liability would be vindicated on appeal.

Immunity from suit is different. It means that the public interest demands that a defendant be able to get out of a case as early as possible and avoid having to litigate the case to the end. The U.S. Supreme Court has held, for example, that qualified immunity is such an immunity, and that a district court’s denial of qualified immunity for a government official is immediately appealable under Section 1291, notwithstanding the lack of a final order. The idea is that the public interest is served when government officials are free to act without fear of consequences when established rights are not implicated, and so determining as soon as possible whether their acts are immune serves that public interest.

Here, the Ninth Circuit held that the district court’s denial of Section 230 immunity for Meta was not immediately appealable under Section 1291’s collateral order doctrine because the immunity is not from suit, but rather from ultimate liability. The panel’s absurd result contravenes the text of Section 230, the statute’s policy goals, and the court’s own prior rulings.

Treating Section 230 as an Immunity from Suit Protects Online Free Speech

Meta rightly argued that Section 230(e)(3) plainly states, “No cause of action may be brought and no liability may be imposed under any State or local law that is inconsistent with this section.” The panel dismissed this argument, stating that this language likely amounts to “redundancy” reflecting only immunity from liability. The court failed to side with the more reasonable position that statutory language should generally not be interpreted as superfluous.

Meta also reminded the panel that the Ninth Circuit has many times over the past two decades framed Section 230 as both an immunity from liability and an immunity from suit. The panel also dismissed this argument, stating, “It is true that we have used the phrase ‘immunity’ somewhat loosely in our section 230 jurisprudence.”

But “loosely” is a gross mischaracterization—the panel did not discuss a seminal prior ruling, Fair Housing Council of San Fernando Valley v. Roommates.com (2008), in which the entire Ninth Circuit, not just a three-judge panel, explicitly ruled that Section 230 is also an immunity from suit. That court rightly explained that Section 230 “must be interpreted to protect websites not merely from ultimate liability, but from having to fight costly and protracted legal battles.”

Why is it important that social media platforms and other internet intermediaries (and their users) have immunity from suit for engaging in publishing activities related to third-party content—and thus a right to immediately appeal when Section 230 immunity is denied?

The Ninth Circuit panel here, using their own words, failed to “evaluate the interests that would be lost through rigorous application of a final judgment requirement” and failed to consider the “substantial public interest” served by treating Section 230 as an immunity from suit.

Section 230 immunity, contrary to what some argue, is not a gift to Big Tech—it applies to all internet intermediaries, big and small, from the large social media companies to smaller entities like community message boards and local ISPs. It even protects internet users who forward others’ emails or host comments on their blogs. In turn, the law supports the free speech of all internet users.

While it is helpful when an internet intermediary can ultimately benefit from Section 230 immunity, if a trial court’s early denial is not immediately appealable, that means the intermediary must bear the extended logistical and financial burdens of defending itself. Under the Ninth Circuit’s logic, anyone hosting others’ speech online would have to endure the pain and expense of discovery, summary judgment, or trial, before they ultimately can be protected by Section 230.

Congress crafted Section 230 to give internet intermediaries legal breathing room, so that they will be incentivized to facilitate online communication and commerce, allowing the rest of us to go online with minimal barriers to entry, without needing to have loads of money or to know how to code. Congress acknowledged in Section 230 itself, “Increasingly Americans are relying on interactive media for a variety of political, educational, cultural, and entertainment services.”

Yet if platforms, especially smaller platforms, know that they will have to defend themselves for years in court before they can ultimately benefit from Section 230 immunity, this alone will create a perverse incentive, as we have explained, to censor user speech, in order to reduce the platforms’ legal exposure. And this incentive is only exacerbated at scale, where the sheer volume of user-generated content hosted by modern platforms makes legal risk astronomical.

Unfortunately, this opinion seems to be part of larger trend reflecting the Ninth Circuit’s increasing disdain for Section 230, and apparently for free speech rights more broadly. The court similarly held last year in Gopher Media v. Melone (2025)—overruling itself—that a trial court’s denial of a defendant’s anti-SLAPP motion also is not immediately appealable under the collateral order doctrine. This is despite the fact that, similar to Section 230, California’s anti-SLAPP law is intended to allow defendants to get harassing lawsuits meant to silence them dismissed early, lest they be chilled from engaging in lawful speech on public issues due to the risk of being mired in litigation, even if they ultimately win a delayed appeal.

Intermediary liability laws around the world recognize that social media platforms, search engines, and other online service providers have become an integral part of our lives: they shape how we access information, communicate with others and participate in public debate, and foster innovation online. These laws generally shield platforms, to varying degrees, from legal liability for user content: the responsibility for unlawful speech should rest primarily with the speaker, not with those who merely host it.  

These liability protections are not a gift for platforms. They exist so that platforms are not encouraged to proactively monitor and filter what we say online, or to remove even lawful speech simply to avoid legal risk. 

This is why a recent judgment by the EU Court of Justice, Coyote System (Joined Cases C-188/24 and C-190/24), is concerning: it could deprive online platforms of liability protection because of how they organize and disseminate user content. The consequences for freedom of expression could be significant. 

Liability Protections in the EU 

The European Union has long embraced a system of limited liability for online service providers. Under the e-Commerce Directive and now the Digital Services Act (DSA), platforms benefit from liability exemptions for user content. To discourage censorship, they also cannot be required to generally monitor user content or actively search for illegal activity. But that liability protection comes with qualifications: Platforms lose this benefit if they play an "active role" such that they have knowledge of, or control over, user-provided information (Recital 42 ECD, Recital 18 DSA, and case law, for example para. 113 in L’Oréal v eBay). For hosting services, providers must remove or disable content they know to be illegal. The DSA has introduced extensive due diligence obligations for platforms but left these foundational immunities intact. The message is clear: platforms bear responsibility for proper systems and processes, but generally not for users' speech. 

Coyote System, however, could undermine this balance. Confronted with a case about restrictions on navigation systems that transmit information to drivers about roadside checks, the Court formulated a general test for when an intermediary ceases to be a "neutral" host and therefore loses the hosting liability exemption. In essence, the Court held that where an intermediary's algorithm goes beyond merely categorizing and indexing user information to determine, "under what conditions, how and in which order of priority" (para. 122) information is disseminated, the intermediary "controls" that information and is deprived of protection under the e-Commerce Directive. 

Let's be clear: the case is not about a service that ranked or recommended user-generated content in the way social media platforms do. It is about the collection and real-time relay of user alerts about roadside checks. However, the Court's reasoning is not confined to navigation services. Recommendation algorithms determine how and in what order user content is disseminated across virtually every major online platform. Should such platforms now cease to qualify as neutral intermediaries and lose the protection of the hosting liability exemption? The answer should be no. 

The Meaning of Control 

Control has never been understood this broadly. Nor should it be. Every hosting service provider, think of Facebook, Amazon or Bluesky, will have some control over users’ content. If that ability alone ruled the analysis, the liability exemption would become largely meaningless. Instead, the disqualifying “active role” must relate to the actual content itself, not merely the technical means by which that content is organised or disseminated. 

The Court’s own case law reinforces this conclusion: In YouTube and Cyando, it examined a platform that categorises, ranks and recommends user content through algorithms, yet still proceeded on the basis that it could generally benefit from the hosting liability exemption. To be sure, the Court was mainly addressing specific knowledge of illegal content rather than the separate category of control. Even so, the underlying premise is clear: those features do not, by themselves, place a platform outside of protection. Advocate General therefore explained that what matters is the provider's "intellectual control of that content" (para 152). The relevant question is who controls the information itself, makes it their own, not who determines how it appears. 

That is precisely where Coyote System breaks new ground and offers a dangerous change of emphasis. By equating algorithmic organisation with content control, the ruling risks excluding social networks and other platforms from the liability exemption and encouraging proactive monitoring of what users say online and removal of lawful content. 

That outcome would have terrible consequences for freedom of expression in the EU. It’s also difficult to reconcile with the structure of the DSA, which certainly does not treat recommendation algorithms as incompatible with intermediary immunity. On the contrary, it accepts them as a defining feature of modern platforms, regulates them extensively through dedicated due diligence obligations, and still leaves the hosting liability regime untouched (it even integrated the YouTube ruling in its preamble!).  

This was no accident: During the DSA negotiations, proposals to deprive platforms of the hosting liability exemption if they optimize, classify, organize or otherwise promote online content were rejected, following successful advocacy by EFF and allies. Would the Court have decided this case differently under the DSA? Probably not. It’s more plausible that the EU judges were influenced by the specific nature of the service, which could explain why the judgment says remarkably, and sadly, little about why intermediary liability exists in the first place and the fundamental rights it serves. Coyote System did not merely transmit user reports but aggregated them into what the Advocate General described as a new "information layer," a distinction omitted by the Court. 

Chipping Away at Intermediary Liability Protections 

The danger is that the Court's broad language on algorithmic curation reaches well beyond that narrow category and, unintentionally or not, chips away at one of the most important safeguards for freedom of expression online.  

Unfortunately, Coyote System does not stand alone. It is the latest in a line of judgments that have gradually narrowed intermediary liability protections. Recently, in Russmedia, the Court privileged preventive content control in the name of data protection, paying little regard to the possibility of reconciling both regimes and the privacy costs of increased monitoring of user content. And in AGCOM, concerning Google's liability for YouTube videos uploaded by creators participating in its Partner Programme, the Court appears to leap from eligibility reviews to specific knowledge of illegal content. 

There is a political risk too. While the top court’s reasoning will be applied by national courts and further refined over time, the European Commission has shown little hesitation in incorporating landmark rulings into legislation. Just recently, in its digital omnibus proposal, it selectively restated part of a recent Court of Justice judgment to justify narrowing privacy rights of users. 

If these trends continue, freedom of expression online will become collateral damage in the EU. 

If you want to overthrow Big Tech, you’ll need Section 230. The paradigm shift being built with the Open Social Web can put communities back in control of social media infrastructure, and finally end our dependency on enshitified corporate giants. But while these incumbents can overcome multimillion-dollar lawsuits, the small host revolution could be picked off one by one without the protections offered by 230.

The internet as we know it is built on Section 230, a law from the 90s that generally says internet users are legally responsible for their own speech — not the services hosting their speech. The purpose of 230 was to enable diverse forums for speech online, which defined the early internet. These scattered online communities have since been largely captured by a handful of multi-billion dollar companies that found profit in controlling your voice online. While critics are rightly concerned about this new corporate influence and surveillance, some look to diminishing Section 230 as the nuclear option to regain control. 

The thing is, that would be a huge gift to Big Tech, and detrimental to our best shot at actually undermining corporate and state control of speech online. 

Dethroning Big Tech

We’re fed up with legacy social media trapping us in walled gardens, where the world's biggest companies like Google and Meta call the shots. Our communities, and our voices, are being held hostage as billionaires’ platforms surveil, betray, and censor us. We’re not alone in this frustration, and fortunately, people are collaborating globally to build another way forward: the Open Social Web. 

This new infrastructure puts the public’s interest first by reclaiming the principles of interoperability and decentralization from the early internet. In short, it puts protocols over platforms and lets people own their connections with others. Whether you choose a Fediverse app like Mastodon or an ATmosphere app like Bluesky, your audience and community stay within reach. It’s a vision of social media akin to our lives offline: you decide who to be in touch with and how, and no central authority can threaten to snuff out those connections. It’s social media for humans, not advertisers and authoritarians.

Behind that vision is a beautiful mess of protocols bringing open social media to life. Each protocol is a unique language for applications, determining how and where messages are sent. While this means there is great variety to these projects, it also means everyone who spins up a server, develops an app, or otherwise hosts others’ speech has skin in the game when it comes to defending Section 230.

What exactly is Section 230?

Section 230 protects freedom of expression online by protecting US intermediaries that make the internet work. Passed in 1996 to preserve new bubbling communities online, 230 enshrined important protections for free expression and the ability to block or filter speech you don’t want on your site. One portion is credited as the “26 words that created the internet”:

“No provider or user of an interactive computer service shall be treated as the publisher or speaker of any information provided by another information content provider.” 

In other words, this bipartisan law recognizes that speech online relies on intermediaries — services that deliver messages between users — and holding them potentially liable for any message they deliver would only stifle that speech. Intuitively, when harmful speech occurs, the speaker should be the one held accountable. The effect is that most civil suits against users and services based on others' speech can quickly be dismissed, avoiding the most expensive parts of civil litigation. 

Section 230 was never a license to host anything online, however. It does not protect companies that create illegal or harmful content. Nor does Section 230 protect companies from intellectual property claims. 

What Section 230 has enabled is the freedom and flexibility for online communities to self-organize. Without the specter of one bad actor exposing the host(s) to serious legal threats, intermediaries can moderate how they see fit or even defer to volunteers within these communities.

Why the Open Social Web Needs Section 230

The superpower of decentralized systems like the Fediverse is the ability for thousands of small hosts to each shoulder some of the burdens of hosting. No single site can assert itself as a necessary intermediary for everyone; instead, all must collaborate to ensure messages reach the intended audience. The result is something superior to any one design or mandate. It is an ecosystem that is greater than the sum of its parts, resilient to disruptions, and enables free experimentation with different approaches to community governance.

The open social web’s kryptonite though, is the liability participants can face as intermediaries. A greater potential for liability comes with more interference from powerful interests in the form of legal threats, more monetary costs, and less space for nuance in moderation. And in practice, participants may simply stop hosting to avoid those risks. The end result is only the biggest and most resourced options can survive.

This isn’t just about the hosts in the Open Social Web, like Mastodon instances or Bluesky PDSes. In the U.S., Section 230’s protections extend to internet users when they distribute another person’s speech. For example, Section 230 protects a user who forwards an email with a defamatory statement. On the open social web, that means when you pass along a message to others through sharing, boosting, and quoting, you’re not liable for the other user’s speech. The alternative would be a web where one misclick could open you up to a defamation lawsuit.

Section 230 also applies to the infrastructure stack, too, like Internet service providers, content delivery networks, and domain or hosting providers. Protections even extend to the new experimental infrastructures of decentralized mesh networks.

Beyond the existential risks to the feasibility of indie decentralized projects in the United States, weakening 230 protections would also make services worse. Being able to customize your social media experience from highly-curated to totally laissez-faire in the open social web is only possible when the law allows space for private experiments in moderation approaches. The algorithmically driven firehose forced on users by antiquated social media giants is driven by the financial interests of advertisers, and would only be more tightly controlled in a post-230 world.

Defending 230

Laws aimed at changing 230 protections put decentralized projects like the open social web in a uniquely precarious position. That is why we urge lawmakers to take careful consideration of these impacts. It is also why the proponents and builders of a better web must be vigilant defenders of the legal tools that make their work possible. 

The open social web embodies what we are protecting with Section 230. It’s our best chance at building a truly democratic public interest internet, where communities are in control.

SmugMug is a family-owned photo hosting and e-commerce platform that helps professional photographers run their businesses online. Founded in 2002, the company provides tools for photographers to show their work, deliver client galleries, sell prints, and manage payments. 

In 2018, SmugMug purchased Flickr, the long-running photo-sharing community, which added tens of millions of active hobbyist photographers to the company’s user base. 

Ben MacAskill is President and COO of SmugMug’s parent company, Awesome, which he co-founded with his family. Awesome also includes the media network This Week in Photo and the nonprofit Flickr Foundation, which focuses on preserving publicly available photography. MacAskill has been an active voice in policy discussions around Section 230 and online platform regulation. He was interviewed by Joe Mullin, a policy analyst on EFF's Activism Team.

Joe Mullin:  How would you explain Section 230 to a SmugMug photographer who hasn't heard of it but relies on you to share their work, run their business.

Ben MacAskill: Section 230 allows us to run our business. We are a small, family run business. We don’t have the resources to police every single upload, every single comment, or every single engagement that happens on the site. 

That includes photographers who have comments on their sites. Anywhere there’s interaction online, Section 230 protects us. 

It doesn't absolve us of liability. We can't run rampant and do anything we want. It  just helps protect us and make it scalable so that we can run our business.

What would you have to change if Section 230 were eliminated or significantly narrowed? 

Honestly, there's a high chance that it would bankrupt platforms like ours. They're not wildly profitable. If Section 230 is done away with, we have to [check] content that goes online to make sure we’re not liable. That means policing tens of millions of uploads per day. 

That would kill the business of a lot of photographers. Can you imagine—you just got married, and you’re waiting for your wedding photos for a week or two because they’re in some moderation queue? 

If we don’t have legal protections, and we get one nefarious customer—if something goes sideways—then I’m liable for that. 

I don't, and can't possibly know, whether every single photo is appropriate or legal, as it's uploaded. We would literally have to moderate everything before it goes online. I don’t think any business can afford that, period. I guess you could have an offshore call-center type thing. Still, it would change the entire nature of the real-time internet. Imagine posting something to Instagram and having the platform say, “Cool, we’ll get back to you in 8 to 12 days.” 

What kind of content moderation do you do on SmugMug? 

If a user uploads something illegal, we will report them as soon as we find it. We're not protecting them. We don’t condone or allow illegal behavior. We work very closely with organizations, nonprofits and governmental agencies to detect CSAM—child exploitative material—and we report that to the National Center for Missing and Exploited Children. We will report users, we eliminate illegal content on our platforms—which is one reason we have such a low prevalence of that problem. 

But that does take effort and time to find, and there is currently no perfect solution. The tech solutions that exist can’t detect it at 100% accuracy, or anywhere close. And with tens of millions of uploads a day, going through them one by one is impossible. 

How do you think more generally about protecting user speech and creative expression? 

On SmugMug, we’re really focusing on professionals running their business. So we don’t have to [weigh in] on content too much. 

On Flickr, we are big proponents of expression and artistic creativity. Photographers have opinions! But we do draw the line at things like hate speech and harassment. We aggressively maintain a friendly platform. Our community guidelines are very specific, that you cannot harass other customers, you cannot upload stuff classified as hate speech, or threats, or anything along those lines. 

Those rules are generally policed by the community. We do have some text analysis tools, but when community members feel harassed or threatened, reports will come in. We’ll address them on a one-by-one basis and remove harassing material from our platform. 

Our ability to moderate is one of the things that makes Flickr what it is. If we lose the ability to enforce our own moderation rules—or have that legislated for us—then it changes the entire nature of the community. And not in a good way. Losing the ability to moderate would permanently and forever change what we've built.

What kind of complaints or takedown requests do you receive, and how do you handle it, both in the U.S. and abroad? 

Flickr is often referred to as the friendliest community online. You know, we're not dealing with a lot of hate. We're not dealing with a lot of threats. Under other frameworks, like the DMCA, we do takedowns on copyrighted material. 

We’re able to handle it with a fully internal team, and we have a great track record. But the user base and the content base is so large that, if we had to assume that those tens of millions of uploads a day are problematic, the burden would be extreme. 

We have a robust Trust and Safety Team, and we operate in every non-embargoed country on Earth. So we are subject to a lot of different laws and regulations: “likeness” rules and privacy rules in certain countries that don't exist here in the United States. Even state to state, there’s some varying laws. It’s a complicated framework, but we pay attention to it. 

The globe responds in much the same way that Section 230 is working. That is, we operate on reports and discovery, not on pre-screening everything. 

What do you think that policy makers most often misunderstand about how platforms like yours operate?

One misconception is that we are not beholden to any laws. That Section 230 absolves us of any responsibility and any liability, and we can just do whatever we want. They talk about it as “reining in tech companies,” or “holding tech companies accountable.” But I am accountable for the content on my platform. We’re not given this “get out of jail free” card. 

And I think they assume all platforms don’t really care about this, that anything that is done is done begrudgingly. But we’re very proactive about keeping a clean, polite, and friendly community. We are already very aggressively policing our platform. 

And even legal content gets moderated, because it might just not be appropriate for a particular community. 

We enforce our rules, and much the way that other private in-person businesses will enforce their rules. If you start screaming hateful things at patrons in a coffee shop, they’re going to throw you out. They want a quiet, chill vibe where people can sip their lattes. We’re doing the same sort of things. 

As an independent family owned company you’re in an ecosystem dominated by much larger platforms. How are these issues different for you as a smaller service? 

I think it's a much more existential threat for middle and small tech companies. It also shuts off the next generation of these platforms. The computer science student in a dorm room right now won't have the legal protections to launch, to even try to build something new. At least not here in the United States. 

EFF filed an amicus brief for the second time in the U.S. Court of Appeals for the Ninth Circuit, arguing that allowing cases against the Apple, Google, and Facebook app stores to proceed could lead to greater censorship of users’ online speech.

Our brief argues that the app stores should not lose Section 230 immunity for hosting “social casino” apps just because they process payments for virtual chips within those apps. Otherwise, all platforms that facilitate financial transactions for online content—beyond app stores and the apps and games they distribute—would be forced to censor user content to mitigate their legal exposure.

Social casino apps are online games where users can buy virtual chips with real money but can’t ever cash out their winnings. The three cases against Apple, Google, and Facebook were brought by plaintiffs who spent large sums of money on virtual chips and even became addicted to these games. The plaintiffs argue that social casino apps violate various state gambling laws.

At issue on appeal is the part of Section 230 that provides immunity to online platforms when they are sued for harmful content created by others—in this case, the social casino apps that plaintiffs downloaded from the various app stores and the virtual chips they bought within the apps.

Section 230 is the foundational law that has, since 1996, created legal breathing room for internet intermediaries (and their users) to publish third-party content. Online speech is largely mediated by these private companies, allowing all of us to speak, access information, and engage in commerce online, without requiring that we have loads of money or technical skills.

The lower court hearing the case ruled that the companies do not have Section 230 immunity because they allow the social casino apps to use the platforms’ payment processing services for the in-app purchasing of virtual chips.

However, in our brief we urged the Ninth Circuit to reverse the district court and hold that Section 230 does apply to the app stores, even when they process payments for virtual chips within the social casino apps. The app stores would undeniably have Section 230 immunity if sued for simply hosting the allegedly illegal social casino apps in their respective stores. Congress made no distinction—and the court shouldn’t recognize one—between hosting third-party content and processing payments for the same third-party content. Both are editorial choices of the platforms that are protected by Section 230.

We also argued that a rule that exposes internet intermediaries to potential liability for facilitating a financial transaction related to unlawful user content would have huge implications beyond the app stores. All platforms that facilitate financial transactions for third-party content would be forced to censor any user speech that may in any way risk legal exposure for the platform. This would harm the open internet—the unique ability of anyone with an internet connection to communicate with others around the world cheaply, easily, and quickly.

The plaintiffs argue that the app stores could preserve their Section 230 immunity by simply refusing to process in-app purchases of virtual chips. But the plaintiffs’ position fails to recognize that other platforms don’t have such a choice. Etsy, for example, facilitates purchases of virtual art, while Patreon enables artists to be supported by memberships. Platforms like these would lose Section 230 immunity and be exposed to potential liability simply because they processed payments for user content that a plaintiff argues is illegal. That outcome would threaten the entire business models of these services, ultimately harming users’ ability to share and access online speech.

The app stores should be protected by Section 230—a law that protects Americans’ freedom of expression online by protecting the intermediaries we all rely on—irrespective of their role as payment processors.

Section 230 helps make it possible for online communities to host user speech: from restaurant reviews, to fan fiction, to collaborative encyclopedias. But recent debates about the law often overlook how it works in practice. To mark its 30th anniversary, EFF is interviewing leaders of online platforms about how they handle complaints, moderate content, and protect their users’ ability to speak and share information. 

Reddit is one of the largest user-generated content platforms on the internet, built around thousands of independent communities known as subreddits. Some subreddits cover everyday interests, while others host discussions about specialized or controversial topics. These communities are created and moderated by volunteers, and the site’s decentralized model means that Reddit hosts a vast range of user speech without relying on centralized editorial control. 

Ben Lee is Chief Legal Officer at Reddit, where he oversees the company’s legal strategy and policy work on issues including content moderation and intermediary liability. Before joining Reddit, Lee held senior legal roles at other tech companies including Plaid, Twitter, and Google. At Reddit, he has been closely involved in litigation and policy debates surrounding Section 230, including cases addressing the legal risks faced by platforms and their users and moderators. He was interviewed by Joe Mullin, a policy analyst on EFF's Activism Team.

Joe Mullin: When we talk about user rights and Section 230, what rights are most at stake on a platform like Reddit? 

Ben Lee: Reddit, we often say, is the most human place on the internet. What’s often missing from the debate is that section 230 protects people—not platforms. 

It protects millions of everyday humans and volunteer moderators who participate in online communities. Without it, people could face lawsuits for voting down a post, enforcing community rules, or moderating a discussion. These are foundational activities on Reddit, and frankly, the whole internet.

If you had to describe section 230 to a regular Reddit user without naming the law, what would you say it does for them?

Section 230 protects your ability to participate in community moderation.

Even if all you are doing is up-voting or down-voting content, that’s participation. On Reddit, everyone is a content moderator, through voting. Up-voting determines the visibility of content. 

We believe, strongly, this is one of the only models to allow Reddit to scale. You make the community part of the moderation process. They’re invested in the community, making it better. 

How would user speech be affected if Section 230 were eliminated or weakened? 

We would undermine community self governance—the notion that humans can do content moderation, and take that responsibility for themselves. Whether you’re a small blog or big forum. I like to think of Reddit as composed of this federation of communities that range from the tiny to the humongous. That’s what the internet is! 

The legal risk would discourage people from moderating, or even speaking at all. The kind of speech we’re trying to protect is often critical of powerful people or entities. If a moderation decision leads to litigation from those powerful entities, that’s an expensive proposition to fight. 

Reddit relies on user-run communities and volunteer moderators. Can you walk me through how content moderation and legal complaints actually work in practice, and where section 230 comes into that? 

We have a tiered structure, like our federal system. Each community is like a state: it has its own rules, and enforces them. The vast majority of content moderation decisions are made by the communities, not by Reddit itself. 

Reddit is built on self-governing communities that are moderated by volunteers, supported by automated tools. Section 230 gives Reddit the freedom to experiment, and lets users shape healthy, interest-based spaces.

Section 230 is fundamental to protecting the moderators from a frivolous lawsuit. A screenwriting community might want to protect their community from scammy competitions—and then they get sued by that competition. 

Or a community wants to keep their conversation civil. And, for example, may not allow Star Trek characters to be called “soy boys,” and they enforce that. Then a person sues. 

I wish these were hypotheticals. But they were actual lawsuits. And we have them, routinely. 

What are policymakers missing about Section 230? 

The [moderation] decisions being criticized in court, are decisions to try to make the internet safer. In none of the cases that I mentioned is there a moderator saying, “I want to increase harmful content!” These are good-faith decisions about what makes the internet better. 

Section 230 is, at its core, protecting the ability for people to make those choices for their own communities. 

There's a price to be paid for not having a Section 230. And it will be paid by internet users—not the biggest platforms.

Some see 230 as a way to punish Big Tech. But removing it doesn't punish Big Tech—it makes them more powerful. It's startups, community driven platforms, and individual moderators who rely on Section 230 to compete and innovate. Weakening Section 230 will harm the open internet, and reduce the choice, diversity, and resilience of the internet. 

The big guys, they have armies of lawyers. They have the budget to withstand a flood of lawsuits. Weakening Section 230 just entrenches them. 

In Reddit’s amicus brief in the Gonzalez v. Google Supreme Court case, you point out that without Section 230, many moderation decisions wouldn’t be protected. The brief states: “A plaintiff might claim emotional distress from a truthful but hurtful post that gained prominence when a moderator highlighted it as a trending topic. Or, a plaintiff might claim interference with economic relations arising from an honest but very critical two-star restaurant review.” 

When you have situations where moderators get threats or litigation, what can you do? 

We have had cases where our own moderators got sued, along with us. In the “soy boy” case, we worked to help find pro bono counsel for the moderators. 

Someone posted “Wesley Crusher is a soy boy,” and it got removed. I'm enough of a Star Trek fan that I understand both the reference, and why the moderator decided—“hey, it's gone. I don't want this here.”

This would not violate our Reddit rules. But the community took it down under its own rules about being civil. It was just not a kind-hearted action, and the community had a right to decide. 

But the moderator got sued. We got sued, actually, because the poster disagreed with that moderation choice. Section 230 is what allowed us to win that case. 

These are just average people, implicated only because they moderated their own community. They are trying to do the right thing by their community. 

In cases where litigation happens, when does Section 230 come into play? 

Section 230 is usually one of the first things that's talked about in the case. It’s usually the most effective way of saying: if you believe someone who defamed you—please go to the person who has defamed you. If you’re looking to the moderator, or to Reddit itself, this is not a great way of getting the justice that you seek. 

Is there a different workflow internationally? 

There’s a very different workflow. We had a prominent case in France where a company was trying to sue moderators, and of course, we didn't have section 230 to protect them. So we had to do all sorts of other things to protect them. It got much more complicated. 

The breadth of content that's considered illegal in certain jurisdictions can be somewhat breathtaking. 

Our goal is always to preserve as much freedom of expression as possible for our community. In the U.S., we look at it through the lens of the First Amendment, and other aspects. Outside the U.S., we rely more on the lens of international human rights. 

How would you characterize legal demands around user content, the ones you see most often? 

They tend to be: somebody said something mean about me—take this down. Or someone says: you didn’t allow me to say something mean about someone or some entity. It completely runs the spectrum. 

One law that has already passed that weakens Section 230 is SESTA/FOSTA. From Reddit’s perspective, what changed after that? 

There's some communities we had to shut down, in particular, support communities. There was a cost. Every time Section 230 is narrowed, there’s a cost—some types of speech and communities have a harder time staying online. 

The cost may not seem high to some people, because those communities are not for them. But if they visited them, they’d see that these are actual people, interacting in a positive way. If it wasn’t positive, we have rules for that—but that’s a different question. 

Section 230 helps make it possible for online communities to host user speech: from restaurant reviews, to fan fiction, to collaborative encyclopedias. But recent debates about the law often overlook how it works in practice. To mark its 30th anniversary, EFF is interviewing leaders of online platforms about how they handle complaints, moderate content, and protect their users’ ability to speak and share information.

Yelp hosts millions of reviews written by internet users about local businesses. Most reviews are positive, but over the years, some businesses have tried to pressure Yelp to remove negative reviews, including through legal threats. Since its founding more than two decades ago, Yelp has fought major legal battles to defend reviewers’ rights and preserve the legal protections that allow consumers to share honest feedback online.

Aaron Schur is General Counsel at Yelp. He joined the company in 2010 as one of its first lawyers and has led its litigation strategy for more than a decade, helping secure court decisions that strengthened legal protections for consumer speech. He was interviewed by Joe Mullin, a policy analyst on EFF's Activism Team. 

Joe Mullin: How would you describe Section 230 to a regular Yelp user who doesn’t know about the law?   

Aaron Schur: I'd say it is a simple rule that, generally speaking, when content is posted online, any liability for that content is with the person that created it, not the platform that is displaying it. That allows Yelp to show your review and keep it up if a business complains about it. It also means that we can develop ways to highlight the reviews we think are most helpful and reliable, and mitigate fake reviews in a way, without creating liability for Yelp, because we're allowed to host third party content.

The political debate around Section 230 often centers around the behavior of companies, especially large companies. But we rarely hear about users, even though the law also applies to users. What is the user story that is getting lost? 

Section 230 at heart protects users. It enables a diversity of platforms and content moderation practices—whether it's reviews on Yelp, videos on another platform, whatever it may be. 

Without Section 230, platforms would face heavy pressure to remove consumer speech when we’re threatened with legal action—and that harms users, directly. Their content gets removed. It also harms the greater number of users who would access that content. 

The focus on the biggest tech companies, I think, is understandable but misplaced when it comes to Section 230. We have tools that exist to go after dominant companies, both at the state and the federal level, and Congress could certainly consider competition-based laws—and has, over the last several years. 

Tell me about the editorial decisions that Yelp makes regarding the highlighting of reviews, and the weeding out of reviews that might be fake.  

Yelp is a platform where people share their experiences with local businesses, government agencies, and other entities. People come to Yelp, by the millions, to learn about these places.

With traffic like that come incentives for bad actors to game the system. Some unscrupulous businesses try to create fake reviews, or compensate people to write reviews, or ask family and friends to write reviews. Those reviews will be biased in a way that won’t be transparent. 

Yelp developed an automated system to highlight reviews we find most trustworthy and helpful. Other reviews may be placed in a “not recommended” section where they don’t affect a business’s overall rating, but they’re still visible. That helps us maintain a level playing field and keep user trust. 

Tell me about what your process around complaints around user reviews look like. 

We have a reporting function for reviews. Those reports get looked at by an actual human, who evaluates the review and looks at data about it to decide whether it violates our guidelines. 

We don't remove a review just because someone says it's “wrong,” because we can't litigate the facts in your review. If someone says “my pizza arrived cold,” and the restaurant says, no, the pizza was warm—Yelp is not in a position to adjudicate that dispute. 

That's where Section 230 comes in. It says Yelp doesn’t have to [decide who’s right]. 

What other types of moderation tools have you built? 

Any business, free of charge, can respond to a review, and that response appears directly below it. They can also message users privately. We know when businesses do this, it’s viewed positively by users.

We also have a consumer alert program, where members of the public can report businesses that may be compensating people for positive reviews—offering things like free desserts or discounted rent. In those cases, we can place an alert on the business’s page and link to the evidence we received. We also do this when businesses make certain types of legal threats against users.

It’s about transparency. If a business’s rating is inflated, because that business is threatening reviewers who rate less than five stars with a lawsuit, consumers have a right to know what’s happening. 

How are international complaints, where Section 230 doesn’t come into play, different? 

We have had a lot of matters in Europe, in particular in Germany. It’s a different system there—it’s notice-and-takedown. They have a line of cases that require review sites to basically provide proof that the person was a customer of the business. 

If a review was challenged, we would sometimes ask the user for documentation, like an invoice, which we would redact before providing it. Often, they would do that, in order to defend their own speech online. Which was surprising to me! But they wouldn’t always—which shows the benefit of Section 230. In the U.S., you don’t have this back-and-forth that a business can leverage to get content taken down. 

And invariably, the reviewer was a customer. The business was just using the system to try to take down speech. 

Yelp has been part of some of the most important legal cases around Section 230, and some of those didn’t exist when we spoke in 2012. What happened in the Hassel v. Bird case, and why was that important for online reviewers?

Hassel v. Bird was a case where a law firm got a default judgment against an alleged reviewer, and the court ordered Yelp to remove the review—even though Yelp had not been a party to the case. 

We refused, because the order violated Section 230, due process, and Yelp’s First Amendment rights as a publisher. But the trial court and the appeal court both ruled against us, allowing a side-stepping of Section 230. 

The California Supreme Court ultimately reversed those rulings, and recognized that plaintiffs cannot accomplish indirectly [by suing a user and then ordering a platform to remove content] what they could not accomplish directly by suing the platform itself.

We spoke to you in 2012, and the landscape has really changed. Section 230 is really under attack in a way that it wasn’t back then. From your vantage point at Yelp, what feels different about this moment? 

The biggest tech companies got even bigger, and even more powerful. That has made people distrustful and angry—rightfully so, in many cases. 

When you read about the attacks on 230, it’s really politicians calling out Big Tech. But what is never mentioned is little tech, or “middle tech,” which is how Yelp bills itself. If 230 is weakened or repealed, it’s really the biggest companies, the Googles of the world, that will be able to weather it better than smaller companies like Yelp. They have more financial resources. It won’t actually accomplish what the legislators are setting out to accomplish. It will have unintended consequences across the board. Not just for Yelp, but for smaller platforms. 

This interview was edited for length and clarity.

For thirty years, internet users have benefited from a key federal law that allows everyone to express themselves, find community, organize politically, and participate in society. Section 230, which protects internet users’ speech by protecting the online intermediaries we rely on, is the legal support that sustains the internet as we know it.

Yet as Section 230 turns 30 this week, there are bipartisan proposals in Congress to either repeal or sunset the law. These proposals seize upon legitimate concerns with the harmful and anti-competitive practices of the largest tech companies, but then misdirect that anger toward Section 230.

But rolling back or eliminating Section 230 will not stop invasive corporate surveillance that harms all internet users. Killing Section 230 won’t end to the dominance of the current handful of large tech companies—it would cement their monopoly power. 

The current proposals also ignore a crucial question: what legal standard should replace Section 230? The bills provide no answer, refusing to grapple with the tradeoffs inherent in making online intermediaries liable for users’ speech.

This glaring omission shows what these proposals really are: grievances masquerading as legislation, not serious policy. Especially when the speech problems with alternatives to Section 230’s immunity are readily apparent, both in the U.S. and around the world. Experience shows that those systems result in more censorship of internet users’ lawful speech.

Let’s be clear: EFF defends Section 230 because it is the best available system to protect users’ speech online. By immunizing intermediaries for their users’ speech, Section 230 benefits users. Services can distribute our speech without filters, pre-clearance, or the threat of dubious takedown requests. Section 230 also directly protects internet users when they distribute other people’s speech online, such as when they reshare another users’ post or host a comment section on their blog.

It was the danger of losing the internet as a forum for diverse political discourse and culture that led to the law in 1996. Congress created Section 230’s limited civil immunity  because it recognized that promoting more user speech outweighed potential harms. Congress decided that when harmful speech occurs, it’s the speaker that should be held responsible—not the service that hosts the speech. The law also protects social platforms when they remove posts that are obscene or violate the services’ own standards. And Section 230 has limits: it does not immunize services if they violate federal criminal laws.

Section 230 Alternatives Would Protect Less Speech

With so much debate around the downsides of Section 230, it’s worth considering: What are some of the alternatives to immunity, and how would they shape the internet?

The least protective legal regime for online speech would be strict liability. Here, intermediaries always would be liable for their users’ speech—regardless of whether they contributed to the harm, or even knew about the harmful speech. It would likely end the widespread availability and openness of social media and web hosting services we’re used to. Instead, services would not let users speak without vetting the content first, via upload filters or other means. Small intermediaries with niche communities may simply disappear under the weight of such heavy liability.

Another alternative: Imposing legal duties on intermediaries, such as requiring that they act “reasonably” to limit harmful user content. This would likely result in platforms monitoring users’ speech before distributing it, and being extremely cautious about what they allow users to say. That inevitably would lead to the removal of lawful speech—probably on a large scale. Intermediaries would not be willing to defend their users’ speech in court, even it is entirely lawful. In a world where any service could be easily sued over user speech, only the biggest services will survive. They’re the ones that would have the legal and technical resources to weather the flood of lawsuits.

Another option is a notice-and-takedown regime, like what exists under the Digital Millennium Copyright Act. That will also result in takedowns of legitimate speech. And there’s no doubt such a system will be abused. EFF has documented how the DMCA leads to widespread removal  https://www.eff.org/takedownsof lawful speech based on frivolous copyright infringement claims. Replacing Section 230 with a takedown system will invite similar behavior, and powerful figures and government officials will use it to silence their critics.

The closest alternative to Section 230’s immunity provides protections from liability until an impartial court has issued a full and final ruling that user-generated content is illegal, and ordered that it be removed. These systems ensure that intermediaries will not have to cave to frivolous claims. But they still leave open the potential for censorship because intermediaries are unlikely to fight every lawsuit that seeks to remove lawful speech. The cost of vindicating lawful speech in court may be too high for intermediaries to handle at scale.

By contrast, immunity takes the variable of whether an intermediary will stand up for their users’ speech out of the equation. That is why Section 230 maximizes the ability for users to speak online.

In some narrow situations, Section 230 may leave victims without a legal remedy. Proposals aimed at those gaps should be considered, though lawmakers should pay careful attention that in vindicating victims, they do not broadly censor users’ speech. But those legitimate concerns are not the criticisms that Congress is levying against Section 230.

EFF will continue to fight for Section 230, as it remains the best available system to protect everyone’s ability to speak online.

(This appeared as an op-ed published Friday, Feb. 6 in the Daily Journal, a California legal newspaper.)

Section 230, “the 26 words that created the internet,” was enacted 30 years ago this week. It was no rush-job—rather, it was the result of wise legislative deliberation and foresight, and it remains the best bulwark to protect free expression online.

The internet lets people everywhere connect, share ideas and advocate for change without needing immense resources or technical expertise. Our unprecedented ability to communicate online—on blogs, social media platforms, and educational and cultural platforms like Wikipedia and the Internet Archive—is not an accident. In writing Section 230, Congress recognized that for free expression to thrive on the internet, it had to protect the services that power users’ speech. Section 230 does this by preventing most civil suits against online services that are based on what users say. The law also protects users who act like intermediaries when they, for example, forward an email, retweet another user or host a comment section on their blog.

The merits of immunity, both for internet users who rely on intermediaries—from ISPs to email providers to social media platforms, and for internet users who are intermediaries—are readily apparent when compared with the alternatives.

One alternative would be to provide no protection at all for intermediaries, leaving them liable for anything and everything anyone says using their service. This legal risk would essentially require every intermediary to review and legally assess every word, sound or image before it’s published—an impossibility at scale, and a death knell for real-time user-generated content.

Another option: giving protection to intermediaries only if they exercise a specified duty of care, such as where an intermediary would be liable if they fail to act reasonably in publishing a user’s post. But negligence and other objective standards are almost always insufficient to protect freedom of expression because they introduce significant uncertainty into the process and create real chilling effects for intermediaries. That is, intermediaries will choose not to publish anything remotely provocative—even if it’s clearly protected speech—for fear of having to defend themselves in court, even if they are likely to ultimately prevail. Many Section 230 critics bemoan the fact that it prevented courts from developing a common law duty of care for online intermediaries. But the criticism rarely acknowledges the experience of common law courts around the world, few of which adopted an objective standard, and many of which adopted immunity or something very close to it.

Congress’ purposeful choice of Section 230’s immunity is the best way to preserve the ability of millions of people in the U.S. to publish their thoughts, photos and jokes online, to blog and vlog, post, and send emails and messages.

Another alternative is a knowledge-based system in which an intermediary is liable only after being notified of the presence of harmful content and failing to remove it within a certain amount of time. This notice-and-takedown system invites tremendous abuse, as seen under the Digital Millennium Copyright Act’s approach: It’s too easy for someone to notify an intermediary that content is illegal or tortious simply to get something they dislike depublished. Rather than spending the time and money required to adequately review such claims, intermediaries would simply take the content down.

All these alternatives would lead to massive depublication in many, if not most, cases, not because the content deserves to be taken down, nor because the intermediaries want to do so, but because it’s not worth assessing the risk of liability or defending the user’s speech. No intermediary can be expected to champion someone else’s free speech at its own considerable expense.Nor is the United States the only government to eschew “upload filtering,” the requirement that someone must review content before publication. European Union rules avoid this also, recognizing how costly and burdensome it is. Free societies recognize that this kind of pre-publication review will lead risk-averse platforms to nix anything that anyone anywhere could deem controversial, leading us to the most vanilla, anodyne internet imaginable.

The advent of artificial intelligence doesn’t change this. Perhaps there’s a tool that can detect a specific word or image, but no AI can make legal determinations or be prompted to identify all defamation or harassment. Human expression is simply too contextual for AI to vet; even if a mechanism could flag things for human review, the scale is so massive that such human review would still be overwhelmingly burdensome.

Congress’ purposeful choice of Section 230’s immunity is the best way to preserve the ability of millions of people in the U.S. to publish their thoughts, photos and jokes online, to blog and vlog, post, and send emails and messages. Each of those acts requires numerous layers of online services, all of which face potential liability without immunity.

This law isn’t a shield for “big tech.” Its ultimate beneficiaries are all of us who want to post things online without having to code it ourselves, and so that we can read and watch content that others create. If Congress eliminated Section 230 immunity, for example, we would be asking email providers and messaging platforms to read and legally assess everything a user writes before agreeing to send it. 

For many critics of Section 230, the chilling effect is the point: They want a system that will discourage online services to publish protected speech that some find undesirable. They want platforms to publish less than what they would otherwise choose to publish, even when that speech is protected and nonactionable.

When Section 230 was passed in 1996, about 40 million people used the internet worldwide; by 2025, estimates ranged from five billion to north of six billion. In 1996, there were fewer than 300,000 websites; by last year, estimates ranged up to 1.3 billion. There is no workforce and no technology that can police the enormity of everything that everyone says.

Internet intermediaries—whether social media platforms, email providers or users themselves—are protected by Section 230 so that speech can flourish online.

Lawmakers in Washington are once again focusing on kids, screens, and mental health. But according to Congress, Big Tech is somehow both the problem and the solution. The Senate Commerce Committee held a hearing today on “examining the effect of technology on America’s youth.” Witnesses warned about “addictive” online content, mental health, and kids spending too much time buried in screen. At the center of the debate is a bill from Sens. Ted Cruz (R-TX) and Brian Schatz (D-HI) called the Kids Off Social Media Act (KOSMA), which they say will protect children and “empower parents.” 

That’s a reasonable goal, especially at a time when many parents feel overwhelmed and nervous about how much time their kids spend on screens. But while the bill’s press release contains soothing language, KOSMA doesn’t actually give parents more control. 

Instead of respecting how most parents guide their kids towards healthy and educational content, KOSMA hands the control panel to Big Tech. That’s right—this bill would take power away from parents, and hand it over to the companies that lawmakers say are the problem.  

Kids Under 13 Are Already Banned From Social Media

One of the main promises of KOSMA is simple and dramatic: it would ban kids under 13 from social media. Based on the language of bill sponsors, one might think that’s a big change, and that today’s rules let kids wander freely into social media sites. But that’s not the case.   

Every major platform already draws the same line: kids under 13 cannot have an account. Facebook, Instagram, TikTok, X, YouTube, Snapchat, Discord, Spotify, and even blogging platforms like WordPress all say essentially the same thing—if you’re under 13, you’re not allowed. That age line has been there for many years, mostly because of how online services comply with a federal privacy law called COPPA. 

Of course, everyone knows many kids under 13 are on these sites anyways. The real question is how and why they get access. 

Most Social Media Use By Younger Kids Is Family-Mediated 

If lawmakers picture under-13 social media use as a bunch of kids lying about their age and sneaking onto apps behind their parents’ backs, they’ve got it wrong. Serious studies that have looked at this all find the opposite: most under-13 use is out in the open, with parents’ knowledge, and often with their direct help. 

A large national study published last year in Academic Pediatrics found that 63.8% of under-13s have a social media account, but only 5.4% of them said they were keeping one secret from their parents. That means roughly 90% of kids under 13 who are on social media aren’t hiding it at all. Their parents know. (For kids aged thirteen and over, the “secret account” number is almost as low, at 6.9%.) 

Earlier research in the U.S. found the same pattern. In a well-known study of Facebook use by 10-to-14-year-olds, researchers found that about 70% of parents said they actually helped create their child’s account, and between 82% and 95% knew the account existed. Again, this wasn’t kids sneaking around. It was families making a decision together.

A 2022 study by the UK’s media regulator Ofcom points in the same direction, finding that up to two-thirds of social media users below the age of thirteen had direct help from a parent or guardian getting onto the platform. 

The typical under-13 social media user is not a sneaky kid. It’s a family making a decision together. 

KOSMA Forces Platforms To Override Families 

This bill doesn’t just set an age rule. It creates a legal duty for platforms to police families.

Section 103(b) of the bill is blunt: if a platform knows a user is under 13, it “shall terminate any existing account or profile” belonging to that user. And “knows” doesn’t just mean someone admits their age. The bill defines knowledge to include what is “fairly implied on the basis of objective circumstances”—in other words, what a reasonable person would conclude from how the account is being used. The reality of how services would comply with KOSMA is clear: rather than risk liability for how they should have known a user was under 13, they will require all users to prove their age to ensure that they block anyone under 13. 

KOSMA contains no exceptions for parental consent, for family accounts, or for educational or supervised use. The vast majority of people policed by this bill won’t be kids sneaking around—it will be minors who are following their parents’ guidance, and the parents themselves. 

Imagine a child using their parent’s YouTube account to watch science videos about how a volcano works. If they were to leave a comment saying, “Cool video—I’ll show this to my 6th grade teacher!” and YouTube becomes aware of the comment, the platform now has clear signals that a child is using that account. It doesn’t matter whether the parent gave permission. Under KOSMA, the company is legally required to act. To avoid violating KOSMA, it would likely  lock, suspend, or terminate the account, or demand proof it belongs to an adult. That proof would likely mean asking for a scan of a government ID, biometric data, or some other form of intrusive verification, all to keep what is essentially a “family” account from being shut down.

Violations of KOSMA are enforced by the FTC and state attorneys general. That’s more than enough legal risk to make platforms err on the side of cutting people off.

Platforms have no way to remove “just the kid” from a shared account. Their tools are blunt: freeze it, verify it, or delete it. Which means that even when a parent has explicitly approved and supervised their child’s use, KOSMA forces Big Tech to override that family decision.

Your Family, Their Algorithms

KOSMA doesn’t appoint a neutral referee. Under the law, companies like Google (YouTube), Meta (Facebook and Instagram), TikTok, Spotify, X, and Discord will become the ones who decide whose account survives, whose account gets locked, who has to upload ID, and whose family loses access altogether. They won’t be doing this because they want to—but because Congress is threatening them with legal liability if they don’t. 

These companies don’t know your family or your rules. They only know what their algorithms infer. Under KOSMA, those inferences carry the force of law. Rather than parents or teachers, decisions about who can be online, and for what purpose, will be made by corporate compliance teams and automated detection systems. 

What Families Lose 

This debate isn’t really about TikTok trends or doomscrolling. It’s about all the ordinary, boring, parent-guided uses of the modern internet. It’s about a kid watching “How volcanoes work” on regular YouTube, instead of the stripped-down YouTube Kids. It’s about using a shared Spotify account to listen to music a parent already approves. It’s about piano lessons from a teacher who makes her living from YouTube ads.

These aren’t loopholes. They’re how parenting works in the digital age. Parents increasingly filter, supervise, and, usually, decide together with their kids. KOSMA will lead to more locked accounts, and more parents submitting to face scans and ID checks. It will also lead to more power concentrated in the hands of the companies Congress claims to distrust. 

What Can Be Done Instead

KOSMA also includes separate restrictions on how platforms can use algorithms for users aged 13 to 17. Those raise their own serious questions about speech, privacy, and how online services work, and need debate and scrutiny as well. But they don’t change the core problem here: this bill hands control over children’s online lives to Big Tech.

If Congress really wants to help families, it should start with something much simpler and much more effective: strong privacy protections for everyone. Limits on data collection, restrictions on behavioral tracking, and rules that apply to adults as well as kids would do far more to reduce harmful incentives than deputizing companies to guess how old your child is and shut them out.

But if lawmakers aren’t ready to do that, they should at least drop KOSMA and start over. A law that treats ordinary parenting as a compliance problem is not protecting families—it’s undermining them.

Parents don’t need Big Tech to replace them. They need laws that respect how families actually work.