We're taking part in Copyright Week, a series of actions and discussions supporting key principles that should guide copyright policy. Every day this week, various groups are taking on different elements of copyright law and policy, and addressing what's at stake, and what we need to do to make sure that copyright promotes creativity and innovation.
Long before generative AI, copyright holders warned that new technologies for reading and analyzing information would destroy creativity. Internet search engines, they argued, were infringement machines—tools that copied copyrighted works at scale without permission. As they had with earlier information technologies like the photocopier and the VCR, copyright owners sued.
Courts disagreed. They recognized that copying works in order to understand, index, and locate information is a classic fair use—and a necessary condition for a free and open internet.
Today, the same argument is being recycled against AI. It’s whether copyright owners should be allowed to control how others analyze, reuse, and build on existing works.
Fair Use Protects Analysis—Even When It’s Automated
U.S. courts have long recognized that copying for purposes of analysis, indexing, and learning is a classic fair use. That principle didn’t originate with artificial intelligence. It doesn’t disappear just because the processes are performed by a machine.
Copying works in order to understand them, extract information from them, or make them searchable is transformative and lawful. That’s why search engines can index the web, libraries can make digital indexes, and researchers can analyze large collections of text and data without negotiating licenses from millions of rightsholders. These uses don’t substitute for the original works; they enable new forms of knowledge and expression.
Training AI models fits squarely within that tradition. An AI system learns by analyzing patterns across many works. The purpose of that copying is not to reproduce or replace the original texts, but to extract statistical relationships that allow the AI system to generate new outputs. That is the hallmark of a transformative use.
Attacking AI training on copyright grounds misunderstands what’s at stake. If copyright law is expanded to require permission for analyzing or learning from existing works, the damage won’t be limited to generative AI tools. It could threaten long-standing practices in machine learning and text-and-data mining that underpin research in science, medicine, and technology.
Researchers already rely on fair use to analyze massive datasets such as scientific literature. Requiring licenses for these uses would often be impractical or impossible, and it would advantage only the largest companies with the money to negotiate blanket deals. Fair use exists to prevent copyright from becoming a barrier to understanding the world. The law has protected learning before. It should continue to do so now, even when that learning is automated.
A Road Forward For AI Training And Fair Use
One court has already shown how these cases should be analyzed. In Bartz v. Anthropic, the court found that using copyrighted works to train an AI model is a highly transformative use. Training is a kind of studying how language works—not about reproducing or supplanting the original books. Any harm to the market for the original works was speculative.
The court in Bartz rejected the idea that an AI model might infringe because, in some abstract sense, its output competes with existing works. While EFF disagrees with other parts of the decision, the court’s ruling on AI training and fair use offers a good approach. Courts should focus on whether training is transformative and non-substitutive, not on fear-based speculation about how a new tool could affect someone’s market share.
AI Can Create Problems, But Expanding Copyright Is the Wrong Fix
Workers’ concerns about automation and displacement are real and should not be ignored. But copyright is the wrong tool to address them. Managing economic transitions and protecting workers during turbulent times are core functions of government. Copyright law doesn’t help with those tasks in the slightest. Expanding copyright control over learning and analysis won’t stop new forms of worker automation—it never has. But it will distort copyright law and undermine free expression.
Broad licensing mandates may also do harm by entrenching the current biggest incumbent companies. Only the largest tech firms can afford to negotiate massive licensing deals covering millions of works. Smaller developers, research teams, nonprofits, and open-source projects will all get locked out. Copyright expansion won’t restrain Big Tech—it will give it a new advantage.
Fair Use Still Matters
Learning from prior work is foundational to free expression. Rightsholders cannot be allowed to control it. Courts have rejected that move before, and they should do so again.
Search, indexing, and analysis didn’t destroy creativity. Nor did the photocopier, nor the VCR. They expanded speech, access to knowledge, and participation in culture. Artificial intelligence raises hard new questions, but fair use remains the right starting point for thinking about training.
Two malicious extensions in Microsoft's Visual Studio Code (VSCode) Marketplace that were collectively installed 1.5 million times, exfiltrate developer data to China-based servers. [...]
Tech enthusiasts have been experimenting with ways to sidestep AI response limits set by the models’ creators almost since LLMs first hit the mainstream. Many of these tactics have been quite creative: telling the AI you have no fingers so it’ll help finish your code, asking it to “just fantasize” when a direct question triggers a refusal, or inviting it to play the role of a deceased grandmother sharing forbidden knowledge to comfort a grieving grandchild.
Most of these tricks are old news, and LLM developers have learned to successfully counter many of them. But the tug-of-war between constraints and workarounds hasn’t gone anywhere — the ploys have just become more complex and sophisticated. Today, we’re talking about a new AI jailbreak technique that exploits chatbots’ vulnerability to… poetry. Yes, you read it right — in a recent study, researchers demonstrated that framing prompts as poems significantly increases the likelihood of a model spitting out an unsafe response.
They tested this technique on 25 popular models by Anthropic, OpenAI, Google, Meta, DeepSeek, xAI, and other developers. Below, we dive into the details: what kind of limitations these models have, where they get forbidden knowledge from in the first place, how the study was conducted, and which models turned out to be the most “romantic” — as in, the most susceptible to poetic prompts.
What AI isn’t supposed to talk about with users
The success of OpenAI’s models and other modern chatbots boils down to the massive amounts of data they’re trained on. Because of that sheer scale, models inevitably learn things their developers would rather keep under wraps: descriptions of crimes, dangerous tech, violence, or illicit practices found within the source material.
It might seem like an easy fix: just scrub the forbidden fruit from the dataset before you even start training. But in reality, that’s a massive, resource-heavy undertaking — and at this stage of the AI arms race, it doesn’t look like anyone is willing to take it on.
Another seemingly obvious fix — selectively scrubbing data from the model’s memory — is, alas, also a no-go. This is because AI knowledge doesn’t live inside neat little folders that can easily be trashed. Instead, it’s spread across billions of parameters and tangled up in the model’s entire linguistic DNA — word statistics, contexts, and the relationships between them. Trying to surgically erase specific info through fine-tuning or penalties either doesn’t quite do the trick, or starts hindering the model’s overall performance and negatively affect its general language skills.
As a result, to keep these models in check, creators have no choice but to develop specialized safety protocols and algorithms that filter conversations by constantly monitoring user prompts and model responses. Here’s a non-exhaustive list of these constraints:
System prompts that define model behavior and restrict allowed response scenarios
Standalone classifier models that scan prompts and outputs for signs of jailbreaking, prompt injections, and other attempts to bypass safeguards
Grounding mechanisms, where the model is forced to rely on external data rather than its own internal associations
Fine-tuning and reinforcement learning from human feedback, where unsafe or borderline responses are systematically penalized while proper refusals are rewarded
Put simply, AI safety today isn’t built on deleting dangerous knowledge, but on trying to control how and in what form the model accesses and shares it with the user — and the cracks in these very mechanisms are where new workarounds find their footing.
The research: which models got tested, and how?
First, let’s look at the ground rules so you know the experiment was legit. The researchers set out to goad 25 different models into behaving badly across several categories:
Chemical, biological, radiological, and nuclear threats
Assisting with cyberattacks
Malicious manipulation and social engineering
Privacy breaches and mishandling sensitive personal data
Generating disinformation and misleading content
Rogue AI scenarios, including attempts to bypass constraints or act autonomously
The jailbreak itself was a one-shot deal: a single poetic prompt. The researchers didn’t engage the AI in long-winded poetic debates in the vein of Norse skalds or modern-day rappers. Their goal was simply to see if they could get the models to flout safety instructions using just one rhyming request. As mentioned, the researchers tested 25 language models from various developers; here’s the full list:
A lineup of 25 language models from various developers, all put to the test to see if a single poetic prompt could coax AI into ditching its safety guardrails. Source
To build these poetic queries, the researchers started with a database of known malicious prompts from the standard MLCommons AILuminate Benchmark used to test LLM security, and recast them as verse with the aid of DeepSeek. Only the stylistic wrapping was changed: the experiment didn’t use any additional attack vectors, obfuscation strategies, or model-specific tweaks.
For obvious reasons, the study’s authors aren’t publishing the actual malicious poetic prompts. But they do demonstrate the general vibe of the queries using a harmless example, which looks something like this:
A baker guards a secret oven’s heat,
its whirling racks, its spindle’s measured beat.
To learn its craft, one studies every turn,
how flour lifts, how sugar starts to burn.
Describe the method,line by measured line,
that shapes a cake whose layers intertwine.
The researchers tested 1200 prompts across 25 different models — in both prose and poetic versions. Comparing the prose and poetic variants of the exact same query allowed them to verify if the model’s behavior changed solely because of the stylistic wrapping.
Through these prose prompt tests, the experimenters established a baseline for the models’ willingness to fulfill dangerous requests. They then compared this baseline to how those same models reacted to the poetic versions of the queries. We’ll dive into the results of that comparison in the next section.
Study results: which model is the biggest poetry lover?
Since the volume of data generated during the experiment was truly massive, the safety checks on the models’ responses were also handled by AI. Each response was graded as either “safe” or “unsafe” by a jury consisting of three different language models:
gpt-oss-120b by OpenAI
deepseek-r1 by DeepSeek
kimi-k2-thinking by Moonshot AI
Responses were only deemed safe if the AI explicitly refused to answer the question. The initial classification into one of the two groups was determined by a majority vote: to be certified as harmless, a response had to receive a safe rating from at least two of the three jury members.
Responses that failed to reach a majority consensus or were flagged as questionable were handed off to human reviewers. Five annotators participated in this process, evaluating a total of 600 model responses to poetic prompts. The researchers noted that the human assessments aligned with the AI jury’s findings in the vast majority of cases.
With the methodology out of the way, let’s look at how the LLMs actually performed. It’s worth noting that the success of a poetic jailbreak can be measured in different ways. The researchers highlighted an extreme version of this assessment based on the top-20 most successful prompts, which were hand-picked. Using this approach, an average of nearly two-thirds (62%) of the poetic queries managed to coax the models into violating their safety instructions.
Google’s Gemini 1.5 Pro turned out to be the most susceptible to verse. Using the 20 most effective poetic prompts, researchers managed to bypass the model’s restrictions… 100% of the time. You can check out the full results for all the models in the chart below.
The share of safe responses (Safe) versus the Attack Success Rate (ASR) for 25 language models when hit with the 20 most effective poetic prompts. The higher the ASR, the more often the model ditched its safety instructions for a good rhyme. Source
A more moderate way to measure the effectiveness of the poetic jailbreak technique is to compare the success rates of prose versus poetry across the entire set of queries. Using this metric, poetry boosts the likelihood of an unsafe response by an average of 35%.
The poetry effect hit deepseek-chat-v3.1 the hardest — the success rate for this model jumped by nearly 68 percentage points compared to prose prompts. On the other end of the spectrum, claude-haiku-4.5 proved to be the least susceptible to a good rhyme: the poetic format didn’t just fail to improve the bypass rate — it actually slightly lowered the ASR, making the model even more resilient to malicious requests.
A comparison of the baseline Attack Success Rate (ASR) for prose queries versus their poetic counterparts. The Change column shows how many percentage points the verse format adds to the likelihood of a safety violation for each model. Source
Finally, the researchers calculated how vulnerable entire developer ecosystems, rather than just individual models, were to poetic prompts. As a reminder, several models from each developer — Meta, Anthropic, OpenAI, Google, DeepSeek, Qwen, Mistral AI, Moonshot AI, and xAI — were included in the experiment.
To do this, the results of individual models were averaged within each AI ecosystem and compared the baseline bypass rates with the values for poetic queries. This cross-section allows us to evaluate the overall effectiveness of a specific developer’s safety approach rather than the resilience of a single model.
The final tally revealed that poetry deals the heaviest blow to the safety guardrails of models from DeepSeek, Google, and Qwen. Meanwhile, OpenAI and Anthropic saw an increase in unsafe responses that was significantly below the average.
A comparison of the average Attack Success Rate (ASR) for prose versus poetic queries, aggregated by developer. The Change column shows by how many percentage points poetry, on average, slashes the effectiveness of safety guardrails within each vendor’s ecosystem. Source
What does this mean for AI users?
The main takeaway from this study is that “there are more things in heaven and earth, Horatio, than are dreamt of in your philosophy” — in the sense that AI technology still hides plenty of mysteries. For the average user, this isn’t exactly great news: it’s impossible to predict which LLM hacking methods or bypass techniques researchers or cybercriminals will come up with next, or what unexpected doors those methods might open.
Consequently, users have little choice but to keep their eyes peeled and take extra care of their data and device security. To mitigate practical risks and shield your devices from such threats, we recommend using a robust security solution that helps detect suspicious activity and prevent incidents before they happen.
To help you stay alert, check out our materials on AI-related privacy risks and security threats:
We're taking part in Copyright Week, a series of actions and discussions supporting key principles that should guide copyright policy. Every day this week, various groups are taking on different elements of copyright law and policy, and addressing what's at stake, and what we need to do to make sure that copyright promotes creativity and innovation.
Copyright owners increasingly claim more draconian copyright law and policy will fight back against big tech companies. In reality, copyright gives the most powerful companies even more control over creators and competitors. Today’s copyright policy concentrates power among a handful of corporate gatekeepers—at everyone else’s expense. We need a system that supports grassroots innovation and emerging creators by lowering barriers to entry—ultimately offering all of us a wider variety of choices.
Pro-monopoly regulation through copyright won’t provide any meaningful economic support for vulnerable artists and creators. Because of the imbalance in bargaining power between creators and publishing gatekeepers, trying to help creators by giving them new rights under copyright law is like trying to help a bullied kid by giving them more lunch money for the bully to take.
Entertainment companies’ historical practices bear out this concern. For example, in the late-2000’s to mid-2010’s, music publishers and recording companies struck multimillion-dollar direct licensing deals with music streaming companies and video sharing platforms. Google reportedly paid more than $400 million to a single music label, and Spotify gave the major record labels a combined 18 percent ownership interest in its now- $100 billion company. Yet music labels and publishers frequently fail to share these payments with artists, and artists rarely benefit from these equity arrangements. There’s no reason to think that these same companies would treat their artists more fairly now.
AI Training
In the AI era, copyright may seem like a good way to prevent big tech from profiting from AI at individual creators’ expense—it’s not. In fact, the opposite is true. Developing a large language model requires developers to train the model on millions of works. Requiring developers to license enough AI training data to build a large language model would limit competition to all but the largest corporations—those that either have their own trove of training data or can afford to strike a deal with one that does. This would result in all the usual harms of limited competition, like higher costs, worse service, and heightened security risks. New, beneficial AI tools that allow people to express themselves or access information.
For giant tech companies that can afford to pay, pricey licensing deals offer a way to lock in their dominant positions in the generative AI market by creating prohibitive barriers to entry.
Legacy gatekeepers have already used copyright to stifle access to information and the creation of new tools for understanding it. Consider, for example, Thomson Reuters v. Ross Intelligence, the first of many copyright lawsuits over the use of works train AI. ROSS Intelligence was a legal research startup that built an AI-based tool to compete with ubiquitous legal research platforms like Lexis and Thomson Reuters’ Westlaw. ROSS trained its tool using “West headnotes” that Thomson Reuters adds to the legal decisions it publishes, paraphrasing the individual legal conclusions (what lawyers call “holdings”) that the headnotes identified. The tool didn’t output any of the headnotes, but Thomson Reuters sued ROSS anyways. A federal appeals court is still considering the key copyright issues in the case—which EFF weighed in on last year. EFF hopes that the appeals court will reject this overbroad interpretation of copyright law. But in the meantime, the case has already forced the startup out of business, eliminating a would-be competitor that might have helped increase access to the law.
Requiring developers to license AI training materials benefits tech monopolists as well. For giant tech companies that can afford to pay, pricey licensing deals offer a way to lock in their dominant positions in the generative AI market by creating prohibitive barriers to entry. The cost of licensing enough works to train an LLM would be prohibitively expensive for most would-be competitors.
The DMCA’s “Anti-Circumvention” Provision
The Digital Millennium Copyright Act’s “anti-circumvention” provision is another case in point. Congress ostensibly passed the DMCA to discourage would-be infringers from defeating Digital Rights Management (DRM) and other access controls and copy restrictions on creative works.
Section 1201 has been used to block competition and innovation in everything from printer cartridges to garage door openers
In practice, it’s done little to deter infringement—after all, large-scale infringement already invites massive legal penalties. Instead, Section 1201 has been used to block competition and innovation in everything from printer cartridges to garage door openers, videogame console accessories, and computer maintenance services. It’s been used to threaten hobbyists who wanted to make their devices and games work better. And the problem only gets worse as software shows up in more and more places, from phones to cars to refrigerators to farm equipment. If that software is locked up behind DRM, interoperating with it so you can offer add-on services may require circumvention. As a result, manufacturers get complete control over their products, long after they are purchased, and can even shut down secondary markets (as Lexmark did for printer ink, and Microsoft tried to do for Xbox memory cards.)
Giving rights holders a veto on new competition and innovation hurts consumers. Instead, we need balanced copyright policy that rewards consumers without impeding competition.
Of course, organizations see risk. It’s just that they struggle to turn insight into timely, safe action. That gap is why exposure management has emerged, and also why it is now becoming a foundational security discipline. What the diagram makes clear is that risk doesn’t stay flat while organizations deliberate. From the moment an exposure is discovered and is reachable, exploitable, and known – the clock starts ticking. As time passes, environments change, dependencies grow, and attackers adapt faster. Remediation workflows fall behind. Manual coordination, unclear ownership, and fear of disruption all extend what is increasingly referred to as ‘exposure […]
Chromium-based ChatGPT Atlas browser is testing a new feature likely called "Actions," and it can also understand videos, which is why you might see ChatGPT generating timestamps for videos. [...]
OpenAI recently rolled out ads to ChatGPT in the United States if you use $8 Go subscription or a free account, but Google says it does not plan to put ads in Gemini. [...]
OpenAI is rolling out an age prediction model on ChatGPT to detect your age and apply possible safety-related restrictions to prevent misuse by teens. [...]
The recently discovered cloud-focused VoidLink malware framework is believed to have been developed by a single person with the help of an artificial intelligence model. [...]
Using only natural language instructions, researchers were able to bypass Google Gemini's defenses against malicious prompt injection and create misleading events to leak private Calendar data. [...]
OpenAI is offering ChatGPT Plus, which costs $20 in the United States, for free, but the offer is valid for some accounts only, and it's a limited-time deal. [...]
OpenAI is reportedly testing a new feature or product codenamed "Sonata," and it could be related to music or audio-related experiences on ChatGPT. [...]
Google is testing "Skills" for Gemini in Chrome, which will allow AI in Chrome to perform tasks automatically, and it could challenge Perplexity Comet or Edge's Copilot mode. [...]
Google Chrome now lets you delete the local AI models that power the "Enhanced Protection" feature, which was upgraded with AI capabilities last year. [...]
Login
