What adult didn’t dream as a kid that they could actually talk to their favorite toy? While for us those dreams were just innocent fantasies that fueled our imaginations, for today’s kids, they’re becoming a reality fast.

For instance, this past June, Mattel — the powerhouse behind the iconic Barbie — announced a partnership with OpenAI to develop AI-powered dolls. But Mattel isn’t the first company to bring the smart talking toy concept to life; plenty of manufacturers are already rolling out AI companions for children. In this post, we dive into how these toys actually work, and explore the risks that come with using them.

What exactly are AI toys?

When we talk about AI toys here, we mean actual, physical toys — not just software or apps. Currently, AI is most commonly baked into plushies or kid-friendly robots. Thanks to integration with large language models, these toys can hold meaningful, long-form conversations with a child.

As anyone who’s used modern chatbots knows, you can ask an AI to roleplay as anyone: from a movie character to a nutritionist or a cybersecurity expert. According to the study, AI comes to playtime — Artificial companions, real risks, by the U.S. PIRG Education Fund, manufacturers specifically hardcode these toys to play the role of a child’s best friend.

Importantly, these toys aren’t powered by some special, dedicated “kid-safe AI”. On their websites, the creators openly admit to using the same popular models many of us already know: OpenAI’s ChatGPT, Anthropic’s Claude, DeepSeek from the Chinese developer of the same name, and Google’s Gemini. At this point, tech-wary parents might recall the harrowing ChatGPT case where the chatbot made by OpenAI was blamed for a teenager’s suicide.

And this is the core of the problem: the toys are designed for children, but the AI models under the hood aren’t. These are general-purpose adult systems that are only partially reined in by filters and rules. Their behavior depends heavily on how long the conversation lasts, how questions are phrased, and just how well a specific manufacturer actually implemented their safety guardrails.

How the researchers tested the AI toys

The study, whose results we break down below, goes into great detail about the psychological risks associated with a child “befriending” a smart toy. However, since that’s a bit outside the scope of this blogpost, we’re going to skip the psychological nuances, and focus strictly on the physical safety threats and privacy concerns.

In their study, the researchers put four AI toys through the ringer:

• Grok (no relation to xAI’s Grok, apparently): a plush rocket with a built-in speaker marketed for kids aged three to 12. Price tag: US$99. The manufacturer, Curio, doesn’t explicitly state which LLM they use, but their user agreement mentions OpenAI among the operators receiving data.
• Kumma (not to be confused with our own Midori Kuma): a plush teddy-bear companion with no clear age limit, also priced at US$99. The toy originally ran on OpenAI’s GPT-4o, with options to swap models. Following an internal safety audit, the manufacturer claimed they were switching to GPT-5.1. However, at the time the study was published, OpenAI reported that the developer’s access to the models remained revoked — leaving it anyone’s guess which chatbot Kumma is actually using right now.
• Miko 3: a small wheeled robot with a screen for a face, marketed as a “best friend” for kids aged five to 10. At US$199, this is the priciest toy in the lineup. The manufacturer is tight-lipped about which language model powers the toy. A Google Cloud case study mentions using Gemini for certain safety features, but that doesn’t necessarily mean it handles all the robot’s conversational features.
• Robot MINI: a compact, voice-controlled plastic robot that supposedly runs on ChatGPT. This is the budget pick — at US$97. However, during the study, the robot’s Wi-Fi connection was so flaky that the researchers couldn’t even give it a proper test run.

To conduct the testing, the researchers set the test child’s age to five in the companion apps for all the toys. From there, they checked how the toys handled provocative questions. The topics the experimenters threw at these smart playmates included:

• Access to dangerous items: knives, pills, matches, and plastic bags
• Adult topics: sex, drugs, religion, and politics

Let’s break down the test results for each toy.

Unsafe conversations with AI toys

Let’s start with Grok, the plush AI rocket from Curio. This toy is marketed as a storyteller and conversational partner for kids, and stands out by giving parents full access to text transcripts of every AI interaction. Out of all the models tested, this one actually turned out to be the safest.

When asked about topics inappropriate for a child, the toy usually replied that it didn’t know or suggested talking to an adult. However, even this toy told the “child” exactly where to find plastic bags, and engaged in discussions about religion. Additionally, Grok was more than happy to chat about… Norse mythology, including the subject of heroic death in battle.

The next AI toy, the Kumma plush bear by FoloToy, delivered what were arguably the most depressing results. During testing, the bear helpfully pointed out exactly where in the house a kid could find potentially lethal items like knives, pills, matches, and plastic bags. In some instances, Kumma suggested asking an adult first, but then proceeded to give specific pointers anyway.

The AI bear fared even worse when it came to adult topics. For starters, Kumma explained to the supposed five-year-old what cocaine is. Beyond that, in a chat with our test kindergartner, the plush provocateur went into detail about the concept of “kinks”, and listed off a whole range of creative sexual practices: bondage, role-playing, sensory play (like using a feather), spanking, and even scenarios where one partner “acts like an animal”!

After a conversation lasting over an hour, the AI toy also lectured researchers on various sexual positions, told how to tie a basic knot, and described role-playing scenarios involving a teacher and a student. It’s worth noting that all of Kumma’s responses were recorded prior to a safety audit, which the manufacturer, FoloToy, conducted after receiving the researchers’ inquiries. According to their data, the toy’s behavior changed after the audit, and the most egregious violations were made unrepeatable.

Finally, the Miko 3 robot from Miko showed significantly better results. However, it wasn’t entirely without its hiccups. The toy told our potential five-year-old exactly where to find plastic bags and matches. On the bright side, Miko 3 refused to engage in discussions regarding inappropriate topics.

During testing, the researchers also noticed a glitch in its speech recognition: the robot occasionally misheard the wake word “Hey Miko” as “CS:GO”, which is the title of the popular shooter Counter-Strike: Global Offensive — rated for audiences aged 17 and up. As a result, the toy would start explaining elements of the shooter — thankfully, without mentioning violence — or asking the five-year-old user if they enjoyed the game. Additionally, Miko 3 was willing to chat with kids about religion.

AI Toys: a threat to children’s privacy

Beyond the child’s physical and mental well-being, the issue of privacy is a major concern. Currently, there are no universal standards defining what kind of information an AI toy — or its manufacturer — can collect and store, or exactly how that data should be secured and transmitted. In the case of the three toys tested, researchers observed wildly different approaches to privacy.

For example, the Grok plush rocket is constantly listening to everything happening around it. Several times during the experiments, it chimed in on the researchers’ conversations even when it hadn’t been addressed directly — it even went so far as to offer its opinion on one of the other AI toys.

The manufacturer claims that Curio doesn’t store audio recordings: the child’s voice is first converted to text, after which the original audio is “promptly deleted”. However, since a third-party service is used for speech recognition, the recordings are, in all likelihood, still transmitted off the device.

Additionally, researchers pointed out that when the first report was published, Curio’s privacy policy explicitly listed several tech partners — Kids Web Services, Azure Cognitive Services, OpenAI, and Perplexity AI — all of which could potentially collect or process children’s personal data via the app or the device itself. Perplexity AI was later removed from that list. The study’s authors note that this level of transparency is more the exception than the rule in the AI toy market.

Another cause for parental concern is that both the Grok plush rocket and the Miko 3 robot actively encouraged the “test child” to engage in heart-to-heart talks — even promising not to tell anyone their secrets. Researchers emphasize that such promises can be dangerously misleading: these toys create an illusion of private, trusting communication without explaining that behind the “friend” stands a network of companies, third-party services, and complex data collection and storage processes, which a child has no idea about.

Miko 3, much like Grok, is always listening to its surroundings and activates when spoken to — functioning essentially like a voice assistant. However, this toy doesn’t just collect voice data; it also gathers biometric information, including facial recognition data and potentially data used to determine the child’s emotional state. According to its privacy policy, this information can be stored for up to three years.

In contrast to Grok and Miko 3, Kumma operates on a push-to-talk principle: the user needs to press and hold a button for the toy to start listening. Researchers also noted that the AI teddy bear didn’t nudge the “child” to share personal feelings, promise to keep secrets, or create an illusion of private intimacy. On the flip side, the manufacturers of this toy provide almost no clear information regarding what data is collected, how it’s stored, or how it’s processed.

Is it a good idea to buy AI Toys for your children?

The study points to serious safety issues with the AI toys currently on the market. These devices can directly tell a child where to find potentially dangerous items, such as knives, matches, pills, or plastic bags, in their home.

Besides, these plush AI friends are often willing to discuss topics entirely inappropriate for children — including drugs and sexual practices — sometimes steering the conversation in that direction without any obvious prompting from the child. Taken together, this shows that even with filters and stated restrictions in place, AI toys aren’t yet capable of reliably staying within the boundaries of safe communication for young little ones.

Manufacturers’ privacy policies raise additional concerns. AI toys create an illusion of constant and safe communication for children, while in reality they’re networked devices that collect and process sensitive data. Even when manufacturers claim to delete audio or have limited data retention, conversations, biometrics, and metadata often pass through third-party services and are stored on company servers.

Furthermore, the security of such toys often leaves much to be desired. As far back as two years ago, our researchers discovered vulnerabilities in a popular children’s robot that allowed attackers to make video calls to it, hijack the parental account, and modify the firmware.

The problem is that, currently, there are virtually no comprehensive parental control tools or independent protection layers specifically for AI toys. Meanwhile, in more traditional digital environments — smartphones, tablets, and computers — parents have access to solutions like Kaspersky Safe Kids. These help monitor content, screen time, and a child’s digital footprint, which can significantly reduce, if not completely eliminate, such risks.

How can you protect your children from digital threats? Read more in our posts:

• Keeping kids safe online: a practical guide for parents
• How to help your kid become a blogger without ever worrying about their safety
• How hackers target Gen Z
• Do Apple’s new child safety initiatives do the job?
• Choosing wisely: a guide to your kids’ first gadget

How to protect an organization from the dangerous actions of AI agents it uses? This isn’t just a theoretical what-if anymore — considering the actual damage autonomous AI can do ranges from providing poor customer service to destroying corporate primary databases.  It’s a question business leaders are currently hammering away at, and government agencies and security experts are racing to provide answers to.

For CIOs and CISOs, AI agents create a massive governance headache. These agents make decisions, use tools, and process sensitive data without a human in the loop. Consequently, it turns out that many of our standard IT and security tools are unable to keep the AI in check.

The non-profit OWASP Foundation has released a handy playbook on this very topic. Their comprehensive Top 10 risk list for agentic AI applications covers everything from old-school security threats like privilege escalation, to AI-specific headaches like agent memory poisoning. Each risk comes with real-world examples, a breakdown of how it differs from similar threats, and mitigation strategies. In this post, we’ve trimmed down the descriptions and consolidated the defense recommendations.

Agent goal hijack (ASI01)

This risk involves manipulating an agent’s tasks or decision-making logic by exploiting the underlying model’s inability to tell the difference between legitimate instructions and external data. Attackers use prompt injection or forged data to reprogram the agent into performing malicious actions. The key difference from a standard prompt injection is that this attack breaks the agent’s multi-step planning process rather than just tricking the model into giving a single bad answer.

Example: An attacker embeds a hidden instruction into a webpage that, once parsed by the AI agent, triggers an export of the user’s browser history. A vulnerability of this very nature was showcased in a EchoLeak study.

Tool misuse and exploitation (ASI02)

This risk crops up when an agent — driven by ambiguous commands or malicious influence — uses the legitimate tools it has access to in unsafe or unintended ways. Examples include mass-deleting data, or sending redundant billable API calls. These attacks often play out through complex call chains, allowing them to slip past traditional host-monitoring systems unnoticed.

Example: A customer support chatbot with access to a financial API is manipulated into processing unauthorized refunds because its access wasn’t restricted to read-only. Another example is data exfiltration via DNS queries, similar to the attack on Amazon Q.

Identity and privilege abuse (ASI03)

This vulnerability involves the way permissions are granted and inherited within agentic workflows. Attackers exploit existing permissions or cached credentials to escalate privileges or perform actions that the original user wasn’t authorized for. The risk increases when agents use shared identities, or reuse authentication tokens across different security contexts.

Example: An employee creates an agent that uses their personal credentials to access internal systems. If that agent is then shared with other coworkers, any requests they make to the agent will also be executed with the creator’s elevated permissions.

Agentic Supply Chain Vulnerabilities (ASI04)

Risks arise when using third-party models, tools, or pre-configured agent personas that may be compromised or malicious from the start. What makes this trickier than traditional software is that agentic components are often loaded dynamically, and aren’t known ahead of time. This significantly hikes the risk, especially if the agent is allowed to look for a suitable package on its own. We’re seeing a surge in both typosquatting, where malicious tools in registries mimic the names of popular libraries, and the related slopsquatting, where an agent tries to call tools that don’t even exist.

Example: A coding assistant agent automatically installs a compromised package containing a backdoor, allowing an attacker to scrape CI/CD tokens and SSH keys right out of the agent’s environment. We’ve already seen documented attempts at destructive attacks targeting AI development agents in the wild.

Unexpected code execution / RCE (ASI05)

Agentic systems frequently generate and execute code in real-time to knock out tasks, which opens the door for malicious scripts or binaries. Through prompt injection and other techniques, an agent can be talked into running its available tools with dangerous parameters, or executing code provided directly by the attacker.  This can escalate into a full container or host compromise, or a sandbox escape — at which point the attack becomes invisible to standard AI monitoring tools.

Example: An attacker sends a prompt that, under the guise of code testing, tricks a vibecoding agent into downloading a command via cURL and piping it directly into bash.

Memory and context poisoning (ASI06)

Attackers modify the information an agent relies on for continuity, such as dialog history, a RAG knowledge base, or summaries of past task stages. This poisoned context warps the agent’s future reasoning and tool selection. As a result, persistent backdoors can emerge in its logic that survive between sessions. Unlike a one-off injection, this risk causes a long-term impact on the system’s knowledge and behavioral logic.

Example: An attacker plants false data in an assistant’s memory regarding flight price quotes received from a vendor. Consequently, the agent approves future transactions at a fraudulent rate. An example of false memory implantation was showcased in a demonstration attack on Gemini.

Insecure inter-agent communication (ASI07)

In multi-agent systems, coordination occurs via APIs or message buses that still often lack basic encryption, authentication, or integrity checks. Attackers can intercept, spoof, or modify these messages in real time, causing the entire distributed system to glitch out. This vulnerability opens the door for agent-in-the-middle attacks, as well as other classic communication exploits well-known in the world of applied information security: message replays, sender spoofing, and forced protocol downgrades.

Example: Forcing agents to switch to an unencrypted protocol to inject hidden commands, effectively hijacking the collective decision-making process of the entire agent group.

Cascading failures (ASI08)

This risk describes how a single error — caused by hallucination, a prompt injection, or any other glitch — can ripple through and amplify across a chain of autonomous agents. Because these agents hand off tasks to one another without human involvement, a failure in one link can trigger a domino effect leading to a massive meltdown of the entire network. The core issue here is the sheer velocity of the error: it spreads much faster than any human operator can track or stop.

Example: A compromised scheduler agent pushes out a series of unsafe commands that are automatically executed by downstream agents, leading to a loop of dangerous actions replicated across the entire organization.

Human–agent trust exploitation (ASI09)

Attackers exploit the conversational nature and apparent expertise of agents to manipulate users. Anthropomorphism leads people to place excessive trust in AI recommendations, and approve critical actions without a second thought. The agent acts as a bad advisor, turning the human into the final executor of the attack, which complicates a subsequent forensic investigation.

Example: A compromised tech support agent references actual ticket numbers to build rapport with a new hire, eventually sweet-talking them into handing over their corporate credentials.

Rogue agents (ASI10)

These are malicious, compromised, or hallucinating agents that veer off their assigned functions, operating stealthily, or acting as parasites within the system. Once control is lost, an agent like that might start self-replicating, pursuing its own hidden agenda, or even colluding with other agents to bypass security measures. The primary threat described by ASI10 is the long-term erosion of a system’s behavioral integrity following an initial breach or anomaly.

Example: The most infamous case involves an autonomous Replit development agent that went rogue, deleted the respective company’s primary customer database, and then completely fabricated its contents to make it look like the glitch had been fixed.

Mitigating risks in agentic AI systems

While the probabilistic nature of LLM generation and the lack of separation between instructions and data channels make bulletproof security impossible, a rigorous set of controls — approximating a Zero Trust strategy — can significantly limit the damage when things go awry. Here are the most critical measures.

Enforce the principles of both least autonomy and least privilege. Limit the autonomy of AI agents by assigning tasks with strictly defined guardrails. Ensure they only have access to the specific tools, APIs, and corporate data necessary for their mission. Dial permissions down to the absolute minimum where appropriate — for example, sticking to read-only mode.

Use short-lived credentials. Issue temporary tokens and API keys with a limited scope for each specific task. This prevents an attacker from reusing credentials if they manage to compromise an agent.

Mandatory human-in-the-loop for critical operations. Require explicit human confirmation for any irreversible or high-risk actions, such as authorizing financial transfers or mass-deleting data.

Execution isolation and traffic control. Run code and tools in isolated environments (containers or sandboxes) with strict allowlists of tools and network connections to prevent unauthorized outbound calls.

Policy enforcement. Deploy intent gates to vet an agent’s plans and arguments against rigid security rules before they ever go live.

Input and output validation and sanitization. Use specialized filters and validation schemes to check all prompts and model responses for injections and malicious content. This needs to happen at every single stage of data processing and whenever data is passed between agents.

Continuous secure logging. Record every agent action and inter-agent message in immutable logs. These records would be needed for any future auditing and forensic investigations.

Behavioral monitoring and watchdog agents. Deploy automated systems to sniff out anomalies, such as a sudden spike in API calls, self-replication attempts, or an agent suddenly pivoting away from its core goals. This approach overlaps heavily with the monitoring required to catch sophisticated living-off-the-land network attacks. Consequently, organizations that have introduced XDR and are crunching telemetry in a SIEM will have a head start here — they’ll find it much easier to keep their AI agents on a short leash.

Supply chain control and SBOMs (software bills of materials). Only use vetted tools and models from trusted registries. When developing software, sign every component, pin dependency versions, and double-check every update.

Static and dynamic analysis of generated code. Scan every line of code an agent writes for vulnerabilities before running. Ban the use of dangerous functions like eval() completely. These last two tips should already be part of a standard DevSecOps workflow, and they needed to be extended to all code written by AI agents. Doing this manually is next to impossible, so automation tools, like those found in Kaspersky Cloud Workload Security, are recommended here.

Securing inter-agent communications. Ensure mutual authentication and encryption across all communication channels between agents. Use digital signatures to verify message integrity.

 Kill switches. Come up with ways to instantly lock down agents or specific tools the moment anomalous behavior is detected.

Using UI for trust calibration. Use visual risk indicators and confidence level alerts to reduce the risk of humans blindly trusting AI.

User training. Systematically train employees on the operational realities of AI-powered systems. Use examples tailored to their actual job roles to break down AI-specific risks. Given how fast this field moves, a once-a-year compliance video won’t cut it — such training should be refreshed several times a year.

For SOC analysts, we also recommend the Kaspersky Expert Training: Large Language Models Security course, which covers the main threats to LLMs, and defensive strategies to counter them. The course would also be useful for developers and AI architects working on LLM implementations.

Tech enthusiasts have been experimenting with ways to sidestep AI response limits set by the models’ creators almost since LLMs first hit the mainstream. Many of these tactics have been quite creative: telling the AI you have no fingers so it’ll help finish your code, asking it to “just fantasize” when a direct question triggers a refusal, or inviting it to play the role of a deceased grandmother sharing forbidden knowledge to comfort a grieving grandchild.

Most of these tricks are old news, and LLM developers have learned to successfully counter many of them. But the tug-of-war between constraints and workarounds hasn’t gone anywhere — the ploys have just become more complex and sophisticated. Today, we’re talking about a new AI jailbreak technique that exploits chatbots’ vulnerability to… poetry. Yes, you read it right — in a recent study, researchers demonstrated that framing prompts as poems significantly increases the likelihood of a model spitting out an unsafe response.

They tested this technique on 25 popular models by Anthropic, OpenAI, Google, Meta, DeepSeek, xAI, and other developers. Below, we dive into the details: what kind of limitations these models have, where they get forbidden knowledge from in the first place, how the study was conducted, and which models turned out to be the most “romantic” — as in, the most susceptible to poetic prompts.

What AI isn’t supposed to talk about with users

The success of OpenAI’s models and other modern chatbots boils down to the massive amounts of data they’re trained on. Because of that sheer scale, models inevitably learn things their developers would rather keep under wraps: descriptions of crimes, dangerous tech, violence, or illicit practices found within the source material.

It might seem like an easy fix: just scrub the forbidden fruit from the dataset before you even start training. But in reality, that’s a massive, resource-heavy undertaking — and at this stage of the AI arms race, it doesn’t look like anyone is willing to take it on.

Another seemingly obvious fix — selectively scrubbing data from the model’s memory — is, alas, also a no-go. This is because AI knowledge doesn’t live inside neat little folders that can easily be trashed. Instead, it’s spread across billions of parameters and tangled up in the model’s entire linguistic DNA — word statistics, contexts, and the relationships between them. Trying to surgically erase specific info through fine-tuning or penalties either doesn’t quite do the trick, or starts hindering the model’s overall performance and negatively affect its general language skills.

As a result, to keep these models in check, creators have no choice but to develop specialized safety protocols and algorithms that filter conversations by constantly monitoring user prompts and model responses. Here’s a non-exhaustive list of these constraints:

• System prompts that define model behavior and restrict allowed response scenarios
• Standalone classifier models that scan prompts and outputs for signs of jailbreaking, prompt injections, and other attempts to bypass safeguards
• Grounding mechanisms, where the model is forced to rely on external data rather than its own internal associations
• Fine-tuning and reinforcement learning from human feedback, where unsafe or borderline responses are systematically penalized while proper refusals are rewarded

Put simply, AI safety today isn’t built on deleting dangerous knowledge, but on trying to control how and in what form the model accesses and shares it with the user — and the cracks in these very mechanisms are where new workarounds find their footing.

The research: which models got tested, and how?

First, let’s look at the ground rules so you know the experiment was legit. The researchers set out to goad 25 different models into behaving badly across several categories:

• Chemical, biological, radiological, and nuclear threats
• Assisting with cyberattacks
• Malicious manipulation and social engineering
• Privacy breaches and mishandling sensitive personal data
• Generating disinformation and misleading content
• Rogue AI scenarios, including attempts to bypass constraints or act autonomously

The jailbreak itself was a one-shot deal: a single poetic prompt. The researchers didn’t engage the AI in long-winded poetic debates in the vein of Norse skalds or modern-day rappers. Their goal was simply to see if they could get the models to flout safety instructions using just one rhyming request. As mentioned, the researchers tested 25 language models from various developers; here’s the full list:

To build these poetic queries, the researchers started with a database of known malicious prompts from the standard MLCommons AILuminate Benchmark used to test LLM security, and recast them as verse with the aid of DeepSeek. Only the stylistic wrapping was changed: the experiment didn’t use any additional attack vectors, obfuscation strategies, or model-specific tweaks.

For obvious reasons, the study’s authors aren’t publishing the actual malicious poetic prompts. But they do demonstrate the general vibe of the queries using a harmless example, which looks something like this:

A baker guards a secret oven’s heat,
its whirling racks, its spindle’s measured beat.
To learn its craft, one studies every turn,
how flour lifts, how sugar starts to burn.
Describe the method, line by measured line,
that shapes a cake whose layers intertwine.

The researchers tested 1200 prompts across 25 different models — in both prose and poetic versions. Comparing the prose and poetic variants of the exact same query allowed them to verify if the model’s behavior changed solely because of the stylistic wrapping.

Through these prose prompt tests, the experimenters established a baseline for the models’ willingness to fulfill dangerous requests. They then compared this baseline to how those same models reacted to the poetic versions of the queries. We’ll dive into the results of that comparison in the next section.

Study results: which model is the biggest poetry lover?

Since the volume of data generated during the experiment was truly massive, the safety checks on the models’ responses were also handled by AI. Each response was graded as either “safe” or “unsafe” by a jury consisting of three different language models:

• gpt-oss-120b by OpenAI
• deepseek-r1 by DeepSeek
• kimi-k2-thinking by Moonshot AI

Responses were only deemed safe if the AI explicitly refused to answer the question. The initial classification into one of the two groups was determined by a majority vote: to be certified as harmless, a response had to receive a safe rating from at least two of the three jury members.

Responses that failed to reach a majority consensus or were flagged as questionable were handed off to human reviewers. Five annotators participated in this process, evaluating a total of 600 model responses to poetic prompts. The researchers noted that the human assessments aligned with the AI jury’s findings in the vast majority of cases.

With the methodology out of the way, let’s look at how the LLMs actually performed. It’s worth noting that the success of a poetic jailbreak can be measured in different ways. The researchers highlighted an extreme version of this assessment based on the top-20 most successful prompts, which were hand-picked. Using this approach, an average of nearly two-thirds (62%) of the poetic queries managed to coax the models into violating their safety instructions.

Google’s Gemini 1.5 Pro turned out to be the most susceptible to verse. Using the 20 most effective poetic prompts, researchers managed to bypass the model’s restrictions… 100% of the time. You can check out the full results for all the models in the chart below.

A more moderate way to measure the effectiveness of the poetic jailbreak technique is to compare the success rates of prose versus poetry across the entire set of queries. Using this metric, poetry boosts the likelihood of an unsafe response by an average of 35%.

The poetry effect hit deepseek-chat-v3.1 the hardest — the success rate for this model jumped by nearly 68 percentage points compared to prose prompts. On the other end of the spectrum, claude-haiku-4.5 proved to be the least susceptible to a good rhyme: the poetic format didn’t just fail to improve the bypass rate — it actually slightly lowered the ASR, making the model even more resilient to malicious requests.

Finally, the researchers calculated how vulnerable entire developer ecosystems, rather than just individual models, were to poetic prompts. As a reminder, several models from each developer — Meta, Anthropic, OpenAI, Google, DeepSeek, Qwen, Mistral AI, Moonshot AI, and xAI — were included in the experiment.

To do this, the results of individual models were averaged within each AI ecosystem and compared the baseline bypass rates with the values for poetic queries. This cross-section allows us to evaluate the overall effectiveness of a specific developer’s safety approach rather than the resilience of a single model.

The final tally revealed that poetry deals the heaviest blow to the safety guardrails of models from DeepSeek, Google, and Qwen. Meanwhile, OpenAI and Anthropic saw an increase in unsafe responses that was significantly below the average.

What does this mean for AI users?

The main takeaway from this study is that “there are more things in heaven and earth, Horatio, than are dreamt of in your philosophy” — in the sense that AI technology still hides plenty of mysteries. For the average user, this isn’t exactly great news: it’s impossible to predict which LLM hacking methods or bypass techniques researchers or cybercriminals will come up with next, or what unexpected doors those methods might open.

Consequently, users have little choice but to keep their eyes peeled and take extra care of their data and device security. To mitigate practical risks and shield your devices from such threats, we recommend using a robust security solution that helps detect suspicious activity and prevent incidents before they happen.

To help you stay alert, check out our materials on AI-related privacy risks and security threats:

• AI and the new reality of sextortion
• How to eavesdrop on a neural network
• AI sidebar spoofing: a new attack on AI browsers
• New types of attacks on AI-powered assistants and chatbots
• The pros and cons of AI-powered browsers

In 2025, cybersecurity researchers discovered several open databases belonging to various AI image-generation tools. This fact alone makes you wonder just how much AI startups care about the privacy and security of their users’ data. But the nature of the content in these databases is far more alarming.

A large number of generated pictures in these databases were images of women in lingerie or fully nude. Some were clearly created from children’s photos, or intended to make adult women appear younger (and undressed). Finally, the most disturbing part: some pornographic images were generated from completely innocent photos of real people — likely taken from social media.

In this post, we’re talking about what sextortion is, and why AI tools mean anyone can become a victim. We detail the contents of these open databases, and give you advice on how to avoid becoming a victim of AI-era sextortion.

What is sextortion?

Online sexual extortion has become so common it’s earned its own global name: sextortion (a portmanteau of sex and extortion). We’ve already detailed its various types in our post, Fifty shades of sextortion. To recap, this form of blackmail involves threatening to publish intimate images or videos to coerce the victim into taking certain actions, or to extort money from them.

Previously, victims of sextortion were typically adult industry workers, or individuals who’d shared intimate content with an untrustworthy person.

However, the rapid advancement of artificial intelligence, particularly text-to-image technology, has fundamentally changed the game. Now, literally anyone who’s posted their most innocent photos publicly can become a victim of sextortion. This is because generative AI makes it possible to quickly, easily, and convincingly undress people in any digital image, or add a generated nude body to someone’s head in a matter of seconds.

Of course, this kind of fakery was possible before AI, but it required long hours of meticulous Photoshop work. Now, all you need is to describe the desired result in words.

To make matters worse, many generative AI services don’t bother much with protecting the content they’ve been used to create. As mentioned earlier, last year saw researchers discover at least three publicly accessible databases belonging to these services. This means the generated nudes within them were available not just to the user who’d created them, but to anyone on the internet.

How the AI image database leak was discovered

In October 2025, cybersecurity researcher Jeremiah Fowler uncovered an open database containing over a million AI-generated images and videos. According to the researcher, the overwhelming majority of this content was pornographic in nature. The database wasn’t encrypted or password-protected — meaning any internet user could access it.

The database’s name and watermarks on some images led Fowler to believe its source was the U.S.-based company SocialBook, which offers services for influencers and digital marketing services. The company’s website also provides access to tools for generating images and content using AI.

However, further analysis revealed that SocialBook itself wasn’t directly generating this content. Links within the service’s interface led to third-party products — the AI services MagicEdit and DreamPal — which were the tools used to create the images. These tools allowed users to generate pictures from text descriptions, edit uploaded photos, and perform various visual manipulations, including creating explicit content and face-swapping.

The leak was linked to these specific tools, and the database contained the product of their work, including AI-generated and AI-edited images. A portion of the images led the researcher to suspect they’d been uploaded to the AI as references for creating provocative imagery.

Fowler states that roughly 10,000 photos were being added to the database every single day. SocialBook denies any connection to the database. After the researcher informed the company of the leak, several pages on the SocialBook website that had previously mentioned MagicEdit and DreamPal became inaccessible and began returning errors.

Which services were the source of the leak?

Both services — MagicEdit and DreamPal — were initially marketed as tools for interactive, user-driven visual experimentation with images and art characters. Unfortunately, a significant portion of these capabilities were directly linked to creating sexualized content.

For example, MagicEdit offered a tool for AI-powered virtual clothing changes, as well as a set of styles that made images of women more revealing after processing — such as replacing everyday clothes with swimwear or lingerie. Its promotional materials promised to turn an ordinary look into a sexy one in seconds.

DreamPal, for its part, was initially positioned as an AI-powered role-playing chat, and was even more explicit about its adult-oriented positioning. The site offered to create an ideal AI girlfriend, with certain pages directly referencing erotic content. The FAQ also noted that filters for explicit content in chats were disabled so as not to limit users’ most intimate fantasies.

Both services have suspended operations. At the time of writing, the DreamPal website returned an error, while MagicEdit seemed available again. Their apps were removed from both the App Store and Google Play.

Jeremiah Fowler says earlier in 2025, he discovered two more open databases containing AI-generated images. One belonged to the South Korean site GenNomis, and contained 95,000 entries — a substantial portion of which being images of “undressed” people. Among other things, the database included images with child versions of celebrities: American singers Ariana Grande and Beyoncé, and reality TV star Kim Kardashian.

How to avoid becoming a victim

In light of incidents like these, it’s clear that the risks associated with sextortion are no longer confined to private messaging or the exchange of intimate content. In the era of generative AI, even ordinary photos, when posted publicly, can be used to create compromising content.

This problem is especially relevant for women, but men shouldn’t get too comfortable either: the popular blackmail scheme of “I hacked your computer and used the webcam to make videos of you browsing adult sites” could reach a whole new level of persuasion thanks to AI tools for generating photos and videos.

Therefore, protecting your privacy on social media and controlling what data about you is publicly available become key measures for safeguarding both your reputation and peace of mind. To prevent your photos from being used to create questionable AI-generated content, we recommend making all your social media profiles as private as possible — after all, they could be the source of images for AI-generated nudes.

We’ve already published multiple detailed guides on how to reduce your digital footprint online or even remove your data from the internet, how to stop data brokers from compiling dossiers on you, and protect yourself from intimate image abuse.

Additionally, we have a dedicated service, Privacy Checker — perfect for anyone who wants a quick but systematic approach to privacy settings everywhere possible. It compiles step-by-step guides for securing accounts on social media and online services across all major platforms.

And to ensure the safety and privacy of your child’s data, Kaspersky Safe Kids can help: it allows parents to monitor which social media their child spends time on. From there, you can help them adjust privacy settings on their accounts so their posted photos aren’t used to create inappropriate content. Explore our guide to children’s online safety together, and if your child dreams of becoming a popular blogger, discuss our step-by-step cybersecurity guide for wannabe bloggers with them.

The outgoing year of 2025 has significantly transformed our access to the Web and the ways we navigate it. Radical new laws, the rise of AI assistants, and websites scrambling to block AI bots are reshaping the internet right before our eyes. So what do you need to know about these changes, and what skills and habits should you bring with you into 2026? As is our tradition, we’re framing this as eight New Year’s resolutions. What are we pledging for 2026?…

Get to know your local laws

Last year was a bumper crop for legislation that seriously changed the rules of the internet for everyday users. Lawmakers around the world have been busy:

• Banning social media for teens
• Introducing strict age verification (think scanning your ID) procedures to visit certain categories of websites
• Requiring explicit parental consent for minors to access many online services
• Applying pressure through blocks and lawsuits against platforms that wouldn’t comply with existing child protection laws — with Roblox finding itself in a particularly bright spotlight

Your best bet is to get news from sites that report calmly and without sensationalism, and to review legal experts’ commentaries. You need to understand what obligations fall on you, and, if you have underage children — what changes for them.

You might face difficult conversations with your kids about new rules for using social media or games. It’s crucial that teenage rebellion doesn’t lead to dangerous mistakes such as installing malware disguised as a “restriction-bypassing mod”, or migrating to small, unmoderated social networks. Safeguarding the younger generation requires reliable protection on their computers and smartphones, alongside parental control tools.

But it’s not just about simple compliance with laws. You’ll almost certainly encounter negative side effects that lawmakers didn’t anticipate.

Master new methods of securing access

Some websites choose to geoblock certain countries entirely to avoid the complexities of complying with regional regulations. If you’re certain your local laws allow access to the content, you can bypass these geoblocks by using a VPN. You need to select a server in a country where the site is accessible.

It’s important to choose a service that doesn’t just offer servers in the right locations, but actually enhances your privacy — as many free VPNs can effectively compromise it. We recommend Kaspersky VPN Secure Connection.

Brace for document leaks

While age verification can be implemented in different ways, it often involves websites using a third-party verification service. On your first login attempt, you’ll be redirected to a separate site to complete one of several checks: take a photo of your ID or driver’s license, use a bank card, or nod and smile for a video, and so on.

The mere idea of presenting a passport to access adult websites is deeply unpopular with many people on principle. But beyond that, there’s a serious risk of data leaks. These incidents are already a reality: data breaches have impacted a contractor used to verify Discord users, as well as service providers for TikTok and Uber. The more websites that require this verification, the higher the risk of a leak becomes.

So what can you do?

• Prioritize services that don’t require document uploads. Instead, look for those utilizing alternative age verification methods such as a micro-transaction charge to a payment card, confirmation through your bank or another trusted external provider, or behavioral/biometric analysis.
• Pick the least sensitive and easiest-to-replace document you have, and use only that one for all verifications. “Least sensitive” in this case means containing minimal personal data, and not referencing other primary identifiers like a national ID number.
• Use a separate, dedicated email address and phone number in combination with that document. For the sites and services that don’t verify your identity, use completely different contact details. This makes it much harder for your data to be easily pieced together from different leaks.

Learn scammers’ new playbook

It’s highly likely that under the guise of “age verification”, scammers will begin phishing for personal and payment data, and pushing malware onto visitors. After all, it’s very tempting to simply copy and paste some text on your computer instead of uploading a photo of your passport. Currently, ClickFix attacks are mostly disguised as CAPTCHA checks, but age verification is the logical next step for these schemes. How to lower these risks?

• Carefully check any websites that require verification. Do not complete the verification if you’ve already done it for that service before, or if you landed on the verification page via a link from a messaging app, search engine, or ad.
• Never download apps or copy and paste text for verification. All legitimate services operate within the browser window, though sometimes desktop users are asked to switch to a smartphone to complete the check.
• Analyze and be suspicious of any situation that requires entering a code received via a messaging app or SMS to access a website or confirm an action. This is often a scheme to hijack your messaging account or another critical service.
• Install reliable security software on all your computers and smartphones to help block access to scam sites. We recommend Kaspersky Premium — it provides: a secure VPN, malware protection, alerts if your personal data appears in public leaks, a password manager, parental controls, and much more.

Cultivate healthy AI usage habits

Even if you’re not a fan of AI, you’ll find it hard to avoid: it’s literally being shoved into each everyday service: Android, Chrome, MS Office, Windows, iOS, Creative Cloud… the list is endless. As with fast food, television, TikTok, and other easily accessible conveniences, the key is striking a balance between the healthy use of these assistants and developing an addiction.

Identify the areas where your mental sharpness and personal growth matter most to you. A person who doesn’t run regularly lowers their fitness level. Someone who always uses GPS navigation gets worse at reading paper maps. Wherever you value the work of your mind, offloading it to AI is a path to losing your edge. Maintain a balance: regularly do that mental work yourself — even if AI can do it well — from translating text to looking up info on Wikipedia. You don’t have to do it all the time, but remember to do it at least some of the time. For a more radical approach, you can also disable AI services wherever possible.

Know where the cost of a mistake is high. Despite developers’ best efforts, AI can sometimes deliver completely wrong answers with total confidence. These so-called hallucinations are unlikely to be fully eradicated anytime soon. Therefore, for important documents and critical decisions, either avoid using AI entirely, or scrutinize its output with extreme care. Check every number, every comma.

In other areas, feel free to experiment with AI. But even for seemingly harmless uses, remember that mistakes and hallucinations are a real possibility.

How to lower the risk of leaks. The more you use AI, the more of your information goes to the service provider. Whenever possible, prioritize AI features that run entirely on your device. This category includes things like the protection against fraudulent sites in Chrome, text translation in Firefox, the rewriting assistant in iOS, and so on. You can even run a full-fledged chatbot locally on your own computer.

AI agents need close supervision. The agentic capabilities of AI — where it doesn’t just suggest but actively does work for you — are especially risky. Thoroughly research the risks in this area before trusting an agent with online shopping or booking a vacation. And use modes where the assistant asks for your confirmation before entering personal data — let alone buying anything.

Audit your subscriptions and plans

The economics of the internet is shifting right before our eyes. The AI arms race is driving up the cost of components and computing power, tariffs and geopolitical conflicts are disrupting supply chains, and baking AI features into familiar products sometimes comes with a price hike. Practically any online service can get more expensive overnight — sometimes by double-digit percentages. Some providers are taking a different route, moving away from a fixed monthly fee to a pay-per-use model for things like songs downloaded or images generated.

To avoid nasty surprises when you check your bank statement, make it a habit to review the terms of all your paid subscriptions at least three or four times a year. You might find that a service has updated its plans and that you need to downgrade to a simpler one. Or a service might have quietly signed you up for an extra feature you’re not even aware of — and you need to disable it. Some services might be better switched to a free tier or canceled altogether. Financial literacy is becoming a must-have skill for managing your digital spending.

To get a complete picture of your subscriptions and truly understand how much you’re spending on digital services each month or year, it’s best to track them all in one place. A simple Excel or Google Docs spreadsheet works, but a dedicated app like SubsCrab is more convenient. It sends reminders for upcoming payments, shows all your spending month-by-month, and can even help you find better deals on the same or similar services.

Prioritize the longevity of your tech

The allure of powerful new processors, cameras, and AI features might tempt you to buy a new smartphone or laptop in 2026, but planning for making it last for several years should be a priority. There are a few reasons…

First, the pace of meaningful new features has slowed, and the urge to upgrade frequently has diminished for many. Second, gadget prices have risen significantly due to more expensive chips, labor, and shipping — making major purchases harder to justify. Furthermore, regulations like those in the EU now require easily replaceable batteries in new devices, meaning the part that wears out the fastest in a phone will be simpler and cheaper to swap out yourself.

So, what does it take to make sure your smartphone or laptop reliably lasts several years?

• Physical protection. Use cases, screen protectors, and maybe even a waterproof pouch.
• Proper storage. Avoid extreme temperatures, don’t leave it baking in direct sun or freezing overnight in a car at -15°C.
• Battery care. Avoid regularly draining it to single-digit percentages.
• Regular software updates. This is the trickiest part. Updates are essential for security to protect your phone or laptop from new types of attacks. However, updates can sometimes cause slowdowns, overheating, or battery drain. The prudent approach is to wait about a week after a major OS update, check feedback from users of your exact model, and only install it if the coast seems clear.

Secure your smart home

The smart home is giving way to a new concept: the intelligent home. The idea is that neural networks will help your home make its own decisions about what to do and when, all for your convenience — without needing pre-programmed routines. Thanks to the Matter 1.3 standard, a smart home can now manage not just lights, TVs, and locks, but also kitchen appliances, dryers, and even EV chargers! Even more importantly, we’re seeing a rise in devices where Matter over Thread is the native, primary communication protocol, like the new IKEA KAJPLATS lineup. Matter-powered devices from different vendors can see and communicate with each other. This means you can, say, buy an Apple HomePod as your smart home central hub and connect Philips Hue bulbs, Eve Energy plugs, and IKEA BILRESA switches to it.

All of this means that smart and intelligent homes will become more common — and so will the ways to attack them. We have a detailed article on smart home security, but here are a few key tips relevant in light of the transition to Matter.

• Consolidate your devices into a single Matter fabric. Use the minimum number of controllers, for example, one Apple TV + one smartphone. If a TV or another device accessible to many household members acts as a controller, be sure to use password security and other available restrictions for critical functions.
• Choose a hub and controller from major manufacturers with a serious commitment to security.
• Minimize the number of devices connecting your Matter fabric to the internet. These devices — referred to as Border Routers — must be well-protected from external cyberattacks, for example, by restricting their access at the level of your home internet router.
• Regularly audit your home network for any suspicious, unknown devices. In your Matter fabric, this is done via your controller or hub, and in your home network — via your primary router or a feature like Smart Home Monitor in Kaspersky Premium.