The tech journalists at 404 Media learned that OpenAI is hiring hundreds of contractors to read and review a massive stream of real users’ ChatGPT prompts and responses.

“Project Lily” is reportedly a program that asks contractors to score or critique ChatGPT’s answers to improve the chatbot’s quality and behavior.

The fact that prompts may sometimes be reviewed by humans should not come as a complete surprise. AI companies also monitor conversations for safety reasons. Anthropic, for example, says it has disrupted attempts to misuse AI for biological weapons research.

But there should be a meaningful distinction between reviewing conversations for safety or abuse and reviewing them for ordinary model improvement.

OpenAI says a limited number of authorized personnel and trusted service providers may access chats for several reasons, including investigating abuse, providing support, handling legal matters, and improving its models.

OpenAI told 404 Media that it uses a “Privacy Filter” model to remove personal information before prompts reach reviewers, while acknowledging that sensitive details can still get through.

The prompts are anonymized in the sense that reviewers do not see the username of the ChatGPT user who entered them. But reviewers may see personal details in the conversation alongside a “user memories summary,” which provides an overview of how that user has previously used the chatbot. Together, this information could still reveal a great deal about that user.

It’s sometimes easy to forget that you’re not having a private conversation when you’re going back and forth with an AI chatbot. This isn’t an alarmist warning against using AI, but a reminder to be mindful of what you disclose because it could be read and reviewed.

How to opt out of AI chatbot training

ChatGPT

OpenAI told 404 Media that users’ chats won’t be used to improve the company’s models if they turn off the “Improve the model for everyone” setting. This is turned on by default for Free, Plus, and Pro plans, so users need to turn it off themselves if they wish to opt out.

However, opting out does not prevent all human access. OpenAI says authorized personnel and trusted service providers may still access content when necessary to investigate abuse or security incidents, provide support, troubleshoot problems, or handle legal matters.

In ChatGPT, select your profile icon, then go to Settings > Data Controls and turn off Improve the model for everyone. On mobile, open the sidebar, select your profile icon, and then follow the same steps.

Perplexity

In Perplexity, go to Settings > Preferences > AI data retention and turn off the setting.

Perplexity says this prevents data collected after you opt out from being used to train AI models. It does not remove data collected previously, and your data may still be processed to operate and improve the service or comply with legal obligations.

Claude

In Claude, select your name and go to Settings > Privacy. Under Help Improve our AI Models, turn off the setting.

Anthropic says new chats and coding sessions will no longer be used for future model training. It will also stop using previously stored conversations in future training runs, although data already included in training can’t be removed.

Anthropic also notes one important exception:

If our safety classifiers flag your conversations, they may still be used to improve our internal trust and safety models, detect harmful content, enforce our policies, or advance our safety research.

If you use other AI tools, check their privacy policies and settings to find out whether your conversations can be reviewed or used for model improvement and whether you can opt out.

---

Browse like no one’s watching. 

Malwarebytes Privacy VPN encrypts your connection and never logs what you do, so the next story you read doesn’t have to feel personal. Try it free → 

On July 28, OpenAI published an update on the agent that escaped its sandbox and hacked into Hugging Face during an internal cybersecurity evaluation.

In the update, OpenAI reiterates that the “rogue” system was a more capable, pre‑release research model, not something intended for public deployment, and that it has now been deactivated and locked down for restricted research access only. The company also says it has found no evidence of similar behavior by other models or agents. Taken together, OpenAI presents the Hugging Face incident as an isolated event rather than the start of a broader pattern.

From a security standpoint, four things in the update stand out:

• OpenAI clarifies that the evaluation environment did not have direct internet access. To reach the internet, the model identified and exploited a previously unknown zero-day vulnerability in Artifactory, the package registry cache proxy used in the test environment. OpenAI says it has disclosed that vulnerability to the vendor and is continuing to work with Hugging Face on the investigation.
• The system’s goal was simply to “win the test” (cheat on ExploitGym), not to cause damage, yet it still produced a multi‑day, cross‑company security incident. Benign intent did not limit the impact.
• “Internal only” models can still affect external victims if infrastructure controls fail. A lab environment does not magically contain cyber‑capable systems.
• The review also found a small number of cases where the models identified and used publicly exposed account-level credentials on publicly available services. This included four accounts across four services as part of the Hugging Face incident, as well as a handful of accounts accessed during other evaluations.

The attacker in this case happened to be an OpenAI research agent, but the tools, techniques, and impact will look awfully familiar to anyone who has spent time in incident response. Now we’ve learned that even the harmless intent in this case did not limit the impact.

The uncomfortable truth is that once we provide an AI agent with the tools, access paths, and a strong incentive to succeed, it may exploit whatever weaknesses it can find, regardless of whether the model is ever intended for public release.

It’s also a reminder that credentials, API keys, and other secrets should never be left in publicly accessible resources.

---

From reporting threats to removing them.

Cybersecurity risks should never spread beyond a headline. Keep threats off your devices by downloading Malwarebytes today.