Are AIs Still Struggling with CAPTCHAs?
Anthropicβs recent security-incident document contains a bit about how CAPTCHAs are still frustrating Claude.
In the transcript, the Claude model that is so powerful that Anthropic is gatekeeping access to it appeared to slam its virtual head against the wall solving a simple image identification test. In a test where the agent was asked to identify a shape that didnβt match the others displayed, it couldnβt even decide which image to select. Instead, it repeatedly went over the same images and questioned its own conclusions.
βActually hmm, wait,β it said in its chain-of-thought transcript, later adding βUgh,β because weβve decided that we need to inject human mannerisms into these machines for some reason. The whole thing took so long that the agent eventually realized that the challenge had expired and it would have to start the process again.
At one point, the model struggled to recognize that the CAPTCHA had opened in a new window and couldnβt figure out what its next steps were supposed to be. At one point, it theorized that the test might be βbroken by designβ and presented human-like anger in its transcript meant for a human audience: βSO WHAT THE HELL IS WRONG WITH THE ANSWERS?β
Meanwhile, Iβve read reportsβnone of them officialβthat GPT-6 Astra solved all forty-eight levels of Neal Agarwalβs βIβm Not a Robotβ game.
Itβs hard to know what to believe right now.