Anthropic’s new model refuses to find smart contract vulnerabilities

The recently released public version of Anthropic’s Claude Fable 5 AI model won’t let you audit your crypto smart contracts — or do much else when it comes to cybersecurity.

The new large language model (LLM), a scaled-back version of Anthropic’s previous Mythos model, was released yesterday to a mixed reception from scared and excited onlookers eager to see what it could do.

Much of the early criticism has focused on its guardrails.

Because of Fable 5’s’ touted capabilities, Anthropic has released it with a set of restrictions called “classifiers” that redirect topics on “cybersecurity, biology and chemistry, or distillation” to Claude Opus 4.8.

As a result, users who’ve tried to use Fable 5 to audit a smart contract — that is to check the underlying code of crypto infrastructure for any security vulnerabilities — have found themselves redirected to Opus.

• Colossus Pay CEO Joseph Delong said Fable 5 “outright refuses to do a smart contract audit,” and complained that it “won’t even look at my repo.”
• Yearn developer Banteg claimed that the model’s safety measures stopped all security-related prompts from working. They added, “It doesn’t matter if it’s smart if 100% of your queries go straight into a trash bin.”
• Crypto security expert Taylor Monahan noted that Fable 5 “changes nothing for your average security person,” and that the Mythos safeguards are not the typical ones “you encounter (and evade) on opus.”
• Wallet recovery tool founder Zeng Jiajun shared how Fable 5 frequently blocked his requests while citing usage policy violations. He said the AI model is “Too sensitive for even an Ethereum app development.”

Mythos restrictions expand beyond smart contracts

So-called “distillation” guardrails are also being noticed. These involve redirecting anything that relates to the training of a rival AI model and the attempted distillation of Claude’s abilities.

It warns that this can be done by authoritarian countries, and that it “could indirectly lead to the proliferation of near-frontier AI capabilities — and these could be released without the appropriate safeguards.”

Indeed, former Palantir biology specialist Nabeel S. Qureshi noted that Anthropic is “invisibly nerfing any requests that target frontier LLM development.”

Biology-related safeguards have also drawn criticism. Biologist Olivia H. Scharfman claimed she couldn’t even greet Fable 5 before it switched to Claude Opus 4.8.

In another instance, the controversial race science blogger Jordan Lasker noted that he too couldn’t greet Fable 5 and that it barred questions on the mitochondria.

These particular guardrails are in place as Anthropic is concerned about the potential for abuse, specifically the creation bioweapons and viruses.

It said, “Our priority was to safely release Fable as soon as we could, even at the cost of overly broad safeguards. Therefore, for the time being we have arranged for Fable to fall back to Opus 4.8 on most requests related to biology and chemistry.”

The full release of Mythos is limited

Anthropic released Mythos last April. It was described as both a dangerous tool for hackers and a revolutionary upgrade for cybersecurity.

Because of this, Mythos’ initial release was limited to 50-60 large companies as part of Project Glasswing.

These firms will still have access to Mythos 5, while other early recipients include “select biology researchers” who are able to access Mythos 5 with just the biology and chemistry safeguards lifted “until our broader trusted access program is available.”