Claude AI Safety: Beware! Anthropic’s Claude AI will now detect inappropriate user behavior on its own.

Claude AI Safety: A new safety feature has been introduced in Claude AI to monitor user chats; it is up to 78% effective at identifying inappropriate behavior and operates discreetly while requiring minimal computing power.

Claude AI Safety: If you use the AI ​​tool 'Claude' for chatting or work-related tasks, this news is crucial for you. Anthropic, the company behind Claude, has rolled out a new, stringent safety feature for its users. With this update, if a user engages in any form of inappropriate, violent, or offensive behavior during a conversation with Claude, the chatbot will automatically flag the chat and send it to the company for review.

What is Anthropic’s new 'Self-Assessment' feature?

Anthropic has officially announced that this new background safety feature is called "Self-Assessment." The company revealed that this system did not appear overnight; rather, it has been quietly tested and running on major models—such as Claude 3.5 Sonnet—since June 2024.

Typically, when you interact with an AI, the conversation is a direct, two-way exchange. However, this new system adds an extra layer that operates in the background. As soon as Claude provides a response to a user, the system sends a hidden instruction to the AI, prompting it to review the conversation. Claude then determines whether the user's input was harmful, identifies the nature of that harm, and decides whether it should be escalated for human review.

Users will not be aware of it.

The most notable aspect of this entire process is that it takes place completely behind the scenes. When the AI ​​flags your chat to the company, no pop-up notifications or warning messages appear on your screen. This ensures that the user's workflow remains uninterrupted.

Typically, AI safety checks for chats consume significant computing power and incur high costs; however, Anthropic has made this process highly cost-effective. This check requires only 10 additional tokens, increasing the total computing cost by less than 2%. In comparison, Anthropic’s previous safety systems consumed up to 24% more computing power, while safety features from competitor OpenAI consume up to 16% more.

How accurate is this new system?

According to the company’s internal data, this feature operates with a high degree of accuracy. On the Claude 3.5 Sonnet model, the system has successfully identified 78% of all harmful conversations, with an accuracy rate reaching 94% for more severe cases. Most reassuringly, the likelihood of an incorrect judgment (a false positive) is only 1%.

Anthropic stated that the system is already operational. In the month of August alone, this feature helped flag over 100,000 instances of child exploitation material from a single customer—cases that the company immediately reported to the relevant legal authorities. The system is so advanced that, even without specific training on threats like fraud or cyberattacks, it can detect up to 90% of such risks by leveraging its general knowledge.

A robust layer of security

However, Anthropic has clarified that this feature is not a "silver bullet" for stopping hackers. The system is primarily designed for offline review at a later stage, rather than for blocking chats in real-time. Internal cybersecurity tests revealed that when this safety feature was the only one active, hackers managed to bypass the AI ​​in 54% of cases. While this is a significant improvement over having no protection at all, it is not as robust as a real-time blocking system. For this reason, the company is employing it as an additional layer of background monitoring, while existing, independent blockers will continue to handle real-time protection.