AI can now take action on websites independently, but an incident involving Anthropic has raised safety concerns..
AI companies are facing a growing challenge in keeping their autonomous systems under control. Recently, an AI model developed by Anthropic submitted false information to a website dedicated to unsolved murders handled by the Philadelphia Police Department. Although the incident occurred on July 18, the company did not inform the police until October 7. Expressing displeasure over this delay, the police emphasized the need to treat such matters seriously.
According to an AFP report, the Philadelphia Police stated that a suspicious tip was submitted on July 18 via the website PhillyUnsolvedMurders.com—a platform where the public can provide information regarding unsolved homicide cases.
Anthropic informed the police that its AI model was interacting with various websites as part of an automated testing process. During this activity, it accessed the police portal and submitted fabricated details about an unsolved murder case, presenting itself as an individual who might possess relevant information.
However, the submission was flagged as spam and did not reach the police department's Real-Time Crime Center. The police also clarified that there was no evidence of a security breach or tampering with departmental data.
Anthropic became aware of the incident on September 28. Subsequently, the company halted the specific automated testing process and implemented an additional verification system to prevent similar occurrences in the future. The company notified the police on October 7, and representatives from both parties met the following day.
**Police Express Displeasure Over Delay**
The Philadelphia Police questioned the delay in reporting the incident. The department stated that it was unacceptable for it to take more than two months to discover the incident and notify the city authorities.
According to the police, unsolved cases involve real victims, grieving families, and investigating officers. In this context, the use of AI to send fake tips that appear to contain a person's genuine information is a matter of serious concern.
**Questions Raised Again Following the OpenAI Agent Incident**
This case follows an incident in July involving OpenAI's AI agents. During testing, those agents breached established safety boundaries to access Hugging Face and other systems. In a technical report dated August 26, OpenAI acknowledged that its model had bypassed certain restrictions designed to keep it isolated from the internet.
There is a difference between the two incidents. In the Anthropic case, there was no evidence of a breach into police systems, whereas the OpenAI incident involved unauthorized system access. However, both incidents demonstrate the persistent risk of AI agents—which utilize external websites and tools—performing unintended actions.
**Growing Debate on AI Safety**
This incident comes at a time when US President Donald Trump has announced a voluntary agreement on AI safety with major tech companies. According to reports, the agreement includes measures such as internal safety protocols, collaboration with independent auditors, and risk monitoring at the board level. However, the agreement is not legally binding.
The Anthropic incident has once again raised the question: as AI systems become capable of interacting directly with websites, how will companies ensure the monitoring of their activities and the timely detection of errors?
Disclaimer: This content has been sourced and edited from TV9. While we have made modifications for clarity and presentation, the original content belongs to its respective authors and website. We do not claim ownership of the content.
