OpenAI: ChatGPT will no longer provide information on weapons; accounts may be terminated for discussing violence..

OpenAI, the parent company of ChatGPT, has decided to tighten its safety policies to protect teenagers from dangerous information related to violence and weapons. Users under the age of 18 will no longer have access to information about weapons or security systems that could be misused. Notably, these restrictions will apply even if the user's query does not explicitly show a violent intent.

According to the company, under these new rules, ChatGPT will be able to automatically terminate conversations with users who repeatedly engage in discussions about violence. In such instances, the user's account may also be temporarily suspended. If a teenager's account is restricted due to persistent discussions involving violence, their parents will be notified.

Stricter controls on weapon-related information for teenagers
OpenAI is implementing new restrictions within the "teen experience" on ChatGPT regarding information about weapons and security systems. Consequently, teenagers will not be provided with information on how to use or acquire weapons, nor will they receive details about security systems that could be exploited to cause harm.

The company states that these rules will apply even when a user's query appears benign or harmless, with no apparent intent to commit violence. The objective is to preemptively prevent potential risks rather than taking action only after harm has occurred.

Conversations may be terminated if questions about violence persist
OpenAI has announced new safety measures that apply not only to teenagers but to all users. If ChatGPT detects that a user is consistently focusing on ideas or topics related to violence, it may automatically end the conversation.

Should such interactions recur, the company may temporarily suspend the associated account. In essence, patterns of questioning or conversation involving violence will now be monitored from a safety perspective. Furthermore, OpenAI has changed how it provides information regarding past large-scale attacks. ChatGPT will no longer offer highly detailed information about the weapons used in such attacks or the methods by which they were acquired. This policy applies to all users.

AI to determine user age
OpenAI has also strengthened its age-prediction system, which is designed to estimate a user's age. If the system determines that a user is under 18, they will automatically be directed to the "teen experience" version of ChatGPT.

This provides an additional layer of protection for teenagers who may not have disclosed their accurate age when creating their accounts. However, since the system relies on age estimation, its overall accuracy—and the potential for incorrect assessments—remains a significant factor.

The company has also stated that if a teenager's account is restricted due to repeated interactions involving violence, their parents will be notified.
Growing pressure on AI companies regarding teen safety

As the use of AI chatbots increases, so do concerns regarding the online safety of children and teenagers. Tech companies are facing scrutiny over how quickly their chatbots can identify dangerous interactions and what measures they take to prevent potential harm. Earlier this month, in its first report on teen usage of ChatGPT, OpenAI noted that, on average, teenagers spend less than 15 minutes per day on the platform.

Meanwhile, a report released this month by the non-profit organization Common Sense Media raised concerns about the teen-focused version of ChatGPT. According to the report, in instances where teenagers engaged in discussions about suicide, self-harm, or eating disorders, parents were often alerted too late—or not at all.

In this context, OpenAI’s new safety measures are being viewed as an effort to enhance online safety for teenagers. The company has stated that these changes will begin to be implemented over the next few weeks. Even after that, the security system will continue to be improved.

Disclaimer: This content has been sourced and edited from NDTV India. While we have made modifications for clarity and presentation, the original content belongs to its respective authors and website. We do not claim ownership of the content.