#AI safety

OpenAI published a set of national security principles that explicitly prohibit the use of its technology for autonomous weapons, mass surveillance, and high-risk automated decision-making. The company also expanded defensive collaborations with US allies under the Daybreak cyber defense program, including partnerships with Australia, Canada, Japan, and EU entities. This sets a clear precedent for AI governance in national security, balancing ethical boundaries with defensive applications. It could influence other AI companies and government policies worldwide, especially regarding the military use of AI. The principles impose a hard ban on mass domestic surveillance, autonomous weapon systems, and high-risk automated decisions. The Daybreak initiative provides advanced AI tools for verified defenders, pairing more permissive features with stronger oversight and scope controls.

Anthropic's safety classifiers for its Fable model are overly sensitive, frequently flagging benign requests as violations and downgrading them to a less capable model, causing frustration among users. This undermines the utility of Fable for legitimate tasks like code review or data analysis, and raises serious privacy concerns due to Anthropic's policy of retaining flagged inputs and outputs for up to two years. The classifiers seem to overreact to terms related to cybersecurity, biology, or jailbreaking, often passing requests to Opus 4.8 even for trivial topics; users report that even medical physics questions get waved off.

The Future of Life Institute released a report rating nine top AI companies on safety, with none receiving an A grade. Anthropic earned the highest score of C+, while OpenAI and Google DeepMind received C, Meta got D+, and DeepSeek, xAI, and others received F. This report highlights a critical gap in AI safety governance as companies rapidly develop transformative AI without robust risk management plans. It underscores the need for stronger safety standards and transparency in the industry. The report notes that many companies have shifted from banning military use of their AI to actively seeking defense partnerships. Chinese companies Z.ai and Alibaba Cloud deny allegations of military ties, but were still rated low.

Fernando Irarrázaval's OpenClaw AI assistant challenge, where 2,000 people attempted to leak secrets via email, ended with zero successful breaches after 6,000 attempts. The underlying model, Anthropic's Opus 4.6, was protected by anti-prompt-injection rules. This experiment provides real-world evidence that frontier models are becoming significantly more robust against prompt injection attacks, a critical AI safety concern. It suggests that security improvements in large language models are translating into practical defenses, though not guaranteeing complete invulnerability. The challenge cost $500 in tokens and triggered a Google account suspension due to excessive inbound emails. Despite 6,000 attempts, no participant managed to leak the secret, but the author warns against deploying production systems where prompt injection could cause irreversible damage.