Trump, AI CEOs Sign Voluntary Safety Pact as Pressure Grows Over Advanced AI Risks

US President Donald Trump and executives from some of the world's biggest technology companies have signed a voluntary agreement aimed at strengthening safety controls around increasingly powerful artificial intelligence systems, placing responsibility for much of the oversight directly with the companies developing the technology.

The agreement, announced after a White House meeting on September 29, brings together leaders from Google, Meta, OpenAI, Anthropic, Nvidia and xAI. It is formally described as the White House Accord on Super Intelligence: Joint Commitment on Frontier Responsibilities. Trump characterised the commitment as "morally binding", while reporting from the meeting indicates that it does not currently carry the force of law or impose government penalties for non-compliance.

The accord comes as governments, researchers and the technology industry grapple with the growing capabilities of so-called frontier AI models. These systems can perform increasingly complex tasks, interact with computer systems and, in some circumstances, act with a degree of autonomy that has raised concerns about cybersecurity, biological and chemical risks, data access and loss of human control.

Under the new framework, participating companies are expected to establish several layers of safeguards covering the development and deployment of their most advanced systems. The measures include internal controls to monitor AI capabilities and potential risks, an internal team responsible for checking those controls, independent external audits and an independent board committee to review safety findings.

The companies are also expected to work together regularly on safety standards and practices. The accord says the objective is to ensure that frontier models behave as intended and that potential problems are identified and addressed before they develop into larger risks.

The White House pact represents a notable shift toward voluntary corporate oversight rather than immediately imposing a new federal regulatory regime on America's leading AI developers.

Trump has repeatedly argued that excessive regulation could slow US technological development and weaken the country's position in its competition with China. His administration has generally emphasised accelerating AI investment while attempting to address security concerns through existing laws, executive action and industry commitments.

The latest agreement therefore attempts to address safety concerns without creating a comprehensive new regulatory system. Its supporters can point to the inclusion of independent auditors and board-level oversight as steps beyond simple corporate promises. But because participation is voluntary and the framework does not establish a statutory enforcement mechanism, questions remain about how consistently the commitments will be applied.

The Washington Post and other outlets reported that critics have questioned whether companies should be allowed to determine the terms of their own oversight, particularly when the same companies have strong commercial incentives to release increasingly capable systems quickly. The Guardian likewise noted that the agreement does not provide a legal enforcement mechanism and allows companies significant control over how the auditing process is conducted.

The debate is particularly significant because several of the companies involved are already developing systems that operate far beyond conventional chatbot functions. AI agents are increasingly being designed to browse the internet, write and execute code, interact with software and perform multi-step tasks on behalf of users.

That growing autonomy has changed the nature of the safety discussion. The question is no longer simply whether an AI system generates an inaccurate answer, but whether a highly capable system could gain access to sensitive systems, assist malicious actors or behave in ways its developers did not anticipate.

Anthropic, one of the companies involved in the agreement, has separately been developing its own frontier-safety policies and security research. Its published work includes efforts to strengthen protections against sophisticated attacks on AI infrastructure and to make model outputs more reliably attributable to specific model versions.

The White House meeting came alongside another significant move in the administration's AI policy. Trump signed an executive order directing the executive branch to use the term "Super Intelligence", or "SI", rather than "Artificial Intelligence" or "AI" in specified government communications.

The administration argues that the new terminology better reflects the rapidly advancing capabilities of the technology. The change applies to executive-branch terminology and does not by itself eliminate the established legal and technical use of the term artificial intelligence.

The terminology change has added another layer to an already intense debate over how the US should approach advanced AI. While the administration is promoting rapid development and investment, the new accord acknowledges that increasingly capable systems require stronger safety processes.

The companies involved also represent an unusually concentrated group of technology power. Google CEO Sundar Pichai, Meta CEO Mark Zuckerberg, Anthropic CEO Dario Amodei, OpenAI President Greg Brockman, Nvidia CEO Jensen Huang and xAI founder Elon Musk were identified as signatories.

That concentration means the agreement could influence safety practices well beyond the United States, particularly because the participating companies operate globally and their AI products are used by millions of people and businesses around the world.

For now, however, the pact remains a commitment rather than a new law. Its broader significance will depend on whether the companies actually implement meaningful independent testing, disclose serious safety failures and act on problems identified by auditors and their own oversight bodies. The agreement marks a growing recognition among governments and AI developers that increasingly powerful systems require stronger safeguards, but the effectiveness of voluntary self-regulation will ultimately be measured by what companies do when safety requirements collide with commercial pressure and the race to build the next generation of AI.