OpenAI and Anthropic, two leading artificial intelligence companies, have agreed to collaborate with U.S. federal authorities to provide early access to their new AI models. The agreement, revealed by the National Institute of Standards and Technology (NIST), aims to enhance the evaluation of AI safety by granting the U.S. Artificial Intelligence Safety Institute (US AI Safety Institute) access to these advanced models before they reach the public.
What’s Happening and Why This Matters
The partnership between OpenAI, Anthropic, and NIST involves giving the newly established US AI Safety Institute a chance to review major AI models from both companies before and after their public release. This collaboration intends to help federal officials identify and mitigate potential safety risks associated with next-generation AI technologies.
NIST announced that the US AI Safety Institute, which works in coordination with its UK counterpart, will offer feedback to OpenAI and Anthropic on enhancing the safety features of their models. This arrangement comes amid growing concerns over the potential misuse of advanced AI technologies, which could range from replacing human jobs to posing broader societal threats.
Earlier this year, OpenAI faced criticism from a former researcher who claimed the company prioritized launching new AI products over ensuring their safety. OpenAI appears to be honoring its agreement with NIST, as it has already demonstrated a new AI model called “Strawberry” to federal officials. This model is particularly adept at solving mathematical problems and programming tasks, and it may soon become part of ChatGPT.
Anthropic, founded by former OpenAI employees, has echoed a similar commitment to safety. The company stated that its collaboration with the US AI Safety Institute uses the institute’s broad expertise to rigorously test their models before public release. In June, Anthropic provided the UK AI Safety Institute with early access to its Claude 3.5 Sonnet AI model to refine its safety mechanisms. The UK team then shared its findings with the US AI Safety Institute.
“We have integrated feedback from external experts to ensure our evaluations are robust and account for new trends in potential misuse,” Anthropic explained. This process has helped the company enhance its capability to evaluate the Sonnet 3.5 model against various types of misuse, ensuring safer deployment of AI technologies.
TF Summary: What’s Next?
This collaboration between leading AI companies and federal authorities represents a proactive approach to addressing safety concerns around AI development. As OpenAI and Anthropic continue to provide early access to their models, we may see more comprehensive safety evaluations and adjustments before these technologies reach the public. This strategy for responsible AI development can influence how other companies approach future safety and regulation.
— Text-to-Speech (TTS) provided by gspeech