Anthropic launches a program fund the development of new types of benchmarks capable of assessing the performance and impact of AI models, including generative models like his own Claude.
Unveiled Monday, Anthropic’s program will provide payments to third-party organizations that can, as the company explains in a blog post, “effectively measure the advanced capabilities of AI models.” Interested parties can submit applications, which will be evaluated on a rolling basis.
“Our investment in these assessments aims to improve the entire field of AI safety, providing valuable tools that benefit the entire ecosystem,” Anthropic wrote on its official blog. “Developing high-quality, relevant safety assessments remains a challenge, and demand outstrips supply.”
as we have Underlines Previously, AI had a benchmarking problem. The most commonly cited benchmarks for AI fail to capture how the average person actually uses the systems under test. It’s also questionable whether some benchmarks, especially those published before the advent of modern generative AI, even measure what they’re supposed to measure, given their age.
The very high-level and more difficult-than-it-seems solution proposed by Anthropic is to create ambitious benchmarks with a focus on AI safety and societal implications through new tools, infrastructures and methods.
The company is calling for tests that assess a model’s ability to perform tasks such as conducting cyberattacks, “enhancing” weapons of mass destruction (e.g. nuclear weapons), and manipulating or deceiving people (e.g. through deepfakes or disinformation). For AI risks related to national security and defense, Anthropic says it is committed to developing some kind of “early warning system” to identify and assess risks, though it doesn’t reveal in the blog post what such a system might entail.
Anthropic also says its new program aims to support research on “end-to-end” benchmarks and tasks that probe AI’s potential to aid scientific study, converse in multiple languages, and mitigate entrenched biases and the toxicity of self-censorship.
To achieve this, Anthropic is considering new platforms that would allow subject matter experts to develop their own assessments and conduct large-scale testing of models involving “thousands” of users. The company says it has hired a full-time coordinator for the program and that it could buy or expand projects that it believes have the potential to grow.
“We offer a range of funding options tailored to the needs and stage of each project,” Anthropic wrote in the post, though an Anthropic spokesperson declined to provide further details on those options. “Teams will have the opportunity to interact directly with Anthropic’s domain experts from Frontier Red, Focus, Trust & Safety, and other relevant teams.”
Anthropic’s efforts to support new AI benchmarks are commendable, provided of course that there are enough funds and personnel to support them. But given the company’s commercial ambitions in the AI race, it might be hard to trust it entirely.
In the blog post, Anthropic is quite transparent about the fact that it wants some of the evaluations it funds to be aligned with the AI Safety Classifications he developed (with help from third parties like the nonprofit AI research organization METR). This is well within the company’s prerogative. But it can also force program applicants to accept definitions of “safe” or “risky” AI that they may not agree with.
Some in the AI community are also likely to take issue with Anthropic’s references to “catastrophic” and “misleading” risks of AI, such as those associated with nuclear weapons. Many experts Researchers say there is little evidence to suggest that AI as we know it will acquire world-destroying capabilities and surpass humans in the near future, if ever. Claims of imminent “superintelligence” only serve to distract from today’s pressing AI regulatory issues, such as hallucinatory trends, these experts add.
In its article, Anthropic writes that it hopes its program will serve as a “catalyst for progress toward a future where comprehensive AI assessment is an industry standard.” It’s a mission that many open, unaffiliated company Efforts to create better AI benchmarks can relate. But it remains to be seen whether those efforts are willing to join forces with an AI vendor whose loyalty ultimately lies with shareholders.
