Anthropic болон METR байгууллагын туршлагатай мэргэжилтнүүд аж ахуйн нэгжүүдэд зориулсан хиймэл оюун ухааны аудитын шинэ стандартыг танилцууллаа
Anthropic-ийн ажилтан асан Рүн Квист болон METR-ийн үйл ажиллагаа хариуцсан захирал асан Ражив Даттани нар хиймэл оюун ухааны агентуудын эрсдэлийг хянах зорилготой Artificial Intelligence Underwriting Company (AIUC) стартапыг үүсгэн байгуулжээ. Тус компани нь аж ахуйн нэгжүүдийн ашиглаж буй хиймэл оюун ухааны загваруудад гуравдагч талын аудитын болон баталгаажуулалтын давхаргыг бий болгохыг зорьж байна. Одоогоор Cursor, Lovable, Harvey, ElevenLabs зэрэг компаниуд AIUC-ийн үйлчилгээг ашиглаж байна.
AIUC мягмар гарагт Ribbit Capital-аар удирдуулсан 40 сая долларын А цуврал санхүүжилтийг амжилттай татан төвлөрүүлснээ зарлалаа. Үүнээс өмнө тус стартап 15 сая долларын үрийн санхүүжилт босгож байсан бөгөөд нийт санхүүжилтийн хэмжээ 55 сая долларт хүрээд байна. Хөрөнгө оруулагчдын дунд Nat Friedman-ийн NFDG сан, Emergence, Terrain болон Anthropic-ийн үүсгэн байгуулагч Бэн Манн нар багтжээ.
Тус компани кибер аюулгүй байдлын SOC 2 стандартыг загварчлан, AIUC-1 хэмээх шинэ стандартыг боловсруулсан байна. Энэхүү стандартын дагуу агентуудыг 5000 орчим сорилтоор шалгаж, хиймэл оюун ухааны “jailbreak”, хий үзэгдэл (hallucination) болон мэдээллийн алдагдлаас урьдчилан сэргийлэх 100 орчим хуудас бүхий дэлгэрэнгүй тайланг гаргадаг. Шалгалтын явцад хиймэл оюун ухаан ашигладаг хэдий ч эцсийн аудитын дүгнэлтийг хүний оролцоотойгоор баталгаажуулдаг байна.
Энэхүү шийдэл нь байгууллагуудад өөрсдийн ашиглаж буй хиймэл оюун ухааны агентуудын аюулгүй байдлын түвшинг бие даасан байдлаар үнэлэх боломжийг олгож байна. Аж ахуйн нэгжүүд хиймэл оюун ухааныг нэвтрүүлэхдээ аюулгүй байдлын баталгаа шаарддаг тул AIUC-ийн үйлчилгээ нь зах зээлд чухал ач холбогдолтой гэж үүсгэн байгуулагчид үзэж байна. Anthropic-ийн гүйцэтгэх захирал Дарио Амодей ч мөн адил салбарын хэмжээнд гуравдагч талын аюулгүй байдлын үнэлгээг нэвтрүүлэх шаардлагатай байгааг онцолсон юм.
Дэлгэрэнгүйг эх сурвалжаас харах
↓Эх сурвалжийг нээх ↓
A day after Anthropic researcher Jacob Coxon quit his job over concerns that AI could kill us all by the end of the decade, I met with founders and brothers-in-law Rune Kvist and Rajiv Dattani. They think they have a solution that could save us all, or at least help prevent AI agents from going rogue inside enterprises.
“AI is getting smarter at an increasingly rapid rate. The surprising thing about AI is that it becomes harder to adopt and harder to control as AI gets smarter, not easier,” said Kvist, an early Anthropic employee who is also married to Dattani’s sister). Dattani is the former COO of the AI safety research organization METR.
The pair launched a startup called Artificial Intelligence Underwriting Company (AIUC) that hopes to bring AI safety to enterprises and companies building AI models and agents. The startup names Cursor, Lovable, Harvey, and ElevenLabs as customers.
On Tuesday, AIUC announced a $40 million Series A led by Ribbit Capital, with participation from First Harmonic. It previously closed a $15 million seed round from Nat Friedman through his fund NFDG along with Emergence, Terrain, and Anthropic co-founder Ben Mann, among others, bringing its total funding to $55 million.
What caught the attention of this A-list group of investors is AIUC’s attempt to apply a familiar cybersecurity model to a new set of AI risks. The company has built a third-party audit and certification layer for AI agents.
“Banks, hospitals, governments and militaries no longer decline to deploy AI because a model isn’t smart enough,” Kvist said. “They decline because they’ve made commitments to their own customers about what a system will and won’t do, and nobody can currently guarantee that.”
Using the widely adopted cybersecurity standard SOC 2 as its muse, AIUC has developed a standard called AIUC-1 and a testing service to validate agents against the standard.
To build the standard, AIUC assembled a consortium of about 250 security and risk leaders — the buyers of agents. “These are the people who we meet with on a monthly basis, and the question we ask them is: When you’re buying agents from someone, what would you look for? What are the questions you’d want to ask, and what would you want to see addressed?” Dattani told TechCrunch.
That feedback shapes the tests. The startup then runs an agent through a suite of some 5,000 tests to see how it behaves in scenarios involving jailbreaks, hallucinations, and data leaks. The results produce a roughly 100-page report detailing where an agent performs safely and reliably — and where it doesn’t. Interestingly, AIUC uses AI agents to run the tests and AI to analyze the data. Humans, however, verify the final audit, Kvist said.
If this sounds a bit familiar, it is. Dattani’s former employer METR, where he was COO from 2024 to 2025 and remains a board member, does similar testing for the frontier labs, though its work until recently has focused mostly on performance (whether agents can reliably complete tasks). METR was one of the independent research orgs OpenAI used to investigate its Hugging Face incident.
Anthropic CEO Dario Amodei has also recently called for the AI industry to pace frontier development, citing a rapid increase in bad-behavior incidents. In his post, Amodei floated the idea of requiring frontier labs to use embedded third-party evaluators to observe and verify safety, and named METR as one possibility.
While AIUC isn’t proposing to embed itself at customer sites, the overall idea is similar: give enterprises an independent assessment of how safe their AI agents are. “Here’s where it passes and where you can trust it. And here’s where there’s concerns. You should be aware of those references before you make the decision to buy,” Dattani said.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
