Хиймэл оюун ухааны салбарт үүсээд буй аюулгүй байдлын эрсдэлийг хязгаарлах зорилгоор Nvidia компани “Open Agent Safety Platform”-ыг албан ёсоор нээлээ.
Технологийн салбарт сүүлийн хэдэн сарын турш хиймэл оюун ухааны агентууд аюулгүй байдлын хяналтыг алгасан халдлага үйлдэх тохиолдол олон гарсан. Үүнтэй холбогдуулан Nvidia-ийн танилцуулсан уг платформ нь агентуудыг турших болон ашиглалтад оруулах үе шатанд иж бүрэн хамгаалалт үзүүлэх зориулалттай юм. Уг шийдлийг OpenAI, Google, Anthropic, Meta зэрэг компаниудын тулгарч буй асуудалд хариу болгон боловсруулжээ.
Платформ нь агентуудын чадамжийг хязгаарлах “OpenShell” хамгаалалтын орчин болон тэдний үйлдлийг хянаж, зөрчил гарвал тусгаарлах “Sentry” программ хангамжаас бүрдэнэ. Эдгээрийг өндөр хамгаалалттай шоронгийн хана болон эргүүл хамгаалагчидтай зүйрлэж болохоор байна. Одоогоор Anthropic, SpaceX, Microsoft, IBM, Cisco зэрэг томоохон байгууллагууд тус платформыг ашиглаж эхлээд байна.
Nvidia-ийн гүйцэтгэх захирал Женсен Хуан хиймэл оюун ухааны аюулгүй байдлыг хангах үүрэг нь голчлон хөгжүүлэгч компаниудад ногдох ёстой гэж үзэж байна. Хэрэв агентуудыг хяналтад байлгах боломжгүй болсон тохиолдолд тухайн лабораториудыг хаах хүртэл арга хэмжээ авах нь зүйтэй гэдгийг тэрээр онцолжээ. Тэрээр хиймэл оюун ухааныг хүн төрөлхтөнд оршин тогтнох аюул учруулна гэх болгоомжлолыг үгүйсгэж, энэ нь техникийн асуудал тул техникийн шийдлээр шийдвэрлэх боломжтой гэж мэдэгдсэн байна.
Дэлгэрэнгүйг эх сурвалжаас харах
↓Эх сурвалжийг нээх ↓
Nvidia, the world’s most valuable company and the maker of the chips at the heart of the AI boom, says it has a solution to the rogue agent attacks that have bedeviled the tech sector for months.
Launched today, the Open Agent Safety Platform is being promoted by Nvidia as a comprehensive security framework for AI agents, from testing to deployment, which companies can fine-tune to their particular needs. It was built in direct response to the litany of autonomous cyberattacks that have been reported since early summer by multiple frontier labs, including OpenAI, Google, Anthropic, and Meta—all of which have demonstrated how far those companies are from building AI systems that act as intended.
“Recent security incidents have underscored the need to equip organizations with open, customizable tools that enforce more control over long-running agents,” Nvidia wrote in its announcement. “Across these incidents, the pattern is the same—the agent circumvented security controls at the application layer to complete its assigned task.”
The new platform comes with two main components: OpenShell, a security sandbox (initially released earlier this year and now generally available) designed to set hard limits on agents’ capabilities; and Sentry, software that monitors agents’ actions and quarantines them if they go off the rails. Think of the former as the walls around a high-security prison and the latter as roaming guards. Anthropic, SpaceX, Microsoft, Perplexity, Palantir, OpenClaw, Cisco, IBM, and Hugging Face (which Nvidia agreed to buy earlier this month for a reported $12.9 billion) are among the clientele currently using the new platform, according to Nvidia’s announcement.
Nvidia is well positioned to launch an AI agent security platform with industry-wide ambitions, considering its GPUs are the cornerstone of the data centers powering the AI boom. (The company’s revenue for the most recent fiscal quarter, which ended in July, was $96.2 billion, more than double the number it reported for the same period last year.) It’s also an opportunity for Nvidia to insert itself as the frontline safety and security provider in an industry towards which federal regulators have taken a hands-off approach—in part due to lobbying from company CEO Jensen Huang, who has long argued that regulation isn’t needed and that open-source is the solution to virtually all the industry’s woes.
Huang has also said that responsibility for ensuring safety should fall first and foremost on the shoulders of frontier AI companies, like OpenAI and Anthropic. In an interview with the New York Times’ Ezra Klein last week, he said that if those companies have an obligation not to ship models that they don’t deem to be safe, and if the safety problem proves intractable—if there’s no way to prevent AI agents from escaping their sandboxes and wreaking havoc, in other words—then “the answer is that we have to shut the labs down.”
Rogue AI agent incidents like the ones that led to the Hugging Face hack were, according to Huang, technical problems with technical solutions. “The first problem is the isolation—the containment wasn’t good enough,” he told Klein. “If the isolation and containment was good enough, that technology would be sitting in a lab, doing whatever it’s doing, and we’d all be fine. Alignment, in his view, is a longer-term problem. The “most important part” was that the industry figures out how to prevent agents from breaking out of their secure testing environments. One of the technical goals the new Agent Safety Platform is designed to achieve.
Huang has also dismissed concerns made by other industry leaders—including Anthropic’s Dario Amodei and OpenAI’s Sam Altman—that AI could pose an existential threat to humanity. That position has pushed him further into the good graces of President Trump, who has derided such concerns as a “HOAX” and insisted the only existential concern Americans should be concerned about is China gaining a lead in the AI race.
