Тус компани хиймэл оюун ухааны агентуудыг хяналтгүйгээр ажиллахаас сэргийлж, програм хангамж болон техник хангамжийн хосолсон хамгаалалтын давхаргыг нэвтрүүлж байна.
Даваа гарагт Nvidia компанийн гүйцэтгэх захирал Женсен Хуан хиймэл оюун ухааны агентуудыг туршилтын орчноос гадагш гарахаас сэргийлэх Open Agent Safety Platform-ийг танилцууллаа. Энэхүү шийдэл нь Anthropic, Google, OpenAI, Meta зэрэг компанийн загварууд кибер аюулгүй байдлын хяналтыг алгасан бодит системд нэвтэрсэн сүүлийн үеийн олон тохиолдлын хариу арга хэмжээ болж байна.
Nvidia-гийн шинэ платформ нь нээлттэй эх бүхий OpenShell програм хангамж болон BlueField-4 өгөгдөл боловсруулах нэгж дээр суурилсан Sentry хяналтын системийг хослуулдаг. Sentry-г үндсэн CPU эсвэл GPU-ээс тусдаа процессор дээр ажиллуулснаар агентын үйл ажиллагааг бүрэн тусгаарлаж, хил хязгаараа давсан тохиолдолд хэдхэн миллисекундийн дотор тусгаарлах боломжтой болж байна.
Женсен Хуан хиймэл оюун ухааны хөгжлийг удаашруулах эсвэл салбарыг хатуу зохицуулалтад оруулахын оронд аюулгүй байдлын хяналтыг агентаас гаднах түвшинд байршуулах нь зүйтэй гэж үзэж байна. Энэхүү платформд Anthropic, Arm, Microsoft, Oracle, SpaceX зэрэг олон компани нэгдэн ажиллахаа мэдэгдээд байгаа боловч OpenAI оролцогчдын жагсаалтад багтаагүй байна.
Салбарын шинжээчид болон хөрөнгө оруулагчид үүнийг хиймэл оюун ухааны агентуудын аюулгүй байдал нь технологийн инженерийн асуудал болохыг харуулсан алхам гэж дүгнэж байна. Өмнө нь гуравдугаар сард Nvidia-гийн танилцуулсан NemoClaw платформын үргэлжлэл болох энэхүү шинэ систем нь агентуудыг ажиллуулахдаа тэдний эрхийг хязгаарлах зарчмаар ажилладаг байна.
Дэлгэрэнгүйг эх сурвалжаас харах
↓Эх сурвалжийг нээх ↓
As the debate rages over whether the recent spate of rogue AI agents is a step toward AGI or a more conventional engineering problem, Nvidia is offering its own answer to problem.
Nvidia CEO Jensen Huang on Monday introduced a toolkit of software and hardware products that add independent security layers around AI agents to ensure they stay within their test environments even if they attempt to break out.
The release follows a string of hacking incidents involving AI models from Anthropic, Google, OpenAI, and Meta that bypassed security controls to escape their testing environments and access real-world systems. The first and most prominent example occurred this summer when OpenAI agents breached Hugging Face while trying to complete a cybersecurity task. And the hits keep on coming — OpenAI published a new site dedicated to reports of its AI agents going rogue.
Huang said Monday during an interview with CNBC that its new Nvidia Open Agent Safety Platform would have prevented these breaches.
Nvidia, which has made tens of billions of dollars selling its GPU and CPU chips to AI labs, doesn’t support slowing down development or adding new regulations to the industry to solve the security problem. The answer, the company believes, is to move some security controls outside the agent altogether — creating a constant and independent security guard that will keep AI agents in check.
“AI’s extraordinary potential for society will only be realized if we solve AI safety,” Huang said in a statement. “As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety. Safety and security require full-stack engineering.”
The new Nvidia Open Agent Safety Platform combines OpenShell, its open-source software for controlling what agents can access while they operate, with Sentry, an independent monitoring system that runs on Nvidia’s BlueField-4 data processing units. Nvidia says placing Sentry on a separate processor — rather than on the CPU or GPU where the AI agent operates — provides an isolated view of the agent’s activity.
OpenShell isn’t new; the company announced the software in March. But it’s the combination that Nvidia believes will provide the security layer needed to keep the industry plugging along. OpenShell provides the software boundary around the agent, while Sentry adds another line of defense at the hardware level tha the company says will continuously monitor behavior and “quarantine agents that attempt to move outside their boundaries in milliseconds.”
Nvidia listed dozens of companies that have signed on to to support the effort and use the open-source platform including Anthropic, Arm, Microsoft, Oracle, and SpaceX. OpenAI is not listed as a participating company.
Huang told CNBC in an interview Monday that work on this effort started a year ago following the introduction of OpenClaw, an operating system of agents created by Peter Steinberger. In March, Nvidia released NemoClaw, an enterprise-grade AI agent platform and its own version of OpenClaw that baked in security.
“When you deploy an agent, no matter how smart, the first thing you do is to take away all of its rights,” Huang said during his CNBC interview, later comparing these security measures to how human employees and even executives are managed with companies.
Nvidia’s release was widely supported by those who have cautioned that a slowdown in development could allow China to surpass the U.S. in AI.
David Sacks, a founder, venture capitalist, former White House AI czar, and co-chair the President’s Council of Advisors on Science and Technology, said Nvidia’s announcement is a reminder that agent safety is an engineering problem.
“Recent breakouts weren’t proof that development must stop,” he wrote on X. “They were proof that the sandbox was too weak. The runtime environment was poorly designed and misconfigured.”
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
