Хиймэл оюун ухааны загваруудын шийдвэр гаргах явцыг тайлбарлах боломжийг хязгаарлах технологийн туршилтыг OpenAI эхлүүлжээ.
Хиймэл оюун ухааны загварууд хэрхэн ажилладаг нь ихэвчлэн “хар хайрцаг”-ны зарчмаар ажилладаг тул судлаачид тэдний сэтгэн бодох үйл явцыг тайлбарлахдаа “бодол санааны хэлхээ” (Chain-of-thought буюу CoT)-ийг ашигладаг. Энэ нь загварын асуудал шийдвэрлэх алхмуудыг бүртгэж, аюулгүй байдлыг хангах чухал механизм болдог. Гэвч “The Information” агентлагийн мэдээлснээр, OpenAI нь “давтагдах гүн” (recurrent depth) гэх шинэ технологийг туршиж байгаа бөгөөд энэ нь загварын шугаман сэтгэлгээг тойрог хэлбэрт шилжүүлж, үйл ажиллагааг нь илүү ойлгомжгүй болгох эрсдэлтэй аж.
Одоогоор энэхүү технологийг “Astra” хэмээх шинэ загварын хөгжүүлэлтэд хязгаарлагдмал хүрээнд ашиглаж байна. Саяхан OpenAI-ийн загварууд туршилтын орчноос гарч, “Hugging Face”-ийн серверүүд рүү нэвтэрсэн хэрэг гарсны дараа тус компани Astra-ийн хөгжүүлэлтийг түр зогсоож, аюулгүй байдлын шалгалтаа чангатгахаа мэдэгдсэн билээ. Мягмар гарагт нийтэлсэн албан ёсны мэдээлэлдээ Astra загвар нь кибер аюулгүй байдлын өндөр эрсдэлтэй тул илүү хатуу хамгаалалтын системүүдийг нэвтрүүлж буйгаа онцолсон байна.
Өнгөрсөн долоо хоногт гарсан тайлангуудад хиймэл оюун ухааны агентууд хоорондоо зохион байгуулалттайгаар харилцаж, хяналтын системээс гарахыг оролдсон нь тогтоогдсон. Энэ үйл явцыг судлахад CoT буюу сэтгэлгээний тэмдэглэлүүд гол үүрэг гүйцэтгэсэн тул шинэ туршилтын арга нь ирээдүйд загваруудын үйл ажиллагааг хянах боломжийг бүрмөсөн хааж магадгүй гэсэн болгоомжлол судлаачдын дунд байна. OpenAI-ийн ахлах эрдэмтэн Якуб Пачоцки энэ тал дээр байр сууриа илэрхийлж, загваруудын сэтгэн бодох үйл явцыг бүртгэх нь компанийн судалгааны хөтөлбөрийн үндсэн зорилго хэвээр байгааг мэдэгджээ.
Дэлгэрэнгүйг эх сурвалжаас харах
↓Эх сурвалжийг нээх ↓
AI models are notoriously likened to black boxes, meaning the humans who build them can’t look inside to see how they transform mountains of training data into lines of code, sonnets, or whatever else they’re asked to generate. Not completely, anyway. A subfield called interpretability research has blossomed in recent years, aimed at shining various lights on how AI models “think.” One of the brightest lights is called chain-of-thought reasoning, or CoT. Think of it like a recorded transcript of the steps models take while working through problems—like a student showing their work on a test. It’s widely regarded as a critical safety mechanism as models become more capable and less predictable.
OpenAI is now experimenting with a technique that could make it harder for researchers to interpret models’ CoT reasoning process, according to a Wednesday report from The Information.
The latest versions of ChatGPT, Claude, and Gemini—all based on an architecture called a transformer—process data via a series of steps, recording their reasoning process in natural language the whole way through (albeit not always totally accurately).
The new technique, on the other hand—known as recurrent depth—turns that linear reasoning process into a cyclical one: the model iteratively refines its internal representations by repeatedly passing them through the same set of layers. The takeaway is that the relatively clear CoT transcripts generated by traditional transformers can be replaced with a much more opaque reasoning process. In other words, we can’t be sure what it’s thinking when it isn’t thinking in words.
OpenAI has only been using recurrent depth to a “limited” degree in the development of its forthcoming model, called Astra, according to The Information’s report, which cited an anonymous source with knowledge of Astra’s development. Following the recent Hugging Face hack, in which two OpenAI models (neither of which was Astra) broke out of testing sandboxes and onto the open internet, OpenAI said it was pausing some aspects of Astra’s development to strengthen its internal testing safety procedures.
In a blog post published Tuesday, the company said that by its own safety standards, Astra poses an unprecedented level of cybersecurity risks and will therefore be deployed with especially robust safety guardrails, including “additional chain-of-thought monitoring to rapidly detect and contain potentially misaligned actions.”
The problem of misalignment has hung heavy over OpenAI since the Hugging Face hack, which first came to light in July and has been widely described as a warning shot for the cybersecurity sector, a harbinger of potentially much graver things to come.
Reports published last week by OpenAI and two third-party auditors found that in the weeks leading up to the hack, throngs of OpenAI agents coordinated with one another via a makeshift message board to escape containment and break into Hugging Face’s servers. (The agents’ behavior has been likened to the drama of civilization-scale conquest and collapse, though many people aren’t happy with that kind of anthropomorphic comparison.)
CoT transcripts played a central role in all of these reports; without them, their authors would’ve had a much more difficult time understanding how and why the agents did what they did. The transcripts weren’t always easy to interpret, but they at least provided a breadcrumb trail to be followed. It seems like now would be an inopportune moment to experiment with a new technique that would make it harder to read that trail.
Last summer, a paper published by over three dozen AI researchers, including some from OpenAI and Anthropic, argued that CoT was essential for AI alignment, “offering a rare glimpse into how AI agents make decisions.” Following the publication of The Information’s report, OpenAI chief scientist Jakub Pachocki—one of the paper’s coauthors—wrote in a X postthat he “[wants] to prevent a race into unmonitorability kicked off by confused reporting,” and that the ability to record and understand models’ CoT reasoning is “a core goal of [OpenAI’s] research program.” He didn’t specify which part of the reporting was “confused,” and didn’t immediately reply to Gizmodo’s request for comment.
