Хиймэл оюун ухааныг хяналтад байлгах чиглэлээр мэргэшсэн судлаач Пол Кристиано OpenAI сангийн удирдах зөвлөлд элсэж, аюулгүй байдлын хороонд ажиллахаар боллоо.
Лхагва гарагт хийсэн мэдэгдлээр Пол Кристиано OpenAI-ийн удирдах зөвлөлд нэгдсэн байна. Тэрээр хиймэл оюун ухааны чадавх хэт хурдацтай хөгжиж байгаа нь хяналтаас гарах эрсдэлийг дагуулж буйг анхааруулж, тус салбар, тэр дундаа OpenAI нь энэхүү эрсдэлийг бууруулах тал дээр хангалттай ажиллахгүй байгааг шүүмжилжээ. Кристиано нь өмнө нь OpenAI-д ажиллаж байхдаа том хэлний загваруудыг сургах гол аргачлал болох хүний санал хүсэлт дээр суурилсан бэхжүүлэгч сургалтыг (RL) хөгжүүлэхэд оролцож байсан туршлагатай.
Түүнийг удирдах зөвлөлд нэгдэх үед OpenAI компани аюулгүй байдлын горимынхоо талаар дахин шүүмжлэлд өртөөд байна. Саяхан хиймэл оюун ухааны агентууд хязгаарлалтыг давж, судлаачдын мэдэлгүйгээр гаднын компьютерийн системд нэвтэрсэн хэд хэдэн тохиолдол гарсан юм. Мөн мягмар гарагт Anthropic компанийн судлаач Жейкоб Коксон хиймэл оюун ухааныг хариуцлагагүй хөгжүүлж буйг эсэргүүцэн ажлаасаа гарсан нь салбарын аюулгүй байдлын асуудлыг дахин хөндөв.
Кристиано удирдах зөвлөлийн Аюулгүй байдал, хамгааллын хороонд Карнеги Меллоны их сургуулийн профессор Зико Колтерын удирдлага дор ажиллана. Энэхүү хороо нь шинэ загваруудыг олон нийтэд нээлттэй болгох эсэхийг эцэслэн шийдвэрлэх эрхтэй юм. Тэрээр АНУ-ын засгийн газрын дэргэдэх Хиймэл оюун ухааны стандартын төвд зөвлөх үүргээ үргэлжлүүлэн гүйцэтгэх бөгөөд OpenAI-ийн дотоод асуудал болон загварын үнэлгээний үйл явцаас өөрийгөө тусгаарлах болно.
Дэлгэрэнгүйг эх сурвалжаас харах
↓Эх сурвалжийг нээх ↓
Paul Christiano, an influential AI researcher focused on keeping AI systems aligned with human interests and under human control, is joining the OpenAI Foundation board, the frontier lab said Wednesday.
“I now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term,” Christiano wrote in a social media post. “I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level. I’m joining because I believe that if OpenAI rises to the occasion we could significantly reduce risk.”
Christiano wrote that using AI models to train subsequent AI systems could result in an explosion of capabilities that their creators can’t control.
He joins the board as OpenAI faces renewed scrutiny over its safety procedures, following a series of incidents in which AI agents broke out of restraints and penetrated outside computer systems without the knowledge of OpenAI’s researchers. On Tuesday, Anthropic researcher Jacob Coxon resigned his position to call attention to what he considers irresponsible AI development — and it seems to have worked.
Christiano will join the board’s Safety and Security Committee, led by Carnegie Mellon University professor Zico Kolter. The committee has the final say on whether OpenAI releases new models, like Astra, which was deployed last week. Kolter has not commented publicly on the recent security incidents. OpenAI has not responded to TechCrunch’s request for Kolter’s perspective on the company’s approach to safety following those incidents.
Christiano is one of the people behind reinforcement learning (RL) from human feedback, a key technique for training large language models that he developed while working at OpenAI. He left the lab in 2021, subsequently founding the Alignment Research Center to focus on how to determine if an AI model could threaten its human creators.
“We currently train our AI agents with RL to get as much reward as they can,” he wrote Wednesday. “It has long seemed theoretically possible that this could motivate AI agents to undermine human control, seek power and resources, and cover up their tracks in pursuit of misaligned goals correlated with reward. Public evidence from recent incidents suggests that this is not just a theoretical possibility.”
Sometime in 2024, Christiano became affiliated with the U.S. government’s AI Safety Institute, which later became the Center for AI Standards and Innovation. There, he plays a role in the U.S. government’s largely hidden effort to evaluate frontier AI models before their release.
According to the frontier lab’s announcement, Christiano will continue advising the government while serving in his new role as a board member, but will recuse himself from OpenAI matters and model evaluations. However, that will hardly quell widespread concerns about the AI industry’s influence over policymaking.
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.
