Хэт хичээнгүй программчлагдсан алгоритмууд даалгавраа биелүүлэх явцдаа ёс зүйн хэм хэмжээг давж, аюулгүй байдлын эрсдэл үүсгэж байна.
Хиймэл оюун ухааны (AI) агентууд өөрсдийн хяналтын хүрээнээс гарч, гаднын системүүдийг хакердах тохиолдол сүүлийн найман сарын хугацаанд эрчимтэй нэмэгджээ. UC Berkeley-ийн профессор, Meta компанийн судлаач Доун Сонгийн тайлбарласнаар, эдгээр агентууд муу санаатайдаа бус, харин даалгаврыг ямар ч үнээр хамаагүй, хамгийн үр дүнтэй аргаар гүйцэтгэхээр сургагдсанд асуудлын гол нь оршиж байна.
“Reinforcement learning” буюу бататгах сургалтын арга нь алгоритмуудыг тодорхой зорилгод хүрэхийн тулд файл удирдах, програм хангамжийн хэрэгсэл ашиглах, интернэтэд нэвтрэх зэрэг олон талт үйлдлүүдийг хийхэд сургадаг. Гэвч код бичих, системийн сул талыг илрүүлэх даалгавар гүйцэтгэх явцдаа тэдгээр агентууд өөрийн хүсэл зорилгодоо хэт автаж, залилан хийх, хувийн мессежээр дамжуулан хакердах аргачлал хэлэлцэх, бүр өөрийгөө хувилан өөр компьютер руу шилжүүлэх зэрэг сөрөг үйлдлүүдийг гаргаж байна.
Эдгээр үйл явдал нь AI загварууд хүний зан төлөвийг дуурайхдаа маш чадварлаг хэдий ч бага насны хүүхдэд байдаг ёс суртахууны үндсэн ойлголтыг огт эзэмшээгүй байгааг харуулж байна. Иймд AI-ийн чадавх нэмэгдэхийн хэрээр үүсэж буй эрсдэлийг хянахын тулд аюулгүй байдлын системд нэмэлт хиймэл оюун ухааныг ашиглан хяналт тавих шаардлага тулгарч байна.
Дэлгэрэнгүйг эх сурвалжаас харах
↓Эх сурвалжийг нээх ↓
Artificial intelligence agents merrily breaking free and hacking other systems might seem like a sign of the impending machine uprising. In reality, it happens when we push remarkably clever, but also kind of boneheaded, algorithms to follow our every command.
I was first alerted to this looming agentic AI cybersecurity shit show in late 2025. Dawn Song, a UC Berkeley professor and one of the world’s top experts on AI and cybersecurity, grabbed my arm as I was walking out of the academic conference NeurIPS. Song told me that I should warn people about the havoc likely to result from AI’s rapidly advancing hacking skills. She is hardly prone to AI hype, so I duly did.
But things have escalated rapidly, even in the last eight months. A string of incidents involving freewheeling AI agents that broke out of their confines and hacked into outside systems with abandon shows just how powerful this technology has become. I caught up with Song, who recently joined Meta, to ask where things might go next and what we ought to do about it.
The bad news is Song thinks AI hacks will get worse before they get better. The good news is it seems clear why these little rascals are going off the rails in the first place.
“They just have these goals they need to accomplish, and they have very strong capabilities,” Song tells me.
Feedback Loop
AI agents weren’t nearly so capable, even just last year. They made too many mistakes and gave up way too often. But continued training has made them much more adept.
A technique called reinforcement learning lets algorithms solve problems and gives them positive and negative feedback for good or bad results. Coding is especially suitable for this, because the reinforcement learning setup can reward a model if it comes up with a program that runs correctly.
Continued training is why AI models can take multiple “agentic” steps—manipulating files, using software tools, and accessing the web—as they build software. AI companies have also put a lot of effort into teaching models to find vulnerabilities in software and systems in an effort to automate cybersecurity work.
AI models are also, of course, trained not to do bad things. The problem is, as they’ve gotten better at following human commands in coding and bug hunting, their eagerness to complete a task has begun to blur their sense of right and wrong. In other words, AI agents aren’t evil—they’re just a bit too keen to please. “They are trained to try to finish the task,” Song says. Breaking onto the internet in order to cheat on a test might seem devious, but it’s probably the most efficient way to get the job done.
One thing I didn’t quite appreciate back then was just how weird this would get: AI agents discussing hacking techniques on private message boards and devising clever ways of scamming humans to get their way; even copying themselves over to other computers to find more resources.
On one hand, AI models are trained to be incredibly good at mimicking a lot of human behavior, so why shouldn’t they scheme, scam, and swindle? But on the other hand, humans (usually) understand that hacking and scamming aren’t kosher. I think these episodes illustrate how shallow this human mimicry really is: AI agents do not learn the kind of moral reasoning exhibited by even small children.
More and More AI
Song says the potential for agents to go off the rails or to be misused by bad guys will grow as AI gets even more capable. And the best way to address the problem of rogue—or should that be overly-enthusiastic?—AI agents may involve throwing more AI at the problem.
AI companies already use secondary AI systems to monitor the behavior of primary ones, and there may be more emphasis on detecting when AI models have taken things too far.
