Хиймэл оюун ухаан ашиглан хүний зөвшөөрөлгүйгээр нүцгэн зураг үүсгэдэг аппликейшнуудыг платформдоо байршуулж, ашиг олж буй үйл ажиллагаагаа зогсоохыг Сан-Франциско хотын прокурорын газраас шаарджээ.
Сан-Франциско хотын прокурор Дэвид Чиу Apple болон Google компаниудад албан ёсны мэдэгдэл хүргүүлж, хэрэглэгчийн зөвшөөрөлгүйгээр нүцгэн зураг үүсгэдэг 13 аппликейшныг өөрсдийн дэлгүүрээс нэн даруй устгахыг шаардлаа. Прокурорын үзэж буйгаар, эдгээр технологийн аварга компаниуд уг хор хөнөөлтэй аппликейшнуудаас сая сая долларын шимтгэл авч, тэднийг дэмжих замаар бэлгийн хүчирхийллийг хөхиүлэн дэмжиж байна. Калифорнийн хуулийн дагуу гүнзгий хуурамч (deepfake) порнограф агуулга бүхий үйлчилгээг дэмжихийг хориглодог аж.
Судлаачдын олж тогтоосноор, Apple болон Google-ийн дэлгүүрүүдэд хиймэл оюун ухаан ашиглан бэлгийн шинж чанартай зураг үүсгэдэг олон аппликейшн байсаар байгаа бөгөөд зарим нь хүүхдэд зориулсан гэсэн ангилалтай байжээ. Эдгээр аппликейшн нь хэдхэн секундын дотор хүний зургийг ашиглан нүцгэн эсвэл бэлгийн шинж чанартай дүр зураг үүсгэх чадвартай бөгөөд сургуулийн насны хүүхдүүдийн дунд ийм төрлийн дарамт үүсгэж буй тохиолдлууд олон улсад бүртгэгдээд байна.
Google-ийн төлөөлөгч Дан Жексоны мэдэгдснээр, тус компани бодлого зөрчсөн олон зуун аппликейшныг устгаж, “nudify” гэх мэт хайлтын үр дүнг хязгаарлах арга хэмжээ авчээ. Гэсэн хэдий ч технологийн компаниуд өөрсдийн платформ дээрх контентын хяналтыг сайжруулж, ийм төрлийн аппликейшн гарч ирэхээс урьдчилан сэргийлэх үүрэгтэй гэж Дэвид Чиу онцолсон байна. Apple компани энэ асуудлаар одоогоор албан ёсны тайлбар өгөөгүй байна.
Дэлгэрэнгүйг эх сурвалжаас харах
Эх сурвалжийг нээх ↓
Apple and Google have been ordered to take down apps that can “nudify” or “undress” people and told that they must stop profiting from the harmful technology, according to cease-and-desist letters sent to the companies seen by WIRED.
On Thursday, San Francisco city attorney David Chiu sent legal notices to Apple and Google demanding that they remove from their app stores 13 face-swapping apps, which allow users to create AI-generated nonconsensual nude images. The letters say the Silicon Valley giants should stop “aiding and abetting” the sale of explicit deepfake images and “sever” business relationships with the app developers.
“Generating non-consensual intimate images is illegal, harmful, and completely unacceptable,” Chiu tells WIRED. The city attorney, whose office previously took legal action against 16 popular deepfake websites, says Apple and Google have likely “made millions of dollars in fees” from apps that offer nudification, and they should improve their moderation processes to stop them appearing in their stores in the first place.
“These companies have responsibility to ensure that apps on their platforms do not facilitate sexual abuse,” Chiu says. The city’s legal letters say California’s laws prohibit supporting services that create deepfake pornography. The apps use in-app payments, which the tech companies take a cut of, the letters says. “The fact that some of the world’s largest and most established technology companies are facilitating this has to stop.”
Researchers have repeatedly found and reported apps in Apple’s App Store and Google’s Play Store that allow people to generate sexual images using AI—including some apps being rated as suitable for use by children. While new laws and bans aim to tackle the scourge of explicit deepfakes online, technology and social media companies consistently direct millions of people toward the harmful tech.
Both Apple and Google have developer policies that prohibit pornography, abuse, and harassment on their platforms. They have previously removed dozens of nudify and deepfake apps, after reports by researchers and journalists.
Google spokesperson Dan Jackson tells WIRED that the company has deleted “hundreds” of apps with nudifying features for policy violations, including the five Android apps flagged by Chiu’s office, among other steps to restrict access to them.
“Google Play does not allow apps that contain sexual content, and we continually take proactive steps to detect and remove apps with harmful content,” Jackson says in a statement. “When violations are reported to us, we investigate and take swift action, which in the case of these apps has included suspending hundreds of violating apps and restricting related search terms like ‘nudify’ on our store.”
Apple did not provide comment ahead of publication.
Over the last five years, a highly lucrative slurry of deepfake “nudification” tech has emerged online—most transparently with xAI’s Grok being used to create millions of sexualized images in January. A host of apps, websites, and bots allow people (mostly men) to upload pictures of people (overwhelmingly women and girls) and digitally “remove” clothing or place them into graphic sexual scenarios.
Often all it takes to create sexual deepfakes is a reference photo and a couple of clicks, with some results available in seconds. Images and videos have become more realistic as the underlying generative AI technology has improved, with services providing some results for free or charging small fees to create the harmful content. Previous reporting by WIRED and Indicator Media has uncovered incidents in at least 90 schools where deepfake sexual abuse images have been created of minors.
