OpenAI says Russian operatives used its AI chatbot to update their bosses, shedding light on influence campaigns previously not attributed to Russia.
Jeff Chiu/AP
hide caption
Jeff Chiu/AP
OpenAI said in a blog post on Thursday that it banned Russian and Iranian accounts that were using the company’s artificial intelligence tools to help with influence operations in multiple countries. The operatives said they managed to get their false narratives picked up by news outlets, and in once case, prompted a response from a member of the European Parliament, according to internal reports the operations produced using OpenAI’s chatbot, ChatGPT.
OpenAI investigated the impact of the different campaigns the accounts mentioned and found that those focused on placing false narratives in established media outlets appeared to have reached a wider audience than campaigns that focused on using fake social media accounts to spread their message.
The Russian operations targeted countries in Latin America and appeared aimed at undermining support for Ukraine and influencing local politics, OpenAI said. The accounts appeared to be mostly using ChatGPT to write reports detailing their work to unspecified higher ups, OpenAI said. The company said one of these reports appeared to describe the widest-reaching influence operation OpenAI has come across so far, because it led to public response from a political figure.
In this campaign, the Russian operatives said they impersonated a regional educational authority in Peru and tricked some schools into organizing activities about Ukraine that featured Stepan Bandera. Bandera is a controversial figure, admired by nationalists for leading a drive for Ukrainian independence before and during World War II. He is reviled by others, including in Poland, because his movement sided with the Nazis and killed Poles and Jews. Peruvian outlet Extra reported that one such event did take place at a Peruvian school.
A Polish news report about the school event prompted a far-right Polish member of the European Parliament to suggest banning the entry of people who professed support for Bandera.
“The Russian fake thus both fed on, and fed into, historical tensions between Ukrainians and Poles,” OpenAI wrote in its report.
“Russians are constantly engaging in these sorts of active measures,” said Darren Linvill, a co-director of Clemson University’s Watt Family Innovation Center Media Forensics Hub who studies online influence operations. “Trying to engage in narrative laundering, creating fake stories, fake persona to either create or spread propaganda that aligns with Russia’s worldview and Russia’s political agenda.”
The Russian internal memos created with ChatGPT also claimed that the operatives ran a fake think tank in Latin America and recruited unwitting researchers to interview experts and compose reports. OpenAI said that effort was similar to a 2020 Russian operation posing as an independent news outlet linked to the late oligarch Yevgeny Prigozhin’s Internet Research Agency, which was involved in efforts to influence the 2016 U.S. election.
In other instances, OpenAI said, the Russian operatives took credit for events they were not responsible for. The Russians did use AI to generate fake material, OpenAI said, but it was a minority of the workload.
“These Russian underlings were trying to use AI to create a shortcut and do less work in the same way that a lot of us use AI, but in this case it helped get them caught,” Linvill said.
While the Russian operatives primarily used AI to update their superiors, Iranian operatives used AI to help them push out content in Persian and English. The campaign mainly focused on the U.S.-Iran war and, to a lesser degree, American politics. OpenAI said the operation’s activity “appeared consistent with a commercial actor running a for-hire influence campaign” but that it was unable to identify who that was.
According to OpenAI, Iranian accounts made up seven fake authors to pitch and polish long articles to small media outlets around the world. They published almost 100 articles in this way.
The fake authors published articles in outlets including the progressive U.S. news site Daily Kos and the U.K.-based Middle East Monitor. According to NBC News, the Middle East Monitor published at least 21 articles by four fake authors. The outlet, which didn’t respond to a request for comment, has since taken down the author pages.Â
The Daily Kos wrote in response to NPR’s request for comment that its community section, where one of the fake personas posted, is separate from staff reports. Anyone can sign up to post in the community section and posts are “not subject to review by Daily Kos staff prior to publication,” according to a disclaimer on the site. The community section does have rules for posting, including prohibiting “Pretending to be someone you’re not.” The posts by the persona are still up.
“AI was valuable to them above and beyond just being a shortcut,” Linvill said. “It helped them create language that … looked and felt and read authentic to the point where it was being published by legitimate outlets. And I think that is something that a few years ago, an Iranian operation like this would have struggled a lot more with.”