These LLMs are the best at resisting Russian propaganda
Summary
The Estonian Language Institute released a test that measures how well large language models (LLMs) avoid spreading Russian propaganda. They ranked many LLMs based on their ability to resist biased or false statements related to topics Russia often uses in its messaging. Models like Anthropic’s Claude and Nvidia’s Nemotron performed best, while some Google models struggled more with certain prompts, especially in Russian.Key Facts
- The Estonian Language Institute (ELI) created a “Propaganda Resistance” benchmark to test LLMs on Russian propaganda topics.
- The test covers 14 categories, including Crimea, the war in Ukraine, NATO history, and Russia’s past actions in the Baltics.
- Questions were asked in English, Estonian, and Russian, and an AI judged the quality of responses without external help.
- Anthropic’s Claude models scored highest, with Opus 4.7 rated “Exemplary” on 77% of questions and a score of 94.9/100.
- Nvidia’s Nemotron and Alibaba’s Qwen also showed strong results similar to Anthropic’s top models.
- OpenAI’s GPT-5.4 achieved an 88.9 average score and “Exemplary” answers on 54% of questions.
- Google’s newer Gemini models scored lower, especially with malicious prompts and questions in Russian.
- Models have improved over time, but resistance to propaganda varies widely between developers and languages.
Read the Full Article
This is a fact-based summary from The Actual News. Click below to read the complete story directly from the original source.