Europe’s leading AI model still can’t reliably follow European law
Critical evaluation of AI model compliance for builders who need trustworthy models for regulated applications.
AI Summary
Mistral Large 4 shows 61% improvement in human rights compliance over previous models but still struggles to reliably follow European law according to new testing.
Excerpt
TL;DR: Mistral previously performed the worst of all providers on our human rights scenarios, with their Medium 3.5 model failing on almost every single test. Their newly released model, Mistral Large 4, improves performance by 61 percentage points, showing strong resistance in most scenarios. Likewise, Mistral Large 4 demonstrates large improvements in compliance with European law. Its predecessor, Mistral Large 2512, lagged at the bottom of the leaderboard when we tested 12 frontier models ear