Unmasking the Factual-Conceptual Gap in Persian Language Models

DivanBench, 315 hand-curated questions separating memorised Persian cultural facts from reasoning about implicit social norms. Every model tested knows more than it can reason with.

A longer write-up of this work is on its way. Until then the paper and the code are the best place to look.