Multi‑language Consistency

Explanation

Multi-language consistency checks whether an AI system gives users the same quality of answer regardless of the language they write in - a real concern for Indian banks serving customers in Hindi, Tamil, Bengali, English and dozens of other tongues through document extraction and RAG-based chat. The worry is that a model trained mostly on English performs well there but mangles vernacular queries, quietly disadvantaging non-English speakers. It is measured by the Language Consistency Score, calculated as min(F1_lang)/max(F1_lang): the system's task accuracy (F1) is measured separately for each language, then the worst-performing language's score is divided by the best-performing one. A ratio near 1.0 means every language is served equally well; a low ratio exposes a weak link. To implement, build a parallel evaluation set covering all supported languages, run it on a schedule and after each model change, and log per-language F1 plus the resulting ratio as audit evidence. The target threshold is greater than or equal to 0.90; scores below 0.85 should trigger an investigation, and scores below 0.80 should trigger remediation such as adding training data or routing weak languages to human review (thresholds proposed).