Model validation is a point-in-time assessment. A credit scoring model that passed its initial validation in 2023 may be materially degraded by 2026 — the applicant population has shifted, the macroeconomic environment has changed, and the behavioral patterns the model learned from pre-COVID data no longer describe the post-moratorium, post-rate-cycle India that the model now scores every day. Ongoing monitoring is the governance mechanism that connects the validation that happened in 2023 to the model that is running decisions today.
RBI MRMF Section 6 makes this ongoing monitoring a formal obligation, not a best practice recommendation. It requires banks to demonstrate, at any point during a supervisory examination, that they are actively monitoring their AI models' production performance — and that they have governance processes for responding when that performance deteriorates.
The technical methods for this monitoring — PSI, CSI, champion-challenger testing, and outcome backtesting — are well understood in the Indian credit industry. CIBIL and India's other credit bureaus have used them for years. The governance gap is not in the techniques themselves but in their connection to MRMF documentation requirements and to the escalation protocols that determine what happens when a monitoring metric signals a problem.
Why Drift Matters Especially in India
The Indian credit market has experienced structural disruptions that cause the kind of population shift and concept drift that destroys model validity faster than in more stable markets.
Demonetization (November 2016). The sudden withdrawal of ₹500 and ₹1,000 currency notes caused a structural shift in financial transaction patterns across the economy. Credit models trained on pre-demonetization behavioral data were scoring applicants whose transaction patterns had been permanently altered by the shift to digital payments — a new feature correlation structure that the models had not been trained to handle.
COVID-19 + RBI moratorium (2020). The loan moratorium (March–August 2020) suspended the standard credit event signal — customer defaults were not recorded during the moratorium period regardless of payment behavior. Models trained on pre-moratorium default data were using historical default patterns as the target variable; post-moratorium, the distribution of "defaults" changed as the artificial suppression of delinquency unwound. Any credit model that was not recalibrated post-moratorium was scoring against a target distribution that no longer matched the production environment.
Rate cycle 2022-2023. RBI's repo rate increases (250 basis points from May 2022 to February 2023) changed affordability relationships for floating-rate loan products. Credit models built on low-rate-cycle behavior had not captured the performance patterns of borrowers whose EMI burdens increased materially — particularly relevant for home loan models and personal loan models where EMI-to-income ratios are a key feature.
UPI and digital banking penetration. The explosive growth of UPI has created new behavioral data sources — transaction velocity, merchant category distributions, payment network characteristics — that are increasingly used in credit models. But this data has only 7-9 years of history at scale, meaning models trained on UPI behavioral features have thin historical training data and are more susceptible to concept drift as UPI usage patterns evolve.
Statistical Drift vs Concept Drift: Which Matters More for Which Model
The distinction between statistical drift and concept drift is not academic — it determines which monitoring metrics are relevant and what the governance response looks like.
Statistical drift (covariate shift)
Statistical drift occurs when the distribution of input features changes. For a credit scoring model, this might mean: the proportion of self-employed applicants has increased relative to the training population, the average age of applicants has shifted younger, or geographic distribution has shifted toward tier-2 and tier-3 cities as digital lending expands. The model's learned relationships (feature-to-score mappings) may still be valid, but the model is now scoring a different population than it was built for.
PSI (Population Stability Index) measures this directly at the score level — has the distribution of credit scores shifted? CSI (Characteristic Stability Index) measures it at the feature level — has the distribution of individual input variables shifted? Together they answer: is the model scoring the same type of population it was trained on?
Concept drift (target distribution shift)
Concept drift occurs when the relationship between features and outcomes changes. Even if the applicant population is identical to the training population, a macroeconomic shift can mean that the features that predicted credit default in 2022 do not predict it in 2025. Concept drift requires monitoring actual model performance against observed outcomes — the Gini coefficient, KS statistic, and AUC on production data over time.
Concept drift is harder to detect than statistical drift because it requires outcome observation — which takes time. For 90-day delinquency outcomes, you need 90 days of data before you can measure whether the model is predicting correctly. A governance framework that only monitors PSI and CSI will miss concept drift. A complete monitoring program monitors both input distributions and output performance.
Which models face which drift types
| Model type | Primary drift risk | Primary monitoring metric | India-specific driver |
|---|---|---|---|
| Retail credit scoring | Both statistical and concept | PSI + Gini on 90-day outcomes | Moratorium residue, rate cycle, digital applicant shift |
| MSME credit scoring | Concept drift dominant | Monthly Gini on 30-day outcomes | GST system changes, seasonal patterns, supply chain disruptions |
| UPI fraud detection | Concept drift dominant | Precision/recall on weekly labeled fraud data | Fraudster behavioral evolution, UPI usage expansion to new populations |
| AML transaction monitoring | Concept drift dominant | Alert-to-STR conversion rate, typology coverage | Evolving money laundering typologies, UPI international expansion |
| Underwriting AI (insurance) | Statistical drift primary | CSI + loss ratio on cohorts | Post-COVID claims patterns, climate risk in agricultural insurance |
PSI and CSI: The RBI Examination Standard
PSI and CSI are the primary quantitative metrics that RBI off-site examination teams use to assess whether a bank's credit models remain valid. Understanding the thresholds and what they trigger in an examination context is essential for MRMF Section 6 compliance.
PSI thresholds and governance responses
| PSI range | Interpretation | RBI examination signal | Required governance response under MRMF |
|---|---|---|---|
| < 0.10 | No significant shift | No concern | Document monthly/quarterly result; no action required |
| 0.10 – 0.25 | Moderate shift | Flag for follow-up if sustained | Escalate to model risk; conduct CSI drill-down; assess whether shift is benign (new market) or adverse (model deterioration); document investigation and conclusion |
| > 0.25 | Significant shift | Likely examination finding | Immediate model risk committee notification; initiate model redevelopment or recalibration assessment; document escalation timeline; if model remains in production, document risk acceptance with Board/senior management sign-off |
A PSI above 0.25 in an MRMF context does not automatically require the model to be taken out of service — but it requires documented evidence that the bank has assessed the materiality of the shift, made a conscious decision about continued use, and initiated a remediation plan. A bank that discovers a PSI above 0.25 during an RBI examination — and cannot show documentation of prior awareness and response — faces an examination finding that goes beyond the technical metric.
CSI: drilling into which features are driving shift
CSI applies PSI's methodology to individual input features, identifying which specific characteristics of the applicant population are shifting. A PSI of 0.15 at the model level may be explained by a CSI shift in one feature (geographic distribution of applicants) that is benign, or by a CSI shift in a key predictive feature (employment type distribution) that materially affects model validity.
CSI drill-down is the investigation step that follows a PSI alert. The governance documentation for a PSI investigation should include: which features showed the highest CSI, what caused the shift for those features, and whether the shift affects the model's predictive accuracy for the affected segment.
Champion-Challenger Testing Under RBI Norms
MRMF Section 6's benchmarking requirement is the regulatory anchor for champion-challenger testing. The key elements of a MRMF-compliant champion-challenger program:
Challenger selection. The challenger model must be a legitimate alternative — either a different model architecture (logistic regression vs gradient boosted tree), a retrained version of the same architecture on more recent data, or an industry benchmark model (such as a CIBIL bureau score used as a comparison point). The selection rationale should be documented: why was this challenger chosen as the comparison basis?
Comparison period and sample. For a credit scoring champion-challenger test, the comparison typically runs for 3-6 months on a representative sample of incoming applications — large enough to be statistically significant but not so large that the bank is taking material risk on an unvalidated challenger model. The sample should be stratified to cover the bank's key applicant segments.
Comparison metrics. The primary comparison metrics for credit scoring champion-challenger: Gini coefficient, KS statistic, approval rate at a given risk threshold, expected loss ratio for approved applications. The challenger does not need to outperform the champion on all metrics to be informative — a challenger that performs equivalently on credit risk but with higher explainability (a logistic regression vs. a gradient boosted tree) may be preferred on MRMF governance grounds.
Documentation and decision record. MRMF requires that the champion-challenger comparison be documented with a formal decision: retain champion, transition to challenger, continue extended comparison, or initiate model redevelopment. The decision and its rationale must be documented in the model governance record.
Production Monitoring Architecture for India BFSI
A MRMF Section 6-compliant production monitoring architecture for an Indian bank's credit scoring AI has the following components:
Monthly monitoring dashboard. PSI, CSI for top 10 predictive features, Gini/KS/AUC on available outcomes (rolling 90-day cohort), alert-rate for any anomaly detection layers, and a status indicator (Green/Amber/Red based on threshold criteria). Distributed to model risk team and MLCO/relevant business head.
Quarterly model performance review. Formal model risk committee review of monitoring dashboard results, champion-challenger testing status, and any investigation reports from PSI/CSI alerts. The review produces a documented sign-off: "Model performance remains adequate / Model placed on watch / Model redevelopment initiated."
Annual revalidation cycle. Full model revalidation — documentation review, backtesting on current data, CSI/PSI assessment against current training baseline, independent validation sign-off. This is the MRMF requirement for "periodic validation" distinct from ongoing monitoring.
Event-triggered review. Outside the regular cycle, certain events require immediate model performance assessment: a significant macroeconomic event (rate change, policy announcement), a data quality incident affecting training or inference data, a regulatory change affecting the feature set or decision criteria, or a PSI/CSI breach beyond defined thresholds. The governance framework must define what constitutes a trigger event and what the required response timeline is.
MRMF Section 6 Compliance Checklist: India Credit AI
- ☐ Monthly PSI monitoring implemented for all production credit scoring models; results documented
- ☐ CSI monitoring implemented for top predictive features; thresholds defined (0.10/0.25 PSI equivalents)
- ☐ Outcome backtesting on rolling 90-day cohorts (or appropriate outcome window); Gini/KS/AUC tracked monthly
- ☐ Threshold breach escalation protocol defined: who is notified, within what timeframe, what investigation is required
- ☐ PSI investigations documented: root cause analysis, materiality assessment, conclusion
- ☐ Champion-challenger testing conducted at least annually; comparison methodology and results documented
- ☐ Champion-challenger decision record: retain/transition/continue — with rationale
- ☐ Quarterly model performance review by model risk committee; minutes document outcome
- ☐ Annual full revalidation: methodology updated for current data, independent validation sign-off
- ☐ Event-triggered review criteria defined: macroeconomic events, data quality incidents, regulatory changes
- ☐ UPI fraud detection models: weekly precision/recall monitoring on labeled fraud data; concept drift protocol defined
- ☐ MRMF monitoring records retained: minimum retention period aligned to model lifecycle + supervisory enquiry window
- ☐ SEBI algo-trading models (if applicable): periodic performance reporting per SEBI AI/ML circular
- ☐ DAKSH-ready documentation: monitoring results formatted for CIMS submission
The PSI and CSI thresholds that RBI examination teams use did not come from MRMF June 2026 — they predate it, having been standard credit industry practice for decades. What MRMF adds is the requirement to document that monitoring, connect it to escalation protocols, and demonstrate to a regulatory examiner that a bank's production AI models are under active governance, not just deployed and forgotten. The technical methods are available; the governance documentation framework is what most banks still need to build.