Explainability is not a feature of good AI governance in India BFSI — it is a regulatory requirement with specific operational implications. RBI MRMF Section 4 defines what constitutes an explainable model, at what level explanations must be available, and what the acceptable documentation format looks like for a regulatory audit. RBI's digital lending guidelines add an adverse action notice obligation that creates a customer-facing implementation requirement alongside the internal audit trail requirement.
Existing coverage of this topic summarizes the regulatory announcement. This guide provides the technical implementation detail: how SHAP and LIME produce compliant explanations, what format those explanations must take for MRMF audit purposes, and when a model's complexity requires an interpretable-by-design approach rather than post-hoc explanation.
What MRMF Section 4 Actually Requires
RBI MRMF Section 4 establishes a two-tier explainability standard for AI models used in material decisions.
Tier 1: Model-level explainability
The bank must be able to document, for each AI model used in a material decision, how the model works at a general level: what features it uses, what types of relationships it has learned, what output scale it produces, and what the model's overall decision logic represents. This is primarily satisfied through model documentation — model cards, model risk reports, and validation summaries that describe model architecture, training approach, and general behavior.
For credit scoring models, model-level explainability documentation typically includes: feature list with descriptions and value ranges; global feature importance (which features most influence scores across the portfolio); model performance metrics by population segment; and a summary of what model outputs mean in terms of credit risk — what a score of 650 represents versus a score of 750 in terms of expected default probability.
Tier 2: Instance-level explainability
For each material decision — each individual credit application that is approved, declined, or resulted in an adverse condition — the bank must be able to produce a decision-specific explanation: which factors, in that specific applicant's profile, contributed most to the decision and in which direction.
Instance-level explainability cannot be satisfied by model documentation alone. It requires that the model produce (or that a post-hoc method compute) an explanation for each individual inference. This explanation must be available for regulatory audit — meaning it must be generated at inference time and stored alongside the decision record.
The operational implication most banks miss: MRMF's instance-level requirement means explanations cannot be generated retrospectively on request. They must be generated at the time the decision is made and logged as part of the decision record. A bank that receives a supervisory inquiry about a specific declined application three months later must be able to produce the explanation from its decision audit trail — not regenerate it from the model.
SHAP for Credit Model Compliance: What It Produces and What Format Satisfies MRMF
SHAP (SHapley Additive exPlanations) is the most widely used post-hoc explanation method for tabular credit models. It computes a contribution value for each feature in each individual prediction — indicating how much each feature increased or decreased the model's output relative to the average prediction across the population.
What SHAP produces for a credit decision
For a gradient boosted tree credit scoring model, SHAP produces a set of values for each applicant, one per input feature, that sum to the difference between the model's average output and the individual prediction. A positive SHAP value means the feature increased the credit score; a negative value means it decreased it.
Example output for a declined application (score 580, threshold 650):
- Employment type: self-employed → SHAP value −45 (decreased score by 45 points vs average)
- Credit utilization: 87% → SHAP value −38 (decreased by 38 points)
- Account age: 2 years → SHAP value −22 (decreased by 22 points)
- Income: ₹4.2L annual → SHAP value +18 (increased by 18 points)
- Number of active loans: 3 → SHAP value −16 (decreased by 16 points)
- [remaining features] → sum to remaining difference
Translating SHAP to adverse action notice
RBI digital lending guidelines require that adverse action notices communicate the principal reasons for the adverse decision in plain language comprehensible to the applicant. The top 3–5 negative SHAP values, translated from feature names to plain language, form the basis of a compliant adverse action notice:
"Your loan application was not approved. The principal factors contributing to this decision were: (1) High credit utilization — your existing credit cards and loans are being used at a level that indicates high existing debt burden. (2) Self-employment income — self-employed income is assessed differently than salaried income for credit purposes. (3) Limited credit history — your accounts have been open for a shorter period than typically required for this loan category."
This notice references the model's SHAP values without exposing the numeric scores — providing actionable information to the applicant without disclosing the model's internal parameters.
MRMF audit trail format for SHAP explanations
For MRMF compliance, the SHAP explanation stored in the decision audit trail should include:
- Decision identifier (linking to the loan application record)
- Decision timestamp
- Model version used for the decision
- Raw model output (score, probability)
- Decision threshold at time of decision
- Full SHAP value vector (all features, not just top contributors)
- Top 5 contributing features by absolute SHAP value (for adverse action notice generation)
- SHAP computation method and baseline used
The full SHAP vector — not just the top contributors — must be stored because regulatory review may examine any aspect of the decision, not just the reasons cited in the adverse action notice.
LIME for Credit Models: When to Use It and Its MRMF Limitations
LIME (Local Interpretable Model-agnostic Explanations) produces explanations by approximating the model's behavior in the neighborhood of a specific input with a simple interpretable model. It is computationally lighter than SHAP for some model types but has a characteristic that creates MRMF compliance risk: LIME explanations are local approximations that may not faithfully represent the underlying model's actual decision process.
MRMF's validation requirement includes a faithfulness criterion for explanations: the explanation must accurately represent what the model did, not just provide a plausible story. LIME explanations can fail this criterion when the model's decision boundary is complex or when the local linear approximation is a poor fit in the specific input region.
Practical guidance for India BFSI: SHAP (specifically TreeSHAP for gradient boosted models) produces explanations with exact mathematical faithfulness guarantees for tree-based models — the explanations sum to the model output exactly. For gradient boosted credit scoring models (XGBoost, LightGBM), TreeSHAP is the preferred method for MRMF compliance. LIME is acceptable for models where SHAP is computationally prohibitive, but must be validated to confirm explanation faithfulness before deployment.
The Interpretability Threshold: When Post-Hoc Explanation Is Not Enough
RBI MRMF Section 4 establishes that for the highest-consequence credit decisions, post-hoc explanation from a complex model may not satisfy the explainability requirement — the model itself must be interpretable by design.
The threshold condition is: decisions with high social consequence for individuals with limited credit history and limited access to alternative credit sources. In India BFSI context, this specifically covers:
- PMJDY and Jan Dhan account credit decisions: First-time borrowers with no credit bureau history, where a model decline forecloses financial inclusion. Post-hoc explanation from a complex neural network may not be adequate — a logistic regression or scorecard with directly auditable coefficients is preferred.
- Microfinance and SHG (Self Help Group) credit: Group-guarantee lending where the credit decision affects multiple members. Interpretable-by-design models allow loan officers to explain decisions directly to group members without relying on SHAP outputs.
- Priority sector agricultural lending: Seasonal credit with complex income patterns. Scorecard models that loan officers can apply manually provide a fallback when AI systems are unavailable and provide direct interpretability for dispute resolution.
For standard retail lending, home loans, and commercial credit, complex models with SHAP explanation are acceptable under MRMF — provided instance-level explanations are generated at inference time and stored in the decision audit trail.
SEBI Suitability Reasoning: The Advisory Parallel
SEBI's AI/ML Guidelines 2026 apply the same logic to investment advisory AI that MRMF applies to credit AI. Robo-advisory platforms and AI-assisted investment recommendations must document suitability reasoning — why a specific recommendation is appropriate for a specific investor profile.
The SEBI suitability explanation must reference: the investor's documented risk category (conservative/moderate/aggressive), the recommended instrument's risk classification, the investor's investment horizon vs. the instrument's maturity profile, and any specific exclusions or preferences documented in the investor profile.
As with credit adverse action notices, "the algorithm recommended it" is not a compliant suitability explanation. The AI system must produce a structured rationale that maps recommendation to investor profile, stored in the interaction audit trail for regulatory review.
IRDAI AI WG: Underwriting Algorithm Auditability
IRDAI AI Working Group June 2026 requires that underwriting algorithms used in insurance be auditable — meaning that for any policy decision (acceptance, loading, exclusion, decline), the insurer must be able to document the factors that contributed to the decision. This parallels MRMF's credit explainability requirement.
For insurance AI, the specific challenge is actuarial defensibility: the explanation must not only satisfy the applicant's right to understand the decision, but must also be defensible under actuarial standards — the factors cited must have statistically valid relationships to the risk being priced. Explaining an underwriting decision using features with legally questionable actuarial justification (features that are proxies for protected characteristics) creates dual exposure: regulatory non-compliance and potential discrimination liability.
XAI Compliance Checklist: India BFSI Credit, Advisory, and Underwriting AI
- ☐ Model-level documentation complete: feature list, global feature importance, decision logic summary, performance metrics by population segment
- ☐ Instance-level explanation method selected and validated for faithfulness (TreeSHAP for gradient boosted models)
- ☐ Explanation generation implemented at inference time (not retrospective) — explanations logged with every decision
- ☐ MRMF audit trail format for explanations: decision ID, timestamp, model version, raw output, threshold, full SHAP vector, top contributors
- ☐ Adverse action notice generation pipeline: top-N negative SHAP values translated to plain language, reviewed for comprehensibility
- ☐ Explanation faithfulness validation completed before deployment (SHAP consistency test, explanation fidelity metric)
- ☐ Interpretability threshold assessed: high-consequence decisions for underserved populations use by-design interpretable models
- ☐ SEBI suitability reasoning documented per recommendation: risk category match, horizon match, exclusion compliance
- ☐ IRDAI underwriting explanation validated for actuarial defensibility of cited factors
- ☐ Explanation retention period defined (minimum: duration of credit facility or regulatory dispute window)
- ☐ Model card template complete with explainability section per MRMF documentation requirements
- ☐ Explanation monitoring in production: check that SHAP distributions remain stable post-deployment (distribution drift = explanation drift)
The RBI Governor's formulation — that "the model decided" is not an acceptable answer — sets the practical test for XAI compliance in India BFSI: can the bank, at any point during the life of a credit facility, produce a human-readable explanation for the credit decision that a customer, an internal audit team, or an RBI supervisor would find responsive? If the answer depends on regenerating the explanation from a model that may have been retrained since the decision, the bank is not compliant. The explanation must be stored with the decision. That is the implementation requirement that MRMF's explainability section actually imposes.