Counterfactuals

Explanation

Counterfactual validity checks whether the system can produce actionable, correct "what would have to change" explanations for an adverse decision - statements like "had your income been higher or your existing debt lower, the application would have been approved." It matters because customers (and regulators) need realistic, feasible recourse, not just a denial; an invalid counterfactual that recommends impossible or model-inconsistent changes is misleading. It is measured as Counterfactual Validity = Count(Valid Counterfactuals) / Count(Generated Counterfactuals); in card denial cases, generate at least 3 valid counterfactuals. In other words, of all counterfactuals the system generates, what fraction are valid - meaning they are feasible and genuinely flip the model's decision when applied. To implement it, run a counterfactual generator on adverse decisions, then validate each candidate by feeding the altered feature values back through the model to confirm the outcome actually changes and by checking feasibility constraints (only mutable, realistic changes), recording the valid count over the generated count and retaining the counterfactuals as evidence; for card denials ensure at least three valid ones are produced. The threshold is a >90% validity rate; a rate at or below 90% is a breach that should trigger investigation and tuning of the counterfactual generator before its explanations are shown to customers. Aligns with RBI Understandable by Design, NIST MEASURE 3.1 and 4.1, and ISO Clauses 6.2.2 and 9.1.