Managing AI Risk: Beyond Explainability

The banking industry has rapidly adopted machine learning for various applications. Large banks in the US are typically more cautious in adopting the methodology for high risk and regulated areas such as credit underwriting. The adoption of so called Explainable AI, which is typically ‘black box’ machine learning models accompanied by post-hoc explainability tools, are becoming more common for low risk applications; the concern remains in high risk area: can we trust post-hoc explainers? Alternatively, there are many recent developments on inherently interpretable, self-explanatory machine learning models without the problem of post-hoc explainers. The later offers many advantages beyond explainability such as model diagnostics and control to manage model risk. This is the focus of my talk where I will present examples including methods to incorporate model constraints (e.g.., monotonicity or other shape constraints) easily and adverse action reason code required by regulation in the US.