1/ For machine learning interpretability methods, I argue that
model-agnostic > model-specific
Why?
tl;dr: model-agnostic methods allow more flexibility for the underlying ML model.
model-agnostic > model-specific
Why?
tl;dr: model-agnostic methods allow more flexibility for the underlying ML model.
2/ First let's define model-agnostic methods:
A post-hoc interpretability method that only works with input and output of the machine learning model and doesn't need access to model parameters, like weights.
Examples: partial dependence plots, feature importance, lime
A post-hoc interpretability method that only works with input and output of the machine learning model and doesn't need access to model parameters, like weights.
Examples: partial dependence plots, feature importance, lime
3/ Then there are model-specific methods:
A built-in or post-hoc interpretability method that requires knowledge about the machine learning model, like the structure or the weights.
Examples: xgboost explainer, attention based nn, linear regression model
A built-in or post-hoc interpretability method that requires knowledge about the machine learning model, like the structure or the weights.
Examples: xgboost explainer, attention based nn, linear regression model
4/ Model-specific methods have an advantage over model-agnostic:
Computation of the methods is often faster, bc you can derive knowledge from the structure, e.g. the partial dependence of a feature in a linear model has the same info as the beta weight, but takes more compute.
Computation of the methods is often faster, bc you can derive knowledge from the structure, e.g. the partial dependence of a feature in a linear model has the same info as the beta weight, but takes more compute.
5/ But I'd argue that the drawbacks of model-specific methods outweigh the benefits: The tie between model and interpretability method makes it hard or impossible to switch one without having to change the other.
6/ Let's say you are predicting fraudulent insurance claims with a random forest and you used random forest specific interpretability methods to forward the claims together with the explanations to your investigator team.
One day you discover that xgboost works better.
One day you discover that xgboost works better.
7/ Now, in order to use xgboost instead of random forests, you also have to change the interpretability method. Luckily, you can use "xgboost explainer". You also have to change the claim reporting. And communicate the changes to the other team, and they dislike changes!
8/ Some time later, you discover CATBoost, which works better and finally you can include categorical features! But is there an explanation method for it? Will the investigator team despise you for again changing the explanation report?
9/ This problem goes away if you use a model-agnostic method in the first place to create explanations, for example LIME , Shapley or SHAP.
10/ Machine Learning is moving and improving at incredible speed:
gbm -> xgboost -> LightGBM -> ...
AlexNet -> VGG -> Inceptiion -> ResNets -> ...
Only model-agnostic interpretability methods are robust in these wild times.
gbm -> xgboost -> LightGBM -> ...
AlexNet -> VGG -> Inceptiion -> ResNets -> ...
Only model-agnostic interpretability methods are robust in these wild times.
11/ Using model-agnostic methods gives you stability to build stuff on top, like a report, a website, an app, any product.
At the same time, you enjoy the freedom to level up your ML algorithm under the hood.
At the same time, you enjoy the freedom to level up your ML algorithm under the hood.
12/ You can also have multiple interpretation methods on top, to satisfy different needs: Your customer might prefer simple, short explanations (-> LIME or shap); your lawyers or engineers prefer detailed contribution of features towards a prediction (->Shapley)
