Glossary

• Artificial Intelligence (AI)
• Bias, Computational/Model
• Bias, Data
• Bias, Decision-making
• Ethical AI
• Explainability
• Machine Learning (ML)
• Interpretability
• Responsible AI
• Robust AI
• Trust
• Trustworthiness

Resources and References

• Understanding Machine Learning: From Theory to Algorithms
• Machine Learning, Tom Mitchell, 1997
• Bias vs Variance Decomposition for Regression and Classification, Pierre Guerts, 2010
• Bias, variance, and prediction error for classification rules

Computational/Model Bias

The various forms of mathematical bias that arise due to the choice of model and parameters of interest for a given dataset. These biases include but are not limited to: the bias-variance tradeoff, frequency bias, and/or inductive bias.

Related terms: Data Bias, Decision-Making Bias, Bias-Variance Tradeoff, Frequency Bias, Inductive Bias 

When developing a model for some dataset, we may initially choose a simple model (e.g., a linear model) to describe the data (upper left hand panel). Simple models, though, fail to capture regularities in the data and tend to underfit the data (high bias, low variance). To capture regularities in a dataset often requires a low-bias prediction.  However, lowering the prediction bias comes at the cost of increasing the variance of the model prediction. Low bias and high variance as shown on the right overfits the data, as it pays way too much attention to training data and doesn’t generalize well.  In other words, increasing the prediction variance tends to increase overfitting to the training data.  The bias variance-tradeoff differs from one learning algorithm to another.

A classic illustration of the bias-variance tradeoff (From Fig. 37.1 in Guerts 2010). Left, a linear model fitted to two random samples. Right, a neural network fitted to the same samples.

Decision-Making Bias

The biases that arise due to how people perceive, think, remember, and represent events and phenomena and use these heuristics in their problem-solving and decision-making. 

Related Terms: Computational/Model Bias, Data Bias

Heuristics or preferences that we have in solving problems or making decisions can be a source of bias.  Biases in problem solving can be described as stemming more fundamentally from how people perceive, think, remember, and represent events and phenomena – both strategically and inadvertently, keeping in mind that all of these are extremely context dependent, much more than it seems most people realize. 

For example, people are likely to perceive motion and color, which are more salient to them than orientation, and may be limited in their understanding of complexity by working memory limits (5 plus or minus two), and their ability to perceive categories – which can be greatly facilitated by expertise and an understanding of hierarchical relationships within data.    

Of relevance to the definition of biases, Kahneman has suggested that most common heuristics can be characterized as resulting from “attribute substitution”  – for example, people commonly substitute feelings/emotions in part or entirely for cognitive assessments, or mental availability/accessibility (i.e., does it come readily to mind) for more cognitive or deliberative assessments of prevalence / relative frequency.   Another common type of bias is anchoring, where subjective estimates can be disproportionately influenced by some familiar or salient number for example, and adjust their estimates insufficiently away from that anchor.  An example of a heuristic that can be hugely influential is a preference to use models and tools with which you are familiar. 

“Narrow bracketing” has been highlighted in the judgment and decision making literature as a weakness in intuitive problem solving approaches (related to smaller variance). Even more specifically, people often use lexicographic approaches to problem solving, considering only one aspect or one option at a time, which might similarly result in ‘narrow bracketing’ of problems.

Resources and References

• Kahneman, D., & Frederick, S. (2002). Representativeness revisited: Attribute substitution in intuitive judgment. Heuristics and biases: The psychology of intuitive judgment, 49, 81.
•  Read, D., Loewenstein, G., Rabin, M., Keren, G., & Laibson, D. (1999). Choice bracketing. In Elicitation of preferences (pp. 171-202). Springer, Dordrecht.

Explainability

The degree to which a human can derive meaning from the entire model and its components through the use of post-hoc methods (e.g., verification, visualizations of important predictors).

Related terms: Interpretability

A distinction between interpretability and explainability is needed since some in the ML and statistics community favor producing interpretable models (i.e., restricting model complexity beforehand to impose interpretability; (Rudin et al. 2019, Rudin et al. 2021) while the general trend in the ML community is to continue developing partially interpretable and black box models and implementing post-hoc methods to explain them. A deep neural network (DNN) or dense random forest is uninterpretable, but through external explanation methods, can be approximately understood. Explanation methods must be approximate as they would otherwise be as incomprehensible as the black box model itself. This is not a limitation of explanation methods as suggested by other studies (e.g., Rudin et al. 2019, Rudin et al. 2021) since abstracting a complex model is required for human understanding. For example, it is common in the weather community to replace the full Navier-Stokes equation with conceptual models that are more understandable. The degree of explainability, though, is controlled by the model complexity (Molnar et al. 2020). As the number of features increases or their interactions become more complex, the explanations for the ML model behavior become less compact and possibly less accurate. At the moment, it is unclear how much improvement in understanding of high-dimensional, highly non-linear models current and future explanation methods will offer (Fig. 1). 

Fig. 1 Illustration of the relationship between understandability and model complexity. Fully interpretable models have high understandability (with little to gain from explainability) while partially interpretable or simpler black box models have the most to gain from explainability methods. With increased dimensionality and non-linearity, explainability methods can improve understanding, but there is considerable uncertainty about the ability of future explanation methods to improve the understandability of high-dimensional, highly non-linear methods.

Resources and References

• Interpretable Machine Learning: A Guide for Making Black Box Models Explainable
• Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead, Rudin et al., 2019 
• Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges, Rudin et al., 2021
• Quantifying Model Complexity via Functional Decomposition for Better Post-Hoc Interpretability, Molnar et al. 2020

Interpretability

The degree to which a human can derive meaning from the entire model and its components without the aid of additional methods. 

Related terms: Explainability

A fully interpretable model is one that has simulatability (the entire model can be contemplated at once), decomposability (each component of the model are human-understandable), and algorithmic transparency (one can understand how the model was trained) (Lipton 2016) while a partially interpretable model may only meet one of these criteria. As an example, low-dimensional linear regression is fully interpretable and a shallow decision tree is partially interpretable (see figure below).

Fig. 1 Illustration of the relationship between understandability and model complexity. Fully interpretable models have high understandability (with little to gain from explainability) while partially interpretable or simpler black box models have the most to gain from explainability methods. With increased dimensionality and non-linearity, explainability methods can improve understanding, but there is considerable uncertainty about the ability of future explanation methods to improve the understandability of high-dimensional, highly non-linear methods.

Resources and References

• Interpretable Machine Learning: A Guide for Making Black Box Models Explainable
• The Mythos of Model Interpretability, Zachary Lipton, 2016

Robust AI

An AI system that fails in a responsible and graceful way and is resistant to adversarial attacks. 

Related terms: AI, Ethical AI, Responsible AI, Trust, Trustworthiness

Trustworthiness

A trustee’s evaluation, or perception, of whether, when, why, or to what degree someone or something should or should not be trusted. 

Related terms: Trust, Interpretability, Explainability, Ethical AI, Responsible AI, Robust AI

The dimensions of trustworthiness are dependent on the values and needs of the trustor (Schneiderman, Human-Centered AI, 2022). A model deemed trustworthy by one user does not imply that it will be trustworthy to another user with their own set of values and needs. The common dimensions associated with evaluation of trustworthiness include: model performance and verification, trustor’s experience with model, source of information, and usability of model and information. Dimensions specific to ML are basic performance, reliability, human interaction, and aligned purpose (Varshney 2022). Note, the provided dimensions are meant to be representative examples but not exhaustive of all possible dimensions. 

Resources and References

• Trustworthy Machine Learning, Varshney, 2022
• Human-Centered AI, Schneiderman, 2022
• Trustworthy: International Organization for Standardization