Glossary Term

Computational/Model Bias 

The various forms of mathematical bias that arise due to the choice of model and parameters of interest for a given dataset. These biases include but are not limited to: the bias-variance tradeoff, frequency bias, and/or inductive bias.

Related terms: Data Bias, Decision-Making Bias, Bias-Variance Tradeoff, Frequency Bias, Inductive Bias 

When developing a model for some dataset, we may initially choose a simple model (e.g., a linear model) to describe the data (upper left hand panel). Simple models, though, fail to capture regularities in the data and tend to underfit the data (high bias, low variance). To capture regularities in a dataset often requires a low-bias prediction.  However, lowering the prediction bias comes at the cost of increasing the variance of the model prediction. Low bias and high variance as shown on the right overfits the data, as it pays way too much attention to training data and doesn’t generalize well.  In other words, increasing the prediction variance tends to increase overfitting to the training data.  The bias variance-tradeoff differs from one learning algorithm to another.

A classic illustration of the bias-variance tradeoff (From Fig. 37.1 in Guerts 2010). Left, a linear model fitted to two random samples. Right, a neural network fitted to the same samples.

Resources and References

Return to Glossary