Interview OS
Back to AI / ML

Library

Bias-variance tradeoff

Under- vs over-fitting.

Theory

Overview

The bias-variance tradeoff explains why models underfit or overfit: high bias means the model is too simple to capture the pattern, high variance means it memorizes noise. Managing this tradeoff is the heart of generalization.

How it works

Theory
DataFitDiagnoseTuneTrain/testsplitComplexitycapacityError gapunder/overfitBalance
ClientServiceEdgeData

Step by step, with examples

  1. 1

    Train/test

    • Split data to measure generalization.
  2. 2

    Complexity

    • Simple models bias; complex ones vary.
  3. 3

    Error gap

    • Compare train vs validation error.
  4. 4

    Balance

    • Regularize or add data/features.
    • Example: find the sweet spot

Overview

Total error = bias² + variance + irreducible noise. High bias underfits; high variance overfits.

Common pitfalls

  • Tuning on the test set
  • Ignoring data leakage

Where this content comes from

For full transparency, this content is curated and verified from these sources:

Peer-reviewed ML papers & model cardsProduction ML engineering referencesOppZen-authored ML eval rubrics