Interview OS
Back to AI / ML

Library

Regularization

L1/L2, dropout, early stopping.

Theory

Overview

Regularization curbs overfitting by penalizing complexity — L1/L2 penalties, dropout, early stopping. The goal is a model that generalizes to unseen data rather than memorizing the training set.

How it works

Theory
ModelPenaltyEffectOutputWeightswL1 / L2λ termShrink/sparsesparsityGeneralize
ClientServiceEdgeData

Step by step, with examples

  1. 1

    Weights

    • Penalize overly large weights.
  2. 2

    L1 / L2

    • Add λ·‖w‖ to the loss.
  3. 3

    Shrink/sparse

    • L2 shrinks; L1 zeros out features.
  4. 4

    Generalize

    • Reduce overfitting.
    • Example: dropout, weight decay

Overview

Constrain model complexity to improve generalization. L1 induces sparsity; L2 shrinks weights; dropout averages sub-networks.

When to use it

  • Overfitting on limited data
  • Feature selection (L1)
  • Deep nets (dropout)

Common pitfalls

  • Over-regularizing -> underfit
  • Applying dropout at inference

Where this content comes from

For full transparency, this content is curated and verified from these sources:

Peer-reviewed ML papers & model cardsProduction ML engineering referencesOppZen-authored ML eval rubrics