Interview OS
Back to AI / ML

Library

Backpropagation & gradients

How nets learn.

Training

Overview

Backpropagation trains neural networks by computing gradients of the loss with respect to every weight via the chain rule, then updating with gradient descent. Understanding it demystifies why networks learn and where they get stuck.

How it works

Training
ForwardGradUpdateOutputPredictŷ, LChain rulebackwardOptimizerw−=η·gLearned
ClientServiceEdgeData

Step by step, with examples

  1. 1

    Predict

    • Compute the output and loss.
  2. 2

    Chain rule

    • Compute ∂L/∂w layer by layer.
  3. 3

    Optimizer

    • Step the weights (SGD/Adam).
  4. 4

    Learned

    • Repeat over epochs.
    • Example: watch vanishing grads

Overview

Chain-rule gradient computation flows error backward to update weights via an optimizer (SGD/Adam).

Common pitfalls

  • Vanishing/exploding gradients
  • Bad learning rate
  • Forgetting to zero gradients

Where this content comes from

For full transparency, this content is curated and verified from these sources:

Peer-reviewed ML papers & model cardsProduction ML engineering referencesOppZen-authored ML eval rubrics