Interview OS
Back to Context Engineering

Library

Context window budgeting

Spend tokens where they matter.

Context

Overview

Every model has a finite context window, so you must budget tokens between instructions, retrieved context, and history. This topic covers prioritizing, compressing, and truncating so the important content survives.

How it works

Context
InventoryRankTrimOutputTokensbudgetRelevancescoreCompressfit windowPacked prompt
ClientServiceEdgeData

Step by step, with examples

  1. 1

    Tokens

    • Count system, history, and docs.
  2. 2

    Relevance

    • Keep what matters most.
  3. 3

    Compress

    • Summarize or evict low-value context.
  4. 4

    Packed prompt

    • Leave room for the answer.
    • Example: avoid truncation

Overview

Treat the context window as a budget: system prompt, retrieved chunks, history, and tools all compete. Prioritize and compress.

Key idea

More context isn't better — relevance density beats volume; the 'lost in the middle' effect is real.

Common pitfalls

  • Dumping whole documents
  • Burying the instruction mid-context
  • Ignoring token cost

Where this content comes from

For full transparency, this content is curated and verified from these sources:

Frontier-lab prompting & agent guidesRetrieval-augmented generation literatureOppZen-authored context-engineering playbooks