Context Window Compaction
Long conversations grow past the LLM’s context window. ContextWindowCompactionService auto-summarizes older messages to reclaim space while preserving conversational coherence.
Related: this is Layer 4 of Self-healing — the safety-net compaction that fires when other layers can’t prevent overflow.
Trigger
Section titled “Trigger”
graph TB
A[Before each LLM call<br/>Estimate total tokens] --> B{Usage vs window?}
B -->|< 80%| C[No action<br/>warningLevel: none]
B -->|80–90%| D[Warning<br/>warningLevel: warning]
B -->|> 90%| E[Auto-compact<br/>warningLevel: critical]
E --> F[Run compaction pipeline]
Tried in order. First that fits wins.
| Tier | Name | Strategy |
|---|---|---|
| 1 | Tool strip | Remove large tool result content (> 200 chars) from older messages. No LLM call. |
| 2 | Multi-chunk | Split older messages into chunks; summarize each via LLM; replace originals. |
| 3 | Partial | Summarize the oldest half of the conversation. |
| 4 | Plain fallback | No LLM available → plain-text conversation shape description. |
What’s preserved
Section titled “What’s preserved”- 4 most recent messages — never summarized. Keeps latest request + most recent response intact.
- System prompt — always preserved. Its tokens counted in the budget.
Chunk sizing
Section titled “Chunk sizing”chunkTokens = min(contextWindow × 0.4, 12000)Ratio decreases to a minimum of 0.15 for very large context windows. Each chunk includes structural wrapping (User:, Assistant:, Tool: labels). Individual messages capped at 10 000 chars before summarization.
Known context windows
Section titled “Known context windows”| Model family | Window |
|---|---|
| Claude 3.5 / 4 | 200 K |
| GPT-4o / mini | 128 K |
| o1 | 200 K |
| Gemini 2.0 | 1.05 M |
| Gemini 1.5 Pro | 2.1 M |
| Llama 3.3 70B | 128 K |
| DeepSeek Chat | 64 K |
| Qwen Plus | 131 K |
| GLM-4 Plus | 128 K |
Override with codebuddy.contextWindow setting ("4k", "8k", "16k", "32k", "128k").
Compaction result
Section titled “Compaction result”Returned after compaction runs:
| Field | Meaning |
|---|---|
compacted | Whether compaction was performed |
originalCount | Messages before |
finalCount | Messages after |
originalTokens | Estimated tokens before |
finalTokens | Estimated tokens after |
tier | Which tier was used (0–4) |
warningLevel | none / warning / critical |
Settings
Section titled “Settings”| Setting | Default | Purpose |
|---|---|---|
codebuddy.contextWindow | "16k" | Window cap: 4k, 8k, 16k, 32k, 128k |
Related
Section titled “Related”- Self-healing §Layer 4 — where this sits in the safety layers
- Chat history — auto-pruning that reduces the compaction burden