Table of Contents
Cache-aware compaction
Source snapshot: 2026-10-25 18:22 UTC, commit bc8adfe6a032429c9a3726d81e912e91ceedb6e3. Live availability follows the pinned rollout receipt.
The unchanged budgets
The main exact compaction trigger remains 120000 tokens, main maximum output 50000 tokens, and context window 200000 tokens. This feature changes the construction of an eligible summary request; it does not lower those settings. Summary work keeps its existing 90000-token input cap and 20000-token output allowance.
When a summary can reuse a prefix
The conversation retains evidence from its last successfully completed main request. An eligible fork keeps the same model/endpoint/template identity, system text, ordered tool schemas, request options and unchanged history prefix, including raw reasoning fields. The head ends at a complete tool-group boundary. One summary instruction follows that prefix.
The endpoint must advertise supported vLLM prefix caching, and serving-tokenizer IDs must prove the unchanged head is a prefix of both parent and summary request. Equal-looking JSON or a token count alone is insufficient. Tokenization itself does not warm a generation cache.
For example, an edited prompt or refreshed tool catalog can invalidate the previous snapshot. A resumed session begins without warm evidence until a new main request finishes. Unsupported metrics, missing token IDs, changed prefixes or an oversized head use the existing bounded standalone summary path.
What the receipts mean
Observations include eligible, fork_sent, common_prefix_tokens, a fallback reason and cache_hit_verified=false. An eligible sent fork is an opportunity for reuse, not proof of a cache hit: eviction, backend restart and scheduling can still require prefill. No GPU KV restoration or free-input guarantee follows from these fields.
A summary must be nonempty, visibly complete, have closed thinking and finish with stop. Tool intent, length stops, cancellation and errors do not replace source history or create a successful checkpoint. The utility has no tool dispatcher. Once a fork has generated and failed, its error is surfaced rather than hidden behind another automatic generation.
Sources and memory
Source: docs/cache-aware-compaction.md, src/conversation/Compactor.cpp, src/conversation/Conversation.cpp, and tests/integration/test_compactor.cpp at bc8adfe6a032429c9a3726d81e912e91ceedb6e3.
A memory or wiki note can record the checkpoint/receipt, source commit, observation time and fallback reason. Keep “eligible prefix” separate from measured cache reuse or latency. Preserve cited source history and its original dates when summarizing; later summary creation is a different timestamp. See source and time.
Installed code observation: 2026-10-25T18:36:38.457052+00:00, native release d6949e909ffa08af7951. This page describes the verified repair rollout; live activity and later changes need fresh observations.