Guide
Prompt compression versus prompt caching
Use caching when the same prefix is valuable and stable. Use compression when the context itself contains irrelevant or repeated material. Use both only with measurements.
Caching wins for stable prefixes
It can avoid repeated processing without changing the bytes the provider recognizes.
Compression wins for oversized changing data
It can remove irrelevant spans from tool results and retrieval context before they enter the request.
The decision is workload-specific
Compare achieved ratio, cache reads, cache writes, latency, provider input cost, and Tokz cost together.