AI-agent token optimization
Optimize AI-agent tokens without turning context into a summary.
Agent costs grow when the same logs, API responses, and tool results are sent back to a model on every loop. Tokz selects the context that a request needs and preserves the retained source material instead of inventing a shorter retelling.
import OpenAI from "openai";
import { withTokz } from "@tokz/openai";
const openai = withTokz(new OpenAI(), {
apiKey: process.env.TOKZ_API_KEY!,
});
await openai.chat.completions.create({
model: "gpt-4o",
messages: [
{ role: "user", content: "Which deployment failed?" },
{ role: "tool", tool_call_id: "call_1", content: deploymentJson },
],
});Start where repeated context is largest
Tool results are often the clearest first target: terminal output, JSON, status lists, and search results may recur across several calls even when only a few records matter to the current step.
- Measure inputs first
- Use the local CLI to see spend by model, tool, and MCP server before choosing a compression policy.
- Compress near the tool boundary
- Selecting context before it joins the next request avoids relying on an agent to remember to trim it later.
Net savings are not a token percentage
Provider input cost, cache-read discounts, cache writes, and Tokz credit cost all belong in the calculation. A useful optimization report labels unknown provider data as unknown rather than treating it as zero.
Determinism protects repeatability
Same input and policy produce byte-identical structural output. That lets a local compression cache reuse a prior result and avoids random prompt changes that make operational analysis harder.
When not to compress
Do not compress a stable prefix merely because it is large when it already produces valuable cache hits. Do not call a reduction ratio a quality guarantee; validate your workload's required fields and answers.