Prompt compression API

Send your agent less context. Keep the source text exact.

Tokz is a prompt compression API for AI applications. For structured payloads such as tool results, JSON, logs, tables, and diffs, it selects the relevant source spans and returns byte offsets into the payload you already hold. Your SDK assembles those spans locally, so the model receives less context without receiving a generated summary.

Hosted requests are processed in memory but are not persisted. Eligible hybrid and local deployments keep compression in your process.

Updated August 5, 2026 · Tokz

What a prompt compression API should return

A smaller string by itself is hard to audit. Tokz's structural path returns a span map: ordered UTF-8 byte ranges, the dropped runs between them, and a hash of the source. The response is a selection plan, not rewritten content.

Byte-exact by construction

The structural service emits offsets rather than text. Each retained fragment is sliced from your own payload.

Deterministic output

The same input and options produce byte-identical output, which matters when cache keys and prompt prefixes depend on stability.

Local expansion

The SDK can reassemble the selected spans or reveal a dropped range from the source you already retain. No second compression request is needed.

Compress a tool result before the model sees it

The SDK wrapper intercepts tool-result content. Your OpenAI call stays an OpenAI call; Tokz selects context before the request goes upstream.

import OpenAI from "openai";
import { withTokz } from "@tokz/openai";

const openai = withTokz(new OpenAI(), {
  apiKey: process.env.TOKZ_API_KEY!,
});

const toolOutput = JSON.stringify({
  pods: Array.from({ length: 150 }, (_, i) => ({
    name: `web-${i}`,
    status: i === 42 ? "CrashLoopBackOff" : "Running",
    restarts: i === 42 ? 14 : 0,
  })),
});

const response = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [
    { role: "user", content: "Which pods are unhealthy?" },
    { role: "tool", tool_call_id: "call_1", content: toolOutput },
  ],
});

For structured tool output, the retained content is selected from the original bytes. The model response is otherwise unchanged.

Use it where agent context grows

Tool results

Keep failures, identifiers, statuses, and representative records instead of forwarding every row from a long-running agent loop.

JSON and API responses

Preserve valid structure while selecting fields and records relevant to the request.

Logs and terminal output

Send the error-bearing lines and context an agent needs without treating a noisy transcript as a summary task.

Retrieved documents

Select source passages for a question, with an extractive provenance map when the input is prose.

Know which guarantee applies

Tokz has two paths. Treating them as identical would be inaccurate.

Neither path generates a summary or calls a generative model. The prose selector is a pinned in-process classifier; it cannot write new content.

What this API does not promise

• It does not guarantee that a model will answer every question correctly; compression can only select from the input it receives.
• It does not make every workload cheaper. If a stable cached prompt prefix is more valuable than removing context, measure cache economics before changing that prefix.
• It does not make hosted payload transit disappear. Hosted Tokz receives a payload to process it, then does not persist it.

Put the compression decision next to the payload

Start with tool output and structured retrieval results—the inputs where a span map is most directly verifiable. Use the documented target ratio and preserve controls for data your application must always retain.

Try prompt compression on the context you already send

Use the API for a real tool result, inspect the selected spans, and keep the original source beside the request. That makes the trade-off testable instead of rhetorical.