Shrink long conversations

You choose which messages each completion request sends to the model. As a conversation grows, resending the full history makes every turn slower and more expensive, until it no longer fits the model’s context window. To keep it in bounds, shrink the message list before each request:

	Trim	Compact
How it shrinks	Drops older messages, and optionally tool calls and results	Replaces older messages with one summary message
LLM calls	None	One or more, through your LLM provider, which bills you
Older details	Lost	Kept in the summary

Both endpoints return messages you can send to the completions endpoint as is. Any API key with the search ACL can call them.

Both endpoints are in beta and live under /1/unstable/context/. Their request and response shapes may change before they move to a stable path, so keep a fallback, as the loop below does.

Trim a conversation

Trim drops messages without calling an LLM, so the same input always returns the same output. This request keeps the last two messages of a five-message conversation:

curl -X POST "https://$ALGOLIA_APPLICATION_ID.algolia.net/agent-studio/1/unstable/context/trim" \
  -H "X-Algolia-Application-Id: $ALGOLIA_APPLICATION_ID" \
  -H "X-Algolia-API-Key: $ALGOLIA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      { "role": "user", "content": "I need running shoes for trail races." },
      { "role": "assistant", "content": "Do you prefer more cushioning or more ground feel?" },
      { "role": "user", "content": "Cushioning. My budget is 150 USD." },
      { "role": "assistant", "content": "The Trailblazer 4 costs 139 USD and has a cushioned midsole." },
      { "role": "user", "content": "Does it come in wide sizes?" }
    ],
    "keepLastMessages": 2
  }'

The response holds the trimmed messages and a stats object:

Response

{
  "messages": [
    { "role": "assistant", "content": "The Trailblazer 4 costs 139 USD and has a cushioned midsole." },
    { "role": "user", "content": "Does it come in wide sizes?" }
  ],
  "stats": {
    "messagesBefore": 5,
    "messagesAfter": 2,
    "messagesDropped": 3,
    "tokensBeforeEstimate": 61,
    "tokensAfterEstimate": 27
  }
}

Token counts are heuristic estimates, not counts from your LLM provider. To measure a conversation without changing it, send only messages: trim returns them unchanged, with stats. Set one or more limits:

Parameter	Effect
`keepLastMessages`	Keeps only the last N messages.
`dropToolParts`	Removes tool calls and tool results from the messages it keeps. Default: `false`.
`maxTokensEstimate`	Drops the oldest messages until the estimate fits this budget.

keepLastMessages and dropToolParts apply first, so maxTokensEstimate counts only what remains.

Compact a conversation

Compact summarizes every message except the last keepLastMessages into one user message, followed by the kept messages unchanged. Your LLM provider writes the summary, so set providerID to its ID and model to a model it serves. We recommend a small model: it reads the older messages but writes only a short summary.

curl -X POST "https://$ALGOLIA_APPLICATION_ID.algolia.net/agent-studio/1/unstable/context/compact" \
  -H "X-Algolia-Application-Id: $ALGOLIA_APPLICATION_ID" \
  -H "X-Algolia-API-Key: $ALGOLIA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "providerID": "'"$PROVIDER_ID"'",
    "model": "gpt-4o-mini",
    "messages": [
      { "role": "user", "content": "I need running shoes for trail races." },
      { "role": "assistant", "content": "Do you prefer more cushioning or more ground feel?" },
      { "role": "user", "content": "Cushioning. My budget is 150 USD." },
      { "role": "assistant", "content": "The Trailblazer 4 costs 139 USD and has a cushioned midsole." },
      { "role": "user", "content": "Does it come in wide sizes?" }
    ],
    "keepLastMessages": 2,
    "instructions": "Keep every product name, price, and stated preference.",
    "targetTokensEstimate": 100
  }'

The response adds a compaction object:

Response

{
  "messages": [
    { "role": "user", "content": "The user wants cushioned trail running shoes for under 150 USD." },
    { "role": "assistant", "content": "The Trailblazer 4 costs 139 USD and has a cushioned midsole." },
    { "role": "user", "content": "Does it come in wide sizes?" }
  ],
  "stats": {
    "messagesBefore": 5,
    "messagesAfter": 3,
    "messagesDropped": 2,
    "tokensBeforeEstimate": 61,
    "tokensAfterEstimate": 42
  },
  "compaction": {
    "compacted": true,
    "chunksProcessed": 1,
    "passes": 1,
    "summarizerPromptTokens": 182,
    "summarizerCompletionTokens": 17,
    "targetTokensEstimate": 100
  }
}

Parameter	Effect
`providerID`, `model`	Required. The provider and model that write the summary.
`keepLastMessages`	Number of trailing messages kept verbatim. Default: `4`.
`instructions`	Guidance for the summary, up to 2,000 characters.
`targetTokensEstimate`	Desired summary size in tokens, from 64. Best effort, not a hard cap.

compacted is false when nothing is older than the kept messages, or nothing older has text to summarize. The messages then pass through unchanged. summarizerPromptTokens and summarizerCompletionTokens are the token counts your provider reports, summed across every summary call. They’re null if the provider doesn’t report usage. If a conversation doesn’t fit the summary model’s context window, compact splits it into chunks, summarizes each one, and merges the results. chunksProcessed reports the number of chunks, and passes the number of summary rounds. If compact still can’t summarize the conversation, the endpoint returns 400.

Shrink the conversation on every turn

This example shrinks the messages before every completion request and compacts them once the estimate passes 20,000 tokens:

JavaScript

const appId = "ALGOLIA_APPLICATION_ID";
const apiKey = "ALGOLIA_API_KEY";
const agentId = "AGENT_ID";
const providerID = "PROVIDER_ID";
const tokenLimit = 20_000;
const fallbackMessages = 10;

const baseUrl = `https://${appId}.algolia.net/agent-studio/1`;

async function post(path, body) {
  const response = await fetch(`${baseUrl}${path}`, {
    method: "POST",
    headers: {
      "X-Algolia-Application-Id": appId,
      "X-Algolia-API-Key": apiKey,
      "Content-Type": "application/json",
    },
    body: JSON.stringify(body),
  });
  if (!response.ok) throw new Error(`${path} returned ${response.status}`);
  return response.json();
}

async function shrink(messages) {
  try {
    const { stats } = await post("/unstable/context/trim", { messages });
    if (stats.tokensBeforeEstimate <= tokenLimit) return messages;

    const compacted = await post("/unstable/context/compact", {
      providerID,
      model: "gpt-4o-mini",
      messages,
      keepLastMessages: 4,
    });
    return compacted.messages;
  } catch (error) {
    console.warn("Context shrink failed, sending the last messages instead", error);
    return messages.slice(-fallbackMessages);
  }
}

let messages = [];

async function ask(text) {
  messages.push({ role: "user", parts: [{ type: "text", text }] });
  messages = await shrink(messages);

  const reply = await post(
    `/agents/${agentId}/completions?stream=false&compatibilityMode=ai-sdk-5`,
    { messages },
  );
  messages.push(reply);
  return reply;
}

console.log(await ask("I need running shoes for trail races."));

If either call fails, the loop sends the last ten messages instead, so a change to a beta endpoint doesn’t stop your agent. Messages can use the AI SDK v4 or v5 format, but not both in the same request. The summary stays in the history, so later compactions summarize it again with the newer messages. Compaction doesn’t guarantee a size. For a hard cap, trim the result with maxTokensEstimate before you send it.

Search and Discovery platform

Optimization and Personalization

AI-powered experiences

Production and scale

Shrink long conversations

Trim a conversation

Compact a conversation

Shrink the conversation on every turn

See also

​Trim a conversation

​Compact a conversation

​Shrink the conversation on every turn

​See also

Trim a conversation

Compact a conversation

Shrink the conversation on every turn

See also