Summarize long Cortex Agent conversations with the Compact API

The new Cortex Agents Compact API enables summarizing long conversation histories to save tokens and prevent exceeding the model context window. The feature works in two ways: stateless by passing the transcript inline, or stateful by integrating with threads stored in Snowflake.

Use cases

Imagine you have a corporate AI chatbot handling long user conversations and need to optimize processing costs. With this feature, you can:

• Reduce the number of tokens consumed during prolonged interactions with language models.
• Keep active sessions running without hitting model context window limits.
• Automatically summarize conversation histories via REST requests.

How to use

Send a POST request to the agent endpoint to compact an inline stateless history:

curl -X POST "https://<account>.snowflakecomputing.com/api/v2/cortex/agent:compact" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{
    "model": "auto",
    "messages": [
      { "role": "user", "content": [{ "type": "text", "text": "What were total sales last quarter?" }] },
      { "role": "assistant", "content": [{ "type": "text", "text": "Total sales last quarter were $4.2M..." }] }
    ]
  }'

What happens in this code:

1. The payload sends the model selection and the messages array containing the conversation transcript.
2. Snowflake processes the content and generates a natural-language summary.
3. The endpoint returns a JSON object containing the generated summary to be injected into subsequent calls.

Or perform compaction using a thread identifier to save the state on the server:

curl -X POST "https://<account>.snowflakecomputing.com/api/v2/cortex/agent:compact" \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -d '{
    "model": "auto",
    "thread_id": 123
  }'

What happens in this code:

1. The request targets an existing thread ID without needing to manually pass the conversation transcript.
2. The system reads the thread history, generates the summary, and persists a COMPACTION message.
3. The response provides the summarized string and the ID of the last message to continue on subsequent executions.

What it returns

Field	Description
summary	A natural-language summary of the conversation up to the compacted point.
end_message_id	The ID of the last message included in the compaction (returned only in stateful mode).

Limitations and requirements

• Available in Public Preview for all accounts.
• The endpoint does not support streaming and always returns a single JSON response.
• In stateless mode, the messages array must start with a user message (with an optional single leading system message) and alternate between user and assistant roles.
• If both messages and thread_id are provided, thread_id takes precedence and the messages parameter is ignored.
• Usage is billed in AI Credits per million tokens processed based on the model's pricing rates, logged in the account usage views.
• Requires the SNOWFLAKE.CORTEX_USER or SNOWFLAKE.CORTEX_AGENT_USER database roles in the Snowflake account.