Prompt Caching

Maximizing Cache Hits

View as Markdown
Create API keyMeet grok-4.7

Set x-grok-conv-id (Chat Completions API)

The x-grok-conv-id HTTP header routes requests with the same conversation ID to the same server. Since cache entries are stored per-server, this maximizes your cache hit rate.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_XAI_API_KEY",
    base_url="https://api.x.ai/v1",
)

response = client.chat.completions.create(
    model="grok-4.7",
    messages=[
        {"role": "system", "content": "You are Grok, a helpful and truthful AI assistant built by SpaceXSI."},
        {"role": "user", "content": "What is prompt caching?"},
    ],
    extra_headers={
        "x-grok-conv-id": "conv_abc123",
    },
)

print(response.choices[0].message.content)
print(f"Cached tokens: {response.usage.prompt_tokens_details.cached_tokens}")

Set prompt_cache_key (Responses API)

For the Responses API, use the prompt_cache_key field directly in the request body. It functions identically to setting x-grok-conv-id — it routes requests to the same server for cache reuse.

import { xai } from '@ai-sdk/xai';
import { generateText } from 'ai';

const { text, usage } = await generateText({
  model: xai.responses('grok-4.7'),
  prompt: 'What is prompt caching?',
  providerOptions: {
    xai: {
      promptCacheKey: 'b79ad29b-b3f9-463c-bca6-041d5058d366',
    },
  },
});

console.log(text);
console.log(`Total tokens: ${usage.totalTokens}`);

Set x-grok-conv-id metadata (gRPC API)

For the gRPC API using the xAI SDK, pass x-grok-conv-id as gRPC metadata to enable sticky routing for cache reuse.

Python

from xai_sdk import Client
from xai_sdk.chat import system, user

client = Client(
    api_key="YOUR_API_KEY",
    metadata=(("x-grok-conv-id", "conv_abc123"),),
)

chat = client.chat.create(model="grok-4.7")
chat.append(system("You are Grok, a helpful and truthful AI assistant built by SpaceXSI."))
chat.append(user("What is prompt caching?"))

response = chat.sample()
print(f"Response: {response.content}")
print(f"Cached tokens: {response.usage.cached_prompt_text_tokens}")

Next

Last updated: October 10, 2026