xai_api.Tokenize

An API service to tokenize input prompts.

Methods of xai_api.Tokenize

TokenizeText

unary

Convert text to a sequence of tokens.

Tokenize text

import os
import xai_sdk

client = xai_sdk.Client(api_key=os.getenv("XAI_API_KEY"))

tokens = client.tokenize.tokenize_text(
    text="Hello, world!",
    model="grok-4.6",
)

print(f"Token count: {len(tokens)}")
for token in tokens:
    print(f"  {token.token_id}: {token.string_token!r}")

buf curl --schema . --protocol grpc \
  -H "Authorization: Bearer $XAI_API_KEY" \
  --data '{
    "text": "Hello, world!",
    "model": "grok-4.6"
  }' \
  https://api.x.ai/xai_api.Tokenize/TokenizeText

Last updated: September 2, 2026

gRPC API

Tokenize

Methods of xai_api.Tokenize

Tokenize text