Inference API
Batches
Create a new batch
/v1/batches
Create a new batch for processing requests.
Request Body
name
string
required
The name of the batch created.
Response Body
batch_id
string
The ID of the batch.
create_api_key_id
string
ID of the API key that was used to create the batch.
create_time
string
The time when the batch was created.
name
string
The human-readable name of the batch.
state
object
Holds aggregate information about the current state of a batch process.
List batches
/v1/batches
List all batches for the current team.
Query parameters
limit
integernull
Number of elements to return.
pagination_token
stringnull
Optional page token to retrieve a specific page. Provided by `pagination_token` in `ListBatchesResponse`.
Response Body
batches
array
The information about the batches.
Get batch
/v1/batches/{batch_id}
Get information about a specific batch.
Path parameters
batch_id
string
required
The unique identifier of the batch
Response Body
batch_id
string
The ID of the batch.
create_api_key_id
string
ID of the API key that was used to create the batch.
create_time
string
The time when the batch was created.
name
string
The human-readable name of the batch.
state
object
Holds aggregate information about the current state of a batch process.
List batch requests in a batch
/v1/batches/{batch_id}/requests
List metadata for all requests in a batch.
Path parameters
batch_id
string
required
The unique identifier of the batch
Query parameters
limit
integernull
Maximum number of items to return in a single page (max 1000)
pagination_token
stringnull
Token for retrieving the next page of results
Response Body
batch_request_metadata
array
The batch request metadata for the given batch.
Add batch requests to a batch
/v1/batches/{batch_id}/requests
Add multiple requests to an existing batch.
Path parameters
batch_id
string
required
The unique identifier of the batch
Request Body
batch_requests
array
required
List of batch requests to add to the batch
Get processing results of a batch
/v1/batches/{batch_id}/results
List the processing results for a batch.
Path parameters
batch_id
string
required
The unique identifier of the batch
Query parameters
limit
integernull
Maximum number of items to return in a single page (max 1000)
pagination_token
stringnull
Token for retrieving the next page of results
Response Body
results
array
The results that has been processed.
Cancel processing on a batch
/v1/batches/{batch_id}:cancel
Cancel processing of all requests in a batch.
Path parameters
batch_id
string
required
The unique identifier of the batch to cancel
Response Body
batch_id
string
The ID of the batch.
create_api_key_id
string
ID of the API key that was used to create the batch.
create_time
string
The time when the batch was created.
name
string
The human-readable name of the batch.
state
object
Holds aggregate information about the current state of a batch process.
Did you find this page helpful?