Tokenize a prompt

Count tokens with the tokenizer and chat template used by the served model. Tokenization is free.

POST /v1/tokenize

Authorization

Authorizationheader · string

Bearer $BOARDWALK_API_KEY. Requires inference:write.

Content-Typeheader · stringrequired

application/json

Request

curl https://api.boardwalk.cloud/v1/tokenize \
  -H "Authorization: Bearer $BOARDWALK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "support-agent",
  "prompt": "Hello"
}'

Request body

modelstringrequired

Callable model ID or revision pin.

promptstringoptional

Plain-text prompt, at most 256 KiB in UTF-8. Supply exactly one of prompt or messages.

messagesobject[]optional

Conversation messages; at least one. Includes the served chat template in the count.

Child fields

rolestringrequired

One of system, developer, user, assistant, or tool.

contentstring | object[] | nulloptional

Text content. Text parts contain type and text.

Child fields

type"text"required

Content part type.

textstringrequired

Text of this part.

namestringoptional

Name of the message author.

tool_callsobject[]optional

Calls returned in an assistant message.

Child fields

idstringrequired

Identifier of the tool call.

type"function"required

Tool type.

functionobjectrequired

Function invocation.

Child fields

namestringrequired

Function name.

argumentsstringrequired

JSON-encoded function arguments.

tool_call_idstringoptional

Call answered by a tool message.

reasoningstring | nulloptional

Reasoning from a previous assistant turn, when required by the model.

reasoning_contentstring | nulloptional

Legacy alias for reasoning. Accepted only on assistant messages.

Response

Example response. IDs, token counts, and rates are illustrative.

{
  "model": "support-agent",
  "hf_revision": "0123456789abcdef0123456789abcdef01234567",
  "count": 1,
  "max_model_len": 32768,
  "tokens": [9707]
}

modelstring

Resolved model.

hf_revisionstring

Pinned weight commit.

countinteger

Number of token IDs returned.

max_model_leninteger

Maximum combined input and output tokens.

tokensinteger[]

Token IDs in order.

Errors

Errors use the shared error format. Save the x-request-id response header for receipts and support.