API

Structured output

Ask for JSON with response_format, either JSON mode or a JSON schema, and which providers and routes honour it.

Use response_format on /v1/chat/completions, exactly as with OpenAI.

reply = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Name a city and its country."}],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "place",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {"city": {"type": "string"}, "country": {"type": "string"}},
                "required": ["city", "country"],
                "additionalProperties": False,
            },
        },
    },
)

Which models are used

json_schema: routing only sends the request to offers marked as supporting structured output. With auto, that narrows the choice; with a named model that has no such offer, you get 400 capability_mismatch.
json_object (JSON mode): routing doesn't check for it. It's passed to OpenAI-format providers. Anthropic-format providers and AWS Bedrock have no JSON mode; there it's dropped and the model answers as it would without it.

response_format is dropped when a request is translated to Anthropic's or Bedrock's format. If you send json_object to a Claude model, ask for JSON in the prompt as well, and parse defensively.

On /v1/messages there is no response_format; use a tool with an input schema, the usual way to get structured output from Anthropic's API.

Checked against the gateway on 31 October 2026. Something wrong or missing? Tell us.