Images

Attach an image to a user message and the model reads it alongside the text. Each API has its own content-part spelling: image_url parts on Chat Completions, input_image on Responses, image blocks on Messages.

Works with

Image parts are accepted on every chat model. The catalog does not yet flag which models have vision; if the underlying model can't process images, the upstream provider's error is relayed back to you. The Claude, GPT, Gemini, and Grok families read images today. The per-model column is on the capability matrix.

Request

Chat Completions

response = client.chat.completions.create(
    model="sonnet",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What's in this image?"},
            {"type": "image_url", "image_url": {"url": "https://example.com/chart.png"}},
        ],
    }],
)
print(response.choices[0].message.content)

Responses

response = client.responses.create(
    model="sonnet",
    input=[{
        "role": "user",
        "content": [
            {"type": "input_text", "text": "What's in this image?"},
            {"type": "input_image", "image_url": "https://example.com/chart.png"},
        ],
    }],
)
print(response.output_text)

Messages

message = client.messages.create(
    model="sonnet",
    max_tokens=1024,
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "What's in this image?"},
            {"type": "image", "source": {"type": "url", "url": "https://example.com/chart.png"}},
        ],
    }],
)
print(message.content[0].text)

Chat Completions

const response = await client.chat.completions.create({
  model: "sonnet",
  messages: [{
    role: "user",
    content: [
      { type: "text", text: "What's in this image?" },
      { type: "image_url", image_url: { url: "https://example.com/chart.png" } },
    ],
  }],
});
console.log(response.choices[0].message.content);

Responses

const response = await client.responses.create({
  model: "sonnet",
  input: [{
    role: "user",
    content: [
      { type: "input_text", text: "What's in this image?" },
      { type: "input_image", image_url: "https://example.com/chart.png", detail: "auto" },
    ],
  }],
});
console.log(response.output_text);

Messages

const message = await client.messages.create({
  model: "sonnet",
  max_tokens: 1024,
  messages: [{
    role: "user",
    content: [
      { type: "text", text: "What's in this image?" },
      { type: "image", source: { type: "url", url: "https://example.com/chart.png" } },
    ],
  }],
});
console.log(message.content[0]);

These samples use sonnet, which reads images and draws the prepaid wallet.

Image sources:

• A data: URL (data:<media type>;base64,<data>) or a public http(s) URL, which the upstream provider fetches. On Messages, base64 and url sources both work.
• JPEG, PNG, GIF, and WebP are passed through. Other inline raster formats (data: URLs) are transcoded to PNG server-side; remote URLs are handed to the provider unchanged.
• A malformed data URL or invalid base64 returns 400.

Response

Nothing about the response shape changes: the model's description arrives as ordinary text. Image tokens are metered as input tokens.

Per-model differences

• Anthropic's Files API references ("source": {"type": "file"}) are unsupported on Messages, and behavior is target-dependent: some providers reject them, others ignore them. Use base64 or url sources.
• PDF and other file inputs are not a documented feature yet; a file part reaches only the OpenAI-transport models today and is dropped elsewhere.

Errors you can hit

400 for a malformed data URL or invalid base64. A model without vision returns its provider's own error, relayed with the original status. See Errors.