Inference with Cortex AI Gateway¶

The Cortex AI Gateway exposes an inference endpoint that accepts requests in industry-standard API formats. Any client that lets you override its base URL can send requests through the gateway, including coding agents, agent frameworks, and the official OpenAI and Anthropic SDKs.

All inference runs within the Snowflake perimeter, and usage is attributed to the Snowflake user whose token made the request, so gateway traffic appears in your existing cost and access reporting.

Gateway endpoint¶

The gateway endpoint takes this form:

https://<account-host>/api/v2/aigateways/SNOWFLAKE

Append the path for the API you’re calling, either /v1/chat/completions or /v1/messages. See Choose your API.

To get the endpoint for your account, use:

SHOW AI GATEWAYS;

Pricing¶

Inference through the gateway is billed the same way as the Cortex Inference: on the number of tokens processed, at each model’s rate. Refer to the Snowflake Service Consumption Table for each model’s cost.

For attributing that spend to teams and users, and for capping it, see Cost management for Cortex AI Gateway.

Choose your API¶

The gateway supports two industry-standard API specifications. Pick the one your client requires:

Prerequisites¶

Before you begin, you need:

1. Your gateway endpoint, from SHOW AI GATEWAYS. See Gateway endpoint.
2. The USAGE privilege on the gateway. See Access control.
3. A programmatic access token (PAT) for authentication. See Using programmatic access tokens for authentication.
4. A model name to use in requests. See Model availability.
5. Access to the model itself. The gateway respects model access control: a request still needs whatever access the underlying model requires, such as the SNOWFLAKE.CORTEX_USER database role or the controls described in Privileges and model access for Cortex AI Functions. USAGE on the gateway doesn’t widen the set of models a user is entitled to, so sending traffic through it doesn’t expand your security perimeter.

Setting up authentication¶

The gateway expects the token as a bearer token:

Authorization: Bearer <SNOWFLAKE_PAT>

Quickstart¶

from openai import OpenAI

client = OpenAI(
  api_key="<SNOWFLAKE_PAT>",
  base_url="<gateway-endpoint>/v1"
)

response = client.chat.completions.create(
  model="openai-gpt-5",
  messages=[
    {"role": "user", "content": "How does a snowflake get its unique pattern?"}
  ]
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "<SNOWFLAKE_PAT>",
  baseURL: "<gateway-endpoint>/v1"
});

const response = await client.chat.completions.create({
  model: "openai-gpt-5",
  messages: [
    { role: "user", content: "How does a snowflake get its unique pattern?" }
  ],
});

console.log(response.choices[0].message.content);

import httpx
import anthropic

PAT = "<SNOWFLAKE_PAT>"

http_client = httpx.Client(
  headers={"Authorization": f"Bearer {PAT}"},
)

client = anthropic.Anthropic(
  api_key="not-used",
  base_url="<gateway-endpoint>",
  http_client=http_client,
  default_headers={"Authorization": f"Bearer {PAT}"},
)

response = client.messages.create(
  model="claude-sonnet-4-5",
  max_tokens=1024,
  messages=[
    {"role": "user", "content": "How does a snowflake get its unique pattern?"}
  ]
)

print(response.content[0].text)

curl "<gateway-endpoint>/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <SNOWFLAKE_PAT>" \
  -d '{
    "model": "openai-gpt-5",
    "messages": [
      {"role": "user", "content": "How does a snowflake get its unique pattern?"}
    ]
  }'

Supported features¶

The gateway exposes the same Chat Completions and Messages surfaces as Cortex Inference, so request and response fields, and the features built on them, behave the same way. See the Cortex Inference documentation for instructions on:

• Streaming responses over server-sent events for both APIs. See Streaming.
• Tool calling. See Tool calling.
• Structured output. See Structured output.
• Prompt caching. See Prompt caching.
• Image input. See Image input.
• Thinking and reasoning. See Thinking and reasoning.

For the complete field-level request and response reference, see Cortex Inference.

Instrument a coding agent¶

Agents differ in which API format they use, where their configuration lives, and how they send credentials. Use the section for your agent.

OpenCode¶

To point OpenCode at the gateway, define a custom provider in your ~/.config/opencode/opencode.json file. OpenCode uses the Chat Completions API through the OpenAI-compatible adapter:

{
  "$schema": "https://opencode.ai/config.json",
  "model": "snowflake-cortex/openai-gpt-5.4",
  "small_model": "snowflake-cortex/openai-gpt-5.4",
  "plugin": [
    [
      "@devtheops/opencode-plugin-otel",
      {
        "enabled": true,
        "tracePropagationProviders": ["snowflake-cortex"]
      }
    ]
  ],
  "provider": {
    "snowflake-cortex": {
      "options": {
        "baseURL": "<gateway-endpoint>/v1",
        "account": "<account-host>",
        "apiKey": "<SNOWFLAKE_PAT>",
        "headers": {
          "snow-agent-name": "opencode"
        }
      }
    }
  }
}

Where:

• baseURL is the gateway endpoint from SHOW AI GATEWAYS with /v1 appended.
• account is the host part of that endpoint, without the path.
• model and small_model are both qualified by the provider name, so they read snowflake-cortex/<model>. Setting small_model as well keeps OpenCode’s lightweight calls on the gateway rather than falling back to its default.
• The snow-agent-name header identifies the client on each request.
• The plugin entry loads the OpenTelemetry plugin and lists the providers it propagates trace context to. Naming snowflake-cortex in tracePropagationProviders makes OpenCode send a traceparent header on its gateway requests.

Other clients¶

Any client that lets you set a custom base URL and bearer token, and connects using either the Chat Completions API or Messages API can use the gateway. Set the base URL according to the API format the client uses, as described in Gateway endpoint, and supply your PAT as the bearer token.

Monitor gateway inference¶

Requests sent through the gateway are recorded in two places:

• Traces, for what each request did: models called, timing, tokens, and errors. See Observability for Cortex AI Gateway.
• Usage history, for credits consumed. See AI_GATEWAY_USAGE_HISTORY view.

For cost attribution, budgets, and per-user quotas, see Cost management for Cortex AI Gateway.