Inference with Cortex AI Gateway¶

The Cortex AI Gateway exposes an inference endpoint that accepts requests in industry-standard API formats. Any client that lets you override its base URL can send requests through the gateway, including coding agents, agent frameworks, and the official OpenAI and Anthropic SDKs.

All inference runs within the Snowflake perimeter, and usage is attributed to the Snowflake user whose token made the request, so gateway traffic appears in your existing cost and access reporting.

Gateway endpoint¶

The gateway endpoint takes this form:

https://<account-host>/api/v2/aigateways/SNOWFLAKE

Append the path for the API you’re calling: /v1/chat/completions, /v1/messages, or /v1/responses. See Choose your API.

To get the endpoint for your account, use:

DESCRIBE AI GATEWAY SNOWFLAKE;

Pricing¶

Inference through the gateway is billed the same way as the Cortex Inference: on the number of tokens processed, at each model’s rate. Refer to the Snowflake Service Consumption Table for each model’s cost.

For attributing that spend to teams and users, and for capping it, see Cost management for Cortex AI Gateway.

Choose your API¶

The gateway supports three industry-standard API specifications. Pick the one your client requires:

Prerequisites¶

Before you begin, you need:

1. Your gateway endpoint, from DESCRIBE AI GATEWAY SNOWFLAKE. See Gateway endpoint.
2. The USAGE privilege on the gateway. See Access control.
3. A token for authentication: an OAuth access token, a key-pair JSON Web Token (JWT), or a programmatic access token (PAT). See Setting up authentication.
4. A model name to use in requests. See Model availability.
5. Access to the model itself. The gateway respects model access control: a request still needs whatever access the underlying model requires, such as the SNOWFLAKE.CORTEX_USER database role or the controls described in Privileges and model access for Cortex AI Functions. USAGE on the gateway doesn’t widen the set of models a user is entitled to, so sending traffic through it doesn’t expand your security perimeter.

Setting up authentication¶

The gateway accepts the same tokens as other Snowflake REST APIs, using the methods described in Authenticating Snowflake REST APIs with Snowflake: OAuth, key-pair authentication with a JWT, or a programmatic access token. It’s recommended to use a short-lived token, such as an OAuth access token, rather than a PAT, which stays valid until it expires or is revoked.

Send the token as a bearer token in the Authorization header:

Authorization: Bearer <token>

Quickstart¶

import os
from openai import OpenAI

client = OpenAI(
  api_key=os.environ["SNOWFLAKE_TOKEN"],
  base_url="<gateway-endpoint>/v1"
)

response = client.chat.completions.create(
  model="openai-gpt-5",
  messages=[
    {"role": "user", "content": "How does a snowflake get its unique pattern?"}
  ]
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.SNOWFLAKE_TOKEN,
  baseURL: "<gateway-endpoint>/v1"
});

const response = await client.chat.completions.create({
  model: "openai-gpt-5",
  messages: [
    { role: "user", content: "How does a snowflake get its unique pattern?" }
  ],
});

console.log(response.choices[0].message.content);

import os
import httpx
import anthropic

token = os.environ["SNOWFLAKE_TOKEN"]

http_client = httpx.Client(
  headers={"Authorization": f"Bearer {token}"},
)

client = anthropic.Anthropic(
  api_key="not-used",
  base_url="<gateway-endpoint>",
  http_client=http_client,
  default_headers={"Authorization": f"Bearer {token}"},
)

response = client.messages.create(
  model="claude-sonnet-4-5",
  max_tokens=1024,
  messages=[
    {"role": "user", "content": "How does a snowflake get its unique pattern?"}
  ]
)

print(response.content[0].text)

from openai import OpenAI

client = OpenAI(
  api_key="<SNOWFLAKE_PAT>",
  base_url="<gateway-endpoint>/v1"
)

response = client.responses.create(
  model="openai-gpt-5.6-sol",
  input="How does a snowflake get its unique pattern?",
  store=False,
)

print(response.output_text)

curl "<gateway-endpoint>/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $SNOWFLAKE_TOKEN" \
  -d '{
    "model": "openai-gpt-5",
    "messages": [
      {"role": "user", "content": "How does a snowflake get its unique pattern?"}
    ]
  }'

Supported features¶

The gateway exposes the same Chat Completions, Messages, and Responses surfaces as Cortex Inference, so request and response fields, and the features built on them, behave the same way. See the Cortex Inference documentation for instructions on:

• Streaming responses over server-sent events for both APIs. See Streaming.
• Tool calling. See Tool calling.
• Structured output. See Structured output.
• Prompt caching. See Prompt caching.
• Image input. See Image input.
• Thinking and reasoning. See Thinking and reasoning.

For the complete field-level request and response reference, see Cortex Inference. For the fields and features the Responses API supports, see Responses API reference.

Instrument a coding agent¶

You can launch the agent with Snowflake CLI or configure it yourself:

• Snowflake CLI launches a coding agent, such as OpenCode, with its traffic already routed, using the endpoint and credential from a Snowflake CLI connection.
• Manual setup means configuring the base URL and the credential in the agent yourself. Use it for an agent Snowflake CLI doesn’t launch, or when the configuration needs to persist outside the CLI.

Snowflake CLI¶

The snow ai commands launch a coding agent that’s already installed on your machine, with its model requests routed through the gateway. The Snowflake CLI connection can authenticate with OAuth or with a PAT:

snow ai opencode

To pass arguments to the agent, put them after --agent-args:

snow ai opencode -c <connection> --agent-args -m snowflake-cortex/openai-gpt-5.4

For how snow ai works, the supported authenticators, and what each command configures in its agent, see snow ai commands.

Manual setup¶

Agents differ in which API format they use, where their configuration lives, and how they send credentials. Use the section for your agent.

OpenCode¶

To point OpenCode at the gateway, define a custom provider in your ~/.config/opencode/opencode.json file. OpenCode uses the Chat Completions API through the OpenAI-compatible adapter:

{
  "$schema": "https://opencode.ai/config.json",
  "model": "snowflake-cortex/openai-gpt-5.4",
  "small_model": "snowflake-cortex/openai-gpt-5.4",
  "plugin": [
    [
      "@devtheops/opencode-plugin-otel",
      {
        "enabled": true,
        "tracePropagationProviders": ["snowflake-cortex"]
      }
    ]
  ],
  "provider": {
    "snowflake-cortex": {
      "options": {
        "baseURL": "<gateway-endpoint>/v1",
        "account": "<account-host>",
        "apiKey": "{env:SNOWFLAKE_TOKEN}",
        "headers": {
          "snow-agent-name": "opencode"
        }
      }
    }
  }
}

Where:

• baseURL is the gateway endpoint from DESCRIBE AI GATEWAY SNOWFLAKE with /v1 appended.
• account is the host part of that endpoint, without the path.
• apiKey uses OpenCode’s {env:...} substitution to read the token from the SNOWFLAKE_TOKEN environment variable, so the token isn’t stored in the file.
• model and small_model are both qualified by the provider name, so they read snowflake-cortex/<model>. Setting small_model as well keeps OpenCode’s lightweight calls on the gateway rather than falling back to its default.
• The snow-agent-name header identifies the client on each request.
• The plugin entry loads the OpenTelemetry plugin and lists the providers it propagates trace context to. Naming snowflake-cortex in tracePropagationProviders makes OpenCode send a traceparent header on its gateway requests.

Other clients¶

Any client that lets you set a custom base URL and bearer token, and connects using the Chat Completions API, Messages API, or Responses API can use the gateway. Set the base URL according to the API format the client uses, as described in Gateway endpoint, and supply your token as the bearer token. See Setting up authentication.

Monitor gateway inference¶

Requests sent through the gateway are recorded in two places:

• Traces, for what each request did: models called, timing, tokens, and errors. See Observability for Cortex AI Gateway.
• Usage history, for credits consumed. See AI_GATEWAY_USAGE_HISTORY view.

For cost attribution, budgets, and per-user quotas, see Cost management for Cortex AI Gateway.