Tutorial: Get started with Named Channels using the REST API¶

This guide shows you how to stream data into Snowflake using the Snowpipe Streaming REST API and a JSON Web Token (JWT) generated with Snowflake CLI. It covers Named Channel REST ingestion for ordered, exactly-once workloads.

For the Elastic Channel REST path (simpler, recommended for most new applications), see Tutorial: Get started with Elastic Channels (REST).

Prerequisites¶

Before you begin, ensure you have the following items:

Snowflake User and Objects:

A Snowflake user that is configured for key-pair authentication. Register your public key by using the following SQL command:

ALTER USER MY_USER SET RSA_PUBLIC_KEY='<your-public-key>';

A Snowflake database, schema, and a target table for streaming ingestion. You can create them by using the following SQL commands and replacing placeholders like MY_DATABASE, MY_SCHEMA, MY_TABLE with the names that you want:

-- Create Database and Schema
CREATE OR REPLACE DATABASE MY_DATABASE;
CREATE OR REPLACE SCHEMA MY_SCHEMA;

-- Create Target Table
CREATE OR REPLACE TABLE MY_TABLE (
    id NUMBER,
    c1 NUMBER,
    ts STRING
);

ACCOUNT_IDENTIFIER:

We suggest that you use Format 1 for the ACCOUNT_IDENTIFIER, which uses the account name within your organization; for example, myorg-account123. For more information on the format, see Account identifiers.

Installed tools:

• curl: For making HTTP requests.
• jq: For parsing JSON responses.
• Snowflake CLI: For generating a JWT. The REST API streams the rows. It doesn’t generate that token, and it doesn’t develop or deploy Snowpark code, Streamlit in Snowflake apps, Snowflake Native Apps, Snowpark Container Services services, notebooks, Git repositories, or dbt and DCM Projects projects.

Generated JWT:

Generate your JWT by using Snowflake CLI:

snow connection generate-jwt --account <ACCOUNT_IDENTIFIER> --user MY_USER --private-key-file rsa_key.p8

Prerequisites and setup¶

The following steps configure the environment variables, ingest host, and sample rows required for Named Channel REST ingestion.

Step 1: Set environment variables¶

Set up the necessary environment variables for your Snowflake account and the streaming operation. Note that the PIPE variable targets the default streaming pipe associated with your table.

# Paste the JWT token obtained from Snowflake CLI
export JWT_TOKEN="PASTE_YOUR_JWT_TOKEN_HERE"

# Configure your Snowflake account and resources:
export ACCOUNT="<ACCOUNT_IDENTIFIER>" # For example, ab12345
export USER="MY_USER"
export DB="MY_DATABASE"
export SCHEMA="MY_SCHEMA"
export TABLE="MY_TABLE"

# Replace ACCOUNT with your Account URL Host to form the control plane host:
export CONTROL_HOST="${ACCOUNT}.snowflakecomputing.com"

Step 2: Discover and configure the ingest host¶

Snowpipe Streaming REST uses two hostnames: the Snowflake account endpoint (CONTROL_HOST) and an account-specific ingest endpoint (INGEST_HOST) returned by GET /v2/streaming/hostname.

If you use AWS PrivateLink, Azure Private Link, or Google Cloud Private Service Connect, set CONTROL_HOST to the hostname from the privatelink-account-url value returned by SYSTEM$GET_PRIVATELINK_CONFIG. Don’t include https:// or a trailing slash.

The ingest host is the endpoint for streaming data. Discover the ingest host by using your JWT:

export INGEST_HOST=$(curl -sS -X GET \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -H "X-Snowflake-Authorization-Token-Type: KEYPAIR_JWT" \
  "https://${CONTROL_HOST}/v2/streaming/hostname")

export INGEST_HOST=$(printf '%s' "$INGEST_HOST" | tr '_' '-')
echo "Ingest Host: $INGEST_HOST"

Configure private DNS for the ingest host¶

Complete the shared network connectivity prerequisites before continuing. They cover public and private connectivity, private DNS for the discovered ingest hostname, firewall inventory, and checks from the application runtime. For private connectivity, route the normalized INGEST_HOST through the existing Snowflake private endpoint; a second Snowflake service private endpoint isn’t required.

Obtain a scoped token to authorize operations on the ingest host:

export SCOPED_TOKEN=$(curl -sS -X POST "https://$CONTROL_HOST/oauth/token" \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -d "grant_type=urn:ietf:params:oauth:grant-type:jwt-bearer&scope=${INGEST_HOST}")

echo "Scoped Token obtained for ingest host"

Step 3: Create sample rows¶

Create a batch in newline-delimited JSON (NDJSON) format:

export NOW_TS=$(date -u +"%Y-%m-%dT%H:%M:%SZ")

cat <<EOF > rows.ndjson
{"id":1,"c1":$RANDOM,"ts":"$NOW_TS"}
EOF

Open and use a Named Channel¶

Use a Named Channel when you require ordering or exactly-once recovery. Complete Steps 1 through 3 in Prerequisites and setup, then set the Named Channel variables:

export PIPE="MY_TABLE-STREAMING"
export CHANNEL="MY_CHANNEL"

Step 4: Open the Named Channel¶

curl -sS -X PUT \
  -H "Authorization: Bearer $SCOPED_TOKEN" \
  -H "Content-Type: application/json" \
  "https://${INGEST_HOST}/v2/streaming/databases/$DB/schemas/$SCHEMA/pipes/$PIPE/channels/$CHANNEL" \
  -d '{"offset_token":"0"}' | tee open_resp.json | jq .

Step 5: Append rows with offset and continuation tokens¶

Use the continuation token returned by the open operation and an application-managed source offset:

export CONT_TOKEN=$(jq -r '.next_continuation_token' open_resp.json)
export OFFSET_TOKEN="1"

curl -sS -X POST \
  -H "Authorization: Bearer $SCOPED_TOKEN" \
  -H "Content-Type: application/x-ndjson" \
  "https://${INGEST_HOST}/v2/streaming/data/databases/$DB/schemas/$SCHEMA/pipes/$PIPE/channels/$CHANNEL/rows?continuationToken=$CONT_TOKEN&startOffsetToken=$OFFSET_TOKEN&endOffsetToken=$OFFSET_TOKEN" \
  --data-binary @rows.ndjson | tee append_resp.json | jq .

Use the next_continuation_token from each append response in the next append request.

Step 6: Verify committed progress¶

curl -sS -X POST \
  -H "Authorization: Bearer $SCOPED_TOKEN" \
  -H "Content-Type: application/json" \
  "https://${INGEST_HOST}/v2/streaming/databases/$DB/schemas/$SCHEMA/pipes/$PIPE:bulk-channel-status" \
  -d "{\"channel_names\":[\"$CHANNEL\"]}" | jq ".channel_statuses.\"$CHANNEL\""

Wait until last_committed_offset_token is 1 before advancing the source offset.

Step 7: Verify the data¶

After last_committed_offset_token advances, query the target table:

SELECT * FROM MY_DATABASE.MY_SCHEMA.MY_TABLE ORDER BY id;

(Optional) Step 8: Clean up¶

rm -f rows.ndjson open_resp.json append_resp.json
unset JWT_TOKEN SCOPED_TOKEN ACCOUNT USER DB SCHEMA TABLE PIPE CHANNEL CONTROL_HOST INGEST_HOST NOW_TS CONT_TOKEN OFFSET_TOKEN

Troubleshooting¶

• HTTP 401 (Unauthorized): Verify that your JWT token is valid and not expired. If needed, regenerate it.
• HTTP 404 (Not Found): Double-check that the database, schema, table, or pipe names are spelled correctly and exist in your Snowflake account.
• HTTP 429 (Too Many Requests): Retry using exponential backoff with random variation in retry delays (jitter). Don’t assume a fixed reserved request rate.
• No Ingest Host: Ensure your control plane host URL is correct and accessible.

Private connectivity troubleshooting¶

For unresolved account or ingest hostnames, HTTPS timeouts, or TLS hostname mismatches, use the commands and symptom table in Network connectivity prerequisites. Test from the application runtime, not only a developer laptop.