Build a data copilot¶
This tutorial builds a data engineering copilot: a small web app whose features are each powered by the Cortex Code Agent SDK. Instead of hand-writing SQL, you describe what you want in natural language, and the agent uses its built-in SQL tool to write and run queries against Snowflake, then returns typed, structured results your application can use directly.
The focus is the SDK usage, not the web framework. The examples use a Next.js app because it makes the agent easy to drive from a browser, but every SDK pattern here works the same way in a plain script, a scheduled job, or a continuous integration and continuous delivery (CI/CD) step.
What you’ll build¶
A data engineering copilot with three tools, each backed by the SDK:
- Pipeline health audit: describe a schema, and the agent discovers its tables, counts rows, checks load freshness, and returns a typed health report. Uses structured output.
- Schema drift detector: compare two schemas and get a typed list of added, removed, and type-changed columns, with breaking changes flagged. Plugs directly into a deploy gate.
- AI query optimizer: paste a slow query, and a multi-turn agent session diagnoses the bottleneck in the first turn and produces an optimized rewrite in the second, keeping full context between turns.
Each tool is a small API route that sends a natural-language prompt to the SDK and lets the agent do the SQL work. The app never hard-codes a query.
Prerequisites¶
- Node.js 22 or later.
- Cortex Code CLI installed:
- Snowflake connection configured through Snowflake CLI connection settings, typically in
~/.snowflake/connections.toml, with~/.snowflake/config.tomlalso supported for existing setups (see Configuring connections): - A schema to point the copilot at. Any schema in your account works. Optional: create demo data includes a script that creates a small schema with tables in deliberately mixed health states.
Project setup¶
Create the project and install the SDK:
The SDK authenticates through your Snowflake CLI connection. It uses your default connection unless you point it at a named one. The app reads the connection name from an environment variable so the same code works locally, in Cortex Code Desktop, and in Snowpark Container Services. Two variables are supported, checked in this order:
SNOWFLAKE_DEFAULT_CONNECTION_NAME: the primary variable. Cortex Code Desktop and Snowpark Container Services set this automatically, so you might already have it set without knowing it.SNOWFLAKE_CONNECTION_NAME: an override you can set to point the app at a different named connection.
If neither is set, the SDK falls back to the Snowflake CLI default connection.
Before building the web UI, confirm the SDK can reach Snowflake with a one-file script. Save it as scripts/smoke-test.ts
and run it with npx tsx scripts/smoke-test.ts:
Once that prints results, you’re connected. The rest of this guide wraps this same query() call in API routes.
Project structure¶
The finished app has one API route per tool, a shared SDK helper, and a UI page per tool. Only the API routes and the helper contain SDK code; the pages are ordinary React:
Optional: create demo data¶
To reproduce the audit against a schema with known issues, run this script in a worksheet. It creates five tables in deliberately different states: healthy, stale, empty, and high-null.
Walk through the key SDK usage¶
A shared options helper¶
Every tool configures the SDK the same way, so it’s worth centralizing in lib/sdk.ts. The app runs the agent
unattended (there’s no human to approve each tool call in a web request), so it auto-approves tools with
permissionMode: "bypassPermissions" and the required allowDangerouslySkipPermissions safety flag, disables
external Model Context Protocol (MCP) servers, and picks up the connection name from the environment.
The same file also holds two small helpers that pull text and SQL out of the agent’s streamed messages, which the tools reuse to show the agent’s work in the browser:
Warning
permissionMode: "bypassPermissions" runs every tool without prompting. Use it only in trusted, sandboxed
environments such as your own server or a CI/CD job. For workflows that need human oversight, use allowedTools,
disallowedTools, or the canUseTool callback instead. See
Handle approvals and user input.
Tool 1: Pipeline health audit¶
The audit sends one prompt and a JSON Schema, then lets the agent write and run the SQL. Because the request passes
outputFormat, the final result is validated against the schema, so the route receives a typed report instead of free
text it would have to parse. As the agent works, the route streams its text and each SQL statement to the browser
using the helpers from lib/sdk.ts.
Tool 2: Schema drift detector¶
The drift detector uses the same structured-output pattern, but the typed result drives a decision: if the agent finds any breaking change, the app can fail a deploy. This turns a natural-language request into a deterministic gate.
Tool 3: AI query optimizer¶
The optimizer needs two turns that share context: first diagnose, then rewrite. Instead of query(), it uses a
session so the second prompt can rely on everything the agent learned in the first. It also appends a system prompt to
steer the agent toward Snowflake-specific advice.
Run it¶
Start the app:
Open http://localhost:3000, choose a tool, and describe what you want. The rest of this section shows the kind of
output each tool produces.
Pipeline health audit output¶
As the audit runs, the route streams the agent’s reasoning and each SQL statement it writes, so the browser shows the work in progress:
The final result event carries the typed PipelineReport. The app renders it as a summary count plus one row per
table:
| Table | Rows | Last loaded | Status | Recommendation |
|---|---|---|---|---|
ORDERS | 100,000 | ~1 hour ago | healthy | None |
CUSTOMERS | 10,000 | ~1 hour ago | healthy | None |
PRODUCT_EVENTS | 25,000 | ~3 days ago | stale | Last load is about 3 days old; run the ingestion job to pick up new events. |
SESSION_METRICS | 0 | Never | empty | Table has no rows; confirm the pipeline populates it or drop it. |
AD_SPEND | 8,000 | ~2 hours ago | high_nulls | campaign_id and channel are about 33% NULL; check the source mapping. |
Two tables are healthy; three need attention: PRODUCT_EVENTS is stale, SESSION_METRICS is empty, and AD_SPEND
has a high NULL rate.
Schema drift detector output¶
Comparing a staging schema against production returns a typed SchemaDiff your deploy gate can act on:
Because breakingChanges is above zero, the CI step from the previous section exits non-zero and blocks the deploy.
AI query optimizer output¶
Paste a slow query:
Turn 1 diagnoses the bottleneck. The agent inspects the table metadata and reports the primary problem, a full
micro-partition scan on SALES:
The predicate s.saletime >= DATEADD('month', -6, CURRENT_TIMESTAMP()) can’t prune any micro-partitions because
SALES has no clustering key, so Snowflake scans every partition. The agent also flags a secondary bottleneck: the
e.catid IN (1, 2, 3) filter lives on EVENT and only removes SALES rows after the join completes.
Turn 2 produces the rewrite, reusing the diagnosis from turn 1. It pre-filters EVENT in a CTE, reorders the
joins so the smallest filtered set drives the query, and groups by raw columns instead of positional aliases:
The rewrite delivers its full benefit only after a one-time clustering change, which the agent calls out as a prerequisite:
Extend it¶
The same building blocks support more automation:
- Run it headless in CI/CD: drive the audit or drift check from a scheduled job with no human in the loop. Cap
runaway work with
maxTurns. - Log every query the agent runs: add a
PostToolUsehook to append each SQL statement to an audit log for compliance. See Hooks. - Combine local files with Snowflake: give the agent the
Read,Glob, andWritetools alongsideSQLand pointcwdat a dbt project, so it can cross-reference your models against the actual schema.
Next steps¶
- Structured output: Return validated JSON from agent workflows
- Multi-turn sessions and streaming input: Maintain context across multiple exchanges
- Handle approvals and user input: Control which tools the agent can use
- Hooks: Run custom code at key points in the agent lifecycle
- TypeScript SDK reference: Full TypeScript API reference
- Python SDK reference: Full Python API reference
Legal notices¶
Where your configuration of Cortex Code uses a model provided on the Model and Service Pass-Through Terms, your use of that model is further subject to the terms for that model on that page.
The data classification of inputs and outputs are as set forth in the following table.
| Input data classification | Output data classification | Designation |
|---|---|---|
| Usage Data | Customer Data | Covered AI Features [1] |
For additional information, refer to Snowflake AI and ML.