Run Python scripts in Snowflake¶

This topic walks you through developing, deploying, and scheduling Python scripts in Snowflake. You write and test your code locally, then deploy it as a Notebook Project Object — a schema-level object that runs your script non-interactively on a warehouse or compute pool. Once deployed, you can run your project on demand or schedule it with a task.

Prerequisites¶

To complete this guide, you need the following on your development machine:

Python 3.10, 3.11, or 3.12
The Snowflake CLI. If you haven’t used the Snowflake CLI before, run snow connection add to connect the CLI to your Snowflake account.

Set up your project¶

Clone the sample Python project:
```
git clone https://github.com/sfc-gh-jfreeberg/npo.git
```

Create a virtual environment and install the dependencies:

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements-prod.txt -r requirements-dev.txt

The template project contains the following files:

src/main.py — the main entry point for your script
src/args_example.py — shows how to pass and process command-line arguments
src/udf_example.py — shows how to register and reference UDFs
requirements-prod.txt — production dependencies (installed at run time in Snowflake)
requirements-dev.txt — development dependencies, including the Snowflake CLI
snow_app.yml — configuration for runtime, compute, and dependencies
tests/ — a sample test suite

Develop and test locally¶

Run the demo script locally to verify your environment:

python src/main.py

Run the test suite:

python -m pytest

You can extend the sample scripts to read or write data specific to your team or use case. Iterate locally until your script is ready, then configure and deploy it to Snowflake.

Configure your project¶

Before you deploy, configure runtime and dependency settings in the snow_app.yml file at the root of your project. This file sets default values so you don’t have to specify them every time you run the project.

`runtime`¶

Use the runtime property to specify the runtime version, compute platform, and other properties for running your project.

`runtime.version`¶

Required. The Container Runtime version to run the project on.

Allowed values:

python_3.10
python_3.11
python_3.12

For the full list of allowed values, see EXECUTE NOTEBOOK PROJECT.

`runtime.compute_pool`¶

The compute pool to run the project on. If not specified, the project runs on the active warehouse of the session.

`runtime.query_warehouse`¶

The warehouse to use for SQL queries when a runtime.compute_pool is specified. If not specified, queries run on the active warehouse of the session.

`dependencies`¶

A reference to the requirements.txt file that lists the Python dependencies for the project. Dependencies are installed from the built-in snowflake.snowpark.pypi_shared_repository PyPI repository.

dependencies: 'requirements.txt'

If you split production and development dependencies, specify your production dependencies:

dependencies: 'requirements-prod.txt'

Deploy to Snowflake¶

Deploy your project to Snowflake so it can be run non-interactively.

Create a stage to hold the project files:
```
snow stage create npo_stage
```
Copy the project contents to the stage:
```
snow stage copy --recursive --overwrite . @npo_stage/
```

Create the Notebook Project Object:

snow notebook project create my_project --source @npo_stage/ --overwrite

The project is now deployed to your Snowflake account.

Run your project¶

You can run a deployed project using the Snowflake CLI or SQL.

Snowflake CLI¶

snow notebook project execute -n 'my_project'

SQL¶

EXECUTE NOTEBOOK PROJECT my_db.my_schema.my_project
  MAIN_FILE = 'main.py';

If you provide COMPUTE_POOL, QUERY_WAREHOUSE, RUNTIME, or EXTERNAL_ACCESS_INTEGRATIONS in the command, those values override the defaults set in snow_app.yml.

Schedule recurring runs¶

To run your project on a recurring schedule, wrap the run command in a Snowflake Task:

CREATE OR REPLACE TASK my_task
  WAREHOUSE = my_warehouse
  SCHEDULE = 'USING CRON 0 0 * * * UTC'
AS
  EXECUTE NOTEBOOK PROJECT my_db.my_schema.my_project
    MAIN_FILE = 'main.py';

After creating the task, resume it to start the schedule:

ALTER TASK my_task RESUME;

Automate deployment with CI/CD¶

You can run the deploy commands in a CI/CD pipeline to automatically deploy your project when you merge changes. The following example shows a GitHub Actions workflow that deploys on push to the main branch:

name: Deploy to Snowflake
on:
  push:
    branches: [main]
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: Snowflake-Labs/snowflake-cli-action@v1
        with:
          cli-version: "latest"
          default-config-file-path: "config.toml"
      - run: snow stage copy --recursive --overwrite . @npo_stage/
      - run: snow notebook project create my_project --source @npo_stage/ --overwrite

Adapt this pattern to your CI system by running snow stage copy followed by snow notebook project create --overwrite after authenticating the Snowflake CLI.

Register UDFs from your project¶

If your script depends on User Defined Functions and you want to manage their lifecycle through the project, create them during the script’s run with session.udf.register or the @udf decorator. See src/udf_example.py in the sample project for an example.

Reference¶

SQL commands¶

CREATE NOTEBOOK PROJECT¶

Create a project from a stage:

CREATE NOTEBOOK PROJECT [ IF NOT EXISTS ] <database_name>.<schema_name>.<project_name>
  FROM '@<stage_path>'
  [ COMMENT = '<string_literal>' ];

Create a project from a workspace:

CREATE NOTEBOOK PROJECT <database_name>.<schema_name>.<project_name>
  FROM 'snow://workspace/<workspace_path>'
  [ COMMENT = '<string_literal>' ];

For full syntax details, see CREATE NOTEBOOK PROJECT.

EXECUTE NOTEBOOK PROJECT¶

EXECUTE NOTEBOOK PROJECT <database_name>.<schema_name>.<project_name>
  MAIN_FILE = 'main.py'
  [ COMPUTE_POOL = '<compute_pool_name>' ]
  [ QUERY_WAREHOUSE = '<warehouse_name>' ]
  [ RUNTIME = '<runtime_version>' ]
  [ EXTERNAL_ACCESS_INTEGRATIONS = ( <integration_name> [ , ... ] ) ];

For full syntax details, see EXECUTE NOTEBOOK PROJECT.

Snowflake CLI commands¶

snow notebook project create¶

Usage: snow notebook project create [OPTIONS] [NAME]

Creates a notebook project in Snowflake.

╭─ Arguments ───────────────────────────────────────────────────────────────────────────────────────────────╮
│   name      [NAME]  Name of the Snowpark project.                                                         │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --source                  TEXT  Source location of the notebook project. Supports stage path (starting    │
│                                 with '@') or workspace path (starting with 'snow://workspace/').          │
│ --comment                 TEXT  Comment for the notebook project.                                         │
│ --overwrite                     Overwrite the notebook project if it already exists.                      │
│ --skip-if-exists                Skip the creation of the notebook project if it already exists.           │
│ --help            -h            Show this message and exit.                                               │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯

snow notebook project execute¶

Usage: snow notebook project execute [OPTIONS] [NAME] ARGUMENTS

Executes a notebook project in Snowflake.

╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────────────────╮
│   name           [NAME]  Name of the notebook project.                                                       │
│   arguments      TEXT    Arguments to pass to the notebook project.                                          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --main-file                             TEXT  Main file of the notebook project.                             │
│ --compute-pool                          TEXT  Compute pool to run the notebook project on.                   │
│ --query-warehouse                       TEXT  Query warehouse to run the notebook project on.                │
│ --runtime                               TEXT  Runtime to run the notebook project on.                        │
│ --requirements-file                     TEXT  Requirements file to use for the notebook project.             │
│ --external-access-integrations          TEXT  External access integrations to use for the notebook project.  │
│ --help                          -h            Show this message and exit.                                    │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

snow notebook project drop¶

Usage: snow notebook project drop [OPTIONS] [NAME]

Drops a notebook project in Snowflake.

╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────────────────╮
│   name      [NAME]  Name of the notebook project.                                                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --help  -h        Show this message and exit.                                                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

snow notebook project list¶

Usage: snow notebook project list [OPTIONS]

Lists notebook projects in Snowflake.

╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --help  -h        Show this message and exit.                                                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Limitations¶

The following limitations apply:

.ipynb notebooks can only run on compute pools, not warehouses. .py files can run on either.
Java and Scala are not supported.
Event table logging is not available when running on warehouses.
Only requirements.txt is supported for specifying dependencies. pyproject.toml is not supported.