Run Python Jobs in Snowflake¶

This topic walks you through developing, deploying, and scheduling Python scripts in Snowflake on Warehouse or Compute Pools. You can write and test your code locally using your favorite IDE or with Cortex Code, then deploy it as a Notebook Project Object — a schema-levelobject that runs your script non-interactively on a warehouse or compute pool. Once deployed, you can run your project on demand or schedule it with a task.

Code Samples¶

Please see this repository for multiple sample Python projects with setup and deployment instructions!

Tutorial¶

1. Clone the samples repository and switch to the snowpark-uv/ directory. (There are other examples in the repository)

git clone https://github.com/sfc-gh-jfreeberg/npo.git
cd snowpark-uv

2. If you don’t already have uv installed, follow the installation instructions.

3. Install the development build of Snowflake CLI as a uv tool (isolated from the project .venv):

uv tool install git+https://github.com/snowflakedb/snowflake-cli@notebook-project

If snow is not on your PATH, run uv tool update-shell and restart your terminal.

Connection configuration¶

1. Snowpark sessions use a TOML connection file. This template expects a connection named snowpark (passed as -c snowpark when deploying). Use the same name in src/main.py for local runs.

snow connection add

2. When prompted, use snowpark as the connection name. Verify it works:

snow connection test --connection snowpark

3. Sync the project environment (creates .venv and installs dependencies from pyproject.toml):

uv sync --managed-python

Develop and test locally¶

1. Run the demo script locally to verify your environment:

uv run src/main.py

2. Run the test suite:

uv add --dev pytest
uv run pytest

You can extend the sample scripts to read or write data specific to your team or use case. Iterate locally until your script is ready, then configure and deploy it to Snowflake.

Deployment¶

Notebook Project Objects run on Snowflake using the warehouse or compute pool you configure. The runtime and dependencies are defined in code_bundle.yml at the root of your project.

1. Upload your project files to Snowflake and create a Notebook Project Object called my_snowpark_job.

snow notebook project create my_snowpark_job \
  --source . \
  --overwrite \
  --connection snowpark

2. Now that our project is deployed, we can execute main.py on Snowflake.

snow notebook project execute my_snowpark_job \
  --main-file=src/main.py \
  -c snowpark

Behind the scenes this command is running the following SQL statement. If you go to the Query History page in Snowsight, you’ll be able to see this statement there.

EXECUTE NOTEBOOK PROJECT your_db.your_schema.my_snowpark_job
MAIN_FILE = 'src/main.py';

3. You can run any script in the project by changing the --main-file parameter:

snow notebook project execute -n 'my_project' \
  --main-file 'src/args_example.py' -c snowpark

snow notebook project execute -n 'my_project' \
  --main-file 'src/udf_example.py' -c snowpark

Schedule Your Project¶

To run your project on a recurring schedule, wrap the run command in a Snowflake Task:

snow sql -q "
  CREATE OR REPLACE TASK my_task
    WAREHOUSE = my_warehouse
    SCHEDULE = 'USING CRON 0 0 * * * UTC'
  AS
    EXECUTE NOTEBOOK PROJECT my_db.my_schema.my_project
      MAIN_FILE = 'src/main.py';
  " \
  -c snowpark

After creating the task, resume it to start the schedule:

ALTER TASK my_task RESUME;

Automate deployment with CI/CD¶

You can run the deploy commands in a CI/CD pipeline to automatically deploy your project when you merge changes. The following example shows a GitHub Actions workflow that deploys on push to the main branch:

name: Deploy to Snowflake
on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: snowflakedb/snowflake-actions@v3
        with:
          cli-version: "latest"
          default-config-file-path: "config.toml"

      - run: |
          snow notebook project create my_project \
            --source . \
            --overwrite

Adapt this pattern to your CI system by running snow notebook project create --source . --overwrite after authenticating the Snowflake CLI.

Register UDFs from your project¶

If your script depends on User Defined Functions and you want to manage their lifecycle through the project, create them during the script’s run with session.udf.register or the @udf decorator. See src/udf_example.py in the sample project for an example.

Reference¶

The code_bundle.yml config¶

Before you deploy, configure runtime and dependency settings in the code_bundle.yml file at the root of your project. This file sets default values so you don’t have to specify them every time you run the project.

Here are two examples: one that runs on a warehouse and one that runs on a compute pool.

# Run on a warehouse
bundle:
  type: python
  runs_on:
    warehouse: my_db.my_schema.my_wh
    python_version: "3.12"
  properties:
    python_packages: requirements.txt
  secrets:
    - db.schema.my_secret
  external_access_integrations:
    - db.schema.my_eai
  env_vars:
    - MY_VAR: '101'

# Run on a compute pool
bundle:
  type: python
  runs_on:
    compute_pool: my_db.my_schema.my_pool
    query_warehouse: my_db.my_schema.my_wh
    python_version: "3.12"
    runtime_version: "2.4"
    accelerator: cpu
  properties:
    python_packages: requirements.txt
  secrets:
    - db.schema.my_secret
  external_access_integrations:
    - db.schema.my_eai
  env_vars:
    - MY_VAR: '101'

bundle¶

The top-level key for all configuration.

bundle.type¶

Required. The language runtime for the project. Supported value: python.

bundle.runs_on¶

Required. Defines the compute platform and runtime environment. Set either warehouse or compute_pool, but not both.

For warehouse execution:

For compute pool execution:

bundle.properties¶

Optional type-specific settings.

bundle.artifact_repositories¶

Optional list of artifact repositories to use for installing dependencies. If not specified, dependencies are installed from the built-in snowflake.snowpark.pypi_shared_repository repository. For warehouse execution, specify exactly one repository. For compute pool execution, you can specify multiple.

artifact_repositories:
  - "my_db.my_schema.my_private_repo"

bundle.stage_mounts¶

Optional list of stages to mount into the runtime environment.

stage_mounts:
  - source_stage_url: "@my_stage/data"
    mount_path: "/tmp/data"

bundle.secrets¶

Optional list of Snowflake secret objects to make available to the runtime. Specify as fully qualified names.

secrets:
  - "my_db.my_schema.my_secret"

bundle.external_access_integrations¶

Optional list of external access integrations that allow the project to reach external network endpoints.

external_access_integrations:
  - "my_db.my_schema.my_eai"

bundle.env_vars¶

Optional list of environment variables to set in the runtime, specified as key-value pairs.

env_vars:
  - MY_VAR: "my_value"
  - ANOTHER_VAR: "another_value"

SQL commands¶

CREATE NOTEBOOK PROJECT¶

Create a project from a stage:

CREATE NOTEBOOK PROJECT [ IF NOT EXISTS ] <database_name>.<schema_name>.<project_name>
  FROM '@<stage_path>'
  [ COMMENT = '<string_literal>' ];

Create a project from a workspace:

CREATE NOTEBOOK PROJECT <database_name>.<schema_name>.<project_name>
  FROM 'snow://workspace/<workspace_path>'
  [ COMMENT = '<string_literal>' ];

For full syntax details, see CREATE NOTEBOOK PROJECT.

EXECUTE NOTEBOOK PROJECT¶

EXECUTE NOTEBOOK PROJECT <database_name>.<schema_name>.<project_name>
  MAIN_FILE = 'main.py'
  [ COMPUTE_POOL = '<compute_pool_name>' ]
  [ QUERY_WAREHOUSE = '<warehouse_name>' ]
  [ RUNTIME = '<runtime_version>' ]
  [ EXTERNAL_ACCESS_INTEGRATIONS = ( <integration_name> [ , ... ] ) ];

For full syntax details, see EXECUTE NOTEBOOK PROJECT.

Snowflake CLI commands¶

snow notebook project create¶

Usage: snow notebook project create [OPTIONS] [NAME]

Creates a notebook project in Snowflake.

╭─ Arguments ───────────────────────────────────────────────────────────────────────────────────────────────╮
│   name      [NAME]  Name of the Snowpark project.                                                         │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ─────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --source                  TEXT  Source location of the notebook project. Supports stage path (starting    │
│                                 with '@') or workspace path (starting with 'snow://workspace/').          │
│ --comment                 TEXT  Comment for the notebook project.                                         │
│ --overwrite                     Overwrite the notebook project if it already exists.                      │
│ --skip-if-exists                Skip the creation of the notebook project if it already exists.           │
│ --help            -h            Show this message and exit.                                               │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯

snow notebook project execute¶

Usage: snow notebook project execute [OPTIONS] [NAME] ARGUMENTS

Executes a notebook project in Snowflake.

╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────────────────╮
│   name           [NAME]  Name of the notebook project.                                                       │
│   arguments      TEXT    Arguments to pass to the notebook project.                                          │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --main-file                             TEXT  Main file of the notebook project.                             │
│ --compute-pool                          TEXT  Compute pool to run the notebook project on.                   │
│ --query-warehouse                       TEXT  Query warehouse to run the notebook project on.                │
│ --runtime                               TEXT  Runtime to run the notebook project on.                        │
│ --requirements-file                     TEXT  Requirements file to use for the notebook project.             │
│ --external-access-integrations          TEXT  External access integrations to use for the notebook project.  │
│ --help                          -h            Show this message and exit.                                    │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

snow notebook project drop¶

Usage: snow notebook project drop [OPTIONS] [NAME]

Drops a notebook project in Snowflake.

╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────────────────╮
│   name      [NAME]  Name of the notebook project.                                                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --help  -h        Show this message and exit.                                                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

snow notebook project list¶

Usage: snow notebook project list [OPTIONS]

Lists notebook projects in Snowflake.

╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --help  -h        Show this message and exit.                                                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Migrate from snow_app.yml to code_bundle.yml¶

If you have an existing project that uses snow_app.yml, rename the file to code_bundle.yml and update its contents using the mapping below.

What changed:

• The config file is now named code_bundle.yml.
• All settings are nested under a top-level bundle key.
• runtime.version is split into two separate fields: python_version (for example, 3.12) and runtime_version (the Snowflake Container Runtime version, for example 2.4).
• runtime.compute_pool and runtime.query_warehouse move to bundle.runs_on.
• dependencies (a path to a file) is now bundle.properties.python_packages.
• artifact_repository (a single string) is now bundle.artifact_repositories (a list).

Before (snow_app.yml) and after (code_bundle.yml) — warehouse execution:

Before:

runtime:
  version: python_3.12

dependencies: requirements.txt

After:

bundle:
  type: python
  runs_on:
    warehouse: my_db.my_schema.my_wh
    python_version: "3.12"
  properties:
    python_packages: requirements.txt

Limitations¶

The following limitations apply:

• .ipynb notebooks can only run on compute pools, not warehouses. .py files can run on either.
• Java and Scala are not supported.
• Event table logging is not available when running on warehouses.