Task Graph Object (Preview)¶
Note
To request access to this Private Preview feature, contact your Snowflake account team.
A task graph is a directed acyclic graph (DAG) of tasks that run together
as a single orchestrated unit. The Task Graph Object introduces a first-class
TASK GRAPH database object that encapsulates those tasks, giving a pipeline
a single place for graph-wide settings and the ability to edit a running graph
without suspending it first.
Overview¶
A task graph lets you build multi-step pipelines (extract, transform, validate, publish) where each step runs after its predecessors finish, with branches running in parallel where the dependency structure allows.
If you have used tasks before, you already know most of the building blocks: a task runs SQL or a stored procedure, on a schedule or when triggered, on serverless compute or a user-managed warehouse. The Task Graph Object introduces an explicit graph object that encapsulates those tasks and provides:
- A single place for graph-wide default settings (schedule, overlap policy, retry behavior, and member defaults).
- The ability to edit a running graph without suspending it first. Changes
are staged until you
COMMITthem.
You can author and manage task graphs in two ways:
- Snowflake Python API (
snowflake.core.task.taskgraph): the primary, recommended authoring experience. See Python authoring. - SQL: the
TASK GRAPHcommands documented in SQL reference.
A migration tool is also provided that reads an existing root-task graph and generates Python scripts to convert it into a Task Graph Object.
The Task Graph Object¶
Historically, a task graph had no object of its own. The graph’s identity and all of its settings (schedule, overlap policy, retry behavior) lived on the root task, which was simultaneously a step in the pipeline and the configuration surface for the whole pipeline.
The TASK GRAPH object owns:
- Graph-level properties: the schedule, the root(s), the finalizer, the
overlap policy, notification integrations, and configuration (
CONFIG). - Member defaults (
TASK_DEFAULTS): default settings, such as the warehouse or log level, that apply to every member task unless the member overrides them. - Membership: the set of tasks reachable from the graph’s root(s), linked
by their
AFTER(predecessor) relationships.
Member tasks are still ordinary tasks created with CREATE TASK.
Note
During Private Preview, a task graph has exactly one root. Support for multiple roots will be added in Public Preview.
Atomic deployments¶
You no longer need to suspend your graph to make changes to it. You can
alter a running task graph (its properties, its member tasks, even its shape),
and your changes remain as staged changes until you COMMIT the graph,
at which point subsequent executions use the new version.
This works because a running graph executes against a committed version: a
frozen snapshot of the graph’s shape and settings captured when you run
COMMIT. Running executions always use the committed version they started
with, so staged edits never disturb a run in progress.
Core concepts¶
Graph properties, parameters, and member defaults¶
Task graphs have three kinds of settings.
Graph properties describe the pipeline as a whole and are set directly on the graph object:
SCHEDULEROOTSFINALIZEROVERLAP_POLICYERROR_INTEGRATIONSUCCESS_INTEGRATIONCONFIGWHEN
Graph parameters also apply to the graph as a whole. They follow the standard account → database → schema parameter inheritance hierarchy, but because they describe graph-level behavior they are not meaningful to set on individual member tasks:
TASK_AUTO_RETRY_ATTEMPTSSUSPEND_TASK_AFTER_NUM_FAILURES
Task defaults (TASK_DEFAULTS) supply default values that cascade to every
member task. A task can override any of these by setting the property on itself:
WAREHOUSE- Serverless compute sizing:
USER_TASK_MANAGED_INITIAL_WAREHOUSE_SIZE,SERVERLESS_TASK_MIN_STATEMENT_SIZE,SERVERLESS_TASK_MAX_STATEMENT_SIZE EXECUTE AS USERTARGET_COMPLETION_INTERVALCOMMENT- Session and task parameters, such as
USER_TASK_TIMEOUT_MSandLOG_LEVEL
Editing and committing task graphs¶
Task graphs separate changing a graph or its tasks from releasing those changes, so you can edit a running graph without disrupting it.
-
Change freely. When a graph is resumed, you can alter the graph object, alter its member tasks, and add or remove member tasks. These edits are staged: they do not affect the committed version, so in-flight and scheduled runs continue uninterrupted.
-
COMMIT to release changes.
COMMITfreezes your staged changes into a new committed version. The next scheduled run picks up the new version. -
REVERT to discard changes.
REVERTrestores the graph and its members to the most recently committed version, discarding all staged edits.
With task graphs you edit and then commit, so CI/CD processes no longer need to manage suspend/resume state.
Lifecycle commands at a glance¶
| Command | Effect |
|---|---|
ALTER TASK GRAPH <name> COMMIT | Freeze staged changes into a new committed version. Does not start the schedule. |
ALTER TASK GRAPH <name> RESUME | Start running the graph on its schedule, using the most recently committed version. Requires SCHEDULE or WHEN to be set. |
ALTER TASK GRAPH <name> SUSPEND | Stop scheduling new runs. |
ALTER TASK GRAPH <name> REVERT | Discard staged changes, restoring the graph and its members to the committed version. |
EXECUTE TASK GRAPH <name> | Trigger one immediate run of the committed version, independent of the schedule. |
A graph must have a committed version before it can run. A typical first
deployment is COMMIT followed by RESUME.
RESUME requires the graph to have a SCHEDULE or a WHEN condition
set. If you want a graph you only ever trigger manually with
EXECUTE TASK GRAPH, omit both: you can still COMMIT and run it on
demand. See ALTER TASK GRAPH ... RESUME in the
SQL reference for details.
You can’t drop a task that is in a committed version¶
Important
A task that belongs to a graph’s committed version can’t be dropped or
replaced (DROP TASK, or CREATE OR REPLACE TASK) while it’s still part
of that committed version.
To remove a member task:
- Sever it from the graph: remove the predecessor relationship that
connects it (for example with
ALTER TASK ... REMOVE AFTER) so it is no longer reachable from the root. - COMMIT that change.
- Once the task is no longer part of the committed version, you can drop it.
SQL reference¶
This section covers the task-graph–specific SQL commands. Member tasks are
created and altered with the standard CREATE TASK / ALTER TASK commands.
CREATE TASK GRAPH¶
Creates, replaces, or alters a task graph object over a set of member tasks.
Example:
CREATE OR ALTER TASK GRAPH¶
Creates the graph if it does not exist, or updates an existing graph to match
the statement. Follows the same syntax as CREATE TASK GRAPH. Does not resume
or suspend a graph. Use ALTER TASK GRAPH for lifecycle management.
ALTER TASK GRAPH … SET / UNSET¶
Changes graph-wide properties or member defaults. On a resumed graph, these
edits are staged until you COMMIT.
SET TASK_DEFAULTS performs a merge, not a replace. To remove a default,
use UNSET TASK_DEFAULTS ( <property> ). An unqualified UNSET TASK_DEFAULTS
is not supported.
ALTER TASK GRAPH … COMMIT¶
Validates the graph and freezes the current shape and settings into a new
committed version. The next run uses the new version. COMMIT does not start
the schedule.
COMMIT has no SCHEDULE or WHEN requirement; RESUME does. See
ALTER TASK GRAPH ... RESUME following.
ALTER TASK GRAPH … RESUME¶
Starts running the graph on its schedule using the most recently committed version. Resuming the graph implicitly enables every member task. The graph must have a committed version.
Important
A task graph must have a SCHEDULE or a WHEN condition set before it
can be resumed. RESUME fails on a graph that has neither. This
requirement applies regardless of whether you build the graph with the
Snowflake Python API or SQL.
If you want a graph that never runs on its own and only runs when you
trigger it manually with EXECUTE TASK GRAPH, omit both SCHEDULE and
WHEN: you can still COMMIT it and run it on demand. To avoid unintended
scheduling, don’t call RESUME.
ALTER TASK GRAPH … SUSPEND¶
Stops scheduling new runs of the graph.
ALTER TASK GRAPH … REVERT¶
Discards all staged (uncommitted) changes to the graph and its member tasks, restoring them to the most recently committed version. Member tasks that were newly added but never committed are disconnected from the graph rather than dropped.
EXECUTE TASK GRAPH¶
Triggers one immediate run of the committed version, regardless of schedule.
Useful for testing. Staged (uncommitted) changes aren’t included: if you’ve
edited the graph or its member tasks and want those edits in the run,
COMMIT first, then EXECUTE TASK GRAPH.
Use RETRY LAST to retry the most recent failed run, resuming from the tasks
that failed rather than re-running the entire graph.
DROP TASK GRAPH¶
Drops the graph object. Suspend the graph first. Dropping the graph object does not drop its member tasks.
Adding and removing member tasks¶
Membership is defined by AFTER relationships reachable from the root. Add a
member by creating (or linking) a task with an AFTER that connects it to the
graph; remove a member by severing that relationship. These are ordinary
CREATE TASK / ALTER TASK operations become staged changes on a
resumed graph and take effect on the next COMMIT.
Monitoring task graphs¶
SHOW TASK GRAPHS¶
Lists graph objects as first-class objects, surfacing graph-wide properties and
TASK_DEFAULTS.
SHOW COMMITTED TASKS / DESCRIBE COMMITTED TASK¶
These surfaces show the committed version (the task shape that is actually
running) rather than the live, possibly-staged definition shown by SHOW TASKS
and DESCRIBE TASK.
SHOW COMMITTED TASKS: lists the tasks in a graph’s committed version.DESCRIBE COMMITTED TASK: shows the committed definition of a single task.
Tip
SHOW TASKS reflects your latest (possibly staged) edits. SHOW COMMITTED TASKS
reflects what is deployed. If they differ, you have staged changes waiting for a
COMMIT.
DESCRIBE TASK GRAPH¶
Returns the full definition of a graph object, including graph-wide properties
and TASK_DEFAULTS.
GET_ DDL¶
Returns a runnable CREATE statement that round-trips the graph definition.
TASK_ GRAPH_ CHANGES¶
Note
TASK_GRAPH_CHANGES is releasing in Snowflake release 10.20 (ETA June 10,
2026). Calling the function before this release returns an
Insufficient privileges error.
The TASK_GRAPH_CHANGES table function shows the staged changes on a
graph: the difference between the live definition and the committed version.
Use it to see exactly what a COMMIT would deploy or what a REVERT would
discard.
Each row identifies an entity with uncommitted changes and shows its
committed-version definition alongside its current (staged) definition. The
function returns no rows immediately after a COMMIT or REVERT.
Existing task monitoring surfaces¶
The following surfaces work for task graphs as well as existing root-task graphs:
TASK_HISTORYCURRENT_TASK_GRAPHSCOMPLETE_TASK_GRAPHS
Most task-observability views and table functions now accept a graph-based
identifier (GRAPH_NAME or GRAPH_ID) in addition to the existing root-task
identifier, and return corresponding GRAPH_ID / GRAPH_NAME output columns.
- On input: provide either a graph-based identifier or a root-task identifier, not both.
- On output: for a task graph, the graph columns are non-null and the root-task columns are null; for a root-task graph, the reverse is true.
Python authoring (Snowflake Python API)¶
While task graphs can be authored in both SQL and Python, the Snowflake Python API is the primary, recommended way to author and manage task graphs. It simplifies the development of complex graphs and lets you express advanced orchestration logic without writing DDL by hand.
Task graphs use a dedicated module: snowflake.core.task.taskgraph. This is
distinct from the older snowflake.core.task.dagv1 DAG API; the taskgraph
module targets the task graph object and its committed-version deployment model.
For general Snowflake Python API installation and connection setup, see:
Installing the pre-release package¶
During Private Preview, the taskgraph module ships in a pre-release build of
snowflake-core. Install the latest pre-release before using the library:
Setup (local)¶
Connect to Snowflake and get a handle to the schema you want to work in:
Root accepts either a snowflake.connector connection (shown above) or a
Snowpark Session.
Setup (Snowflake workspace)¶
Using the task graph Python library in a Snowflake workspace requires installing the pre-release package each time the workspace service starts, because installed packages are not persisted across service restarts.
-
Create an external access integration to allow network calls to PyPI. See Set up external access for Snowflake Notebooks for full details. Sample configuration:
-
Open the workspace in Snowsight, open a Python file, and click Connect. Expand the service settings, switch to service runtime v2.6, apply the external access integration, then create the service.
-
When the service launches, open the terminal and run:
Important
This step must be repeated each time the workspace service restarts.
-
Obtain a
Rootinstance:
Migrating existing root-task graphs¶
A migration tool converts an existing root-task graph into a task graph object and generates the Python scripts to manage it going forward. If you have existing pipelines, this is the recommended starting point.
| Argument | Required | Notes |
|---|---|---|
<ROOT_TASK_FQN> | Yes | Fully qualified DB.SCHEMA.TASK of the root task. |
--local-output | Yes | Local directory for the generated files. |
--new-graph-name | No | Name for the generated task graph. Defaults to the root task name. |
--folder-name | No | Output subfolder under --local-output. Defaults to the new graph name. |
--connection | No | Named connection from ~/.snowflake/connections.toml. Defaults to your default connection. |
--no-sql-files | No | Inline each task’s SQL into the generated commit script instead of writing separate .sql files. |
The tool produces three output files:
- commit_<name>.py: the ongoing source of truth for the graph. Edit it to change the graph, then re-run to deploy. Re-running is idempotent.
- migrate_<name>.py: performs the one-time conversion: suspends the existing root task and deploys the task graph object. Delete it after success.
- task_definitions/: one
.sqlfile per task, referenced from the generated graph viasql_from_file(...).
Running the generated scripts:
Defining a task graph¶
Define a task graph with the @task_graph decorator on a function. Inside the
function, declare member tasks as TaskNode objects and connect them. Calling
the decorated function builds an in-memory graph;
schema.task_graphs.create(...) persists it.
The schedule can be a fixed interval (timedelta) or a calendar expression
(Cron):
A TaskNode accepts a name and a SQL body, plus optional per-task settings:
create() persists the graph but does not start it. Use the resource lifecycle
methods to deploy and run it.
Dependencies with flow operators¶
Use >> and << to connect tasks. They accept single tasks or iterables
(tuples, lists, sets) for fan-out and fan-in:
>> and << describe the same edges from either direction.
a >> (b, c) >> d and d << (b, c) << a are equivalent. Adding the same
edge more than once is harmless.
Important
Connecting two iterables directly, such as (a, b) >> (c, d), is not
supported and raises TypeError. Split a many-to-many fan-out into separate
statements instead:
Static branching with graph_ case¶
graph_case adds conditional branches to a graph. It comes in two forms:
- Predicate form:
graph_case()with no argument; eachwhen(...)takes a SQL boolean expression, and the first branch whose predicate is true runs. - Switch form:
graph_case("<expr>")with an expression; eachwhen(...)takes a value to compare against.
Both forms support otherwise(...) or end() (no fallback).
graph_case blocks can be nested. A task may belong to only one graph_case,
and a task’s own condition is ANDed with its branch predicate.
Defaults and warehouse configuration¶
Set graph-wide defaults with TaskDefaults on the @task_graph decorator.
Member tasks inherit these defaults unless they set their own value.
Compute is configured with a warehouse config object:
ServerlessWarehouseConfig(...): serverless compute with optionalinitial_warehouse_size,min_statement_size,max_statement_size, andtarget_completion_interval.UserManagedWarehouseConfig("<warehouse_name>"): a user-managed warehouse.
Finalizer tasks and sql_ from_ file¶
A finalizer task runs after every other task in the graph completes. It is
useful for cleanup or notifications. Declare it with FinalizerTask; it
attaches to the graph as its finalizer rather than connecting via flow
operators.
sql_from_file loads a task’s SQL body from a file (relative to your script):
Lifecycle: the task graph resource¶
schema.task_graphs.create(...) returns a resource representing the
persisted graph. You can also get a resource for an existing graph by name:
The resource exposes the full graph lifecycle:
Most methods also have an _async variant (for example, revert_async(),
drop_async()) that returns a PollingOperation.
To inspect a graph and its committed version:
fetch_committed_tasks() returns the shape that actually runs, which can
differ from your latest staged edits until you call commit().
Known limitations (Private Preview)¶
The following limitations apply during Private Preview and will be resolved before public preview or general availability.
Replacing the root task on a resumed task graph is not yet supported¶
Altering a resumed root task is fully supported. Altering or replacing any
child task is fully supported. However, replacing the root task of a resumed
task graph might produce the following error when you attempt to COMMIT:
Member tasks might show state = “suspended” even when running¶
A task that is a member of a running task graph might show state = "suspended"
in SHOW TASKS and DESCRIBE TASK even though the graph is resumed and the
task is being scheduled and executed normally.
Workaround: The graph’s state from SHOW TASK GRAPHS is accurate and
authoritative.
Snowsight Task Graph History page does not show task graph objects¶
The Task Run History page works for individual tasks and you can view details on the most recent graph run. However, manual executions and retries can’t be triggered from the Snowsight UI for task graph objects.
TASK_ GRAPH_ CHANGES is not yet available¶
Calling INFORMATION_SCHEMA.TASK_GRAPH_CHANGES currently returns an
Insufficient privileges error.
Replication is not supported¶
Replication for task graph objects is not supported during Private Preview. Member tasks belonging to a task graph object can be replicated but will be suspended and non-resumable.