snowflake.core.task.dagv1¶
High level, client-side representation of DAGs.
This set of higher-level classes provides a more convenient way to create, deploy, and manage DAGs of Tasks than the lower-level Task APIs in snowflake.core.task.
Example: >>> def dosomething(session: Session) -> str: … df = session.table(“target”) … df.group_by(“a”).agg(sum_(“b”)).save_as_table(“agg_table”) … return “Success” >>> with DAG(“my_dag”, schedule=timedelta(days=1)) as dag: … # Create a task that runs some SQL. … dag_task1 = DAGTask( … “dagtask1”, … “MERGE INTO target USING source_stream WHEN MATCHED THEN UPDATE SET target.v = source_stream.v”) … # Create a task that runs a Python function. … dag_task2 = DAGTask( … StoredProcedureCall( … dosomething, stage_location=”@mystage”, … packages=[“snowflake-snowpark-python”] … ), … warehouse=”test_warehouse”) … ) >>> # Shift right and left operators can specify task relationships. >>> dag_task2 >> dag_task1 >>> schema = root.databases[“MYDB”].schemas[“MYSCHEMA”] >>> dag_op = DAGOperation(schema) >>> dag_op.deploy(dag)
Classes
|
A graph of tasks composed of a single root task and additional tasks, organized by their dependencies. |
|
Represents a child Task of a DAG. |
|
Contains the history of a DAG run in Snowflake. |
|
Represents a client that has the convenient APIs to manage child tasks of a DAG in the Snowflake database. |
|
An enumeration. |