Serverless Tasks Flex¶

With the Serverless Tasks Flex, you can save money by allowing Snowflake to identify and run regularly scheduled serverless tasks within a window of time rather than a set time, optimizing for performance and costs. For example, you can run a maintenance task every night at a time when the server workload is predicted to be the lowest. For more information about pricing, see Task Costs.

This feature is recommended for the following:

  • Batch extract, transform, and load (ETL) use cases.

  • Maintenance tasks.

  • Short tasks that should be performed regularly, but not necessarily at a specific time.

When setting up the task, define the execution window (that is, the window of time when the task will run) using SCHEDULE and TARGET_COMPLETION_INTERVAL.

Diagram of a flexible task, where the task runs approximately every 6 hours (SCHEDULE), within a 2 hour window of time (TARGET_COMPLETION_INTERVAL).

The minimum target completion interval is 60 minutes. The target completion interval should be at least twice as long as the expected task completion time. This extra time gives Snowflake the flexibility to find run times that avoid peak pricing.

Each time a flexible task is completed, Snowflake evaluates the performance and compares it with previous runs. In the following runs, it estimates an optimal start time to achieve performance while balancing costs.

Considerations¶

  • Flexible tasks must be serverless.

  • Snowflake attempts to schedule tasks to complete within the target completion interval, but tasks aren’t guaranteed to complete within this time.

  • Flexible tasks support replication. Flex tasks will not be replicated to a secondary database if the secondary database does not have the feature enabled.

  • Flexible tasks support task graphs.

    • Tasks Graphs can be defined with a mix of flexible and non-flexible tasks.

    • The target completion interval is specified on each child flexible task. Therefore, the target completion interval of the entire task graph is the sum of the target completion intervals of each child flexible task.

  • Flexible tasks use a pool of XSMALL to XXLARGE size warehouses. More complex tasks might require a larger compute size. For more information, see Choosing a warehouse size.

  • You can’t manually start a flexible task with EXECUTE_TASK.

  • Flexible tasks can’t be triggered tasks.

New parameter: SCHEDULING MODE¶

The CREATE TASK and ALTER TASK commands now support a new parameter, SCHEDULING_MODE, which specifies whether a task starts at the scheduled time, or during a flexible execution window. The SCHEDULING_MODE parameter is optional. If you don’t specify a SCHEDULING_MODE, the task defaults to a standard serverless task that starts at the scheduled time.

SCHEDULING_MODE  = '{ FLEXIBLE | FIXED }'

Specifies whether a task starts at the scheduled time, or during a flexible execution window.

  • FLEXIBLE: The next scheduled run of the root task starts sometime within a flexible execution window, optimizing for costs. For more information, see TARGET_COMPLETION_INTERVAL.

  • FIXED (or NULL): The next scheduled run of a root task runs at the scheduled start time.

Default: NULL

Create a flexible task¶

Use CREATE TASK to create a serverless task.

  • Set SCHEDULING_MODE = ‘FLEXIBLE’.

  • Do not specify a WAREHOUSE.

  • Set a SCHEDULE of at least ‘60 MINUTES’.

  • Set TARGET_COMPLETION_INTERVAL of at least ‘60 MINUTES’.

Examples¶

Create a task that runs within windows of time that start every six hours and are each two hours long. The first window of time starts after the task is created (or resumed).

CREATE TASK T1
  SCHEDULING_MODE = 'FLEXIBLE'
  SCHEDULE='6 HOURS'
  TARGET_COMPLETION_INTERVAL='2 HOURS'
  AS SELECT 1;
Copy

Create a task that runs at some time each night, between 9PM - 5AM in the Europe/Madrid timezone.

CREATE TASK T1
  SCHEDULING_MODE = 'FLEXIBLE'
  SCHEDULE='USING CRON 0 2100 * * * Europe/Madrid'
  TARGET_COMPLETION_INTERVAL='8 HOURS'
  AS SELECT 1;
Copy

Convert an existing task to a flexible task¶

Use ALTER TASK to convert the task:

  1. Suspend the task.

  2. Alter the task:

    • Set the schedule mode to flexible.

    • Set the schedule interval to 60 minutes or more.

    • Set the target completion interval to at least 60 minutes.

    • If a warehouse is specified, remove it.

  3. Resume the task.

Examples¶

This example converts a task named ‘t1’ that ran on a user-managed warehouse to a serverless flexible task. The existing schedule is maintained (for example, every four hours). A new target completion interval is added, and the warehouse is removed. The first task execution window begins when the task is resumed.

ALTER TASK t1 SUSPEND;
ALTER TASK t1 SET
  SCHEDULING_MODE = 'FLEXIBLE'
  TARGET_COMPLETION_INTERVAL = '60 MINUTES'
  UNSET WAREHOUSE;
ALTER TASK t1 RESUME;
Copy

This example converts a task named ‘t1’ from a standard serverless task to a flexible serverless task that runs every day at some time between 12AM - 2AM UTC.

ALTER TASK t1 SUSPEND;
ALTER TASK t1 SET
  SCHEDULE = 'USING CRON 0 0 * * * UTC'
  SCHEDULING_MODE = 'FLEXIBLE'
  TARGET_COMPLETION_INTERVAL = '2 H'
ALTER TASK t1 RESUME;
Copy

Monitor flexible tasks¶

You can view the flexible task history for your account using SQL or Snowsight the same way you can monitor non-flexible tasks. For more information about monitoring tasks, see Viewing tasks and task graphs in Snowsight and Viewing the task history for your account.

To find flexible tasks, use SHOW TASKS or DESCRIBE TASK, and filter for SCHEDULING_MODE and TARGET_COMPLETION_INTERVAL.

In task details:

  • The task definition shows the task SCHEDULING_MODE as 'FLEXIBLE' and displays the TARGET_COMPLETION_INTERVAL.

  • SCHEDULED_TIME and NEXT_SCHEDULED_TIME show the start time of the flexible task scheduled window, not the time the task is scheduled to start.

In task history:

  • Task history only shows flexible tasks that are executing and completed. Scheduled tasks are not shown.

  • To compare the target completion interval to the actual task run time, expand the Total Duration breakdown.

Manage failed or overrun flexible task runs¶

Failed tasks

Flexible tasks can fail similarly to non-flexible tasks. When a flexible task run fails, the run is marked as failed stated in the TASK_HISTORY table function in the Information Schema or the TASK_HISTORY view in Account Usage. By default, Snowflake automatically suspends tasks that fail 10 or more consecutive times.

To avoid paying for failed task run compute, set up notifications for task run errors. For more information about setting up notifications, see Enabling notifications for tasks.

Task timeout

When the task execution duration exceeds 50% of its defined target completion interval, the task execution is considered to have overrun and is timed out.

To view tasks that are suspended due to overrun, view the STATE column for FAILED tasks in the TASK_HISTORY table function output or the TASK_HISTORY view. You can also create alerts for overrun tasks using SHOW TASKS and periodically searching for tasks in the SUSPENDED_DUE_TO_ERRORS state. The SHOW TASKS command also displays the LAST_SUSPENDED_ON column to better help you investigate flexible task overruns.

To avoid overrun tasks, Snowflake recommends that you take one or more of the following actions:

  • Set a longer target completion interval.

  • Convert the task from a flexible serverless task to a standard serverless task (SCHEDULING_MODE = FIXED).

  • Convert the task from a flexible task to a user-managed task (set SCHEDULING_MODE = FIXED, and also select a WAREHOUSE with enough resources to run the task).

  • Specify a larger warehouse size in the serverless task controls for the task to run (SERVERLESS_TASK_MAX_STATEMENT_SIZE).

Language: English