Serverless Tasks Flex¶

With Serverless Tasks Flex you can cost optimize your batch ETL use cases. Serverless Tasks Flex extends serverless tasks by offering features focused on lowering costs on SQL workloads. For less time-critical serverless tasks, you can define a schedule with a target completion interval. Snowflake runs these tasks on a serverless compute pool of warehouses, with variable start times to optimize the cost of running tasks during the specified completion interval.

The following use cases are candidates for Serverless Tasks Flex:

Tasks scheduled to execute every four or more hours.
Tasks that are tolerant of variable start times and longer runtimes.
Tasks that run SQL workloads.

To execute user defined Python/JVM functions or stored procedures in Serverless Tasks Flex, please enroll in the Serverless Tasks Python/ JVM preview.

Create or alter flexible tasks¶

To create a flexible task, specify SCHEDULING_MODE = FLEXIBLE and the TARGET_COMPLETION_INTERVAL when using CREATE TASK. You can also convert existing serverless tasks to flexible tasks using ALTER TASK.

Create a flexible task¶

To create a flexible task, use the flexible scheduling mode and define the target completion interval to four hours or more. For more information about creating tasks, see CREATE TASK.

Syntax

CREATE [ OR REPLACE ] TASK [ IF NOT EXISTS ] <name>
  [ USER_TASK_MANAGED_INITIAL_WAREHOUSE_SIZE = <string> ]
  [ SCHEDULE = '{ <num> {MINUTES | MINUTE | M } | USING CRON <expr> <time_zone> }' ]
  [ SCHEDULING_MODE = 'FLEXIBLE']
  [ TARGET_COMPLETION_INTERVAL = '<num> {MINUTES | MINUTE | M }' ]
...

Copy

Parameters

SCHEDULING_MODE

Specifies the type of serverless task.

FLEXIBLE uses a flexible task compute pool to complete the task. When creating a flexible task, you must also configure the TARGET_COMPLETION_INTERVAL parameter.
FIXED uses the non-flexible serverless compute model for the task. If you don’t specify a scheduling mode, this creates a non-flexible task with the SCHEDULING_MODE set to NULL.

TARGET_COMPLETION_INTERVAL

Defines the amount of time that the task has to complete. The task starts after the scheduled time and attempts to complete within the target completion interval.

The target completion interval must be 60 minutes or more for flexible tasks.

Usage notes

Flexible tasks must be serverless. This command returns an error if you specify a WAREHOUSE.
Tasks will attempt to complete within TARGET_COMPLETION_INTERVAL, but aren’t guaranteed to complete within this time.

Examples

Creates a task that runs every six hours starting at the resume time, with a target completion of four hours.

CREATE TASK T1
  SCHEDULE='360 MIN'
  SCHEDULING_MODE = 'FLEXIBLE'
  TARGET_COMPLETION_INTERVAL='60 MINUTES'
  AS SELECT 1;

Copy

Creates a task that runs every day at midnight with a target completion time of four hours.

CREATE TASK T1
  SCHEDULE='USING CRON 0 0 * * *'
  SCHEDULING_MODE = 'FLEXIBLE'
  TARGET_COMPLETION_INTERVAL='60 MINUTES'
  AS SELECT 1;

Copy

Alter tasks¶

You can alter an existing task to convert it to a flexible task. Set the scheduling mode to flexible and define a target completion interval of greater than 60 minutes to convert your existing task to a flexible task.

To alter a task, suspend the task and then resume it after you finish modifications. For more information about altering tasks, see ALTER TASK.

Syntax

ALTER TASK [ IF EXISTS ] <name> SET
  [ SCHEDULING_MODE = 'FLEXIBLE' ]
  [ TARGET_COMPLETION_INTERVAL = '<num> {MINUTES | MINUTE | M }' ]

Copy

Parameters

SCHEDULING_MODE

Specifies the type of serverless task.

FLEXIBLE uses a flexible task compute pool to complete the task. You must define the target completion interval to use FLEXIBLE.
FIXED uses the non-flexible serverless compute model for the task.

TARGET_COMPLETION_INTERVAL

Defines the amount of time that the task has to complete. The task starts after the scheduled time and attempts to finish within the target completion interval.

The target completion interval must be more than four hours for flexible tasks.

Examples

Converts task t1 to a flexible task that runs every six hours starting at the resume time, with a target completion of four hours.

ALTER TASK t1 SET
  SCHEDULING_MODE = 'FLEXIBLE'
  TARGET_COMPLETION_INTERVAL = '60 MINUTES';

Copy

Converts task t1 to a flexible task that runs every day at midnight with a target completion time of four hours.

ALTER TASK t1 SET
  SCHEDULE='USING CRON 0 0 * * *'
  SCHEDULING_MODE = 'FLEXIBLE'
  TARGET_COMPLETION_INTERVAL = '60 MINUTES'

Copy

Manage failed or overrun flexible task runs¶

Failed tasks

Flexible tasks can fail similarly to non-flexible tasks. When a flexible task run fails, the run is marked as failed stated in the TASK_HISTORY table function in the information schema or the Vue TASK_HISTORY in account usage. By default, Snowflake automatically suspends tasks that fail 10 or more consecutive times.

To avoid paying for failed task run compute, set up notifications for task run errors. For more information about setting up notifications, see Activer les notifications pour les tâches.

Overrun tasks

When a flexible task run fails to finish before its defined target completion interval, it’s considered to have overrun. Snowflake suspends flexible tasks that overrun the target completion interval two out of four executions consecutively.

To view tasks that are suspended due to overrun, view the STATE column for FAILED tasks in the TASK_HISTORY table function or TASK_HISTORY view. You can also create alerts for overrun tasks using SHOW TASKS and periodically searching for tasks in the SUSPENDED_DUE_TO_ERRORS state. The SHOW TASKS command also displays the LAST_SUSPENDED_ON column to better help you investigate flexible task overruns.

To avoid overrun tasks, Snowflake recommends that you take the following actions:

Configuring a longer target completion interval.
Configuring the task to run as a serverless task by setting SCHEDULING_MODE to FIXED, or configuring it to run as a task by selecting a WAREHOUSE and setting SCHEDULING_MODE to FIXED.

Monitor Serverless Tasks Flex¶

You can monitor flexible tasks the same way you can monitor non-flexible tasks. For more information about monitoring tasks, see Affichage des tâches et des graphiques de tâches dans Snowsight and Affichage de l’historique des tâches de votre compte.

You can use the new TARGET_COMPLETION_INTERVAL and SCHEDULING_MODE with the following commands to filter for flexible tasks:

You can also use Snowsight to monitor and view your flexible tasks. For more information about using Snowsight for tasks, see Affichage des tâches et des graphiques de tâches dans Snowsight.

In task details the task definition shows the task scheduling mode as flexible and displays the target completion interval.
You can expand the total duration breakdown in task history to compare the target completion interval to the actual task run duration.

Considerations¶

The following are known limitations of Serverless Tasks Flex:

Flexible tasks use a pool of XSMALL to SMALL size warehouses. Consider the complexity of your tasks when using flexible tasks. More complex tasks might require a larger compute size. For more information, see Sélection d’une taille d’entrepôt.
Task graphs don’t support the use of flexible tasks.
Flexible tasks don’t support overlapping runs.
You can’t manually start a flexible task with EXECUTE_TASK.
Scheduled flexible tasks aren’t visible in TASK_HISTORY. TASK_HISTORY shows executing and completed flexible tasks.
SCHEDULED_TIME and NEXT_SCHEDULED_TIME show the start time of the flexible task scheduled window, not the time the task is scheduled to start.
Flexible tasks doesn’t support triggered tasks.