Snowpark Container Services: Working with services¶

Create a long running service¶

Use CREATE SERVICE to create a long running service.

• In most cases, you create a service by specifying an inline specification, as shown below:

  CREATE SERVICE echo_service
     IN COMPUTE POOL tutorial_compute_pool
     FROM SPECIFICATION $$
     spec:
       containers:
       - name: echo
         image: /tutorial_db/data_schema/tutorial_repository/my_echo_service_image:tutorial
         readinessProbe:
           port: 8000
           path: /healthcheck
       endpoints:
       - name: echoendpoint
         port: 8000
         public: true
     $$;
  

  Copy

• Create a service by referencing a service specification stored on a Snowflake stage. When you deploy the service in a production environment, you can apply the separation of concerns design principle and upload the specification to a stage, providing stage information in the CREATE SERVICE command, as shown:

  CREATE SERVICE echo_service
    IN COMPUTE POOL tutorial_compute_pool
    FROM @tutorial_stage
    SPECIFICATION_FILE='echo_spec.yaml';
  

  Copy

Execute a job service¶

Use EXECUTE JOB SERVICE to create a job service. By default this command runs synchronously, and returns a response after all containers of the job service exit. You can optionally specify the ASYNC parameter to run the job service asynchronously.

• Execute a job service using an inline specification:

  EXECUTE JOB SERVICE
     IN COMPUTE POOL tutorial_compute_pool
     NAME = example_job_service
     FROM SPECIFICATION $$
     spec:
       container:
       - name: main
         image: /tutorial_db/data_schema/tutorial_repository/my_job_image:latest
         env:
           SNOWFLAKE_WAREHOUSE: tutorial_warehouse
         args:
         - "--query=select current_time() as time,'hello'"
         - "--result_table=results"
     $$;
  

  Copy

  You can optionally execute this job asynchronously using the ASYNC property.

  EXECUTE JOB SERVICE
     IN COMPUTE POOL tutorial_compute_pool
     NAME = example_job_service
     ASYNC = TRUE
     FROM SPECIFICATION $$
     ...
     $$;
  

  Copy

• Execute a job service using stage information:

  EXECUTE JOB SERVICE
    IN COMPUTE POOL tutorial_compute_pool
    NAME = example_job_service
    FROM @tutorial_stage
    SPECIFICATION_FILE='my_job_spec.yaml';
  

  Copy

Using specification templates¶

There are times you might want to create multiple services using the same specification but with different configurations. For example, you suppose that you define an environment variable in a service specification and you want to create multiple services using the same specification but different values for the environment variable.

Specification templates enable you to define variables for field values in the specification. When you create a service you provide values for these variables.

In a specification template, you specify variables as values for various specification fields. Use the {{ variable_name }} syntax to specify these variables. Then, in the CREATE SERVICE command, specify the USING parameter to set values for these variables.

For example, the inline specification template in the following CREATE SERVICE command uses a variable named tag_name for the image tag name. You can use this variable to specify a different image tag for each service. In this example, the USING parameter sets the tag_name variable to the value latest.

CREATE SERVICE echo_service
  IN COMPUTE POOL tutorial_compute_pool
  FROM SPECIFICATION $$
  spec:
    containers:
    - name: echo
      image: myorg-myacct.registry.snowflakecomputing.com/tutorial_db/data_schema/tutorial_repository/my_echo_service_image:{{ tag_name }}
        ...
    endpoints:
    - name: ...
      ...
  $$
  USING (tag_name=>'latest');

If you choose to save the specification template to a Snowflake stage in your account, you can point to the location of the template in the CREATE SERVICE command:

CREATE SERVICE echo_service
    IN COMPUTE POOL tutorial_compute_pool
    FROM @STAGE SPECIFICATION_TEMPLATE_FILE='echo.yaml'
    USING (tag_name=>'latest');

USING( var_name=>var_value, [var_name=>var_value, ... ] );

Examples¶

These examples show creating services using specification templates. The CREATE SERVICE commands in these examples use inline specification.

CREATE SERVICE echo_service
   IN COMPUTE POOL tutorial_compute_pool
   MIN_INSTANCES=1
   MAX_INSTANCES=1
   FROM SPECIFICATION_TEMPLATE $$
      spec:
         containers:
         - name: echo
           image: {{ image_url }}
           env:
             SERVER_PORT: {{SERVER_PORT}}
             CHARACTER_NAME: Bob
           readinessProbe:
             port: {{SERVER_PORT}}
             path: /healthcheck
         endpoints:
         - name: echoendpoint
           port: {{SERVER_PORT}}
           public: true
         $$
      USING (image_url=>' "/tutorial_db/data_schema/tutorial_repository/my_echo_service_image:latest" ', SERVER_PORT=>8000 );

spec:
  containers:
  - name: echo
    image: /tutorial_db/data_schema/tutorial_repository/my_echo_service_image:latest
    env:
      SERVER_PORT: 8000
      CHARACTER_NAME: Bob
    readinessProbe:
      port: 8000
      path: /healthcheck
    endpoints:
    - name: echoendpoint
      port: 8000
      public: true

spec:
 containers:
 - name: echo
   image: /tutorial_db/data_schema/tutorial_repository/my_echo_service_image:latest
   env:
     SERVER_PORT: 8000
     CHARACTER_NAME: Bob
   …

CREATE SERVICE echo_service
  IN COMPUTE POOL tutorial_compute_pool
  MIN_INSTANCES=1
  MAX_INSTANCES=1
  FROM SPECIFICATION_TEMPLATE $$
     spec:
       containers:
       - name: echo
         image: /tutorial_db/data_schema/tutorial_repository/my_echo_service_image:latest
         env: {{env_values}}
         readinessProbe:
           port: {{SERVER_PORT}}    #this and next tell SF to connect to port 8000
           path: /healthcheck
       endpoints:
       - name: echoendpoint
         port: {{SERVER_PORT}}
         public: true
        $$
     USING (env_values=>'{"SERVER_PORT": 8000, "CHARACTER_NAME": "Bob"}' );

spec:
  container:
  - name: main
    image: /tutorial_db/data_schema/tutorial_repository/my_job_image:latest
    env:
      SNOWFLAKE_WAREHOUSE: tutorial_warehouse
    args:
    - "--query=select current_time() as time,'hello'"
    - "--result_table=results"

spec:
  container:
  - name: main
    image: /tutorial_db/data_schema/tutorial_repository/my_job_image:latest
    env:
      SNOWFLAKE_WAREHOUSE: tutorial_warehouse
    args: {{ARGS}}
  $$
  USING (ARGS=$$["--query=select current_time() as time,'hello'", "--result_table=results"]$$ );

Enabling autoscaling¶

To configure Snowflake to autoscale the number of service instances running, follow these steps:

1. Specify the CPU requirements for your service instance in the service specification file. For more information, see the container.resources field.

   Example

   resources:
     requests:
      cpu: <cpu-units>
   

   Copy

2. When running the CREATE SERVICE command, set the MIN_INSTANCES and MAX_INSTANCES parameters. You can also use ALTER SERVICE to change these values. Autoscaling occurs when the specified MAX_INSTANCES is greater than MIN_INSTANCES.

Snowflake starts by creating the minimum number of service instances on the specified compute pool. Snowflake then scales up or scales down the number of service instances based on an 80% CPU usage threshold. Snowflake continuously monitors CPU utilization within the compute pool, aggregating the usage data from all currently running service instances.

When the aggregated CPU usage (across all service instances) surpasses 80%, Snowflake deploys an additional service instance within the compute pool. If the aggregated CPU usage falls below 80%, Snowflake scales down by removing a running service instance. Snowflake uses a five-minute stabilization window to prevent frequent scaling.

Note the following scaling behaviors:

• The scaling of service instances is constrained by the MIN_INSTANCES and MAX_INSTANCES parameters configured for the service.

• If scaling up is necessary and the compute pool nodes lack the necessary resource capacity to start up another service instance, compute pool autoscaling can be triggered. For more information, see Autoscaling of compute pool nodes.

• If you specify the MAX_INSTANCES and MIN_INSTANCES parameters when creating a service but don’t specify the CPU and memory requirements for your service instance in the service specification file, no autoscaling occurs; Snowflake starts with the number of instances specified by the MIN_INSTANCES parameter and does not autoscale.

Suspending a service¶

A long-running service consumes compute pool resources, incurring costs, but you can suspend the service when it’s not performing meaningful work. When no services or jobs are active on any compute pool node, Snowflake’s compute pool auto-suspend mechanism suspends the pool to reduce costs.

To suspend a service, you can either explicitly call ALTER SERVICE … SUSPEND to suspend a service or set the AUTO_SUSPEND_SECS property using CREATE SERVICE or ALTER SERVICE to define the idle duration after which Snowflake automatically suspends the service.

When the AUTO_SUSPEND_SECS property is set, Snowflake automatically suspends a service if it’s not already suspended and it’s idle for more than AUTO_SUSPEND_SECS seconds. A service is idle when both of the following are true:

• There is no query currently running that includes a service function invocation to that service.

• The service status is RUNNING.

When Snowflake suspends a service, it shuts down all service instances on the compute pool. If there are no other services running on the compute pool and if auto-suspend is configured for the compute pool, then Snowflake also suspends the compute pool nodes. You thus avoid having to pay for an inactive compute pool.

Also, note the following:

• Auto-suspension is not supported for job services.

• Auto-suspension is not supported on services with public endpoints because Snowflake currently only tracks service function traffic and not ingress traffic in deciding when a service is idle.

Service termination¶

When you suspend a service (ALTER SERVICE … SUSPEND) or drop a service (DROP SERVICE), Snowflake terminates all the service instances. Similarly, when you upgrade service code (ALTER SERVICE … <fromSpecification>), Snowflake applies rolling upgrades by terminating and redeploying one service instance at a time.

When terminating a service instance, Snowflake first sends a SIGTERM signal to each service container. The container has the option to process the signal and shut down gracefully with a 30-second window. Otherwise, after the grace period, Snowflake terminates all the processes in the container.

Updating service code and redeploying the service¶

After a service is created, use the ALTER SERVICE … <fromSpecification> command to update service code and redeploy the service.

You first upload the modified application code to your image repository. You then execute the ALTER SERVICE command, either providing the service specification inline or specifying the path to a specification file in the Snowflake stage. For example:

ALTER SERVICE echo_service
FROM SPECIFICATION $$
spec:
  …
  …
$$;

Upon receiving the request, Snowflake redeploys the service using the new code.

If you are the service owner, the output of the DESCRIBE SERVICE command includes the service specification, which includes the image digest (the value of the sha256 field in the specification), as shown below:

spec:
containers:
- name: "echo"
    image: "/tutorial_db/data_schema/tutorial_repository/my_echo_service_image:latest"
    sha256: "@sha256:8d912284f935ecf6c4753f42016777e09e3893eed61218b2960f782ef2b367af"
    env:
      SERVER_PORT: "8000"
      CHARACTER_NAME: "Bob"
    readinessProbe:
      port: 8000
      path: "/healthcheck"
endpoints:
- name: "echoendpoint"
    port: 8000
    public: true

ALTER SERVICE can impact communications (see Using a service) with the service.

• If ALTER SERVICE … <fromSpecification> removes an endpoint or removes relevant permissions required to use an endpoint (see serviceRoles in Specification Reference), access to the service will fail. For more information, see Using a Service.

• While the upgrade is in progress, new connections might get routed to the new version. If the new service version is not backward compatible, it will disrupt any active service usage. For example, ongoing queries using a service function might fail.

Privileges needed for the service to run (service owner role)¶

The role that creates a service is the service’s owner role. The service executes all SQL in the context of this role. For more information, see Using an OAuth token to execute SQL.

If a service requires a privilege (for example, to access or perform an operation on a database object), the owner role must be granted that privilege.

In service-to-service communication, the owner role determines which endpoints on the destination service are accessible (see service roles).

Privileges needed to perform operations on the service¶

If a role needs to perform an operation on a service (for example, suspend the service), that role must be granted the privilege to perform that operation.

Each SQL command reference provides access control requirements for the commands. For more information, see SHOW SERVICES, ALTER SERVICE, and DROP SERVICE.

Privileges needed to access the service endpoints (service roles)¶

A service can expose one or more endpoints that clients access by sending requests. The service’s owner role has full access to the service and its endpoints. To allow other roles to access the endpoints, you must grant them the appropriate privileges. Snowflake supports defining “service roles” in the service specification to manage access to the exposed endpoints.

Examples where a role needs a service role include:

• An owner role of a service function needs a service role that grants access to the endpoint the service function references. Otherwise, you cannot create the service function. If the owner role of the service function loses permission to the service role after creation of the service function, queries using that service function will fail with a permission error.

• In service-to-service communications, the owner role of the service needs the service role that grants access to another service’s endpoint to call that endpoint.

• A user making ingress requests from outside Snowflake to a public endpoint exposed by a service must be granted a role that is granted a service role to allow access to that endpoint.

To enable a role (say some_role) to access a service endpoint, you do the following:

1. Grant the USAGE privilege on the database and schema where the service is created. These privileges enable resolving the names of objects in the schema. In this case, the object is the service.

   For example, the following commands grant these USAGE privileges to a role (some_role).

   GRANT USAGE ON DATABASE my_db TO ROLE some_role;
   GRANT USAGE ON SCHEMA my_schema TO ROLE some_role;
   

   Copy

2. Grant the service role that has permission to access the endpoints (see GRANT SERVICE ROLE).

   You have these options:

   • Grant access to all endpoints that the service exposes using the all_endpoints_usage service role, a pre-defined service role that Snowflake creates for every service. A service role name uses this syntax: service-name!service-role.

     GRANT SERVICE ROLE my_service!all_endpoints_usage TO ROLE some_role;
     

     Copy

   • Grant access to specific endpoints that a service exposes. This requires you to define one or more service roles in the specification with permission to specific endpoints. Then grant these service roles to manage fine-grained endpoint access.

     In the following CREATE SERVICE command, the inline specification defines two endpoints (ep1 and ep2) and a service role (ep1_role) that is granted access to only the ep1 endpoint.

     USE DATABASE my_db;
     USE SCHEMA my_schema;
     
     
     CREATE SERVICE my_service
     IN COMPUTE POOL tutorial_pool
     FROM SPECIFICATION $$
     spec:
       containers:
       - name: echo
         image: /tutorial_db/data_schema/tutorial_repository/my_echo_service_image:latest
       endpoints:
       - name: ep1
         port: 8000
         public: true
       - name: ep2
         port: 8082
         public: true
     serviceRoles:
     - name: ep1_role
       endpoints:
       - ep1
     $$
     

     Copy

     Now, to grant some_role access to the ep1 endpoint, grant the ep1_role service role as shown:

     GRANT SERVICE ROLE my_service!ep1_role TO ROLE some_role;
     

     Copy

Service functions: Using a service from an SQL query¶

A service function is a user-defined function (UDF) you create using CREATE FUNCTION (Snowpark Container Services). However, instead of writing the UDF code directly, you associate the UDF with your service endpoint. Note that you can associate a service function only with a service endpoint that supports the HTTP protocol (see spec.endpoints field (optional)).

For example, in Tutorial 1, you create a service named echo_service that exposes one endpoint (echoendoint) as defined in the service specification:

spec:
…
  endpoints:
  - name: echoendpoint
    port: 8080

echoendpoint is a user-friendly endpoint name that represents the corresponding port (8080). To communicate with this service endpoint, you create a service function by providing the SERVICE and ENDPOINT parameters as shown:

CREATE FUNCTION my_echo_udf (text varchar)
   RETURNS varchar
   SERVICE=echo_service
   ENDPOINT=echoendpoint
   AS '/echo';

The AS parameter provides the HTTP path to the service code. You get this path value from the service code. For example, the following code lines are from service.py in Tutorial 1.

@app.post("/echo")
def echo():
...

You invoke the service function in a SELECT statement such as the following:

SELECT service_function_name(<parameter-list>);

Snowflake directs the request to the associated service endpoint and path.

"Alex", "2014-01-01 16:00:00"
"Steve", "2015-01-01 16:00:00"
…

SELECT service_func(col1, col2) FROM input_table;

{
   "data":[
      [
         0,
         "Alex",
         "2014-01-01 16:00:00"
      ],
      [
         1,
         "Steve",
         "2015-01-01 16:00:00"
      ],
      …
      [
         <row_index>,
         "<column1>",
         "<column2>"
      ],
   ]
}

{
   "data":[
      [0, "a"],
      [1, "b"],
      …
      [ row_index,  output_column1]
   ]
}

ALTER FUNCTION my_echo_udf(VARCHAR) SET
   MAX_BATCH_ROWS = 15
   MAX_BATCH_RETRIES = 5;

USE ROLE service_owner;
GRANT USAGE ON DATABASE service_db TO ROLE func_owner;
GRANT USAGE ON SCHEMA my_schema TO ROLE func_owner;
GRANT SERVICE ROLE ON service service_db.my_schema.my_service!all_endpoints_usage TO ROLE func_owner;
USE ROLE func_owner;
CREATE OR REPLACE test_udf(v VARCHAR)
  RETURNS VARCHAR
  SERVICE=service_db.my_schema.my_service
  ENDPOINT=endpointname1
  AS '/run';

SELECT test_udf(col1) FROM some_table;

ALTER FUNCTION test_udf(VARCHAR) SET
  SERVICE = service_db.other_schema.other_service
  ENDPOINT=anotherendpoint;

GRANT USAGE ON DATABASE service_db TO ROLE func_user;
GRANT USAGE ON SCHEMA my_schema TO ROLE func_user;
GRANT USAGE ON FUNCTION test_udf(varchar) TO ROLE func_user;
USE ROLE func_user;
SELECT my_test_udf('abcd');

Ingress: Using a service from outside Snowflake¶

You can declare one or more endpoints as public in the service specification to allow users to use the service from the public. Note that users must be Snowflake users in the same Snowflake account that created the service.

spec
  ...
  endpoints
  - name: <endpoint name>
    port: <port number>
    public: true

Note that ingress is allowed only with an HTTP endpoint (see spec.endpoints field (optional)).

Service-to-service communications¶

Service instances can communicate directly with each other over TCP (including HTTP). This is true both for instances that belong to the same service and for instances that belong to different services.

Instances can only receive communications (requests) on the endpoints declared in the service specification. The client (the service sending the request) must have the required roles and grants to connect to that endpoint (see Privileges needed to access the service endpoints (service roles)).

• By default, a service instance can connect to other instances of the same service on the declared endpoints. In broader terms, a service’s owner role has permissions to connect to endpoints of services with the same owner role.

• In order for a client service to connect to an endpoint of a service that has a different owner role, the owner role of the client service needs the service role that grants access to another service’s endpoint to call that endpoint. For more information, see Privileges needed to access the service endpoints (service roles).

• If you want to prevent your services from communicating with each other (for reasons such as security), use different Snowflake roles to create those services.

A service instance can be reached using either the service IP address or the service instance IP addresses.

• Requests using the service IP address are routed to a load-balancer that in turn routes requests to a randomly selected service instance.

• Requests using the service instance IP address are routed directly to the specific service instance. You must use the service instance IP when connecting to an endpoint defined using the portRange field (see spec.endpoints field (optional)).

Both IP addresses are discoverable using the DNS name that Snowflake automatically assigns to each service. Note that it’s not possible to use DNS to connect to a specific instance. For example, it doesn’t make sense to construct a URL using the service instance DNS name, because there is no way to use the service instance DNS name to reference a specific service instance.

The service instance IP addresses are shown in the output of the SHOW SERVICE INSTANCES IN SERVICE command when the 2025_01 behavior change bundle is enabled.

For a service-to-service communication example, see Tutorial 3.

Note that if a service endpoint is created only to allow service-to-service communications, the TCP protocol should be used (see spec.endpoints field (optional)).

<service-name>.<hash>.svc.spcs.internal

instances.<service-name>.<hash>.svc.spcs.internal