Snowpark Container Services: Working with services¶

Create a long running service¶

Use CREATE SERVICE to create a long running service.

• Create a service using an inline specification. In most cases, during development, you might choose inline specification, as shown:

  CREATE SERVICE echo_service
     IN COMPUTE POOL tutorial_compute_pool
     FROM SPECIFICATION $$
     spec:
       containers:
       - name: echo
         image: /tutorial_db/data_schema/tutorial_repository/my_echo_service_image:tutorial
         readinessProbe:
           port: 8000
           path: /healthcheck
       endpoints:
       - name: echoendpoint
         port: 8000
         public: true
     $$;
  

  Copy

• Create a service using stage information. When you deploy the service in a production environment, it’s advisable to apply the separation of concerns design principle and upload the specification to a stage, provide stage information CREATE SERVICE command, as shown:

  CREATE SERVICE echo_service
    IN COMPUTE POOL tutorial_compute_pool
    FROM @tutorial_stage
    SPECIFICATION_FILE='echo_spec.yaml';
  

  Copy

Execute a job service¶

Use EXECUTE JOB SERVICE to create a job service. This command runs synchronously, returns response after all containers of the job service exit.

• Execute a job service using an inline specification:

  EXECUTE JOB SERVICE
     IN COMPUTE POOL tutorial_compute_pool
     NAME = example_job_service
     FROM SPECIFICATION $$
     spec:
       container:
       - name: main
         image: /tutorial_db/data_schema/tutorial_repository/my_job_image:latest
         env:
           SNOWFLAKE_WAREHOUSE: tutorial_warehouse
         args:
         - "--query=select current_time() as time,'hello'"
         - "--result_table=results"
     $$;
  

  Copy

• Execute a job service using stage information:

  EXECUTE JOB SERVICE
    IN COMPUTE POOL tutorial_compute_pool
    NAME = example_job_service
    FROM @tutorial_stage
    SPECIFICATION_FILE='my_job_spec.yaml';
  

  Copy

Creating multiple service instances and enabling autoscaling¶

By default, Snowflake runs one instance of the service in the specified compute pool. To manage heavy workloads, you can run multiple service instances by setting the MIN_INSTANCES and MAX_INSTANCES properties, which specify the minimum number of instances of the service to start with and the maximum instances Snowflake can scale to when needed.

Example

CREATE SERVICE echo_service
   IN COMPUTE POOL tutorial_compute_pool
   FROM @tutorial_stage
   SPECIFICATION_FILE='echo_spec.yaml'
   MIN_INSTANCES=2
   MAX_INSTANCES=4;

When multiple service instances are running, Snowflake automatically provides a load balancer to distribute the incoming requests.

To configure Snowflake to autoscale the number of service instances running, follow these steps:

1. Specify the CPU and memory requirements for your service instance in the service specification file. For more information, see the container.resources field.

   Example

   resources:
     requests:
      memory: <memory-reserved>
      cpu: <cpu-units>
   

   Copy

2. When running the CREATE SERVICE command, set the MIN_INSTANCES and MAX_INSTANCES parameters. You can also use ALTER SERVICE to change these values. Autoscaling occurs when the specified MAX_INSTANCES is greater than MIN_INSTANCES.

Snowflake starts by creating the minimum number of service instances on the specified compute pool. Snowflake then scales up or scales down the number of service instances using an 80% threshold (for both CPU and memory). Snowflake continuously monitors resource utilization (memory or CPU) within the compute pool, aggregating the usage data from all currently running service instances.

When the aggregated resource usage (across all service instances) surpasses 80%, Snowflake deploys an additional service instance within the compute pool. If the aggregated resource usage falls below 80%, Snowflake scales down by removing a running service instance. Snowflake uses a five-minute stabilization window to prevent frequent scaling.

Note the following scaling behaviors:

• The scaling of service instances is constrained by the MIN_INSTANCES and MAX_INSTANCES parameters configured for the service.

• If scaling up is necessary and the compute pool nodes lack the necessary resource capacity to start up another service instance, compute pool autoscaling can be triggered. For more information, see Autoscaling of compute pool nodes.

• If you specify the MAX_INSTANCES and MIN_INSTANCES parameters when creating a service but don’t specify the CPU and memory requirements for your service instance in the service specification file, no autoscaling occurs; Snowflake starts with the number of instances specified by the MIN_INSTANCES parameter and does not autoscale.

Using specification templates¶

There are times you might want to create multiple services using the same specification but with different configurations. For example, you suppose that you define an environment variable in a service specification and you want to create multiple services using the same specification but different values for the environment variable.

Specification templates enable you to define variables for field values in the specification. When you create a service you provide values for these variables.

Using specification templates is a two-step process:

1. Create a specification using variables as values for various specification fields. Use the {{ variable_name }} syntax to specify these variables. For example, the following specification uses a variable named “tag_name” for the image tag name, so that you specify a different image tag for each service.

   spec:
     containers:
     - name: echo
       image: myorg-myacct.registry.snowflakecomputing.com/tutorial_db/data_schema/tutorial_repository/my_echo_service_image:{{ tag_name }}
       ...
     endpoints:
     …
   

   Copy

2. Create a service by providing the specification template in a CREATE SERVICE command. You use SPECIFICATION_TEMPLATE or SPECIFICATION_TEMPLATE_FILE to specify the template. Use the USING parameter to specify the value of the variable. For example, the following statement uses a specification template from a Snowflake stage. The USING parameter sets the tag_name variable to the value 'latest'.

   CREATE SERVICE echo_service
      IN COMPUTE POOL tutorial_compute_pool
      FROM @STAGE SPECIFICATION_TEMPLATE_FILE='echo.yaml'
      USING (tag_name=>'latest');
   

   Copy

USING( var_name=>var_value, [var_name=>var_value, ... ] );

Examples¶

These examples show creating services using specification templates. The CREATE SERVICE commands in these examples use inline specification.

CREATE SERVICE echo_service
   IN COMPUTE POOL tutorial_compute_pool
   MIN_INSTANCES=1
   MAX_INSTANCES=1
   FROM SPECIFICATION_TEMPLATE $$
      spec:
         containers:
         - name: echo
           image: {{ image_url }}
           env:
             SERVER_PORT: {{SERVER_PORT}}
             CHARACTER_NAME: Bob
           readinessProbe:
             port: {{SERVER_PORT}}
             path: /healthcheck
         endpoints:
         - name: echoendpoint
           port: {{SERVER_PORT}}
           public: true
         $$
      USING (image_url=>' "/tutorial_db/data_schema/tutorial_repository/my_echo_service_image:latest" ', SERVER_PORT=>8000 );

spec:
  containers:
  - name: echo
    image: /tutorial_db/data_schema/tutorial_repository/my_echo_service_image:latest
    env:
      SERVER_PORT: 8000
      CHARACTER_NAME: Bob
    readinessProbe:
      port: 8000
      path: /healthcheck
    endpoints:
    - name: echoendpoint
      port: 8000
      public: true

spec:
 containers:
 - name: echo
   image: /tutorial_db/data_schema/tutorial_repository/my_echo_service_image:latest
   env:
     SERVER_PORT: 8000
     CHARACTER_NAME: Bob
   …

CREATE SERVICE echo_service
  IN COMPUTE POOL tutorial_compute_pool
  MIN_INSTANCES=1
  MAX_INSTANCES=1
  FROM SPECIFICATION_TEMPLATE $$
     spec:
       containers:
       - name: echo
         image: /tutorial_db/data_schema/tutorial_repository/my_echo_service_image:latest
         env: {{env_values}}
         readinessProbe:
           port: {{SERVER_PORT}}    #this and next tell SF to connect to port 8000
           path: /healthcheck
       endpoints:
       - name: echoendpoint
         port: {{SERVER_PORT}}
         public: true
        $$
     USING (env_values=>'{"SERVER_PORT": 8000, "CHARACTER_NAME": "Bob"}' );

spec:
  container:
  - name: main
    image: /tutorial_db/data_schema/tutorial_repository/my_job_image:latest
    env:
      SNOWFLAKE_WAREHOUSE: tutorial_warehouse
    args:
    - "--query=select current_time() as time,'hello'"
    - "--result_table=results"

spec:
  container:
  - name: main
    image: /tutorial_db/data_schema/tutorial_repository/my_job_image:latest
    env:
      SNOWFLAKE_WAREHOUSE: tutorial_warehouse
    args: {{ARGS}}
  $$
  USING (ARGS=$$["--query=select current_time() as time,'hello'", "--result_table=results"]$$ );

Service termination¶

When you suspend a service (ALTER SERVICE … SUSPEND) or drop a service (DROP SERVICE), Snowflake terminates all the service instances. Similarly, when you upgrade service code (ALTER SERVICE … <fromSpecification>), Snowflake applies rolling upgrades by terminating and redeploying one service instance at a time.

When terminating a service instance, Snowflake first sends a SIGTERM signal to each service container. The container has the option to process the signal and shut down gracefully with a 30-second window. Otherwise, after the grace period, Snowflake terminates all the processes in the container.

Updating service code and redeploying the service¶

After a service is created, use the ALTER SERVICE … <fromSpecification> command to update service code and redeploy the service.

You first upload modified application code to your image repository and then call ALTER SERVICE, either providing the service specification inline or specifying the path to a specification file in the Snowflake stage. For example:

ALTER SERVICE echo_service
FROM SPECIFICATION $$
spec:
  …
  …
$$;

Upon receiving the request, Snowflake redeploys the service using the new code.

When you run the CREATE SERVICE … <from-Specification> command, Snowflake records the specific version of the provided image. Snowflake deploys that same image version in the following scenarios, even if the image in the repository has been updated:

• When a suspended service is resumed (using ALTER SERVICE … RESUME).

• When autoscaling adds more service instances.

• When service instances are restarted during cluster maintenance.

But if you call ALTER SERVICE … <fromSpecification>, that triggers Snowflake to use the latest version in the repository for that image.

If you are the service owner, the output of the DESCRIBE SERVICE command includes the service specification, which includes the image digest (the value of the sha256 field in the specification), as shown below:

spec:
containers:
- name: "echo"
    image: "/tutorial_db/data_schema/tutorial_repository/my_echo_service_image:latest"
    sha256: "@sha256:8d912284f935ecf6c4753f42016777e09e3893eed61218b2960f782ef2b367af"
    env:
      SERVER_PORT: "8000"
      CHARACTER_NAME: "Bob"
    readinessProbe:
      port: 8000
      path: "/healthcheck"
endpoints:
- name: "echoendpoint"
    port: 8000
    public: true

ALTER SERVICE can impact communications (see Using a service) with the service.

• If ALTER SERVICE … <fromSpecification> removes an endpoint or removes relevant permissions required to use an endpoint (see serviceRoles in Specification Reference), access to the service will fail. For more information, see Using a Service.

• While the upgrade is in progress, new connections might get routed to the new version. If the new service version is not backward compatible, it will disrupt any active service usage. For example, ongoing queries using a service function might fail.

Granting the USAGE privilege on all endpoints using the default service role¶

When you create a service (including job service), Snowflake also creates a default service role, named ALL_ENDPOINTS_USAGE. This role has USAGE privilege on all endpoints the service exposes. You can grant other roles this default service role using the GRANT SERVICE ROLE command:

GRANT SERVICE ROLE my_echo_service_image!ALL_ENDPOINTS_USAGE TO ROLE some_other_role;

Users who are using some_other_role have the USAGE privilege on all the service endpoints.

When you drop a service, Snowflake drops all the service roles (default service role and service roles defined in the service specification) associated with the service and voids all the service role grants.

Granting the USAGE privilege to specific endpoints using service roles defined in the specification¶

Use service roles to manage fine-grained access to service endpoints. You define the service roles, along with the list of endpoints they are granted USAGE privilege to, in the service specification.

Granting privilege on specific endpoints of a service is a two-step process:

1. Define a service role: Use a service specification to define a service role by providing a role name and a list of one or more endpoints for which you want to grant USAGE privilege. For example, in the following specification fragment, the top-level serviceRoles field defines two service roles, each with USAGE privilege on specific endpoints.

   spec:
   ...
   serviceRoles:                 # Optional list of service roles
   - name: <svc_role_name1>
     endpoints:                  # endpoints that role can access
     - <endpoint_name1>
     - <endpoint_name2>
   - name: <svc_role_name2>
     endpoints:
     - <endpoint_name3>
     - <endpoint_name4>
   

   Copy

2. Grant the service role to other roles. Using the GRANT SERVICE ROLE command, you grant the service role to other roles (account roles, application roles, or database roles). For example:

   GRANT SERVICE ROLE <service-name>!<svc_role_name1> TO ROLE <another-role>
   

   Copy

Service functions: Using a service from an SQL query¶

A service function is a user-defined function (UDF) you create using CREATE FUNCTION (Snowpark Container Services). However, instead of writing the UDF code directly, you associate the UDF with your service endpoint. Note that, you can associate a service function only with a service endpoint that supports the HTTP or HTTPS protocol.

For example, in Tutorial 1, you create a service named echo_service that exposes one endpoint (echoendoint) as defined in the service specification:

spec:
…
  endpoints:
  - name: echoendpoint
    port: 8080

echoendpoint is a user-friendly endpoint name that represents the corresponding port (8080). To communicate with this service endpoint, you create a service function by providing the SERVICE and ENDPOINT parameters as shown:

CREATE FUNCTION my_echo_udf (text varchar)
   RETURNS varchar
   SERVICE=echo_service
   ENDPOINT=echoendpoint
   AS '/echo';

The AS parameter provides the HTTP path to the service code. You get this path value from the service code. For example, the following code lines are from service.py in Tutorial 1.

@app.post("/echo")
def echo():
...

When you invoke the service function, Snowflake directs the request to the associated service endpoint and path.

ALTER FUNCTION my_echo_udf(VARCHAR)
   SET SERVICE=other_service
   ENDPOINT=otherendpoint
   MAX_BATCH_ROWS=100

"Alex", "2014-01-01 16:00:00"
"Steve", "2015-01-01 16:00:00"
…

SELECT service_func(col1, col2) FROM input_table;

{
   "data":[
      [
         0,
         "Alex",
         "2014-01-01 16:00:00"
      ],
      [
         1,
         "Steve",
         "2015-01-01 16:00:00"
      ],
      …
      [
         <row_index>,
         "<column1>",
         "<column2>"
      ],
   ]
}

{
   "data":[
      [0, "a"],
      [1, "b"],
      …
      [ row_index,  output_column1]
   ]
}

USE ROLE service_owner;
GRANT USAGE ON service service_db.my_schema.my_service TO ROLE func_owner;

USE ROLE func_owner;
CREATE OR REPLACE test_udf(v VARCHAR)
  RETURNS VARCHAR
  SERVICE=service_db.my_schema.my_service
  ENDPOINT=endpointname1
  AS '/run';

SELECT test_udf(col1) FROM some_table;

ALTER FUNCTION test_udf(VARCHAR) SET
  SERVICE = service_db.other_schema.other_service
  ENDPOINT=anotherendpoint;

GRANT USAGE ON FUNCTION test_udf(varchar) TO ROLE func_user;
USE ROLE func_user;
SELECT my_test_udf('abcd');

Ingress: Using a service from outside Snowflake¶

A service can expose one or more endpoints as public to allow users to use the service from the public web. In this case, Snowflake manages access control. Note that ingress is allowed only with HTTP or HTTPS endpoints.

Mark the endpoint as public in your service specification file:

spec
  ...
  endpoints
  - name: <endpoint name>
    port: <port number>
    public: true

Service-to-service communications¶

Services can communicate with each other using the DNS name that Snowflake automatically assigns to each service. For an example, see Tutorial 3. Note that if a service endpoint is created only to allow service-to-service communications, the TCP protocol should be used.

The DNS name format is:

<service-name>.<schema-name>.<db-name>.snowflakecomputing.internal

Use SHOW SERVICES (or DESCRIBE SERVICE) to get the DNS name of a service. The preceding DNS name is a full name. Services created in the same schema can communicate using just the <service-name>. Services that are in the same database but different schemas must provide the schema name, such as <service-name>.<schema-name>.

Snowflake allows network communications between services created by the same role and blocks network communications between services created by different roles. If you want to prevent your services from communicating with each other (for reasons such as security), use different Snowflake roles to create those services.

DNS names have the following limitations:

• Your database, schema, or service names must be valid DNS labels. (See also https://www.ietf.org/rfc/rfc1035.html#section-2.3.1). Otherwise, creating a service will fail.

• Snowflake replaces an underscore (_) in the names (database, schema, and service name) by a dash (-) in the DNS name.

• After creating a service, do not change the database or the schema name, because Snowflake will not update the DNS name of the service.

• A DNS name is only for internal communications within Snowflake between services running in the same account. It is not accessible from the internet.