Snowpark Container Services: Working with services¶

Create a long running service¶

Use CREATE SERVICE to create a long running service.

• Create a service using an inline specification. In most cases, during development, you might choose inline specification, as shown:

  CREATE SERVICE echo_service
     IN COMPUTE POOL tutorial_compute_pool
     FROM SPECIFICATION $$
     spec:
       containers:
       - name: echo
         image: /tutorial_db/data_schema/tutorial_repository/my_echo_service_image:tutorial
         readinessProbe:
           port: 8000
           path: /healthcheck
       endpoints:
       - name: echoendpoint
         port: 8000
         public: true
     $$;
  

  Copy

• Create a service using stage information. When you deploy the service in a production environment, it’s advisable to apply the separation of concerns design principle and upload the specification to a stage, provide stage information CREATE SERVICE command, as shown:

  CREATE SERVICE echo_service
    IN COMPUTE POOL tutorial_compute_pool
    FROM @tutorial_stage
    SPECIFICATION_FILE='echo_spec.yaml';
  

  Copy

Execute a job service¶

Use EXECUTE JOB SERVICE to create a job service. This command runs synchronously, returns response after all containers of the job service exit.

• Execute a job service using an inline specification:

  EXECUTE JOB SERVICE
     IN COMPUTE POOL tutorial_compute_pool
     NAME = example_job_service
     FROM SPECIFICATION $$
     spec:
       container:
       - name: main
         image: /tutorial_db/data_schema/tutorial_repository/my_job_image:latest
         env:
           SNOWFLAKE_WAREHOUSE: tutorial_warehouse
         args:
         - "--query=select current_time() as time,'hello'"
         - "--result_table=results"
     $$;
  

  Copy

• Execute a job service using stage information:

  EXECUTE JOB SERVICE
    IN COMPUTE POOL tutorial_compute_pool
    NAME = example_job_service
    FROM @tutorial_stage
    SPECIFICATION_FILE='my_job_spec.yaml';
  

  Copy

Create multiple service instances and configure autoscaling¶

By default, Snowflake runs one instance of the service in the specified compute pool. To manage heavy workloads, you can run multiple service instances by setting the MIN_INSTANCES and MAX_INSTANCES properties, which specify the minimum number of instances of the service to start with and the maximum instances Snowflake can scale to when needed.

Example

CREATE SERVICE echo_service
   IN COMPUTE POOL tutorial_compute_pool
   FROM @tutorial_stage
   SPECIFICATION_FILE='echo_spec.yaml'
   MIN_INSTANCES=2
   MAX_INSTANCES=4;

When multiple service instances are running, Snowflake automatically provides a load balancer to distribute the incoming requests.

To configure Snowflake to autoscale the number of service instances running, follow these steps:

1. Specify the memory and CPU requirements for your service instance in the service specification file. For more information, see the container.resources field.

   Example

   resources:
     requests:
      memory: <memory-reserved>
      cpu: <cpu-units>
   

   Copy

2. When running the CREATE SERVICE command, set the MIN_INSTANCES and MAX_INSTANCES parameters. You can also use ALTER SERVICE to change these values.

Snowflake starts by creating the minimum number of service instances on the specified compute pool and then automatically scales the number of instances as needed. Snowflake uses an 80% threshold (for both CPU and memory) to scale up or scale down the number of service instances.

At any given time, one or more service instances can be running on the specified compute pool. Snowflake continuously monitors resource utilization (memory or CPU) within the compute pool, aggregating the usage data from all currently running service instances.

When the aggregated resource usage (across all service instances) surpasses 80%, Snowflake deploys an additional service instance within the compute pool. If the aggregated resource usage falls below 80%, Snowflake scales down by removing a running service instance. Snowflake uses a five-minute stabilization window to prevent frequent scaling.

Note the following scaling behaviors:

• The scaling of service instances is constrained by the MIN_INSTANCES and MAX_INSTANCES parameters configured for the service.

• If scaling up is necessary and the compute pool nodes lack the necessary resource capacity to start up another service instance, compute pool autoscaling can be triggered. For more information, see Autoscaling of compute pool nodes.

• If you specify the MAX_INSTANCES and MIN_INSTANCES parameters when creating a service but don’t specify the memory and CPU requirements for your service instance in the service specification file, no autoscaling occurs; Snowflake starts with the number of instances specified by the MIN_INSTANCES property and will not autoscale.

Updating service code redeploying the service¶

You can modify your application code and configuration, and provide a new service specification in a ALTER SERVICE command to direct Snowflake to redeploy your running service using the new specification.

You first upload modified application code to your image repository and then call ALTER SERVICE providing the service specification either inline or specifying the path to a specification file in Snowflake stage. For example:

• Provide specification inline (partial specification is shown).

  ALTER SERVICE echo_service
  FROM SPECIFICATION $$
  spec:
    …
    …
  $$;
  

  Copy

• Provide Snowflake stage file path:

  ALTER SERVICE echo_service
  FROM @tutorial_stage
  SPECIFICATION_FILE='echo_spec.yaml';
  

  Copy

Upon receiving the request, Snowflake redeploys the service using the new code.

ALTER SERVICE always uses the most recent version of the image from your repository. For example, if you upload multiple versions of an image named “echo_service:latest” or “echo_service:sometag”, Snowflake will use the latest uploaded image version.

You can use the DESCRIBE SERVICE command to find the image version the service is running. If you are the service owner, DESCRIBE SERVICE output includes the service specification, where you get the image digest (SHA256 of the image). For example:

spec:
 containers:
 - name: "echo"
    image: "orgname-acctname.registry-dev.snowflakecomputing.com/tutorial_db/data_schema/tutorial_repository/my_echo_service_image:latest"
    sha256: "@sha256:8d912284f935ecf6c4753f42016777e09e3893eed61218b2960f782ef2b367af"
    env:
       SERVER_PORT: "8000"
       CHARACTER_NAME: "Bob"
    readinessProbe:
       port: 8000
       path: "/healthcheck"
 endpoints:
 - name: "echoendpoint"
    port: 8000
    public: true

Using SYSTEM$GET_SERVICE_LOGS¶

You provide the service name, instance ID, container name, and optionally the number of most recent log lines to retrieve. If only one service instance is running, the service instance ID is 0. For example, the following GET_SERVICE_LOGS command retrieves the trailing 10 lines from the log of a container named echo that belongs to instance 0 of a service named echo_service:

CALL SYSTEM$GET_SERVICE_LOGS('echo_service', '0', 'echo', 10);

If you don’t know service information (such as the instance ID or container name), you can first run GET_SERVICE_STATUS to get information about the service instances and containers running in each instance.

Example output:

+--------------------------------------------------------------------------+
| SYSTEM$GET_SERVICE_LOGS                                                  |
|--------------------------------------------------------------------------|
| 10.16.6.163 - - [11/Apr/2023 21:44:03] "GET /healthcheck HTTP/1.1" 200 - |
| 10.16.6.163 - - [11/Apr/2023 21:44:08] "GET /healthcheck HTTP/1.1" 200 - |
| 10.16.6.163 - - [11/Apr/2023 21:44:13] "GET /healthcheck HTTP/1.1" 200 - |
| 10.16.6.163 - - [11/Apr/2023 21:44:18] "GET /healthcheck HTTP/1.1" 200 - |
+--------------------------------------------------------------------------+
1 Row(s) produced. Time Elapsed: 0.878s

SYSTEM$GET_SERVICE_LOGS output has the following limitations:

• It simply merges standard output and standard error streams, which makes it impossible to distinguish between regular output and error messages.

• It reports the captured data for a specific container in a single service instance.

• It only reports logs for a running container. The function cannot fetch logs from a previous container that was restarted or from a container of a service that is stopped or deleted.

• The function returns up to 100 KB of data.

Using an event table¶

Snowflake can capture and record standard output and standard errors from your containers into the event table configured for your account. For more information, see Logging and Tracing Overview.

For example, the following SELECT query retrieves Snowflake service and job events recorded in the past hour:

SELECT TIMESTAMP, RESOURCE_ATTRIBUTES, RECORD_ATTRIBUTES, VALUE
FROM <current-event-table-for-your-account>
WHERE timestamp > dateadd(hour, -1, current_timestamp())
AND RESOURCE_ATTRIBUTES:"snow.executable.type" = 'SnowparkContainers'
ORDER BY timestamp DESC
LIMIT 10;

Snowflake recommends that you include a timestamp in the WHERE clause of event table queries, as shown in this example. This is particularly important because of the potential volume of data generated by various Snowflake components. By applying filters, you can retrieve a smaller subset of data, which improves query performance.

You control the level of logs (all, errors only, or none) that you want persisted in an event table by using the spec.logExporters field in the service specification file.

The events table includes the following columns, which provide useful information regarding the logs collected by Snowflake from your container:

• TIMESTAMP: Shows when Snowflake collected the log.

• RESOURCE_ATTRIBUTES: Each object in this column identifies which Snowflake service and container in the service generated the log. For example, it furnishes details such as the service name, container name, and compute pool name that were specified when the service was run.

  {
  "snow.containers.compute_pool.id": 549816068,
  "snow.containers.compute_pool.name": "TUTORIAL_COMPUTE_POOL",
  "snow.containers.container.name": "echo",
  "snow.containers.instance.name": "0",
  "snow.containers.restart.id": "cd0cf2",
  "snow.database.id": 549816076,
  "snow.database.name": "TUTORIAL_DB",
  "snow.executable.id": 980991015,
  "snow.executable.name": "ECHO_SERVICE",
  "snow.executable.type": "SnowparkContainers",
  "snow.schema.id": 549816076,
  "snow.schema.name": "DATA_SCHEMA"
  }
  

  Copy

• RECORD_ATTRIBUTES: For a Snowflake service, it identifies an error source (standard output or standard error).

  { "log.iostream": "stdout" }
  

  Copy

• VALUE: Standard output and standard error are broken into lines, and each line generates a record in the event table.

  "echo-service [2023-10-23 17:52:27,429] [DEBUG] Sending response: {'data': [[0, 'Joe said hello!']]}"
  

Service functions: Using a service from an SQL query¶

A service function is a user-defined function (UDF) you create using CREATE FUNCTION. However, instead of writing the UDF code directly, you associate the UDF with your service. For example, in Tutorial 1, you create a service named echo_service that exposes one endpoint (echoendoint) as defined in the service specification:

spec:
…
  endpoints:
  - name: echoendpoint
    port: 8080

echoendpoint is a user-friendly endpoint name that represents the corresponding port (8080). To communicate with this service endpoint, you create a service function by providing the SERVICE and ENDPOINT parameters as shown:

CREATE FUNCTION my_echo_udf (text varchar)
   RETURNS varchar
   SERVICE=echo_service
   ENDPOINT=echoendpoint
   AS '/echo';

The AS parameter provides the HTTP path to the service code. You get this path value from the service code (for example, see service.py in Tutorial 1).

@app.post("/echo")
def echo():
...

When you invoke the service function, Snowflake directs the request to the associated service endpoint and path.

When you run multiple instances of your service, you can create a service function by specifying the optional MAX_BATCH_ROWS parameter to limit the batch size, the maximum rows that Snowflake sends in a batch to the service. For example, suppose MAX_BATCH_ROWS is 10 and you call my_echo_udf service function with 100 input rows. Snowflake partitions the input rows into batches, with each batch having at most 10 rows, and sends a series of requests to the service with the batch of rows in the request body. Configuring batch size can help when processing takes nontrivial time, and distributing rows across all available servers can also help.

You can use ALTER FUNCTION to alter a service function. The following ALTER FUNCTION command changes the service endpoint to which it associates and the batch size:

ALTER FUNCTION my_echo_udf(VARCHAR)
   SET SERVICE=other_service
   ENDPOINT=otherendpoint
   MAX_BATCH_ROWS=100

"Alex", "2014-01-01 16:00:00"
"Steve", "2015-01-01 16:00:00"
…

SELECT service_func(col1, col2) FROM input_table;

{
   "data":[
      [
         0,
         "Alex",
         "2014-01-01 16:00:00"
      ],
      [
         1,
         "Steve",
         "2015-01-01 16:00:00"
      ],
      …
      [
         <row_index>,
         "<column1>",
         "<column2>"
      ],
   ]
}

{
   "data":[
      [0, "a"],
      [1, "b"],
      …
      [ row_index,  output_column1]
   ]
}

USE ROLE service_owner;
GRANT USAGE ON service service_db.my_schema.my_service TO ROLE func_owner;

USE ROLE func_owner;
CREATE OR REPLACE test_udf(v VARCHAR)
  RETURNS VARCHAR
  SERVICE=service_db.my_schema.my_service
  ENDPOINT=endpointname1
  AS '/run';

SELECT test_udf(col1) FROM some_table;

ALTER FUNCTION test_udf(VARCHAR) SET
  SERVICE = service_db.other_schema.other_service
  ENDPOINT=anotherendpoint;

GRANT USAGE ON FUNCTION test_udf(varchar) TO ROLE func_user;
USE ROLE func_user;
SELECT my_test_udf('abcd');

Ingress: Using a service from outside Snowflake¶

A service can expose one or more endpoints as public to allow users to use the service from the public web.

CREATE SECURITY INTEGRATION SNOWSERVICES_INGRESS_OAUTH
TYPE=oauth
OAUTH_CLIENT=snowservices_ingress
ENABLED=true;

Mark the endpoint as public in your service specification file:

spec
  ...
  endpoints
  - name: <endpoint name>
    port: <port number>
    public: true

For more information about service specifications, see Specification Reference.

Service-to-service communications¶

Services can communicate with each other using the DNS name that Snowflake automatically assigns to each service. For an example, see Tutorial 3.

The DNS name format is:

<service-name>.<schema-name>.<db-name>.snowflakecomputing.internal

Use SHOW SERVICES (or DESCRIBE SERVICE) to get the DNS name of a service. The preceding DNS name is a full name. Services created in the same schema can communicate using just the <service-name>. Services that are in the same database but different schemas must provide the schema name, such as <service-name>.<schema-name>.

Snowflake allows network communications between services created by the same role and blocks network communications between services created by different roles. If you want to prevent your services from communicating with each other (for reasons such as security), use different Snowflake roles to create those services.

DNS names have the following limitations:

• Your database, schema, or service names must be valid DNS labels. (See also https://www.ietf.org/rfc/rfc1035.html#section-2.3.1). Otherwise, creating a service will fail.

• Snowflake replaces an underscore (_) in the names (database, schema, and service name) by a dash (-) in the DNS name.

• After creating a service, do not change the database or the schema name, because Snowflake will not update the DNS name of the service.

• A DNS name is only for internal communications within Snowflake between services running in the same account. It is not accessible from the internet.