Service Management & Scaling¶

Using the service helper function¶

Model serving include a built-in helper function that retrieves logs from the event table for running or suspended services:

-- Retrieve logs using the service helper function
SELECT * FROM TABLE(mydb.myschema.my_model_service!SPCS_GET_LOGS())
WHERE
timestamp > dateadd(hour, -1, current_timestamp())
AND instance_id = 0  -- choose all instances or one particular
AND container_name = 'model-inference';

Querying the event table directly¶

If you have an event table configured for your account, you can query it directly to retrieve service logs:

-- Find the event table for your account
SHOW PARAMETERS LIKE 'event_table' IN ACCOUNT;

-- Query the event table for model service logs
SELECT TIMESTAMP, RESOURCE_ATTRIBUTES, RECORD_ATTRIBUTES, VALUE
FROM <current_event_table_for_your_account>
WHERE timestamp > dateadd(hour, -1, current_timestamp())
    AND RESOURCE_ATTRIBUTES:"snow.service.name" = '<model_service_name>'
    AND RECORD_TYPE = 'LOG'
    AND RESOURCE_ATTRIBUTES:"snow.service.container.instance" = '0'  -- choose all instances or one particular
    AND RESOURCE_ATTRIBUTES:"snow.service.container.name" = 'model-inference'
ORDER BY timestamp DESC
LIMIT 10;

Using the system function (Running instances only)¶

For real-time debugging of active containers you can use the SYSTEM$GET_SERVICE_LOGS function:

-- Retrieve logs from a specific service instance
SELECT SYSTEM$GET_SERVICE_LOGS('model_service_name', '0', 'model-inference', 10);

Using the service helper function¶

Model serving services include a built-in helper function that retrieves metrics from the event table for running or suspended services:

-- Retrieve metrics using the service helper function
SELECT *
FROM TABLE(mydb.myschema.my_model_service!SPCS_GET_METRICS())
WHERE
timestamp > dateadd(hour, -1, current_timestamp())
AND instance_id = 0  -- choose all instances or one particular
AND container_name = 'model-inference';

Querying the event table directly¶

You can query the event table directly to retrieve and filter specific metrics:

-- Find the event table for your account
SHOW PARAMETERS LIKE 'event_table' IN ACCOUNT;

-- Query the event table for model service metrics
SELECT
    timestamp,
    RESOURCE_ATTRIBUTES:"snow.service.container.instance" as instance,
    RESOURCE_ATTRIBUTES:"snow.service.container.name" as container,
    RECORD:metric:"name" as metric,
    value
FROM my_event_table_db.my_event_table_schema.my_event_table
WHERE timestamp > DATEADD(hour, -1, CURRENT_TIMESTAMP())
    AND RESOURCE_ATTRIBUTES:"snow.service.name" = '<model_service_name>'
    AND RECORD_TYPE = 'METRIC'
    AND RESOURCE_ATTRIBUTES:"snow.service.container.instance" = '0'  -- choose all instances or one particular
    AND RESOURCE_ATTRIBUTES:"snow.service.container.name" = 'model-inference'
ORDER BY timestamp DESC
LIMIT 100;

Multi-Zone Compute Pools¶

This compute pool prioritizes service uptime over strict symmetry. The system attempts to maintain a healthy footprint across multiple zones, but it does not “hard block” if conditions aren’t perfect.

Node Distribution: Uses a Round Robin strategy to spread nodes across all available zones in a region. If there are insufficient capacity errors in one zone, nodes are provisioned in other zones with capacity, which may cause uneven distribution.

Pod Distribution: Leverages Kubernetes topology spread constraints with a ScheduleAnyway policy. The scheduler tries to spread pods, but will still schedule them even if an even distribution cannot be met (e.g., due to capacity issues).

Zonal Outage Behavior: If a zone fails, the system does not automatically failover nodes to healthy zones yet. You should over-provision capacity (N+1) to ensure the remaining zones can handle the traffic load during an outage. In an event of availability zone outage:

• Scheduling: The scheduler immediately stops placing new pods in the impacted zone.

• Traffic: Ingress automatically routes traffic to pods in the healthy zones.

• Autoscaling: If the control plane remains functional, autoscaling may trigger to place new pods in the healthy zones.

Recovery & Rebalancing: Once a zone recovers, the system does not automatically move pods back to the recovered zone. It will naturally rebalance eventually as node failure or service maintenance occurs.

Convert an Existing Pool¶

ALTER COMPUTE POOL my_pool SUSPEND;

ALTER COMPUTE POOL my_pool
  SET PLACEMENT_GROUP = 'DISTRIBUTED';

ALTER COMPUTE POOL my_pool RESUME;

Revert an Existing Pool¶

ALTER COMPUTE POOL my_pool SUSPEND;

ALTER COMPUTE POOL my_pool
  UNSET PLACEMENT_GROUP;

ALTER COMPUTE POOL my_pool RESUME;

Verification¶

To confirm your pool is correctly configured for HA, check the placement_group column:

DESCRIBE COMPUTE POOL my_service_pool;