Deploying on Snowpark Container Services

You can run the AIM DMV Orchestrator and Workers as Snowpark Container Services (SPCS) in your Snowflake account. This page covers prerequisites, service setup, and the network access that Workers need to download database drivers and connect to source systems.

For shared architecture and deployment options, see Data Migration & Validation overview.

When to use SPCS

Deploy on SPCS when you want Orchestrator and Worker compute inside Snowflake rather than on customer-hosted machines. You can run both components on SPCS, both outside Snowflake, or mix the two (for example Orchestrator on SPCS and Workers in your data center).

Prerequisites you create manually

AIM DMV does not create these objects for you. Prepare them before starting services:

  1. Compute pool — hosts the SPCS services.
CREATE COMPUTE POOL <compute_pool_name>
  MIN_NODES = 1
  MAX_NODES = <max_nodes>
  INSTANCE_FAMILY = CPU_X64_S;

  1. Image repository — stores Orchestrator and Worker container images. Your role needs WRITE on the repository.

  2. Container images — push Orchestrator and Worker images to the repository. The Snowflake AIM Agent for Data Warehouses or SnowConvert AI CLI can help with image preparation and upload.

  3. Warehouse — a warehouse for the service specification’s QUERY_WAREHOUSE.

The Orchestrator service connects only to Snowflake. The Worker service additionally needs egress to your source system and, for some platforms, to package hosts where database drivers are downloaded at container startup.

Orchestrator vs Worker services

ServiceConnects to source?External Access Integration
OrchestratorNo (Snowflake only)Not required
WorkerYesRequired when Workers reach a source database or download drivers at runtime

Run both as Services (not Jobs), monitor usage, and suspend when idle.

The Orchestrator creates the SNOWCONVERT_AI database and schema objects on startup if they do not already exist.

Network access for Workers

Workers in SPCS run inside Snowflake’s network. To connect to a source database and download ODBC or related drivers at container startup, you must allow outbound access through a network rule and external access integration, then attach that integration to the Worker service.

Setup flow

  1. Create a secret with source database credentials (if the Worker reads them from Snowflake).
  2. Create a network rule listing every host and port the Worker must reach.
  3. Create an external access integration that references the network rule.
  4. Create or alter the Worker service with EXTERNAL_ACCESS_INTEGRATIONS = (<integration_name>).

Example templates (replace placeholders):

CREATE OR REPLACE SECRET SNOWCONVERT_AI.PUBLIC.<source_type>_SECRET
  TYPE = PASSWORD
  USERNAME = '<username>'
  PASSWORD = '<password>';

CREATE OR REPLACE NETWORK RULE SNOWCONVERT_AI.PUBLIC.<source_type>_EGRESS_RULE
  MODE = EGRESS
  TYPE = HOST_PORT
  VALUE_LIST = (
    '<source_host>:<source_port>',
    -- driver download hosts (see per-platform lists below)
  );

CREATE OR REPLACE EXTERNAL ACCESS INTEGRATION <source_type>_EAI
  ALLOWED_NETWORK_RULES = (SNOWCONVERT_AI.PUBLIC.<source_type>_EGRESS_RULE)
  ENABLED = TRUE;

When you use SnowConvert AI CLI (scai data worker setup) with a SQL Server source connection, the CLI can create the secret, network rule, and external access integration automatically. For other platforms, or when you use --skip-egress, create these objects manually using the host lists below.

Note

Account-level network policies, source-system firewalls, and corporate proxies are your responsibility. Verify egress from SPCS to every host in your network rule before starting a workflow.

Driver download hosts by platform

Workers download some database drivers when the container starts. Include your source host and port in every network rule, plus the driver download hosts for your platform.

Note

Only the SQL Server host list below is hardcoded in product setup code. Redshift and PostgreSQL lists are derived from the Worker container’s runtime install steps and can change when driver or package versions change. Confirm hosts before locking down egress.

SQL Server and Azure Synapse

Authoritative allowlist (plus your source host):

  • packages.microsoft.com:443
  • pmc-geofence.trafficmanager.net:443
  • deb.debian.org:80
  • deb.debian.org:443

The Worker installs the Microsoft ODBC Driver for SQL Server from Microsoft’s apt repository at startup.

Amazon Redshift

Recommended allowlist (plus your Redshift endpoint):

  • s3.amazonaws.com:443 (Redshift ODBC driver .deb download)
  • deb.debian.org:80
  • deb.debian.org:443

PostgreSQL

Recommended allowlist (plus your PostgreSQL endpoint):

  • deb.debian.org:80
  • deb.debian.org:443

The Worker installs odbc-postgresql from Debian package mirrors at startup.

Oracle

No driver download hosts. The Worker uses the pre-installed thin oracledb driver. Allow only your Oracle listener host and port.

Teradata

No driver download hosts. The Worker uses the pre-installed teradatasql driver. Allow only your Teradata host and port.

Related content