Sync a Snowflake-managed table with Polaris Catalog¶
To query a Snowflake-managed Iceberg table using a third-party engine such as Apache Spark™, you can sync the table with Polaris Catalog.
This topic covers how to sync a Snowflake-managed Iceberg table with Polaris Catalog using a catalog integration in Snowflake and an external catalog in Polaris Catalog.
Step 1: Create an external volume¶
If you don’t have one already, start by creating an external volume in Snowflake that provides access to the cloud storage location where you want to store your table data and metadata.
Complete the instructions for your cloud storage service:
Step 2: Configure Polaris Catalog resources¶
Next, complete the steps in this section to create an external catalog and service connection in your Polaris Catalog account.
Follow the instructions in Create a catalog to create an external catalog in your Polaris Catalog account. Make sure that the following settings for the external catalog are configured:
The External toggle is enabled.
The Default base location matches the
STORAGE_BASE_URL
for the external volume you created in Step 1: Create an external volume.
Polaris Catalog syncs your Snowflake-managed tables to this external catalog.
If you don’t already have a service connection for Snowflake, follow the instructions in Configure a service connection to create a connection for the Snowflake engine in your Polaris Catalog account.
Configure a catalog role for your external catalog with privileges that allow access to your external catalog. For instructions, see Grant privileges to a catalog.
The catalog role must have the following privileges on the catalog:
TABLE_CREATE
TABLE_WRITE_PROPERTIES
TABLE_DROP
NAMESPACE_CREATE
NAMESPACE_DROP
You can either grant each of these privileges to the catalog role, or grant the CATALOG_MANAGE_CONTENT privilege, which includes these privileges. For more information, see Catalog privileges for Polaris Catalog.
Attach the catalog role to the principal role for your service connection. This lets the service connection access the catalog. For instructions, see Grant a catalog role to a principal role.
Step 3: Create a catalog integration for Polaris Catalog¶
Create a catalog integration for Polaris Catalog by using the CREATE CATALOG INTEGRATION (Polaris Catalog) command. For WAREHOUSE, specify the name of the external catalog that you configured in your Polaris Catalog account.
CREATE OR REPLACE CATALOG INTEGRATION my_polaris_int
CATALOG_SOURCE = POLARIS
TABLE_FORMAT = ICEBERG
CATALOG_NAMESPACE = 'myPolarisCatalogNamespace'
REST_CONFIG = (
CATALOG_URI = 'https://myAccount.snowflakecomputing.com/polaris/api/catalog'
WAREHOUSE = 'myPolarisExternalCatalogName'
)
REST_AUTHENTICATION = (
TYPE = OAUTH
OAUTH_CLIENT_ID = 'myClientId'
OAUTH_CLIENT_SECRET = 'myClientSecret'
OAUTH_ALLOWED_SCOPES = ('PRINCIPAL_ROLE:ALL')
)
ENABLED = TRUE;
Note
You can use this catalog integration to sync one or more Snowflake-managed tables.
Step 4: Create a Snowflake-managed Iceberg table¶
Create a Snowflake-managed Iceberg table by using the CREATE ICEBERG TABLE (Snowflake as the Iceberg catalog) command.
For the CATALOG_SYNC parameter, specify the name of your catalog integration for Polaris Catalog.
CREATE OR REPLACE ICEBERG TABLE my_managed_iceberg_table (col1 INT)
CATALOG = 'SNOWFLAKE'
EXTERNAL_VOLUME = 'my_external_volume'
BASE_LOCATION = 'my_managed_iceberg_table'
CATALOG_SYNC = 'my_polaris_int';
When you modify the table in Snowflake, the changes are automatically synchronized with the external catalog in your Polaris Catalog account. Other engines such as Apache Spark™ can query the table by connecting to Polaris Catalog.