Sync a Snowflake-managed table with Polaris Catalog

To query a Snowflake-managed Iceberg table using a third-party engine such as Apache Spark™, you can sync the table with Polaris Catalog.

This topic covers how to sync a Snowflake-managed Iceberg table with Polaris Catalog using a catalog integration in Snowflake and an external catalog in Polaris Catalog.

Step 1: Create an external volume

If you don’t have one already, start by creating an external volume in Snowflake that provides access to the cloud storage location where you want to store your table data and metadata.

Complete the instructions for your cloud storage service:

Step 2: Configure Polaris Catalog resources

Next, complete the steps in this section to create an external catalog and service connection in your Polaris Catalog account.

  1. Follow the instructions in Create a catalog to create an external catalog in your Polaris Catalog account. Make sure that the following settings for the external catalog are configured:

    Polaris Catalog syncs your Snowflake-managed tables to this external catalog.

  2. If you don’t already have a service connection for Snowflake, follow the instructions in Configure a service connection to create a connection for the Snowflake engine in your Polaris Catalog account.

  3. Configure a catalog role for your external catalog with privileges that allow access to your external catalog. For instructions, see Grant privileges to a catalog.

    The catalog role must have the following privileges on the catalog:

    • TABLE_CREATE

    • TABLE_WRITE_PROPERTIES

    • TABLE_DROP

    • NAMESPACE_CREATE

    • NAMESPACE_DROP

    You can either grant each of these privileges to the catalog role, or grant the CATALOG_MANAGE_CONTENT privilege, which includes these privileges. For more information, see Catalog privileges for Polaris Catalog.

  4. Attach the catalog role to the principal role for your service connection. This lets the service connection access the catalog. For instructions, see Grant a catalog role to a principal role.

Step 3: Create a catalog integration for Polaris Catalog

Create a catalog integration for Polaris Catalog by using the CREATE CATALOG INTEGRATION (Polaris Catalog) command. For WAREHOUSE, specify the name of the external catalog that you configured in your Polaris Catalog account.

CREATE OR REPLACE CATALOG INTEGRATION my_polaris_int
  CATALOG_SOURCE = POLARIS
  TABLE_FORMAT = ICEBERG
  CATALOG_NAMESPACE = 'myPolarisCatalogNamespace'
  REST_CONFIG = (
    CATALOG_URI = 'https://myAccount.snowflakecomputing.com/polaris/api/catalog'
    WAREHOUSE = 'myPolarisExternalCatalogName'
  )
  REST_AUTHENTICATION = (
    TYPE = OAUTH
    OAUTH_CLIENT_ID = 'myClientId'
    OAUTH_CLIENT_SECRET = 'myClientSecret'
    OAUTH_ALLOWED_SCOPES = ('PRINCIPAL_ROLE:ALL')
  )
  ENABLED = TRUE;
Copy

Note

You can use this catalog integration to sync one or more Snowflake-managed tables.

Step 4: Create a Snowflake-managed Iceberg table

Create a Snowflake-managed Iceberg table by using the CREATE ICEBERG TABLE (Snowflake as the Iceberg catalog) command.

For the CATALOG_SYNC parameter, specify the name of your catalog integration for Polaris Catalog.

CREATE OR REPLACE ICEBERG TABLE my_managed_iceberg_table (col1 INT)
  CATALOG = 'SNOWFLAKE'
  EXTERNAL_VOLUME = 'my_external_volume'
  BASE_LOCATION = 'my_managed_iceberg_table'
  CATALOG_SYNC = 'my_polaris_int';
Copy

When you modify the table in Snowflake, the changes are automatically synchronized with the external catalog in your Polaris Catalog account. Other engines such as Apache Spark™ can query the table by connecting to Polaris Catalog.

Language: English