Configure a catalog integration for Unity Catalog

Use the CREATE CATALOG INTEGRATION (Apache Iceberg™ REST) command to create a REST catalog integration that uses vended credentials or an external volume to connect to Databricks Unity Catalog.

Note

You can create a catalog integration for Unity Catalog where the Databricks workspace is hosted on one of the following cloud providers:

  • AWS

  • Azure

  • Google Cloud

You can configure a catalog integration for Unity Catalog that uses OAuth or bearer authentication:

Configure an OAuth catalog integration

Step 1: Find your Databricks workspace URL

Your Databricks workspace URL is the URL that you use to access your Databricks workspace. You need to find this URL because you specify it later when you create a catalog integration.

  1. Find your Databricks workspace URL.

    For instructions on how to find this URL, see the topic for where your Databricks account is hosted:

  2. Copy your Databricks workspace URL into a text editor.

Step 2: Add a service principal in Databricks

  1. To add a service principal, see the topic for where your Databricks account is hosted:

  2. Copy the Application ID value for your service principal into a text editor and store it securely. You specify this value later when you create a catalog integration in Snowflake.

Step 3: Create an OAuth secret for your service principal

  1. To create an OAuth secret for your service principal, see the topic for where your Databricks account is hosted:

  2. Copy the Secret value that you generated into a text editor and store it securely. You specify this value later when you create a catalog integration in Snowflake.

    Important

    The client secret is only displayed once. Make sure to copy it before closing the dialog.

Step 4: Enable Snowflake access to your catalog in Unity Catalog

In this step, you use Databricks to enable Snowflake access to your catalog in Unity Catalog.

To enable Snowflake access to your catalog in Unity Catalog through vended credentials, first, at the metastore level, you must enable external data access on the metastore. Next, you need to grant your service principal Unity Catalog privileges to your catalog.

Enable external data access on the metastore (vended credentials only)

If you’re creating a catalog integration that uses vended credentials, you must enable external data access on the metastore in Databricks. If you’re creating a catalog integration that uses an external volume, you can skip this step.

For instructions on how to enable external data access on the metastore, see the topic for where your Databricks account is hosted:

Assign your service principal to a workspace

Next, you need to assign your service principal to your Databricks workspace.

For instructions, see the topic for where your Databricks account is hosted:

Grant your service principal access to your catalog

Next, you must grant your service principal Unity Catalog privileges. You need to grant these privileges to your service principal to allow Snowflake to access the catalog based on the privileges that you specify.

Privileges for full functionality

To enable full functionality in Snowflake, you must grant the following privileges:

Note

If you want to restrict Snowflake access, see Unity Catalog privileges and securable objects in the Databricks documentation.

Privilege

Description

EXTERNAL USE SCHEMA

Allows Unity Catalog to generate and provide temporary, scoped credentials to Snowflake for accessing table data in cloud storage.

Note

This privilege is only required when you create a catalog integration that uses vended credentials; it’s not required when you create a catalog integration that uses an external volume, so if you’re using an external volume, remove it from the example code block.

MODIFY

Allows Snowflake to add, update, and delete data in a table.

SELECT

Allows Snowflake to query tables and access table metadata. Required for all operations in Snowflake, including reading data and discovering tables in the catalog-linked database.

USE CATALOG

Allows Snowflake to access the catalog. Required to connect to and interact with any objects in the Unity Catalog.

USE SCHEMA

Allows Snowflake access to schemas (namespaces) within the catalog. Required to view and work with tables in specific schemas.
Grant privileges

You can grant privileges by using Catalog Explorer or SQL.

To grant permissions by using the Databricks Catalog Explorer, see the topic for where your Databricks account is hosted:

Important

In the Principals field, you must enter the name of your service principal, not the email address for a user or the name of a group.

Step 5: Create a catalog integration

The following example creates a REST catalog integration that uses OAuth:

CREATE OR REPLACE CATALOG INTEGRATION unity_catalog_int_oauth_vended_credentials
CATALOG_SOURCE = ICEBERG_REST
TABLE_FORMAT = ICEBERG
REST_CONFIG = (
  CATALOG_URI = '<databricks_workspace_url>/api/2.1/unity-catalog/iceberg-rest'
  CATALOG_NAME = '<catalog_name>'
  ACCESS_DELEGATION_MODE = VENDED_CREDENTIALS
)
REST_AUTHENTICATION = (
  TYPE = OAUTH
  OAUTH_TOKEN_URI = '<databricks_workspace_url>/oidc/v1/token'
  OAUTH_CLIENT_ID = '<client_id>'
  OAUTH_CLIENT_SECRET = '<oauth_secret>'
  OAUTH_ALLOWED_SCOPES = ('all-apis')
)
ENABLED = TRUE;
Copy

Where:

  • <databricks_workspace_url> specifies the URL for your Databricks workspace. To find this URL, see Step 1: Find your Databricks workspace URL.

    Here is an example of a Databricks workspace URL for each cloud platform:

    • Databricks on AWS: https://dbc-a1a1a1a1-a1a1.cloud.databricks.com

    • Azure Databricks: https://adb-1111111111111111.1.azuredatabricks.net

    • Databricks on Google Cloud: https://1111111111111111.1.gcp.databricks.com

  • <catalog_name> specifies the name of your catalog in Unity Catalog that you want to connect Snowflake to. You can find this name in the Databricks workspace under Data > Catalogs.

  • ACCESS_DELEGATION_MODE = VENDED_CREDENTIALS configures the catalog integration to use vended credentials from Unity Catalog.

    Note

    If you’re creating a catalog integration that uses an external volume, you must exclude the ACCESS_DELEGATION_MODE parameter.

  • <client_id> specifies the OAuth client ID for your Databricks service principal. You copied this value when you added a service principal in Databricks.

    Note

    In Databricks, this value is called the Application ID, not the Client ID.

  • <oauth_secret> specifies the OAuth secret for your Databricks service principal. You copied this value when you created an OAuth secret for your service principal.

Step 6: Verify your catalog integration

Next steps

After you configure a catalog integration for your catalog in Unity Catalog, use the CREATE DATABASE (catalog-linked) command to create a catalog-linked database by specifying the catalog integration that you created. Snowflake then automatically syncs with your catalog in Unity Catalog to detect schemas and Iceberg tables, and registers the remote tables to the catalog-linked database.

When you create your catalog-linked database, you specify the catalog integration.

For example:

CREATE OR REPLACE DATABASE my_linked_db
   LINKED_CATALOG = (
      CATALOG = 'unity_catalog_int_oauth_vended_credentials'
   );
Copy

Note

Configure a bearer token catalog integration

Step 1: Find your Databricks workspace URL

Your Databricks workspace URL is the URL that you use to access your Databricks workspace. You need to find this URL because you specify it later when you create a catalog integration.

  1. Find your Databricks workspace URL.

    For instructions on how to find this URL, see the topic for where your Databricks account is hosted:

  2. Copy your Databricks workspace URL into a text editor.

Step 2: Add a personal access token (PAT) in Databricks

You need to add a personal access token (PAT) because you must specify it when you create a catalog integration that uses a bearer token for authentication.

  1. To add a PAT in Databricks, see the topic for where your Databricks account is hosted:

  2. Copy the value for your PAT into a text editor and store it securely. You specify this value later when you create a catalog integration in Snowflake.

Step 3: Enable Snowflake access to your catalog in Unity Catalog

In this step, you use Databricks to enable Snowflake access to your catalog in Unity Catalog.

To enable Snowflake access to your catalog in Unity Catalog through vended credentials, first, at the metastore level, you must enable external data access on the metastore. Next, you need to grant your Databricks user Unity Catalog privileges to your catalog, which your PAT inherits.

Enable external data access on the metastore (vended credentials only)

If you’re creating a catalog integration that uses vended credentials, you must enable external data access on the metastore in Databricks. If you’re creating a catalog integration that uses an external volume, you can skip this step.

For instructions on how to enable external data access on the metastore, see the topic for where your Databricks account is hosted:

Grant your Databricks user access to your catalog

Next, you must grant your Databricks user Unity Catalog privileges. You need to grant these privileges to your Databricks user to allow Snowflake access to the catalog based on the privileges that you specify. When you use a PAT for authentication, it inherits all the privileges granted on the Databricks user who created the PAT.

Privileges for full functionality

To enable full functionality in Snowflake, you must grant the following privileges:

Note

If you want to restrict Snowflake access, see Unity Catalog privileges and securable objects in the Databricks documentation.

Privilege

Description

EXTERNAL USE SCHEMA

Allows Unity Catalog to generate and provide temporary, scoped credentials to Snowflake for accessing table data in cloud storage.

Note

This privilege is only required when you create a catalog integration that uses vended credentials; it’s not required when you create a catalog integration that uses an external volume, so if you’re using an external volume, remove it from the example code block.

MODIFY

Allows Snowflake to add, update, and delete data in a table.

SELECT

Allows Snowflake to query tables and access table metadata. Required for all operations in Snowflake, including reading data and discovering tables in the catalog-linked database.

USE CATALOG

Allows Snowflake to access the catalog. Required to connect to and interact with any objects in the Unity Catalog.

USE SCHEMA

Allows Snowflake access to schemas (namespaces) within the catalog. Required to view and work with tables in specific schemas.
Grant privileges

You can grant privileges by using Catalog Explorer or SQL.

To grant permissions by using the Databricks Catalog Explorer, see the topic for where your Databricks account is hosted:

Step 4: Create a catalog integration

The following example creates a REST catalog integration that uses a bearer token with vended credentials:

CREATE OR REPLACE CATALOG INTEGRATION unity_catalog_int_bearer_vended_credentials
  CATALOG_SOURCE = ICEBERG_REST
  TABLE_FORMAT = ICEBERG
  REST_CONFIG = (
    CATALOG_URI = '<databricks_workspace_url>/api/2.1/unity-catalog/iceberg-rest'
    CATALOG_NAME = '<catalog_name>'
    ACCESS_DELEGATION_MODE = VENDED_CREDENTIALS
  )
  REST_AUTHENTICATION = (
    TYPE = BEARER
    BEARER_TOKEN = '<personal_access_token>'
  )
  ENABLED = TRUE;
Copy

Where:

  • <databricks_workspace_url> specifies the URL for your Databricks workspace. To find this URL, see Step 1: Find your Databricks workspace URL.

    Here is an example of a Databricks workspace URL for each cloud platform:

    • Databricks on AWS: https://dbc-a1a1a1a1-a1a1.cloud.databricks.com

    • Azure Databricks: https://adb-1111111111111111.1.azuredatabricks.net

    • Databricks on Google Cloud: https://1111111111111111.1.gcp.databricks.com

  • <catalog_name> specifies the name of your catalog in Unity Catalog that you want to connect Snowflake to. You can find this name in the Databricks workspace under Data > Catalogs.

  • ACCESS_DELEGATION_MODE = VENDED_CREDENTIALS configures the catalog integration to use vended credentials from Unity Catalog.

    Note

    If you’re creating a catalog integration that uses an external volume, you must exclude the ACCESS_DELEGATION_MODE parameter.

  • <personal_access_token> specifies your Databricks personal access token (PAT). An example of a PAT is aaaa111aaaa111a1a1a1a111111a111a1111.

Step 5: Verify your catalog integration

Next steps

After you configure a catalog integration for your catalog in Unity Catalog, use the CREATE DATABASE (catalog-linked) command to create a catalog-linked database by specifying the catalog integration that you created. Snowflake then automatically syncs with your catalog in Unity Catalog to detect schemas and Iceberg tables, and registers the remote tables to the catalog-linked database.

When you create your catalog-linked database, you specify the catalog integration.

For example:

CREATE OR REPLACE DATABASE my_linked_db
   LINKED_CATALOG = (
      CATALOG = 'unity_catalog_int_bearer_vended_credentials'
   );
Copy

Note

Language: English