Configure an external volume for Iceberg tables¶

Step 1: Configure access permissions for the S3 bucket¶

Step 2: Create an IAM role in AWS¶

In the AWS Management Console, create an AWS IAM role to grant privileges on the S3 bucket containing your data files.

1. Log in to the AWS Management Console.

2. From the home dashboard, select Identity & Access Management (IAM):

   

3. From the left-hand navigation pane, select Roles.

4. Select Create role.

   

5. For the trusted entity type, select Another AWS account.

6. In the Account ID field, enter your own AWS account ID. In a later step, you modify the trusted relationship and grant access to Snowflake.

7. Select the Require external ID option. An external ID is used to grant access to your AWS resources (such as S3 buckets) to a third party like Snowflake.

   Enter a placeholder ID such as 0000. In a later step, you modify the trust relationship for your IAM role and specify the external ID for your external volume.

8. Select Next.

9. Locate the policy you created in Step 1: Configure access permissions for the S3 bucket, and select this policy.

10. Select Next.

    

11. Enter a name and description for the role, and select Create role. You have now created an IAM policy for a bucket, created an IAM role, and attached the policy to the role.

12. On the role summary page, locate and record the Role ARN value. You use this value in the next step to create a Snowflake external volume that references this role.

    

Step 3: Grant privileges required for SSE-KMS encryption to the IAM role (optional)¶

To upload an object encrypted with an AWS KMS key to Amazon S3, the IAM role that you created in Step 2: Create an IAM role in AWS needs kms:GenerateDataKey permissions on the key. To download an object encrypted with an AWS KMS key, the IAM role needs kms:Decrypt permissions on the key.

If you want to use a KMS key for your server-side encryption, follow these steps to create a key and reference it.

1. In the AWS Management Console, go to the KMS service. From the left navigation, select Customer managed keys, and then select Create key. You must create a key in the same region as your bucket.

2. Create a symmetric key type. For the key usage, select Encrypt and decrypt. Select Next.

3. In the Alias box, create a name for the key and select Next.

4. If needed, provide an administrator for the key and select Next.

5. In the Define key usage permissions step, enter the name of your IAM role. Select the checkbox next to the role, then select Next.

6. Select Finish to create the key.

7. Find the key in the list of customer managed keys, select it, and record its ARN. The following is an example of an ARN for a key: arn:aws:kms:us-west-2:111111122222:key/1a1a11aa-aa1a-aaa1a-a1a1-000000000000.

   When you create your external volume, set the KMS_KEY_ID value to the ARN of your key.

Step 4: Create an external volume in Snowflake¶

Create an external volume using the CREATE EXTERNAL VOLUME command.

The following example creates an external volume that defines an Amazon S3 storage location with encryption:

CREATE OR REPLACE EXTERNAL VOLUME exvol
   STORAGE_LOCATIONS =
      (
         (
            NAME = 'my-s3-us-west-2'
            STORAGE_PROVIDER = 'S3'
            STORAGE_BASE_URL = 's3://MY_EXAMPLE_BUCKET/'
            STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::123456789012:role/myrole'
            ENCRYPTION=(TYPE='AWS_SSE_KMS' KMS_KEY_ID='arn:aws:kms:us-west-2:111111122222:key/1a1a11aa-aa1a-aaa1a-a1a1-000000000000')
         )
      );

Step 5: Retrieve the AWS IAM user for your Snowflake account¶

1. To retrieve the ARN for the AWS IAM user that was created automatically for your Snowflake account, use the DESCRIBE EXTERNAL VOLUME command. Specify the name of the external volume that you created previously.

   예:

   DESC EXTERNAL VOLUME exvol;
   

   Copy

2. Record the values for the following properties:

   Property	Description
   STORAGE_AWS_IAM_USER_ARN	The AWS IAM user created for your Snowflake account; for example, arn:aws:iam::123456789001:user/abc1-b-self1234. Snowflake provisions a single IAM user for your entire Snowflake account. All S3 external volumes in your account use that IAM user.
   STORAGE_AWS_EXTERNAL_ID	The external ID that Snowflake uses to establish a trust relationship with AWS. If you didn’t specify an external ID (STORAGE_AWS_EXTERNAL_ID) when you created the external volume, Snowflake generates an ID for you to use.

   You provide these values in the next step.

Step 6: Grant the IAM user permissions to access bucket objects¶

In this step, you configure permissions that allow the IAM user for your Snowflake account to access objects in your S3 bucket.

1. Log in to the AWS Management Console.

2. Select Identity & Access Management (IAM):

   

3. From the left-hand navigation pane, select Roles.

4. Select the role you created in Step 2: Create an IAM role in AWS.

5. Select the Trust relationships tab.

6. Select Edit trust relationship.

7. Modify the policy document with the output values that you recorded in Step 5: Retrieve the AWS IAM user for your Snowflake account:

   Policy document for IAM role

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Sid": "",
         "Effect": "Allow",
         "Principal": {
           "AWS": "<snowflake_user_arn>"
         },
         "Action": "sts:AssumeRole",
         "Condition": {
           "StringEquals": {
             "sts:ExternalId": "<snowflake_external_id>"
           }
         }
       }
     ]
   }
   

   Copy

   여기서

   • snowflake_user_arn is the STORAGE_AWS_IAM_USER_ARN value you recorded.

   • snowflake_external_id is the STORAGE_AWS_EXTERNAL_ID value you recorded.

   참고

   You must update this policy document if you create a new external volume (or recreate an existing external volume using the CREATE OR REPLACE EXTERNAL VOLUME syntax) and do not provide your own external ID. For security reasons, a new or recreated external volume has a different external ID and cannot resolve the trust relationship unless you update this trust policy.

8. Select Update Trust Policy. The changes are saved.

Next steps¶

After you configure an external volume, you can create an Iceberg table.

• To create a read-only Iceberg table that uses a catalog integration, see Iceberg 테이블의 카탈로그 통합 구성하기.

• To create an Iceberg table with full Snowflake platform support, see Snowflake를 카탈로그로 사용하여 Iceberg 테이블 만들기.

Step 1: Create an external volume in Snowflake¶

Create an external volume using the CREATE EXTERNAL VOLUME command.

The following example creates an external volume that defines a GCS storage location with encryption:

CREATE EXTERNAL VOLUME exvol
  STORAGE_LOCATIONS =
    (
      (
        NAME = 'my-us-east-1'
        STORAGE_PROVIDER = 'GCS'
        STORAGE_BASE_URL = 'gcs://mybucket1/path1/'
        ENCRYPTION=(TYPE='GCS_SSE_KMS' KMS_KEY_ID = '1234abcd-12ab-34cd-56ef-1234567890ab')
      )
    );

Step 2: Retrieve the Cloud Storage service account for your Snowflake account¶

To retrieve the ID for the Cloud Storage service account that was created automatically for your Snowflake account, use the DESCRIBE EXTERNAL VOLUME command. Specify the name of the external volume that you created previously.

예:

DESC EXTERNAL VOLUME exvol;

Copy

Record the value of the STORAGE_GCP_SERVICE_ACCOUNT property in the output (for example, service-account-id@project1-123456.iam.gserviceaccount.com).

Snowflake provisions a single Cloud Storage service account for your entire Snowflake account. All Google Cloud Storage external volumes use that service account.

Step 3: Grant the service account permissions to access bucket objects¶

In this step, you configure IAM access permissions for Snowflake in your Google Cloud Platform Console.

Next steps¶

After you configure an external volume, you can create an Iceberg table.

• To create a read-only Iceberg table that uses a catalog integration, see Iceberg 테이블의 카탈로그 통합 구성하기.

• To create an Iceberg table with full Snowflake platform support, see Snowflake를 카탈로그로 사용하여 Iceberg 테이블 만들기.

Step 1: Create an external volume in Snowflake¶

Create an external volume using the CREATE EXTERNAL VOLUME command.

The following example creates an external volume that defines an Azure storage location with encryption:

CREATE EXTERNAL VOLUME exvol
  STORAGE_LOCATIONS =
    (
      (
        NAME = 'my-azure-northeurope'
        STORAGE_PROVIDER = 'AZURE'
        STORAGE_BASE_URL = 'azure://exampleacct.blob.core.windows.net/my_container_northeurope/'
        AZURE_TENANT_ID = 'a123b4c5-1234-123a-a12b-1a23b45678c9'
      )
    );

Step 2: Grant Snowflake access to the storage location¶

1. To retrieve a URL to the Microsoft permissions request page, use the DESCRIBE EXTERNAL VOLUME command. Specify the name of the external volume that you created previously.

   DESC EXTERNAL VOLUME exvol;
   

   Copy

   Record the values for the following properties:

   Property	Description
   AZURE_CONSENT_URL	URL to the Microsoft permissions request page.
   AZURE_MULTI_TENANT_APP_NAME	Name of the Snowflake client application created for your account. In a later step in this section, you grant this application permission to obtain an access token on your allowed storage location.

   You use these values in the following steps.

2. In a web browser, navigate to the Microsoft permissions request page (the AZURE_CONSENT_URL).

3. Select Accept. This action allows the Azure service principal created for your Snowflake account to obtain an access token on any resource inside your tenant. Obtaining an access token succeeds only if you grant the service principal the appropriate permissions on the container (see the next step).

   The Microsoft permissions request page redirects to the Snowflake corporate site (snowflake.com).

4. Log in to the Microsoft Azure portal.

5. Go to Azure Services » Storage Accounts. Select the name of the storage account you want to grant the Snowflake service principal access to.

   참고

   You must set IAM permissions for an external volume at the storage account level, not the container level.

6. Select Access Control (IAM) » Add role assignment.

7. Select the Storage Blob Data Contributor role to grant read and write access to the Snowflake service principal.

   참고

   The Storage Blob Data Contributor role grants write access to the external volume location. To completely configure write access, the ALLOW_WRITES parameter of the external volume must be set to TRUE (the default value).

8. Search for the Snowflake service principal. This is the identity in the AZURE_MULTI_TENANT_APP_NAME property in the DESC EXTERNAL VOLUME output (in Step 1). Search for the string before the underscore in the AZURE_MULTI_TENANT_APP_NAME property.

   중요

   • It can take an hour or longer for Azure to create the Snowflake service principal requested through the Microsoft request page in this section. If the service principal is not available immediately, wait an hour or two and then search again.

   • If you delete the service principal, the external volume stops working.

   

9. Select Review + assign.

   참고

   It can take up to 10 minutes for changes to take effect when you assign a role. For more information, see Symptom - Role assignment changes are not being detected in the Microsoft Azure documentation.

Next steps¶

After you configure an external volume, you can create an Iceberg table.

• To create a read-only Iceberg table that uses a catalog integration, see Iceberg 테이블의 카탈로그 통합 구성하기.

• To create an Iceberg table with full Snowflake platform support, see Snowflake를 카탈로그로 사용하여 Iceberg 테이블 만들기.

예¶

다음 문은 my_database_1 이라는 데이터베이스의 외부 볼륨(my_s3_vol)을 설정합니다.

ALTER DATABASE my_database_1
  SET EXTERNAL_VOLUME = 'my_s3_vol';

데이터베이스 수준에서 외부 볼륨을 설정한 후 외부 볼륨을 지정하지 않고도 해당 데이터베이스에 Iceberg 테이블을 생성할 수 있습니다. 다음 문은 Snowflake를 카탈로그로 사용하고 데이터베이스에 설정된 기본 외부 볼륨(my_s3_vol)을 사용하는 Iceberg 테이블을 my_database_1 에 생성합니다.

CREATE ICEBERG TABLE iceberg_reviews_table (
  id STRING,
  product_name STRING,
  product_id STRING,
  reviewer_name STRING,
  review_date DATE,
  review STRING
)
CATALOG = 'SNOWFLAKE'
BASE_LOCATION = 'my/product_reviews/';