Configure storage for Iceberg tables¶

Configure an external volume¶

You can create and configure an external volume to use with one or more Iceberg tables. You must create an external volume before you can create an Iceberg table.

For specific instructions, see the following topics:

• Amazon S3

• Google Cloud Storage

• Azure Storage

Grant Snowflake access to your storage¶

To grant Snowflake access to your storage locations for Iceberg tables, you use the identity and access management service for your cloud provider. You grant an identity, or principal, limited access to your storage without exchanging secrets. This is the same access model that Snowflake uses for other integrations, including storage integrations.

Snowflake provisions a principal for your entire Snowflake account when you create an external volume. The principal is as follows, depending on your cloud provider:

Snowflake authenticates directly with your storage provider, and the Snowflake-provisioned principal assumes a role that you specify. The role must have permission to perform operations on your storage location. For example, Snowflake can read from a storage location only if the role has permission to read from that storage location.

Snowflake requires permission to perform the following actions on Iceberg tables:

For full instructions on granting Snowflake access to your storage for Iceberg tables, see the following topics:

• Configure an external volume for Amazon S3

• Configure an external volume for Google Cloud Storage

• Configure an external volume for Azure

Assign an active storage location¶

Each external volume supports a single active storage location. If you specify multiple storage locations in a CREATE EXTERNAL VOLUME statement, Snowflake assigns one location as the active location. The active location remains the same for the lifetime of the external volume.

Assignment occurs the first time you use the external volume in a CREATE ICEBERG TABLE statement. Snowflake uses the following logic to choose an active location:

• If the STORAGE_LOCATIONS list contains one or more local storage locations, Snowflake uses the first local storage location in the list. A local storage location is with the same cloud provider and in the same region as your Snowflake account.

• If the STORAGE_LOCATIONS list does not contain any local storage locations, Snowflake selects the first location in the list.

Verify storage access¶

For Snowflake-managed tables, Snowflake verifies access to the active storage location on your external volume. The ALLOW_WRITES property of the external volume must be set to TRUE.

Verification occurs in the following situations:

• The first time you specify that external volume in a CREATE ICEBERG TABLE statement for a Snowflake-managed table.

• The first time you convert a table to use Snowflake as the Iceberg catalog.

Snowflake tries the following storage operations to verify the storage location.

1. Writing a test file.

2. Reading the file.

3. Listing the contents of the file’s path.

4. Deleting the file.

If any of the storage operations fails, the CREATE ICEBERG TABLE (or ALTER ICEBERG TABLE … CONVERT TO MANAGED) statement fails and you receive an error message.

Manage files in storage¶

This section explains how management of Iceberg table files in storage works, according to the type of Iceberg table.

Configure Time Travel¶

With Time Travel, you can query historical data within a table’s data retention period. Snowflake deletes objects after the table retention period expires when Snowflake-managed table data is deleted or the table is dropped. This might incur costs with your cloud storage provider for longer than the table’s lifetime.

You can configure Time Travel for Iceberg tables by using the DATA_RETENTION_TIME_IN_DAYS object parameter.

Snowflake uses the DATA_RETENTION_TIME_IN_DAYS parameter to handle metadata in different ways, depending on the type of Iceberg table. For more information, see Manage files in storage.

Enable storage access logs¶

To diagnose issues and audit access to the storage locations associated with an external volume, you can enable storage logging. Storage logs help you identify the cause of missing or corrupted files.

Enable logging with your storage provider. Because you own and manage storage for Iceberg tables, Snowflake can’t enable logging or auditing on your Iceberg storage locations.

To learn about storage access logs for your storage provider, see the following external topics:

• Logging options for Amazon S3

• Usage logs and storage logs for Google Cloud Storage

• Azure Storage analytics logging

Protect files with versioning and object retention¶

If your Iceberg table data is in a central data repository (or data lake) that is operated on by multiple tools and services, accidental deletion or corruption might occur. To protect Iceberg table data and ensure retrieval of accidentally deleted or overwritten data, use storage lifecycle management and versioning offered by your storage provider.

With lifecycle management, you can set retention and tracking rules for storage objects. To learn about lifecycle management for your storage provider, see the following external topics:

• Managing your storage lifecycle for Amazon S3

• Object Lifecycle Management for Google Cloud Storage

• Lifecycle management policies in Azure

To support object recovery, you can also enable versioning for your external cloud storage.

• To enable versioning for Amazon S3, see Enabling versioning on buckets.

• To enable versioning for Google Cloud Storage, see Use Object Versioning.

• To enable versioning for Azure, see Enable blob versioning.

Encrypt Iceberg table files¶

Snowflake can read Iceberg table files in storage that you encrypt using common server-side encryption (SSE) schemes. You should use your cloud service provider to manage encryption keys, and grant the Snowflake principal access to your keys if you use a customer-managed key.

For Amazon S3, Snowflake supports the following SSE options:

For Google Cloud Storage, Snowflake supports the following SSE option:

Set an external volume at the account, database, or schema level¶

To define which existing external volume to use for Iceberg tables, you can set the EXTERNAL_VOLUME parameter at the following levels:

Account:

Account administrators can use the ALTER ACCOUNT command to set the parameter for the account. If the value is set for the account, all Iceberg tables created in the account read from and write to this external volume by default.

Object:

Users can execute the appropriate CREATE <object> or ALTER <object> command to override the EXTERNAL_VOLUME parameter value at the database or schema level. The lowest-scoped declaration is used: schema > database > account.

In addition to the minimum privileges required to modify an object using the appropriate ALTER <object_type> command, a role must have the USAGE privilege on the external volume.

ALTER DATABASE my_database_1
  SET EXTERNAL_VOLUME = 'my_s3_vol';

CREATE ICEBERG TABLE iceberg_reviews_table (
  id STRING,
  product_name STRING,
  product_id STRING,
  reviewer_name STRING,
  review_date DATE,
  review STRING
)
CATALOG = 'SNOWFLAKE'
BASE_LOCATION = 'my/product_reviews/';