Openflow Connector for PostgreSQL: Iceberg table destinations¶

The Openflow Connector for PostgreSQL supports writing to Snowflake-managed Apache Apache Iceberg™ tables as an opt-in destination format. Iceberg v2 and v3 are both supported. Setting Table Storage Format = ICEBERG in the connector’s parameter context is the only change required. All Iceberg storage settings (external volume, catalog, version, and serialization policy) are inherited from the Snowflake destination database defaults.

Storage can be either Snowflake-managed (EXTERNAL_VOLUME = 'SNOWFLAKE_MANAGED') or a customer-configured external volume. With Snowflake-managed storage, no external cloud storage or IAM grants are required. For more information, see Snowflake storage for Iceberg tables.

Existing connectors using standard tables aren’t affected.

Prerequisites¶

• Openflow runtime: An existing runtime to host the connector.
• PostgreSQL source configured for CDC: Logical replication enabled (wal_level = logical), a publication created, and a user with replication privileges. For details, see Set up the Openflow Connector for PostgreSQL.
• Snowflake external volume (customer-managed storage only): An external volume configured for Iceberg storage, with USAGE granted to the connector’s Snowflake role. See CREATE EXTERNAL VOLUME. Not required when using Snowflake-managed storage (EXTERNAL_VOLUME = 'SNOWFLAKE_MANAGED').
• Snowflake destination database: An existing database configured with Iceberg parameters (next section).

Step 1: Configure the Snowflake destination database¶

Set the Iceberg defaults on the destination database. The connector reads these defaults at runtime, so no per-connector Iceberg configuration is needed beyond Table Storage Format.

Option A: Snowflake-managed storage¶

With Snowflake-managed storage, Snowflake stores and manages the Iceberg table files for you. No external cloud storage or IAM grants are required.

CREATE DATABASE <db>
  EXTERNAL_VOLUME = 'SNOWFLAKE_MANAGED'
  ICEBERG_VERSION_DEFAULT = <2|3>
  STORAGE_SERIALIZATION_POLICY = <COMPATIBLE|OPTIMIZED>;

To configure an existing database:

ALTER DATABASE <db> SET
  EXTERNAL_VOLUME = 'SNOWFLAKE_MANAGED'
  ICEBERG_VERSION_DEFAULT = <2|3>
  STORAGE_SERIALIZATION_POLICY = <COMPATIBLE|OPTIMIZED>;

Option B: Customer-configured external volume¶

If you need to keep table files in your own cloud storage, configure the database with your external volume:

CREATE DATABASE <db>
  EXTERNAL_VOLUME = '<volume>'
  ICEBERG_VERSION_DEFAULT = <2|3>
  STORAGE_SERIALIZATION_POLICY = <COMPATIBLE|OPTIMIZED>;

To configure an existing database:

ALTER DATABASE <db> SET
  EXTERNAL_VOLUME = '<volume>'
  ICEBERG_VERSION_DEFAULT = <2|3>
  STORAGE_SERIALIZATION_POLICY = <COMPATIBLE|OPTIMIZED>;

If using a customer-configured external volume (Option B), grant the connector’s Snowflake role USAGE on the external volume:

GRANT USAGE ON EXTERNAL VOLUME <volume> TO ROLE <openflow_role>;

This step is not required for Snowflake-managed storage.

Step 2: Set Table Storage Format in the connector’s parameter context¶

Set the Table Storage Format parameter to ICEBERG in the connector’s destination parameter context. The default is STANDARD.

For the full connector creation and configuration workflow, see Set up the Openflow Connector for PostgreSQL.

Step 3: Start and verify¶

Start the connector as usual. After the initial snapshot completes, verify the destination tables are Iceberg:

-- Confirm the table is Iceberg
SELECT GET_DDL('TABLE', '<db>.<schema>.<table>');

-- Confirm the Iceberg version on the database
SHOW PARAMETERS LIKE 'ICEBERG_VERSION_DEFAULT' IN DATABASE <db>;

Known limitations¶

• Tri-Secret Secure accounts and Snowflake-managed storage: Accounts with Tri-Secret Secure (TSS) enabled may be unable to create new Snowflake-managed Iceberg tables on Snowflake storage. For details, see Encryption.
• Incompatible type changes on the source cause table failure: Only same-family type widening is allowed on Iceberg tables (for example, int to long, float to double, or decimal(P,S) to decimal(P',S) with wider precision). Cross-family changes such as int to decimal, long to string, or int to boolean are not allowed. If a source column’s type changes to an incompatible Iceberg type, the table is marked as failed and requires a rebootstrap.
• TIMETZ offset not preserved: Iceberg timestamptz stores only the UTC instant. PostgreSQL TIMETZ values lose the original timezone offset when written to Iceberg tables.
• Do not change Table Storage Format or Iceberg version after the connector starts: The connector’s Table Storage Format and the destination database’s Iceberg version should not be modified while the connector is running or stopped. Mixing settings across destination tables is not supported. To switch, follow the steps in Switching table storage format or Iceberg version.

Type mapping reference¶

The following table shows how PostgreSQL types map to Snowflake standard and Iceberg destination types:

Source types not listed in the table are mapped to TEXT on standard tables and string on Iceberg tables.

Switching table storage format or Iceberg version¶

Switching between Standard and Iceberg, or between Iceberg v2 and v3, requires recreating the connector. Follow these steps:

1. Stop the connector.
2. Delete the process group in Openflow.
3. Manually clean up the destination database (drop the replicated schemas/tables, or use a new database).
4. Reimport the connector with the new Table Storage Format or Iceberg version configured on the destination database.

This ensures all connector state is correctly cleaned up within Openflow. The new connector performs a fresh snapshot into the destination.

References¶

• CREATE EXTERNAL VOLUME
• Data types for Apache Iceberg tables
• ALTER DATABASE
• STORAGE_SERIALIZATION_POLICY
• Set up the Openflow Connector for PostgreSQL