Table support and schema¶

This topic covers the table types, data types, and schema capabilities supported by Snowpipe Streaming.

Apache Iceberg™ table support¶

Snowpipe Streaming supports ingestion into Snowflake-managed Apache Iceberg™ tables, including both Iceberg v2 and Iceberg v3 tables. For more information, see Snowpipe Streaming high-performance architecture with Apache Iceberg™ tables.

Schema evolution¶

Snowpipe Streaming supports automatic table schema evolution on tables with ENABLE_SCHEMA_EVOLUTION = TRUE. Snowflake infers supported new top-level fields in the incoming stream and adds the corresponding columns. Schema evolution occurs asynchronously after the append is durably acknowledged. Snowflake can also drop NOT NULL constraints when incoming rows omit required columns. For more information, see Table schema evolution.

Limitations of schema evolution:

• The precision, scale, or length of existing columns can’t be increased automatically.
• For standard Snowflake tables, schema evolution isn’t supported for structured data types. New columns that contain structured values are inferred as VARIANT instead.
• For Snowflake-managed Iceberg tables, both v2 and v3 support new top-level columns inferred from JSON numbers, booleans, strings, dates, times, and timestamps.
• Nested JSON objects infer as VARIANT and can add a column only on Iceberg v3. VARIANT isn’t supported on Iceberg v2.
• Lists and arrays can’t add columns on Iceberg v2 or v3. Schema evolution doesn’t create structured ARRAY, OBJECT, or MAP columns; define these columns when you create the table.
• BINARY, GEOGRAPHY, and GEOMETRY columns are supported by managed Iceberg tables but aren’t inferred from incoming JSON values. Define these columns when you create the table.
• Nanosecond timestamps require Iceberg v3. Iceberg v2 can ingest the value at microsecond precision when default-scale coercion is enabled; otherwise the row is rejected.
• A new field whose only observed value is null is added as VARCHAR.
• Externally managed Iceberg tables and external tables aren’t supported.

Insert-only operations¶

The API is currently limited to inserting rows. To modify, delete, or combine data, write the “raw” records to one or more staging tables. Merge, join, or transform the data by using continuous data pipelines to insert modified data into destination reporting tables.

Supported Java data types¶

The following table summarizes which Java data types are supported for ingestion into Snowflake columns: