July 29-August 01, 2024 — 8.28 Release Notes¶
Attention
The release has completed.
For differences between the in-advance and final versions of these release notes, see Release notes change log.
SQL updates¶
New SQL functions¶
The following function(s) are now available with this release:
Function category |
New function |
Description |
---|---|---|
Semi-structured and structured |
Returns an array of objects, each of which contains key-value pairs for an nth element in the input arrays. |
Account Usage: New SEARCH_OPTIMIZATION_BENEFITS view¶
With this release, we are pleased to announce the new SEARCH_OPTIMIZATION_BENEFITS view in the ACCOUNT_USAGE schema.
This view provides information about the number of partitions pruned specifically due to search optimization. This view is similar to the TABLE_PRUNING_HISTORY view but provides information about pruning due to search optimization.
For more information, see SEARCH_OPTIMIZATION_BENEFITS view.
Data governance updates¶
Object Tagging: Support added for replication and failover groups¶
With this release, Snowflake is pleased to announce that you can set tags on replication and failover groups.
For more information, see CREATE and ALTER commands for replication and failover groups: Support added for tags (in this topic).
Data Quality and data metric functions (DMFs) — General Availability¶
With this release, Snowflake is pleased to announce the general availability of Data Quality Monitoring with data metric functions (DMFs). Data Quality Monitoring uses DMFs to continuously monitor data quality metrics such as completeness, accuracy, uniqueness, and validity. You can use Snowflake provided system DMFs for common metrics such as row count, duplicates, and freshness. Alternatively, you can create your own custom DMFs to define metrics that are specific to your own data.
You can either use the DMF in a query to test the quality of data in your pipeline or associate the DMF to desired tables to continuously monitor its quality. The continuous monitoring can either be schedule-based for periodic measurement or trigger-based to measure only when the underlying table is modified.
Since announcing the preview availability in March, we’ve made the following updates:
New schema privilege: CREATE DATA METRIC FUNCTION. This is a change from the preview where you needed to use the CREATE FUNCTION privilege.
Now, your role must have the CREATE DATA METRIC FUNCTION privilege to create a DMF.
New table function: DATA_QUALITY_MONITORING_RESULTS
Access control for the new table function.
Support added for new kinds of tables: dynamic table, materialized view, Apache Iceberg™ table, external table, event table, temporary table, and transient table.
Number of DMF associations increased to 10,000 per account.
System DMFs for statistics, which was announced in June.
For more information, see Introduction to Data Quality and data metric functions.
Data loading/unloading updates¶
Snowpipe: New output in SYSTEM$PIPE_STATUS¶
With this release, the output of the PIPE_STATUS system function includes a new field, syncHistoryRemainingEntries
. When a pipe fails over, load history entries might continue to be replicated for the pipe, ensuring that changes from the last refresh operation are up to date. This new field can help you monitor the progress of load history replication for a pipe.
For more information, see SYSTEM$PIPE_STATUS.
Data pipelines updates¶
Dynamic tables: Support for incremental lateral flatten¶
With this release, you can now use lateral flatten with incremental refresh by setting the refresh mode to INCREMENTAL. Selecting the flatten SEQ column from a lateral flatten join is not supported for incremental refresh.
For more information, see Supported queries in incremental refresh.
Data lake updates¶
Apache Iceberg™ tables: Support for Polaris Catalog — Preview¶
With this release, Snowflake is pleased to announce the preview of support for integrating Apache Iceberg™ tables in Snowflake with Polaris Catalog.
Using a catalog integration configure for Polaris Catalog, you can do the following:
Query a table in Polaris Catalog using Snowflake.
Sync a Snowflake-managed Iceberg table with Polaris Catalog.
For more information, see Use Apache Iceberg™ tables with Polaris Catalog in Snowflake.
Release notes change log¶
Announcement |
Update |
Date |
---|---|---|
Release notes |
Initial publication (preview) |
27-Jul-24 |
Snowpipe: New output in SYSTEM$PIPE_STATUS |
Added to Data loading / unloading updates section |
30-Jul-24 |
Iceberg tables: Support for Polaris Catalog |
Added to Data lake updates section |
31-Jul-24 |