October 07-09, 2024 — 8.38 Release Notes (with behavior changes)¶

New SQL functions¶

The following function(s) are now available with this release:

Query objects larger than 16 MB in files on a stage¶

With this release, you can now query objects up to 128 MB in files on a stage. Although you still can’t store objects larger than 16 MB in a column, you can reduce their size before storing them in columns. For example, you can split large objects across multiple columns or rows, transform nested JSON into a tabular format, or simplify complex geometries.

For more information, see Reducing the size of objects larger than 16 MB before loading.

Dynamic tables: Updates to input types¶

Before this release, if a dynamic table read from a regular table that was dropped and replaced by a new dynamic table with the same name, the refresh would fail, requiring the dynamic table to be recreated. Now, the refresh will succeed without user intervention.

The same applies if a dynamic table reads from another dynamic table, and that dynamic table is replaced by a table. In both cases, the refresh now succeeds.

This change makes table/view/dynamic table input types interchangeable, but it only applies to dynamic table created after October 10, 2024.

Data quality: New SYSTEM$DATA_METRIC_SCAN function¶

With this release, we are pleased to announce a new system function, SYSTEM$DATA_METRIC_SCAN, that returns records that failed a data quality check. Previously, you could use data metric functions to identify data quality issues but did not have the ability to pinpoint the records responsible for the problems.

With this new capability, you can use a single command to extract records that failed data quality checks. This simplifies the identification of problematic records in your data and makes it easier to remediate data quality issues.

For more information, see SYSTEM$DATA_METRIC_SCAN.