Just-in-time refresh for external catalog metadata (private preview)¶

Just-in-time (JIT) refresh is a private preview enhancement to automatic refresh for supported externally managed Apache Iceberg™ tables.

Snowflake supports multiple catalog integration types for Iceberg tables. In this private preview, JIT refresh applies to tables that use any of the following:

• Iceberg REST catalog integration — Snowflake connects to a remote catalog through the Apache Iceberg REST OpenAPI specification. Examples include Unity Catalog, Tabular, Apache Polaris™ REST endpoints, and AWS Glue Iceberg REST.
• Open Catalog catalog integration — Snowflake connects to Snowflake Open Catalog, which also uses a REST interface.
• Native AWS Glue catalog integration — Snowflake connects to the AWS Glue Data Catalog directly.

JIT refresh is not supported in this preview for Snowflake-managed Iceberg, object-storage catalog integrations, or other ineligible types. Your account team confirms which objects in your environment qualify.

When JIT is enabled for an account, writes to eligible tables always refresh metadata at query time so DML can proceed against current catalog state. The rest of this topic is about reads and the ICEBERG_METADATA_CACHE_SECONDS parameter, which controls how often read paths may skip a query-time refresh when background metadata is already fresh enough.

This topic is shared after your account team enables phase 1 JIT refresh on your account. Parameter names, defaults, and supported workloads can change during the preview. Contact your account team to report feedback or escalate issues.

Overview¶

Externally managed Iceberg tables rely on auto-refresh: Snowflake periodically calls the catalog getTable() for each table and refreshes when a new metadata.json appears. Background polling cadence is shaped in part by REFRESH_INTERVAL_SECONDS on the catalog integration (default 30 seconds).

That model trades freshness against catalog load:

• A short interval improves freshness but increases getTable() traffic—often painful at large catalog-linked scale.
• A long interval reduces catalog load but increases lag between external commits and what Snowflake queries return.

Intervals are the same for every table even when change rates differ. For reads, ICEBERG_METADATA_CACHE_SECONDS bounds how stale catalog metadata may be before a query triggers a JIT refresh. Background auto-refresh still runs so metadata is often already fresh at query time, which reduces how often read paths must refresh during the query and helps avoid extra query latency.

Common motivations for the preview include:

• Reducing unpredictable latency after external writers commit new snapshots.
• Tuning freshness and catalog load for your traffic pattern (a small TTL with very frequent reads can increase catalog calls and query-time refresh work).
• Making read freshness more predictable for dashboards and pipelines.

JIT refresh does not replace auto-refresh. If background refresh is stuck because of catalog or refresh-plan issues, query-time refresh can fail for the same underlying reasons—and failed JIT refresh on the read path can block queries, whereas stuck auto-refresh alone may still allow reads against the last refreshed metadata.

How JIT fits with automatic refresh¶

Snowflake uses both background auto-refresh and read-side JIT in phase 1: background polling limits how often queries pay the cost of a full metadata refresh at execution time, while JIT enforces your freshness bound when cached metadata is too old.

Private preview phases¶

Which topic to use

• This topic — phase 1 only: query-time read freshness with interval-based background polling.
• Comprehensive phase 2 topic — phase 2: JIT refresh and intelligent polling together. Your account team shares it when intelligent polling is enabled on your account.

ICEBERG_METADATA_CACHE_SECONDS¶

This parameter sets, in seconds, how long Snowflake may treat Iceberg catalog metadata as fresh on the read path before a SELECT (or other supported read) triggers a JIT refresh for an eligible table.

Allowed values: -1, or integers 0 through 86400 (one day), inclusive. -1 is the only negative value allowed.

Default: 300 seconds at table lineage when no value is inherited from a higher scope (JIT refresh is on with that TTL). Background auto-refresh still follows phase rules for REFRESH_INTERVAL_SECONDS.

Where to set: CREATE / ALTER ICEBERG TABLE, CREATE / ALTER SCHEMA, CREATE / ALTER DATABASE, or ALTER ACCOUNT.

Precedence (most specific wins): table → schema → database → account.

Defaults set at schema, database, or account level apply only to supported tables (Iceberg REST catalog integration, Open Catalog catalog integration, or native AWS Glue catalog integration) in that scope. Setting the parameter on a non-eligible table fails.

Semantics (reads only)

• -1 — JIT refresh is off for that table. Background auto-refresh still applies per phase rules for REFRESH_INTERVAL_SECONDS.
• 0 — on reads, always refresh metadata at query time if the table supports JIT refresh.
• Greater than 0 — on reads, bound how stale catalog-backed metadata may be before Snowflake refreshes it at query time.

Examples

Table-level:

ALTER ICEBERG TABLE my_ext_table SET ICEBERG_METADATA_CACHE_SECONDS = 5;

Schema default:

ALTER SCHEMA my_schema SET ICEBERG_METADATA_CACHE_SECONDS = 10;

Database-level:

ALTER DATABASE my_linked_db SET ICEBERG_METADATA_CACHE_SECONDS = 0;

Turn JIT refresh off for a table:

ALTER ICEBERG TABLE my_ext_table SET ICEBERG_METADATA_CACHE_SECONDS = -1;

Account default:

ALTER ACCOUNT SET ICEBERG_METADATA_CACHE_SECONDS = 300;

Use the same parameter name on CREATE ICEBERG TABLE, CREATE SCHEMA, CREATE DATABASE, and account-level configuration where supported.

Statement types during the preview¶

ICEBERG_METADATA_CACHE_SECONDS affects reads (primarily SELECT, including joins, CTEs, and unions). During private preview, which read shapes are supported may still vary by account; confirm with your account team before you rely on JIT for a given query pattern.

REFRESH_INTERVAL_SECONDS (catalog integration)¶

REFRESH_INTERVAL_SECONDS on the catalog integration controls interval-based background polling (default 30 seconds when not specified). Read-side JIT at query time is separate and is off when the effective ICEBERG_METADATA_CACHE_SECONDS for a table is -1.

Testing guidance¶

The ideas below are optional. They are not a required test plan or reporting checklist—use what fits your workload, and reach out to your account team only when you want help or want to share feedback.

When you compare settings, changing one variable at a time (ICEBERG_METADATA_CACHE_SECONDS, REFRESH_INTERVAL_SECONDS, workload, or table count) usually makes results easier to interpret. If something looks wrong, a query_id and a short note of how the table was configured are often enough for follow-up.

Phase 1 — JIT with interval background polling¶

Background polling still honors REFRESH_INTERVAL_SECONDS. Read-side JIT follows ICEBERG_METADATA_CACHE_SECONDS.

Scenarios others have found helpful:

• External write, then read — Try a few TTL values (0, a small positive number, or defaults at account, database, schema, or table level) after a change made outside Snowflake.
• Multi-table reads — Joins, unions, or CTEs over JIT-enabled tables, especially when only one table changed upstream.

To separate background polling from read-side JIT, you can hold TTL fixed and vary REFRESH_INTERVAL_SECONDS, or the reverse.

You may run into catalog throttling, metadata churn after external DDL, or heavy concurrent reads after a bulk commit—worth mentioning if you contact your account team.

Phase 2 — JIT with intelligent polling¶

Use the comprehensive phase 2 topic for JIT refresh, intelligent polling, observability, and testing guidance in one place.

Limitations and expectations¶

• This LIMITEDACCESS topic is not in public navigation. Use the direct link your account team provides.
• In this preview, JIT refresh supports Iceberg REST catalog integrations, Open Catalog catalog integrations, and native AWS Glue catalog integrations. It does not support Snowflake-managed Iceberg, object-storage catalog integrations, or other ineligible types unless your account team states otherwise.
• Not every table type or SQL shape is supported. Your account team confirms what applies to your account.
• Preview features are not committed to a specific region, cloud, edition, or GA date.
• Prefer non-production testing unless your organization accepts preview risk.