Just-in-time refresh for external catalog metadata (private preview)¶

Just-in-time (JIT) refresh is a private preview enhancement to automatic refresh for supported externally managed Apache Iceberg™ tables.

Snowflake supports multiple catalog integration types for Iceberg tables. In this private preview, JIT refresh applies to tables that use either of the following:

• Iceberg REST catalog integration — Snowflake connects to a remote catalog through the Apache Iceberg REST OpenAPI specification. Examples include Unity Catalog, Tabular, Apache Polaris™ REST endpoints, and AWS Glue Iceberg REST.
• Open Catalog catalog integration — Snowflake connects to Snowflake Open Catalog, which also uses a REST interface.

JIT refresh is not supported in this preview for tables that use a native AWS Glue catalog integration (AWS Glue Data Catalog without the Iceberg REST endpoint), Snowflake-managed Iceberg, object-storage catalog integrations, or other ineligible types. Your account team confirms which objects in your environment qualify.

When JIT is enabled for an account, writes to eligible tables always refresh metadata at query time so DML can proceed against current catalog state. The rest of this topic is about reads and the ICEBERG_METADATA_CACHE_SECONDS parameter, which controls how often read paths may skip a query-time refresh when background metadata is already fresh enough.

This feature is available only to selected accounts. Contact your Snowflake account team to request enrollment, confirm eligibility, and report feedback. Parameter names, defaults, and supported workloads can change during the preview.

Overview¶

Externally managed Iceberg tables rely on auto-refresh: Snowflake periodically calls the catalog getTable() for each table and refreshes when a new metadata.json appears. Background polling cadence is shaped in part by REFRESH_INTERVAL_SECONDS on the catalog integration (default 30 seconds).

That model trades freshness against catalog load:

• A short interval improves freshness but increases getTable() traffic—often painful at large catalog-linked scale.
• A long interval reduces catalog load but increases lag between external commits and what Snowflake queries return.

Intervals are the same for every table even when change rates differ. For reads, ICEBERG_METADATA_CACHE_SECONDS bounds how stale catalog metadata may be before a query triggers a JIT refresh. Background auto-refresh still runs so metadata is often already fresh at query time, which reduces how often read paths must refresh during the query and helps avoid extra query latency.

Common motivations for the preview include:

• Reducing unpredictable latency after external writers commit new snapshots.
• Tuning freshness and catalog load for your traffic pattern (a small TTL with very frequent reads can increase catalog calls and query-time refresh work; intelligent polling in phase 2 targets better background scheduling).
• Making read freshness more predictable for dashboards and pipelines.

JIT refresh does not replace auto-refresh. If background refresh is stuck because of catalog or refresh-plan issues, query-time refresh can fail for the same underlying reasons—and failed JIT refresh on the read path can block queries, whereas stuck auto-refresh alone may still allow reads against the last refreshed metadata.

How JIT fits with automatic refresh¶

Snowflake uses both background auto-refresh and read-side JIT in phase 1: background polling limits how often queries pay the cost of a full metadata refresh at execution time, while JIT enforces your freshness bound when cached metadata is too old.

Private preview phases¶

Dates for phase 2 are targets and can change. Your account team confirms timing when your account is enabled.

Choosing a phase

• Request phase 1 if you want to validate query-time freshness first with familiar interval-based background polling.
• Request phase 2 when available if you also want to evaluate intelligent background polling, query performance, and freshness together for your catalog traffic patterns.

You can often start in phase 1 and move to phase 2 when it is offered for your account.

Request access¶

1. Contact your Snowflake account team to enroll in the private preview.
2. Share your account identifiers.
3. After enrollment, Snowflake enables JIT on your account. You configure read-side behavior with ICEBERG_METADATA_CACHE_SECONDS as described below.

If you run experiments or need help with an issue, your account team can point you to the right preview channel.

ICEBERG_METADATA_CACHE_SECONDS¶

This parameter sets, in seconds, how long Snowflake may treat Iceberg catalog metadata as fresh on the read path before a SELECT (or other supported read) triggers a JIT refresh for an eligible table.

Allowed values: -1, or integers 0 through 86400 (one day), inclusive. -1 is the only negative value allowed.

Default: 300 seconds at table lineage when no value is inherited from a higher scope (JIT refresh is on with that TTL). Background auto-refresh still follows phase rules for REFRESH_INTERVAL_SECONDS.

Where to set: CREATE / ALTER ICEBERG TABLE, CREATE / ALTER SCHEMA, CREATE / ALTER DATABASE, or ALTER ACCOUNT.

Precedence (most specific wins): table → schema → database → account.

Defaults set at schema, database, or account level apply only to supported tables (Iceberg REST catalog integration or Open Catalog catalog integration) in that scope. Setting the parameter on a non-eligible table fails.

Semantics (reads only)

• -1 — JIT refresh is off for that table (even after account enrollment). Background auto-refresh still applies per phase rules for REFRESH_INTERVAL_SECONDS.
• 0 — on reads, always refresh metadata at query time if the table supports JIT refresh.
• Greater than 0 — on reads, bound how stale catalog-backed metadata may be before Snowflake refreshes it at query time.

Examples

Table-level:

ALTER ICEBERG TABLE my_ext_table SET ICEBERG_METADATA_CACHE_SECONDS = 5;

Schema default:

ALTER SCHEMA my_schema SET ICEBERG_METADATA_CACHE_SECONDS = 10;

Database-level:

ALTER DATABASE my_linked_db SET ICEBERG_METADATA_CACHE_SECONDS = 0;

Turn JIT refresh off for a table:

ALTER ICEBERG TABLE my_ext_table SET ICEBERG_METADATA_CACHE_SECONDS = -1;

Account default:

ALTER ACCOUNT SET ICEBERG_METADATA_CACHE_SECONDS = 300;

Use the same parameter name on CREATE ICEBERG TABLE, CREATE SCHEMA, CREATE DATABASE, and account-level configuration where supported.

Statement types during the preview¶

ICEBERG_METADATA_CACHE_SECONDS affects reads (primarily SELECT, including joins, CTEs, and unions). During private preview, which read shapes are supported may still vary by account; confirm with your account team before you rely on JIT for a given query pattern.

REFRESH_INTERVAL_SECONDS (catalog integration)¶

REFRESH_INTERVAL_SECONDS on the catalog integration controls interval-based background polling (default 30 seconds when not specified). Read-side JIT at query time is separate and is off when the effective ICEBERG_METADATA_CACHE_SECONDS for a table is -1.

Intelligent polling (phase 2)¶

Intelligent polling targets the tradeoff between short intervals (more getTable() calls) and long intervals (higher lag). In phase 2, Snowflake adapts polling per table using prediction and scheduling based on historical snapshot patterns so getTable() tends to run near when tables are likely to change—without using your catalog integration’s REFRESH_INTERVAL_SECONDS.

Goals include:

• Low lag between snapshot creation in the catalog and Snowflake detecting it.
• High hit rate — a larger share of getTable() calls that return new metadata, which can reduce total catalog traffic for a given freshness target.

For cold start, tables that are new to intelligent polling are polled more often at first until Snowflake has enough history to predict change patterns; quiet tables still receive periodic background polls. Snowflake also budgets catalog rate limits across many tables.

Testing guidance¶

The ideas below are optional. They are not a required test plan or reporting checklist—use what fits your workload, and reach out to your account team only when you want help or want to share feedback.

When you compare settings, changing one variable at a time (ICEBERG_METADATA_CACHE_SECONDS, REFRESH_INTERVAL_SECONDS, workload, or table count) usually makes results easier to interpret. If something looks wrong, a query_id and a short note of how the table was configured are often enough for follow-up.

Phase 1 — JIT with interval background polling¶

Background polling still honors REFRESH_INTERVAL_SECONDS. Read-side JIT follows ICEBERG_METADATA_CACHE_SECONDS.

Scenarios others have found helpful:

• External write, then read — Try a few TTL values (0, a small positive number, or defaults at account, database, schema, or table level) after a change made outside Snowflake.
• Multi-table reads — Joins, unions, or CTEs over JIT-enabled tables, especially when only one table changed upstream.

To separate background polling from read-side JIT, you can hold TTL fixed and vary REFRESH_INTERVAL_SECONDS, or the reverse.

You may run into catalog throttling, metadata churn after external DDL, or heavy concurrent reads after a bulk commit—worth mentioning if you contact your account team.

Phase 2 — JIT with intelligent polling¶

If you move to phase 2, repeating a subset of your phase 1 checks can help spot regressions. You might also see whether REFRESH_INTERVAL_SECONDS still affects background polling, compare catalog load or lag at similar read TTLs, or try new, idle, or bursty tables. Large linked catalogs are a common focus in this phase.

If you share feedback, a short description of what you tried and what stood out (including phase 1 vs phase 2, if both apply) is usually enough; include query_id values only for cases you want investigated.

Limitations and expectations¶

• This LIMITEDACCESS topic is not in public navigation. Use the direct link your account team provides.
• In this preview, JIT refresh supports Iceberg REST catalog integrations and Open Catalog catalog integrations. It does not support native AWS Glue catalog integrations or other non-REST catalog integration types unless your account team states otherwise.
• Not every table type or SQL shape is supported. Your account team confirms what applies to your account.
• Preview features are not committed to a specific region, cloud, edition, or GA date.
• Prefer non-production testing unless your organization accepts preview risk.