Manage and monitor mirrors¶
Use the procedures in the snowflake.postgres schema to alter, drop, inspect, and monitor mirrors after they’re created. These procedures take a mirror name but not an instance name;
mirror names are unique across all Postgres instances in the account, and the instance association is stored when the mirror is created.
To see all mirrors visible to the current role, or to find which instance a mirror belongs to, query list_mirrors without arguments:
Roles and permissions¶
Managing mirrors (create, alter, drop, inspect, monitor) requires the postgres_mirror_admin
application role on snowflake.postgres. ACCOUNTADMIN grants it to the role that manages
mirrors:
The caller of every mirror management procedure must also own the source PostgreSQL instance.
The procedures verify this before doing any privileged work, so a user who holds
postgres_mirror_admin but doesn’t own the source instance can’t manage its mirrors.
For the role that governs reading mirrored data, see Query mirrored data.
Alter a mirror¶
Use alter_mirror to change the refresh interval, add or remove tables and schemas, or change
the warehouse that runs the apply task:
warehouse converts a serverless apply task to a user-managed warehouse. Pass warehouse => NULL
to convert back to serverless. Omit the warehouse parameter to leave compute unchanged: NULL
selects serverless compute; it does not mean “leave this alone.” Grant
USAGE ON WAREHOUSE ... TO APPLICATION snowflake before pointing the task at a warehouse.
Changing refresh_interval or warehouse briefly suspends the apply task to reconfigure it, then
restores its prior state. A mirror that was suspended stays suspended; use resume_mirror to start
it again. alter_mirror does not resume a mirror that was stopped on purpose.
Renaming a schema or table on PostgreSQL doesn’t require reconfiguring the mirror. The rename is captured by the source extension and automatically applied to the target database.
Drop a mirror¶
drop_mirror suspends the apply task and waits for any in-flight run to finish before removing the
mirror. It then removes all associated infrastructure: the CDC publication on the source Postgres
instance, the apply task, all mirrored target tables, the <table>$changes Iceberg change feeds,
and the <MIRROR_NAME>_CATALOG_INTEGRATION catalog integration that was auto-created when the
mirror was set up.
By default the target database is dropped with the mirror. Pass drop_target_database => FALSE
to keep it. A kept database can’t be reassigned to a new mirror (see
Target database names can’t be reused). It is owned by
the snowflake application, so it can’t be dropped or modified directly. To drop it later,
ACCOUNTADMIN can transfer ownership first:
Caution
Dropping the Snowflake target database while a mirror is still active orphans the mirror. The
apply task continues to run but fails on every attempt because its target no longer exists.
To recover, drop the orphaned mirror with drop_mirror, then create a new mirror with a fresh
target_database name.
Drop is blocked by user-created Iceberg tables¶
If another role has created Iceberg tables in a different database that reference the mirror’s
<MIRROR_NAME>_CATALOG_INTEGRATION, drop_mirror fails:
The “not accessible by the current role” wording can be misleading: the blocking tables may be in a database outside your current role’s privileges. To find them, run:
Drop all tables that reference the catalog integration, then retry drop_mirror.
Tip
To keep teardown clean, query a mirror’s Iceberg change feed via a separate catalog integration
that you own rather than the mirror-managed <MIRROR_NAME>_CATALOG_INTEGRATION. That way, dropping
the mirror doesn’t affect your tables.
Teardown order when dropping an instance¶
Each mirror’s <MIRROR_NAME>_CATALOG_INTEGRATION references the Postgres instance, so all mirrors must
be dropped before dropping the instance. Attempting to drop the instance first fails:
Drop in this order:
If you kept a target database with drop_target_database => FALSE, transfer ownership and drop it as shown in Drop a mirror.
Automatic cleanup of orphaned mirrors¶
If a Postgres instance is removed while mirrors still reference it, those mirrors are detected and removed automatically. The target database is not removed automatically; drop it manually when it’s no longer needed.
Inspect mirrors¶
Use describe_mirror to check the configuration and runtime state of a specific mirror
(error_message contains the error from the most recent apply run, or NULL if the mirror is
healthy):
Use list_mirrors to see mirrors on a specific Postgres instance, or call it without arguments
to list all mirrors visible to the current role:
Use list_mirrored_tables to see the replication state of each table in a mirror:
state is the coarse target-side indicator:
SNAPSHOTTING: Snowflake is copying the current contents of the Postgres table into Snowflake for the first time.REPLICATING: The initial sync is complete and the mirror is continuously applying new changes from Postgres to Snowflake. This is the normal healthy state.
The same result also includes source-side snapshot detail (snapshot_state, snapshot_percentage,
snapshot_rows_processed, snapshot_total_rows, snapshot_reason, pending_since,
last_snapshot_error, last_snapshot_error_at). Those columns are NULL if the source is
unreachable. For a cheap fleet-wide view of onboarding progress, use
list_mirrors.initial_snapshot_tables_remaining instead of looping list_mirrored_tables.
For full result column details, see Mirror management procedures.
Monitor mirroring¶
The describe_mirror procedure surfaces a rolling window of recent apply runs in the
recent_query_durations column for a quick look at cadence and latency. Each apply run displays
a state of SUCCEEDED, FAILED, or CANCELLED. Apply lag is in apply_lag_seconds,
apply_lag_operations, and apply_lag_bytes; flush lag is in postgres_status.
For a full history, ACCOUNTADMIN can query Snowflake’s task history directly:
The query_admin_log procedure exposes mirror-internal events with optional filters:
If an apply task run fails, its error message is persisted on the mirror and surfaced by
describe_mirror and list_mirrors. The next successful run clears the error.
Source-side snapshot progress¶
On the Postgres instance, snowflake_cdc.snapshot_progress shows each initial copy that is
running right now (percentage, rows processed, estimated total). An empty result is normal when
no copy is in flight.
snowflake_cdc.snapshot_status lists every tracked table’s snapshot state (PENDING, SNAPSHOTTING,
ERROR, or REPLICATING), why it is snapshotting, and the last snapshot error. This is the catalog view
that list_mirrored_tables surfaces from Snowflake.
Alert on mirror health¶
describe_mirror and list_mirrors support TABLE(...) syntax, so a
Snowflake alert can watch them and notify you
when a mirror stops replicating. The same queries work from a Snowflake task or dashboard.
Set up a mirror health alert¶
Grant the required privileges¶
The role that owns the alert must hold the postgres_mirror_admin application role and own the
source Postgres instance. The procedures only see mirrors on instances the caller owns:
list_mirrors silently returns nothing for other instances.
Create a notification integration¶
Recipients must be verified email addresses of users in the account:
Create the alert¶
The condition fires when either of the following is true:
status = 'SUSPENDED': the apply task was suspended bysuspend_mirroror auto-suspended after repeated failures.error_message IS NOT NULLand nothing has been processed for over an hour. Checkinglast_operation_timeavoids firing for a transient error such as a lock timeout.
Activate and verify the alert¶
Alerts are created suspended. Resume it, then run it once — conditions are validated at run time,
not at CREATE ALERT:
Check what that run did:
Watch all mirrors¶
To watch every mirror instead of one, swap the FROM clause for
TABLE(snowflake.postgres.list_mirrors()) and add mirror_name to the SELECT. Calling
list_mirrors() with no argument covers every mirror on every instance you own, including
mirrors created after the alert. The predicates are unchanged:
Note
Don’t pass NULL to list_mirrors — an untyped NULL doesn’t resolve against the STRING
parameter and errors out.
Tune the alert¶
- Match the staleness threshold to
refresh_interval. Both procedures returnrefresh_interval, so you can derive the bound from it rather than hardcoding one value across mirrors with different cadences. A few multiples of the configured interval is a reasonable starting point. - Exclude mirrors you suspend deliberately, to avoid being alerted by your own maintenance.
- Don’t schedule the alert below
min(refresh_interval, 2 minutes). That’s how often the apply task runs, so below that threshold the alert re-reads the same state on every evaluation.
What the alert doesn’t check¶
- A non-
ACTIVEstatus isn’t treated as a failure. A failing mirror usually staysACTIVEwhile the apply task retries, sostatusalone isn’t a reliable failure signal. See Mirror auto-suspend after repeated failures. STARTINGstatus. A new mirror is legitimatelySTARTINGuntil its first operation is applied, so a bareSTARTINGtest fires on everycreate_mirror.- A missing apply task.
statusisNULLwhendescribe_mirrorcan’t find the mirror’s apply task, and= 'SUSPENDED'doesn’t matchNULL.create_mirroralways creates the task, so addstatus IS NULL ORto theWHEREclause only if you’ve seen one go missing. - An initial copy that never finished.
STARTINGmeans the mirror has processed nothing yet, but neither procedure returns a creation timestamp. Record when the mirror first appeared and alert once it’s beenSTARTINGlonger than a full copy should take.list_mirrored_tablesshows which table is holding up a slow initial copy. - A stall with no error. If the source stops delivering changes,
statusstaysACTIVEanderror_messagestaysNULL. Monitorlast_operation_timeand the lag figures inpostgres_status. See Monitor the source side. recent_query_durationshas a short window. Its 20-slot history fills at the 2-minute task cadence, covering roughly 40 minutes. Count runs wherestate LIKE 'FAILED%'rather than looking for a streak — backoff-skipped runs are recorded as successes and break streaks up.
Suppress repeat notifications¶
The alert fires on every evaluation where the condition matches, so a mirror broken for a day sends 144 identical emails. To suppress repeats, record what you’ve already notified about and exclude it from the condition.
Create a table to track notifications:
Add a NOT EXISTS check to the condition’s WHERE clause, keeping the existing predicates in
a group:
On the list_mirrors variant, alias the procedure call and qualify both sides of the
comparison:
The action then has to both notify and record, so it becomes a procedure.
GET_CONDITION_QUERY_UUID() gives it access to the row the condition matched:
Nothing clears those rows once a mirror recovers, so a second alert or a task has to remove them. When writing the clearing logic:
- Wait for more than one healthy observation before clearing. A single evaluation that reads the mirror as healthy mid-recovery would clear the suppression state and allow the next evaluation to page for an outage that’s already being worked.
- Anchor on
last_operation_time, not a trailing window of task runs. Whether the oldest run in a fixed lookback is older than some threshold depends on where the task’s schedule lands and flips between evaluations.
Monitor the source side¶
The columns described on this page report the Snowflake side: the apply task and what it has processed. A source-side problem shows up there only once it starves apply of work, which looks the same as an idle source.
describe_mirror also returns postgres_status, read live from the source, so a condition can
test the source side without a session on the Postgres instance. slot_lag_bytes grows when the
CDC worker falls behind. It’s a subset of the snowflake_cdc.publication_health view, which has
the rest: worker PID, error detail, and per-table snapshot state.
For alert debugging, see Troubleshoot mirrors.
Alert on mirror lag¶
Health alerts catch a suspended or errored mirror. Lag alerts catch a mirror that is still
ACTIVE but falling behind. Replication has two hops: Postgres → Iceberg (the CDC worker)
and Iceberg → target tables (the apply task). describe_mirror reports them separately:
postgres_status:cdc_flush_lag_secondsandpostgres_status:slot_lag_bytesfor the flush hopapply_lag_seconds,apply_lag_operations, andapply_lag_bytesfor the apply hop
Add the time terms for end-to-end lag, and the byte terms for end-to-end WAL volume. A NULL on
the source-side terms or on apply_lag_operations / apply_lag_bytes means “not known,” never
“caught up.”
Alert on one hop at a time when you want a different owner for each failure:
Alert on backlog volume instead of (or in addition to) freshness:
Use last_applied_commit_lsn and last_applied_commit_time when you need a read-your-writes
check rather than a lag threshold.
Suspend and resume mirroring¶
You have two options when suspending a Postgres instance that has active mirrors: leave the mirror
running, or explicitly suspend it first. The two approaches have different effects on replication
continuity and the $changes feed.
Suspend the instance without pausing the mirror¶
If you suspend the Postgres instance without calling SUSPEND_MIRROR, the apply task keeps
running and mirrored tables continue to show a status of REPLICATING. Once all buffered changes
that were already recorded in cloud storage are merged, no further data is applied until the instance
resumes. No action is needed on the mirror side.
When you resume the instance, mirroring picks up from where it left off. No re-snapshot is
performed, and the $changes feed retains the full incremental history of inserts, updates,
and deletes.
Suspend the mirror before suspending the instance¶
To stop apply task activity entirely while the instance is suspended, call SUSPEND_MIRROR
before suspending:
After resuming the Postgres instance, resume the mirror in place:
If resume_mirror raises RESTART_REQUIRED, the unapplied operations have aged out. Use
restart_mirror instead (see Mirror auto-suspend after repeated failures).
Caution
RESTART_MIRROR performs a full re-snapshot of all mirrored tables. The $changes 7-day feed
is rebuilt from a fresh snapshot baseline: every row appears with _CHANGE_TYPE = 'S' (snapshot),
and incremental change history is lost. Change-feed consumers that rely on the $changes feed
lose continuity across a RESTART_MIRROR call.
By default, any queued-but-unapplied operations in the source-side metalog are preserved
(soft restart). Pass truncate_feed => TRUE to truncate the metalog first, discarding those
queued operations before rebuilding (hard restart). Use a hard restart only for recovery
scenarios where the apply state has diverged significantly from the source.
$changes continuity is not guaranteed in general. Schema changes on mirrored tables also
rebuild the feed.
Mirror auto-suspend after repeated failures¶
Snowflake automatically suspends a mirror’s apply task if it has been failing continuously for approximately 96 hours. The 96-hour window is designed to surface a silently failing mirror before unapplied change records age out on the source side (which happens at 7 days), and covers a Friday-to-Monday outage with time left for investigation.
When a mirror is auto-suspended, describe_mirror shows the mirror as suspended and includes the
error from the most recent failed apply run.
Once you’ve resolved the underlying problem, resume in place. resume_mirror checks that the
unapplied operations are still on the source before it starts the apply task. If those operations
have aged out of the metalog, it raises RESTART_REQUIRED and leaves the mirror suspended:
If resume_mirror succeeds, the mirror continues from the operation it stopped at. If it returns
RESTART_REQUIRED, re-snapshot instead:
Troubleshooting¶
For solutions to common issues, see Troubleshoot mirrors.
