Replication: Stages, pipes, storage integrations, and load history¶

Attention

This behavior change is in the 2024_02 bundle.

For the current status of the bundle, refer to Bundle History.

Note

This behavior change was part of the 2024_01 bundle, however it has been moved to the 2024_02 bundle.
Enabling the 2024_02 bundle might result in dropped objects in the target account.

If you have secondary databases in target accounts and you have manually created stages or pipes in those target accounts, enabling this bundle might result in those objects being dropped. Once dropped, those objects are not recoverable by disabling the BCR bundle. Please review the full text of this behavior change before enabling the bundle.

Snowflake accounts can be replicated across regions and cloud platforms. Supported database objects are replicated to target accounts when a database is replicated.

The replication of internal and external stage objects, pipe objects, storage integrations, and table load history is available in preview. This change makes replication of stages, pipes, storage integrations, and load history generally available when this BCR bundle becomes enabled by default.

Before the change:

Primary stage objects, pipe objects, storage integrations, and table load history are not replicated to target accounts (unless you have enabled the preview feature). Any existing stages and pipes in a target account are not modified during a refresh operation.

If you are participating in the preview for storage integrations replication, and you include storage integrations in a replication or failover group by including integrations in the group’s object_types list and include storage integrations in the allowed_integration_types list, then any existing manually created storage integrations in the target account are dropped.

If you are not participating in the preview for storage integrations replication (that is, you are not replicating storage integrations in a replication or failover group), existing storage integrations in a target account are not modified during a refresh operation.

After the change:

Primary stage objects, pipe objects, and table load history are replicated to target accounts when the database that contains them is replicated in a replication or failover group. Primary storage integrations are replicated to target accounts if they are included in the replication or failover group. To replicate storage integrations, the object_types parameter must include INTEGRATIONS and the allowed_integrations parameter must include STORAGE INTEGRATIONS for the group.

If a target account has secondary databases with manually created internal or external stages, or pipes, these manually created objects are dropped when the replication or failover group is refreshed after this feature is enabled. Similarly, if the primary replication or failover group includes storage integrations, these manually created storage integrations are dropped in the target account during the refresh operation.

If the primary database has an internal stage with directory table enabled, the files on the stage are also replicated. If there are files on the stage that are larger than 5GB, the refresh operation for the replication or failover group fails. To work around this limitation, move any files larger than 5GB to another stage. For more information, see Considerations.

Stage, pipe, and load history replication is supported for databases that are replicated in replication or failover groups. This feature is not supported for database replication.

For more information, see Stage, pipe, and load history replication.

As part of pipe object replication, two new execution states, FAILING_OVER and READ_ONLY are added to SYSTEM$PIPE_STATUS and is generally enabled, not configurable by this BCR bundle.

Ref: 1461