Automating Continuous Data Loading Using Cloud Messaging

Automated data loads leverage event notifications for cloud storage to inform Snowpipe of the arrival of new data files to load. Snowpipe copies the files into a queue, from which they are loaded into the target table in a continuous, serverless fashion based on parameters defined in a specified pipe object.

Note

Automated Snowpipe uses event notifications to determine when new files arrive in monitored cloud storage and are ready to load. Notifications identify the cloud storage event and include a list of the file names. They do not include the actual data in the files.

The following table indicates which cloud storage services are supported for automatically loading data into your Snowflake account using cloud storage event notifications, based on the cloud platform that hosts your account:

Snowflake Account Host

Amazon S3

Google Cloud Storage

Microsoft Azure Blob storage

Microsoft Data Lake Storage Gen2

Microsoft Azure General-purpose v2

Amazon Web Services

Google Cloud Platform

Microsoft Azure

Next Topics: