Data Loading / Unloading DDL

Stages and file formats are named database objects that can be used to simplify and streamline bulk loading data into and unloading data out of database tables.

Pipes are named database objects that define COPY statements for loading micro-batches of data using Snowpipe.

Stage Management

Snowflake supports two types of stages for storing data files used for loading/unloading:

  • Internal stages store the files internally within Snowflake.

  • External stages store the files in an external location (i.e. S3 bucket) that is referenced by the stage. An external stage specifies location and credential information, if required, for the S3 bucket.

Both external and internal stages can include file format and copy options.

File Format Management

A file format encapsulates information, such as file type (CSV, JSON, etc.) and formatting options specific to each type, for data files used for bulk loading/unloading.

Git Repository Management

A Git repository stage represents a local Git repository in Snowflake.

Pipe Management

A pipe encapsulates a single COPY statement for loading a set of data files from an ingestion queue into a table.

Language: English