Schema:

ACCOUNT_USAGE

SNOWPIPE_STREAMING_FILE_MIGRATION_HISTORY View

This Account Usage view can be used to query the history of data migrated into Snowflake tables using Snowpipe Streaming within the last 365 days (1 year). The view displays the number of rows and bytes migrated and credits used for migration billed for your entire Snowflake account.

Columns

Column Name

Data Type

Description

START_TIME

TIMESTAMP_LTZ

Start of the time (in the local time zone) range in which data migration took place.

END_TIME

TIMESTAMP_LTZ

End of the time (in the local time zone) range in which data migration took place.

CREDITS_USED

FLOAT

Number of credits billed for Snowpipe Streaming data migration during the START_TIME and END_TIME window.

NUM_BYTES_MIGRATED

NUMBER

Number of bytes migrated during the START_TIME and END_TIME window.

NUM_ROWS_MIGRATED

NUMBER

Number of rows migrated during the START_TIME and END_TIME window.

TABLE_ID

NUMBER

Internal/system-generated identifier for the target table that the Snowpipe Streaming client loads data into.

TABLE_NAME

TEXT

The name of the target table that the Snowpipe Streaming client loads data into.

SCHEMA_ID

NUMBER

Internal/system-generated identifier for the schema that the target table belongs to.

SCHEMA_NAME

TEXT

The name of the schema that the target table belongs to.

DATABASE_ID

NUMBER

Internal/system-generated identifier for the database that the target table belongs to.

DATABASE_NAME

TEXT

The name of the database that the target table belongs to.

Usage Notes

  • Latency for the view may be up to 12 hours. Note that file migration sometimes may be pre-empted by clustering or other DML operations. Migration may not always occur and therefore the migration history will be empty even after 12 hours.

  • The NUM_BYTES_MIGRATED and NUM_ROWS_MIGRATED columns only show the number of bytes and rows processed during the migration process. These numbers may not equal the actual numbers of rows and bytes inserted by Snowpipe Streaming to the table because some rows and bytes are processed outside of the migration process due to clustering or other DML operations.

    For example, Snowpipe Streaming inserts 1M rows and the table has 1M rows, but the NUM_ROWS_MIGRATED column in the migration history view only shows 800K rows. This is because the other 200K rows are processed outside of the migration process.

Examples

Query the history of data migrated into Snowflake tables using Snowpipe Streaming within the last 365 days.

SELECT * FROM SNOWFLAKE.ACCOUNT_USAGE.SNOWPIPE_STREAMING_FILE_MIGRATION_HISTORY;
Copy

The query returns the following results.

+-------------------------------+-------------------------------+--------------+--------------------+------------------+----------+-----------------+------------+--------------+---------------+--------------+
| START_TIME                    | END_TIME                      | CREDITS_USED | NUM_BYTES_MIGRATED | NUM_ROWS_MIGRATED| TABLE_ID |      TABLE_NAME | SCHEMA_ID  |  SCHEMA_NAME |   DATABASE_ID | DATABASE_NAME|
|-------------------------------+-------------------------------+--------------+--------------------+------------------+----------+----------------------------------------------------------------------------|
|2023-02-08 19:00:00.000 +0000  |2023-02-08 20:00:00.000 +0000  | 0.0000325    |                 0  |                0 |  16849926| STREAMING_TABLE |   101351   |   SNOW       |  3166         |STREAMING     |
|2023-02-07 19:00:00.000 +0000  |2023-02-07 20:00:00.000 +0000  | 0.000096761  |             7,850  |               39 |  16849926| STREAMING_TABLE |   101351   |   SNOW       |  3166         |STREAMING     |
+-------------------------------+-------------------------------+--------------+--------------------+------------------+----------+-----------------+------------+--------------+---------------|--------------+
Copy