With Snowpipe’s serverless compute model, users can initiate any size load without managing a virtual warehouse. Instead, Snowflake provides and manages the compute resources, automatically growing or shrinking capacity based on the current Snowpipe load. Accounts are charged based on their actual compute resource usage; in contrast with customer-managed virtual warehouses, which consume credits when active, and may sit idle or be overutilized.
In this Topic:
Resource Consumption and Management Overhead¶
Snowflake tracks the resource consumption of loads for all pipes in an account, with per-second/per-core granularity, as Snowpipe actively queues and processes data files. Per-core refers to the physical CPU cores in a compute server. The utilization recorded is then translated into familiar Snowflake credits, which are listed on the bill for your account.
Using a multi-threaded client application enables submitting data files in parallel, which initiates additional servers and loads the data in less time. However, the actual overall compute time required would be identical to using a single-threaded client application, just spread out over more internal Snowpipe servers.
Decisions with regard to data file size and staging frequency impact the cost and performance of Snowpipe. For recommended best practices, see Continuous Data Loads (i.e. Snowpipe) and File Sizing.
In addition to resource consumption, an overhead is included in the utilization costs charged for Snowpipe. This overhead is charged regardless if the event notifications or REST API calls resulted in data loaded. This overhead charge appears as Snowpipe charges in your billing statement.
For more information, see the “Serverless Feature Credit Table” in the Snowflake service consumption table.
Estimating Snowpipe Charges¶
Given the number of factors that can differentiate Snowpipe loads, it is very difficult for Snowflake to provide sample costs. File formats and sizes, and the complexity of COPY statements (including SELECT statement used for transformations), all impact the resource consumption and file overhead charged for a Snowpipe load.
We suggest that you experiment by performing a typical set of loads to estimate future charges.
Viewing the Data Load History for Your Account¶
Account administrators (users with the ACCOUNTADMIN role) or users with a role granted the MONITOR USAGE global privilege can use Snowsight, the classic web interface, or SQL to view the credits billed to your Snowflake account within a specified date range.
To view the credits billed for Snowpipe data loading for your account:
Select Admin » Usage
- Classic Web Interface
Query either of the following:
Resource monitors provide control over virtual warehouse credit usage; however, you cannot use them to control credit usage for the Snowflake-provided warehouses, including the SNOWPIPE warehouse.