Snowflake key concepts and architecture¶

Database storage¶

Snowflake supports the following kinds of data:

• Structured data — such as rows and columns in a table — follows a strict tabular schema.

• Semi-structured data — such as a JSON file or an XML file — has a flexible schema.

• Unstructured data — such as a document, image, or audio file — has no inherent schema.

Snowflake supports several types of tables for data storage, including the following table types:

• Snowflake tables

• Apache Iceberg™ tables

• Hybrid tables

Compute¶

A virtual warehouse is a cluster of compute resources in Snowflake. Virtual warehouses process SQL statements and, using Snowpark, run code in languages, such as Java, Python, and Scala. With Snowpark Connect for Spark, you can also run Apache Spark™ workloads on virtual warehouses.

Each virtual warehouse is an independent compute cluster that doesn’t share compute resources with other virtual warehouses. As a result, each virtual warehouse has no effect on the performance of other virtual warehouses.

For more information, see Virtual warehouses.

Cloud services¶

The cloud services layer is a collection of services that coordinate activities across Snowflake. These services tie together all of the different components of Snowflake in order to process user requests, from sign-in to query dispatch. The cloud services layer also runs on compute instances that are provisioned by Snowflake from the cloud provider.

Services managed in this layer include the following:

• Security, authentication, and access control

• Snowflake Horizon Catalog

• Infrastructure management with cloud platforms

• Metadata management, including the SNOWFLAKE database and the Snowflake Information Schema

• Query parsing and optimization

• Regulatory compliance

Data engineering¶

Snowflake separates storage and compute, which simplifies some traditional challenges of data engineering, such as infrastructure management and performance tuning. Data engineers can focus on implementing pipelines that ingest, transform, and deliver data.

Snowflake provides several ways to ingest data, including the following options:

• COPY INTO <table> command — Loads data from files to a table.

• Snowpipe — Loads data from files as soon as they are available in a stage.

• Snowpipe Streaming — Loads row-level data continuously and with low latency, using the Snowflake SDKs or a REST API, directly into Snowflake tables and Snowflake-managed Iceberg tables, instead of loading data from files.

• Openflow connectors — Ingest data from specific sources by using connectors built on Apache NiFi, such as Microsoft Sharepoint and Google Drive.

• Snowflake Connectors — Connect from external applications and systems and stream data into Snowflake.

Snowflake also provides several ways to transform data, including the following options:

• Dynamic tables — Define tables that automatically refresh based on target freshness and a query that performs data transformations.

• Streams and tasks — Capture changes made to base objects with streams and define tasks to perform data transformations.

• Snowpark — Perform more complex transformations by using programming languages, such as Python, Java, and Scala.

• dbt — Use an open-source data transformation tool and framework to define, test, and deploy SQL transformations.

In addition, SnowConvert AI can ingest and transform data, and Snowpark Migration Accelerator can convert code from various platforms to Snowflake.

For more information, see Overview of data loading.

Analytics¶

With Snowflake, you can scale workloads dynamically based on demand, access different types of data — including structured, semi-structured, and unstructured — and share data easily. These features let you analyze data stored in Snowflake to extract meaningful insights, patterns, and trends for analytical use cases, such as business intelligence or predictive modeling.

Snowflake provides several ways to analyze data, including the following options:

• System functions and SQL constructs — Perform calculations and statistical analysis with the following Snowflake system functions and SQL constructs:

  • Aggregate functions — Summarize data by performing calculations on a set of related rows and returning a single value.

  • Window functions — Perform calculations on a set of related rows in partitions for rolling operations on subsets of the rows in each partition, such as calculating running totals or moving averages.

  • Common table expressions (CTEs) — Improve the readability and reusability of complex queries, which might perform multiple steps of data transformation.

• Snowflake Cortex AISQL — Run unstructured analytics on text and images with large language models (LLMs) from OpenAI, Anthropic, Meta, Mistral AI, and DeepSeek.

• Semantic views — Store semantic business concepts directly in the database to define business metrics and model business entities and their relationships.

AI and ML¶

Snowflake simplifies the use of artificial intelligence (AI) and machine learning (ML) capabilities so you can perform AI and ML feature engineering, training, and inference with your Snowflake data. Models can access your most up-to-date data in a secure environment. With Snowflake, you can avoid the cost and complexity of moving your data to a separate platform for AI and ML tasks.

Snowflake offers AI and ML capabilities in two broad suites of features:

• Snowflake Cortex — AI features that use LLMs to understand unstructured data, answer freeform questions, and provide intelligent assistance. AISQL functions can automate routine tasks, such as simple summaries and quick translations.

• Snowflake ML — Features that you can use to build your own models. ML functions give you automated predictions and insights into your data by using ML. Snowflake ML is a unified environment for ML development.

For more information, see Snowflake AI and ML.

Applications and collaboration¶

Snowflake offers many ways to build applications and share them with your teams, partners, and customers. When you use Snowflake to share data, you control access to the data, and avoid the challenges of keeping it synchronized in different places.

The following list shows some of the tools and services you can use to build, deploy, and manage applications in Snowflake:

• Streamlit — Use an open-source Python library to create and share custom web apps with an interactive user interface (UI) for ML and data science.

• Snowpark Container Services — Deploy, manage, and scale containerized applications from directly inside Snowflake.

• Snowflake Native App Framework — Build applications that expand the capabilities of other Snowflake features by sharing data and related business logic with other Snowflake accounts. The business logic of an application might include a Streamlit app, stored procedures, and functions written by using Snowpark API, JavaScript, and SQL. A Snowflake Native App can also run container workloads with Snowpark Container Services.

Snowflake includes support for the following kinds of collaboration:

• Secure Data Sharing — Share selected objects in a database in your account with other Snowflake accounts.

• Listings — Provide data and other information to other Snowflake users, or access data and other information shared by Snowflake providers. You can explore, access, and provide listings to consumers privately and on the Snowflake Marketplace.

• Data Clean Rooms — Define what analyses can be run against the shared data, which allows the consumer to gather insights from the data without having unrestricted access to it.