Snowflake DCM Projects¶

Key terms¶

The following are the key terms you should know when working with DCM Projects.

Declarative definitions

In DCM Projects, you define the desired state of your Snowflake environment, such as which tables, schemas, or roles should exist, independent of the current state of the objects. You do not specify each step to create or modify them. You describe what you want, and Snowflake figures out how to make it happen.

Specifically, DCM Projects leverage DEFINE statements with templating features. This makes project files reusable and customizable for different environments. The order and location of DEFINE statements within a project don’t affect the results. Snowflake collects and sorts all statements before applying changes, so you don’t need to manually handle sequencing or dependencies.

DCM project files

A DCM project is based on a set of SQL and YAML source files, usually managed in a Git repository or your local workspace. You define Snowflake objects, their attributes, relationships, and constraints for a DCM project in your project definition files (SQL files). You update your project files in a development workspace. Changes only take effect in Snowflake after you deploy them through a DCM project object.

DCM project object

A DCM project is a schema-level object in Snowflake that you use to deploy and manage the objects defined in DCM project files. You need a DCM project object for each target environment.

The DCM project object is used to execute DCM commands and stores the immutable artifacts and definition files of all executed deployments.

Though a DCM project is a schema-level object, you can use it to create and manage objects in other databases. You can also execute a DCM project to perform a dry run of changes to your workflow so that you can preview changes before deploying them.

Requirements¶

• Use Snowsight, Snowflake CLI, SQL, or Cortex CLI to manage DCM Projects.

• You need a database and a schema where you can create your DCM project objects.

• Store your DCM project definitions locally or in Snowflake Workspace.

• Use Git for collaboration, versioning, and for synchronizing changes.

• If you want to execute local definitions using Snowflake CLI, you also need privileges to create a temporary stage in the schema of your target DCM project object.

Considerations and limitations¶

• Project size

  • Currently, DCM Projects supports up to 1,000 source files and 10,000 rendered object definitions or grants.

    Beyond 1,000 files or 10,000 definitions, you can experience performance degradation and, in some cases, execution failure.

    Consolidating definitions into fewer files generally shows faster execution times for PLAN and DEPLOY commands.

    This limit will be raised during the public preview period as performance and scalability continue to improve.

• Changeset

  Both PLAN and DEPLOY commands list all DDL changes inside the plan_result.json file. The changeset lists the operations performed or planned (CREATE, ALTER, DROP) and the individual attributes affected, such as comment, schedule, and timeout.

  Important

  During the preview phase of DCM Projects, it’s not guaranteed that the changeset captures every granular change across all properties of each object.

• Templating

  • Because definition files are Jinja2 templates, all of the limitations for Jinja2 templates apply.

  • DCM templating variables are not intended for sensitive information like credentials. The rendered SQL definitions don’t redact any values inserted by environment variables.

Key use cases for DCM Projects¶

This section describes key use cases for DCM Projects and how they help address challenges that data businesses face at scale. These use cases fall into two general categories based on team’s responsibility:

• Platform teams that manage infrastructure and governance

• Feature teams that manage individual data products and pipelines

DCM Projects for managing infrastructure¶

DCM Projects help address the following challenges that platform teams often encounter:

When platform teams want to deploy and maintain standardized infrastructure for multiple business units, they can use DCM Projects to define a standard set of objects in code as SQL files. And with Jinja, this template can be parameterized, for example, by team name, and deployed multiple times.

Example: Create a dedicated DCM project for each business unit¶

One approach is to create a dedicated DCM project for each business unit, with all projects referencing the same parameterized definition files, as shown in the following definitions.sql example:

CopyExpand

DEFINE DATABASE {{team_name}}_DB;

DEFINE ROLE {{team_name}}_ADMIN;

DEFINE WAREHOUSE {{team_name}}_WH WITH
  warehouse_size = '{{wh_size}}'
  auto_suspend = 300;

GRANT OWNERSHIP ON DATABASE {{team_name}}_DB TO ROLE {{team_name}}_ADMIN;

GRANT OWNERSHIP ON WAREHOUSE {{team_name}}_WH TO ROLE {{team_name}}_ADMIN;

GRANT ROLE {{team_name}}_ADMIN TO ROLE SYSADMIN;

Show lessSee more

Scroll to top

Execute the DCM project with the following command:

CopyExpand

EXECUTE DCM PROJECT FINANCE_INFRA PLAN
  USING (team_name => 'Finance', wh_size => 'LARGE')
FROM
  ...

Show lessSee more

Scroll to top

Example: Create a single DCM project for multiple business units¶

In this approach, you manage infrastructure for multiple business units in one DCM project by using loops in your Jinja template, as shown in the following definitions.sql example:

CopyExpand

{% for team_name in teams %}

  DEFINE DATABASE {{team_name}}_DB;
  DEFINE ROLE {{team_name}}_ADMIN;
  DEFINE WAREHOUSE {{team_name}}_WH
    WITH
      warehouse_size = '{{wh_size}}'
      auto_suspend = 300;

  GRANT OWNERSHIP ON DATABASE {{team_name}}_DB TO ROLE {{team_name}}_ADMIN;
  GRANT OWNERSHIP ON WAREHOUSE {{team_name}}_WH TO ROLE {{team_name}}_ADMIN;
  GRANT ROLE {{team_name}}_ADMIN TO ROLE SYSADMIN;

{% endfor %}

Show lessSee more

Scroll to top

Execute the DCM project with the following command:

CopyExpand

EXECUTE DCM PROJECT FINANCE_INFRA PLAN
  USING (teams => ['Finance', 'HR', 'Engineering'], wh_size => 'MEDIUM')
FROM
  ...

Show lessSee more

Scroll to top

This makes it easy for platform teams and admins to make changes such as:

• Add a new team to the list to deploy the existing infrastructure template for that team.

• Remove a team from the list to drop the infrastructure of that team.

• Add a new READ_ONLY role for all teams.

• Change specific configurations such as grants or warehouse size across all teams or for a specific team.

• Run PLAN to compare the current state against expected standards and re-deploy to reinstate standards.

DCM Projects for data pipelines¶

DCM Projects help address the following challenges that feature teams often encounter:

Business units that want to easily author and manage their data pipelines can use DCM Projects to define, test, deploy, and iterate over their business logic.

You can:

• Manage Snowflake object types like tables, dynamic tables, views, warehouses, roles, grants, data metric functions, and expectations all in one project.

• Test and deploy incremental changes to pipelines. You can change configurations, implement transformation logic, and add columns and views.

• Preview data samples to validate transformation logic before deploying objects.

• Deploy the same pipeline definition to multiple environments.

• Test data quality expectations on pre-prod environments before deploying changes to production.

DCM Projects provide additional functionality for authoring and managing data pipelines. See DCM Projects for data pipelines for details.