Classify sensitive data automatically¶

Get started¶

The basic workflow to automatically classify sensitive data consists of the following:

1. Create a classification profile that controls how often sensitive data in a schema is automatically classified, including whether system tags should be automatically applied after classification.

2. Optionally, use the classification profile to map user-defined tags to system tags so a column with sensitive data can be associated with a user-defined tag based on its classification. You can add the tag mapping while creating the classification profile or after creating it.

3. Optionally, add a custom classifier to the classification profile so sensitive data can be automatically classified with user-defined semantic and privacy categories. You can add custom classifiers while creating the classification profile or after creating it.

4. Set the classification profile on a schema so that tables in the schema get automatically classified.

For end-to-end examples of this workflow, see Examples.

About classification profiles¶

A data engineer creates a classification profile by creating an instance of the CLASSIFICATION_PROFILE class to define the criteria that are used to automatically classify tables in a schema. This criteria includes:

• How long a table should exist before automatically classifying it.

• How long before previously classified tables should be reclassified.

• Whether system and custom tags should be set on columns after the automatic classification.

• A mapping between system classification tags and user-defined object tags so the user-defined tags can be applied automatically.

When the data engineer assigns the classification profile to a schema, sensitive data in the tables of the schema are automatically classified on the schedule defined by the profile. A data engineer can assign the same classification profile to multiple schemas, or can create multiple classification profiles if there is a need to set different classification criteria for different schemas.

The process of automatically classifying data requires access to the raw data in the table. The raw data includes tables that have a masking policy assigned to a column. However, Snowflake preserves the intention of regulating access to protected data by using an internal role to automatically classify data. The internal role can access data protected by a masking policy, but this role is not accessible to users.

For an example of using the CREATE CLASSIFICATION_PROFILE command to create a classification profile, see Examples.

About tag mapping¶

You can use the classification profile to map SEMANTIC_CATEGORY system tags to one or more object tags. This tag mapping allows a column with sensitive data to be automatically assigned a user-defined tag based on its classification. The tag map can be added while creating the classification profile or later by calling the <classification_profile_name>!SET_TAG_MAP method.

Because user-defined object tags can have a masking policy associated with them, you can use a tag map to enable automatic tag-based masking, which facilitates column protection aligned with the SEMANTIC_CATEGORY system tag.

Regardless of whether you are defining the tag map while creating the classification profile or after, the contents of the map are specified as a JSON object. This JSON object contains the 'column_tag_map' key, which is an array of objects that specify a user-defined tag, the string value of that tag, and the semantic categories to which the tag is being mapped. After the tag map is associated with a classification profile and you automatically classify tables in a schema, the tag is assigned to the columns that correspond to the semantic categories.

The following is an example of a tag map:

'tag_map': {
  'column_tag_map': [
    {
      'tag_name':'tag_db.sch.pii',
      'tag_value':'Highly Confidential',
      'semantic_categories':[
        'NAME',
        'NATIONAL_IDENTIFIER'
      ]
    },
    {
      'tag_name': 'tag_db.sch.pii',
      'tag_value':'Confidential',
      'semantic_categories': [
        'EMAIL'
      ]
    }
  ]
}

Based on this mapping, if you have a column of email addresses and the classification process determines that the column contains these addresses, the tag_db.sch.pii = 'Confidential' tag is set on the column containing the email addresses.

If your tag map includes multiple JSON objects that map tags, tag values, and category values, the order of the JSON objects determines which tag and value to set on the column if there is a conflict. Specify the JSON objects in the desired assignment order from left to right, or top to bottom if you are formatting JSON.

If there is a conflict with a manually assigned tag and a tag applied by automatic classification, an error occurs. For information about tracking these errors, see Troubleshooting.

View results of automatic classification¶

You can view the results of automatic classification in the following ways:

• Call the SYSTEM$GET_CLASSIFICATION_RESULT stored procedure. For example:

  CALL SYSTEM$GET_CLASSIFICATION_RESULT('mydb.sch.t1');
  

  Copy

• Use a role that is granted the SNOWFLAKE.GOVERNANCE_VIEWER database role to query the DATA_CLASSIFICATION_LATEST view. For example:

  SELECT * FROM snowflake.account_usage.data_classification_latest;
  

  Copy

Limitations¶

• Classification profiles cannot be set on a reader account.

• Only one classification profile can be set on a schema.

• The same classification profile cannot be set on more than 10,000 schemas.

• A maximum of 10,000 tables can be classified in a schema.

Access control¶

This section describes the privileges and roles that let you work with classification profiles and enable automatic sensitive data classification.

For an example of granting these privileges and database roles to the role of a data engineer, see Basic example: Automatically classifying tables in a schema.

Cost of automatically classifying sensitive data¶

Automatic sensitive data classification consumes credits as it uses serverless compute resources to classify tables in the schema. For more information about pricing for this consumption, see Table 5 in the Snowflake Service Consumption Table.

You can query views in the ACCOUNT_USAGE and ORGANIZATION_USAGE schemas to determine how much was spent on automatically classifying sensitive data. To monitor credit consumption, query the following views:

METERING_HISTORY view (ACCOUNT_USAGE)

Lets you retrieve the hourly cost of automatic classification by focusing on SENSITIVE_DATA_CLASSIFICATION in the SERVICE_TYPE column. For example:

SELECT
  service_type,
  start_time,
  end_time,
  entity_id,
  name,
  credits_used_compute,
  credits_used_cloud_services,
  credits_used,
  budget_id
  FROM snowflake.account_usage.metering_history
  WHERE service_type = 'SENSITIVE_DATA_CLASSIFICATION';

Copy

METERING_DAILY_HISTORY view (ACCOUNT_USAGE and ORGANIZATION_USAGE)

Lets you retrieve the daily cost of automatic classification by focusing on SENSITIVE_DATA_CLASSIFICATION in the SERVICE_TYPE column. For example:

SELECT
  service_type,
  usage_date,
  credits_used_compute,
  credits_used_cloud_services,
  credits_used
  FROM snowflake.account_usage.metering_daily_history
  WHERE service_type = 'SENSITIVE_DATA_CLASSIFICATION';

Copy

USAGE_IN_CURRENCY_DAILY (ORGANIZATION_USAGE)

Lets you retrieve the daily cost of automatic classification by focusing on SENSITIVE_DATA_CLASSIFICATION in the SERVICE_TYPE column. Use this view to determine the cost in currency, not credits.

Examples¶

• Basic example: Automatically classifying tables in a schema

• Example: Using a tag map and custom classifiers

• Example: Testing a classification profile before enabling automatic classification

Troubleshooting¶

Automatic classification errors are persisted in the default event table of the account. You can use the following query to access the error messages:

SELECT
  record_type,
  record:severity_text::string log_level,
  parse_json(value) error_message
  FROM log_db.log_schema.log_table
  WHERE record_type='LOG' and scope:name ='snow.automatic_sensitive_data_classification'
  ORDER BY log_level;