Validating Data from Amazon Redshift¶

This page covers Amazon Redshift-specific setup for Data validation. For workflow field definitions, see Data validation configuration reference.

Prerequisites¶

• ODBC connectivity to Redshift from Worker hosts (standard or IAM auth; same TOML as Migrating Data from Amazon Redshift).
• Live Redshift access for L1 and L2. Schema and metrics validation always run SQL against live Redshift.
• L3 object storage (Recommended): Use UNLOAD (extraction.strategy: unload) so signatures land on Amazon S3 with the same Worker unload_* TOML and Snowflake externalStage as migration. See L3 extraction and Amazon S3.

Connectivity¶

Reuse the same [connections.source.redshift] Worker TOML as Migrating Data from Amazon Redshift.

Standard authentication example:

[connections.source.redshift]
username = "myuser"
password = "mypassword"
database = "mydatabase"
host = "my-cluster.abcdef123456.us-west-2.redshift.amazonaws.com"
port = 5439
auth_method = "standard"

Validation behavior¶

• After UNLOAD migrations: Schema and metrics validation still run SQL against live Redshift. For L3, reuse UNLOAD with the same Worker unload_* TOML and Snowflake externalStage as migration. Grant the Redshift cluster IAM role the Amazon S3 writer actions. Ensure the Worker can reach the cluster and that large partition result sets stay within timeout and spool limits.
• Iceberg targets: For spec v2 tables, set tableType: iceberg and icebergConfig.icebergSpec: v2 on the validation workflow so L1 expects VARCHAR for both TIMESTAMP and TIMESTAMPTZ, and L3 compares those columns as the stored UTC ISO text. Metrics validation runs on Iceberg tables and leaves out columns stored as VARCHAR that no metric can read as the source type. See Apache Iceberg™ tables (spec v2 and v3).

For partitioning and indexColumnList, see Data validation configuration reference.

Example workflow excerpt:

sourcePlatform: redshift
targetDatabase: TARGET_DB
validationConfiguration:
  schemaValidation: true
  metricsValidation: true
  rowValidation: false
tables:
  - fullyQualifiedName: snowconvert_demo.ecommerce_raw.customers
    columnNamesToPartitionBy:
      - customer_id
  - fullyQualifiedName: snowconvert_demo.ecommerce_raw.orders
    columnNamesToPartitionBy:
      - order_id
    indexColumnList:
      - order_id
    validationConfiguration:
      rowValidation: true

Data type mappings¶

Platform-specific considerations¶

• Data shares: To validate tables from a Redshift data share, set the Worker’s database and the workflow’s source.databaseName to the database created from the share, and set indexColumnList on each table, because shared tables expose no primary key. See Migrating from a Redshift data share.

• Starting validation: Ask the agent to generate a validation workflow with the depth you need (schema validation, metrics validation, and row-level validation, per table).

  Prompt:

  Copy code

  Run cloud data validation for my Redshift tables, with schema and metrics validation on all tables and row-level validation on orders
  

• For tables migrated via UNLOAD, confirm the live Redshift source still reflects the data snapshot you expect to compare.

  Prompt:

  Copy code

  Validate the customers and orders tables that were migrated with UNLOAD extraction
  

• Anti-locking: No automatic hint is added on Redshift. Set queryModifiers only when you need custom source SQL hints. See Anti-locking and query modifiers.

Related content¶

• Data validation
• Migrating Data from Amazon Redshift
• Apache Iceberg™ tables (spec v2 and v3)
• Manual Migration: Data validation