The SNOWCONVERT_AI database

AIM DMV stores workflow definitions, task queue state, migration progress, and validation results in a Snowflake metadata database. By default this database is named SNOWCONVERT_AI. The Orchestrator creates and updates it on startup.

Use this database to monitor migration and validation workflows, inspect errors and warnings, and manage workflow lifecycle (pause, resume, cancel, retry). You can rename the database and schemas through Orchestrator configuration if your account requires a different name.

Schemas

SchemaPurpose
DATA_MIGRATIONWorkflows, task queue, migration table and partition metadata, workflow management procedures, and migration monitoring views. Validation workflows also register here; both workflow types share WORKFLOW and TASK_QUEUE.
DATA_VALIDATIONValidation table and partition metadata, L1/L2/L3 result tables, and validation monitoring views.
COMMONInternal schema migration tracking (not customer-facing workflow data).
TEMPTransient Snowpipe stage and pipe objects used during migration.

Tables and views

When querying any object below, filter by WORKFLOW_ID to scope results to a single workflow. Other useful filter columns include TABLE_METADATA_ID, PARTITION_NUMBER, RESULT, and STATUS.

DATA_MIGRATION schema

Tables

ObjectPurpose
WORKFLOWOne row per migration or validation workflow (configuration, status, affinity, lifecycle timestamps).
TASK_QUEUEPending and in-progress tasks for the Orchestrator and Workers.
TABLE_METADATAPer-workflow registry of source tables being migrated.
PARTITION_METADATAPer-partition migration progress (boundaries, row counts, status).

Views

ObjectPurpose
TABLE_PROGRESSPer-table partition rollup (extraction, loading, completed, and failed counts).
TABLE_PROGRESS_WITH_EXAMPLE_ERRORTABLE_PROGRESS plus a sample error or warning message per table.
DATA_MIGRATION_ERRORFirst failed task error per workflow, table, and partition.
DATA_MIGRATION_WARNINGNon-fatal migration warnings (for example type fallbacks).
DATA_MIGRATION_WORKFLOWFiltered view of WORKFLOW for data-migration workflows only.
DATA_VALIDATION_WORKFLOWFiltered view of WORKFLOW for data-validation workflows, with L1/L2/L3 rollups.

Dashboard

ObjectPurpose
DATA_MIGRATION_DASHBOARDStreamlit app for table-centric migration monitoring (Tables, Errors, Overview tabs).

DATA_VALIDATION schema

Tables

ObjectPurpose
TABLE_METADATAPer-workflow registry of validated tables and views (source and target identifiers, configuration).
PARTITION_METADATAValidation partition boundaries and row counts per table.
SCHEMA_VALIDATION_RESULTSL1 schema comparison results (column existence, types, ordinals, and similar).
METRICS_VALIDATION_RESULTSL2 metrics comparison results per partition.
ROW_VALIDATION_RESULTSL3 per-partition row comparison results. The RESULT column includes MISMATCH, POSSIBLE_MISMATCH (provisional, pending accepted-transformation reconcile), NOT_FOUND_SOURCE, NOT_FOUND_TARGET, DUPLICATE_SOURCE, DUPLICATE_TARGET, and DUPLICATE_BOTH_SIDES. See Validation levels and result codes for each code’s meaning.
ROW_VALIDATION_SUMMARYL3 per-table or per-partition summary (matching vs differing chunks).
CELL_VALIDATION_RESULTSL3 cell-level diffs (row index, column, source vs target values).

Views

ObjectPurpose
TABLE_PROGRESSPer-table validation rollup with L1/L2/L3 pass flags.
TABLE_PROGRESS_DETAILPer-table partition-level L2/L3 status counts.
PARTITION_PROGRESSDerived L1/L2/L3 and overall validation status per partition.
DATA_VALIDATION_ERRORFailed or abandoned validation tasks with error messages (distinct from row-level MISMATCH results).
DATA_VALIDATION_WARNINGNon-fatal validation warnings (unsupported types, metric exclusions).

Dashboard

ObjectPurpose
DATA_VALIDATION_DASHBOARDStreamlit app for validation monitoring (Tables, Schema, Metrics, Rows, Cell, Errors tabs).

Workflow management procedures

These stored procedures in SNOWCONVERT_AI.DATA_MIGRATION are intended for direct use by operators. They apply to both migration and validation workflows.

Replace <workflow_id> with the integer WORKFLOW_ID from the WORKFLOW table.

-- Retry failed or abandoned tasks
CALL SNOWCONVERT_AI.DATA_MIGRATION.RETRY_WORKFLOW(<workflow_id>);

-- Pause all pending and executing tasks
CALL SNOWCONVERT_AI.DATA_MIGRATION.PAUSE_WORKFLOW(<workflow_id>);

-- Pause tasks for one source table
CALL SNOWCONVERT_AI.DATA_MIGRATION.PAUSE_TABLE(<workflow_id>, '<source_identifier>');

-- Resume a paused workflow
CALL SNOWCONVERT_AI.DATA_MIGRATION.RESUME_WORKFLOW(<workflow_id>);

-- Resume tasks for one source table
CALL SNOWCONVERT_AI.DATA_MIGRATION.RESUME_TABLE(<workflow_id>, '<source_identifier>');

-- Cancel a workflow (active tasks fail with a manual cancellation message)
CALL SNOWCONVERT_AI.DATA_MIGRATION.CANCEL_WORKFLOW(<workflow_id>);

-- Cancel tasks for one source table
CALL SNOWCONVERT_AI.DATA_MIGRATION.CANCEL_TABLE(<workflow_id>, '<source_identifier>');

-- Delete all tasks and table metadata for a workflow (cleanup)
CALL SNOWCONVERT_AI.DATA_MIGRATION.DELETE_WORKFLOW_TASKS(<workflow_id>);

Internal procedures

AIM DMV also defines internal stored procedures such as PULL_TASKS, PULL_SINGLE_TASK, and COMPLETE_TASK that implement the distributed task queue. The Orchestrator and Workers call these automatically. Do not call internal procedures directly unless Snowflake Support instructs you to.

Sample queries

Use these patterns as starting points. Replace <workflow_id> with your workflow’s WORKFLOW_ID.

Find your latest workflow

SELECT WORKFLOW_ID, WORKFLOW_NAME, WORKFLOW_TYPE, STATUS, CREATED_AT
FROM SNOWCONVERT_AI.DATA_MIGRATION.WORKFLOW
ORDER BY CREATED_AT DESC
LIMIT 10;

Migration progress per table

SELECT *
FROM SNOWCONVERT_AI.DATA_MIGRATION.TABLE_PROGRESS
WHERE WORKFLOW_ID = <workflow_id>
ORDER BY SOURCE_IDENTIFIER;

Migration tables with errors

SELECT SOURCE_IDENTIFIER, FAILED_PARTITIONS, EXAMPLE_ERROR_MESSAGE
FROM SNOWCONVERT_AI.DATA_MIGRATION.TABLE_PROGRESS_WITH_EXAMPLE_ERROR
WHERE WORKFLOW_ID = <workflow_id>
  AND FAILED_PARTITIONS > 0;

Migration error details

SELECT SOURCE_IDENTIFIER, PARTITION_NUMBER, ERROR_MESSAGE, CREATED_AT
FROM SNOWCONVERT_AI.DATA_MIGRATION.DATA_MIGRATION_ERROR
WHERE WORKFLOW_ID = <workflow_id>
ORDER BY SOURCE_IDENTIFIER, PARTITION_NUMBER;

Validation status per table

SELECT SOURCE_IDENTIFIER, L1_STATUS, L2_STATUS, L3_STATUS, OVERALL_STATUS
FROM SNOWCONVERT_AI.DATA_VALIDATION.TABLE_PROGRESS
WHERE WORKFLOW_ID = <workflow_id>
ORDER BY SOURCE_IDENTIFIER;

Validation partition detail

SELECT SOURCE_IDENTIFIER, PARTITION_NUMBER, L2_STATUS, L3_STATUS, OVERALL_STATUS
FROM SNOWCONVERT_AI.DATA_VALIDATION.TABLE_PROGRESS_DETAIL
WHERE WORKFLOW_ID = <workflow_id>
ORDER BY SOURCE_IDENTIFIER, PARTITION_NUMBER;

Row-level mismatches (L3)

SELECT TABLE_NAME, PARTITION_NUMBER, RESULT, SOURCE_INDEX_VALUES, TARGET_INDEX_VALUES
FROM SNOWCONVERT_AI.DATA_VALIDATION.ROW_VALIDATION_RESULTS
WHERE WORKFLOW_ID = <workflow_id>
  AND RESULT = 'MISMATCH'
ORDER BY TABLE_NAME, PARTITION_NUMBER
LIMIT 100;

Provisional mismatches (accepted transformations)

When a table defines acceptedTransformations, row-hash mismatches are recorded as POSSIBLE_MISMATCH until reconcile completes. On a completed workflow, this query should return no rows.

SELECT TABLE_NAME, PARTITION_NUMBER, SOURCE_INDEX_VALUES
FROM SNOWCONVERT_AI.DATA_VALIDATION.ROW_VALIDATION_RESULTS
WHERE WORKFLOW_ID = <workflow_id>
  AND RESULT = 'POSSIBLE_MISMATCH'
ORDER BY TABLE_NAME, PARTITION_NUMBER;

Cell-level diffs (L3)

SELECT TABLE_NAME, ROW_INDEX, COLUMN_NAME, SOURCE_VALUE, TARGET_VALUE
FROM SNOWCONVERT_AI.DATA_VALIDATION.CELL_VALIDATION_RESULTS
WHERE WORKFLOW_ID = <workflow_id>
ORDER BY TABLE_NAME, ROW_INDEX, COLUMN_NAME
LIMIT 100;

Validation task errors and warnings

SELECT SOURCE_IDENTIFIER, PARTITION_NUMBER, ERROR_MESSAGE
FROM SNOWCONVERT_AI.DATA_VALIDATION.DATA_VALIDATION_ERROR
WHERE WORKFLOW_ID = <workflow_id>;

SELECT SOURCE_IDENTIFIER, WARNING_MESSAGE
FROM SNOWCONVERT_AI.DATA_VALIDATION.DATA_VALIDATION_WARNING
WHERE WORKFLOW_ID = <workflow_id>;

Correlate queries in QUERY_HISTORY

Both the Orchestrator and Worker set Snowflake QUERY_TAG on every query:

SELECT query_id, query_text, start_time
FROM TABLE(INFORMATION_SCHEMA.QUERY_HISTORY())
WHERE TRY_PARSE_JSON(query_tag):DMVF_WORKFLOW_ID = <workflow_id>
ORDER BY start_time DESC
LIMIT 50;

Related content