Categories:: System functions (Control)

SYSTEM$CREATE_EVALUATION_DATASET¶

Creates a Snowflake dataset for Cortex Agent evaluations from an existing table. The new dataset is registered in the same database and schema as given in the dataset name argument.

Use this function when you want to create an evaluation dataset with SQL (for example, following the Getting Started with Cortex Agent Evaluations guide). You can instead create a dataset from a table in Snowsight when you start an evaluation, or define a dataset block in your evaluation YAML file. For the table and column requirements for evaluation data, see Dataset format.

See also:: EXECUTE_AI_EVALUATION , GET_AI_EVALUATION_DATA (SNOWFLAKE.LOCAL)

Syntax¶

CALL SYSTEM$CREATE_EVALUATION_DATASET(
  '<agent_dataset_type>' ,
  '<source_table_name>' ,
  '<dataset_name>' ,
  <column_mapping>
);

Arguments¶

'agent_dataset_type'

The dataset type for Cortex Agent evaluations. Use the string Cortex Agent. This value is case-insensitive.

'source_table_name'

The fully qualified name of the source table that contains evaluation inputs and ground truth columns (for example, MY_DB.MY_SCHEMA.MY_EVAL_TABLE).

'dataset_name'

The fully qualified name for the dataset Snowflake creates (for example, MY_DB.MY_SCHEMA.MY_EVAL_DATASET). The dataset is created in the database and schema you specify.

column_mapping

A SQL OBJECT that maps evaluation fields to source column names. Include:

query_text: The name of the column that contains the input query (VARCHAR).
ground_truth: The name of the VARIANT column that contains ground truth JSON for each row.

The keys in this object correspond to the column_mapping keys in the dataset section of the Agent Evaluation YAML specification.

Returns¶

A string message indicating whether the dataset was created successfully.

Access control requirements¶

You need permissions to create a dataset from an input table, including CREATE DATASET ON SCHEMA for the schema where the dataset is created.

Examples¶

The following example creates the dataset MARKETING_CAMPAIGNS_DB.AGENTS.MARKETING_CAMPAIGN_EVALSET from the table MARKETING_CAMPAIGNS_DB.AGENTS.EVALS_TABLE, using INPUT_QUERY as the query column and GROUND_TRUTH_DATA as the ground truth column:

CALL SYSTEM$CREATE_EVALUATION_DATASET(
  'Cortex Agent',
  'MARKETING_CAMPAIGNS_DB.AGENTS.EVALS_TABLE',
  'MARKETING_CAMPAIGNS_DB.AGENTS.MARKETING_CAMPAIGN_EVALSET',
  OBJECT_CONSTRUCT(
    'query_text', 'INPUT_QUERY',
    'ground_truth', 'GROUND_TRUTH_DATA'
  )
);

Confirm the dataset exists:

SHOW DATASETS IN SCHEMA MARKETING_CAMPAIGNS_DB.AGENTS;