Enabling Snowpipe error notifications for Microsoft Azure Event Grid¶

Retrieve the Tenant ID¶

Retrieve your Azure tenant ID, which you will need later in these instructions.

1. Log into the Microsoft Azure portal.

2. Navigate to Azure Active Directory » Properties. Record the Tenant ID value for reference later. The directory ID, or tenant ID, is needed to generate the consent URL that grants Snowflake access to the Event Grid topic.

Create the Notification Integration¶

Create an integration using the CREATE NOTIFICATION INTEGRATION command. An integration is a Snowflake object that references the Azure storage queue you created.

CREATE NOTIFICATION INTEGRATION <integration_name>
  ENABLED = true
  TYPE = QUEUE
  NOTIFICATION_PROVIDER = AZURE_EVENT_GRID
  DIRECTION = OUTBOUND
  AZURE_EVENT_GRID_TOPIC_ENDPOINT = '<event_grid_topic_endpoint>'
  AZURE_TENANT_ID = '<azure_tenant_id>'

For example:

CREATE NOTIFICATION INTEGRATION myint
  ENABLED = true
  TYPE = QUEUE
  NOTIFICATION_PROVIDER = AZURE_EVENT_GRID
  DIRECTION = OUTBOUND
  AZURE_EVENT_GRID_TOPIC_ENDPOINT = 'https://myaccount.region-1.eventgrid.azure.net/api/events'
  AZURE_TENANT_ID = 'mytenantid';

Where:

• event_grid_topic_endpoint is the Event Grid topic endpoint you recorded in section Step 1.

• azure_tenant_id is your Azure directory ID, or tenant ID, which you recorded earlier in this section.

Grant Snowflake Access to the Topic¶

1. Execute the DESCRIBE INTEGRATION command to retrieve the consent URL:

   DESC NOTIFICATION INTEGRATION <integration_name>;
   

   Copy

   Where:

   • integration_name is the name of the integration you created in Create the Notification Integration.

   Note the values in the following columns:

   AZURE_CONSENT_URL:

   URL to the Microsoft permissions request page.

   AZURE_MULTI_TENANT_APP_NAME:

   Name of the Snowflake client application created for your account. In a later step in this section, you will need to grant this application the permissions necessary to obtain an access token on your allowed topic.

2. In a web browser, navigate to the URL in the AZURE_CONSENT_URL column. The page displays a Microsoft permissions request page.

3. Click the Accept button. This action allows the Azure service principal created for your Snowflake account to be granted an access token on specified resources inside your tenant. Obtaining an access token succeeds only if you grant the service principal the appropriate permissions on the container (see the next step).

   The Microsoft permissions request page redirects to the Snowflake corporate site (snowflake.com).

4. Log into the Microsoft Azure portal.

5. Navigate to Azure Active Directory » Enterprise applications. Verify the Snowflake application identifier you recorded in Step 2 in this section is listed.

   Important

   If you delete the Snowflake application in Azure Active Directory at a later time, the notification integration stops working.

6. Navigate to Event Grid Topics » topic_name, where topic_name is the name of the topic you created to receive event notifications.

7. Click Access Control (IAM) » Add role assignment.

8. Search for the Snowflake service principal. This is the identity in the AZURE_MULTI_TENANT_APP_NAME property in the DESC NOTIFICATION INTEGRATION output (in Step 1). Search for the string before the underscore in the AZURE_MULTI_TENANT_APP_NAME property.

   Important

   • It can take an hour or longer for Azure to create the Snowflake service principal requested through the Microsoft request page in this section. If the service principal is not available immediately, we recommend waiting an hour or two and then searching again.

   • If you delete the service principal, the notification integration stops working.

9. Grant the Snowflake application the EventGrid Data Sender permission.

New pipe¶

Create a new pipe using CREATE PIPE:

CREATE PIPE <name>
  AUTO_INGEST = TRUE
  [ INTEGRATION = '<string>' ]
  ERROR_INTEGRATION = <integration_name>
  AS <copy_statement>

Where:

ERROR_INTEGRATION = <integration_name>

Name of the notification integration you created in Step 4: Creating the Notification Integration (in this topic).

For example:

CREATE PIPE mypipe
  AUTO_INGEST = TRUE
  INTEGRATION = 'my_storage_int'
  ERROR_INTEGRATION = my_notification_int
  AS
  COPY INTO mydb.public.mytable
  FROM @mydb.public.mystage;

Existing pipe¶

Modify an existing pipe using ALTER PIPE:

ALTER PIPE <name> SET ERROR_INTEGRATION = <integration_name>;

Where <integration_name> is the name of the notification integration you created in Step 4: Creating the Notification Integration (in this topic).

For example:

ALTER PIPE mypipe SET ERROR_INTEGRATION = my_notification_int;