Planning an External Function for AWS

This topic helps you prepare to create an external function for AWS (Amazon Web Services) using either the AWS Management Console or an AWS CloudFormation template provided by Snowflake.

In this Topic:

Prerequisites

These instructions assume that you are an experienced AWS Management Console user.

You need:

  • An account with AWS, including privileges to:

    • Create AWS roles via IAM (identity and access management).

    • Create AWS Lambda Functions.

    • Create an API Gateway endpoint.

  • A Snowflake account in which you have ACCOUNTADMIN privileges or a role with the CREATE INTEGRATION privilege.

  • If you plan to use a private endpoint, you need your Virtual Private Cloud (VPC) ID. (You must use a VPC ID, not a VPC Endpoint ID. VPC Endpoint IDs can change over time.)

    If you do not already have your VPC ID, you can look it up by executing the following command in the Snowflake web interface:

    select system$get_snowflake_platform_info();
    

    The output should look similar to the following:

    {"snowflake-vpc-id":["vpc-12345678"]}
    

    Record the VPC ID. After you decide whether to create your external function by using the AWS Management Console or an AWS CloudFormation template, copy the VPC ID to the appropriate tracking worksheet:

Choosing Your Endpoint Type: Regional Endpoint vs. Private Endpoint

You access a proxy service (such as Amazon API Gateway) via a URI, often referred to as an endpoint. The instructions for creating your Amazon API Gateway ask you to choose one of the following types of endpoints:

  • A regional endpoint.

  • A private endpoint.

The following information can help you choose the type of endpoint.

A regional endpoint can be accessed across AWS regions, or even across cloud platforms. Your Snowflake instance, your proxy service, and your remote service can all be in different regions or even on different cloud platforms. For example, a Snowflake instance running on Azure could send requests to an Amazon API Gateway regional endpoint, which in turn could forward data to a remote service running on GCP.

A private endpoint can be configured to allow access only within a region. For example, you can configure a private endpoint to allow access from only a Snowflake VPC (Virtual Private Cloud) in the same AWS region. Communication between a Snowflake VPC and a private endpoint uses AWS PrivateLink.

For more details about the types of endpoints on AWS, see:

If you want to use a private endpoint, and you are not sure which region you are using, you can look up your region by doing either of the following:

  • Call the SQL function CURRENT_REGION() (e.g. SELECT CURRENT_REGION()).

  • Check your Snowflake account hostname, which normally indicates the cloud provider and region. For more information about account hostnames, regions, and cloud providers, see Supported Cloud Regions.

To use a private endpoint, your account must meet the following requirements:

  • Business Critical (or higher) edition of Snowflake.

Choosing the Method for Creating the External Function

Snowflake provides instructions for two ways to create an external function on AWS:

  • AWS Management Console web interface

  • AWS CloudFormation template provided by Snowflake

AWS Management Console

You can use the AWS Management Console to create a Lambda Function (as the remote service) and an Amazon API Gateway instance (as the proxy service). If you choose this method, you also use the AWS Management Console to configure security-related settings.

The instructions for creating an external function using the AWS Management Console include a sample Lambda Function and details for creating a basic API Gateway:

  • First-time users can use the instructions with little or no modification.

  • Experienced users can use the instructions and sample Lambda Function as a starting point for creating a custom Lambda Function and a custom-configured API Gateway.

AWS CloudFormation Template

The CloudFormation template performs both of the following steps in creating an external function:

  • Creating the remote service (an AWS Lambda Function).

  • Creating and configuring the proxy service (an Amazon API Gateway).

The template also:

  • Creates two IAM roles (one for the Lambda Function and one for the API Gateway).

  • Configures a resource policy for the API Gateway.

Preparing to Use the AWS Management Console

Create a Worksheet for Tracking Required Information

As you create your external function, you should record specific information that you enter (e.g. the Resource Invocation URL) so that you can use that information in subsequent steps. The worksheet below helps you track this information.

===========================================================================
================ Tracking Worksheet: AWS Management Console ===============
===========================================================================

****** Step 1: Information about the Lambda Function (remote service) *****

Your AWS Account ID: ______________________________________________________

Lambda Function Name: _____________________________________________________


******** Step 2: Information about the API Gateway (proxy Service) ********

New IAM Role Name: ________________________________________________________

New IAM Role ARN: _________________________________________________________

Snowflake VPC ID (optional): ______________________________________________

New API Name: _____________________________________________________________

API Gateway Resource Name: ________________________________________________

Resource Invocation URL: __________________________________________________

Method Request ARN: _______________________________________________________


*** Step 3: Information about the API Integration and External Function ***

API Integration Name: _____________________________________________________

API_AWS_IAM_USER_ARN: _____________________________________________________

API_AWS_EXTERNAL_ID: ______________________________________________________

External Function Name: ___________________________________________________

Preparing to Use an AWS CloudFormation Template

Download the Template

The template is available for download from the deployment templates directory in the Snowflake repository in GitHub.

Create a Worksheet for Tracking Required Information

As you create your external function, you should record specific information that you enter (e.g. the Resource Invocation URL) so that you can use that information in subsequent steps. The worksheet below helps you track this information.

===========================================================================
================== Tracking Worksheet: CloudFormation Template ============
===========================================================================

New IAM Role Name: ________________________________________________________

New IAM Role ARN: _________________________________________________________

Resource Invocation URL: __________________________________________________

API_AWS_IAM_USER_ARN: _____________________________________________________

API_AWS_EXTERNAL_ID: ______________________________________________________

Additional Resources for Building External Functions on AWS

When you are ready to create your own remote service for your own external function, you might want to look at the examples of remote services based on Lambda Functions that are available in The Snowflake Labs.