Step 2: Create and Configure the Proxy Service (Google Cloud API Gateway)

These instructions show how to a Google Cloud API Gateway that you can use as a proxy service.

Snowflake does not send data (HTTP POST requests) directly to the remote service (e.g. GCP Cloud Function). Instead, Snowflake sends the data to a proxy service that relays the data from Snowflake to the remote service, and from the remote service back to Snowflake. On GCP, Snowflake supports the Google Cloud API Gateway as the proxy service.

In this Topic:

Create a Google Cloud API Gateway to Route Requests to the Cloud Function

If you use the Google Cloud Console, then the specific steps to execute are:

  1. Create an API definition by creating a configuration file.

  2. Create an API Gateway.

    1. Create an API.

    2. Create an API Config.

    3. Create a Gateway with the API Config.

These steps are documented in more detail below.

Create an API Definition

On your local file system, create and customize a YAML-formatted configuration file that specifies the API that you are creating. The file should have the extension .yaml or .yml. A template for the configuration file is below:

swagger: '2.0'
info:
  title: API Gateway config for Snowflake external function.
  description: This configuration file connects the API Gateway resource to the remote service (Cloud Function).
  version: 1.0.0
schemes:
  - https
produces:
  - application/json
paths:
  /<PATH>:
    post:
      summary: Echo the input.
      operationId: echo
      x-google-backend:
        address: <HTTP ENDPOINT TO ROUTE REQUEST TO>
        protocol: h2
      responses:
        '200':
          description: <DESCRIPTION>
          schema:
            type: string

Fill or update the following fields:

  1. Replace the <PATH> with a unique name. This will be incorporated into URLs, so use only characters that are valid in URLs. For example, enter demo-func-resource.

    Note that for the <PATH>, unlike the other fields in this configuration file, you should enter the value before the colon, rather than after the colon. For example, the following is correct:

    paths:
      /demo-func-resource:
    

    The path name should not contain a path parameter . (Google supports path parameters when setting the path to a URL . However, Snowflake does not support path parameters in the corresponding URL specified in the CREATE EXTERNAL FUNCTION statement.)

  2. Copy the path from the immediately preceding step (e.g. demo-func-resource) to the “Path Suffix” field in the worksheet.

  3. Find the address field under the x-google-backend field, and replace the <HTTP ENDPOINT TO ROUTE REQUEST TO> with the value in the “Cloud Function Trigger URL” field of the worksheet. The result should look similar to:

    x-google-backend:
      address: https:// ...
    

    The URL should not have quotation marks around it.

    The URL can be the path to any HTTP endpoint, not only an endpoint hosted by Google.

    If you selected Require HTTPS when you created the remote service, then ensure that the URL you enter into the address field starts with https.

  4. Optionally, you can update any of the following:

    • The title in the info section.

    • The description in the info section.

    • The operationId in the post subsection of the paths section.

    • The summary in the post subsection of the paths section.

  5. Review your sample configuration file. It should look similar to the following:

    swagger: '2.0'
    info:
      title: "API Gateway config for Snowflake external function"
      description: "This configuration file connects the API Gateway resource to the remote service (Cloud Function)."
      version: 1.0.0
    schemes:
      - https
    produces:
      - application/json
    paths:
      /demo-func-resource:
        post:
          summary: "echo the input"
          operationId: echo
          x-google-backend:
            address: https://my_dev.cloudfunctions.net/demo-cloud-function-01
            protocol: h2
          responses:
            '200':
              description: echo result
              schema:
                type: string
    
  6. Save this configuration file.

  7. Record the file’s path and name in the “Configuration File Name” field in the worksheet.

If you want to learn more about the API configuration file, see the following GCP documentation:

Create an API Gateway

Create an API

This step creates a GCP API, which is a container that can contain one or more API Gateways and one or more configuration files.

  1. If you have not already done so, go to the Google Cloud API Gateway screen by clicking on the GCP menu and selecting API Gateway.

  2. Click on CREATE GATEWAY.

  3. Enter the Display Name and the API ID (e.g. demo-api-display-name-for-external-function1 and demo-api-id-for-external-function1).

    You do not need to record these in the worksheet because you do not need to enter these later to create your external function. However, you might want to record the API ID so that you can delete it when you are done with it.

Create an API Config

Upload your configuration file to GCP, which creates an API Config.

  1. Scroll to the API Config section of the screen.

  2. Search for the field that contains Upload an API Spec.

    Click on BROWSE and select your configuration file. The name of your configuration file was recorded in the “Configuration File Name” field of the worksheet.

  3. Enter a display name into the field that contains Display Name.

  4. Select a service account.

    If you created the sample function, then in the field that contains Select a Service Account, select App Engine default service account.

    If you are creating a function to use in production (rather than as a sample), you might choose a different service account.

    The selected service account must have appropriate privileges, including privileges to call the Cloud Function.

Create a Gateway with the API Config

  1. Scroll to the Gateway details section of the screen.

  2. Enter the Display Name of the new API Gateway.

  3. Click in the Location field and select the appropriate region (e.g. us-central1).

  4. Click on CREATE GATEWAY.

    This takes you to the APIs screen and shows you a list of your APIs.

    If your new API is not visible immediately, wait a few minutes, then click the Refresh button.

  5. Copy the value of the API’s Managed Service to the “Managed Service Identifier” field in the worksheet.

  6. At this point, you should still see a list of your APIs. Click on the name of the API.

    You should see 4 tabs: OVERVIEW, DETAILS, CONFIGS, and GATEWAYS.

  7. Click on the GATEWAYS tab.

  8. Copy the Gateway URL to the “Gateway Base URL” field in the worksheet.

Next Step

Continue on to the next step:

Step 3: Create the API Integration for GCP in Snowflake