Setting up Snowflake to use Git

When you connect your Snowflake account to a remote Git repository, Snowflake creates a Git repository clone, copying the latest version of all files in the repository (a shallow clone) and storing metadata about the location of the remote repository, credentials (if needed), and configuration details about how Snowflake should interact with the Git repository API.

Depending on your requirements, by configuring components for authentication, interaction with the Git API, and communication over a private link between Snowflake and your cloud service provider, you can set up Snowflake so that a remote Git repository becomes an integral part of your workflow within Snowflake.

Choose a configuration model

Depending on your network and workflow requirements, you can configure Snowflake for access to a remote Git repository in any of several ways. The following lists example use cases, along with the repository access strategies you might use to support them.

  • Work with files on a Git repository through a workflow that includes pulling, pushing, and creating files.

    Consider using Snowflake Workspaces. If you’re using a repository at github.com, you can authenticate with OAuth.

  • Reference files on a Git repository as part of a data pipeline or ML project.

    If a scripted process will access the repository, consider authenticating using a token.

  • Get started by cloning a public repository (including Snowflake Labs) to run scripts or notebook files.

    You can use Workspaces for .sql files, Snowflake notebooks for .ipynb files, or Snowflake Workspaces for .py files.

The following describes options in terms of whether you want access over a public network or a private network:

Access over a public network

Access over a private network

Access over a public network allows you to authenticate to your remote Git repository server using the entire IP range of your Snowflake cloud provider deployment (because Snowflake does not provide a static IP range).

  1. Configure Snowflake for access to the repository.

    Choose one of the following authentication methods:

    • No authentication.

      Configure an API integration with details about the Git repository server. You don’t provide credentials.

    • Authenticate with a token, such as a personal access token.

      Configure a secret containing the username and token to use, then configure an API integration that allows Snowflake to use the secret when authenticating.

    • Authenticate through an OAuth flow.

      Configure an API integration to support OAuth2 authentication. In this case, you don’t need to create a secret.

  2. Create a Git repository clone to which you can synchronize files from the remote repository.

Access over a private network helps you avoid allowing access to the Git server for the entire IP range of your Snowflake cloud provider deployment.

You can configure Snowflake to establish outbound connectivity through an outbound private link connection between Snowflake and your cloud infrastructure. Snowflake routes Git traffic through this connection to the Git repository server.

  1. Configure the private link connection.

    Before you can configure Snowflake for access to the remote Git repository, you’ll need to set up a private link between Snowflake and your cloud service provider.

  2. Configure Snowflake access to the remote Git repository.

    After you’ve set up private link between Snowflake and your cloud service provider, you can configure Snowflake access to the remote Git repository.

  3. Create a Git repository clone to which you can synchronize files from the remote repository.

Configure Snowflake for access over a public network

You can set up Snowflake to access your Git repository over a public network. You can have Snowflake authenticate using any of the following strategies:

  • No authentication.

    Configure an API integration with details about the Git repository server.

  • Authenticate with a token, such as a personal access token.

    Configure a secret containing the username and token to use, then configure an API integration that allows Snowflake to use the secret when authenticating.

  • Authenticate through an OAuth flow.

    Configure an API integration to allow for any of the following authenticate types: OAuth2, a personal access token, or no authentication.

    Note

    Using OAuth to work with a Git repository is supported only from Workspaces and only when the repository is hosted at github.com. For more information, see Create a Git workspace.

Configure for no authentication

Diagram showing components needed to configure Git connection requiring no authentication

To set up Snowflake to use a Git repository without authenticating, follow these steps:

  1. Create an API integration that supports access without authenticating, and specify the following details:

    • git_https_api as the value of the API_PROVIDER parameter

    • HTTPS endpoints to which requests must be limited as values of the API_ALLOWED_PREFIXES parameter

    For more information, see CREATE API INTEGRATION.

    CREATE OR REPLACE API INTEGRATION my_git_api_integration
      API_PROVIDER = git_https_api
      API_ALLOWED_PREFIXES = ('https://example.com/my-account')
      ENABLED = TRUE;
    
    Copy
  2. Create a Git repository clone as described in Create a Snowflake Git repository clone.

Configure for authenticating with a token

Diagram showing components needed to configure Git connection requiring no authentication

To have Snowflake authenticate with the Git repository by using a username and token such as a personal access token (PAT), follow these steps:

  1. Provide credentials in a basic authentication secret.

    To provide the credentials that Snowflake uses to authenticate with the repository, create a secret that contains the following:

    • A TYPE value of password

    • A username and token, such as a personal access token (PAT)

      If your Git repository is hosted on Bitbucket, specify x-token-auth as the username value.

      Note

      For information about creating a personal access token in GitHub, see Managing your personal access tokens in the GitHub documentation.

    For more information on the SQL command for creating a secret, see the CREATE SECRET.

    Code in the following example creates a secret called my_git_secret with a username and the user’s personal access token to use as credentials:

    CREATE OR REPLACE SECRET my_git_secret
      TYPE = password
      USERNAME = 'gladyskravitz'
      PASSWORD = 'ghp_token';
    
    Copy
  2. Create an API integration that supports authenticating with a token.

    To create an API integration for access to a Git repository without authenticating, specify the following details:

    • git_https_api as the value of the API_PROVIDER parameter

    • HTTPS endpoints to which requests must be limited as values of the API_ALLOWED_PREFIXES parameter

    For more information, see CREATE API INTEGRATION.

    CREATE OR REPLACE API INTEGRATION my_git_api_integration
      API_PROVIDER = git_https_api
      API_ALLOWED_PREFIXES = ('https://github.com/my-account')
      ALLOWED_AUTHENTICATION_SECRETS = (my_git_secret)
      ENABLED = TRUE;
    
    Copy
  3. Create a Git repository clone as described in Create a Snowflake Git repository clone.

Configure for authenticating with OAuth

Note

Using OAuth to work with a Git repository is supported only from Workspaces and only when the repository is hosted at github.com. For more information, see Create a Git workspace.

Diagram showing components needed to configure Git connection requiring no authentication

To set up Snowflake so that it authenticates with the remote Git repository using an OAuth flow, follow these steps:

  1. Create an API integration that supports authenticating through OAuth.

    When specifying authentication with an OAuth2 flow, create an API integration that specifies the following:

    • git_https_api as the value of the API_PROVIDER parameter

    • HTTPS endpoints to which requests must be limited as values of the API_ALLOWED_PREFIXES parameter

      This must be a prefix at https://github.com. Authenticating with OAuth is supported only for repositories hosted at github.com.

    • An API_USER_AUTHENTICATION whose value is (TYPE = SNOWFLAKE_GITHUB_APP)

    For more information, see CREATE API INTEGRATION.

    Code in the following example creates an API integration called my_git_api_integration:

    CREATE OR REPLACE API INTEGRATION my_git_api_integration
      API_PROVIDER = git_https_api
      API_ALLOWED_PREFIXES = ('https://github.com')
      API_USER_AUTHENTICATION = (TYPE = SNOWFLAKE_GITHUB_APP)
      ENABLED = TRUE;
    
    Copy
  2. Create a Git repository clone as described in Create a Snowflake Git repository clone.

Configure Snowflake for access over a private network

You can configure Snowflake to establish outbound connectivity through an outbound private link connection between Snowflake and your cloud infrastructure. Snowflake routes Git traffic through this connection to the Git repository server.

With a private link connection, you avoid allowing access to the Git server for the entire IP range of your Snowflake cloud provider deployment. This section describes the steps at a high level.

  1. Configure the private link connection.

    You’ll apply configuration changes to both Snowflake and your cloud service infrastructure. This topic describes the steps on the Snowflake side. For details about all the steps, including about configuring your cloud service provider, see the knowledge base article Configuring Git Integration with Snowflake over Private Link.

  2. Configure Snowflake access to the remote Git repository.

Note

Snowflake supports only connections within the same cloud and region. For example, if your Snowflake deployment is on AWS in the us-west-2 region, then your other components must also be in that region.

Configure Snowflake access to the remote Git repository

After you set up a private link between Snowflake and your cloud service provider, you can configure Snowflake access to the remote Git repository.

  1. Create an API integration that supports authenticating with a certificate.

    Because Snowflake will reach your Git server using the HTTPS protocol, the domain name needs to have a valid certificate. The configuration you use differs depending on whether you use a self-signed certificate or a certificate signed by a certificate authority.

    • Using a self-signed certificate:

      Diagram showing components needed to configure Git connection requiring no authentication
      1. Provide credentials in a generic string secret.

        This should be a public key of a self-signed domain to establish an HTTPS connection. To provide to Snowflake the credentials it will use to authenticate with the server, create a secret that contains the following details:

        • A TYPE parameter value of GENERIC_STRING

        • A public certificate string as the value of the SECRET_STRING parameter

          For the parameter’s value, specify a secret string, such as a public certificate body.

        CREATE OR REPLACE SECRET my_public_certificate
          TYPE = GENERIC_STRING
          SECRET_STRING = '-----BEGIN CERTIFICATE-----
                    <certificate_body>
                    -----END CERTIFICATE-----';
        
        Copy
      2. Create an API integration to integrate with the Git API, and specify the following details:

        • An API_PROVIDER parameter set to git_https_api

        • An API_ALLOWED_PREFIXES set to the base URL beneath which access is allowed

        • A USE_PRIVATELINK_ENDPOINT parameter set to TRUE

        • A TLS_TRUSTED_CERTIFICATES parameter set to the name of the secret you created, which contains the certificate

        For more information, see CREATE API INTEGRATION.

        CREATE OR REPLACE API INTEGRATION my_git_api_integration
          API_PROVIDER = git_https_api
          API_ALLOWED_PREFIXES = ('https://example.com/my-account')
          ALLOWED_AUTHENTICATION_SECRETS = ALL
          USE_PRIVATELINK_ENDPOINT = TRUE
          TLS_TRUSTED_CERTIFICATES = (my_public_certificate)
          ENABLED = TRUE;
        
        Copy
    • Using a certificate signed by a certificate authority:

      Diagram showing components needed to configure Git connection requiring no authentication
      1. Create an API integration to integrate with the Git API, and specify the following details:

        • An API_PROVIDER parameter set to git_https_api

        • An API_ALLOWED_PREFIXES set to the base URL beneath which access is allowed

        • A USE_PRIVATELINK_ENDPOINT parameter set to TRUE

        • A TLS_TRUSTED_CERTIFICATES parameter set to the name of the secret you created, which contains the certificate

        For more information, see CREATE API INTEGRATION.

        CREATE OR REPLACE API INTEGRATION my_git_api_integration
          API_PROVIDER = git_https_api
          API_ALLOWED_PREFIXES = ('https://example.com/my-account')
          ALLOWED_AUTHENTICATION_SECRETS = ALL
          USE_PRIVATELINK_ENDPOINT = TRUE
          ENABLED = TRUE;
        
        Copy
  2. Provide credentials in a basic authentication secret.

    After successfully connecting to the Git server over private link, you must still authenticate with the repository by creating another secret that provides credentials for the repository.

    To provide the credentials that Snowflake uses to authenticate with the repository, create a secret that contains the following:

    • A TYPE value of password

    • A username and token, such as a personal access token (PAT)

      Note

      For information about creating a personal access token in GitHub, see Managing your personal access tokens in the GitHub documentation.

    For more information on the SQL command for creating a secret, see the CREATE SECRET.

  3. Create a Git repository clone as described in Create a Snowflake Git repository clone.

Create a Snowflake Git repository clone

To set up Snowflake to work with a remote Git repository, create a Git repository clone in Snowflake to contain files fetched from the remote repository.

Note

Before beginning the steps in this section, consider first configuring components you might need, including a secret (if the remote repository requires authentication), an API integration, and private link connection between Snowflake and your cloud service provider.

Note

For information on creating a Git workspace in Snowsight, see Create a Git workspace.

A Git repository clone in Snowflake specifies the following details:

  • The remote repository’s origin

    In Git, origin is the remote repository’s URL. Use that URL when setting up Snowflake to use a remote Git repository. The URL must use HTTPS. For example, you can retrieve the origin URL in the following ways:

    • In the GitHub user interface, you can get the origin URL from the repository home page. Select the Code button, and then copy the HTTPS URL from the box displayed beneath the button.

    • From the command line, use the git config command from within your local repository, as in the following example:

      $ git config --get remote.origin.url
      
      Copy

      The command produces output such as the following:

      https://github.com/my-account/snowflake-extensions.git
      

      For reference information about git config, see the git documentation.

  • Credentials, if needed, for Snowflake to use when authenticating with the repository

    For the GIT_CREDENTIALS parameter, specify a Snowflake secret you created.

  • An API integration specifying details for Snowflake interaction with the repository API

You can create a Git repository clone by using either Snowsight or SQL.

Note

Before creating a Git repository clone, you’ll need to create a secret (if the remote repository requires authentication) and an API integration.

Code in the following example creates a Git repository clone called snowflake_extensions. The clone specifies the my_git_api_integration API integration and the my_git_secret secret with credentials for authenticating.

USE ROLE ACCOUNTADMIN;
GRANT CREATE GIT REPOSITORY ON SCHEMA myco_db.integrations TO ROLE myco_git_admin;

USE ROLE myco_git_admin;

CREATE OR REPLACE GIT REPOSITORY snowflake_extensions
  API_INTEGRATION = my_git_api_integration
  GIT_CREDENTIALS = my_git_secret
  ORIGIN = 'https://github.com/my-account/snowflake-extensions.git';
Copy