Configuring Snowpipe Streaming¶

Required Properties¶

authorization_type

Configure the authentication and authorization method for the user. You can use one of the following methods:

• JWT : key pair authentication with JSON Web Token (JWT). This is the default method. If authorization_type is not configured, the default method JWT will be used. Configure the following private_key for key pair authentication.

  • private_key Private key to authenticate the user. Include only the key, not the header or footer. If the key is split across multiple lines, remove the line breaks.

    You can provide an unencrypted key, or you can provide an encrypted key and provide the snowflake.private.key.passphrase parameter to enable Snowflake to decrypt the key. Use this parameter if and only if the snowflake.private.key parameter value is encrypted.

• OAuth : Snowflake OAuth. This option is only available with Snowflake Ingest SDK versions 2.0.3 and later. Configure the following parameters for Snowflake OAuth in the profile.json file.

  • oauth_client_id : The client ID of the OAuth integration.

  • oauth_client_secret : The client secret of the OAuth integration.

  • oauth_refresh_token : A valid refresh token of the OAuth integration.

url

URL for accessing your Snowflake account. This URL must include your account identifier. Note that the protocol (https://) and port number are optional.

Note that url is not required if you are already using the Snowflake Ingest SDK and have set the host, scheme, and port properties in the profile.json file.

user

User login name for the Snowflake account.

Optional Properties¶

snowflake.private.key.passphrase: (Optional)

Passphrase to decrypt the private key when the key is encrypted. For information, see Using Key Pair Authentication & Key Rotation (in this topic).

role

Access control role to use for the session after connecting to Snowflake.

Note that the role property is optional for Snowflake Ingest SDK versions 2.0.3 and later. It is required for earlier Ingest SDK versions.

Using Snowflake OAuth¶

With Snowflake Ingest SDK versions 2.0.3 and later, you can use Snowflake OAuth as an authorization method.

Follow the workflow to create a Snowflake OAuth integration and call OAuth endpoints to request authorization codes and to request and refresh access tokens. The response of token requests contains oauth_refresh_token. After a Snowflake OAuth integration is created, run the SYSTEM$SHOW_OAUTH_CLIENT_SECRETS function to obtain oauth_client_id and oauth_client_secret.

To enable Snowflake OAuth, in the profile.json file, set authorization_type as OAuth and fill the fields oauth_refresh_token, oauth_client_id and oauth_client_secret with the parameters obtained above.

Using Key Pair Authentication & Key Rotation¶

API calls rely on key pair authentication with JSON Web Token (JWT). JWTs are signed using a public/private key pair with RSA encryption. This authentication method requires a 2048-bit (minimum) RSA key pair. Generate the public-private key pair using OpenSSL. The public key is assigned to the Snowflake user defined in the properties file.

Complete the key pair authentication instructions described in key pair rotation. Copy and paste the entire private key into the snowflake.private.key field in the properties file. Save the file.

See Java Example for an example of creating a fingerprint and generating a JWT token.

Next, evaluate the recommendation for Externalizing Secrets (in this topic).

Externalizing Secrets¶

Snowflake strongly recommends externalizing secrets such as the private key and storing them in an encrypted form or in a key management service such as AWS Key Management Service (KMS), Microsoft Azure Key Vault, or HashiCorp Vault.

For more information, see the Confluent description of this service.