Openflow Connector for Confluence Data Center: Set up Snowflake¶

Note

The connector is subject to the Snowflake Connector Terms.

This topic describes the steps required to configure Snowflake to support the Openflow Connector for Confluence Data Center.

As a Snowflake administrator, perform the following tasks:

Note

The connector uses a Snowflake role to read from and write to Snowflake. Runtimes on Openflow Snowflake deployments use a role assigned when the role is created.

For BYOC deployment runtimes, create a new role or identify an existing role with the following privileges to read from, write to, and create objects in the schema in which you intend to store your Confluence data. See example below.

For BYOC deployment runtimes, create a new Snowflake service user or use an existing one. The service user is used to operate the connector. Configure with key-pair auth.
```
USE ROLE USERADMIN;
CREATE USER <OPENFLOW_SERVICE_USER> TYPE=SERVICE COMMENT='Service user for Openflow automation';
```

Create a role to manage the connector and the associated data. Grant it to the Snowflake service user.

USE ROLE SECURITYADMIN;
CREATE ROLE <OPENFLOW_CONNECTOR_DATA_ENGINEER_ROLE>;
GRANT ROLE <OPENFLOW_CONNECTOR_DATA_ENGINEER_ROLE> TO USER <OPENFLOW_SERVICE_USER>;

If you are using Cortex search:

Create a role for read access to the Cortex Search service created by this connector. This role should be granted to any role that uses the service.
```
CREATE ROLE <CORTEX_SEARCH_SERVICE_READ_ONLY_ROLE>;
GRANT ROLE <CORTEX_SEARCH_SERVICE_READ_ONLY_ROLE> TO ROLE <whatever_roles_will_access_search_service>;
```

Create or identify an existing database in Snowflake.

--Create the database the data will be stored in and grant usage to the roles created
USE ROLE ACCOUNTADMIN; --use whatever role you want to own your DB
CREATE DATABASE IF NOT EXISTS <destination_database>;
GRANT USAGE ON DATABASE <destination_database> TO ROLE <openflow_connector_admin_role>;

Create the schema the data will be stored in. Grant the necessary database privileges to the role created in the first step for BYOC runtimes or the “runtime” role for Openflow - Snowflake Deployment runtimes to the connector admin role:

USE DATABASE <destination_database>;
CREATE SCHEMA IF NOT EXISTS <destination_schema>;
GRANT USAGE ON SCHEMA <destination_schema> TO ROLE <openflow_connector_admin_role>;
GRANT CREATE TABLE, CREATE DYNAMIC TABLE, CREATE STAGE, CREATE SEQUENCE, CREATE CORTEX SEARCH SERVICE ON SCHEMA <destination_schema> TO ROLE <openflow_connector_admin_role>;

If using the Cortex use case:

Note

The following block is for the Cortex use case ONLY.

Grant the Cortex read-only role access to the database and schema.

GRANT USAGE ON DATABASE <destination_database> TO ROLE <CORTEX_SEARCH_SERVICE_READ_ONLY_ROLE>;
GRANT USAGE ON SCHEMA <destination_schema> TO ROLE <CORTEX_SEARCH_SERVICE_READ_ONLY_ROLE>;

If you haven’t already set up Openflow with a secrets manager such as AWS, Azure, or HashiCorp, configure it now. Then store your public and private keys in the secret store. For more information, see Set up Openflow - BYOC.

Note

If you don’t use a secrets manager, then you are responsible for safeguarding public key and private key files used for key-pair authentication according to the security policies of your organization.
1. Once the secrets manager is configured, determine how you will authenticate to it. On AWS, you should use the EC2 instance role associated with Openflow, as this way no other secrets have to be persisted.
2. In Openflow, configure a parameter provider associated with this secrets manager:
  1. From the menu in the upper right, select Controller Settings.
  2. Select the Parameter providers tab.
  3. Fetch the parameter values.
When using a secrets manager, all credentials can be referenced with the associated parameter paths and no sensitive values need to be persisted within Openflow.
Optional: If any other Snowflake users or roles require access to the documents ingested and tables created by the connector, then grant those users or roles the role created in a previous step.
Create a warehouse for the connector or use an existing one. Start with the smallest warehouse size, then experiment with size depending on the number of documents being replicated and the amount of data transferred. Large document numbers typically scale better with multi-cluster warehouses, rather than larger warehouse sizes. Grant the warehouse access to the role created or identified previously.
```
CREATE WAREHOUSE <CONFLUENCE_WAREHOUSE>
WITH
  WAREHOUSE_SIZE = 'SMALL'
  AUTO_SUSPEND = 300
  AUTO_RESUME = TRUE;

GRANT USAGE, OPERATE ON WAREHOUSE <CONFLUENCE_WAREHOUSE> TO ROLE <OPENFLOW_CONNECTOR_ADMIN_ROLE>;
```
Create an external access integration to enable network access outside of Snowflake.

Important

If your runtime executes in Openflow - BYOC, you do not need to create an External Access Integration (EAI). Instead, configure your cloud network egress to allow access to your Confluence site URL.

To allow the connector to access the Confluence site from a Snowflake-hosted runtime, you must create a network rule and an external access integration (EAI). Then, grant the Snowflake role usage privileges on the EAI.

To create the external access integration and network rule and grant access, perform the following steps:
1. Create a network rule to all the connector to access the required REST APIs:
  USE ROLE ACCOUNTADMIN; USE DATABASE <openflow_network_db>; CREATE OR REPLACE NETWORK RULE openflow_<runtime_name>_network_rule TYPE = HOST_PORT MODE = EGRESS VALUE_LIST = ( '<confluence_site_url>' );
2. Create an External Access Integration that references the network rule:
  CREATE OR REPLACE EXTERNAL ACCESS INTEGRATION openflow_<runtime_name>_eai ALLOWED_NETWORK_RULES = (openflow_<runtime_name>_network_rule) ENABLED = TRUE;
3. Grant your Snowflake Role USAGE on the integration:
  GRANT USAGE ON INTEGRATION openflow_<runtime_name>_eai TO ROLE openflow_runtime_role_<runtime_name>;

Next steps¶

After setting up Snowflake, continue with Set up a connector.