Set up the Openflow Connector for Jira Cloud¶

This topic describes the steps to set up the Openflow Connector for Jira Cloud.

Prerequisites¶

Ensure that you have reviewed About Openflow Connector for Jira Cloud.
Ensure that you have Set up Openflow - Snowflake Deployments or Set up Openflow - BYOC.
If using Openflow - Snowflake Deployments, ensure that you’ve reviewed configuring required domains and have granted access to the required domains for the Jira Cloud connector.

Get the credentials¶

As a Jira Cloud administrator, perform the following tasks in your Atlassian account:

Navigate to the API tokens page.
Select Create API token with scopes.
In the Create an API token dialog box, provide a descriptive name for the API token and select an expiration date for the API token. This can range from 1 to 365 days.
Select the Api token app Jira.
Select jira scopes read:jira-work and read:jira-user.
Select Create token.
In the Copy your API token dialog box, select Copy to copy your generated API token and then paste the token to the connector parameters, or save it securely.
Select Close to close the dialog box.

Set up Snowflake account¶

As an Openflow administrator, perform the following tasks to set up your Snowflake account. With the default SNOWFLAKE_MANAGED authentication strategy, the runtime’s execute-as role is the identity the connector uses to access Snowflake, so you grant it the following privileges.

Note

If you’re deploying the connector in Openflow - BYOC Deployments and using the KEY_PAIR authentication strategy instead of the recommended SNOWFLAKE_MANAGED, you’ll also grant this same execute-as role to a service user rather than relying on the runtime’s managed token. See Set up key-pair authentication for Openflow - BYOC Deployments to create the service user.

Create database, schema, and warehouse¶

Create the destination database:

USE ROLE OPENFLOW_ADMIN;
CREATE DATABASE IF NOT EXISTS <destination_database>;

Create the destination schema:

CREATE SCHEMA IF NOT EXISTS <destination_database>.<destination_schema>;

Grant the required privileges to the runtime’s execute-as role:

GRANT USAGE ON DATABASE <destination_database> TO ROLE OPENFLOW_<RUNTIME_NAME>_EXECUTE_AS_RL;
GRANT USAGE ON SCHEMA <destination_database>.<destination_schema> TO ROLE OPENFLOW_<RUNTIME_NAME>_EXECUTE_AS_RL;
GRANT CREATE TABLE, CREATE VIEW ON SCHEMA <destination_database>.<destination_schema> TO ROLE OPENFLOW_<RUNTIME_NAME>_EXECUTE_AS_RL;

Create a warehouse (or use an existing one) and grant usage privileges:

CREATE WAREHOUSE IF NOT EXISTS <openflow_warehouse>
  WITH
  WAREHOUSE_SIZE = 'XSMALL'
  AUTO_SUSPEND = 300
  AUTO_RESUME = TRUE;

GRANT USAGE, OPERATE ON WAREHOUSE <openflow_warehouse> TO ROLE OPENFLOW_<RUNTIME_NAME>_EXECUTE_AS_RL;

If any other Snowflake users require access to the tables and views ingested by the connector (for example, for custom processing in Snowflake), grant those users the execute-as role.

Set up the connector¶

As a data engineer, perform the following tasks to install and configure the connector:

Install the connector¶

To install the connector, do the following as a data engineer:

Navigate to the Connector library tab in Openflow.
On the Openflow connectors page, find the connector and select Install.
In the Select runtime dialog, select your runtime from the Available runtimes drop-down list and click Install.

Note
Before you install the connector, ensure that you have created a database and schema in Snowflake for the connector to store ingested data.
Authenticate to the deployment with your Snowflake account credentials and select Allow when prompted to allow the runtime application to access your Snowflake account. The connector installation process takes a few minutes to complete.
Authenticate to the runtime with your Snowflake account credentials.

The Openflow canvas appears with the connector process group added to it.

Configure the connector¶

Right-click on the imported process group and select Parameters.
Populate the required parameter values as described in Flow parameters.

Flow parameters¶

This section describes the flow parameters that you can configure based on the following parameter contexts:

Jira Cloud Source Parameters: Used to establish connection with Jira API.
Jira Cloud Destination Parameters: Used to establish connection with Snowflake.
Jira Cloud Ingestion Parameters: Used to define the configuration of data downloaded from Jira.

Note

Modifying the parameters related to ingestion configuration (for example, Search Type, JQL Query, Project Names, and Created After) will reset the state of the FetchJiraIssues processor, allowing it to fetch all issues again. This is useful if you want to change the issue query criteria or restart the ingestion from scratch. This reset action does not truncate the destination table.

Jira Cloud Source Parameters¶

Parameter	Description
Jira Email	Email address for the Atlassian account.
Jira API Token	API access token for your Atlassian Jira account with the necessary scopes (`read:jira-work` and `read:jira-user`).
Environment URL	URL to the Atlassian Jira environment. For example, `https://your-domain.atlassian.net`.
Connection Method	Must be set to `DIRECT` unless otherwise instructed by Snowflake.

Jira Cloud Destination Parameters¶

Parameter	Description	Required
Destination Database	The database where data will be persisted. It must already exist in Snowflake. The name is case-sensitive. For unquoted identifiers, provide the name in uppercase.	Yes
Destination Schema	The schema where data will be persisted, which must already exist in Snowflake. The name is case-sensitive. For unquoted identifiers, provide the name in uppercase. See the following examples: `CREATE SCHEMA SCHEMA_NAME` or `CREATE SCHEMA schema_name`: use `SCHEMA_NAME` `CREATE SCHEMA "schema_name"` or `CREATE SCHEMA "SCHEMA_NAME"`: use `schema_name` or `SCHEMA_NAME`, respectively	Yes
Snowflake Authentication Strategy	When using: Snowflake Openflow Deployment or BYOC: Use SNOWFLAKE_MANAGED. This token is managed automatically by Snowflake. BYOC deployments must have previously configured execute-as roles to use SNOWFLAKE_MANAGED. BYOC: Alternatively, BYOC can use KEY_PAIR as the value for the authentication strategy.	Yes
Snowflake Account Identifier	When using: SNOWFLAKE_MANAGED Authentication Strategy: Must be blank. KEY_PAIR: Snowflake account name formatted as [organization-name]-[account-name].	Yes
Snowflake Private Key	When using: SNOWFLAKE_MANAGED Authentication Strategy: Must be blank. KEY_PAIR: Must be the RSA private key used for authentication, formatted according to PKCS8 standards and including standard PEM headers and footers. Note that either a Snowflake Private Key File or a Snowflake Private Key must be defined.	No
Snowflake Private Key File	When using: SNOWFLAKE_MANAGED Authentication Strategy: The private key file must be blank. KEY_PAIR: Upload the file that contains the RSA private key used for authentication to Snowflake, formatted according to PKCS8 standards and including standard PEM headers and footers. The header line begins with `-----BEGIN PRIVATE`. To upload the private key file, select the Reference asset checkbox.	No
Snowflake Private Key Password	When using: SNOWFLAKE_MANAGED Authentication Strategy: Must be blank. KEY_PAIR: Provide the password associated with the Snowflake private key file.	No
Snowflake Role	When using: SNOWFLAKE_MANAGED Authentication Strategy: Use the runtime’s execute-as role (or a child role granted to it). You can find your execute-as role in the Openflow UI by navigating to View Details for your runtime. KEY_PAIR: Use a valid role configured for your service user.	Yes
Snowflake Username	When using: SNOWFLAKE_MANAGED Authentication Strategy: Must be blank. KEY_PAIR: Provide the username used to connect to the Snowflake instance.	Yes
Snowflake Warehouse	Snowflake warehouse used to run queries.	Yes

Jira Cloud Ingestion Parameters¶

Parameter	Description
Search Type	Type of search to perform. It has one of these possible values `SIMPLE` and `JQL`. Default value: `SIMPLE`.
Destination Table	The Snowflake table where data is stored. It will be created if it doesn’t exist. The name of the table must be unquoted and must be provided in uppercase. Additionally to the destination table, a flattened view based on destination table is created. The view name is a concatenation of the table name and the suffix `_VIEW`
JQL Query	A JQL query used to search for Jira issues to fetch. It should be used only when Search Type is `JQL`.
Project Names	List of projects from which the issues should be fetched. You can search for issues belonging to a particular project by project name, project key, or project ID. It should be used only when Search Type is `SIMPLE`. Provide a list of items, separated by commas. For example: `Project1, Project2`.
Status Category	Status category filter for simple search. It should be used only when Search Type is `SIMPLE`. Example values are: `Done`, `In Progress`, `To Do`.
Updated After	Filter issues updated after a specified date and time. It should be used only when Search Type is `SIMPLE`. It should be in the yyyy-MM-dd format, such as 2023-10-01.
Created After	Filter issues created after a specified date and time. It should be used only when Search Type is `SIMPLE`. It should be in the yyyy-MM-dd format, such as 2023-10-01.
Issue Fields	A list of fields to return for each issue, which is used to retrieve a subset of fields. IDs of custom fields can be obtained by following this guide. This parameter accepts a comma-separated list. You can use special values: `all` to fetch all fields, `navigable` to fetch navigable fields, field prefixed with minus (`-`) to exclude field. For example, `all,-description` returns all fields except description. Default value: `all`.
Fetch All Worklogs	Determines whether to fetch all worklogs for each issue. Default value: `false`. When set to `true`, the connector enriches issues with all associated worklogs, beyond the default 20 worklogs per issue returned by the Jira Cloud REST API. When set to `false`, only the first 20 worklogs per issue are fetched. Note Setting this parameter to `true` can impact performance due to the increased number of API calls required to fetch all worklogs for issues with more than 20 worklogs.
Maximum Page Size	Maximum number of issues to return per request, with a default and maximum value of `1000`. Note that the Jira API may return fewer results depending on the total response size.

Run the flow¶

Right-click on the plane and select Enable all Controller Services.
Right-click on the imported process group and select Start. The connector starts the data ingestion.

If you need to change the issue query criteria or want to restart the ingestion from scratch, perform the following steps to ensure that the data in the destination table is consistent:

Right-click on the FetchJiraIssues processor and stop it.
Right-click on the FetchJiraIssues processor and then select View State.
In the State dialog box, select Clear State. This action clears the state of the processor and allows it to fetch all issues again.
Optional: If you want to change the issue query criteria, right-click on the imported process group and select Parameters. Update the parameters as needed.
Optional: If you want to change the destination table name, right-click on the imported process group and select Parameters. Update the Destination Table parameter.
Right-click on the FetchJiraIssues processor and select Start. The connector starts the data ingestion.
After ingestion, the data is available in the Snowflake destination table and in a flattened format in the destination view. The view includes all fields available in the Jira instance.

Accessing the data¶

Data fetched from Jira is available in the destination table. All fields fetched for Jira issue is available in the ISSUE column as an object in raw form fetched from the API.

To help with querying the data, a flattened view is created based on the destination table. The view name is a concatenation of the table name and the suffix _VIEW. For example, if the destination table is named JIRA_ISSUES, then the view will be named JIRA_ISSUES_VIEW. In the view, all issue fields are extracted and available as separate columns. The column name is set to the field label. If there are many issues with the same label, a suffix with field ID is added to the column name to ensure uniqueness. For example, if there are two fields with IDs customfield_1, customfield_2, the label set in both fields to Custom Field, then the columns in the view will be named Custom Field (customfield_1), Custom Field (customfield_2).