Set up the Atlassian Jira Cloud (Agile) flow¶

Prerequisites¶

1. Ensure that you have reviewed About Openflow Connector for Jira Cloud.

2. Ensure that you have Set up Openflow - BYOC or Set up Openflow - Snowflake Deployments.

3. If using Openflow - Snowflake Deployments, ensure that you’ve reviewed configuring required domains and have granted access to the required domains for the Jira Cloud connector.

Get the credentials¶

As a Jira Cloud administrator, perform the following tasks in your Atlassian account. You can reuse the API token from the core flow or create a separate token. The core flow and agile flow can use the same token, but they always share the underlying Jira API rate budget regardless.

1. Navigate to the API tokens page.

2. Select Create API token with scopes.

3. In the Create an API token dialog box, provide a descriptive name for the API token and select an expiration date for the API token. This can range from 1 to 365 days.

4. Select the API token app Jira.

5. Select the agile scopes listed in Required API scopes.

6. Select Create token.

7. In the Copy your API token dialog box, select Copy to copy your generated API token and then paste the token to the connector parameters, or save it securely.

8. Select Close to close the dialog box.

Set up Snowflake account¶

If you’ve already completed the Snowflake account setup for the core flow, you can reuse the same role, service user, key pair, database, schema, and warehouse for the agile flow. The agile flow parameters point at this same Snowflake configuration.

Otherwise, perform the following tasks:

As a Snowflake account administrator, perform the following tasks:

1. Create a new role or use an existing role.

2. Create a new Snowflake service user with the type as SERVICE.

3. Grant the Snowflake service user the role you created in the previous steps.

4. Configure with key-pair auth for the Snowflake SERVICE user from step 2.

5. Configure a secrets manager supported by Openflow (recommended), for example, AWS, Azure, and HashiCorp, and store the public and private keys in the secret store.

   Note

   If for any reason, you don’t want to use a secrets manager, then you are responsible for safeguarding the public key and private key files used for key-pair authentication according to the security policies of your organization.

   1. After the secrets manager is configured, determine how you will authenticate to it. On AWS, use the EC2 instance role associated with Openflow as this way no other secrets have to be persisted.

   2. In Openflow, configure a Parameter Provider associated with this Secrets Manager, from the main menu (⋮) in the upper-right corner. Navigate to Controller Settings » Parameter Provider and then fetch your parameter values.

   3. At this point, all credentials can be referenced with the associated parameter paths and no sensitive values need to be persisted within Openflow.

6. If any other Snowflake users require access to the tables ingested by the connector (for example, for custom processing in Snowflake), then grant those users the role created in step 1.

7. Create a database and schema in Snowflake for the connector to store ingested data. Grant the following Database privileges to the role created in the first step.

   CopyExpand

   CREATE DATABASE jira_destination_db;
   CREATE SCHEMA jira_destination_db.jira_destination_schema;
   GRANT USAGE ON DATABASE jira_destination_db TO ROLE <jira_connector_role>;
   GRANT USAGE ON SCHEMA jira_destination_db.jira_destination_schema TO ROLE <jira_connector_role>;
   GRANT CREATE TABLE ON SCHEMA jira_destination_db.jira_destination_schema TO ROLE <jira_connector_role>;
   

   Show lessSee more

   Scroll to top

8. Create a warehouse that the connector will use or use an existing one. Start with the smallest warehouse size, then experiment with size depending on the amount of data transferred. Large data volumes typically scale better with multi-cluster warehouses, rather than larger warehouse sizes.

9. Ensure that the user with the role used by the connector has the required privileges to use the warehouse. If that’s not the case then grant the required privileges to the role.

   CopyExpand

   CREATE WAREHOUSE jira_connector_warehouse WITH WAREHOUSE_SIZE = 'X-Small';
   GRANT USAGE ON WAREHOUSE jira_connector_warehouse TO ROLE <jira_connector_role>;
   

   Show lessSee more

   Scroll to top

Set up the connector¶

The agile flow is shipped as the Atlassian Jira Cloud (Agile) process group. As a data engineer, perform the following tasks to install and configure it.

Run the flow¶

1. Right-click on the canvas and select Enable all Controller Services.

2. Right-click on the Atlassian Jira Cloud (Agile) process group and select Start. The flow starts the data ingestion.

On first run, the flow creates the required Snowflake tables in the destination schema. See Destination tables for the full list of tables created by the agile flow and the parameters that control which optional tables are populated.

Resetting the connector state¶

If you want to restart the ingestion from scratch, clear the agile flow’s ingestion state. The agile flow uses its own centralized state service rather than per-processor state.

To reset the state, perform the following steps:

1. Right-click the Atlassian Jira Cloud (Agile) process group and select Stop.

2. Navigate to the Controller Settings for the process group.

3. Find the StandardJiraIngestionStateService controller service and select View State.

4. Select Clear State. This clears the agile flow’s ingestion tracking.

5. Optionally, update the connector parameters if needed.

6. Right-click the Atlassian Jira Cloud (Agile) process group and select Start.

Next steps¶

• Set up the Atlassian Jira Cloud (Core) flow if you haven’t yet installed the core flow.

• Migrate from the legacy Openflow Connector for Jira Cloud if you’re moving from a previous version of the Jira Cloud connector.