Manage Openflow¶

This topic describes the steps to manage Openflow components.

Delete a deployment¶

From the AWS Console:

  1. Navigate to EC2 Instances

  2. Select the openflow-agent-{deployment-key} instance with your deployment key.

  3. Click Connect at the top of the page.

  4. Switch from EC2 Instance Connect to Connect using EC2 Instance Connect Endpoint. Leave the default EC2 Instance Connect Endpoint in place.

  5. Click Connect. A new browser tab or window will appear with a command line interface.

  6. Run ./destroy.sh from the shell.

    • This may take 20-30 minutes. If your connection is interrupted, the process continues running in the background.

    • You can log back in and view its status with the command: journalctl -u docker -f -n 250

    • The destroy process is complete when you see output of delete successful.

  7. Navigate to CloudFormation in the AWS Console for your region.

  8. Delete the CloudFormation stack for your deployment.

From Snowsight:

  1. Navigate to Openflow.

  2. Select the Deployments tab.

  3. In the row of the deployment you want to delete, select the More options icon.

  4. Select Delete.

  5. In the confirmation dialog, type delete to confirm deletion.

  6. Click Delete deployment.

Upgrade a deployment¶

Within the Openflow deployment, there are several components that may require updating. Currently, this is executed through an Openflow deployment agent script. Components include the agent, the deployment service, the deployment UI, the runtime gateway, and the runtime operator. All components will be upgraded by the following procedures.

Connect to the deployment agent¶

  1. Navigate to Openflow.

  2. Select the Deployments tab.

  3. View your deployment details and note the deployment key.

  4. In your AWS account, view the EC2 instances and filter using the deployment key.

  5. Locate the deployment agent EC2 instance named openflow-agent-{deployment-key}.

  6. Connect using EC2 Instance Connect Endpoint and accepting all defaults.

  7. Run the remaining commands from the new browser tab or window that appears with a command line interface.

Check for available upgrades¶

cat ~/.upgrade
Copy

The script will display the latest available version of the various deployment components.

If no upgrades are available, you will see an output similar to this:

AGENT_IMAGE_VERSION_UPGRADE=
OPERATOR_CHART_VERSION_UPGRADE=
GATEWAY_IMAGE_VERSION_UPGRADE=
DPS_CHART_VERSION_UPGRADE=
DPUI_CHART_VERSION_UPGRADE=

Otherwise, you will see the version that upgraded components will use, such as:

AGENT_IMAGE_VERSION_UPGRADE=0.17.0
OPERATOR_CHART_VERSION_UPGRADE=0.31.0
GATEWAY_IMAGE_VERSION_UPGRADE=
DPS_CHART_VERSION_UPGRADE=
DPUI_CHART_VERSION_UPGRADE=

Initiate the upgrade¶

If the output indicates that upgrades are available, run the following script to initiate the upgrade:

./upgrade-data-plane.sh
Copy

You will see output similar to this:

openflow-data-plane-agent-aws is set to version 0.16.0
   Upgrade set to version 0.17.0
openflow-dataplane-service-chart is set to version 0.47.0
   No upgrade is available
openflow-dataplane-ui-chart is set to version 0.5.0
   No upgrade is available
openflow-runtime-gateway is set to version 2025.6.8.2
   No upgrade is available
runtime-operator-chart is set to version 0.30.0
   Upgrade set to version 0.31.0

Then, you have two options:

  • Wait for an automatic upgrade: The system will automatically initiate the upgrade process within approximately 10 minutes.

  • Manual upgrade: To start the upgrade immediately, run the following command:

./create.sh
Copy

Monitor the upgrade process¶

To track the progress of the upgrade, use the journalctl command:

journalctl -u openflow-apply-infrastructure -f -n 250
Copy

Verify a successful upgrade¶

A successful upgrade will typically show output similar to this:

All resources applied successfully and log uploaded to s3
openflow-apply-infrastructure.service: Deactivated successfully

Upgrade a runtime¶

Updates will be made available by Snowflake, both when introducing new Openflow processors or newer versions of existing processors, as well as when introducing new Openflow runtime functionality. When updates are available for Openflow runtimes, you will see an Upgrade icon next to your runtime on the Openflow UI.

To upgrade a runtime, do the following:

  1. Navigate to Openflow.

  2. Select the Runtimes tab.

  3. An indicator next to the Runtime name will show if an upgrade is available.

  4. In the row of the runtime you want to upgrade, select the More options icon.

  5. Select Upgrade.

Upgrade a connector¶

Connector updates are made available by Snowflake when functionality is added, processing logic is improved, or new processor versions are used.

For example, to add support for a new source API version.

When connector updates are available, you will see an Upgrade icon in your process group on the canvas.

Note

You can only upgrade connectors after you have upgraded their runtime.

To upgrade a connector, do the following:

  1. Navigate to Openflow.

  2. Select the Runtimes tab.

  3. Click the runtime name or select ‘View Canvas’ in the More Options menu to navigate to the canvas.

  4. Locate the processor group(s) which displays a red upgrade arrow next to its name.

  5. Right click on the process group and select Version » Change Version.

  6. Select the latest available version and select Change.

  7. Confirm that your connector was upgraded to the latest version which should now show a green check mark. You can also validate the version by hovering over the speech bubble at the bottom right of the process group.

Configure Snowflake Connector Flow Registry¶

Important

Early preview releases of Openflow did not configure a runtime for connector upgrades. If you don’t see the Version option when right clicking on a process group, you have to configure the Snowflake Connector Flow Registry and manually enable version control for existing connectors.

To configure the Snowflake Connector Flow Registry, do the following:

  1. Navigate to the canvas.

  2. Click on the menu in the top right corner and select Controller Settings.

  3. Switch to the Registry Clients tab.

  4. Click the + icon to add a new Registry Client.

  5. Select the ConnectorFlowRegistryClient and select Add.

  6. Click More Options for the ConnectorFlowRegistryClient row and select Edit.

  7. Enter /nifi/configuration_resources/connector_flow_registry as the value for Storage Location and select Apply.

After configuring the Snowflake Connector Flow Registry you can now enable version control for your existing connectors.

To enable version control for existing connectors, do the following:

  1. Navigate to the canvas and locate the process group where you want to add version control.

  2. Right click on the process group and select Version » Set Version.

  3. In the Set Version dialog, choose the flow that matches your process group.

    For example choose sqlserver if you are using the SQL Server connector.

    Note that flow names do not exactly match the connector name.

  4. Select the latest version and then select Set version to enable version control.

  5. From the canvas, right click on the process group again and select Version » Revert Local Changes to apply the latest connector version.

  6. Review the list of changes and select Revert.

  7. Confirm that your connector was upgraded to the latest version which should now show a green check mark. You can also validate the version by hovering over the speech bubble at the bottom right of the process group.

Language: English