Set up Openflow - Snowflake Deployment: Connect to private data sources¶

Data Connectivity Proxy (DCP) lets connectors on an Openflow - Snowflake Deployment reach private data sources in your VPC or on-premises network. You deploy a lightweight agent in your network which establishes an encrypted tunnel to Snowflake over port 443. As a result, Snowflake never needs inbound access into your network.

While the agent enables network connectivity to data sources, all data processing still happens in Openflow runtimes hosted by Snowflake. Connector state stays with the runtime, so replacing an agent doesn’t change it.

When to use DCP with Openflow¶

DCP is for Snowflake Deployments, where the runtime runs in Snowflake and can’t otherwise reach hosts behind your firewall.

Choose a connectivity option based on where the runtime runs and whether Snowflake can already reach the source:

For a fuller comparison, see How DCP compares to other private connectivity options.

Prerequisites¶

Before you configure Openflow to use DCP, make sure you have:

• An Openflow - Snowflake Deployment and a runtime. See Set up Openflow - Snowflake Deployment - Task overview.
• An execute-as role for the runtime, with USAGE on the DCP external access integration. See Set up Openflow - Snowflake Deployment: Create the execute-as role and external access integrations.
• An existing Data Connectivity Proxy object that’s enabled. See Create a DCP object in Snowflake.
• A DCP agent deployed where it can:
  • Make outbound TCP/TLS connections on port 443 to Snowflake DCP endpoints. See Outbound ports.
  • Resolve Snowflake DNS names. See DNS.
  • Reach the private data source (host and port) from the agent host. See Deploy the agent.

Configure Openflow to use the proxy¶

After the proxy object is enabled and the agent is connected, complete the following steps to ensure Openflow connectors can connect to your data source. To verify the agent is connected, see Verify the agent is connected.

The following diagram shows where each component runs and how the connection is established:

Create a network rule for your data source¶

This network rule of mode DATA_CONNECTIVITY_PROXY_EGRESS ensures that any traffic to the specified TARGET_HOSTNAME:TARGET_PORT is routed through DCP and not through public internet gateways. You can specify multiple hostname and port combinations in a network rule.

CREATE NETWORK RULE IF NOT EXISTS <DCP_NETWORK_RULE>
  MODE = DATA_CONNECTIVITY_PROXY_EGRESS
  TYPE = HOST_PORT
  VALUE_LIST = ('<TARGET_HOSTNAME>:<TARGET_PORT>');

Create an external access integration for your data source¶

An external access integration provides secure access to external resources for Openflow runtimes and connectors. You attach it to Openflow runtimes, and its ALLOWED_NETWORK_RULES determine which endpoints are accessible through the integration.

CREATE EXTERNAL ACCESS INTEGRATION IF NOT EXISTS <DCP_EAI_NAME>
  ALLOWED_NETWORK_RULES = (<DCP_NETWORK_RULE>)
  ENABLED = TRUE;

Grant the execute-as role and Openflow admin access to the integration¶

Grant USAGE on the DCP external access integration to the execute-as role associated with the runtime:

GRANT USAGE ON INTEGRATION <DCP_EAI_NAME> TO ROLE <OPENFLOW_EXECUTE_AS_ROLE>;

To allow users to associate the DCP external access integration with a runtime, grant USAGE to the role that users use to create or update runtimes:

GRANT USAGE ON INTEGRATION <DCP_EAI_NAME> TO ROLE <OPENFLOW_ADMIN>;

Attach the external access integration to the runtime¶

The final step to allow the Openflow runtime and connector to use the DCP tunnel is to attach the same external access integration you associated with the network rule for your source system.

To attach it in the Openflow UI:

1. Sign in to Snowsight.
2. In the navigation menu, select Ingestion » Openflow.
3. Select Launch Openflow.
4. Navigate to the runtime listing to view your existing runtimes, or create one as described in Set up Openflow - Snowflake Deployment: Create runtime.
5. In the runtime list view, select next to the runtime » External access integrations. Pick the DCP external access integration and select Save. You can select more than one integration if the runtime also needs allowlisted public destinations.

On a gen 2 runtime, you can do the same thing in SQL. To add the integration to an existing runtime:

ALTER OPENFLOW RUNTIME <DATABASE>.<SCHEMA>.<RUNTIME_NAME>
  ADD EXTERNAL_ACCESS_INTEGRATIONS = (<DCP_EAI_NAME>);

To create a runtime with the integration already attached:

CREATE OPENFLOW RUNTIME <DATABASE>.<SCHEMA>.<RUNTIME_NAME>
  IN DEPLOYMENT <DEPLOYMENT_NAME>
  NODE_TYPE = MEDIUM
  MIN_NODES = 1
  MAX_NODES = 1
  EXECUTE_AS_ROLE = <OPENFLOW_EXECUTE_AS_ROLE>
  EXTERNAL_ACCESS_INTEGRATIONS = (<DCP_EAI_NAME>);

Adding or removing integrations with ALTER OPENFLOW RUNTIME requires OWNERSHIP on the runtime. To confirm which integrations are attached, run DESCRIBE OPENFLOW RUNTIME and check the external_integrations column.

Configure the connector with the private DNS name¶

Set the connector’s source host and port to the same fully qualified hostname you listed in the DCP network rule. Don’t substitute a Snowflake hostname, a proxy URL, or the source’s private IP.

The host and port have to match the network rule exactly. The rest of the connection string works as usual, so keep the scheme and any parameters your source requires. For example, if the network rule lists mydb.internal.corp:5432, a PostgreSQL connector can use jdbc:postgresql://mydb.internal.corp:5432/mydb?sslmode=require.

Then run the connector’s usual connectivity check from the Openflow UI. A successful check confirms the path from the Snowflake-hosted runtime through the DCP tunnel to the source.

Start your connector and validate that data is being ingested successfully.

Considerations¶

• The agent must be able to resolve the hostname you use in the network rule and in the connector. If you don’t have DNS for the target, add a host entry on the agent host (not in Snowflake), as described in Create a network rule for the private destination.
• Enable the Data Connectivity Proxy object before you expect connector traffic. If the object exists but is disabled, the agent can appear to connect and then retry with an error that the agent isn’t known or is disabled. See Agent shows disconnected or down health.
• Deploy more than one agent in separate failure domains if you need redundancy. Connector checkpoints remain in Openflow if you replace an agent. See High availability.
• Rotate the bootstrap credential before it expires. Rotation is a DCP agent operation; you don’t reconfigure the Openflow connector when the token changes. See Rotate the bootstrap token without downtime.

Troubleshooting Openflow connectivity¶

If the connector can’t reach the source after the agent is running, check the Openflow side first:

For agent startup errors, bootstrap JWT issues, outbound port 443, DNS to Snowflake, and Prometheus metrics, see Troubleshoot Data Connectivity Proxy and Monitor Data Connectivity Proxy.