Setting up the Openflow Connector for Shopify¶

This topic describes the steps to set up the Openflow Connector for Shopify.

Prerequisites¶

1. Review About the Openflow Connector for Shopify.

2. Set up your runtime deployment.

   • Set up Openflow - BYOC
   • Set up Openflow - Snowflake Deployment - Task overview

3. If you are using Openflow - Snowflake Deployments, ensure that you have reviewed the required domain configuration and have granted access to the domains required by the connector.

4. Ensure you have access to the Openflow admin role or a similar role you use to manage Openflow.

5. If you’re creating a Snowflake service user to manage the connector, set up key pair authentication. For more information, see key pair authentication.

Required endpoints¶

The following endpoint is required for the connector to function:

• <your_store>.myshopify.com:443 (for example, mystore.myshopify.com:443)

If you are using Openflow - BYOC Deployments, configure your cloud network egress to allow HTTPS (port 443) access to this endpoint. If you are using Openflow - Snowflake Deployments, you must create a network rule and an external access integration (EAI). For more information, see Create a network rule (Openflow - Snowflake Deployments only).

Set up Shopify¶

A Shopify store administrator must create a custom app and generate an Admin API access token for the connector to authenticate.

1. Log in to your Shopify admin at https://admin.shopify.com/store/<your_store>.

2. Navigate to Settings » Apps » Develop apps.

3. Select Create an app. Provide an app name and select a developer.

4. On the Configuration tab, select Configure Admin API scopes.

5. Select the read_* scopes corresponding to the object types you want to replicate. For example:

   • read_orders: orders, transactions, fulfillments (access limited to the last 60 days by default; request the read_all_orders scope in the Partner Dashboard to access the full order history)
   • read_products: products, product variants, collections
   • read_customers: customers, segments
   • read_inventory: inventory items, locations
   • read_merchant_managed_fulfillment_orders: fulfillment orders

   For the full list of available scopes, see the Shopify access scopes reference.

   Note

   Grant only the scopes required for the objects you intend to replicate. You can update scopes later by editing the app configuration.

   Some GraphQL fields require write scopes to read (for example, marketingUnsubscribeUrl on the Customer object requires write_customers). If you don’t grant the corresponding write scope, the Shopify API returns an error for that field. To avoid this, either omit the field from the graphqlFields list in the Object Definitions Override parameter, or add it to ignoredFields. Note that ignoredFields works on top-level field names only. For nested fields, you must remove them from the graphqlFields sub-selection.

6. Select Save.

7. Go to the API credentials tab and select Install app. Confirm the installation.

8. Copy the Admin API access token and store it in a secure location. You need this token when configuring the connector.

For more information, see Generate access tokens for custom apps in the Shopify developer documentation.

Set up your Snowflake account¶

As an Openflow administrator, perform the following tasks to set up your Snowflake account.

Create a Snowflake service user (Openflow - BYOC Deployments only)¶

1. Create a service user:

   Copy codeExpand code block

   USE ROLE USERADMIN;
   CREATE USER <openflow_service_user>
     TYPE=SERVICE
     COMMENT='Service user for the Shopify connector';

   Show lessSee more

   Scroll to top

2. Store the private key for that user in a file to supply to the connector’s configuration. For more information, see key pair authentication.

   Copy codeExpand code block

   ALTER USER <openflow_service_user> SET RSA_PUBLIC_KEY = '<pubkey>';

   Show lessSee more

   Scroll to top

Create database, schema, and warehouse¶

1. Create the destination database:

   Copy codeExpand code block

   USE ROLE ACCOUNTADMIN;
   CREATE DATABASE IF NOT EXISTS <shopify_database>;

   Show lessSee more

   Scroll to top

2. Create the destination schema:

   Copy codeExpand code block

   CREATE SCHEMA IF NOT EXISTS <shopify_database>.<shopify_schema>;

   Show lessSee more

   Scroll to top

3. Create a role for the connector and grant the required privileges:

   Copy codeExpand code block

   CREATE ROLE IF NOT EXISTS <shopify_connector_role>;
   
   GRANT USAGE ON DATABASE <shopify_database> TO ROLE <shopify_connector_role>;
   GRANT USAGE ON SCHEMA <shopify_database>.<shopify_schema> TO ROLE <shopify_connector_role>;
   GRANT CREATE TABLE ON SCHEMA <shopify_database>.<shopify_schema> TO ROLE <shopify_connector_role>;

   Show lessSee more

   Scroll to top

4. Create a warehouse (or use an existing one) and grant usage privileges:

   Copy codeExpand code block

   CREATE WAREHOUSE IF NOT EXISTS <shopify_warehouse>
     WITH
     WAREHOUSE_SIZE = 'SMALL'
     AUTO_SUSPEND = 300
     AUTO_RESUME = TRUE;
   
   GRANT USAGE, OPERATE ON WAREHOUSE <shopify_warehouse> TO ROLE <shopify_connector_role>;

   Show lessSee more

   Scroll to top

5. If using Openflow - BYOC Deployments, assign the role to the service user:

   Copy codeExpand code block

   GRANT ROLE <shopify_connector_role> TO USER <openflow_service_user>;
   ALTER USER <openflow_service_user> SET DEFAULT_ROLE = <shopify_connector_role>;

   Show lessSee more

   Scroll to top

Create a network rule (Openflow - Snowflake Deployments only)¶

To allow the connector to call the Shopify API from a Snowflake-hosted runtime, create a network rule and an external access integration (EAI), and then grant the Snowflake role usage privileges on the EAI.

1. Create a network rule:

   Copy codeExpand code block

   USE ROLE ACCOUNTADMIN;
   
   CREATE OR REPLACE NETWORK RULE openflow_<runtime_name>_shopify_network_rule
     TYPE = HOST_PORT
     MODE = EGRESS
     VALUE_LIST = ('<your_store>.myshopify.com:443');

   Show lessSee more

   Scroll to top

2. Create an External Access Integration:

   Copy codeExpand code block

   CREATE OR REPLACE EXTERNAL ACCESS INTEGRATION openflow_<runtime_name>_shopify_eai
     ALLOWED_NETWORK_RULES = (openflow_<runtime_name>_shopify_network_rule)
     ENABLED = TRUE;

   Show lessSee more

   Scroll to top

3. Grant your Snowflake role USAGE on the integration:

   Copy codeExpand code block

   GRANT USAGE ON INTEGRATION openflow_<runtime_name>_shopify_eai
     TO ROLE openflow_runtime_role_<runtime_name>;

   Show lessSee more

   Scroll to top

Install the connector¶

Configure the connector¶

To configure the connector, perform the following steps:

1. Right-click on the added connector process group and select Parameters.
2. Populate the required parameter values as described in the following sections.

Shopify parameters¶

Snowflake destination parameters¶

Run the flow¶

1. Right-click on an empty area of the canvas and select Enable all Controller Services.
2. Right-click on the connector process group and select Start.

The connector starts querying the Shopify Admin API and loading data into Snowflake.

Next steps¶

• For information on customizing which fields are extracted and registering custom object types, see Object definition overrides for the Openflow Connector for Shopify.
• For information on forcing a full reload of connector state, see Maintaining the Openflow Connector for Shopify.
• For information on monitoring the flow, see Monitor the flow.