Set up the Openflow Connector for HubSpot¶

Prerequisites¶

1. Ensure that you have reviewed About Openflow Connector for HubSpot.

2. Ensure that you have Set up Openflow - BYOC or Set up Openflow - Snowflake Deployment - Task overview.

Get the credentials¶

As a HubSpot administrator, generate a HubSpot private app token or create one in your HubSpot account. This lets you authenticate your requests to the HubSpot API.

1. Log in to your HubSpot account.

2. Navigate to Settings by selecting the gear icon in the top navigation bar.

3. In the left navigation, go to Integrations » Private Apps.

4. Select Create a private app.

   1. Enter a name for your app.

   2. Navigate to the Scopes tab.

   3. Select the scopes required for the API requests you intend to make. To find scopes required for the API requests, see Scopes.

   4. Select Create app.

   5. Set the required scopes for the API requests you intend to make for each endpoint.

5. Select View access token to view the access token. Paste the token in the connector parameters, or save it securely.

Set up Snowflake account¶

As a Snowflake account administrator, perform the following tasks:

1. Create a new role or use an existing role and grant the Database privileges and View privileges.

2. Create a new Snowflake service user with the type as SERVICE.

3. Grant the Snowflake service user the role you created in the previous steps.

4. Configure with key-pair auth for the Snowflake SERVICE user from step 2.

5. Snowflake strongly recommends this step. Configure a secrets manager supported by Openflow, for example, AWS, Azure, and Hashicorp, and store the public and private keys in the secret store.

   Note

   If for any reason, you do not wish to use a secrets manager, then you are responsible for safeguarding the public key and private key files used for key-pair authentication according to the security policies of your organization.

   1. Once the secrets manager is configured, determine how you will authenticate to it. On AWS, it’s recommended that you the EC2 instance role associated with Openflow as this way no other secrets have to be persisted.

   2. In Openflow, configure a Parameter Provider associated with this Secrets Manager, from the hamburger menu in the upper right. Navigate to Controller Settings » Parameter Provider and then fetch your parameter values.

   3. At this point all credentials can be referenced with the associated parameter paths and no sensitive values need to be persisted within Openflow.

6. If any other Snowflake users require access to the raw ingested documents and tables ingested by the connector (for example, for custom processing in Snowflake), then grant those users the role created in step 1.

7. Create a database and schema in Snowflake for the connector to store ingested data. Grant the following Database privileges to the role created in the first step.

   CREATE DATABASE hubspot_destination_db;
   CREATE SCHEMA hubspot_destination_db.hubspot_destination_schema;
   GRANT USAGE ON DATABASE hubspot_destination_db TO ROLE <hubspot_connector_role>;
   GRANT USAGE ON SCHEMA hubspot_destination_db.hubspot_destination_schema TO ROLE <hubspot_connector_role>;
   GRANT CREATE TABLE, CREATE VIEW ON SCHEMA hubspot_destination_db.hubspot_destination_schema TO ROLE <hubspot_connector_role>;
   

   Copy

8. Create a warehouse that will be used by the connector or use an existing one. Start with the smallest warehouse size, then experiment with size depending on the number of tables being replicated, and the amount of data transferred. Large table numbers typically scale better with multi-cluster warehouses, rather than larger warehouse sizes.

9. Ensure that the user with role used by the connector has the required privileges to use the warehouse. If that’s not the case then grant the required privileges to the role.

   CREATE WAREHOUSE hubspot_connector_warehouse WITH WAREHOUSE_SIZE = 'X-Small';
   GRANT USAGE ON WAREHOUSE hubspot_connector_warehouse TO ROLE <hubspot_connector_role>;
   

   Copy

Set up the connector¶

As a data engineer, perform the following tasks to install and configure the connector:

Run the flow¶

1. Right-click on the plane and select Enable all Controller Services.

2. Right-click on the imported process group and select Start. The connector starts the data ingestion.

Data structure and views¶

The connector stores data in the following two formats within your Snowflake database: