Set up the Openflow Connector for SharePoint¶

Prerequisites¶

1. Ensure that you have reviewed Openflow Connector for SharePoint.

2. Ensure that you have set up Openflow.

Get the credentials¶

As an SharePoint administrator, perform the following actions:

1. Ensure that you have a Microsoft Graph application with the following Microsoft Graph permissions:

   1. Sites.Selected: limits access only to specified sites.

   2. Files.SelectedOperations.Selected: limits access only to files in specified sites.

   3. GroupMember.Read.All: used for resolving SharePoint group permissions.

   4. User.ReadBasic.All: used for resolving Microsoft365 user emails.

2. Configure SharePoint to enable OAuth authentication as described in Get access without a user. The connector uses the following Microsoft Graph APIs to fetch data from SharePoint:

   1. Download driveItem content

   2. Get driveItem metadata

   3. driveItem: delta

   4. List driveItem permissions

   5. group: delta

   6. List group members

   7. Get user

3. Get the site URL of your Microsoft 365 SharePoint site with files or folders that you want to ingest into Snowflake and the credentials from your Azure or Office 365 account administrator.

Set up Snowflake account¶

As a Snowflake account administrator, perform the following tasks manually or by using the script included below:

1. Create a new role or use an existing role and grant the Database privileges.

2. Create a new Snowflake service user with the type as SERVICE.

3. Grant the Snowflake service user the role you created in the previous steps.

4. Configure with key-pair auth for the Snowflake SERVICE user from step 2.

5. Snowflake strongly recommends this step. Configure a secrets manager supported by Openflow, for example, AWS, Azure, and Hashicorp, and store the public and private keys in the secret store.

   Note

   If for any reason, you do not wish to use a secrets manager, then you are responsible for safeguarding the public key and private key files used for key-pair authentication according to the security policies of your organization.

   1. Once the secrets manager is configured, determine how you will authenticate to it. On AWS, it’s recommended that you the EC2 instance role associated with Openflow as this way no other secrets have to be persisted.

   2. In Openflow, configure a Parameter Provider associated with this Secrets Manager, from the hamburger menu in the upper right. Navigate to Controller Settings » Parameter Provider and then fetch your parameter values.

   3. At this point all credentials can be referenced with the associated parameter paths and no sensitive values need to be persisted within Openflow.

6. If any other Snowflake users require access to the raw ingested documents and tables ingested by the connector (for example, for custom processing in Snowflake), then grant those users the role created in step 1.

7. Designate a warehouse for the connector to use. Start with the smallest warehouse size, then experiment with size depending on the number of tables being replicated, and the amount of data transferred. Large table numbers typically scale better with multi-cluster warehouses, rather than larger warehouse sizes.

Use case 1: Use the connector to ingest files only¶

Use the connector to:

• Perform custom processing on ingested files

• Ingest Sharepoint files and permissions and keep them up to date

Use case 2: Use the connector definition to ingest files and perform processing with Cortex¶

Use the predefined flow definition to:

• Create AI assistants for public documents within your organization’s SharePoint site.

• Enable your AI assistants to adhere to access controls specified in your organization’s SharePoint site.

Use case 3: Customise the connector definition¶

Customize the connector definition to:

• Process the ingested files with Document AI.

• Perform custom processing on ingested files.

Enabling Sharepoint site groups¶

Query the Cortex Search service¶

You can use the Cortex Search service to build chat and search applications to chat with or query your documents in SharePoint.

After you install and configure the connector and it begins ingesting content from Sharepoint, you can query the Cortex Search service. For more information about using Cortex Search, see Query a Cortex Search service.

Filter responses

To restrict responses from the Cortex Search service to documents that a specific user has access to in SharePoint, you can specify a filter containing the user ID or email address of the user when you query Cortex Search. For example, filter.@contains.user_ids or filter.@contains.user_emails. The name of the Cortex Search service created by the connector is search_service in the schema Cortex.

Run the following SQL code in a SQL worksheet to query the Cortex Search service with files ingested from your SharePoint site.

Replace the following:

• application_instance_name: Name of your database and connector application instance.

• user_emailID: Email ID of the user who you want to filter the responses for.

• your_question: The question that you want to get responses for.

• number_of_results: Maximum number of results to return in the response. The maximum value is 1000 and the default value is 10.

SELECT PARSE_JSON(
  SNOWFLAKE.CORTEX.SEARCH_PREVIEW(
    '<application_instance_name>.cortex.search_service',
      '{
        "query": "<your_question>",
         "columns": ["chunk", "web_url"],
         "filter": {"@contains": {"user_emails": "<user_emailID>"} },
         "limit": <number_of_results>
       }'
   )
)['results'] AS results

Here’s a complete list of values that you can enter for columns:

Example: Query an AI assistant for human resources (HR) information

You can use Cortex Search to query an AI assistant for employees to chat with the latest versions of HR information, such as onboarding, code of conduct, team processes, and organization policies. Using response filters, you can also allow HR team members to query employee contracts while adhering to access controls configured in SharePoint.

SELECT PARSE_JSON(
     SNOWFLAKE.CORTEX.SEARCH_PREVIEW(
          '<application_instance_name>.cortex.search_service',
          '{
             "query": "What is my vacation carry over policy?",
             "columns": ["chunk", "web_url"],
             "filter": {"@contains": {"user_emails": "<user_emailID>"} },
             "limit": 1
          }'
     )
 )['results'] AS results

import snowflake.snowpark as snowpark
from snowflake.snowpark import Session
from snowflake.core import Root

def main(session: snowpark.Session):

   root = Root(session)

   # fetch service
   my_service = (root
     .databases["<application_instance_name>"]
     .schemas["cortex"]
     .cortex_search_services["search_service"]
   )

   # query service
   resp = my_service.search(
     query="What is my vacation carry over policy?",
     columns = ["chunk", "web_url"],
     filter = {"@contains": {"user_emails": "<user_emailID>"} },
     limit=1
   )
   return (resp.to_json())

curl --location "https://<account_url>/api/v2/databases/<application_instance_name>/schemas/cortex/cortex-search-services/search_service" \
     --header 'Content-Type: application/json' \
     --header 'Accept: application/json' \
     --header "Authorization: Bearer <CORTEX_SEARCH_JWT>" \
     --data '{
         "query": "What is my vacation carry over policy?",
         "columns": ["chunk", "web_url"],
         "limit": 1
     }'

{
  "results" : [ {
  "web_url" : "https://<domain>.sharepoint.com/sites/<site_name>/<path_to_file>",
  "chunk" : "Answer to the question asked."
  } ]
}