Set up Openflow¶
This topic describes the steps to set up Openflow.
Setting up Openflow involves the following steps:
Prerequisites¶
The prerequisites to be completed on your Snowflake and AWS accounts are as follows:
Snowflake account¶
If you’ve never used image repositories in your Snowflake account, you need to set the following to make sure the deployment service can pull the Openflow images from Snowflake.
USE ROLE ACCOUNTADMIN; CREATE DATABASE IF NOT EXISTS OPENFLOW; USE OPENFLOW; CREATE SCHEMA IF NOT EXISTS OPENFLOW; USE SCHEMA OPENFLOW; CREATE IMAGE REPOSITORY IF NOT EXISTS OPENFLOW; grant usage on database OPENFLOW to role public; grant usage on schema OPENFLOW to role public; grant read on image repository OPENFLOW.OPENFLOW.OPENFLOW to role public;
Setting up Openflow requires defining new privileges at the Snowflake Account level. The new privileges are assigned to the ACCOUNTADMIN role as part of the default set of privileges. ACCOUNTADMIN will automatically have the following two privileges and will be able to grant them to a role of their choosing for the Openflow admin role, denoted as $OPENFLOW_ADMIN_ROLE in the following code:
USE ROLE ACCOUNTADMIN; GRANT CREATE OPENFLOW DATA PLANE INTEGRATION ON ACCOUNT TO ROLE $OPENFLOW_ADMIN_ROLE; GRANT CREATE OPENFLOW RUNTIME INTEGRATION ON ACCOUNT TO ROLE $OPENFLOW_ADMIN_ROLE;
Additionally, you need to set
default_secondary_rolestoALLfor all Openflow users. This is because Openflow actions are authorized by using any of the authenticated user’s roles and not just the default role. Replace $OPENFLOW_USER in the following code for each Openflow user:USE ROLE ACCOUNTADMIN; ALTER USER $OPENFLOW_USER SET DEFAULT_SECONDARY_ROLES = ('ALL');
Deployment integration privileges¶
The deployment integration object represents a set of resources provisioned to deploy one or more Snowflake Openflow runtimes. For organizations bringing their own cloud resources, the deployment integration object represents a managed Kubernetes cluster along with its associated nodes.
Users with the CREATE DATA PLANE INTEGRATION privilege on the Snowflake account can create and delete the deployment integration objects.
Additional privileges can be defined on deployment integration objects directly to support differentiation of access.
You can grant the following privileges on a deployment integration object:
OWNERSHIP: Enables full control over deployment actions objects, including deletion of the deployment.
USAGE: Enables creation of runtime child objects.
Runtime privileges¶
The runtime object represents a cluster of one or more Snowflake Openflow runtime servers, provisioned to run flow definitions. For Kubernetes deployments, the runtime object represents a stateful set of Snowflake Openflow Runtime containers deployed in a namespace, along with supporting components.
Users with the OWNERSHIP privilege on the parent deployment integration object and the CREATE RUNTIME INTEGRATION account-level privilege can create runtime integration objects. Additional privileges can be defined on runtime integration objects directly to support differentiation of access.
You can grant the following privileges on a runtime integration object:
OWNERSHIP: Enables full control over runtime actions, including deletion of the associated runtime and modification of runtime flow definitions.
USAGE: Enables read access to the deployed Runtime for observing health and status, without making any changes.
Example for role setup¶
Let’s consider the following scenario where we want the following roles to be set up:
accountadmin: Out-of-the box role from Snowflake, which will have these two CREATE privileges:
CREATE OPENFLOW DATA PLANE INTEGRATION
CREATE OPENFLOW RUNTIME INTEGRATION
deployment_manager: a role that can create, manage, and delete deployments
deployment1_runtime_manager_1: a role that can create a runtime only within deployment 1. it can modify and delete runtime within deployment 1 that it created but not runtime created by deployment1_runtime_manager_2.
deployment1_runtime_manager_2: a role that can create a runtime only within deployment 1. it can modify and delete a runtime within deployment 1 that it created but not runtime created by deployment1_runtime_manager_1.
deployment1_runtime_viewer_1: a role that can view runtime canvas within deployment 1 that’s created by deployment1_runtime_manager_1.
deployment1_runtime_viewer_2: a role that can view runtime canvas within deployment 1 that’s created by deployment1_runtime_manager_2.
deployment2_runtime_manager: a role that can create a runtime only within deployment 2.
deployment2_runtime_viewer: a role that can view a runtime canvas within deployment 2.
To set up Openflow with these roles, do the following:
Create new roles and assign the relevant privileges.
use role accountadmin; create role if not exists deployment_manager; create role if not exists deployment1_runtime_manager_1; create role if not exists deployment1_runtime_manager_2; create role if not exists deployment1_runtime_viewer_1; create role if not exists deployment1_runtime_viewer_2; create role if not exists deployment2_runtime_manager; create role if not exists deployment2_runtime_viewer; -- assign create deployment privilege to roles, since this privilege cannot be granted in openflow ui grant create openflow data plane integration on account to role deployment_manager; -- assign create runtime privilege to roles, since this privilege cannot be granted in cp ui grant create openflow runtime integration on account to role deployment1_runtime_manager_1; grant create openflow runtime integration on account to role deployment1_runtime_manager_2; grant create openflow runtime integration on account to role deployment2_runtime_manager; -- grant these roles to desired users grant role <role name> to user <username>; ........
Create a deployment.
Log in as deployment_manager.
Create deployment 1 in the Openflow UI. Grant the USAGE privilege to deployment1_runtime_manager_1 and deployment1_runtime_manager_2.
Create deployment 2 in the Openflow UI. Grant the USAGE privilege to deployment2_runtime_manager.
You need the CREATE OPENFLOW RUNTIME INTEGRATION privilege as well as the USAGE privilege on a deployment to be able to create a runtime in that deployment.
Create runtimes in deployment 1 as deployment1_runtime_manager_1.
Log in as deployment1_runtime_manager_1.
Create a runtime as decribed in the following sections. deployment1_runtime_manager_1 should be able to create runtimes and manage any runtimes it created within this deployment.
In the Openflow UI, select deployment1_runtime_viewer_1 and grant it the USAGE privilege.
AWS account¶
Ensure the following on your AWS account:
You have an AWS account with permissions required to create a CloudFormation stack.
You have an AWS administrator in your organization with the ability to execute CloudFormation script to set up EKS inside a new VPC (created by CloudFormation) or an existing VPC. See Prerequisites for BYO-VPC (existing VPC).
Note
To learn about how the Openflow installation happens in your AWS account and the permissions that are configured by the CloudFormation template, see Installation process.
Prerequisites for BYO-VPC (existing VPC)¶
If you want to use an existing VPC and your own private subnets, ensure that you have the following:
One VPC
One internet gateway attached to the VPC
Two public subnets in different availability zones:
The public subnets should be at least a /27 CIDR range with 32 IPs for public subnets.
Both public subnets should have routes for destination 0.0.0.0/0 and target internet gateway.
The public subnets must be in different availability zones because the AWS Network Load Balancer requires high availability.
The public subnets must be tagged as follows for the AWS Load Balancer to be created:
Key: kubernetes.io/role/elb
Value: 1
One NAT gateway with a “Public” connectivity type attached to a public subnet.
Two private subnets in different availability zones.
The subnets should be at least a /24 CIDR range with 256 available IPs to allocate. This limits the number and scale of runtimes you can create, so it may be more appropriate to use a larger range for the deployment.
Each private subnet needs a route table entry for 0.0.0.0/0 to the NAT gateway in the public subnet.
These subnets must be in different availability zones, which is a prerequisite for EKS provisioning.
Here are some points to bear in mind:
The Openflow deployment agent runs in Private Subnet 1, which needs to be bound to an internet gateway. Otherwise, it will not initialize or set up properly and no infrastructure will be provisioned.
Without a public NAT gateway that’s routable from the private subnets, resources such as the EKS cluster will not have internet access. Furthermore, the deployment agent will not be able to install the Openflow deployment.
Accept the Openflow terms of service¶
This step is only required once for your organization.
Sign in to Snowflake as a user with the ORGADMIN role.
Navigate to Data » Openflow.
Accept Openflow terms of services.
Create a deployment in your cloud¶
Important
Users cannot sign in to Openflow if their default role is ACCOUNTADMIN, ORGADMIN, GLOBALORGADMIN, or SECURITYADMIN. You must change the default role for your user to a role other than ACCOUNTADMIN, ORGADMIN, GLOBALORGADMIN, or SECURITYADMIN to log in to Openflow. For more information, see Prerequisites.
Sign in to Snowflake with a role that your ACCOUNTADMIN assigned for using Openflow.
Navigate to Data » Openflow.
Select Launch Openflow.
In the Openflow UI, select Create a deployment. The Deployments tab opens.
Select Create a deployment. The Creating a deployment wizard opens.
In the Prerequisites step, ensure that you meet all the requirements. Select Next.
In the Deployment location step, select Amazon Web Services as the deployment location. Enter a name for your deployment. Select Next.
In the Configuration step, you can select one of the following configuration options:
Fully managed VPC: Choose this option if you want your VPC to be managed by Snowflake
Bring your own VPC: Choose this option if you want to use an existing VPC.
In the PrivateLink step, you can select if you want to establish communication with Snowflake over the private link. Enabling this option requires additional setup in your AWS account.
Select Create Deployment.
Once your deployment is configured, a dialog box appears that lets you download the CloudFormation template to complete the setup process in your AWS account. Download this template and navigate to your AWS account.
In your AWS account, create a new CloudFormation Stack using the template. After the Openflow deployment agent’s EC2 instance is created, it completes the rest of the Installation process using infrastructure as code scripts. You can track the installation progress as described in Track the installation progress.
If you’re using an existing VPC, upon uploading the CloudFormation template, select the respective values in the drop-downn lists for the two private subnets and your VPC.
Navigate to your Snowflake account.
[Optional] This step is required only if you’re using network policies to control access to Snowflake. Create a network rule for Openflow and add it to your existing network policy for your Snowflake account. A network policy is a set of rules that control which IP addresses can access your Snowflake account. Replace {$NAT_GATEWAY_PUBLIC_IP} in the following code snippet with the NAT gateway public IP address that was created as part of the CloudFormation stack (search for NAT Gateway on AWS console or check the output of the CFN stack). NAT gateway is responsible for Openflow egress for both DPA and EKS. Both DPA and EKS run in the Private Subnet 1 of the installation.
USE ROLE ACCOUNTADMIN; USE DATABASE {REPLACE_WITH_YOUR_DB_NAME}; CREATE NETWORK RULE allow_openflow_deployment MODE = INGRESS TYPE = IPV4 VALUE_LIST = ('{$NAT_GATEWAY_PUBLIC_IP}/32'); -- Run this command to find your currently active network policy and copy the value column SHOW PARAMETERS LIKE 'NETWORK_POLICY' IN ACCOUNT; -- Now add the new network rule to this policy ALTER NETWORK POLICY {ENTER_YOUR_ACTIVE_NETWORK_POLICY_NAME} ADD ALLOWED_NETWORK_RULE_LIST = (allow_openflow_deployment);
Create an event table. An event table is designed to store and manage event data, such as user actions, system logs, and transactions. You can choose to use an Openflow-specific event table or an account-specific event table.
Create an Openflow-specific event table
USE ROLE accountadmin; GRANT create event table on schema OPENFLOW.OPENFLOW to role $ROLE_OF_DEPLOYMENT_OWNER; USE ROLE $ROLE_OF_DEPLOYMENT_OWNER; CREATE event table if not exists openflow.openflow.openflow_events; -- Find the Data Plane Integrations SHOW openflow data plane integrations; ALTER openflow data plane integration $OPENFLOW_deployment_UUID SET event_table = 'openflow.openflow.openflow_events';
Create an account-specific event table
USE database openflow; CREATE schema if not exists openflow.telemetry; CREATE event table if not exists openflow.telemetry.events; ALTER ACCOUNT SET EVENT_TABLE = openflow.telemetry.events;
Navigate to Openflow UI. Creating a deployment takes about 45 minutes on AWS. Once it’s created, you can view your deployment in the Deployments tab of Openflow UI with its state marked as Active.
Create a runtime environment in your cloud¶
In Openflow Control Plane, select Create a runtime. The Create Runtime dialog box appears.
From the Deployment drop-down list, choose the deployment in which you want to create a runtime.
Enter a name for your runtime.
Choose a node type from the Node type drop-down list. This specifies the size of your nodes.
In the Min/Max node range selector, select a range. The minimum value specifies the number of nodes that get created when the runtime starts and the maximum value specifies the number of nodes that the runtime can scale up to, in the event of high data volume or CPU load.
Select Create. The runtime takes a couple of minutes to get created.
Once created, you can view your runtime by navigating to the Runtimes tab of the Openflow control plane. Click the runtime to open the Openflow canvas.
Next step¶
Deploy a connector in a runtime. For a list of connectors available in Openflow, see Openflow connectors.
Networking considerations: Openflow EKS to source systems¶
The Openflow CloudFormation stack creates:
One VPC with two public subnets and two private subnets
Public subnets host the AWS Network Load Balancer, which is created later Private subnets host the EKS Cluster and all of the EC2 instances backing the node groups. Openflow Runtimes run within Private subnet 1.
NAT Gateway is currently the egress for both DPA and EKS. Both DPA and EKS run in the Private subnet 1 of the installation.
Configuring PrivateLink in AWS¶
Complete creating your new deployment and ensure that the PrivateLink option is enabled.
Retrieve Snowflake’s VPC endpoint service ID as follows:
Run the following SQL command:
SELECT SYSTEM$GET_PRIVATELINK_CONFIG()
From the output, read and save the ID of the VPC endpoint service maintained by Snowflake. This ID is located under the
privatelink-vpce-idkey.
Create a VPC endpoint with parameters:
Type: PrivateLink Ready partner services
Service: Service ID obtained in the previous step
VPC: The VPC of your deployment
Configure security group inbound rule as follows:
Add inbound rule to the security group associated with your new VPC endpoint. The rule should allow all inbound traffic from the security group assigned to your EKS cluster.
Use the naming convention:
eks-cluster-sg-<runtime-key>-*for the EKS cluster’s security group.
Set up Route 53 private hosted zone with parameters:
Domain:
privatelink.snowflakecomputing.comType: Private hosted zone
Add a
CNAMErecord to your private hosted zone:Record name:
*.privatelink.snowflakecomputing.comRecord type:
CNAMEValue: DNS name of your VPC endpoint
Example: BYOC deployment with a new VPC to communicate with RDS in a different VPC of the same account¶
To enable communication between the Openflow EKS cluster and the RDS instance, you need to create a new security group, with the EKS cluster security group as the source for the inbound rule for RDS connectivity, and attach the group in RDS.
Find the EKS cluster security group, navigate to EKS and find your deployment key. You can also find it on the Openflow UI by performing the following steps:
Sign in to Openflow.
Go to the Deployments tab.
Select the More options icon next to your deployment.
Select View details. The value in the field Key is your deployment key.
After finding the deployment key, you can use it to filter your AWS resources by the key value.
Create a new security group that allows access over the relevant database port (5432 in the case of PostgreSQL by default) to the Openflow EKS cluster.
Attach it in RDS as a new security group.
If you need to troubleshoot, the Reachability Analyzer can be useful. It will give you detailed information about what may be blocking connectivity by using tracing capabilities within the AWS platform.
See the following AWS docs for accessing DB instances using VPC peering and the associated security group configuration:
Installation process¶
Between the CloudFormation stack and the Openflow Agent, there are several coordinated steps that the BYOC deployment installation process manages. The goal is to separate responsibilities between a cold-start that gives organizations an easy way to provide inputs to their BYOC deployment (solved via CloudFormation), and the configuration of the deployment and its core software components that will need to change over time (solved by the Openflow Agent).
The deployment Agent facilitates the creation of the Openflow deployment infrastructure and installation of the deployment software components including the deployment service. The deployment agent authenticates with Snowflake System Image Registry to obtain Openflow container images.
The steps are as follows:
Note
When using BYO-VPC, you will choose a VPC ID and two private subnet IDs from the template, and the CloudFormation stack will use the selected ones rather than creating the resources mentioned in steps 1a, 1b, and 1c.
The CloudFormation template creates the following and configures with the AWS permissions mentioned in Configured AWS permissions:
One VPC with two public subnets and two private subnets. Public subnets host the AWS Network Load Balancer (created later). Private Subnets host the EKS cluster and all of the EC2 instances backing the NodeGroups. Openflow runtimes run within a private subnet.
Internet Gateway for egress from the VPC
NAT Gateway for egress from the private subnets
AWS Secrets Manager entry for the OIDC configuration input by the user
IAM role and instance profile for the Openflow Agent to use from its EC2 instance
An EC2 instance for Openflow deploymnent agent, complete with a UserData script to automatically run the initialization process. This script sets environment variables for the Openflow deploymnent agent to use, derived from the input CloudFormation parameters.
EC2 Instance Connect endpoint for the Openflow deploymnent agent to upgrade the deployment when needed.
When using BYO-VPC, by default the CloudFormation stack will create an EC2 Instance Connect endpoint. However, this default behaviour can be modified. When using the managed VPC option, the CloudFormation stack will always create an EC2 Instance Connect endpoint.
The Instance Connect endpoint can be shared across many VPCs.
If a deployment is deleted, along with deleting the CloudFormation stack, it will also remove the endpoint. This would block access to other BYO-VPC agents if the endpoint is shared.
To add an EC2 Instance Connect endpoint, perform the following steps in your AWS account:
In the left navigation, navigate to VPC » Endpoints.
Select Create Endpoint.
Choose the endoint type as EC2 Instance Connect Endpoint.
Select a VPC. Leave all the security groups clear (not selected) to use the default VPC security group.
When selecting a subnet, use the same value as Private Subnet 1 in the CloudFormation parameters.
Select Create. It takes apporximately 5 minutes for the endpoint to be created.
S3 Bucket that stores the Terraform state, logs, and outputs for the Openflow Agent
The Openflow deployment agent creates the following:
An EKS cluster containing:
Node groups
Autoscaling groups
VPC CNI add-on
EBS CSI add-on
Secrets manager records for PostgreSQL, OAuth credentials, and so on.
IAM policies and roles for various K8s service accounts to retrieve their secrets from AWS Secrets Manager.
K8s components
Namespaces
Cluster autoscaler
EBS CSI expandable storage
AWS Load Balancer Controller, which creates the publicly accessible Network Load Balancer
Let’s Encrypt certificate issuer
Nginx Ingress, configured for Let’s Encrypt
Metrics Server
Certificate manager from Jetstack
Service accounts for Temporal, deployment service, and OIDC
Secrets stores for Temporal, deployment service, and OIDC
External secrets for Temporal and deployment service. The external secret for OIDC is created and managed by the runtime operator.
PostgreSQL
Temporal
Self-signed certificate issuer and ingress configuration for communications between runtime nodes
Openflow runtime operator
Openflow deployment service
By default, all AWS accounts have a quota of five Elastic IP addresses per region, because public (IPv4) internet addresses are a scarce public resource. Snowflake strongly recommends that you use Elastic IP addresses primarily for their ability to remap the address to another instance in the case of instance failure, and to use DNS hostnames for all other inter-node communication.
Track the installation progress¶
After the CloudFormation stack moves into the CREATE_COMPLETE state, the Openflow agent automatically creates the rest of the infrastructure.
There are a few steps that can take 10-15 minutes each, such as:
Creating the EKS cluster
Installing the EBS CSI add-on to the EKS cluster
Creating the RDS PostgreSQL database
Status reporting for the Openflow agent is not available yet. In the meantime, you can view logs on the Openflow agent to verify whether the BYOC deployment is ready for runtimes. To do this, perform the following steps:
In the EC2 instances list, locate the following two instances:
openflow-agent-{data-plane-key}: This is the Openflow agent that you will use to manage runtimes
{data-plane-key}-mgmt-group: This is a node in the BYOC deployment’s EKS cluster that runs an operator and other core software
Right-click on the openflow-agent-{data-plane-key} instance and select Connect.
Switch from EC2 Instance Connect to Connect using EC2 Instance Connect Endpoint. Leave the default EC2 Instance Connect Endpoint in place.
Click Connect. A new browser tab or window will appear with a command line interface.
Run the following command to tail the installation logs of the docker image that is configuring your deployment:
journalctl -xe -f -n 100 -u docker
Once the installation is complete, you’ll see the following output:
{timestamp} - app stack applied successfully {timestamp} - All resources applied successfully
Configured AWS permissions¶
This section lists the AWS permissions configured by Openflow BYOC stack based on the roles.
Note
{key} represents the deployment key that uniquely identifies cloud resources created and managed by Openflow for a particular deployment.
Administrative user
cloudformation and all of the following permissions.
IAM Role: openflow-agent-role-{key}
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"autoscaling:DescribeTags",
"ec2:DescribeImages",
"ec2:DescribeInstances",
"ec2:DescribeLaunchTemplates",
"ec2:DescribeLaunchTemplateVersions",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeTags",
"ec2:DescribeVolumes",
"ec2:DescribeVpcs",
"ec2:DescribeVpcAttribute",
"iam:GetRole",
"iam:GetOpenIDConnectProvider",
"ecr:GetAuthorizationToken",
"ec2:RunInstances",
"ec2:CreateLaunchTemplate",
"ec2:CreateSecurityGroup",
"ec2:CreateTags",
"ec2:DeleteTags"
],
"Resource": "*",
"Effect": "Allow"
},
{
"Condition": {
"StringLike": {
"aws:ResourceTag/Name": [
"{key}-oidc-provider"
]
}
},
"Action": [
"iam:CreateOpenIDConnectProvider",
"iam:DeleteOpenIDConnectProvider",
"iam:TagOpenIDConnectProvider"
],
"Resource": "arn:aws:iam::{Account_ID}:oidc-provider/oidc.eks.{Region}.amazonaws.com/id/*",
"Effect": "Allow"
},
{
"Action": [
"iam:DeletePolicy",
"iam:CreatePolicy",
"iam:GetPolicy",
"iam:GetPolicyVersion",
"iam:ListPolicyVersions"
],
"Resource": [
"arn:aws:iam::{Account_ID}:policy/dp-service-role-policy-{key}",
"arn:aws:iam::{Account_ID}:policy/oauth2-role-policy-{key}",
"arn:aws:iam::{Account_ID}:policy/temporal-service-role-policy-{key}",
"arn:aws:iam::{Account_ID}:policy/oidc-service-role-policy-{key}",
"arn:aws:iam::{Account_ID}:policy/dps-temporal-role-policy-{key}"
"arn:aws:iam::{Account_ID}:policy/dps-postgres-role-policy-{key}"
],
"Effect": "Allow"
},
{
"Action": [
"iam:UpdateAssumeRolePolicy",
"iam:PutRolePolicy",
"iam:ListInstanceProfilesForRole",
"iam:ListAttachedRolePolicies",
"iam:ListRolePolicies",
"iam:GetRolePolicy",
"iam:CreateRole",
"iam:AttachRolePolicy",
"iam:DeleteRole",
"iam:DeleteRolePolicy",
"iam:DetachRolePolicy",
"iam:TagRole"
],
"Resource": [
"arn:aws:iam::{Account_ID}:role/openflow-agent-role-{key}",
"arn:aws:iam::{Account_ID}:role/{key}-*",
"arn:aws:iam::{Account_ID}:role/dps-temporal-role-{key}",
"arn:aws:iam::{Account_ID}:role/dps-postgres-role-{key}",
"arn:aws:iam::{Account_ID}:role/dp-service-role-{key}",
"arn:aws:iam::{Account_ID}:role/oauth2-role-{key}",
"arn:aws:iam::{Account_ID}:role/oidc-service-role-{key}"
],
"Effect": "Allow"
},
{
"Action": [
"autoscaling:CreateOrUpdateTags",
"autoscaling:DeleteTags"
],
"Resource": "arn:aws:autoscaling:{Region}:{Account_ID}:autoScalingGroup:*:autoScalingGroupName/eks-{key}-*",
"Effect": "Allow"
},
{
"Condition": {
"StringLike": {
"aws:ResourceTag/Name": [
"{key}-EC2SecurityGroup-*",
"k8s-traffic-{key}-*",
"eks-cluster-sg-{key}-*",
"{key}-cluster-sg",
"postgres-{key}-sg"
]
}
},
"Action": [
"ec2:AuthorizeSecurityGroupEgress",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:RevokeSecurityGroupEgress",
"ec2:DeleteSecurityGroup",
"ec2:CreateTags",
"ec2:DeleteTags",
"ec2:CreateNetworkInterface",
"ec2:DeleteNetworkInterface"
],
"Resource": "arn:aws:ec2:{Region}:{Account_ID}:security-group/*",
"Effect": "Allow"
},
{
"Condition": {
"StringLike": {
"aws:ResourceTag/elbv2.k8s.aws/cluster": "{key}"
}
},
"Action": [
"ec2:AuthorizeSecurityGroupEgress",
"ec2:AuthorizeSecurityGroupIngress",
"ec2:RevokeSecurityGroupEgress",
"ec2:DeleteSecurityGroup",
"ec2:CreateTags",
"ec2:DeleteTags",
"ec2:CreateNetworkInterface",
"ec2:DeleteNetworkInterface"
],
"Resource": "arn:aws:ec2:{Region}:{Account_ID}:security-group/*",
"Effect": "Allow"
},
{
"Action": [
"ec2:CreateSecurityGroup"
],
"Resource": "arn:aws:ec2:{Region}:{Account_ID}:vpc/vpc-018d2da0fde903de4",
"Effect": "Allow"
},
{
"Condition": {
"StringLike": {
"ec2:ResourceTag/Name": "openflow-agent-{key}"
}
},
"Action": [
"ec2:AttachNetworkInterface"
],
"Resource": "arn:aws:ec2:{Region}:{Account_ID}:instance/*",
"Effect": "Allow"
},
{
"Condition": {
"StringLike": {
"aws:ResourceTag/Name": "{key}-*-group"
}
},
"Action": [
"ec2:DeleteLaunchTemplate"
],
"Resource": "arn:aws:ec2:{Region}:{Account_ID}:launch-template/*",
"Effect": "Allow"
},
{
"Action": [
"eks:CreateCluster",
"eks:CreateAccessEntry",
"eks:CreateAddon",
"eks:CreateNodegroup",
"eks:DeleteCluster",
"eks:DescribeCluster",
"eks:ListClusters",
"eks:ListNodeGroups",
"eks:DescribeUpdate",
"eks:UpdateClusterConfig",
"eks:TagResource"
],
"Resource": "arn:aws:eks:{Region}:{Account_ID}:cluster/{key}",
"Effect": "Allow"
},
{
"Action": [
"eks:DescribeAddon",
"eks:DescribeAddonVersions",
"eks:UpdateAddon",
"eks:DeleteAddon",
"eks:DescribeUpdate"
],
"Resource": "arn:aws:eks:{Region}:{Account_ID}:addon/{key}/*",
"Effect": "Allow"
},
{
"Action": [
"eks:DeleteNodegroup",
"eks:DescribeNodegroup",
"eks:ListNodegroups",
"eks:UpdateNodegroupConfig",
"eks:TagResource",
"eks:DescribeUpdate"
],
"Resource": "arn:aws:eks:{Region}:{Account_ID}:nodegroup/{key}/*",
"Effect": "Allow"
},
{
"Action": [
"s3:CreateBucket",
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::byoc-tf-state-{key}",
"Effect": "Allow"
},
{
"Action": [
"s3:DeleteObject",
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::byoc-tf-state-{key}/*",
"Effect": "Allow"
},
{
"Action": [
"secretsmanager:CreateSecret",
"secretsmanager:DeleteSecret",
"secretsmanager:DescribeSecret",
"secretsmanager:GetResourcePolicy",
"secretsmanager:GetSecretValue",
"secretsmanager:PutSecretValue",
"secretsmanager:UpdateSecretVersionStage"
],
"Resource": "arn:aws:secretsmanager:{Region}:{Account_ID}:secret:*-{key}*",
"Effect": "Allow"
},
{
"Action": [
"ecr:BatchCheckLayerAvailability",
"ecr:BatchGetImage",
"ecr:DescribeImages",
"ecr:DescribeRepositories",
"ecr:GetDownloadUrlForLayer",
"ecr:ListImages"
],
"Resource": "arn:aws:ecr:{Region}:{Account_ID}:*",
"Effect": "Allow"
},
{
"Action": [
"ecr:CreateRepository",
"ecr:CompleteLayerUpload",
"ecr:InitiateLayerUpload",
"ecr:PutImage",
"ecr:UploadLayerPart"
],
"Resource": "arn:aws:ecr:{Region}:{Account_ID}:repository/snowflake-openflow/*",
"Effect": "Allow"
},
{
"Condition": {
"StringLike": {
"iam:AWSServiceName": "eks.amazonaws.com"
}
},
"Action": [
"iam:CreateServiceLinkedRole"
],
"Resource": "arn:aws:iam::*:role/aws-service-role/eks.amazonaws.com/AWSServiceRoleForAmazonEKS",
"Effect": "Allow"
},
{
"Condition": {
"StringLike": {
"iam:AWSServiceName": "eks-nodegroup.amazonaws.com"
}
},
"Action": [
"iam:CreateServiceLinkedRole"
],
"Resource": "arn:aws:iam::*:role/aws-service-role/eks-nodegroup.amazonaws.com/AWSServiceRoleForAmazonEKSNodegroup",
"Effect": "Allow"
},
{
"Action": [
"eks:AssociateAccessPolicy",
"eks:ListAssociatedAccessPolicies",
"eks:DisassociateAccessPolicy"
],
"Resource": "arn:aws:eks:{Region}:{Account_ID}:access-entry/{key}/*",
"Effect": "Allow"
},
{
"Action": "iam:PassRole",
"Resource": "*",
"Effect": "Allow"
}
]
}
IAM Role: {key}-cluster-ServiceRole
AWS-managed policies:
AmazonEKSClusterPolicy
AmazonEKSVPCResourceController
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"cloudwatch:PutMetricData"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ec2:DescribeAccountAttributes",
"ec2:DescribeAddresses",
"ec2:DescribeInternetGateways"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
IAM Role: {key}-addon-vpc-cni-Role
AWS-managed policies:
AmazonEKS_CNI_Policy
IAM Role: {key}-eks-role
AWS-managed policies:
AmazonEBSCSIDriverPolicy
AmazonEC2ContainerRegistryReadOnly
AmazonEKS_CNI_Policy
AmazonEKSWorkerNodePolicy
AmazonSSMManagedInstanceCore
AutoScalingFullAccess
ElasticLoadBalancingFullAccess
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ec2:CreateSecurityGroup",
"ec2:CreateTags"
],
"Effect": "Allow",
"Resource": [
"arn:aws:ec2:{Region}:{Account_ID}:security-group/*",
"arn:aws:ec2:{Region}:{Account_ID}:vpc/{VPC_ID}"
],
"Sid": "CreateOpenflowEKSSecurityGroupAndTags"
},
{
"Action": [
"ec2:AuthorizeSecurityGroupIngress",
"ec2:DeleteSecurityGroup"
],
"Condition": {
"StringLike": {
"aws:ResourceTag/Name": "eks-cluster-sg-{key}-*"
}
},
"Effect": "Allow",
"Resource": [
"arn:aws:ec2:{Region}:{Account_ID}:security-group/*"
],
"Sid": "OpenflowManageEKSSecurityGroup"
}
]
}
Note
{VPC_ID} represents the identifier of the VPC that was either created by BYOC or used by BYO-VPC.
IAM Role: oidc-service-role-{key}
{
"Statement": [
{
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:GetResourcePolicy",
"secretsmanager:ListSecretVersionIds"
],
"Effect": "Allow",
"Resource": [
"arn:aws:secretsmanager:{Region}:{Account_ID}:secret:oidc-{key}*"
]
}
],
"Version": "2012-10-17"
}
IAM Role: dps-postgres-role-{key}
{
"Statement": [
{
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:GetResourcePolicy",
"secretsmanager:ListSecretVersionIds"
],
"Effect": "Allow",
"Resource": [
"arn:aws:secretsmanager:{Region}:{Account_ID}:secret:postgres_creds-{key}*"
]
}
],
"Version": "2012-10-17"
}
IAM Role: dps-temporal-role-{key}
{
"Statement": [
{
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:GetResourcePolicy",
"secretsmanager:ListSecretVersionIds"
],
"Effect": "Allow",
"Resource": [
"arn:aws:secretsmanager:{Region}:{Account_ID}:secret:temporal_creds-{key}*"
]
}
],
"Version": "2012-10-17"
}
IAM Role: dp-service-role-{key}
{
"Statement": [
{
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:GetResourcePolicy",
"secretsmanager:ListSecretVersionIds"
],
"Effect": "Allow",
"Resource": [
"arn:aws:secretsmanager:{Region}:{Account_ID}:secret:dps_creds-{key}*",
"arn:aws:secretsmanager:{Region}:{Account_ID}:secret:snowflake-oauth2-{key}*"
]
}
],
"Version": "2012-10-17"
}
IAM Role: oauth2-role-{key}
{
"Statement": [
{
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:GetResourcePolicy",
"secretsmanager:ListSecretVersionIds"
],
"Effect": "Allow",
"Resource": [
"arn:aws:secretsmanager:{Region}:{Account_ID}:secret:snowflake-oauth2-{key}*"
]
}
],
"Version": "2012-10-17"
}
IAM Role: {key}-nodegroup-NodeInstanceRole
AWS-managed policies:
AmazonEBSCSIDriverPolicy
AmazonEC2ContainerRegistryReadOnly
AmazonEKS_CNI_Policy
AmazonEKSWorkerNodePolicy
AmazonSSMManagedInstanceCore
AutoScalingFullAccess
ElasticLoadBalancingFullAccess
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"servicediscovery:CreateService",
"servicediscovery:DeleteService",
"servicediscovery:GetService",
"servicediscovery:GetInstance",
"servicediscovery:RegisterInstance",
"servicediscovery:DeregisterInstance",
"servicediscovery:ListInstances",
"servicediscovery:ListNamespaces",
"servicediscovery:ListServices",
"servicediscovery:GetInstancesHealthStatus",
"servicediscovery:UpdateInstanceCustomHealthStatus",
"servicediscovery:GetOperation",
"route53:GetHealthCheck",
"route53:CreateHealthCheck",
"route53:UpdateHealthCheck",
"route53:ChangeResourceRecordSets",
"route53:DeleteHealthCheck",
"appmesh:*"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeAutoScalingInstances",
"autoscaling:DescribeLaunchConfigurations",
"autoscaling:DescribeScalingActivities",
"autoscaling:DescribeTags",
"ec2:DescribeInstanceTypes",
"ec2:DescribeLaunchTemplateVersions"
],
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"autoscaling:SetDesiredCapacity",
"autoscaling:TerminateInstanceInAutoScalingGroup",
"ec2:DescribeImages",
"ec2:GetInstanceTypesFromInstanceRequirements",
"eks:DescribeNodegroup"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"iam:CreateServiceLinkedRole"
],
"Condition": {
"StringEquals": {
"iam:AWSServiceName": "elasticloadbalancing.amazonaws.com"
}
},
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"ec2:DescribeAccountAttributes",
"ec2:DescribeAddresses",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeInternetGateways",
"ec2:DescribeVpcs",
"ec2:DescribeVpcPeeringConnections",
"ec2:DescribeSubnets",
"ec2:DescribeSecurityGroups",
"ec2:DescribeInstances",
"ec2:DescribeNetworkInterfaces",
"ec2:DescribeTags",
"ec2:GetCoipPoolUsage",
"ec2:DescribeCoipPools",
"elasticloadbalancing:DescribeLoadBalancers",
"elasticloadbalancing:DescribeLoadBalancerAttributes",
"elasticloadbalancing:DescribeListeners",
"elasticloadbalancing:DescribeListenerCertificates",
"elasticloadbalancing:DescribeSSLPolicies",
"elasticloadbalancing:DescribeRules",
"elasticloadbalancing:DescribeTargetGroups",
"elasticloadbalancing:DescribeTargetGroupAttributes",
"elasticloadbalancing:DescribeTargetHealth",
"elasticloadbalancing:DescribeTags"
],
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"cognito-idp:DescribeUserPoolClient",
"acm:ListCertificates",
"acm:DescribeCertificate",
"iam:ListServerCertificates",
"iam:GetServerCertificate",
"waf-regional:GetWebACL",
"waf-regional:GetWebACLForResource",
"waf-regional:AssociateWebACL",
"waf-regional:DisassociateWebACL",
"wafv2:GetWebACL",
"wafv2:GetWebACLForResource",
"wafv2:AssociateWebACL",
"wafv2:DisassociateWebACL",
"shield:GetSubscriptionState",
"shield:DescribeProtection",
"shield:CreateProtection",
"shield:DeleteProtection"
],
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"ec2:AuthorizeSecurityGroupIngress",
"ec2:RevokeSecurityGroupIngress"
],
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"ec2:CreateSecurityGroup"
],
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"ec2:CreateTags"
],
"Condition": {
"Null": {
"aws:RequestTag/elbv2.k8s.aws/cluster": "false"
},
"StringEquals": {
"ec2:CreateAction": "CreateSecurityGroup"
}
},
"Effect": "Allow",
"Resource": "arn:aws:ec2:*:*:security-group/*"
},
{
"Action": [
"ec2:CreateTags",
"ec2:DeleteTags"
],
"Condition": {
"Null": {
"aws:RequestTag/elbv2.k8s.aws/cluster": "true",
"aws:ResourceTag/elbv2.k8s.aws/cluster": "false"
}
},
"Effect": "Allow",
"Resource": "arn:aws:ec2:*:*:security-group/*"
},
{
"Action": [
"ec2:AuthorizeSecurityGroupIngress",
"ec2:RevokeSecurityGroupIngress",
"ec2:DeleteSecurityGroup"
],
"Condition": {
"Null": {
"aws:ResourceTag/elbv2.k8s.aws/cluster": "false"
}
},
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"elasticloadbalancing:CreateLoadBalancer",
"elasticloadbalancing:CreateTargetGroup"
],
"Condition": {
"Null": {
"aws:RequestTag/elbv2.k8s.aws/cluster": "false"
}
},
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"elasticloadbalancing:CreateListener",
"elasticloadbalancing:DeleteListener",
"elasticloadbalancing:CreateRule",
"elasticloadbalancing:DeleteRule"
],
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"elasticloadbalancing:AddTags",
"elasticloadbalancing:RemoveTags"
],
"Condition": {
"Null": {
"aws:RequestTag/elbv2.k8s.aws/cluster": "true",
"aws:ResourceTag/elbv2.k8s.aws/cluster": "false"
}
},
"Effect": "Allow",
"Resource": [
"arn:aws:elasticloadbalancing:*:*:targetgroup/*/*",
"arn:aws:elasticloadbalancing:*:*:loadbalancer/net/*/*",
"arn:aws:elasticloadbalancing:*:*:loadbalancer/app/*/*"
]
},
{
"Action": [
"elasticloadbalancing:AddTags",
"elasticloadbalancing:RemoveTags"
],
"Effect": "Allow",
"Resource": [
"arn:aws:elasticloadbalancing:*:*:listener/net/*/*/*",
"arn:aws:elasticloadbalancing:*:*:listener/app/*/*/*",
"arn:aws:elasticloadbalancing:*:*:listener-rule/net/*/*/*",
"arn:aws:elasticloadbalancing:*:*:listener-rule/app/*/*/*"
]
},
{
"Action": [
"elasticloadbalancing:ModifyLoadBalancerAttributes",
"elasticloadbalancing:SetIpAddressType",
"elasticloadbalancing:SetSecurityGroups",
"elasticloadbalancing:SetSubnets",
"elasticloadbalancing:DeleteLoadBalancer",
"elasticloadbalancing:ModifyTargetGroup",
"elasticloadbalancing:ModifyTargetGroupAttributes",
"elasticloadbalancing:DeleteTargetGroup"
],
"Condition": {
"Null": {
"aws:ResourceTag/elbv2.k8s.aws/cluster": "false"
}
},
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"elasticloadbalancing:AddTags"
],
"Condition": {
"Null": {
"aws:RequestTag/elbv2.k8s.aws/cluster": "false"
},
"StringEquals": {
"elasticloadbalancing:CreateAction": [
"CreateTargetGroup",
"CreateLoadBalancer"
]
}
},
"Effect": "Allow",
"Resource": [
"arn:aws:elasticloadbalancing:*:*:targetgroup/*/*",
"arn:aws:elasticloadbalancing:*:*:loadbalancer/net/*/*",
"arn:aws:elasticloadbalancing:*:*:loadbalancer/app/*/*"
]
},
{
"Action": [
"elasticloadbalancing:RegisterTargets",
"elasticloadbalancing:DeregisterTargets"
],
"Effect": "Allow",
"Resource": "arn:aws:elasticloadbalancing:*:*:targetgroup/*/*"
},
{
"Action": [
"elasticloadbalancing:SetWebAcl",
"elasticloadbalancing:ModifyListener",
"elasticloadbalancing:AddListenerCertificates",
"elasticloadbalancing:RemoveListenerCertificates",
"elasticloadbalancing:ModifyRule"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"route53:ChangeResourceRecordSets"
],
"Effect": "Allow",
"Resource": "arn:aws:route53:::hostedzone/*"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"route53:GetChange"
],
"Effect": "Allow",
"Resource": "arn:aws:route53:::change/*"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"route53:ListResourceRecordSets",
"route53:ListHostedZonesByName"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ec2:CreateSnapshot",
"ec2:AttachVolume",
"ec2:DetachVolume",
"ec2:ModifyVolume",
"ec2:DescribeAvailabilityZones",
"ec2:DescribeInstances",
"ec2:DescribeSnapshots",
"ec2:DescribeTags",
"ec2:DescribeVolumes",
"ec2:DescribeVolumesModifications"
],
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"ec2:CreateTags"
],
"Condition": {
"StringEquals": {
"ec2:CreateAction": [
"CreateVolume",
"CreateSnapshot"
]
}
},
"Effect": "Allow",
"Resource": [
"arn:aws:ec2:*:*:volume/*",
"arn:aws:ec2:*:*:snapshot/*"
]
},
{
"Action": [
"ec2:DeleteTags"
],
"Effect": "Allow",
"Resource": [
"arn:aws:ec2:*:*:volume/*",
"arn:aws:ec2:*:*:snapshot/*"
]
},
{
"Action": [
"ec2:CreateVolume"
],
"Condition": {
"StringLike": {
"aws:RequestTag/ebs.csi.aws.com/cluster": "true"
}
},
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"ec2:CreateVolume"
],
"Condition": {
"StringLike": {
"aws:RequestTag/CSIVolumeName": "*"
}
},
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"ec2:DeleteVolume"
],
"Condition": {
"StringLike": {
"ec2:ResourceTag/ebs.csi.aws.com/cluster": "true"
}
},
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"ec2:DeleteVolume"
],
"Condition": {
"StringLike": {
"ec2:ResourceTag/CSIVolumeName": "*"
}
},
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"ec2:DeleteVolume"
],
"Condition": {
"StringLike": {
"ec2:ResourceTag/kubernetes.io/created-for/pvc/name": "*"
}
},
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"ec2:DeleteSnapshot"
],
"Condition": {
"StringLike": {
"ec2:ResourceTag/CSIVolumeSnapshotName": "*"
}
},
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"ec2:DeleteSnapshot"
],
"Condition": {
"StringLike": {
"ec2:ResourceTag/ebs.csi.aws.com/cluster": "true"
}
},
"Effect": "Allow",
"Resource": "*"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"route53:ChangeResourceRecordSets"
],
"Effect": "Allow",
"Resource": "arn:aws:route53:::hostedzone/*"
}
]
}
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"route53:ListHostedZones",
"route53:ListResourceRecordSets",
"route53:ListTagsForResource"
],
"Effect": "Allow",
"Resource": "*"
}
]
}