Skip to main content

Setup Snowplow BDP on AWS

info
This documentation only applies to Snowplow BDP. See the feature comparison page for more information about the different Snowplow offerings.

To setup Snowplow, simply follow the 'Getting Started' steps in the Snowplow BDP Console. You will receive an account as part of your onboarding.

What are the steps​

The first setup steps are designed to get your infrastructure in place and have you sending data as quickly as possible. The initial steps include:

  • providing the right cloud environment for Snowplow to be installed
  • setting up your first sandbox environment
  • selecting which warehouse, if any, you want to load your data into

Completing the forms for these initial steps should take you around 30 minutes.

What will I need​

To setup your cloud environment as required you will need

We often find our point of contact requires support from their DevOps or Networking colleagues to complete the cloud setup step; in Snowplow BDP Console you can easily create accounts for colleagues who can complete this step for you.

Preparing your AWS sub-account​

These instructions are also provided as part of the setup flow in Snowplow BDP Console.

Create sub-account​

  1. From your main AWS account, set up an Organisation if you haven't done so already.
  2. Create a member account (the sub-account) in that organization
  3. Sign out and sign into the new sub-account. Everything Snowplow-related will take place within this account from here in.
  4. Follow these instructions to create a policy using the JSON policy document below.

Set up Role and IAM permissions​

  1. Access the IAM control panel within the sub-account
  2. Go to Access management > Roles and select Create role
  3. Select "Another AWS account"
    (Account ID: 793733611312 Require MFA: true)
  4. Select the policy you created earlier
  5. Call the role "SnowplowAdmin" or similar

You will need to share this role with us as part of filling out the setup form in Snowplow BDP console.

JSON Policy Document​

{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Effect":"Allow",
      "Action":[
        "acm:*",
        "application-autoscaling:*",
        "autoscaling:*",
        "cloudformation:*",
        "cloudfront:*",
        "cloudwatch:*",
        "dynamodb:*",
        "ec2:*",
        "ecs:*",
        "eks:*",
        "elasticloadbalancing:*",
        "elasticmapreduce:*",
        "es:*",
        "execute-api:*",
        "events:*",
        "globalaccelerator:*",
        "iam:*",
        "kinesis:*",
        "kinesisanalytics:*",
        "kms:DescribeKey",
        "kms:List*",
        "lambda:*",
        "logs:*",
        "rds:*",
        "redshift:*",
        "route53:*",
        "route53resolver:*",
        "s3:*",
        "secretsmanager:CreateSecret",
        "secretsmanager:DeleteSecret",
        "secretsmanager:DescribeSecret",
        "secretsmanager:GetResourcePolicy",
        "secretsmanager:GetSecretValue",
        "secretsmanager:PutSecretValue",
        "secretsmanager:TagResource",
        "servicequotas:GetAWSDefaultServiceQuota",
        "servicequotas:GetServiceQuota",
        "servicequotas:ListAWSDefaultServiceQuotas",
        "servicequotas:ListRequestedServiceQuotaChangeHistory",
        "servicequotas:ListServiceQuotas",
        "servicequotas:ListServices",
        "servicequotas:RequestServiceQuotaIncrease",
        "sns:*",
        "sqs:*",
        "ssm:*",
        "support:*"
      ],
      "Resource":"*"
    }
  ]
}

For complete documentation from Amazon go here.

Set up Role and IAM Permissions with CloudFormation​

We also provide a CloudFormation template that will create a role named SnowplowAdmin with the full permission set here.

  1. Access the CloudFormation service within the sub-account
  2. Go to Stacks select Create stack > With new resources (standard)
  3. Select Template is ready within the Prepare template block
  4. Specify an Amazon S3 URL with the full path to the SnowplowAdmin CloudFormation template and proceed
  5. Provide the stack with a meaningful name such as SnowplowAdmin stack
  6. Now proceed through the remainder of the prompts and choose Create stack

For complete documentation from Amazon go here.

Setup the Snowplow deployment role​

The last step is to set up the Snowplow deployment role. This is a role assumed by the machine user to make changes with Terraform.

  1. Navigate to https://console.aws.amazon.com/iam/home#/roles$new?step=type&roleType=crossAccount
  2. Select Create role and for trusted entity type select AWS account.
  • Account ID: 793733611312
  • Do not require MFA, as Snowplow needs to be able to assume the role via headless jobs.
  1. Attach the IAMFullAccess policy. If a Permission Boundary was set on the admin role, then add this boundary to the bottom section of permissions page.
  • Role name: SnowplowDeployment
  • Role description: Allows the Snowplow Team to programmatically deploy to this account.
  1. Copy the Snowplow deployment role ARN. You will need to share this role with us as part of filling out the setup form in Snowplow BDP console.

Final checklist​

If you are sending a request to our team to set up your account for you. Please ensure you provide the following information:

  1. SnowplowAdmin role ARN
  2. SnowplowDeployment role ARN
  3. AWS region to deploy into
  4. VPC CIDR requirements for VPC Peering (if applicable)
  5. The IAM permission boundary ARN (if applicable)
Was this page helpful?