Setup Snowplow BDP on GCP

info

This documentation only applies to Snowplow BDP. See the feature comparison page for more information about the different Snowplow offerings.

Request your pipeline through Snowplow BDP Console

Login to Snowplow BDP Console where you'll be able to follow a step-by-step guide to getting started (including the steps below).

Setting up your Google project

To set up your cloud environment ready for your Snowplow pipeline to be installed:

Create a new project

Please create a new project segregated from anything else you have running in GCP. For the latest documentation on setting up a new project please refer to Google Cloud's documentation here.

Set up required permissions

Please add techops-cloud-admin@snowplowanalytics.com to your project. This account will be used to install and maintain your pipeline in GCP. The following roles are required:

• editor
• Roles/resourcemanager.projectIamAdmin
• Roles/errorreporting.admin
• Roles/logging.admin
• Roles/monitoring.admin
• Roles/iam.roleAdmin
• Roles/iam.serviceAccountAdmin

The following roles are also required if using RDB Loader within GCP:

• Roles/secretmanager.secretAccessor
• Roles/secretmanager.secretVersionAdder

Enable billing for the project

Please enable billing on the project as failing to do so will prevent the pipeline from being installed. For details on enabling billing for your project, please refer to the GCP documentation for APIs and billing.

Ensure that the following constraints are turned off

The following is a list of Organizational Policy constraints that we require to be turned off, otherwise we are unable to deploy the pipeline. For some, more specific configuration would enable the constraint to stay and the pipeline to deploy. This list will be updated as we discover more constraints that are not compatible with the pipeline deployment.

• constraints/compute.requireShieldedVm
• constraints/gcp.resourceLocations
• constraints/iam.disableServiceAccountCreation
• constraints/iam.disableServiceAccountKeyCreation
• constraints/sql.restrictPublicIp