Modeling your data with dbt
dbt enables analytics engineers to transform data in their warehouses by simply writing select statements. Snowplow has written and maintain a number of dbt packages to model your snowplow data for various purposes and produce derived tables for use in analytics, AI, ML, BI, or reverse ETL tools.
Using Snowplow's dbt packages means you can draw insight and value from your data quicker, easier, and cheaper than building your own modeling from scratch.
To setup dbt, Snowplow Community Edition users can start with the dbt User Guide and then we have prepared some introduction videos for working with the Snowplow dbt packages.
For Snowplow BDP customers, dbt projects can be configured and scheduled in the console meaning you can get started running dbt models alongside your Snowplow pipelines.
Snowplow dbt Packages
Our dbt packages are under a mix of licenses, including some under the Snowplow Personal & Academic License. For more information about what this means please see the FAQ or contact us to gain access to our Data Models Pack that enables use of all our packages.
Our dbt packages come with powerful built-in features such as an optimization to the incremental materialization to save you cost on warehouse compute resources compared to the standard method, a custom incremental logic to ensure we process just the required data for each run and keep your models in sync, plus the ability to build your own custom models using both of these!
There are 4 core snowplow dbt packages:
- Snowplow Unified Digital (dbt model docs): for modeling your web and mobile data for page and screen views, sessions, users, and consent
- Snowplow Media Player (dbt model docs): for modeling your media elements for play statistics
- Snowplow E-commerce (dbt model docs): for modeling your E-commerce interactions across carts, products, checkouts, and transactions
- Snowplow Attribution (dbt model docs): used for Attribution Modeling with Snowplow
We also have 2 utility packages:
- Snowplow Normalize package that makes it easy for you to build models that transform your events data into a different structure that may be better suited for downstream consumers
- Snowplow Utils contains all our shared macros and features used across our packages
There are also 3 legacy dbt packages for web, mobile (superseded by unified) and fractribution (superseeded by attribution):
- Snowplow Web (dbt model docs): for modeling your web data for page views, sessions, users, and consent
- Snowplow Mobile (dbt model docs): for modeling your mobile app data for screen views, sessions, users, and crashes
- Snowplow Fractribution (dbt model docs): used for Attribution Modeling with Snowplow
Each package comes with a set of standard models to take your Snowplow tracker data and produce tables aggregated to different levels, or to perform analysis for you. You can also add your own models on top, see the page on custom modules for more information on how to do this.
The supported data warehouses per version can be seen below:
- Snowplow Unified Digital
- Snowplow Media Player
- Snowplow Normalize
- Snowplow E-commerce
- Snowplow Attribution
snowplow-unified version | dbt versions | BigQuery | Databricks | Redshift | Snowflake | Postgres | Spark |
---|---|---|---|---|---|---|---|
0.5.0 | >=1.6.0 to <2.0.0 | β | β | β | β | β | β |
0.4.5 | >=1.6.0 to <2.0.0 | β | β | β | β | β | β |
snowplow-media-player version | snowplow-web version | dbt versions | BigQuery | Databricks | Redshift | Snowflake | Postgres | Spark |
---|---|---|---|---|---|---|---|---|
0.9.1 | N/A | >=1.4.0 to <2.0.0 | β | β | β | β | β | β |
0.8.0 | N/A | >=1.4.0 to <2.0.0 | β | β | β | β | β | β |
0.5.3 | >=0.14.0 to <0.16.0 | >=1.4.0 to <2.0.0 | β | β | β | β | β | β |
0.4.2 | >=0.13.0 to <0.14.0 | >=1.3.0 to <2.0.0 | β | β | β | β | β | β |
0.4.1 | >=0.12.0 to <0.13.0 | >=1.3.0 to <2.0.0 | β | β | β | β | β | β |
0.3.4 | >=0.9.0 to <0.12.0 | >=1.0.0 to <1.3.0 | β | β | β | β | β | β |
0.1.0 | >=0.6.0 to <0.7.0 | >=0.20.0 to <1.1.0 | β | β | β | β | β | β |
snowplow-normalize version | dbt versions | BigQuery | Databricks | Redshift | Snowflake | Postgres |
---|---|---|---|---|---|---|
0.3.5 | >=1.4.0 to <2.0.0 | β | β | β | β | β |
0.2.3 | >=1.3.0 to <2.0.0 | β | β | β | β | β |
0.1.0 | >=1.0.0 to <2.0.0 | β | β | β | β | β |
snowplow-ecommerce version | dbt versions | BigQuery | Databricks | Redshift | Snowflake | Postgres | Spark |
---|---|---|---|---|---|---|---|
0.9.0 | >=1.4.0 to <2.0.0 | β | β | β | β | β οΈ | β |
0.8.2 | >=1.4.0 to <2.0.0 | β | β | β | β | β οΈ | β |
0.3.0 | >=1.3.0 to <2.0.0 | β | β | β | β | β | β |
0.2.1 | >=1.0.0 to <2.0.0 | β | β | β | β | β | β |
Postgres is technically supported in the models within the package, however one of the contextsβ names is too long to be loaded via the Postgres Loader.
snowplow-attribution version | dbt versions | BigQuery | Databricks | Redshift | Snowflake | Postgres | Spark |
---|---|---|---|---|---|---|---|
0.4.0 | >=1.6.0 to <2.0.0 | β | β | β | β | β | β |
0.3.0 | >=1.6.0 to <2.0.0 | β | β | β | β | β | β |
dbt Version Compatibility Checkerβ
You may be using other dbt packages that require you to use a specific version of dbt, or a specific version of the dbt_utils
package. Or you simply may not wish to upgrade your existing setup.
Hereβs a way to check the latest version of our packages you can install for a given setup. Simply enter your dbt version and (optionally) dbt_utils
version below.
Please enter a valid dbt version.