Skip to main content

Normalize

Contents
Variables Definitions
Config Generator
Schema Generator

Package Configuration Variables​

This package utilizes a set of variables that are configured to recommended values for optimal performance of the models. Depending on your use case, you might want to override these values by adding to your dbt_project.yml file.

note
All variables in Snowplow packages start with `snowplow__` but we have removed these in the below tables for brevity.

Warehouse and Tracker


Operation and Logic


Contexts, Filters, and Logs


Warehouse Specific


Config Generator​

You can use the below inputs to generate the code that you need to place into your dbt_project.yml file to configure the package as you require. Any values not specified will use their default values from the package.

Warehouse and Tracker
Schema (dataset) that contains your atomic events
Database that contains your atomic events
Target name of your development environment as defined in your `profiles.yml` file
Reference to your events table
Grant Select List
> Click the plus sign to add a new entry
Enable granting usage on schemas
Operation and Logic
The maximum numbers of days of new data to be processed since the latest event processed
The maximum allowed number of days between the event creation and it being sent to the collector
The number of hours to look before the latest event processed - to account for late arriving data, which comes out of order
The date to start processing events from in the package on first run or a full refresh, based on `collector_tstamp`
Determines which timestamp is used to process sessions of data
Number of days to look back over the incremental derived tables during the upsert
Contexts, Filters, and Logs
App IDs
> Click the plus sign to add a new entry
Warehouse Specific
The catalogue your atomic events table is in

Project Variables:

vars:
  snowplow_normalize: null

Output Schemas​

By default all scratch/staging tables will be created in the <target.schema>_scratch schema, the derived tables, will be created in <target.schema>_derived and all manifest tables in <target.schema>_snowplow_manifest. Some of these schemas are only used by specific packages, ensure you add the correct configurations for each packages you are using. To change, please add the following to your dbt_project.yml file:

tip

If you want to use just your connection schema with no suffixes, set the +schema: values to null

models:
  snowplow_normalize:
    base:
      manifest:
        +schema: my_manifest_schema
      scratch:
        +schema: my_scratch_schema