Skip to main content

Snowflake Loader

The Snowflake Loader consists of two independent applications:

  • Snowplow Snowflake Transformer - Spark job responsible for transformning enriched events into Snowflake-compatible format
  • Snowplow Snowflake Loader - CLI application responsible for loading Snowplow-compatible enriched events into Snowflake DB

Both applications communicate through DynamoDB table, called "processing manifest" and used to maintain pipeline state. Both applications use same self-describing JSON configuration file of schema, which contain both transformer- and loader-specific properties.

The Snowflake loader is publicly available since version 0.3.1.