Skip to main content

Stream Enrich

note

stream-enrich-kinesis and stream-enrich-kafka will soon be deprecated in favor of enrich-kinesis and enrich-kafka, respectively. We recommend switching when you have the opportunity.

Available on Terraform Registry

A Terraform module is available which deploys Stream Enrich on a AWS EC2 without the need for this manual setup.

Run Stream Enrich

Stream Enrich is on docker hub with several different flavours. Pull the image that matches the source/sink you are using:

docker pull snowplow/stream-enrich-kinesis:3.6.0
docker pull snowplow/stream-enrich-kafka:3.6.0
docker pull snowplow/stream-enrich-nsq:3.6.0

The application is configured by passing a hocon file on the command line:

docker run --rm \
--log-driver awslogs \
--log-opt awslogs-group=${log_group_name} \
--log-opt awslogs-stream=$(ec2metadata --instance-id) \
-v ${path_to_config_dir}:/snowplow/config \
snowplow/stream-enrich-${message_queue}:3.6.0 \
--config /snowplow/config/config.hocon \
--resolver file:/snowplow/config/iglu_resolver.json \
--enrichments file:/snowplow/config/enrichments/

Alternatively, you can download and run a jar file from the github release.

java -jar snowplow-stream-enrich-kinesis-3.6.0.jar \
--config /path/to/config.hocon \
--enrichments file:/path/to/enrichments/ \
--resolver file:/path/to/iglu_resolver.json \
--force-cached-files-download

The configuration guide describes how to construct the config, enrichments, and resolver input files.