Skip to main content

Transformer Kafka

info

This application is available since v5.7.0.

Downloading the artifact​

The asset is published as a jar file attached to the Github release notes for each version.

It's also available as a Docker image on Docker Hub under snowplow/transformer-kafka:6.1.2

Configuring snowplow-transformer-kafka​

The transformer takes two configuration files:

  • a config.hocon file with application settings
  • an iglu_resolver.json file with the resolver configuration for your Iglu schema registry.

An example of the minimal required config for the Transformer Kafla can be found here and a more detailed one here. For details about each setting, see the configuration reference.

See here for details on how to prepare the Iglu resolver file.

tip

All self-describing schemas for events processed by the transformer must be hosted on Iglu Server 0.6.0 or above. Iglu Central is a registry containing Snowplow-authored schemas. If you want to use them alongside your own, you will need to add it to your resolver file. Keep it mind that it could override your own private schemas if you give it higher priority. For details on this see here.

Running the Transformer Kafka​

The two config files need to be passed in as base64-encoded strings:

$ docker run snowplow/transformer-kafka:6.1.2 \
--iglu-config $RESOLVER_BASE64 \
--config $CONFIG_BASE64
Telemetry notice

By default, Snowplow collects telemetry data for Transformer Kafka (since version 5.7.0). Telemetry allows us to understand how our applications are used and helps us build a better product for our users (including you!).

This data is anonymous and minimal, and since our code is open source, you can inspect what’s collected.

If you wish to help us further, you can optionally provide your email (or just a UUID) in the telemetry.userProvidedId configuration setting.

If you wish to disable telemetry, you can do so by setting telemetry.disable to true.

See our telemetry principles for more information.