# S3 Loader
> Archive Snowplow events from Kinesis to S3 in LZO or Gzip format for raw payloads, enriched events, and failed events.
> Source: https://docs.snowplow.io/docs/api-reference/loaders-storage-targets/s3-loader/
> Documentation index: https://docs.snowplow.io/llms.txt

Snowplow S3 Loader consumes records from an [Amazon Kinesis](http://aws.amazon.com/kinesis/) stream and writes them to [S3](http://aws.amazon.com/s3/). A typical Snowplow pipeline would use the S3 loader in several places:

- Load enriched events from the "enriched" stream. These serve as input for [the RDB loader](/docs/api-reference/loaders-storage-targets/snowplow-rdb-loader/) when loading to a warehouse.
- Load failed events from the "bad" stream.

Records that can't be successfully written to S3 are written to a second Kinesis stream with the error message.

## Output format : GZIP

The records are treated as byte arrays containing UTF-8 encoded strings (whether CSV, JSON or TSV). New lines are used to separate records written to a file. This format can be used with the Snowplow Kinesis Enriched stream, among other streams.

Gzip encoding is generally used for both enriched data and bad data.

## Running

### Available on Terraform Registry

A Terraform module which deploys the Snowplow S3 Loader on AWS EC2 for use with Kinesis. For installing in other environments, please see the other installation options below.

### Docker image

We publish two different flavours of the docker image:

- `snowplow/snowplow-s3-loader:3.3.0`

- `snowplow/snowplow-s3-loader:3.3.0-distroless` (lightweight alternative)

Here is a standard command to run the loader on a EC2 instance in AWS:

```bash
docker run \
    -d \
    --name snowplow-s3-loader \
    --restart always \
    --log-driver awslogs \
    --log-opt awslogs-group=snowplow-s3-loader \
    --log-opt awslogs-stream='ec2metadata --instance-id' \
    --network host \
    -v $(pwd):/snowplow/config \
    -e 'JAVA_OPTS=-Xms512M -Xmx1024M -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN' \
    snowplow/snowplow-s3-loader:3.3.0 \
    --config /snowplow/config/config.hocon
```

## Monitor the loader

The S3 Loader reports metrics covering event counts and latency. It can send them to StatsD, to Prometheus, or to both.

See [Monitoring the S3 Loader](/docs/api-reference/loaders-storage-targets/s3-loader/monitoring/) for the full list of metrics and for how to configure each protocol.