Storage options
Data warehouses and data lakes are primary destinations for Snowplow data. For other options, see the destinations overview page.
Data warehouse loadersโ
Cloud
The cloud selection is for where your Snowplow pipeline runs. The warehouse itself can be deployed in any cloud.
- AWS
- GCP
- Azure
Destination | Type | Loader application | Status |
---|---|---|---|
Redshift (including Redshift serverless) | Batching (recommended) or micro-batching | RDB Loader | Production-ready |
Snowflake | Batching (recommended) or micro-batching | Snowplow RDB Loader | Production-ready |
Snowflake | Streaming ๐ | Snowflake Streaming Loader | Early release |
Databricks | Batching (recommended) or micro-batching | Snowplow RDB Loader | Production-ready |
Postgres | Streaming | Postgres Loader | Not recommended for production use |
Destination | Type | Loader application | Status |
---|---|---|---|
BigQuery | Streaming | BigQuery Loader | Production-ready |
Snowflake | Micro-batching | RDB Loader | Production-ready |
Snowflake | Streaming ๐ | Snowflake Streaming Loader | Early release |
Databricks | Micro-batching | RDB Loader | Production-ready |
Postgres | Streaming | Postgres Loader | Not recommended for production use |
Destination | Type | Loader application | Status |
---|---|---|---|
Snowflake | Micro-batching | RDB Loader | Early release |
Snowflake | Streaming ๐ | Snowflake Streaming Loader | Early release |
Databricks | Micro-batching (via a data lake) | Lake Loader | Early release |
Synapse Analytics | Micro-batching (via a data lake) | Lake Loader | Early release |
Data lake loadersโ
All lake loaders are micro-batching.
- AWS
- GCP
- Azure
Lake | Format | Compatibility | Loader application | Status |
---|---|---|---|---|
S3 | Delta | Athena | Lake Loader | Early release |
S3 | Iceberg | Athena, Redshift | Lake Loader | Early release |
S3 | TSV/JSON | Athena | S3 Loader | Only recommended for use with RDB Batch Transformer or for failed events |
Lake | Format | Compatibility | Loader application | Status |
---|---|---|---|---|
GCS | Delta | Databricks | Lake Loader | Early release |
GCS | JSON | BigQuery | GCS Loader | Only recommended for failed events |
Lake | Format | Compatibility | Loader application | Status |
---|---|---|---|---|
ADLS Gen2 | Delta | Synapse Analytics, Fabric, Databricks | Lake Loader | Early release |