Versioning Data Structures using Iglu
When evolving your schema and uploading it to your Iglu Server, you will need to choose how to increment its version.
There are two kinds of schema changes:
- Non-breaking - a non-breaking change is backward compatible with historical data and increments the
patchnumber i.e.1-0-0->1-0-1. - Breaking - a breaking change is not backwards compatible with historical data and increments the
modelnumber i.e.1-0-0->2-0-0.
Different data warehouses handle schema evolution slightly differently. Use the table below as a guide for incrementing the schema version appropriately.
| Redshift | Snowflake, BigQuery & Databricks | |
| Add / remove / rename an optional field | Non-breaking | Non-breaking |
| Add / remove / rename a required field | Breaking | Breaking |
| Change a field from optional to required | Breaking | Breaking |
| Change a field from required to optional | Breaking | Non-breaking |
| Change the type of an existing field | Breaking | Breaking |
| Change the size of an existing field | Non-breaking | Non-breaking |
warning
In Redshift and Databricks, changing size may also mean type change; e.g. changing the maximum integer from 30000 to 100000. See our documentation on how schemas translate to database types.