Versioning Data Structures with the JSON Editor
How do I version?
Breaking and non-breaking changes
In Data Structures UI at the point of publishing a schema you'll be asked to select which version you'd like to create.
There are two kinds of schema changes:
- Non-breaking - a non-breaking change is backward compatible with historical data and increments the
patch
number i.e.1-0-0
->1-0-1
. - Breaking - a breaking change is not backwards compatible with historical data and increments the
model
number i.e.1-0-0
->2-0-0
.
Different data warehouses handle schema evolution slightly differently. Use the table below as a guide for incrementing the schema version appropriately.
Redshift | Snowflake, BigQuery & Databricks | |
Add / remove / rename an optional field | Non-breaking | Non-breaking |
Add / remove / rename a required field | Breaking | Breaking |
Change a field from optional to required | Breaking | Breaking |
Change a field from required to optional | Breaking | Non-breaking |
Change the type of an existing field | Breaking | Breaking |
Change the size of an existing field | Non-breaking | Non-breaking |
In Redshift and Databricks, changing size may also mean type change; e.g. changing the maximum
integer from 30000
to 100000
. See our documentation on how schemas translate to database types.
Incrementing the middle digit
For particular workflows you may want to make use of the middle digital as part of your versioning strategy. For simplicity, the UI allows only breaking or non-breaking changes.
Should you wish to use the middle versioning digit this is possible via the Data Structures API.