Tracking Schema Migrations with JSON Diff

Schema migration is the process of evolving the structure of your data over time. In document databases (MongoDB, CouchDB), JSON-based APIs, and configuration systems, the "schema" is often implicit in the JSON structure itself. JSON diff becomes an essential tool for tracking how schemas evolve.

Migration example (user document):

// Schema v1
{
  "name": "Alice Johnson",
  "email": "alice@example.com",
  "address": "123 Main St, Portland, OR 97201"
}

// Schema v2
{
  "firstName": "Alice",
  "lastName": "Johnson",
  "email": "alice@example.com",
  "address": {
    "street": "123 Main St",
    "city": "Portland",
    "state": "OR",
    "zip": "97201"
  },
  "version": 2
}

The diff reveals three categories of schema changes:

Field splitting:

• name (single string) was removed and replaced by firstName and lastName (two strings). This is a breaking change that requires a migration function.

Scalar to object promotion:

• address changed from a flat string to a structured object with street, city, state, and zip fields. This provides better queryability but requires all consumers to update their parsing logic.

Version tracking:

• A version field was added to identify which schema iteration a document conforms to. This enables progressive migration strategies.

Migration strategies revealed by diffs:

1. Big bang migration: Transform all documents at once. The diff shows the exact transformation needed.
2. Lazy migration: Transform documents on read. The diff helps write the transformation function.
3. Dual write: Write both old and new formats during a transition period. The diff validates both formats are consistent.

Documenting migrations with diffs:

For each schema version bump:

1. Capture a representative document in the old format.
2. Capture the same document in the new format.
3. Run a JSON diff and document every change.
4. Write migration code that reproduces each change.
5. Write rollback code that reverses each change.

This creates a clear audit trail that future developers can reference when they need to understand why the schema looks the way it does.