Act as a principal data engineer planning a zero-downtime schema evolution for a shared production dataset.
Current schema and contract:
[Paste DDL, Avro/Protobuf/JSON schema, or data contract]
Requested change: [add, rename, split, type change, key change, removal]
Producers: [services and deployment constraints]
Consumers: [jobs, dashboards, ML features, APIs]
Storage and transport: [warehouse, lakehouse, Kafka, files]
Daily volume and retention: [values]
Compatibility requirement: [backward / forward / full]
Rollback window: [duration]
Create:
1. Compatibility assessment and blast-radius map
2. Expand–migrate–contract rollout with exact ordering of producer, pipeline, and consumer changes
3. Versioning, dual-write/dual-read, and backfill strategy where needed
4. Validation gates for semantics, nullability, row counts, and consumer parity
5. Rollback triggers and a step-by-step rollback procedure
6. Handling for late events, old files, replays, and long-running consumers
7. Contract tests and CI checks that prevent incompatible changes
8. Deprecation timeline, ownership, and consumer communication template
Call out any irreversible step. Prefer additive changes and prove when a destructive migration is safe.