Legacy migration stalls on inconsistent data
A migration from an old system stalls because the old data breaks rules the new system enforces.
Over the years fields were used inconsistently: several values in one field, free text where structure was expected.
Key facts
- The old system accepted any input, and the new one validates it.
- The same field was used for different purposes by different teams.
- Profiling the data comes before designing the target model.
- Cleaning everything before go-live moves the date. Migrating as-is carries the confusion into the new system.
- Screens designed and tested on a small, clean dataset break on migrated data with long names and messy history.
Why it happens
No validation in the old system. Fields accepted any input, and users filled them with whatever fit the moment.
Meaning changed over time. Different teams used the same field for different purposes.
The migration was scheduled as a technical task. It requires a decision about what each inconsistent field meant, and that decision belongs to the business.
How it shows up
- Import scripts fail on a share of records
- Single fields hold several comma-separated values
- The migration date slips
- Someone proposes importing everything into a notes field
- Users of the new system find records they do not recognise
Options and trade-offs
What helps:
- Profile the data before designing the target model
- Mark migrated records visibly
- A queue for records that need a person
- Keep part of the history as read-only reference
The tension: cleaning all data before go-live moves the date. Migrating as-is keeps it, and the old confusion arrives in the new system.
Keeping old history as read-only reference avoids cleaning it, and users then work in two places when they need old records.
Common mistakes
- Designing the target model before looking at the real data
- Importing inconsistent values into a free-text field to keep the date
- Leaving migrated records indistinguishable from new ones
- Designing screens on clean demo data