Data migration for platform implementations: cleanse before you move
What are the data migration best practices for a platform implementation?
Agree cleansing rules, rehearse on a copy, reconcile counts and samples, then migrate in a window with a rollback. Data migration best practices are about sequence: cleansing decisions belong to the business, the rehearsal proves the mapping, and a signed reconciliation makes the new system trusted on day one.
Data migration best practices come down to four steps in a fixed order: agree the cleansing rules with the people who own the data, rehearse the full migration on a copy until it runs clean, reconcile counts and samples with a named business owner signing off, and then migrate in a planned window with a tested rollback. Every migration that went wrong in our experience skipped or reordered one of those steps, usually by treating migration as a technical chore for the last fortnight. This article sets out the method we use inside an enterprise platform implementation, whether the target is a CRM, an ERP, an HRMS or a custom platform.
The reason migration matters more than its budget line suggests is trust. Users judge a new system by the first record they open. If it is a duplicate, has the wrong owner or is missing the phone number they know by heart, they conclude the system is wrong and the spreadsheet is right. Nothing in training undoes that.
What a platform data migration involves
Migration is extraction from one or more sources, transformation into the target's model, loading, and verification. The hard parts are not technical. They are deciding what a "customer" is when three systems disagree, deciding whether ten years of closed tickets need to move, and deciding who is allowed to say the numbers match. CRM data migration adds the ownership problem: every record needs a current owner, and half the owners have left. Legacy data cleansing adds the history problem: fields that changed meaning in 2019 and were never re-coded.
The migration plan at a glance
| Step | Owner | Output | Common failure |
|---|---|---|---|
| Source inventory | Data engineer with business owners | List of every source, record count, freshness, owner | A spreadsheet nobody mentioned holds the real data |
| Cleansing rules | Business owners, facilitated by us | Written rules for duplicates, dead records, defaults, ownership | Engineers decide rules the business later rejects |
| Field mapping | Data engineer with process owner | Source-to-target map with transformations and defaults | Free-text fields mapped to picklists without a review |
| Rehearsal one | Data engineer | Full load on a copy; error log; timing | Only a sample is loaded, so scale problems hide |
| Reconciliation | Business owner signs | Counts by entity, totals, and a sampled record check | Counts match but content is wrong in the sample |
| Rehearsal two | Data engineer | Clean run with fixes applied; cut-over runbook | Fixes applied by hand and not repeatable |
| Cut-over | Project manager | Migration in a window; rollback tested; delta load | No freeze, so records change during the load |
Agree cleansing rules before anything is moved
Cleansing is a business decision dressed as a technical task. Which of two duplicate accounts survives? What happens to leads untouched for three years? What is the default owner for orphaned records? Is a contact without an email or phone worth keeping? Engineers can propose rules and show the impact of each ("this rule removes this many records"), but the sales director, the finance controller or the HR head must choose. Write the rules down, version them, and apply them by script so the same rules run identically in every rehearsal. A rule applied by hand in one rehearsal will be forgotten in the next.
Duplicates deserve their own workshop
Duplicate detection is where migrations stall. Exact matches are easy; the hard cases are the same company with three spellings, the same person with two phone numbers, and the two accounts that should be one because the business merged. Agree a match key (for example normalised phone plus name, or GST number for Indian businesses), a survivor rule, and a merge rule for child records. Then show the business a list of proposed merges and let them veto. Automated matching with human review beats either alone.
Rehearse on a full copy, not a sample
A migration rehearsal is a complete run of extraction, transformation and load against a copy of the target, using the full data volume. Samples hide the problems that matter: the one record with a character that breaks the loader, the ten-thousandth attachment that exceeds a limit, the load that takes fourteen hours when the window is eight. Run the first rehearsal early, expect it to fail, and treat the error log as the work list. Every fix goes into the script, never into the data by hand, so the second rehearsal proves the fixes rather than the engineer's memory.
Reconcile counts and samples, then get a signature
Reconciliation has two halves. Counts: records per entity in source and target, with every difference explained by a cleansing rule. Totals: pipeline value, open ticket count, headcount, ledger balances, whatever the business measures itself by. Then samples: a business owner opens twenty records they know well in the new system and confirms each is right. Counts prove nothing was lost; samples prove nothing was mangled. The owner signs the reconciliation, and that signature is the go-live gate. If nobody in the business will sign, the migration is not ready, however clean the logs look.
Migrate in a window with a rollback
The cut-over runbook lists every step with an owner and a time: freeze the source, take a final backup, run the migration, run reconciliation, open the target, run the delta for anything that changed during the freeze. The rollback is the step nobody wants to use and everybody must have: restore the target to its pre-migration state and reopen the source. Test the rollback in rehearsal two. A rollback that has never been run is a hope, not a plan. Where the old and new systems must coexist for a period, plan the delta loads and the point at which the old system becomes read-only; phased ERP delivery covers coexisting systems in more depth.
Where AI helps and where it does not
Language models are useful at the edges of migration: classifying free-text fields into picklists with a confidence score, proposing merges for messy company names, extracting structure from notes fields, and flagging records that look anomalous. They are not a substitute for cleansing rules, and their output must be reviewed on a sample like any other transformation. The approach we use for document intelligence, where extraction is scored and low-confidence cases go to a person, applies directly to messy legacy fields. Our AI/ML development team handles this when the volume justifies it.
A worked example
A last-mile logistics operator moving to a new dispatch platform had customer, site and rider data spread across a legacy system, the finance package and a set of regional spreadsheets. The source inventory found that the spreadsheets were the most accurate for site addresses, which changed the whole mapping. Cleansing rules retired sites with no deliveries in a year and merged customers by GST number. The first rehearsal exposed address formats that broke geocoding; the fix went into the script. Reconciliation counted sites per region and sampled fifty addresses with the regional heads. Cut-over ran overnight with a delta the next morning, and the rollback was tested but never needed. The dispatch platform went live on data the operations team recognised as their own.
Team and timeline
A data engineer owns the scripts, rehearsals and runbook; a solutions consultant facilitates the cleansing workshops and mapping; a project manager runs the cut-over. Your side provides a data owner per entity with authority to decide rules and sign reconciliation. Within a platform implementation the migration stream runs four to eight weeks in parallel with configuration, and it is included in the enterprise platform implementation service from $28,000 (from ₹18,40,000). As a standalone piece of work it fits API and integrations from $7,000. Post-migration fixes and delta loads are covered by a Care Plan; see the pricing page.
Before you start: a checklist
- Inventory every source, including spreadsheets and personal exports
- Name a data owner per entity who can decide rules and sign off
- Agree duplicate match keys, survivor rules and dead-record rules in writing
- Map every field with transformations and defaults, and review free-text to picklist mappings
- Schedule two full-volume rehearsals with time to fix between them
- Define reconciliation counts, totals and the sample size per entity
- Write the cut-over runbook with a freeze, a delta and a tested rollback
- Decide when the old system becomes read-only and how long it stays available
Glossary
- Cleansing rule: a written, scripted decision about what to merge, retire, default or correct
- Survivor rule: which of two duplicates keeps its identity and absorbs the other's children
- Migration rehearsal: a full-volume dry run on a copy of the target, with an error log
- Reconciliation: comparing counts, totals and samples between source and target, signed by a business owner
- Freeze: the period during cut-over when the source is read-only
- Delta load: migrating records that changed between the main load and the freeze
Related reading
Why enterprise software implementations fail on adoption shows where dirty data does its damage, and Integrating a new platform with finance, identity and messaging covers the streams that run alongside migration. For loading and constraint behaviour in the most common target database, the PostgreSQL documentation is the reference we point engineers to.
Cleanse first, rehearse twice, reconcile with a signature, and migrate with a rollback you have already tested.
Frequently asked questions
What are the main data migration best practices?
▾
Agree cleansing rules with business owners, rehearse the full-volume migration on a copy at least twice, reconcile counts, totals and samples with a signed sign-off, and cut over in a window with a freeze, a delta load and a tested rollback.
How do you handle duplicates in a CRM data migration?
▾
Agree a match key such as normalised phone or GST number, a survivor rule and a merge rule for child records, generate proposed merges by script, and have the business review and veto before the rehearsal applies them.
How long does data migration take in a platform implementation?
▾
Typically four to eight weeks running in parallel with configuration, most of it spent on cleansing decisions and rehearsals rather than the load itself. It is included in Eazyware's enterprise platform implementation service.