Data Migration in New Zealand

Profile the data before anyone quotes a fixed price

The single most useful thing you can buy at the start of a migration is a week of somebody counting things.

For each table or record type: how many rows, how many are duplicates, how many have empty required fields, how many distinct values are in each column, and what the oldest and newest records are. It is unglamorous and it predicts the project cost better than anything else.

What it always turns up:

• A field labelled one thing that the business uses for another, because there was nowhere else to put it.
• A period — often around a system change or a staff departure — where the data is shaped differently.
• Free-text where a list should have been, with fourteen spellings of the same value.
• Duplicates that are obvious to a person and invisible to a matching rule.
• Rows that cannot be explained by anyone still working there.

Every one of those is a decision the business has to make, and each is a change request if it surfaces mid-project. A fixed price quoted without profiling is a guess with a number attached.

Decide what not to move

This is where the money is, and it is the question least often asked.

Migrating everything is the default because it feels safe. It is also the most expensive option, and it carries fifteen years of accumulated mess into a system you are paying to be clean.

Work through it deliberately:

• How much history does the business genuinely need live? Three years of transactions is often plenty.
• Which customers matter — all of them, or those with activity since a date?
• What must be kept for compliance but never looked at? That goes to an archive, not into the new system.
• What can simply be deleted, because you have no reason to hold it any more?

Every record you exclude is one nobody has to map, clean, test or explain — and the new system is faster for it. Get the decision in writing, signed by someone who can make it, so nobody discovers it in month two.

Cleansing is your team's job, not the developer's

A developer can find you every duplicate, every missing value and every record that fails a rule. They cannot tell you which of the three "Smith Builders" records is the real one, or which contact should own an account nobody has touched since 2021.

Plan for that time explicitly. The pattern that works:

1. The developer produces exception reports.
2. Your people work through them in the old system, where they know what they are looking at.
3. The next rehearsal picks up the corrections automatically.

Cleaning in the old system rather than in a spreadsheet matters: it means the work is not lost if a rehearsal is rerun, and it keeps one version of the truth.

Rehearse at least twice

A migration attempted once, live, on the weekend, is not a plan.

• Rehearsal one tells you what is broken. Expect it to be ugly. That is what it is for.
• Rehearsal two proves the fixes and puts a realistic dataset in front of the people who will use it. This is when your team finds the things nobody thought to specify.
• The dress rehearsal is timed. If the migration takes eleven hours, everyone needs to know that before the weekend it has to happen in.

A quote with a single attempt is quoting a happy path that migrations do not have.

Agree how you will prove it worked

Decide the reconciliation before the migration, not after, and get it in writing:

• Record counts by type and by year, old system against new.
• Financial totals matching to the cent — debtors, creditors, stock value.
• A sample of records checked by the people who use them daily, not by the project team.
• A named list of everything deliberately excluded, and why.
• A written statement of what did not migrate and where it now lives.

Anyone who cannot describe this is asking you to take the result on trust, and the moment to discover a gap is not three months later when someone needs a 2023 invoice.

Keep the old system readable for a while

Do not switch it off on go-live day. Keep it available read-only for at least a few months — long enough to cover a full reporting cycle and the first audit. It is cheap insurance and it removes most of the fear from the weekend.

Put a date on it, though, and decommission deliberately. A "temporary" read-only system still running four years later is a licence bill and a security liability nobody owns.

Migration is a good moment to stop holding things

Under the Privacy Act you should not keep personal information you no longer have a purpose for. A migration is the natural point to act on that: the alternative is copying it into a new system and inheriting the obligation for another decade.

Balance it against retention rules — IRD requires financial records for seven years, and public sector records have their own regime. Decide the retention policy during planning and let it drive what moves.

Comparing five quotes

Read for:

• Who proposed profiling the data before giving a fixed price?
• How many rehearsals are included?
• Who described the reconciliation you would receive?
• Who asked what you do not want migrated?
• Who was honest about how much of your team's time this needs?

Describe the systems, roughly how many records and how many years, and anything you already know is messy. Developers who have done migrations will ask to see the data before they price it — and that request is a good sign, not a delay.