Software Audit in New Zealand

Buy it before the decision, not after the problem

An audit is worth its cost when it sits in front of something expensive and hard to reverse.

• You are about to inherit a system. The developer is leaving, or the work is coming in-house. Find out what you are accepting before you accept it.
• You are buying a business and a large part of what you are paying for is software. What it actually is changes the price and the warranties.
• You have been told it needs a rewrite. That may be true. An independent read tells you whether it is, and what the alternatives cost.
• Your quotes differ by a factor of five. That gap is usually five different assumptions about the state of the system, and an audit makes everyone quote the same thing.

In each case the audit is a small fraction of the decision, and it frequently changes it.

Ask to see a report before you engage

The quality gap between audit reports is enormous and it is instantly visible. Ask each firm for a redacted example.

A weak report is a long undifferentiated list of observations, most of them true and none of them ranked, ending in a recommendation to rebuild.

A good report tells you:

• What it is built on, with versions, and whether each is still supported.
• Known vulnerabilities in the dependencies, with severity.
• Who else could maintain it — is this a common stack in New Zealand or a niche one?
• What it costs to run, including hosting and licences nobody has looked at for three years.
• How it is released and tested, and whether one person is the only one who can do it.
• What documentation exists, and what a new developer would face in week one.
• The two or three things that would hurt most, with a cost and an effort estimate against each.

The ranking is the part that makes it useful. A list without priorities transfers the work back to you.

Ask about independence, plainly

A firm that audits and then quotes to remediate has an interest in a long list of findings. That does not make them dishonest, and often the auditor is the person best placed to do the work — but you should know the arrangement before you read the report.

The useful question is simpler: will you tell us if it is fine?

An auditor willing to report that the system is in reasonable shape has just talked themselves out of the follow-on project. That is worth a great deal, and you can usually tell within one conversation who would say it.

Get the access sorted early

The audit itself takes two to four weeks. The access frequently takes longer, and it is the most common cause of delay.

What is needed:

• Read access to the source code repository — all of it, including whatever is not in the main one.
• Access to hosting, or at least a full description of it.
• An hour with whoever knows the system, and an hour with whoever runs the business.
• The hosting, licensing and third-party bills.
• Any documentation, however out of date.

If the current developer is the one leaving, start that conversation early. How willingly the access arrives is itself a finding, and it tells you what the handover will be like.

The findings that actually matter

Audits generate long lists. In practice a small number of things determine whether a system is a liability:

1. Is it patchable? An unsupported runtime or framework is a security problem with a date attached.
2. Can anyone else work on it? A stack nobody in New Zealand hires for is a commercial risk regardless of code quality.
3. Can it be deployed by more than one person?
4. Is there a tested backup and restore? Not a backup — a restore somebody has actually performed.
5. Who owns the accounts? Domain, DNS, hosting, app stores, certificates. This is where inherited systems most often go wrong.
6. Is there anything that would fail an audit of yours? Customer data in a test environment, credentials in the repository, personal information with no retention policy.

Ask each auditor to answer those six explicitly. Everything else is detail against them.

For due diligence, ask for commercial framing

If the audit is part of an acquisition, a list of technical findings is not enough. What the buyer needs is:

• What must be spent in the first twelve months, as a number.
• Which key people the system depends on, and what happens if they leave.
• Whether any licensing or open-source obligation affects the deal.
• Whether the software is actually owned — contractors without an assignment clause are a recurring problem.
• What would be needed to scale it to the buyer's plan.

That last one is often the finding that moves the price.

Comparing five quotes

Read for:

• Who showed you a redacted example report?
• Who ranks findings by cost and risk rather than listing them?
• Who was clear about whether they would also bid for the remediation?
• Who told you exactly what access they need and how long your side takes?
• Who would be willing to report that the system is fine?

Describe the system, why you are asking now, and what decision the audit is feeding. Firms who do this regularly will ask what decision it is for before they ask what the software does.