What an AI code audit should actually tell you

A report can look thorough and still leave you stuck. Thirty pages of warnings, a colourful score and a recommendation to “improve test coverage” don't tell you whether to launch on Monday.

A useful audit should help you decide what to fix, what can wait and what needs a closer look. You should understand the answer even if you didn't write the code. Your developer should have enough detail to act on it without starting the investigation again.

Here, an AI code audit means a review of software built with AI assistance. If the product itself uses a model, reviewing that model's behaviour and data handling is a separate area to agree. The name alone doesn't tell you what is included.

Start with the decision you're trying to make

Tell the reviewer what happens next. Are you inviting ten trial users, accepting payments, handing the app to another team or preparing for a public launch? Those plans give the review something concrete to assess.

Imagine a booking tool moving from an internal trial to paying customers. The review should follow a booking from availability through payment, confirmation and cancellation. Looking only at whether the screens load would miss much of the work the business relies on.

Agree the boundaries in writing. Which repository version is being reviewed? Is there access to a test environment, deployment settings, database rules and accounts with different permissions? Are performance testing, infrastructure and third-party integrations included? Record missing access as a limitation, because reading code cannot confirm every detail of the running service.

For security checks, OWASP's Application Security Verification Standard provides testable requirements for web applications. Ask which version and requirements the reviewer will use. A broad statement that an app was “checked against OWASP” gives you much less information than a record of the checks performed and their results.

Every important finding needs evidence

“Access control is weak” sounds worrying, but it isn't a useful handover. A finding should identify the affected behaviour, explain the consequence and show how the reviewer reached the conclusion.

Here's a hypothetical example from a customer portal:

Now there's something to fix. The report should include the affected handler and reviewed version, safe reproduction steps using test data, and the expected result. It should also distinguish a reproduced issue from a concern inferred from reading the code.

In this example, the fix belongs in the trusted server-side permission check. Hiding invoice links in the browser wouldn't address it. OWASP's authorisation guidance recommends checking permissions on every request and testing the access rules.

The acceptance test should be equally clear: a user can download their own company's permitted invoices; requests for another company's invoices are denied without exposing the document. Include both cases. A fix that prevents everyone from downloading invoices isn't finished.

Automated tools can contribute useful leads. The report should explain which alerts were confirmed, which need investigation and which were dismissed with a reason. An unexplained scanner export leaves that judgement with you.

Make the priorities useful to your team

If every item is urgent, the report hasn't helped you choose. Priorities need the application's context: who can reach the problem, what they could affect, what protection already exists and how confidently the issue has been established.

A numerical security score can help describe severity, but it isn't the whole decision. FIRST's CVSS guidance explicitly says that a Base score measures severity and should not be used alone to assess risk. Ask for the explanation behind a priority, including the assumptions that could change it.

For the hypothetical portal, a sensible initial plan might look like this:

1. Before onboarding customers: fix the reproduced cross-company invoice access, check related document routes and verify the permission tests. It exposes customer information through an ordinary account.
2. Before relying on the service for daily work: demonstrate a database restore in a separate environment. The current evidence shows scheduled backups, but no successful recovery test.
3. In the next maintenance cycle: consolidate duplicated invoice formatting. It makes changes awkward, but the review hasn't found an immediate effect on access or invoice values.

Those are example decisions, not a universal ranking. If the app already holds live customer documents, the first item also needs an immediate containment decision and investigation of possible exposure. If the duplicated code calculates totals differently, it belongs much higher on the list.

Ask for effort estimates with assumptions attached. “Small change, provided all downloads use this handler” is more honest than a precise deadline before anyone has checked the other routes.

Look beyond security findings

A broader engineering audit should also explain how easy the application will be to operate and change. Can someone else build it? Where are the business rules? What happens when an external service times out? How would you notice a failed background job?

For maintainability, ask the reviewer to trace a realistic change. Suppose customers need a second billing contact. Which screens, validation rules, database changes and exports would need updating? That exercise can reveal scattered assumptions more clearly than a general complaint that the code is messy.

The report should also say what is worth keeping. A straightforward structure, clear data boundaries or good tests around payments are useful foundations. Recommendations to replace them need a reason tied to your plans, plus an explanation of the cost and disruption.

Whether a person or an AI tool produced a function doesn't change what the review needs to establish. Assess its behaviour and the evidence around it. Don't accept “AI-generated” as a diagnosis.

Leave with a plan someone can follow

Before the engagement ends, ask for a short walkthrough. A useful handover includes:

• A plain-English summary of the decision, its conditions and remaining uncertainty.
• A scope record covering the version, environments and checks, including exclusions.
• Findings with evidence, consequences and a reason for each priority.
• A work list with owners, estimated effort and a way to verify each fix.
• A clear agreement on whether retesting is included and what will trigger another review.

Where several findings share a cause, the plan should address that cause. Three missing permission checks may call for a consistent authorisation approach and tests, rather than three isolated patches. NIST's Secure Software Development Framework includes addressing the root causes of vulnerabilities to help prevent recurrence.

An audit is a review of a particular system with particular access at a particular time. It cannot promise that every defect has been found. Unchecked areas and unresolved questions should be visible, so you can decide whether further testing is needed.

You should finish with fewer unanswered questions and a manageable next step. If you can explain what needs doing, why it matters and how you'll know it's done, the report is earning its keep.

Further reading

• OWASP ASVS — a reference for agreeing application security checks.
• OWASP Authorisation Cheat Sheet — practical guidance on permissions and their tests.
• FIRST CVSS user guide — how to interpret vulnerability scores.
• NIST SSDF, version 1.1 — secure development practices across the software lifecycle.

For the wider launch decision, read Your AI-built app works. But is it ready for production?