Independent AI review
a delivery confidence rating from reviewers who cannot win the fix

What do you receive?

A delivery confidence rating, findings traced to their evidence, and recommendations with no Precision work attached. If a fix is needed, you also receive a transition pack that any firm can start from.

• A delivery confidence rating on a defined scale, which the scope sets out with the review method.
• Confidential interviews with the people who sponsor, build, run and use the system.
• Findings on six questions: does the value case still hold; is the data ready; what will it cost to run in production; are the controls adequate; where do people oversee and approve; and has security been scoped. Each finding is traced to its evidence.
• The standard, named: your adopted standard, or a public framework such as the Commonwealth policy, the NSW framework, the APRA letter or the Guidance for AI Adoption.
• Recommendations written to stand without us.
• A transition pack where a fix is needed: the evidence base, the recovery objectives in order, and a short guide organised so another firm can start in days. It is included in every review. What the transition pack holds

What does engineer testing add?

Testing shows how the system behaves on your data, beyond what its documents and its builders say. It is available on request, scoped to your system.

Our testing method has not yet run on its first system. Until it has, we lead with the delivery confidence review, and a scope that adds testing says so.

An Independent AI Review with testing adds, to everything above:

• an evaluation harness run on your data: accuracy on an agreed question set, robustness, drift and bias where relevant;
• tests of the agent's permissions and of each point where a person must approve;
• your data flows mapped against what the hosting arrangement claims;
• misuse and prompt injection scenarios at design level, with any penetration testing carried out by a specialist partner under a disclosed subcontract, its fee shown as a separate line;
• a report written for your board, with every finding traced to its evidence and the limits of the testing stated;
• the harness handed over as built and documented, with a fixed period for questions and an optional support retainer;
• for government, inputs to the impact assessment and to any review committee referral.

Before testing starts, the scope names a test environment and an approved data extract, and includes either a cooperation letter from the supplier or a plain statement that the testing is black-box. You confirm in writing that you own, or are authorised to permit testing of, every system in scope, including any hosted by a third party, and you name anything out of bounds. Bias testing uses sensitive information only with your authority, in de-identified form where possible, after a privacy impact assessment.

Planned elapsed time: four to eight weeks. The work is scoped to your system, with any testing partner's fee shown as a separate line.

What happens if the finding is that the system needs fixing?

You choose who fixes it, and it will not be us. After an independent review we do not build, fix or run that system, or a system that replaces it, for 24 months, and the transition pack lets another firm start in days.

The transition pack holds:

• the evidence base behind the findings;
• the recovery objectives, in order;
• a short guide for the firm that takes on the work;
• the testing harness, where we built one.

How do you keep a review independent?

By rules we publish and apply to every scope. The scope names the reviewers and engineers, says whether we might compete with the supplier whose system is under review, and discloses the fees we have earned from you.

• We give no independent review of a system we designed, built, recovered or operate, or that people we supplied helped design or build.
• An engineer who has worked on a competing build for you is excluded from the review.
• When we test a system another supplier built, we protect that supplier's confidential information, and the engineers who tested it do not build anything that replaces, competes with or integrates with it for you for 12 months.
• We decline the review where our delivery fees from you over the past 24 months exceed a threshold we set.
• The review lead signs the findings. Neither the firm's owner nor a delivery lead edits them.
• Reviewers' pay and targets do not depend on selling delivery work to the clients they review.
• What a reviewer learns on a review is used only for that review.

Each line above restates one of the fourteen conflict rules we publish and apply to every scope. Read our independence and conflict rules

What happens after the review?

If the system stays in production, Continuous AI Oversight keeps the view current. You receive a quarterly delivery confidence statement for each system, an annual review and reporting to your committee.

Oversight is only for systems we did not design, build, recover or operate, and only where your contract with the supplier gives us the access we need. A named reviewer and engineer work on it each month, over a 12-month term that you can renew.

Fees are scoped to the systems it covers, as a monthly fee agreed before the term starts.

What do you offer government agencies?

A delivery confidence review of a single system, and on request the testing on its own, as an AI System Technical Assessment.

An AI System Technical Assessment is the testing without a delivery confidence rating or advice on the programme's governance: the evaluation harness run on agreed data, the agent's permissions and approval points tested, data flows mapped against the hosting claim, and a technical report with every finding traced to its evidence and the limits of the testing stated. The harness is handed over. We work in your environment by default, and the scope allows for your onboarding time and any clearances you require. It is planned at three to six weeks from data access.

It uses the same testing method, which has not yet run on its first system, and the scope says so.

Where our work defines requirements for a later procurement, we declare our interest in that later work and accept any probity measure you set, including exclusion from it.