Hey Procurement, Did You Buy AI Without Knowing It?

AI procurement governance does not end when the contract is signed. Suppliers can introduce AI later through software updates, new delivery methods, third-party models and contract renewals, changing the commercial arrangement without triggering a new procurement event.

Your organisation may be using supplier-provided AI that procurement never explicitly bought. When suppliers introduce AI after contract award, it can change how data is handled, how services are delivered, what they cost to provide and which third parties the organisation depends on. Procurement needs to know when those changes become commercially material.

Picture this.

You open a piece of software your organisation has used for years. Something has changed. Beside a familiar menu item is that little cluster of stars ✨that has quickly come to mean one thing: AI has arrived.

You click.

It summarises a document. Drafts a response. Analyses some data. Perhaps it even recommends what you should do next.

And it is good. Better still, it is already included in the SaaS subscription your organisation pays for. No new business case. No sourcing event. No new supplier. Nobody in procurement bought an AI product.

Imagine a reasonably ordinary enterprise contract.

You appoint a professional services firm for three years. You evaluate the proposed team, their experience, their methodology and their price. The contract is signed on that basis.

Eighteen months later, the supplier has changed how it works.

Research that was previously undertaken by analysts is partly automated. Drafting is being done with generative AI. Documents are being reviewed by AI before they reach senior staff. Work that once required several people can now be produced by one person supervising a set of tools.

The supplier may be providing exactly the outcome you contracted for. But commercially, quite a lot has changed.

Its cost to deliver may have changed. Its use of your information may have changed. The assumptions behind its staffing model may be different. The source of errors may have changed too, along with your dependency on technology you never evaluated.

And none of it necessarily triggered a procurement event.

How suppliers can introduce AI after contract award

Victorian Government guidance on AI procurement makes the problem unusually explicit.

It identifies four main ways AI can enter an organisation: buying ready-to-use AI or software containing it; commissioning bespoke AI; modifying existing software so it includes AI; or buying a service where AI is an essential part of delivery.

Then it identifies the less obvious risk:

Suppliers can add AI to existing goods and services after the contract has been signed. Suppliers can also use AI during service delivery without the buyer knowing.

That is a very different governance problem from evaluating an AI platform during a tender.

That makes AI procurement governance a lifecycle issue, not simply a technology-selection exercise. (See: Comprara’s AI & Automation in Procurement service).

Traditional procurement controls are strongest before award.

We define requirements. We test the market. We conduct due diligence. Legal negotiates terms. Cybersecurity reviews the technology. Commercial teams challenge pricing.

Then the contract moves into management.

But AI can materially change the service after all of those controls have already been applied.

• A software supplier adds generative AI functionality in a release.
• A technology provider changes from a conventional licence structure to an AI-enabled consumption model.
• An outsourcing provider automates work previously performed by people.
• A consulting firm introduces AI into its delivery methodology.
• A supplier incorporates a third-party model that did not form part of the original solution.

The supplier may not even regard the change as particularly significant. From its perspective, it is improving its product or becoming more efficient.

From the buyer’s perspective, the commercial arrangement may no longer be exactly the one originally assessed.

How AI procurement risk differs from cloud procurement

There is a fair objection to all of this, Procurement has been here before: when cloud services became mainstream, organisations discovered consumption pricing after decades of predictable licences.

Business units could buy technology faster than central IT could govern it. Infrastructure costs became difficult to forecast. Supplier lock-in changed shape. Renewal negotiations became much more important.

AI shares many of those characteristics.

Even the Australian Government’s cloud policy puts stronger cost management, transparency and governance alongside security and capability because cloud buying has long since stopped being a simple licence negotiation.

So procurement should resist the temptation to describe every AI problem as unprecedented. Some of it isn’t. But there are differences that matter.

• The first is that AI can change the nature of the service, not merely the technology used to host it;
• The second is that evaluating the change often requires expertise procurement does not have inside the function;
• The third is that AI can change the economics of the supplier relationship while the contractual economics remain exactly where they were.

Those are not minor differences.

Why supplier AI use creates post-award contract risk

Victoria tells government buyers to manage AI risks throughout contract delivery, including model drift, bias and hallucination, and recommends periodic reassessment as those risks change. It also advises buyers to monitor the reliability of AI as part of supplier performance. The technical risks matter.

However, there is a broader commercial point underneath the guidance.

What you evaluated at the beginning of the contract may not remain static.

That has always been true to some extent. Staff change. Supplier ownership changes. Technology changes. AI increases the speed and potential significance of those changes.

Take enterprise software: a product selected three years ago may now include AI capabilities that did not exist during the sourcing process. The functionality might be optional today and embedded tomorrow. The way data is processed may evolve. The commercial model may evolve with it.

Supplier economics can change while your price doesn’t

Professional services makes the issue particularly easy to see.

Suppose an organisation contracted for a substantial piece of analytical work.

The supplier’s original proposal involved a partner, a director, two managers and six analysts. Its price reflected the labour required to undertake the research, conduct analysis, prepare material and perform quality assurance.

Two years later, the supplier can perform substantial parts of that work with AI.

There is nothing inherently wrong with that.

A more productive supplier should be encouraged.

But the commercial question is obvious.

If the economics of delivering the service have changed substantially, should the economics of buying it remain untouched?

Procurement has spent decades challenging labour pyramids, offshore ratios, utilisation assumptions and rate cards.

AI introduces another variable. How much human effort is actually required to produce the contracted outcome? That does not mean buyers are automatically entitled to every productivity gain a supplier creates.

Suppliers invest in technology precisely because they expect to benefit from doing so. A contract based on outcomes should not suddenly become an audit of every efficiency the supplier finds.

But neither should procurement assume that a commercial model designed around one method of delivery remains competitive indefinitely when the method changes.

That becomes particularly important at renewal.

Why contract renewal is becoming a critical AI procurement control

Much of the procurement discussion around AI concentrates on acquisition. The more interesting control point may eventually be renewal.

Think about what normally happens:

A large incumbent software agreement approaches expiry. The business wants continuity. Users do not want disruption. The supplier presents its new product structure.

Some AI capability has been added.

Another AI capability is now separately priced.

A premium tier includes features the business believes it needs. Consumption charges have appeared somewhere that previously had fixed economics. The renewal is presented as an extension of an established relationship. Yet, commercially, it may be much closer to buying a different service.

The same issue applies to outsourcing and professional services.

Before renewing a significant supplier, procurement increasingly needs to understand what has changed in the supplier’s delivery model since the contract was awarded.

• Has AI been introduced?
• Where?
• Has it changed staffing requirements?
• Has it changed the supplier’s cost base?
• Has it changed how organisational data is handled?
• Has it created dependencies on third-party AI providers?
• Has the supplier’s pricing model changed?
• Has functionality that used to be included become an AI add-on?
• Could the organisation disable that functionality if it chose to?
• What happens to the service if it does?

Those questions belong in commercial review long before a renewal recommendation reaches an approval committee. That is why renewal needs to be treated as a fresh commercial decision, not an administrative extension.

What supplier AI disclosure and contract clauses should cover

Victoria Government’s guidance suggests procurement documents can require suppliers to disclose whether and how they intend to use AI to deliver goods and services. Contracts can define permitted AI uses. They can prevent suppliers from introducing new AI without buyer approval. They can also require suppliers to provide a mechanism allowing the buyer to opt out of, or disable, newly introduced AI.

The guidance even provides a simple professional-services example.

A buyer expects a report to be prepared by the skilled personnel named in the supplier’s proposal. The resulting work contains factual errors and fabricated references. It later emerges that the supplier used AI without approval and without adequate human oversight. That is easy to recognise as an AI risk.

But consider the same scenario without the errors.

The report is good. The supplier used AI extensively. Does the buyer care?

The answer will depend on the contract and the service.

In some cases, probably not. In others, absolutely:

• If confidential data has been placed into a third-party model, it matters.
• If the organisation selected named specialists because their judgement was central to the engagement, it matters.
• If regulatory obligations require human review, it matters.
• If AI has reduced a supplier’s delivery effort so significantly that the existing commercial model no longer represents value, it may matter.
• If the supplier has transferred an important part of the service to another technology provider, it may matter from a resilience and concentration perspective.

The objective is not to police every use of AI in a supplier’s business.

That would be unworkable. It is to define when a change in AI use becomes commercially material.

Who should govern supplier AI changes?

There is another reason existing contract-management models will struggle: AI expertise sits outside procurement.

The US Government Accountability Office recently examined 13 AI acquisitions across four federal agencies. Officials reported difficulty accessing technical experts such as data scientists to help evaluate supplier proposals. They also reported difficulty understanding AI-related costs. That is US government evidence, and it should not be treated as a survey of Australian CPOs. But the underlying problem travels well.

Procurement may own the commercial process while depending on others to tell it whether the supplier’s technology has materially changed. The business understands the use case. Technology understands the architecture. Cyber understands security.

Legal understands the contractual exposure. Data and privacy specialists understand information use.

Procurement needs enough information from all of them to answer a different question:

Has this change altered the commercial proposition we originally agreed to?

If nobody owns the commercial picture of AI across the organisation, each individual supplier change can look too small to matter.

Collectively, they may be moving substantial cost, capability and dependency outside the organisation.

How AI changes contract management

Most large organisations already have mechanisms for managing supplier change.

They have contract variations, key-person provisions, technology roadmaps, security reviews, price reviews, audit rights, supplier performance processes … and so on.

The requirement is not to invent an entirely separate AI contract-management bureaucracy. It is to make existing controls sensitive to the ways AI changes a commercial arrangement.

Four procurement governance decisions are particularly important:

1. Define when new supplier AI use triggers reassessment

Not every software update deserves a procurement review. A material change should.

A new AI subprocess handling sensitive information might qualify. A change in pricing architecture might qualify. Replacing substantial contracted human activity with automated delivery might qualify. Introducing a third-party model into a critical service might qualify. The important thing is to define the trigger. An organisation that simply adds “review AI annually” to a contract-management checklist has not solved the problem.

2. Require disclosure of material supplier AI use

The answer is unlikely to be “all AI must be disclosed”. That will become increasingly unrealistic as AI becomes embedded in ordinary business tools. The useful requirement is disclosure of AI use that is relevant to the organisation’s data, risk, performance or commercial position.

3. Review AI changes before contract renewal

Renewals should ask what changed, not merely what the supplier wants to charge next year. For strategic technology, outsourcing and professional-services suppliers, that increasingly includes changes in AI capability and delivery.

This is where category management, supplier relationship management and contract management start to overlap:

• The category strategy may determine the organisation’s overall exposure.
• Supplier management should understand how a critical partner is changing.
• Contract management provides the rights and obligations.

The renewal brings the commercial consequences together.

4. Assign accountability for the commercial decision

A risk review can conclude that new AI use is acceptable. That does not mean the commercial arrangement is still good. A cybersecurity approval can say the architecture is secure. That does not mean the organisation should accept the new pricing model. Legal can confirm that a contract variation is enforceable.

That does not mean it represents value.

Somebody still has to make the commercial call. For major supplier relationships, that responsibility should be explicit.

AI exposes a wider post-award contract management problem

It would be easy to treat this as another argument for better AI clauses. That would miss the more important point.

The arrival of AI exposes a weakness that already exists in many procurement functions: they invest far more effort in selecting suppliers than in continually testing whether the commercial arrangement still makes sense after award.

AI simply makes that weakness harder to ignore.

During the lifetime of a contract supplier capabilities change, cost structures change, technology changes, dependencies change, etc.

AI speeds that process up. For the CPO, that means the commercial work cannot end at signature. Nor can it wait until six weeks before renewal.

Do you know where AI has already entered your supplier base?

You probably do not need to review every supplier that uses AI.

You do need to know where AI has materially changed your organisation’s data exposure, delivery model, supplier economics, technology dependencies or contractual position,  particularly before a major contract is renewed.

Comprara works with enterprise and government procurement teams to assess AI readiness, procurement technology and governance, and to build practical commercial controls around emerging technology. If you are trying to work out where supplier AI has already changed the deal, talk to us.

Frequently asked questions

Can a supplier introduce AI after a contract is signed?

Yes. AI can be introduced through software updates, changes in service-delivery methods, new subcontractors or third-party models, or additional functionality introduced during the life of an existing agreement. Whether the supplier needs approval will depend on the contract and the materiality of the change.

What should AI clauses in supplier contracts cover?

The appropriate clauses depend on the service, but material issues can include disclosure of AI use, permitted and prohibited uses, treatment of confidential data, human oversight, use of third-party models, notification of material changes, audit or information rights, and the buyer’s ability to opt out of or disable newly introduced functionality.

When should new supplier use of AI trigger contract reassessment?

Reassessment is most useful when AI materially changes data handling, risk, pricing, supplier performance, the delivery model, reliance on named personnel, or dependency on another technology provider. The objective is not to review every use of AI. It is to identify changes that alter the commercial proposition.

Why should AI be reviewed at contract renewal?

Renewal is one of the few points where procurement can step back and test whether the service being bought is still the service originally negotiated. AI may have changed functionality, delivery effort, supplier costs, pricing structures, data use and technology dependencies during the contract term. Those changes should form part of the commercial renewal decision.