Thirty Questions for Any AI Vendor
The AI services market has expanded far faster than the pool of people who have actually delivered anything into production. These questions separate the two, and surface the commercial terms that matter long after the pitch.
Run the same list past every vendor and record the answers. Ticks save in this browser so you can compare properly rather than from memory.
Vendor Due Diligence Checklist
Same thirty questions to every vendor. Record the answers.
Your ticks are saved in this browser, so you can work through the list over several sessions.
01Capability and evidence
0/5Delivered is different from demonstrated.
02Approach and delivery
0/6How they scope, and what happens when reality intervenes.
03Ownership and lock-in
0/6The section buyers skip and later regret.
04Data and security
0/7Accountability stays with us regardless of the contract.
05Commercials
0/5A total against the same scope, from every vendor.
06Support and exit
0/5After go-live, and after the relationship.
General procurement guidance, not legal advice. It does not replace your organisation's procurement process or legal review of contract terms.
What You Are Testing For
Three things separate vendors who will deliver from those who will produce an impressive deck and an expensive proof of concept.
Evidence of production delivery
Ask specifically about systems running in production today, with users, being maintained. A great many AI engagements end at proof of concept, and a vendor whose portfolio is entirely pilots has not yet solved the hard part.
What you own at the end
The question buyers most consistently skip and most consistently regret. If the build lives in the vendor’s environment on the vendor’s licences, you have rented a capability rather than acquired one, and your position at renewal is weak.
Willingness to say no
A vendor who agrees enthusiastically with every requirement has either not understood the constraints or is not telling you about them. The most useful answer in any of these conversations is a clear "that will be harder than you think, and here is why".
The Six Sections
Thirty questions grouped so each section can be assessed by whoever is best placed to judge it.
Capability and evidence
What they have actually delivered, for whom, and whether it is still running.
Approach and delivery
How they scope, what happens when discovery changes the picture, and who does the work.
Ownership and lock-in
Whose environment, whose licences, whose intellectual property, and what you can take away.
Data and security
What they touch, where it goes, and what governs their access.
Commercials
Total cost, what changes cost, and what happens at renewal.
Support and exit
What happens after go-live, and what happens when it ends.
The Answers Worth Probing
Four responses that are common, superficially reasonable, and worth following up before you sign anything.
"We have delivered dozens of AI projects"
Ask how many are in production with real users today, and how many are still being maintained. The gap between projects delivered and systems running is where the AI services market’s reputation problem lives, and it is a fair question that good vendors answer readily.
- Ask specifically how many are in production with users right now
- Ask for a reference from a client twelve months post-delivery
- Ask what happened to the ones that did not reach production
- Ask who at their firm would actually do your work, by name
"We will start with a proof of concept"
Reasonable, provided the path beyond it is defined upfront. Proofs of concept that exist to justify the next phase rather than to answer a specific question are how organisations accumulate impressive demonstrations and no working systems.
- Ask what specific question the proof of concept will answer
- Ask what would constitute a negative result, and what happens then
- Ask what it takes to move from proof of concept to production, and what that costs
- Ensure the output is something you own and could hand to another vendor
"Your data is secure with us"
Too general to assess. What you need is specific: which country, which sub-processors, whether content trains models, how long it is retained, and who at their end can access it. A vendor who cannot answer in writing has not thought about it.
- Ask which countries data is stored and processed in
- Ask for the sub-processor list, including the model provider
- Ask whether content is used for training, and get the answer in writing
- Ask what security certifications they hold and what scope those cover
"Pricing depends on scope"
Fair as an opening position, unacceptable as a final one. Before comparing vendors you need a total figure against the same documented scope, including running costs and what post-launch changes cost.
- Insist on a total including build, running and first-year support
- Ask the day rate and the hourly rate for post-launch changes
- Ask what happens to pricing at renewal and how much notice is given
- Have every vendor quote the same written scope so totals are comparable
Next Steps
AI Project Cost Estimator
Sanity-check any quote against a realistic total cost including your own time.
Estimate the cost →How to Choose an AI Consultant
The longer written guide to running a fair selection process.
Read the guide →Frequently Asked Questions
Three is the practical number for most engagements. One gives no market reference, and beyond four the process consumes more effort than the decision warrants and tends to stall. Give each the same written scope, ask the same thirty questions, and record the answers. The differences that matter usually emerge in the ownership and delivery-evidence sections rather than in the capability pitch, which tends to look similar across the market because everyone is describing the same underlying technology.
It depends on the shape of the problem more than the size of the firm. Specialists generally bring deeper technical judgement and move faster on the build itself. General consultancies bring change management, sector knowledge and the capacity to handle the organisational work around the technology, which is frequently where projects actually fail. For a well-defined technical build, a specialist usually delivers better value. For something requiring substantial process redesign and stakeholder management, the broader capability often earns its premium. Ask either one who specifically will do the work.
At minimum you should own the configuration, prompts, evaluation sets and any custom code built specifically for you, and receive documentation sufficient for another competent provider to maintain it. Vendors reasonably retain their own pre-existing tooling and frameworks, and that is normal. The distinction is between their generic platform and your specific implementation. What you should not accept is an arrangement where your business logic lives in their environment and cannot be extracted, because that converts every renewal into a negotiation you cannot walk away from.
Ask questions where the quality of the answer is legible even without deep technical knowledge. Ask how they would evaluate whether the system is working, and listen for whether they describe a measurable approach or an impressionistic one. Ask what would make them recommend against proceeding. Ask them to explain their proposed approach in plain language, since genuine understanding usually survives translation and bluffing does not. Reference calls with clients twelve months post-delivery are also more informative than any technical assessment you could run yourself.
Milestone-based payment tied to demonstrable deliverables, rather than time elapsed, aligns incentives well and is standard for competent providers. A modest deposit is normal. Full payment upfront is not, and paying substantially in advance of delivered value removes your only real use if things go wrong. For longer engagements, tying a final milestone to successful production deployment rather than to handover of code is worth negotiating, since it keeps the vendor engaged through the phase where most problems surface.
No. It is a practical prompt list to help you ask better questions and structure a comparison, and it is not exhaustive. It does not replace your organisation’s procurement process, legal review of contracts, or professional advice on obligations specific to your sector. Use it alongside those rather than instead of them, particularly for engagements involving personal information or regulated activity.
Ask Us All Thirty
We would rather answer the uncomfortable questions in writing before an engagement than have a client discover the answers in month four.