Start with a task your team recognises
Describe the input, required result and person who uses it. In a fictional distributor, the task is to turn an incoming delivery enquiry into a correctly assigned internal follow-up. A convincing summary is only part of that outcome. Ask the vendor to show where the task appears and which details staff receive. Keep the example narrow enough to inspect. Avoid a generic request to demonstrate AI when the business has not decided which activity it wants to improve.
Prepare expected results in advance
Ask the relevant staff to define what a good result includes and what should cause the process to stop for review. Use fictional or appropriately approved material that reflects ordinary variation. Include a missing detail and an ambiguous request, not only a clean example. Keep some cases unfamiliar to the presenter so the session does not test only a rehearsed path. Record the expected result before seeing the output, which helps prevent a fluent but incomplete answer from becoming the new acceptance standard.
Identify what is live and what is simulated
Ask which parts of the demonstration use a working integration and which use prepared data or a mock destination. Both can be useful, but they support different conclusions. A simulated handover may show the interface without proving that your actual system permits the required write. Record those boundaries in the notes. Do not allow a live demonstration to send messages or change real customer records outside an approved test scope simply to make the presentation appear more convincing.
Inspect the destination and failed cases
For each example, open the resulting record or task and compare it with the expected outcome. Check the owner, source reference and unresolved questions. Ask what happens when a required detail is absent or the destination is unavailable. In the fictional distributor example, a task assigned to the wrong depot does not pass because its summary reads well. Separate a correctly handled exception from an unexplained failure. Record any intervention the presenter needed to complete the demonstration.
Question the operating effort
Ask who maintains the source data, handles exceptions and checks changes after deployment. Have the vendor show how an operator finds an affected item and pauses the workflow. A demonstration may conceal preparation work that your staff would later need to do. Record that effort without assuming it makes the product unsuitable. Clarify which operational questions remain unanswered and who will investigate them. Avoid converting a smooth demonstration into a savings estimate before measuring the work required in your own setting.
Choose a next step supported by the evidence
Summarise what was demonstrated, what was simulated and which outcomes were not achieved. Keep vendor statements separate from observations. The next step may be a small pilot, a further integration check or a decision not to proceed. Agree the evidence required for that step and its owner. Do not label the service proven for the whole business after a short session. Preserve the reviewed examples so later comparisons use the same business expectations rather than whichever presentation was most memorable.