An AI assistant that works in a demo and fails in production is not a broken model. It is an untested one. The gap between the two is almost entirely made of inputs nobody thought to try.
We test in three passes. The first is the happy path: the request the assistant was built for, phrased the way the client phrased it when they described the job. If this fails, nothing else matters.
The second pass is the same request phrased badly. Half a sentence. A voicemail with background noise. A question asked three ways in one message. Real customers do not write clean prompts, and an assistant that only handles clean ones will look excellent right up until launch day.
The third pass is the one that catches most problems: requests the assistant should refuse. Questions outside its knowledge, asks that need a human decision, anything touching money or a commitment the business has not authorised. We want it to hand off, not to guess. An assistant that says "let me get someone" is doing its job.
Only after all three does anything go in front of a customer, and the first week runs with a person reviewing output before it sends. That review is not a formality — it is where we find the last set of cases, the ones that only exist in this particular business.