How to Evaluate Chatbot Accuracy Before Launch

Build a representative test set

Sample real question types and record the approved answer, supporting source and expected next step. Include common questions, difficult exceptions and deliberately unanswerable requests. Keep some cases separate from development examples so improvements are tested on unseen questions. Remove personal information from conversation samples before sharing them with reviewers.

Score the result, not fluency alone

Use separate scores for factual correctness, source support, completeness and appropriate refusal or handover. For action tools, check authorisation and the actual system outcome. Define severity: an incorrect opening hour is different from disclosure of another customer’s records. Avoid hiding serious errors inside a single average score.

Exercise adversarial and failure cases

Try contradictory documents, misleading instructions in retrieved text, unavailable APIs and repeated requests. Verify that a missing source does not become a confident invented answer. Run important cases more than once because outputs may vary. Have a subject specialist review failures instead of relying exclusively on another model’s score.

Define a release gate and monitoring

Agree which failures block release and who can accept remaining limitations. Save test versions, model configuration and results so later comparisons are meaningful. After launch, review unresolved enquiries and repeat the regression set after changes. A rising containment rate is useful only if the retained conversations are resolved correctly.

FAQ

Technical References

• NIST: Generative AI Risk Management Profile