A customer assistant that cannot be talked into a price
I replaced LiveChat at Sportsafe UK with an in-house live chat and AI support assistant. It has answered customers on its own since August 2026 and hands over as soon as someone asks for a person. Retrieval is PostgreSQL full-text search with pgvector, so there is no separate search service.
The rule that matters is enforced in code. The assistant never states a price, stock level, lead time or delivery date from its own knowledge. That check runs before the model is called, so no prompt can argue it out of the rule. The scope guard’s vocabulary came from ten years of real customer chats, 11,606 conversations, measured both ways, for what it would wrongly refuse as well as what it catches. The AI agent it replaced got 4 of its 14 refusals wrong, and that was the bar.
An eval suite runs in CI. Its judge model sees the facts the assistant was entitled to use but not the rules that shaped the reply, and it was checked in both directions before its scores were trusted, scoring 5 out of 5 for a known-good reply and 1 out of 5 for a known-bad one. It is also my first Elixir project.