Shipping AI Features That Users Actually Trust

The failure mode of AI features is not that the model is wrong. It is that the feature was added because the technology was available rather than because a problem needed solving, and users can tell. What follows is how we scope this work: which problems fit, what it costs to run, what has to be checked, and where the honest answer is to build something conventional instead.

Solve a real problem, not a demo

Good candidates share a shape. There is a repetitive task involving unstructured text or images, a human is currently doing it slowly, the cost of an occasional wrong answer is low or a human catches it, and there is enough volume to be worth automating. Summarising incoming enquiries, drafting a first reply, extracting fields from an invoice, classifying tickets, searching across documents nobody has time to read.

Bad candidates share a shape too. A deterministic rule would do it correctly and cheaply. The wrong answer is expensive, irreversible or regulated. The task needs a guarantee rather than a probability. Or nobody was doing it at all, which usually means nobody needed it done.

When not to use AI

This deserves its own heading because it is the most valuable judgement in the whole area, and the least often exercised. Do not use a model for arithmetic, for anything that must be exactly reproducible, for a decision that affects someone's money or rights without a human in the loop, or for a task where a lookup table is the actual answer.

• Use a rule when the logic is stable and enumerable. "If the order is over Rs 5,000, free delivery" is not a machine learning problem.
• Use search when the user knows what they are looking for. A good search box beats a chatbot that paraphrases your documentation.
• Use a form when you need structured data. Extracting fields from free text you asked the user to write is solving a problem you created.
• Use a person when the stakes are high, the volume is low, or being wrong damages trust more than being slow does.

Retrieval over your own documents

Retrieval-augmented generation, where the system searches your documents and answers using what it found, is the most useful pattern for most businesses. It works because it grounds answers in text you control and can cite. It fails, predictably, on retrieval rather than on generation: if the search step returns the wrong three passages, no amount of prompt work rescues the answer.

• Fix the corpus first. Contradictory, outdated policy documents produce contradictory, outdated answers, and now with a confident tone attached.
• Chunk by meaning, not by character count. A chunk that cuts a table in half retrieves badly.
• Combine keyword and vector search. Pure semantic search misses exact terms like part numbers, policy codes and names, which is precisely what people search for.
• Always cite the source document and let the user open it. Being able to check is most of what makes it trustworthy.
• Say "I could not find this" when retrieval returns nothing relevant. A system that always answers is a system that sometimes invents.
• Evaluate on real questions from real users, scored by a human, before and after every change. Vibes are not a test suite.

Automating customer support without making it worse

Support is the most common first AI project and the easiest to get wrong, because the failure is visible to customers. The version that works is narrow: handle the handful of questions that make up most of the volume, answer from your own documented policies, and hand over to a person the moment it is out of scope or the customer asks.

• Start by reading a hundred real tickets. The top five questions are usually more than half the volume, and some of them are better fixed in the product than answered.
• Make the escalation path obvious and one tap away. Trapping people in a bot is the fastest way to lose them.
• Never let it promise refunds, discounts, dates or exceptions. Those need a human and a policy.
• Tell the user they are talking to an automated system. People forgive a bot for being limited; they do not forgive being deceived.
• Measure resolution and satisfaction, not deflection. Deflection counts people who gave up.

Guardrails for anything user-facing

Generated text that reaches a customer or a public page is published content, and it carries the same liability as content a person wrote. The controls are not exotic but they have to exist before launch, not after an incident.

• A human reviews anything that goes out under your name. Assisted drafting, not autonomous publishing.
• Constrain the output shape: structured JSON with validated fields beats free text you then have to parse and trust.
• Keep prompts and model versions in version control. An answer that changed because someone edited a prompt in a console at midnight is unexplainable.
• Log inputs and outputs so a complaint can be investigated, with personal data handled to the same standard as the rest of your system.
• Treat prompt injection as a real attack when the model reads untrusted content. Text in a document can and will try to instruct your system.
• Never put anything in a prompt you would not want disclosed, and be explicit with users about what is sent where. Regulatory expectations here are moving, as covered in what AI regulation changes for client work.

Controlling what it costs to run

The prototype is cheap and the production system is not, because cost scales with usage and with how much context you send. Model this before launch rather than discovering it in a bill.

• Work out cost per request, then multiply by realistic monthly volume. If that number is uncomfortable, the design is wrong, not the pricing.
• Use a small model for the easy majority and escalate to a large one only when needed. Most requests do not need the biggest model available.
• Cache aggressively. Identical and near-identical questions are extremely common in support and search workloads.
• Trim the context. Sending an entire document when three paragraphs would do is the most common source of avoidable spend.
• Cap spend per user and per day, and alert on anomalies. A loop that retries on failure can spend a month's budget overnight.
• Re-check the arithmetic quarterly. Prices and model options move fast enough that last quarter's optimal choice often is not this quarter's.

Keep the human in control

The interface decisions matter as much as the model. Show what the system used to reach its answer. Make editing the output easier than regenerating it. Let people turn the feature off. Never make an automated action irreversible without confirmation. Confidence displayed without grounds is worse than no confidence indicator at all.

Users forgive a system that says it does not know. They do not forgive one that was confidently wrong about their money.

Scope tightly, then expand

Ship one workflow, for one team, with a human checking the output, and measure whether it saved time. If it did, widen it. If it did not, the honest move is to remove it rather than to add a feature that might rescue it. We wrote about what this looks like inside our own team in AI coding agents in a small team, and the build side is our AI and machine learning service and automation service.