Building Reliable AI Features with RAG, Tools, and Human Review
A practical framework for deciding when to use RAG or agents and how to add evaluation, permission boundaries, and human review.
The difficult part of an AI feature is rarely the first model call. The difficult part is turning an uncertain model response into a product behavior that users can understand, operators can diagnose, and a business can afford.
That starts with a precise job. “Add AI” is not a job. “Help a support agent find the relevant policy and draft a cited answer that the agent approves” is a job. It identifies the user, the source material, the output, and the person responsible for the final decision.
Choose the least complex useful pattern
Three patterns cover many product needs:
- Structured generation transforms supplied context into a defined output, such as classifying a request or drafting a summary.
- Retrieval-augmented generation (RAG) finds relevant private or changing knowledge before producing an answer.
- Tool use or an agent loop lets the model choose and sequence permitted operations.
Do not start with an agent if one validated model call can do the work. Agent loops increase latency, cost, possible actions, and the number of ways a task can fail.
Build retrieval as a measurable pipeline
RAG does not automatically make answers correct. Retrieval can return irrelevant, outdated, duplicated, or unauthorized material. The system needs a deliberate ingestion and query path.
During ingestion, preserve source identity, ownership, update time, and access rules. Chunking should match how readers use the material. A legal clause, product procedure, and API reference may require different boundaries.
At query time, filter by tenant and permission before the model sees the content. Retrieve a small candidate set, rerank when necessary, and keep source references attached through generation so the interface can show where an answer came from.
Evaluate retrieval separately from generation. If the correct document never reaches the prompt, changing the prompt will not solve the problem.
Treat tools as privileged interfaces
A model should not receive a general-purpose database or shell connection. Give it narrow tools with typed inputs and explicit authorization. A draftRefundRequest tool is easier to reason about than a runSql tool.
For every tool, define:
- who may request it;
- what resources it can affect;
- which inputs are allowed;
- whether it only reads or also changes state;
- what requires human confirmation;
- and what is recorded for audit.
The application—not the model—enforces these rules. The model can propose an action; trusted code decides whether that action is valid.
Put human review where failure matters
Human review should not be a vague promise. It should be a product state. An AI-created price, outbound message, account change, or operational recommendation can be saved as a draft with the model, source context, and rationale visible to the reviewer.
The reviewer needs to edit, approve, reject, or request regeneration. Those decisions become evaluation data. Silent auto-application removes accountability and makes errors harder to learn from.
Low-risk, reversible actions may earn more automation after the system demonstrates quality. The permission boundary can evolve with evidence.
Create an evaluation set before optimizing prompts
Collect representative tasks, difficult edge cases, and known failure examples. Define what a good output means for each task. Depending on the feature, that can include citation correctness, required fields, policy compliance, factual consistency, latency, and cost.
Run the set when changing models, prompts, retrieval, or tools. Model upgrades are dependency changes; they should not bypass regression testing.
Production feedback should be sampled and reviewed, not copied indiscriminately into training or evaluation data. Personal and confidential information needs an explicit retention policy.
Design for failure and cost
Models time out, providers throttle requests, schemas fail validation, and retrieved context becomes stale. The interface should distinguish retryable system failure from uncertainty in the answer.
Set time and token limits. Cache safe repeated work. Use a smaller model when evaluation shows it is sufficient. Keep provider-specific code behind an adapter so the product is not tightly coupled to one response shape.
When the AI path is unavailable, decide whether the user should retry, continue manually, or receive a deterministic fallback. That decision belongs in the product design, not in an exception handler added after launch.
The useful definition of an AI feature
A production AI feature is a complete system: a scoped job, trusted context, constrained capabilities, measurable quality, visible uncertainty, an operating model, and a person or policy responsible for consequential actions.
The model is important. The surrounding engineering is what makes it useful.