Why We Reach for Retrieval-Augmented Chatbots Before Fine-Tuning

Almost every week a client tells us they want “a chatbot that knows our company.” It is a reasonable ask, and the instinct in the room is usually the same: let’s fine-tune a model on our documents. We understand the appeal, but in most cases we steer the conversation somewhere else first — toward retrieval-augmented generation, or RAG.

What we mean by retrieval-augmented generation

Instead of baking your knowledge into the model’s weights, we keep your knowledge in a searchable index. When a user asks a question, we retrieve the most relevant passages from your own content and hand them to the model as context. The model answers using what we just gave it, not what it vaguely remembers from training.

The practical payoff is that your assistant stays current. When a policy changes, we update one document in the index and the next answer reflects it. There is no retraining cycle, no waiting, and no risk that the model confidently quotes a version of the truth from three months ago.

When we do reach for fine-tuning

We are not against fine-tuning — we use it when the goal is to change how the model behaves rather than what it knows. Teaching a model your house tone of voice, a strict output format, or a narrow classification task: those are good reasons. Stuffing facts into weights so a chatbot can recite your help centre is usually not.

• Knowledge that changes often → retrieval, every time.
• Behaviour, tone or rigid formatting → fine-tuning can earn its keep.
• Both → we layer a light fine-tune on top of a solid retrieval system.

“The best AI feature is the one your team can keep correct by editing a document, not by booking an ML engineer.”

So when we start an AI assistant project, we begin with retrieval, prove it answers honestly on your real questions, and only add complexity when the use case clearly demands it. It is the shortest path we know to something you can trust in front of customers.