Fine-tune, RAG, or neither: a decision framework for lending workflows
Somewhere in your institution, a team is proposing to fine-tune a language model. Somewhere else, a vendor is insisting that retrieval-augmented generation is the answer. Both conversations tend to share a flaw: the technique has been chosen before the problem has been diagnosed.
The choice matters — the three paths differ by an order of magnitude in cost, in maintenance burden, and in what they do to your governance obligations. Here is the framework we use with lending executives to make the call deliberately, in plain language, before the spend.
The three paths, stated plainly
Neither. Use a capable model as-is, with carefully engineered instructions, well-designed workflow, and the right context handed to it at the moment of the task. Unfashionable, unglamorous — and the correct starting point for almost every use case.
Retrieval (the industry says “RAG”). At the moment of each question, fetch the relevant documents — the policy section, the guideline, the customer file — and put them in front of the model, so its answer is grounded in your current sources and can cite them.
Fine-tuning. Train lasting habits into the model itself with many examples: a format, a tone, a classification scheme. Think of it as teaching reflexes — not teaching facts.
The misconception that drives most wasted spend: treating fine-tuning as the way to make a model “know our business.” Tuning is poor at storing knowledge and worse at updating it; what it changes is behavior. Knowledge belongs in retrieval, where it stays current and citable.
Five questions that make the decision
1. Is the gap knowledge, behavior — or neither? When output quality disappoints, diagnose before prescribing. If the model didn’t know something (your policy, this borrower’s file), that’s a knowledge gap — retrieval territory. If it knew but answered in the wrong shape (wrong format, wrong tone, inconsistent structure), that’s behavior — instructions first, tuning if instructions plateau. And in our experience the most common case is neither: the model was never given the context a competent employee would have had. That’s a workflow defect. No technique fixes a context you didn’t provide.
2. How often does the underlying truth change? Rates change daily. Credit policy changes quarterly. Underwriting guidelines change with every committee cycle. Anything that changes must live in retrieval, where an update to the source updates every answer from that moment on. Bake changing truth into a tuned model and you’ve created a system that is confidently out of date, with no way to correct it short of retraining — a control failure waiting for its moment.
3. Must the answer show its sources? In lending, usually yes. An underwriter’s assistant that cites the specific policy section it relied on is reviewable — by the underwriter in the moment and by risk teams after the fact. A tuned model’s knowledge can’t cite anything; it simply asserts. If your model-risk or compliance function will ask “why did it say that?” — and they will — retrieval isn’t just the better architecture, it’s the governable one.
4. Do the economics actually favor tuning? Fine-tuning earns its keep in one honest scenario: a narrow, stable, high-volume task — classifying thousands of servicing documents a day, extracting the same fields from the same forms — where a small tuned model can replace a large general one at a fraction of the per-task cost. That’s real money at real volume. But note what this is: a cost optimization applied to a proven workflow. It is not the way to make a pilot work. If the use case hasn’t proven itself with the simpler paths, tuning adds expense and rigidity to something that doesn’t yet deserve either.
5. Can you evaluate it? Whichever path you choose, one artifact decides whether the investment is real: a test set of genuine cases with known-good answers, and a threshold the system must clear. No evaluation set, no spend — that rule alone would have saved most of the disappointed AI budgets we’ve seen. The evaluation set is also what makes the comparison honest: run the same cases through “neither,” through retrieval, through a tuned variant, and let the results — not the vendor’s roadmap — pick the architecture.
What this looks like in a lending shop
- Credit memo drafting: retrieval for every fact — financials, covenants, relationship history, applicable policy — so each statement is grounded and citable. If the house style is rigid, a light behavioral tune later, once the workflow has proven out.
- Policy questions from underwriters: pure retrieval against the current guideline, with the answer quoting the section it relied on. Nothing about this should be tuned; policy is precisely the truth that changes.
- Servicing correspondence triage: the genuine fine-tuning case — high volume, stable categories, measurable accuracy, meaningful unit economics.
- Adverse-action language: neither. Some outputs are too consequential to generate freely; controlled templates with system-assisted selection beat generative flexibility, and your compliance team will agree.
The sequence we recommend
Start with neither and take it seriously — instructions, context, workflow. Add retrieval when answers must be current and citable, which in lending is early and often. Consider tuning last, as an optimization on a workflow that has already earned its place with evaluation evidence. And whichever door you walk through, governance walks with you: the evaluation set, the source lineage, the documented rationale for the choice itself.
The institutions that get this right aren’t the ones with the most sophisticated architecture. They’re the ones whose architecture can be explained — to the board, to the examiner, and to the next leader who inherits it.
Eastern Point advises financial services leadership on AI strategy and governance. If the architecture debate has started before the diagnosis, start a conversation.