Your agent pilot isn't stuck on the agent — it's stuck on the data
There is a pattern to stalled AI pilots in financial institutions, and it rarely involves the AI.
The demo went well. The model was capable. The team was competent. And yet the pilot has been “three weeks from production” for two quarters — because the agent that dazzled on a curated sample turned out to need customer data from four systems, permissions nobody could articulate, and documents whose freshness no one could vouch for.
The uncomfortable truth: an agent is a new employee who reads everything you hand it and believes all of it. Handing it your data estate as-is means handing it every duplicate record, stale policy, and permissions shortcut you’ve been living with. The institutions that get agents into production aren’t the ones with the best models. They’re the ones that answered six architecture questions before the build started.
1. Where does the truth live?
For every entity the workflow touches — the customer, the loan, the account, the guarantee — name the system of record. Not “mostly in the servicing platform”; name it, and name the exceptions. If the same customer exists three times across your estate with three addresses, a human employee knows which one to trust through folklore. An agent doesn’t get folklore. Either the golden record exists, or the agent’s confident answer is a coin flip you’ve automated.
You do not need to resolve this for the whole institution. You need to resolve it for the entities this workflow touches. That’s a tractable project; “fix the data first” as an enterprise program is where momentum goes to die.
2. Can the agent see what a good employee sees — and nothing more?
Take the workflow’s best human performer and write down what they actually consult: core system screens, the document repository, the policy manual, that spreadsheet everyone pretends not to depend on. That list is the agent’s required field of view, and it’s usually longer than the pilot assumed.
Then invert it. The agent authenticates as something — and that something must not see more than the role it serves. Entitlements designed for humans (who can be trained, warned, and fired) transfer poorly to software that will cheerfully use everything it can reach. Least-privilege access for the agent is both a security control and, increasingly, what reviewers expect to find documented.
3. Is the case file assembled, or scattered?
Watch an underwriter work and you’ll see the real job: assembling a case file from a half-dozen systems before judgment ever begins. Most agent pilots quietly assume this assembly problem away — the demo used a tidy folder someone prepared by hand.
Production requires that assembly to be an engineered capability: given a loan application, gather the financials, the collateral documents, the relationship history, and the applicable policy sections, reliably and on demand. This integration layer — not the model — is usually the longest lead-time item in the entire program, which is why it should be scoped first, not discovered late.
4. Is the data allowed to be used this way?
Data acquired for one purpose doesn’t automatically carry the right to be used for another. Before any build: what personal information enters the agent’s context, under what purpose limitation? Does anything cross a border it shouldn’t? If the model is externally hosted, what leaves your perimeter, and does that square with what you’ve told customers and regulators?
These questions have real answers, and privacy and compliance teams can produce them quickly — if they’re asked at design time. Asked at go-live, the same questions produce a stopped project and a bruised relationship with the second line.
5. How fresh is it, and can you trace it?
An agent reading last quarter’s rate sheet doesn’t fail loudly. It answers fluently, confidently, and wrong. Every source the agent consults needs a known refresh cadence and an owner accountable for it — and when the agent produces an answer, you should be able to trace which sources, in which versions, it drew from. Freshness protects the decision; lineage protects you when someone asks, months later, why the decision was made.
6. What does the exhaust look like?
The most overlooked design question: the agent’s own activity is a new data stream, and it should be treated as a first-class one. Every retrieval, every draft, every human correction — logged, structured, and retained. That exhaust is your audit trail when a reviewer asks what happened. It is also, quietly, your most valuable asset: the record of human corrections is exactly the evidence you’ll need to measure the agent honestly and improve it deliberately.
The sequencing advice
None of this argues for a grand data-transformation program before your first agent. The opposite: scope data readiness to one workflow’s case file, and do it completely. One workflow’s entities resolved, one field of view permissioned, one assembly pipeline built, one set of sources dated and owned. The second workflow reuses most of it. The third goes faster still. Data readiness compounds — but only if it’s done as engineering, not as aspiration.
The pilots that die are the ones where these six questions were answered implicitly, optimistically, and late. The programs that ship answer them explicitly, skeptically, and first.
Eastern Point advises financial services leadership on strategy, transformation, and AI governance. If your pilot has been “almost ready” for a quarter, start a conversation.