What your model-risk team will ask about your first AI agent
For fifteen years, model risk management in U.S. banking had a script. SR 11-7 told institutions what a model was, what validation meant, and what documentation looked like. That script is gone: the interagency guidance that replaced it this spring is deliberately principles-based, and generative AI sits outside its formal scope entirely while the agencies gather comment.
Some institutions have read that as permission to move fast. We’d suggest reading it the other way: when the rulebook is thin, the burden of explanation moves to you. An examiner who can no longer check your work against a prescriptive standard will instead ask whether your own reasoning holds together. So will your board.
The good news is that the questions a competent model-risk team will ask about an AI agent are knowable in advance. Here are the seven we would ask — and that we help institutions prepare to answer before the first one is deployed.
1. Is it a model?
Don’t litigate this one. If the system produces outputs — drafts, classifications, recommendations, summaries — that influence business decisions, treat it as within scope of your model risk framework, whatever the current guidance technically requires. The principles-based era scopes by risk, not by category. An agent that drafts credit memos is shaping credit decisions; arguing that it’s “just a productivity tool” will cost you credibility precisely when you need it.
The mature posture: put it in the model inventory voluntarily, risk-tier it honestly, and calibrate the rigor of everything below to that tier.
2. What exactly does it decide?
The single most important design document for an AI agent is a plain statement of its decision rights: what it may do on its own, what it may only propose, and what it must never touch. Where does a human review its output, and is that review real — or a click-through?
Map the blast radius while you’re at it. If the agent is wrong once, what happens? If it’s wrong the same way four hundred times before anyone notices, what happens? The answers determine everything from monitoring frequency to whether the use case should exist.
3. How do you know it works?
“We tried it and the outputs looked good” is a demo, not evidence. Review-ready institutions can show:
- A golden set — a curated collection of real (or realistic) cases with known-good outputs, run against the system on every meaningful change.
- Adversarial cases — the inputs most likely to produce failure: incomplete files, contradictory data, edge-of-policy scenarios, and inputs designed to pull the agent off task.
- Acceptance thresholds decided in advance — what score, on what measure, was required before go-live, and who signed off on that bar.
If the evaluation set doesn’t exist, that’s the first engagement, before any build proceeds.
4. What happens when it’s wrong?
Every model is wrong sometimes; the discipline is in the failure design. A reviewable answer names the error taxonomy (what kinds of wrong are possible — fabricated facts, omitted facts, wrong tone, wrong policy applied), the detection mechanism for each, and the escalation path — who is alerted, what gets paused, and how affected outputs are found after the fact.
The last part is where audit trails earn their keep: if you discover on Thursday that the agent has been misreading a covenant clause since Monday, can you produce every memo it touched?
5. Where does its knowledge come from?
An agent’s answer is only as governed as the data it saw. Expect questions about provenance (which sources feed it, and who owns them), permissions (does it see anything the requesting employee couldn’t see?), and currency (when policy changed last month, did its sources change too?). Systems that retrieve from designated, versioned sources have clean answers here; systems that were trained or tuned on internal data have much harder ones — a point worth weighing at design time, not after.
6. How will you know when it drifts?
Two kinds of change matter. Your inputs drift — new products, new customer segments, new fraud patterns — and the agent’s performance on yesterday’s golden set says nothing about next quarter’s cases. And the underlying model itself changes: if a third party hosts it, updates can arrive on their schedule, not yours. Version pinning, re-evaluation triggers, and a monitoring cadence proportionate to the risk tier are the answer; “we’ll keep an eye on it” is not.
This is also a third-party risk conversation. Your vendor-management framework almost certainly hasn’t contemplated a supplier who changes the product’s behavior without a release note. It should.
7. Who owns it?
Not the project team — they disband. A named first-line owner accountable for the agent’s behavior in production; a documentation pack that a new risk analyst could pick up cold; a standing review cycle. Ownership is the difference between a governed system and an orphaned one, and orphaned systems are what examination findings are made of.
What “ready” looks like
None of this requires a hundred-page validation report on day one. It requires a coherent file: purpose and decision rights, risk tier and rationale, evaluation evidence with thresholds, failure and escalation design, data provenance, monitoring and change triggers, and a named owner. A dozen well-considered pages beat a binder of boilerplate — especially in front of an examiner who is, for the first time in fifteen years, grading the quality of your judgment rather than your compliance with a checklist.
The deeper lesson is about sequence. Institutions that treat model risk as the final gate lose months in a review cycle that was never going to pass. Institutions that bring the second line in at design — that build the evaluation set before the agent — find that governance stops being the obstacle and starts being the reason the project ships.
Eastern Point advises financial services leadership on AI governance and adoption. If your first agent is closer than your answers to these questions, start a conversation.