You Cannot Outsource the Obligation to Govern

KellerAI

The problem on your desk

You licensed an AI agent. The vendor ran its benchmarks, published a model card (its own report on what the model is and how it behaves), scored a safety test, and signed the contract, so the hard part of governance looks done. It is not. When that agent moves money, sends an email, or amends a record, the consequence lands on you, the deployer, not on the lab that trained it. The vendor's test is evidence about the vendor's model under the vendor's conditions. It is not your governance, because governance is not a property of the model: it is a property of the institution that lets the model act in its name. You own the leftover risk of every model you run, including the ones you bought and cannot inspect.

What the solution is

A governance posture that treats the bought agent as exactly what it is: a vendor model that takes actions, and therefore your risk to own. Three disciplines follow, and the vendor supplies none of them: - Vendor attribution: when the agent uses a vendor model or outside tool, its mistakes count against your own allowed error limit, formally the escape-rate budget (your limit for wrong actions that survive every check and are actually carried out), not the vendor's reputation. There is no separate, more forgiving ledger for bought models. - A reconstructable trace: every gated action emits an add-only, tamper-evident record (a log that cannot be quietly rewritten) of what it did, at what risk level, and who or what checked it, so you can defend it to a regulator or to internal audit from your own records. - Consequence-scaled scrutiny: a separate approval check, running outside the agent so the agent cannot switch it off, reads what an action actually touches and sizes the review to its potential damage. The agent proposes; it never grades its own consequence, so it cannot label a large external transfer routine to dodge review.

Why it works

This is not a new AI rulebook. It extends the discipline banks already run. The interagency US model-risk standard SR 26-2 (which in April 2026 replaced the long-standing SR 11-7) already puts vendor and outside models squarely inside the deployer's responsibility: you validate, monitor, and own the risk of models you buy, because the regulator holds you, not the supplier, accountable. SR 26-2 places generative and agentic AI outside its formal model-risk scope as novel and rapidly evolving, deferring them to broader risk management, so the duty to govern an agent's actions falls to the deploying institution to build, which is the gap the LLM-Agent Assurance Standard (LAAS) was written to fill. The cautionary tale is concrete: in 2007-08, banks and investors treated AAA ratings from Moody's and S&P as a substitute for their own analysis. The ratings were catastrophically wrong, and the losses landed on the institutions that had leaned on them, not on the agencies that issued them. An outsourced assessment never transferred the obligation. Across airlines (the earned, revocable authority of ETOPS), banks (SR 26-2), and autonomous-product safety (the UL 4600 standard), the same rule keeps surfacing: a supplier's certificate helps, but the operator still owns what happens in operation.

The bottom line

The discipline is not a compliance tax on your agent: it is the asset that lets you operate it widely. The same trace that defends an action to a regulator is the one that proves the agent is reliable enough to earn a wider, revocable mandate (more autonomy, granted on evidence and taken back if the record slips). The natural champion is the model-risk or risk-governance owner responsible for records that hold up to examiners and internal audit, the very records this posture produces. It is grounded in a named standard, LAAS, and deployable as a posture today: own the trace, attribute the vendor to your own budget, and size scrutiny to the damage an action can do, before you let a bought model commit anything.

Section 01

The Eval You Bought Is Not the Governance You Owe

The pitch is seductive: the foundation-model vendor ran a benchmark suite, published a model card, scored a safety eval, and signed an enterprise agreement — so the hard part of governance is done, and what remains is integration. That description is true, and it misses the point entirely. The vendor's eval is the vendor's evidence about the vendor's artifact under the vendor's conditions. It is not your governance, because governance is not a property of the model. It is a property of the deploying institution that lets the model commit actions in its name.

Buying the model does not buy the accountability. When a bought agent moves money, sends an email, files a ticket, or amends a record, the consequence lands on the deployer — not on the lab that trained the weights. The deployer owns the residual risk of every model it runs, including the ones it did not build, and especially the ones it cannot inspect.

Section 02

Banking Already Litigated This

Model-risk management settled the question fifteen years ago and re-settled it in 2026. The interagency US standard SR 26-2 (federalreserve.gov/supervisionreg/srletters/sr2602.htm) — which superseded SR 11-7 in April 2026 — places vendor and third-party models explicitly inside its scope. Generative and agentic AI sit outside SR 26-2's formal scope for now, in an interim posture while agencies prepare further guidance. A model bought from outside is still the deploying institution's model risk to own. The validation lifecycle, the ongoing monitoring, the outcomes analysis: the bank performs them on the vendor's model, because the regulator holds the bank, not the vendor, accountable for what the model is allowed to do.

The doctrine has a name and a shape. Rigor is risk-tiered by materiality — capital is proportional to consequence — and a full audit trail sufficient to reconstruct the decision is the supervisory expectation. You do not get to point at a supplier's certificate when the model is wrong. You get to explain, from your own records, what it did and why you let it.

Section 03

The Ratings That Were AAA Until They Weren't

The cleanest enforcement anchor is not an enforcement action at all — it is the 2007–08 reliance on external credit-rating-agency models. Banks and investors treated AAA ratings from Moody's and S&P on structured products as a substitute for their own model-risk assessment. The ratings were the outsourced judgment. They were catastrophically wrong, and the losses landed on the deployers who had leaned on them, not on the agencies that issued them.

An external party's assessment of a model never transferred the obligation. The deployer that relied on someone else's eval still owned the failure when the eval was wrong.

The load-bearing lesson

That is the whole of it. SR 11-7's explicit vendor-model clause was, in part, written to foreclose exactly this move — the move of treating a supplier's score as a discharge of your own duty to govern.

Section 04

What This Means for a Bought Agent

The agent you license is, governance-wise, a vendor model that takes actions. Three disciplines follow directly. First, vendor attribution: when the agent uses a vendor model or a third-party tool, its errors count against your escape-rate budget, not the vendor's reputation. Second, a reconstructable trace: every gated action emits an append-only, tamper-evident record sufficient to reconstruct what happened, at what tier, checked by whom. Third, consequence-scaled rigor: the gate derives the tier from the action's blast radius and prices scrutiny to it — the agent proposes, it never grades its own consequence.

None of this is the vendor's to provide. The vendor cannot trace your decisions, cannot attribute its errors to your budget, and cannot set your tiers. Accountability is non-delegable downward, to a supplier, for the same reason it is non-delegable inward, to the builder. You cannot outsource the obligation to govern.

The in-depth companion develops the full argument — vendor attribution to the deployer's escape budget, the append-only hash-chained trace as the conformance artifact, and consequence-scaled rigor where the gate derives the tier and the actor only proposes.

Read the in-depth companion →

End of paper↑ Back to top