Effective Challenge

KellerAI

The problem on your desk

When an autonomous AI agent (an AI system allowed to take actions in software without a person approving each step) is about to do something costly and hard to undo, the instinct is to have a second AI model check it first. But a second model grading the first is not independence. A reviewer from the same model line shares the actor's training data, its biases, and its blind spots, so it tends to fail on the same hard cases, for the same reasons, at the same moment. Two reviewers who fail together are one reviewer. This is exactly why the agent that demos beautifully still cannot be trusted to commit a hard-to-reverse or materially consequential action without independent validation: for the Head of Model Risk or Independent Validation, the open question is whether an agent can ever be allowed to take such an action on its own.

What the solution is

Effective challenge: the review discipline US bank model-risk guidance has run for fifteen years, now applied to AI agent actions. That guidance requires any consequential model to be reviewed by an independent validation function with the authority to change or reject it, not a courtesy read from the next desk over. This paper turns that requirement into three things an agent's checker must prove before it is allowed to approve a hard-to-reverse action: - Different roots: the checker is not the same model line, the same instructions and context, or the same owner as the agent it grades, so it does not inherit the same blind spots. - Proven competence: the checker is itself tested and shown qualified on the task before it is trusted to gate it, the same way aviation tool-qualification rules require any tool you rely on to be proven first. - Mistakes that do not move together: you measure how often the checker is wrong at the same moment the agent is wrong, on a frozen, version-locked set of at least 1,000 hard cases (re-measured whenever the model, prompts, tools, or data shift), and admit the checker only if that measured overlap stays safely below a strict ceiling (the paper sets it as a correlation at or under 0.2, judged on the cautious end of the measurement, not the best-case estimate). Routine actions stay light. Hard-to-reverse or materially consequential actions trigger the full independent check; irreversible or high-consequence ones also require a human to sign off.

Why it works

This is not a new and unproven AI control. It is the discipline regulated industries already trust, extended to a new kind of actor. US bank model-risk guidance calls it effective challenge (the rules SR 11-7 and its 2026 successor SR 26-2): a consequential model must be validated by someone other than the people who built it. Aviation certification reaches the same place, requiring certain high-consequence checks to be done by someone other than the builder and any tool relied on to be proven qualified before its verdict counts. The cost of skipping it is documented: in 2012 JPMorgan's London Whale loss reached roughly 6.2 billion dollars after a risk model was reviewed by a challenger too close to the desk it was meant to police. Independence existed on the org chart and not in the errors. An AI setup where a model of the same line signs off on its own kind makes that identical mistake, at machine speed, on every action.

The bottom line

Reading this as a tax on the agent's speed is backwards. A checker you can prove is independent is precisely what lets you grant an agent a wider range of action without a human approving every step, because you can finally show where the machine check does the job a person would have done. A natural champion is the Head of Model Risk or Independent Validation, the function that already owns effective challenge for models; what they gain is a measurable standard and an audit trail they can show the board and the regulator. The deployer inherits this obligation whether or not the rules name AI agents, and the paper hands them the way to meet it: name the specific checker, version it, prove it is independent, qualified, and below the 0.2 correlation ceiling, then decide which hard-to-reverse actions can proceed without blanket human approval, while irreversible or high-consequence ones keep the human gate. Ready now.

Section 01

A Second Pass Is Not Independence.

A second model grading the first is not independence. It is a second pass by a sibling of the thing under test — and a sibling that shares lineage, training data, prompt scaffolding, and blind spots will be wrong in the same places, at the same time, on the same inputs. Two reviewers who fail together are one reviewer. Independence is not "a different forward pass." It is structural: separate model lineage, separate context, a separate owning function that did not build the thing it grades, and — the part everyone skips — demonstrably uncorrelated errors.

Section 02

What Banking Already Settled.

Bank supervisors named this discipline and gave it a name: effective challenge. Under SR 11-7, and now under its April 2026 successor SR 26-2 (federalreserve.gov/supervisionreg/srletters/SR2602.htm), a consequential model must be reviewed by a validation function organizationally separate from the people who built it: credible, independent review with the authority to change or reject the model. Not a courtesy read. Not a sign-off from the next desk over. A reviewer with the standing to say no, who reports through a different chain, and whose job is to find the flaw the builder could not see.

The phrase that carries the weight is "the authority to change." A challenger who cannot stop the action is not a control; it is a comment.

Effective challenge

Section 03

The Failure When Independence Is Nominal.

In 2012, JPMorgan's Chief Investment Office changed the Value-at-Risk model governing its Synthetic Credit Portfolio. The new model understated the position's risk. The change was reviewed — but not by a genuinely independent, empowered challenger; the review was nominal, captured, close to the desk it was meant to police. The portfolio ran on a number that flattered it. The loss reached roughly US$6.2 billion, and a US Senate subcommittee faulted the model-risk governance and the absence of real independent validation.

The lesson is exact. The model's own judgment of its risk — or a sibling reviewer's, which is the same judgment in a different chair — was the failure. Independence was on the org chart. It was not in the errors.

Section 04

The Mapping to Agents.

An autonomous agent committing a hard-to-reverse action needs the same control banking already requires: an independent, qualified, pre-commit verifier — not a second forward pass by a model that fails the way the actor fails. "Independent" has to be earned in three dimensions. Separate lineage, so the verifier does not inherit the actor's blind spots. Measured qualification, so the challenger is itself competent on the task class before it is allowed to gate. And measured low correlation of errors — the actor and verifier wrong together no more than a small, bounded fraction of the time — because a challenger whose mistakes track the actor's adds no independence regardless of where it sits on the org chart.

That last one is the whole discipline, and it is the one almost no AI system measures.

The in-depth companion develops the full argument — the SR 11-7 / SR 26-2 effective-challenge spine, the London Whale as the failure of nominal independence, the three structural conditions for a real challenger, and the precise units that make independence measurable for an agent: separate lineage, verifier qualification as a DO-330 analogue, and an error-correlation ceiling of ρ ≤ 0.2 on a frozen adversarial set (a KellerAI proposed default from our LLM-Agent Assurance Standard, derived from a binomial confidence bound). Read it at Effective Challenge: Independent Validation for Autonomous Agents.

End of paper↑ Back to top

The Second Model Is Not a Check

Context

The Finding