The Second Model Is Not a Check

A second model grading the first is not independence. It is a second pass by a sibling of the thing under test — and a sibling that shares lineage, training data, prompt scaffolding, and blind spots will be wrong in the same places, at the same time, on the same inputs. Two reviewers who fail together are one reviewer. Independence is not "a different forward pass." It is structural: separate model lineage, separate context, a separate owning function that did not build the thing it grades, and — the part everyone skips — demonstrably uncorrelated errors.

Bank supervisors named this discipline and gave it a name: effective challenge. Under SR 11-7, and now under its 2026 successor SR 26-2, a consequential model must be reviewed by a validation function organizationally separate from the people who built it — credible, independent review with the authority to change or reject the model. Not a courtesy read. Not a sign-off from the next desk over. A reviewer with the standing to say no, who reports through a different chain, and whose job is to find the flaw the builder could not see.

In 2012, JPMorgan's Chief Investment Office changed the Value-at-Risk model governing its Synthetic Credit Portfolio. The new model understated the position's risk. The change was reviewed — but not by a genuinely independent, empowered challenger; the review was nominal, captured, close to the desk it was meant to police. The portfolio ran on a number that flattered it. The loss reached roughly US$6.2 billion, and a US Senate subcommittee faulted the model-risk governance and the absence of real independent validation.

The lesson is exact. The model's own judgment of its risk — or a sibling reviewer's, which is the same judgment in a different chair — was the failure. Independence was on the org chart. It was not in the errors.

An autonomous agent committing a hard-to-reverse action needs the same control banking already requires: an independent, qualified, pre-commit verifier — not a second forward pass by a model that fails the way the actor fails. "Independent" has to be earned in three dimensions. Separate lineage, so the verifier does not inherit the actor's blind spots. Measured qualification, so the challenger is itself competent on the task class before it is allowed to gate. And measured low correlation of errors — the actor and verifier wrong together no more than a small, bounded fraction of the time — because a challenger whose mistakes track the actor's adds no independence regardless of where it sits on the org chart.

That last one is the whole discipline, and it is the one almost no AI system measures.

The in-depth companion develops the full argument — the SR 11-7 / SR 26-2 effective-challenge spine, the London Whale as the failure of nominal independence, the three structural conditions for a real challenger, and the precise units that make independence measurable for an agent: separate lineage, verifier qualification as a DO-330 analogue, and an error-correlation ceiling of ρ ≤ 0.2 on a frozen adversarial set. Read it at Effective Challenge: Independent Validation for Autonomous Agents.