Fail-Closed AI: What a Decision System Should Do When Evidence Is Missing
Updated: 1 day ago
The most dangerous AI decision is not the one based on bad data. It is the one based on data that was never there.
It happens quietly, every day, in financial institutions. A field arrives empty and the system fills it with a default. A bureau query times out and the pipeline proceeds without it. A financial statement is two years old, but nothing checks its date. The model receives something that looks like a complete input, produces a confident score, and a decision is made.
No one decided to approve that customer without the evidence. The decision was made by omission.
This article is about one design principle that prevents it, and about why it matters more as decisions become faster and more automated: AI decision systems must fail closed.
Fail-open and fail-closed
The terms come from engineering. When a system fails, it can fail in one of two directions.
A fail-open system keeps working when something breaks. A door that unlocks when the power goes out. A firewall that lets traffic through when it can't inspect it.
A fail-closed system stops when something breaks. A door that stays locked. A firewall that blocks traffic it can't inspect.
Neither is universally right. A fire exit should fail open. A vault should fail closed.
Consequential financial decisions belong with the vault. When the evidence required to make a decision is missing, the system should not proceed as if it were there.
How decisions fail open today
Most decision pipelines fail open by default, and not because anyone chose it. It is a side effect of how they are built.
Imputation hides absence. Models need complete inputs, so missing values are filled with averages, medians or zeros. That is a reasonable technique for training a model. It becomes dangerous when the imputed value silently replaces evidence that a policy requires.
Timeouts become defaults. When an external source doesn't respond in time, the pipeline continues with whatever it has, so the customer isn't kept waiting. The decision is made on less information than the policy assumes, and nothing records that.
Staleness goes unchecked. Data has a date. A financial statement, an income verification or a collateral appraisal can be accurate and still too old to support a decision. Pipelines rarely check whether evidence is current, only whether it exists.
Scores look equally confident. A model scoring a customer with complete, current data and one scoring a customer with half the inputs imputed can produce the same number. Nothing downstream can tell the difference.
In each case, the decision proceeds. The absence of evidence is invisible to everyone, including the people accountable for the outcome.
The principle
The principle is simple to state: before any decision is made, the system checks that the evidence the policy requires is present, current and sufficient. If it isn't, the decision does not proceed as if it were. It is conditioned, escalated or refused, and the reason is recorded.
Three elements make this work.
Evidence is declared. For each decision, the policy defines what evidence it requires: which data, from which sources, how recent. Requirements are explicit, not implied by what the model happens to use.
Sufficiency is checked before deciding. The check happens before the policy is applied, not after. A decision should never be issued and then flagged.
Absence has a defined outcome. What happens when evidence is missing is itself part of the policy, not left to the pipeline's defaults.
Fail-closed doesn't mean "always say no"
A common objection is that failing closed will block legitimate customers and slow the business down. It won't, if it is designed properly. Failing closed doesn't mean refusing. It means refusing to pretend. The policy can choose among several outcomes, depending on what is missing and why:
Situation | Typical outcome |
Evidence is missing but can be obtained quickly | Condition: request it and proceed once it arrives |
Evidence is stale but recent data suggests no change | Condition: proceed partially, request an update for the rest |
Evidence is missing and the case needs judgment | Escalate to an analyst with the gap highlighted |
Evidence that is legally or materially essential is missing | Deny or block, with the reason recorded |
A credit limit increase without updated financial statements can be partially approved, with the rest released once the statements arrive. A payment without a matching invoice can wait for the invoice. Only when the missing evidence is essential does the system refuse.
The customer experience is often better, not worse: a clear request for a specific document instead of an unexplained rejection months later, when the risk materializes.
Failing closed at execution
The principle applies after the decision, too.
Once a decision is made, it must be executed: a payment sent, a limit updated, a contract issued. Execution can fail in ways that leave the state uncertain. Did the payment go through or not? Was the limit updated?
When the state is uncertain, the system should not guess. Retrying blindly can execute an action twice. Assuming failure can leave a customer without what they were approved for. The safe pattern has three parts:
Idempotent execution: every action carries a unique decision ID, so a retry can never produce a duplicate effect.
Fail-closed on uncertainty: when the outcome of an execution is unknown, the system stops and resolves the state before doing anything else.
Reversal defined in advance: the way to undo or compensate an action is defined before the action exists in production, not improvised after something goes wrong.
Measure the gaps
Failing closed produces a valuable by-product: visibility into where evidence is missing.
The evidence insufficiency rate, the share of decisions that were conditioned, escalated or refused because evidence was missing, is one of the most useful operational metrics an institution can track. Broken down by source, product and channel, it shows exactly where data pipelines are failing, which integrations are unreliable and which documents customers struggle to provide.
In a fail-open system, all of those gaps exist too. They just aren't visible until they show up as losses.
A design checklist
Does every decision type declare the evidence it requires, including how recent it must be?
Is evidence sufficiency checked before the decision, not after?
Is the outcome of missing evidence defined by policy: condition, escalate or refuse?
Can the model's inputs be distinguished from imputed values in the decision record?
Do timeouts and source failures produce an explicit outcome rather than a silent default?
Is execution idempotent, with uncertain states resolved before retrying?
Is the evidence insufficiency rate measured and reviewed?
No decision by omission
As decisions become faster and more automated, failing open becomes more dangerous, because there is no longer a human in the loop to notice that something is missing.
The fix isn't slower decisions. It is decisions that know what they require, check it every time, and stop, visibly and for a stated reason, when it isn't there.
A missing piece of evidence should never become an invisible assumption.
Want to see how fail-closed decisioning would work in your institution's processes? Talk to our team →
Comments