Today, AI agents can generate analyses and decision-making frameworks in a very short amount of time. However, they cannot take responsibility.
On a single morning, a division head signs off on forty documents an agent has prepared. Supplier assessments, risk reviews, two draft contracts. Every one of them is cleanly structured, sourced and better written than much of what used to land on his desk. He read three of them in full. What exactly is he accountable for with his initials on the other thirty-seven?
The honest answer: for now, only for the fact that a sign-off happened. That is not what a sign-off was meant for. And it is not a problem AI created. AI runs into one that has been growing in many companies for years, and makes it bigger.
In “Trust Is The Scarce Asset”, Abraham Thomas describes how every technology that makes something cheap first sets off a flood, and only afterward produces the rules a society uses to contain it. For writing, it was seals and witnesses. For coins, the royal mints. For the printing press, copyright and peer review. With AI, we are in the middle of the flood. What interests me is what happens inside companies during this phase, because that is where it is being decided right now whether cheap intelligence produces accountability or its opposite.
Proof instead of accountability
In 1988, under the impression of Chernobyl, the German sociologist Ulrich Beck coined a term that fits better today than it did then: organized irresponsibility. He meant a system in which every participant acts correctly and, in the end, nobody answers for the result. Beck had nuclear plants and chemical accidents in mind. Today you find the pattern in every compliance department.
ESG reporting, supply chain laws, sustainability disclosures: the regulations of recent years have anchored a particular logic inside companies. Accountability is documented, not carried. What gets checked is whether the process is documented, whether the risk analysis exists, whether the approval chain is complete. No form asks whether someone stands behind the result. Anyone working in that system learns fast that an airtight paper trail protects you better than a good decision, because when something goes wrong, the paper trail is what people ask for.
Europe is now rolling some of this back itself. Since March 2026, the EU’s Omnibus directive limits sustainability reporting to companies with at least 1,000 employees and 450 million euros in revenue, and supply chain due diligence kicks in only at 5,000 employees and 1.5 billion euros. In Germany, the federal export control office, BAFA, has stopped accepting reports under the country’s supply chain act since October 2025. The obligations are shrinking. The departments, the approval layers and the habit of thinking in proof remain. They have long since become part of the organization.
Madeleine Clare Elish found a precise term for the role humans play in such systems: the moral crumple zone. At the end of an automated chain, a person absorbs the blame, even though they had little control. Every approval layer a company adds in response to a risk creates another one. The division head with his forty documents is one of them.
A system with nothing to lose
This organization now meets a tool that delivers exactly what it asks for: proof. A risk analysis for every supplier, a rationale for every approval, a complete log, all at close to zero cost. Effort used to cap how much paper got produced. That cap is gone.
The agent brings two properties that make the problem worse from its side. First, it has nothing to lose. An employee who does sloppy work risks their reputation and the next promotion, and because they know that, they usually work carefully. A language model has no next round. When it writes “I take responsibility,” that is a sentence like any other.
Second, it is not predictable. A language model computes with probabilities, and the same task does not reliably lead to the same result. I have written about that in detail elsewhere. Andreas Matthias drew the consequence as early as 2004: if a system’s behavior cannot be fully foreseen, accountability cannot simply be delegated to the system. He called it the responsibility gap. Twenty-some years later, that gap sits inside every company that deploys agents, and it has not gotten smaller.
When both sides meet
It gets uncomfortable where the two sides meet. An investment memo consists of a market analysis, a business plan and a valuation. If all three come from the same model and the same input data, they can rest on the same unchecked assumption. Then three documents agree, and that very agreement reassures everyone involved, even though nobody checked anything independently. Formally, everything is in order: the memo is complete, the sources are cited, the sign-offs are in place.
It does not take much imagination to carry this further. The company’s agent writes the sustainability report, the auditor’s agent reviews it, and at the end two people sign who have not traced either run. Everyone acted correctly, and nobody is accountable. That is Beck’s organized irresponsibility in its most consistent form yet, and it shows the real danger: AI can scale insight. It can also scale false certainty.
Accountability has to be engineered
An agent cannot carry accountability. That does not mean a company using agents has to act irresponsibly. It means accountability has to be built into the system.
Filippo Santoni de Sio and Jeroen van den Hoven formulated two conditions for this, originally for autonomous weapons. First, the system must demonstrably respond to the reasons of the people who deploy it and to the facts of the situation. Second, every one of its results must be traceable to at least one person who understands what the system does and who can change it. They call it meaningful human control. Someone who only signs meets neither condition.
For a company, that first means clear chains. Every result an agent produces belongs to a person who set its task, limits and review, and who changes them when needed. It also has to stay visible whether a change to a process rests on a deliberate decision or on an inference by the agent that nobody ever confirmed. And whoever is responsible for a chain needs, above all, real authority to stop it, otherwise they are just a spectator.
Control needs independence
There is a second requirement: a result counts only once it has survived more than one run, and independent runs at that. Asking the same model three times does not give you three independent opinions. People knew this long before AI. In 1986, John Knight and Nancy Leveson had 27 students at two universities write the same program independently and tested the versions against a million inputs. The programs failed conspicuously often in the same places, because people across many fields of knowledge still think in similar patterns. In June 2026, a group at KTH in Stockholm repeated the experiment with coding agents. The errors were correlated again, but groups of three with a majority vote cut the error count by about two thirds on average. Independence is achievable, then, but it does not happen on its own. You have to engineer it deliberately: different models, different data, a separate review or human judgment. The point is not to process every result several times in full. The point is to spot dependencies before repetition turns into false certainty.
This leads to a shift many people will not like. Accountability moves from the result to the design. Trust in that design, in turn, comes from a track record of verified performance. Not all at once, but by observing the agent’s performance, evaluating it and widening its mandate step by step. Recognizing its own knowledge gaps and handing over to a person at the right moment count toward that track record, too. For the division head, that means concretely: he is not accountable for the individual document, which he cannot check anyway, but for the machine that produces documents. Which runs exist, how independent they are, who reviews, and what happens to an error that slipped through. Such an error becomes a fixed test case for every future version, otherwise the design learns nothing. Strictly speaking, that is an engineering job and a leadership job, and it cannot be handed off to the compliance department.
What really gets expensive
That intelligence is getting cheap is no longer a forecast. A model that gives a company an edge today can be used by its competitor tomorrow. Something else becomes valuable: an organization in which someone actually stands behind a result, because they know the design underneath it. You cannot simply buy that. It comes from experience, clear lines of responsibility, technical limits, independent review and the ability to actually learn from mistakes.
Most companies are building the opposite right now. They are putting agents into processes that were trained for years to produce proof, and they are pleased with the speed. As an investor, I do not ask such a company which AI model it uses. I ask who is allowed to stop the chain, and when that last happened.
