Epistemic Fragility: Why IT Systems Fail at Scale

Most failures in IT are not caused by ignorance.
They stem from a misunderstanding of how systems break.

Book VI of Nassim Nicholas Taleb’s Antifragile is not about chaos in the abstract. It is about non-linearity. It explains why small causes can remain harmless while slightly larger ones suddenly produce disproportionate damage. This single idea explains incident blast radius, cascading failures, customer loss, and why the phrase “it worked yesterday” is operationally meaningless.

This section is uncomfortable for IT because it challenges optimism, forecasting, and the illusion of control. It forces a confrontation with fragility.

Linearity Is a Comfortable Fiction

Linear thinking assumes proportionality.
Double the traffic and you double the load.
Double the load and you double the degradation.
Small errors lead to small consequences.

Real systems do not behave this way.

They are either concave or convex.
Concave systems absorb stress and degrade slowly.
Convex systems amplify stress and collapse suddenly.

Most IT platforms prove to be convex under pressure, even if they appear stable under normal conditions.

Consider traffic. A moderate capacity issue may affect 5% of users. A 10% overshoot can suddenly impact 70%. Nothing fundamentally changed. A threshold was crossed. Feedback loops took over.

This is not an edge case. This is normal system behaviour.

Incident Blast Radius Is Non-Linear by Nature

IT organisations often discuss incidents as if impact scales with the severity of the root cause. This is a dangerous illusion.

Blast radius grows non-linearly because infrastructure is shared, domains are coupled, queues saturate abruptly, latency compounds across synchronous calls, and humans react late rather than early.

A minor scalability flaw can therefore destroy customer trust overnight. Customers do not experience root causes. They experience outcomes. Outcomes are convex.

If a system only works when nothing goes wrong, then it does not work.

Probability Thinking Fails in Convex Systems

This is where traditional risk language fails. Phrases such as low probability, unlikely scenario, or edge case are meaningless when consequences are extreme.

A rare event with catastrophic impact is not low risk. It is poorly priced risk.

This is also where the limits of forecasting become visible.

Following Karl Popper’s critique of historicism, the future cannot be reliably inferred from past regularities when systems evolve, interact, and change their own conditions. Forecasts fail not because data is missing, but because the system itself is altered by time, scale, and intervention.

In such environments, prediction is structurally weaker than falsification. You do not design safety by estimating how often something happens. You design safety by understanding what must not happen.

Disconfirmation Beats Validation

No amount of past success proves that a system is safe. A single failure can invalidate years of apparent stability.

This asymmetry matters operationally.

You can validate success millions of times without learning anything about the limits of a system. You only learn when something breaks. This is why negative knowledge is more reliable than positive knowledge.

We know the impossible far better than the possible. We can often state with confidence what would destroy a system, while remaining largely ignorant of all the paths that might appear to work.

Design should therefore start from known failure modes, not optimistic scenarios.

Via Negativa and Engineering Maturity

Taleb’s via negativa approach fits engineering better than any roadmap ever will.

Removing fragility is more effective than adding capability. Eliminating known failure paths delivers more value than building new features.

Yet IT organisations consistently underinvest in resilience, ignore known risks, reward heroics, and treat outages as bad luck rather than structural signals. This is not courage. It is negligence disguised as confidence.

Stability Is an Asset, Not a Liability

There is a persistent myth that stable systems are brittle and that constant change creates strength.

The opposite is often true.

A system that has been stable for a long time is more likely to remain stable, provided it is concave under stress. Time without failure increases confidence because failure surfaces have already been exposed and removed. This directly relates to meaningful SLAs, increasing MTTBF, and reduced operational surprises.

Stability must be earned through subtraction.

Complexity Turns Systems Convex

Most IT systems become fragile because they accumulate complexity.

This complexity is rarely necessary. It often emerges from hero culture, lack of domain isolation, architectural vanity, and poor system boundaries.

As complexity grows, delivery becomes non-linear. Small delays explode into months. Minor scope changes derail entire plans. Integration work becomes the real product.

Other engineering disciplines learned long ago to constrain and price complexity. Many large structures were delivered on time precisely because complexity was controlled, isolated, and respected.

IT often does the opposite.

Negative Error and Error Containment

In convex systems, errors are asymmetric. Small errors are survivable. Large errors are fatal.

The objective is therefore not precision or prediction. It is error containment. This means limiting blast radius, isolating domains, designing for failure, and preferring removal over optimisation.

If a system cannot fail gracefully, it is not robust.

System Thinking and Why Leadership Resists It

System thinking forces an uncomfortable shift. It replaces stories with structures and intentions with consequences.

Linear narratives allow leaders to believe that effort scales with outcome and that good intent compensates for fragile design. System thinking removes that comfort. It shows that outcomes emerge from interactions, delays, thresholds, and feedback loops rather than individual decisions.

This is precisely why it is resisted.

System thinking makes responsibility unavoidable. It reveals that most large incidents are not accidents, but predictable consequences of known structures. It exposes optimism as a choice, not a virtue.

Executives often reject this framing because it conflicts with incentive models built around growth, speed, and visibility. The cost of fragility is paid by customers, operators, and engineers, while the benefits of optimism accrue higher up the hierarchy.

Delivery Failure Is Also Non-Linear

The same non-linearity applies to delivery.

Most IT projects do not fail gradually. They appear to progress normally until a small delay, a dependency slip, or a late integration suddenly collapses the schedule.

This happens because modern IT delivery is convex by design. Excessive coupling, shared ownership, and layered abstractions mean that time risk compounds rather than accumulates linearly.

Historic engineering disciplines learned to constrain complexity. Large structures were delivered on time not because they were simple, but because complexity was isolated, priced, and governed by system boundaries.

IT often celebrates complexity as sophistication and then expresses surprise when delivery timelines explode.

Epistemic Fragility and the Illusion of Numbers

Epistemic fragility describes a system state in which confidence increases faster than understanding, where beliefs are reinforced by volume of data and repetition rather than constrained by causal models, failure analysis, and disconfirming evidence.

Optimism persists because it is inexpensive for those who express it and because it demands little commitment.

In modern IT, this optimism increasingly hides behind numbers.

Large volumes of data create an illusion of mastery. Dashboards, metrics, and forecasts generate confidence without necessarily generating understanding. Numbers are trusted even when their meaning, limits, and assumptions are poorly understood. This numerical comfort often substitutes for mathematical literacy.

Without a solid grounding in probability, non-linearity, and system dynamics, data becomes a form of magic. It reassures rather than constrains. It supports belief rather than discipline.

When optimistic assumptions fail, the damage is externalised. Customers lose trust. Operators absorb fatigue. Engineers are asked to compensate through heroics.

System thinking exposes this asymmetry. It shows that optimism is not neutral. It is a bet placed with other people’s capital, reinforced by data volume rather than scientific rigour and made fragile by misunderstanding the numbers themselves.

IT and the Loss of Engineering Foundations

Modern IT increasingly behaves like an engineering discipline without engineering.

Large parts of the field have lost contact with applied mathematics, probability theory, and the physical intuition that governs real systems. Non-linearity, feedback loops, saturation, and thresholds are treated as abstract concepts rather than operational realities.

In older engineering cultures, optimism without calculation was not seen as confidence. It was seen as incompetence. Bridges, power grids, aircraft, and industrial systems were not declared safe because they had worked yesterday. They were designed against known failure modes, worst cases, and asymmetries. Margins were explicit. Assumptions were conservative. Responsibility was personal.

IT gradually replaced this discipline with narrative. Roadmaps replaced models. Confidence replaced proof. Success metrics replaced safety margins.

This shift explains why optimism flourishes. It fills the void left by weakened foundations in mathematics, science, and system behaviour.

Optimism as a Leadership Shortcut

Optimism is attractive to leadership because it simplifies decision making.

It allows complexity to be ignored, uncertainty to be postponed, and accountability to be deferred. It converts structural risk into motivational language. When systems fail, the explanation is framed as execution error, bad luck, or exceptional circumstances rather than predictable outcomes of known structures.

System thinking breaks this shortcut. It forces leaders to engage with coupling, delay, convexity, and irreversible failure. It replaces intention with consequence.

That is why it is resisted.

The Uncomfortable Conclusion

IT continues to fail because it clings to linear thinking, optimistic forecasting, and validation through repetition.

Reality is non-linear.
Disconfirmation outweighs confirmation.
Fragility accumulates quietly.

Antifragility is not about embracing chaos. It is about refusing to ignore what can go wrong when the cost of being wrong is catastrophic.

System thinking exposes this clearly. That is precisely why it remains rare. It replaces comforting narratives with responsibility and removes plausible deniability.