Residuality Theory

Residuality Theory is an innovative approach to software architecture that provides a robust foundation for architectural decisions in the face of uncertainty.

Software architects constantly face tough decisions with incomplete and uncertain information. However, in competitive markets, we don’t have the option to gather all the necessary information before making a commitment. While feature-level choices are often reversible, architectural decisions have long-term impacts and are harder to undo.

Moreover, when time and experience delivered the necessary feedback, the original decisions will progressively become a burden compared to more modern alternatives. No architectural choice is perfect in retrospect.

To make things worse, not every constraint to the technical decision space is technical. Service providers can go bankrupt or be acquired by other companies. They can discontinue services or change their pricing without notice. No matter how technically skilled we are, uncertainty won’t go away.

Residuality Theory offers a unique perspective on tackling these challenges effectively.

Software architecture is a special place

One reason that makes it challenging to frame software architecture as a discipline is that it sits at the intersection of uncertainty and determinism. The technical side involves code and computers that perform tasks predictably. Distributed and asynchronous components can require special attention and a more sophisticated design. Component failures may require redundancy and load balancing. Still, we are in the complicated domain.

We don’t have the luxury of designing software systems in a vacuum. Software architecture is also about strategic choices. Choosing an implementation language for server-side performance might turn out to be a recruiting nightmare if your company needs to grow. A third-party component you rely on can change the license cost, or worse.

In Cynefin terms, while the architecture of a software system resides in the complicated domain, software architecture as a discipline falls within the complex domain. And this requires a different mindset.

Regardless of your technical ability in building reliable and robust software applications, a large part of your problem space cannot be controlled. Unknown unknowns are not an exception here.

Being ready for “theoretically” everything can result in overengineering, leading to longer development cycles now, to protect against many possible futures, much like wearing a raincoat on a sunny summer day.

Coping with uncertainty

Despite uncertainty, the expectation for software architecture is robustness, resilience, or antifragility. We should address crises promptly, turning them into opportunities for improvement, and ensure that system failures never happen again.

The typical response from the engineering mindset to uncertainty is to insist on complicated approaches, such as spreadsheets, checklists, coefficients, and probability. We assess crisis scenarios in terms of likelihood and associated risks. We rank and prioritise threats, often with a touch of bureaucracy.

Less “disciplined” professionals will rely on experience and gut feeling, on the thin line that separates a guru from a gambler. Brilliant choices without a robust foundation can turn out weak when decisions involve budget approval.

Enter Residuality Theory

Residuality Theory leverages the special properties of complicated systems situated in a complex space.

These systems, known as Hyperliminal Systems, exhibit some special properties: changes in the outer sphere tend to impact the inner side in a few specific locations, known as attractors.

The more stressors we apply to a system, the more the system gives us information about what needs to change to make our design more resilient.

Barry O’Reilly’s research work discovered that if there is a sufficient number of stressors on a hyperliminal system, we can detect the key attractors, regardless of the probability of the stressors.

Coping with the consequences of a very unlikely catastrophic event, like the giant flame-throwing lizard mentioned in some examples, will ultimately impact your architectural choices not so differently than a more mundane bankruptcy of your cloud provider.

While the source of a future stressor can be highly uncertain, the vulnerabilities of your system will cluster around the same attractors. Improving the design of the attractors will make your architecture more future-proof.

Residuality Theory relieves us from worrying about the probability of a crisis scenario. A sufficient quantity of stressors would do the job.

Stressor analysis

Stressor analysis shifts the architectural design into a different form of hypothetical space, where we leverage creativity and lateral thinking to produce many hypotheses. Then we use math and a spreadsheet to process the resulting evidence.

If we collect enough stressors, we can analyse their impact, adjust our initial architectural sketch, and redesign the attractors. Separation of concerns could be triggered by the evidence that too many sources stress a single component.

Repeating the impact analysis on the improved design will show improved numbers. With enough stressors and adjustments, the numbers will converge. After passing a given threshold, new scenarios won’t deliver any new insights.

At the same time, the shape of the architecture becomes more refined: a residue is a component that isn’t threatened by stressors.

But quantity and randomness of the stressors matter. So does the framing while investigating them. For example, asking ourselves, “What could possibly go wrong?” might create a blind spot in failure modes coming from too much success.

We need multiple scenarios; they should come from diverse sources, and they should be affordable.

Why does it matter?

Coping with uncertainty is stressful. Good design is a quest for perfection, but perfection can never be fully achieved. Still, projecting enough unlikely futures into your architectural decision can be the best foundation for making your architectural decisions as future-proof as possible.

Residuality Theory gives you proven, repeatable principles that work across diverse scenarios. By analysing the residues left behind after stressors impact a system, architects can improve resilience and adaptability, ensuring long-term system stability.

How can I know more?

There’s plenty of material available from Barry O’Reilly; you may want to take a look at one of his videos or read the two short books available on Leanpub: Residues – Time, Uncertainty, and Change in Software Architecture and The Architect’s Paradox – Uncertainty and the Philosophy of Software Architecture.

However, if you truly want to dive into the matter, we offer an engaging in-person workshop: Advanced Software Architecture with Resilience Theory, led by Barry O’Reilly himself!

 

 

 

 

 

 

 

---