{"id":1344,"date":"2020-11-17T15:26:26","date_gmt":"2020-11-17T15:26:26","guid":{"rendered":"https:\/\/tomgeraghty.co.uk\/?p=1344"},"modified":"2023-06-19T08:46:15","modified_gmt":"2023-06-19T08:46:15","slug":"resilience-engineering-and-devops","status":"publish","type":"post","link":"https:\/\/tomgeraghty.co.uk\/index.php\/resilience-engineering-and-devops\/","title":{"rendered":"Resilience Engineering and DevOps – A Deeper Dive"},"content":{"rendered":"

[This is a work in progress. If you spot an error, or would like to contribute, please get in touch<\/a>]<\/strong><\/em><\/p>\n

The term “Resilience Engineering” is appearing more frequently in the DevOps domain, field of physical safety, and other industries, but there exists some argument about what it really means. That disagreement doesn’t seem to occur in those domains where Resilience Engineering has been more prevalent and applied for some time now, such as healthcare and aviation. Resilience Engineering is an academic field of study and practice in its own right. There is even a Resilience Engineering Association<\/a>.<\/p>\n

Resilience Engineering<\/strong> is a multidisciplinary field associated with safety science, complexity, human factors and associated domains that focuses on understanding how complex adaptive systems cope with, and learn from, surprise.<\/p>\n

It addresses human factors, ergonomics, complexity, non-linearity, inter-dependencies, emergence, formal and informal social structures, threats and opportunities. A common refrain in the field of resilience engineering is “there is no root cause<\/a>“, and blaming incidents on “human error” is also known to be counterproductive, as Sidney Dekker explains so eloquently in “The Field Guide To Understanding Human Error”<\/a>.<\/p>\n

Resilience engineering is “<\/span>The intrinsic ability of a system to adjust its functioning prior to, during, or following changes and disturbances, so that it can sustain required operations under both expected and unexpected conditions.<\/span><\/i>” Prof Erik Hollnagel<\/a><\/span><\/p>\n

It is the \u201csustained adaptive capacity\u201d of a system, organisation, or community.<\/b><\/p>\n

Resilience engineering has the word \u201cengineering\u201d in, which makes us typically think of machines, structures, or code, and this is maybe a little misleading. Instead, maybe try to think about engineering being the process of response, creation and change.<\/span><\/p>\n

Systems<\/h2>\n

Resilience Engineering also refers to \u201csystems\u201d, which might also lead you down a certain mental path of mechanical or digital systems. Widen your concept of systems from software and machines, to organisations, societies, ecosystems, even solar systems. They\u2019re all systems in the broader sense.<\/span><\/p>\n

Resilience engineering refers in particular to complex systems, and typically, complex systems involve people. Human beings like you and I (I don’t wish to be presumptive but I’m assuming that you’re a human reading this).<\/span><\/p>\n

Consider Dave Snowden\u2019s Cynefin framework<\/a>:<\/span><\/p>\n

\"cynefin\"<\/p>\n

Systems in an Obvious<\/strong> state are fairly easy to deal with. There are no unknowns<\/strong> – they\u2019re fixed and repeatable in nature, and the same process achieves the same result each time, so that we humans can use things like Standard Operating Procedures to work with them.<\/span><\/p>\n

Systems in a Complicated<\/strong> state are large, usually too large for us humans to hold in our heads in their entirety, but are finite and have fixed rules. They possess known unknowns<\/strong> – by which we mean that you can find the answer if you know where to look. A modern motorcar, or a game of chess, are complicated – but possess fixed rules that do not change. With expertise and good practice, such as employed by surgeons or engineers or chess players, we can work with systems in complicated states.<\/span><\/p>\n

Systems in a Complex<\/strong> state possess unknown-unknowns<\/strong>, and include realms such as battlefields, ecosystems, organisations and teams, or humans themselves. The practice in complex systems is probe, sense, and respond. Complexity resists reductionist attempts at determining cause and effect because the rules are not fixed, therefore the effects of changes can themselves change over time, and even the attempt of measuring or sensing in a complex system can affect the system. When working with complex states, feedback loops that facilitate continuous learning about the changing system are crucial.<\/span><\/p>\n

Systems in a Chaotic<\/strong> state are impossible to predict<\/strong>. Examples include emergency departments or crisis situations. There are no real rules to speak of, even ones that change. In these cases, acting first is necessary. Communication is rapid, and top-down or broadcast, because there is no time, or indeed any use, for debate.<\/span><\/p>\n

Resilience<\/h2>\n

As Erik Hollnagel has said repeatedly since Resilience Engineering began (Hollnagel & Woods, 2006), resilience is about what a system can do \u2014 including its capacity:\u00a0<\/a><\/span><\/p>\n