The Puppet State of DevOps Report 2021 – A Summary

I get a bit confused every year about who is writing the State of DevOps Report, and how that gets decided, and in the past it’s been Puppet, Google, DORA and others, but this year, 2021, it was definitely Puppet.

[Edit: apparently there are two State of DevOps reports now… I’m staying out of that particular argument though!]

The state of DevOps report each year attempts to synthesise and aggregate the current state of the technology industry across the world in respect to our collective transformation towards delivering value faster and more reliably. Or as Jonathan Smart puts it, “Sooner. Safer, Happier”. The DevOps shift has been in progress for over a decade now, and whilst DevOps was always really about culture, the most recent reports are now emphasising the importance of culture, progressive leadership, inclusion, and diversity more than ever before.

Last year, in 2020, the core findings of the State of DevOps Report focussed on:

  1. The technology industry in general still had a long way to go and there remained significant areas for improvement across all sectors.
  2. Internal platforms and platform teams are a key enabler of performance, and more organisations were starting to adopt this approach.
  3. Adopting a long-term product approach over short-term project-oriented improves performance and facilitates improved adoption of DevOps cultures and practices.
  4. Lean, automated, and people-oriented change management processes improve velocity and performance over traditional gated approaches.

 

This year (2021), there are a number of key findings building on previous DevOps reports:

1. Well defined and architected Team Topologies improve flow.

Clear organisational dynamics including well-defined boundaries, responsibilities, and interactions, are critical to achieve fast flow of of value. Whilst last year highlighted the importance of internal platforms, this report emphasises the importance of Conway’s Law, and shows that well defined team structures and interactions, such as platform teams (which scale out the benefits of DevOps transformations across multiple teams), cross-functional value-stream aligned teams, and enabling teams strongly influence the architecture and performance of the technology they build. Team “Interaction Modes” as seen in the diagram below are also critical to define, in the same way that we would define API specifications.

DevOps and Team Topologies

The book Team Topologies expands upon this concept in great detail, and Matthew and Manuel, the authors, also provide excellent training in order to apply these concepts to your contexts.

Clear team responsibilities

What is also clear from the State of DevOps Report this, and has been for some time, is that siloing DevOps practices into separate “DevOps teams” is an antipattern to success in most cases. And there should still be no such thing as a “DevOps Engineer”.

2. Use of cloud technology remains immature in many organisations.

Whilst the majority of organisations are now using cloud technology such as IaaS (infrastructure-as-a-service), most organisations are still using it in ways that are analogous to the ways we used to manage on-premise or datacentre technology. High performers are adopting “cloud-native” technologies and ways of working, including the NIST (National Institute of Standards and Technology)essential characteristics of cloud computing: “on-demand self-service, broad network access, resource pooling, rapid elasticity or expansion, and measured service.” How these are implemented is very context-specific, but includes the principles of platform(s) as a product or service, and high competencies in monitoring and alerting and SRE (Site Reliability Engineering) capabilities, whether as SRE teams, or SRE roles in cross-functional teams.

Cloud native capabilities and devops

3. Security is shifting left.

High performers in the technology space integrate security requirements early in the value chain, including security stakeholders into the design phase and build phases rather than just at deploy, or even worse, run, phases. Traditional “inspection” approaches to security, governance and compliance significantly impact flow and quality, resulting in higher risk and lower reliability. Applying DevOps principles and practices to include change management, security and compliance improves flow, reliability, performance, and keeps the auditors off your back.

DevSecOps transformation

Whilst some call this DevSecOps, many would simply call it DevOps the way it was always intended to be.

4. DevOps and Digital Transformation must be delivered from the bottom-up, and empowered from the top-down.

Culture is the reflection of what we do, the behaviours we manifest, the practices we perform, the way we interact and what we believe. Culture change is never successfully implemented only from the top-down, and must be driven and engaged with by those expected to actually change their behaviours and practices.

DevOps transformation promotion

Cultural barriers to change include unclear responsibilities (enter Team Topologies), insufficient feedback loops, fear of change and a low prioritisation for fast flow, and most importantly, a lack of psychological safety.

Psychological safety and risk

A lot of these findings, unsurprisingly, echo the findings from Google’s 2013 Project Aristotle, which showed that psychological safety, clarity, dependability, meaning and impact were crucial for high performance in teams.

Extra note on “Legacy” workloads.

The report highlighted the “dragging” effect that legacy workloads can have on flow and change rate, as an effect of their architecture, codebase, or infrastructure, or the fact that nobody in the organisation understands it any longer. Rather than leave alone your legacy workloads, invest in them “so that they’re no longer an inhibitor of progress”. This could be as simple as virtualisation of physical hardware, or decomposing part of the system and moving certain components to cloud-native platforms such as Kubernetes or OpenShift. Even if you have to do something a bit “ugly” such as creating 18GB containers, it’s still a step forward.

TL;DR

  1. Organisational dynamics must be considered crucial to transformation.
  2. Cloud-native approaches are critical. It is no good to simply move traditional workloads to the cloud.
  3. Shift security, compliance and change governance left, and include security stakeholders in all stages of value delivery.
  4. Culture change is key, and must be promoted from the very “top” as well as delivered from the “bottom”. Psychological safety is at the core of digital and cultural transformations.

If you’re interested in finding out more about DevOps and Digital Transformations, Psychological Safety, or Cloud Native approaches, please get in touch.

Thanks to Nigel Kersten, Kate McCarthy, Michael Stahnke and Caitlyn O’Connell for working on the 2021 State of DevOps Report and providing us with these insights.

View the 2021 Accelerate State of DevOps Report summary here.

Adoption vs Adaption in Resilience Engineering and DevOps

tree - adoption vs adaptation

DevOps was, and still is to a degree, a “ground-up” phenomenon. It became to be adopted, adapted and evolved by engineering teams before “management” even really understood what it was. 

The openness and flexibility that was expounded by DevOps meant that it was able to be interpreted in different ways by different teams in different contexts. This was a key strength, because unlike rigid frameworks such as ITIL, the people responsible for doing the work were able to modify and apply DevOps to their own work, in the ways that best suited them.

But this loose definition also proved to be a weakness. Because there were no limits to how DevOps could be interpreted and applied, it was often (and still is) interpreted as a technology solution rather than cultural change. This resulted in “DevOps engineers”, or “DevOps teams” whose remit is focussed on cloud technology, CI/CD pipelines, or automation. 

Due to this, we’re still far behind from where we could have been as an industry. Despite everyone in technology knowing the term “DevOps” and almost every firm adopting some degree of DevOps practices, these transformations have often stuttered or even failed, in part because it’s unclear to many what DevOps really is and how to “do” DevOps.

Resilience Engineering is a field of applied research that considers organisational-scale capability to anticipate, detect, respond and adapt to change. The principle of socio-technicality is core to RE: the premise that you can’t separate people from technology; if you change the technology, it will affect people, and if you change the way people work or communicate or the way teams are structured, it will impact the technology created or consumed by those people. 

RE as a field has been around for almost two decades, but only now (for various reasons including the Covid pandemic) is beginning to touch mainstream discussions and discussed in the same conversations as digital and organisational transformation. 

Researchers and Practitioners of RE are quick to clarify what RE is, and is not, during these discussions. Whilst it may seem dogmatic to be so strict about what is within the remit of the field, I think this could be the valuable lesson learned from one of the weaknesses of DevOps. In order for organisations to successfully adopt and adapt to a new operating model and principles such as RE, it’s essential to understand very clearly what it is. 

Resilience Engineering, despite being nearly twenty years old as a field, is somewhat embryonic in its adoption outside of a narrow field of specialist researchers and practitioners, and as such, it’s crucial that we define accurately what it is, what it is not, and resist attempts (intentional or unintentional) to co-opt the term to mean something more akin to chaos engineering, automation, or system hardening efforts. 

A balance must be struck between defining accurately what RE is, and tolerating (indeed, encouraging) a flexibility of interpretation and adoption in different contexts. DevOps was maybe too loose in this respect, other paradigms such as ITIL or SAFe were maybe too strict and dogmatic. Maybe with Resilience Engineering, the sweet spot will be found.