The Puppet State of DevOps Report 2021 – A Summary

I get a bit confused every year about who is writing the State of DevOps Report, and how that gets decided, and in the past it’s been Puppet, Google, DORA and others, but this year, 2021, it was definitely Puppet.

[Edit: apparently there are two State of DevOps reports now… I’m staying out of that particular argument though!]

The state of DevOps report each year attempts to synthesise and aggregate the current state of the technology industry across the world in respect to our collective transformation towards delivering value faster and more reliably. Or as Jonathan Smart puts it, “Sooner. Safer, Happier”. The DevOps shift has been in progress for over a decade now, and whilst DevOps was always really about culture, the most recent reports are now emphasising the importance of culture, progressive leadership, inclusion, and diversity more than ever before.

Last year, in 2020, the core findings of the State of DevOps Report focussed on:

  1. The technology industry in general still had a long way to go and there remained significant areas for improvement across all sectors.
  2. Internal platforms and platform teams are a key enabler of performance, and more organisations were starting to adopt this approach.
  3. Adopting a long-term product approach over short-term project-oriented improves performance and facilitates improved adoption of DevOps cultures and practices.
  4. Lean, automated, and people-oriented change management processes improve velocity and performance over traditional gated approaches.

 

This year (2021), there are a number of key findings building on previous DevOps reports:

1. Well defined and architected Team Topologies improve flow.

Clear organisational dynamics including well-defined boundaries, responsibilities, and interactions, are critical to achieve fast flow of of value. Whilst last year highlighted the importance of internal platforms, this report emphasises the importance of Conway’s Law, and shows that well defined team structures and interactions, such as platform teams (which scale out the benefits of DevOps transformations across multiple teams), cross-functional value-stream aligned teams, and enabling teams strongly influence the architecture and performance of the technology they build. Team “Interaction Modes” as seen in the diagram below are also critical to define, in the same way that we would define API specifications.

DevOps and Team Topologies

The book Team Topologies expands upon this concept in great detail, and Matthew and Manuel, the authors, also provide excellent training in order to apply these concepts to your contexts.

Clear team responsibilities

What is also clear from the State of DevOps Report this, and has been for some time, is that siloing DevOps practices into separate “DevOps teams” is an antipattern to success in most cases. And there should still be no such thing as a “DevOps Engineer”.

2. Use of cloud technology remains immature in many organisations.

Whilst the majority of organisations are now using cloud technology such as IaaS (infrastructure-as-a-service), most organisations are still using it in ways that are analogous to the ways we used to manage on-premise or datacentre technology. High performers are adopting “cloud-native” technologies and ways of working, including the NIST (National Institute of Standards and Technology)essential characteristics of cloud computing: “on-demand self-service, broad network access, resource pooling, rapid elasticity or expansion, and measured service.” How these are implemented is very context-specific, but includes the principles of platform(s) as a product or service, and high competencies in monitoring and alerting and SRE (Site Reliability Engineering) capabilities, whether as SRE teams, or SRE roles in cross-functional teams.

Cloud native capabilities and devops

3. Security is shifting left.

High performers in the technology space integrate security requirements early in the value chain, including security stakeholders into the design phase and build phases rather than just at deploy, or even worse, run, phases. Traditional “inspection” approaches to security, governance and compliance significantly impact flow and quality, resulting in higher risk and lower reliability. Applying DevOps principles and practices to include change management, security and compliance improves flow, reliability, performance, and keeps the auditors off your back.

DevSecOps transformation

Whilst some call this DevSecOps, many would simply call it DevOps the way it was always intended to be.

4. DevOps and Digital Transformation must be delivered from the bottom-up, and empowered from the top-down.

Culture is the reflection of what we do, the behaviours we manifest, the practices we perform, the way we interact and what we believe. Culture change is never successfully implemented only from the top-down, and must be driven and engaged with by those expected to actually change their behaviours and practices.

DevOps transformation promotion

Cultural barriers to change include unclear responsibilities (enter Team Topologies), insufficient feedback loops, fear of change and a low prioritisation for fast flow, and most importantly, a lack of psychological safety.

Psychological safety and risk

A lot of these findings, unsurprisingly, echo the findings from Google’s 2013 Project Aristotle, which showed that psychological safety, clarity, dependability, meaning and impact were crucial for high performance in teams.

Extra note on “Legacy” workloads.

The report highlighted the “dragging” effect that legacy workloads can have on flow and change rate, as an effect of their architecture, codebase, or infrastructure, or the fact that nobody in the organisation understands it any longer. Rather than leave alone your legacy workloads, invest in them “so that they’re no longer an inhibitor of progress”. This could be as simple as virtualisation of physical hardware, or decomposing part of the system and moving certain components to cloud-native platforms such as Kubernetes or OpenShift. Even if you have to do something a bit “ugly” such as creating 18GB containers, it’s still a step forward.

TL;DR

  1. Organisational dynamics must be considered crucial to transformation.
  2. Cloud-native approaches are critical. It is no good to simply move traditional workloads to the cloud.
  3. Shift security, compliance and change governance left, and include security stakeholders in all stages of value delivery.
  4. Culture change is key, and must be promoted from the very “top” as well as delivered from the “bottom”. Psychological safety is at the core of digital and cultural transformations.

If you’re interested in finding out more about DevOps and Digital Transformations, Psychological Safety, or Cloud Native approaches, please get in touch.

Thanks to Nigel Kersten, Kate McCarthy, Michael Stahnke and Caitlyn O’Connell for working on the 2021 State of DevOps Report and providing us with these insights.

View the 2021 Accelerate State of DevOps Report summary here.

Psychological safety, homogeneity and diversity in group contexts.

Psychological safety stems in large part from a shared set of group values, norms, and experiences. When we’re in a group that shares the same perspectives, life experiences, language and culture, we feel safer and more able to listen, contribute, admit mistakes and challenge the ideas of others.

Think about tribalism – which is sometimes framed as a negative, but can be extremely powerful for good and for bad. Football fans who all passionately support the same team, are an incredible support group and safe space for many people. But of course there can be a dark side to the same phenomenon, as manifested by football hooliganism and violence, whereby outgroup members are seen as “The Other” and thus either a threat, or a group to be dominated or beaten.

Being part of a “tribe”, a group of similar people (similar in whatever way brings that tribe together, whether it’s a football club or a sexuality) brings with it a sense of belonging and safety.

Groups of people who are more similar, who have similar life experiences, language, values, and expectations (of each other and of themselves) will find it easier to attain higher levels of psychological safety because they understand more implicitly whats expected, what behaviour is accepted and “good”, and feel less fear from other members of the group

This is why it’s useful to have “safe spaces” where people, for example, LGBTQ+, or people of colour, can talk openly about the things they’re dealing with. Having people in the group without that life experience can make psychological safety harder to attain – but not impossible. This applies not just to those who we’d typically view as disadvantaged or disenfranchised in some way – we are all somewhere on the myriad spectrum of intersectionality.

For example, I, as a white, english speaking man, may feel very safe in a very heterogenous group of different genders, ethnicities and languages by dint of the privilege as a white man that I’m lucky to benefit from – but if I was in a group that also consists of various Elon Musks and Jeff Bezos characters, I’m probably going to feel very psychologically unsafe, and I’d be highly unlikely to speak up, admit mistakes, and certainly not challenge the ideas of others in the group.

Creating a homogenous group, whether it’s of gender, sexuality, class, colour or background is a shortcut, a cheat code, for psychological safety.

But ultimately, homogeneous groups will never attain the performance level that a highly diverse, heterogenous group will. Diversity brings different experiences, knowledge, insights, that increase the ultimate capability of the group to do the thing they’re trying to do – whether that’s build a product or run a country.

So, whilst a homogenous group can be an effective way to increase, quickly, the psychological safety of a group, it is just a shortcut. It’s not a reason to suggest that we should create homogenous teams – and certainly not an argument to segregate society, if one was that way inclined.

Diversity arguably does make psychological safety harder to attain, because there isn’t as much of a set of inherent shared group norms, beliefs and experiences. However, that “weakness” is also a strength. By working on creating those group norms, values and behaviours, a more diverse group can reach far higher potential than a less diverse, more homogenous group – through exploiting that wide spectrum of experiences, knowledge and insights.