The Puppet State of DevOps Report 2021 – A Summary

I get a bit confused every year about who is writing the State of DevOps Report, and how that gets decided, and in the past it’s been Puppet, Google, DORA and others, but this year, 2021, it was definitely Puppet.

[Edit: apparently there are two State of DevOps reports now… I’m staying out of that particular argument though!]

The state of DevOps report each year attempts to synthesise and aggregate the current state of the technology industry across the world in respect to our collective transformation towards delivering value faster and more reliably. Or as Jonathan Smart puts it, “Sooner. Safer, Happier”. The DevOps shift has been in progress for over a decade now, and whilst DevOps was always really about culture, the most recent reports are now emphasising the importance of culture, progressive leadership, inclusion, and diversity more than ever before.

Last year, in 2020, the core findings of the State of DevOps Report focussed on:

  1. The technology industry in general still had a long way to go and there remained significant areas for improvement across all sectors.
  2. Internal platforms and platform teams are a key enabler of performance, and more organisations were starting to adopt this approach.
  3. Adopting a long-term product approach over short-term project-oriented improves performance and facilitates improved adoption of DevOps cultures and practices.
  4. Lean, automated, and people-oriented change management processes improve velocity and performance over traditional gated approaches.

 

This year (2021), there are a number of key findings building on previous DevOps reports:

1. Well defined and architected Team Topologies improve flow.

Clear organisational dynamics including well-defined boundaries, responsibilities, and interactions, are critical to achieve fast flow of of value. Whilst last year highlighted the importance of internal platforms, this report emphasises the importance of Conway’s Law, and shows that well defined team structures and interactions, such as platform teams (which scale out the benefits of DevOps transformations across multiple teams), cross-functional value-stream aligned teams, and enabling teams strongly influence the architecture and performance of the technology they build. Team “Interaction Modes” as seen in the diagram below are also critical to define, in the same way that we would define API specifications.

DevOps and Team Topologies

The book Team Topologies expands upon this concept in great detail, and Matthew and Manuel, the authors, also provide excellent training in order to apply these concepts to your contexts.

Clear team responsibilities

What is also clear from the State of DevOps Report this, and has been for some time, is that siloing DevOps practices into separate “DevOps teams” is an antipattern to success in most cases. And there should still be no such thing as a “DevOps Engineer”.

2. Use of cloud technology remains immature in many organisations.

Whilst the majority of organisations are now using cloud technology such as IaaS (infrastructure-as-a-service), most organisations are still using it in ways that are analogous to the ways we used to manage on-premise or datacentre technology. High performers are adopting “cloud-native” technologies and ways of working, including the NIST (National Institute of Standards and Technology)essential characteristics of cloud computing: “on-demand self-service, broad network access, resource pooling, rapid elasticity or expansion, and measured service.” How these are implemented is very context-specific, but includes the principles of platform(s) as a product or service, and high competencies in monitoring and alerting and SRE (Site Reliability Engineering) capabilities, whether as SRE teams, or SRE roles in cross-functional teams.

Cloud native capabilities and devops

3. Security is shifting left.

High performers in the technology space integrate security requirements early in the value chain, including security stakeholders into the design phase and build phases rather than just at deploy, or even worse, run, phases. Traditional “inspection” approaches to security, governance and compliance significantly impact flow and quality, resulting in higher risk and lower reliability. Applying DevOps principles and practices to include change management, security and compliance improves flow, reliability, performance, and keeps the auditors off your back.

DevSecOps transformation

Whilst some call this DevSecOps, many would simply call it DevOps the way it was always intended to be.

4. DevOps and Digital Transformation must be delivered from the bottom-up, and empowered from the top-down.

Culture is the reflection of what we do, the behaviours we manifest, the practices we perform, the way we interact and what we believe. Culture change is never successfully implemented only from the top-down, and must be driven and engaged with by those expected to actually change their behaviours and practices.

DevOps transformation promotion

Cultural barriers to change include unclear responsibilities (enter Team Topologies), insufficient feedback loops, fear of change and a low prioritisation for fast flow, and most importantly, a lack of psychological safety.

Psychological safety and risk

A lot of these findings, unsurprisingly, echo the findings from Google’s 2013 Project Aristotle, which showed that psychological safety, clarity, dependability, meaning and impact were crucial for high performance in teams.

Extra note on “Legacy” workloads.

The report highlighted the “dragging” effect that legacy workloads can have on flow and change rate, as an effect of their architecture, codebase, or infrastructure, or the fact that nobody in the organisation understands it any longer. Rather than leave alone your legacy workloads, invest in them “so that they’re no longer an inhibitor of progress”. This could be as simple as virtualisation of physical hardware, or decomposing part of the system and moving certain components to cloud-native platforms such as Kubernetes or OpenShift. Even if you have to do something a bit “ugly” such as creating 18GB containers, it’s still a step forward.

TL;DR

  1. Organisational dynamics must be considered crucial to transformation.
  2. Cloud-native approaches are critical. It is no good to simply move traditional workloads to the cloud.
  3. Shift security, compliance and change governance left, and include security stakeholders in all stages of value delivery.
  4. Culture change is key, and must be promoted from the very “top” as well as delivered from the “bottom”. Psychological safety is at the core of digital and cultural transformations.

If you’re interested in finding out more about DevOps and Digital Transformations, Psychological Safety, or Cloud Native approaches, please get in touch.

Thanks to Nigel Kersten, Kate McCarthy, Michael Stahnke and Caitlyn O’Connell for working on the 2021 State of DevOps Report and providing us with these insights.

View the 2021 Accelerate State of DevOps Report summary here.

Digital Transformation and DevOps: Enterprise Resilience

digital transformation

Digital Transformation is having a real moment in industry, in part due to the huge changes as a result of the pandemic of 2020.  But as usual, there’s little agreement about what it means. In contrast to previous “transformations” such as ITIL, Lean, Agile, or DevOps, digital transformation doesn’t simply mean automating processes, becoming more efficient, offering your existing products and services online, creating an app, or shifting your infrastructure to the cloud. Even the annual State of DevOps Reports are beginning to focus more on digital and organisational transformation rather than a specific focus solely on DevOps.

What is digital transformation?

True digital transformation means transforming everything about your organisation in respect to people and technology towards an engaged, agile, happy and high performing organisation. DevOps was (and still is) one key aspect of this approach. The only way to truly achieve organisational resilience or enterprise agility is to fundamentally transform the foundations of the organisation. The list below describes just some of the aspects of digital transformation and the areas to address

  • Culture, values and behaviours
  • Practices and ways of working
  • Communication structures
  • Hierarchies
  • How financial budgets and funding models are managed
  • How teams and people are measured and incentivised
  • How and what metrics are used
  • Cloud native architectures and practices
  • Moving from projects to products
  • Team structures, topologies and interactions
  • Recruitment and onboarding/offboarding practices
  • Value stream alignment
  • Breaking down silos
  • Embedding the ability to change and adapt
  • Reducing cognitive load
  • Psychological safety in delivery teams, senior leadership teams and functional teams
  • IT services and operational technologies
  • Facilities, colocation, office layouts (especially options for open-plan or not)
  • And many many more – in fact, here is an (incomplete) list of organisational factors relevant to transformation.

Why digital transformation?

What’s your organisational goal? Maybe it’s increasing your speed to market for new products and features, maybe it’s reducing risk of failure in production and improving reliability, or maybe it’s to keep doing what you’re doing but with less stress and improved flow. If you’re only looking to reduce costs however, digital transformation is not for you: one of the core requirements for a transformation to succeed is for everyone in the organisation to be psychologically safe, engaged and get behind it, so reducing costs and potentially cutting workforce numbers is not going to create that movement.

What is Enterprise Resilience?

Resilience Engineering is a decades-old field of applied research that focusses on the capacity to anticipate, detect, respond to and adapt to change. Organisational “robustness” might mean being able to withstand massive disrupting events such as pandemics or competition, but enterprise agility represents the resilience engineering concept of true resilience – not just “coping” with change, but improving from it and future challenges. I believe that Resilience Engineering is the direction that DevOps is evolving into.

Why is digital transformation so complex?

Despite many attempts to simplify the concept of digital transformation, it remains one of the most challenging endeavours we could embark upon.

Galbraith Star model
Galbraith Star model

I’m not a huge fan of over-simplifying organisational complexity into components, especially models such as Galbraith’s Star that place “people” as one of the components (and certainly not models that consider anything other than people to be the primary element). Whilst models such as this may help people compartmentalise the transformation challenge, in almost every case, the fractures between the various components don’t actually exist in the way they’re presented.

Organisations are not simply jigsaw pieces of technology, tools, and people that react and function in predictable ways. As the Cynefin model shows us, complex systems, such as sociotechnical systems (the organisations that we work in) require a probe-sense-respond approach that has built-in feedback loops to determine what effect the intervention you’re working on is having. Everything in digital transformation is an experiment.

upload.wikimedia.org/wikipedia/commons/1/15/Cyn...

It’s also important to avoid localised optimisation – applying digital transformation approaches to one part of an organisation whilst ignoring other parts will only result in tension, bottlenecks, high-friction and failures elsewhere. Likewise, changing one small part of a system, especially a complex system, will have unintended and unanticipated effects elsewhere, so a complete, holistic view of the entire organisation is critical.

Digital transformation is a series of experiments

This is why, if anyone suggests that there is a detailed “roadmap”, or even worse, a Gannt chart, for a digital transformation project, at best it’s naive and worst, it’s fiction. Any digital transformation process must be made not of a fixed plan, but a series of experiments that allow for iterative improvements to be made.

Digital transformaiton - everything is an experiment

When you think about digital transformation in this way, it also becomes clear why it will never be “finished”. Organisations, like the people they consist of, constantly change and evolve, just like the world we operate in, so whilst digital transformation is undoubtedly of huge value and an effective approach to organisational change, you will never, ever, be “done”.

In my role as Transformation Lead at Red Hat Open Innovation Labs, we use the Mobius Loop approach to provide structure to this experimental, feedback-based, continuous improvement and transformation journey.  If you’re interested in digital transformation, DevOps, Psychological Safety and how you can begin to set transformation in motion in your own organisation, get in touch.