{"id":821,"date":"2018-12-24T05:31:08","date_gmt":"2018-12-24T05:31:08","guid":{"rendered":"https:\/\/tomgeraghty.co.uk\/?p=821"},"modified":"2021-08-19T15:45:08","modified_gmt":"2021-08-19T15:45:08","slug":"compliance-in-devops-and-public-cloud","status":"publish","type":"post","link":"https:\/\/tomgeraghty.co.uk\/index.php\/compliance-in-devops-and-public-cloud\/","title":{"rendered":"Compliance in DevOps: Using public cloud services and automated governance"},"content":{"rendered":"

As a DevOps engineer, you\u2019ve achieved greatness. You\u2019ve containerised everything, built your infrastructure and systems in the cloud and you\u2019re deploying every day, with full test coverage and hardly any outages. You\u2019re even starting to think you might really enjoy your job.<\/span><\/p>\n

Then why are your compliance teams so upset?<\/span><\/p>\n

Let\u2019s take a step back. You know how to build secure applications, create back ups, control access to the data and document everything, and in general you\u2019re pretty good at it. You\u2019d do this stuff whether there were rules in place or not, right? <\/span><\/p>\n

Not always. Back in the late 90\u2019s, a bunch of guys in suits decided they could get rich by making up numbers in their accounts. Then in 2001 Enron filed for bankruptcy and the suits went to jail for fraud. That resulted in the Sarbanes-Oxley Act, legislation which forced publicly listed firms in the US to enforce controls to prevent fraud and enable effective audits. <\/span><\/p>\n

Sarbanes-Oxley isn\u2019t the only law that makes us techies do things certain ways though. Other compliance rules include HIPAA, ensuring that firms who handle clinical data do so properly; GDPR, which ensures adequate protection of EU citizens\u2019 personal data; and PCI-DSS, which governs the use of payment card data in order to prevent fraud (and isn\u2019t a law, but a common industry standard). Then there are countless other region and industry specific rules, regulations, accreditations and standards such as ISO 27001 and Cyber Essentials. <\/span><\/p>\n

Aside from being good practice, the main reason you\u2019d want to abide by these rules is to avoid losing your job and\/or going to jail. It\u2019s also worth recognising that demonstrating compliance can provide a competitive advantage over organisations that don\u2019t comply, so it makes business sense too. <\/span><\/p>\n

The trouble is, compliance is an old idea applied to new technology. HIPAA was enacted in 1996, Sarbanes-Oxley in 2002 and PCI DSS in 2004 (though it is frequently updated). In contrast the AWS EC2 service only went out of beta in late 2008, and the cloud has we know it has been around for just a few years. Compliance rules are rarely written with cloud technology in mind, and compliance teams sometimes fail to keep up to date with these platforms or modern DevOps-style practices. This can make complying with those rules tricky, if not downright impossible at times. How do you tell an auditor exactly where your data resides, if the only thing you know is that it\u2019s in Availability Zone A in region EU-West-1? (And don\u2019t even mention to them that one customer\u2019s Zone A isn\u2019t the same as another\u2019s). <\/span><\/p>\n

As any tech in a regulated industry will appreciate, compliance with these rules is checked by regular, painful and disruptive audits. Invariably, audits result in <\/span>compliance levels looking something like a sine wave:<\/span><\/p>\n

\"\"<\/p>\n

This is because when an audit is announced, the pressure is suddenly on to patch the systems, resolve vulnerabilities, update documents and check procedures. Once the audit is passed, everyone relaxes a bit, patching lags behind again, documentation falls out of date and the compliance state drifts away from 100%. This begs the question, if we only become non-compliant between audits, is the answer to have really, really frequent audits? <\/span><\/p>\n

In a sense, yes. However, we can no longer accept that audits with spreadsheet tick box exercises, and infosec sign-off at deployment approval stage actually work. Traditional change management and compliance practices deliberately slow us down, with the intention of reducing the risk of mistakes. <\/span><\/p>\n

This runs counter to modern DevOps approaches. We move fast, making rapid changes and allowing teams to be autonomous in their decision making. Cloud technology confuses matters even further. For example, how can you easily define how many servers you have and what state they\u2019re in, if your autoscaling groups are constantly killing old ones and creating new ones? <\/span><\/p>\n

From a traditional compliance perspective, this sounds like a recipe for disaster. But we know that making smaller, more frequent changes will result in lower overall risk than large, periodic changes. What\u2019s more, we take humans out of the process wherever possible, implementing continuous integration and using automated tests to ensure quality standards are met.<\/span><\/p>\n

\"\"<\/p>\n

From a DevOps perspective, let\u2019s consider compliance in three core stages. The first pillar represents achieving compliance. That\u2019s the technical process of ensuring workloads and data are secure, everything is up to date, controlled and backed up. This bit\u2019s relatively easy for competent techs like us.<\/span><\/p>\n

The second pillar is about demonstrating that you\u2019re compliant. How do you show someone else, without too much effort, that your data is secure and your backups actually work? This is a little more difficult, and far less fun.<\/span><\/p>\n

The third pillar stands for maintaining compliance. This is a real challenge. \u00a0How do you ensure that with rapid change, new technology, and multiple teams\u2019 involvement, the system you built a year ago is still compliant now? This comes down to process and culture, and it\u2019s the most difficult of the three pillars to achieve.<\/span><\/p>\n

But it can be done. In DevOps and Agile culture, we shift left. We shorten feedback loops, decrease batch size, and improve quality through automated tests. This approach is now applied to security too, by embedding security tests into the development process and ensuring that it\u2019s automated, codified, frictionless and fast. It\u2019s not a great leap from there towards shifting compliance left too, codifying the compliance rules and embedding them within development and build cycles.<\/span><\/p>\n

First we had Infrastructure as Code. Now we\u2019re doing Compliance as Code. After all, what is a Standard Operating Procedure, if not a script for humans? If we can \u201ccode\u201d humans to carry out a task in exactly the same way every time, we should be able to do it for machines.<\/span><\/p>\n

Technologies such as AWS Config or Inspec allow us to constantly monitor our environment for divergence from a \u201ccompliant\u201d state. For example, if a compliance rule deems that all data at rest is encrypted, we can define that rule in the system and ensure we don\u2019t diverge from it – if something, human or machine, creates some unencrypted storage, it will be either be flagged for immediate attention or automatically encrypted.<\/span><\/p>\n

One of the great benefits of this approach is that the \u201cproof\u201d of compliance is baked into your system itself. When asked by an auditor whether data is encrypted at rest, you can reassure them that it\u2019s so by showing them your rule set. Since the rules are written as code, the documentation (the proof) is the control itself. <\/span><\/p>\n

If you really want your compliance teams to love you (or at least quit hassling you), this automation approach can be extended to documentation. Since the entire environment can be described in code at any point in time, you can provide point-in-time documentation of what exists in that environment to an auditor, or indeed at any point in time previously, if it\u2019s been recorded. <\/span><\/p>\n

By involving your compliance teams in developing and designing your compliance tools and systems, asking what\u2019s important to them, and building in features that help them to do their jobs, you can enable your compliance teams to serve themselves. In a well designed system, they will be able to find the answers to their own questions, and have confidence that high-trust control mechanisms are in place.<\/span><\/p>\n

Compliance as Code means that your environment, code, documentation and processes are continuously audited, and that third, difficult pillar of maintaining compliance becomes far easier: <\/span><\/p>\n

\"\"<\/p>\n

This is what continuous compliance looks like. Achieve this, and you\u2019ll see what a happy compliance team looks like too.<\/span><\/p>\n