Re:Develop.io History of DevOps talk

I recently gave a talk at Re:Develop.io about the history of DevOps, and this page is where you can find the slide deck and other relevant resources.

Re:develop.io DevOps Evolution Slide deck

The Theory of constraints

Agile 2008 presentation by Patrick DuBois

10+Deploys per day, Dev and Ops at Flickr – Youtube

10+deploys a day at Flickr – slide deck

DORA 

Nicole Forsgren research paper

2018 State of DevOps Report

My “Three Ways” slide deck

Safety culture

Amazon DevOps Book list

Deming’s 14 points of management

Lean management

The Toyota Way

The Agile Manifesto

Beyond the phoenix project (audio)

Netflix – simian army

My Continuous Lifecycle London talk about compliance in DevOps and the cloud

And while you’re here, please check out and maybe even donate to Danyadara, where we’re trying to reverse desertification in Andalucia!

 

 

 

Going solo

This is the view from where I’m living now.

I’ll explain. A couple of months ago, my partner was offered a once-in-a-lifetime opportunity to live and work in Andalusia, doing digital marketing for an organisation that is part retreat centre, part permaculture farm, and part yoga teacher training school. With brexit looming and faced with such a great opportunity to do something very different, we both rapidly left our jobs in the UK, packed up what stuff we didn’t get rid of or put in storage, and moved.

All of which means I’m now working with the organisations here (suryalila.comdanyadara.com and froglotusyogainternational.com ) alongside developing my own consultancy business and working as CTO for ydentity.com (so new we still have lorem ipsum text). I will freely admit that coming off the salary drug is a tough task, but having the freedom to do my own thing, develop my skills and work the hours that I want is proving very satisfying so far.

For the moment, I’m getting involved in a really wide range of work, from project management, tech consultancy, AWS engineering, to digital marketing and analytics, security consultancy and more. The reason for me getting stuck into such a wide range of tasks is so I can really work on evaluating what the most suitable area is for me to focus on in the future, both from a perspective of what I’m good at and enjoy doing, and also what I find there is most demand for in the marketplace.

If you would like to work with me or you’d like to discuss an opportunity for us to collaborate, drop me a line through LinkedIn, email me at tom@tomgeraghty.co.uk or pop in to see me in Andalusia, if you can get past the goats on the highway.

GDPR, and how I spent a month chasing my data.

sherlock holmes basil rathbone

In May 2018, I received a letter from a local firm of solicitors, Roythornes, advertising a property investment event. I hadn’t heard of them and I was damn sure I hadn’t given them my permission write to me at home. They were wide of the mark to say the least- I’m an unlikely potential property tycoon, unless we’re playing Monopoly. Even then, I’m a long shot.

It was a quiet week at work so given the recent implementation of GDPR and the fact that I really don’t like junk mail, I thought I’d give the new Data Subject Access Request (DSAR) process a whirl.

I couldn’t find contact at Roythornes to send a DSAR to but, helpfully, GDPR places no restriction on the medium someone can use to make a request, so at around 9am that day I filled in their online contact form, despite my concerns that it would get picked up by a clueless admin assistant. I requested a copy of the data they hold on me, the source of that data and the evidence of my opt-in for direct mail. I also asked that they delete the data they hold on me and send no further marketing material.

At 1:42pm, I had a from “Norma” of Roythornes (not joking, sorry Norma), asking for a copy of the letter and stating that she couldn’t find me in their database. So far, so good…

At 1:55pm, I received an automated recall email from Norma.

A few hours later, another email arrived, this time from the firm’s “compliance partner”, stating that they had acquired my personal data in a mailing list they purchased from Lloyd James Media, on the 1st of May 2018, and that my letter was sent out on the 21st May 2018. She stressed that the purchase of the list, and the sending of the letter itself was prior to the GDPR implementation date of 25th May 2018, and therefore legal.

Solicitors abiding by the letter of the law, not the spirit of the law? Imagine that.

Dancing around technicalities notwithstanding, Roythornes did confirm that my data had been deleted and I wouldn’t be hearing from them again. Phase 1 complete, but who exactly were Lloyd James Media, and how did my data fall into their hands?

For Phase 2 of my quest, a quick google told me that Lloyd James Media “is a multi-channel data agency focusing on the intelligent use of data to optimise customer acquisition and retention.” I can only assume this translates as “we make and sell mailing lists”. So, off went my DSAR email to their sales email address, because yet again, there was no contact information available for non-sales enquiries, let alone DSARs.

Andrew of the compliance team at Lloyd James Media only took 24 hours to get back to me. He confirmed that they sold my personal data to Roythornes for a postal campaign. They had acquired my data from another firm, “My Offers”, and the consent for postal marketing was obtained by them, apparently. Helpfully,  Andrew suggested I get in touch with the “compliance team” at My Offers. Evidently, this is a team of one, someone called Saydan, whose email address Andrew provided. I reminded Andrew to remove my data from their system and headed off to continue the hunt for the true source of my data, feeling like a geek version of Bear Grylls tracking an elusive squirrel. Phase 3 had begun.

My Offers are “Europe’s leading online direct marketing company with a database of 22.2million registered users.” I fired my third DSAR email off to Saydan later that day. One week later, I’d heard nothing. According to GDPR, there is no need for an immediate response, as long as the DSAR is executed within a month, but the silence was unnerving. Was Saydan trying to ghost me? I found their Facebook page and sent a message to whichever poor soul supports their social channels. For good measure, I also dredged LinkedIn for their employees, emailed Ivan, one of their directors, and in true annoying millennial style, tweeted at them. The only response was from their Facebook team, who reiterated that I should email the enigmatic Saydan, then also went quiet on me.

Over the next few weeks, because nothing seemed to be happening, I pinged their facebook team a courtesy message once each week with a gentle reminder of the impending deadline for a response. Part of me was relishing the prospect of not getting a reply, and I began googling, “What to do if someone doesn’t respond to a DSAR.” I was way too invested in this.

Then, exactly one month to the day since the original request, an email from Ivan, the My Offers director arrived in my inbox. Ivan’s email was straight to the point and only had a few spelling mistakes. Attached was a password protected CSV file containing all the information I’d requested. The password was sent in a separate email. So far, so good (though, yet again, I had to remind him to remove my data from their systems).

The CSV file was interesting. And by interesting, I mean in the way that hearing the creak of a stair when you’re in bed, and there’s nobody else in the house is interesting. The data contained my full name, birth date, gender, home address, email address, phone number, data subject acquisition date and source (Facebook), as well as a list of businesses that my data had been shared with in the past year. The list totalled around 60, including Lloyd James Media, various PPI and no-win no-fee firms, and more. That explains all the marketing calls over the past year then.

This CSV file was the smoking gun. However, the trigger was evidently pulled by my own fair hand. At some point, possibly whilst drunk, bored at work, or both, I’d clicked on a campaign offering me a beard trimmer. I still don’t have a beard trimmer (I do have a beard), so I presumably I didn’t pursue this purchase but in getting only that far, I inadvertently provided My Offers with access to my personal data, and consent for direct marketing. Sounding eerily familiar, I wondered if my voting choices in the last election were my own making.

So, just over a month after I sent my first DSAR to a local firm, what have I learned from this?

Firstly, GDPR actually works. Not only was the DSAR process easy to do, it was free (for me), and two out of three firms responded within 24 hours. Presumably GDPR is also helping to reduce unwanted junk mail; after all, Roythornes as good as admitted that they wouldn’t have posted the initial letter to me after the GDPR implementation date.

Secondly, once your data is out there, it gets around. It only takes one “online direct marketing company” to get hold of it, and your personal information will spread faster than nationalism in Europe.

Finally, don’t be dumb on facebook (like me). We know about Cambridge Analytica of course, but they’re not the only guys trying to harvest information and profit from it. Resist the lure of data-harvesting surveys and competitions, even when drunk.

Compliance in the Cloud

Continuous Lifecycle London slide deck:

https://docs.google.com/presentation/d/103csVh12upw-VfC7JoGBDElThYqeuWzg1SripJA_phs/edit?usp=sharing 

AWS compliance information:

https://aws.amazon.com/compliance/

ENISA cloud computing security doc:

https://www.enisa.europa.eu/publications/cloud-computing-risk-assessment

AWS security products:

https://aws.amazon.com/products/security/

Life without SSH:

https://www.youtube.com/watch?v=fEuN5LkXfZk

https://www.cisecurity.org/

www.devsecops.org

https://www.sans.org/reading-room/whitepapers/analyst/cloud-security-compliance-primer-34910

Contino Compliance as Code pdf:

https://www.contino.io/files/Compliance-as-Code-March-18-v13.pdf

Curve is making using money whilst traveling much easier.

Most people who know me are somewhat aware that I travel quite a lot. I love visiting new places, and I really enjoy being on the move. One thing that often bugs me, however, is how awkward using money in different countries can be.

Getting local cash at a decent rate can be a pain. Using an ATM abroad incurs fees from your bank, and you don’t necessarily get good rates. Going to a currency exchange can be a hassle too. I usually carry some dollar notes on me anyway, because they’re accepted almost everywhere.

Using your debit or credit card abroad incurs yet more fees, with unpredictable and usually poor rates.

This is where my new Curve card comes in. A Curve card essentially acts as a proxy between the world and your existing cards. You can add multiple credit and debit cards to it, and select which one to use for each payment (or even change it afterwards!). Using your Curve card doesn’t incur any currency conversion fees, and you receive a better rate than most high street currency exchanges.

Cleverly, all the transactions on your Curve appear as purchases on the “source” cards, which means you can also take cash out of an ATM using your Curve card, and choose to source the cash from any of your cards, including credit cards (without incurring the cash advance fees that you normally would), debit cards and pre-paid cards.

So what does this mean for travelers? Firstly, using your Curve card saves you money abroad, both when using it to purchase goods and services, or getting cash from an ATM.

Secondly, it’s far more secure. Just take your Curve out with you, and leave your main cards in the safe at the hotel, or simply keep them elsewhere on your person. If your Curve card gets stolen or lost, simply open the app and deactivate it. Even if it somehow got cloned by some unscrupulous bartender or waiter, you’ll receive instant alerts of any payments, so you’d know the instant it was used, which means you could instantly disable the card and request a refund.

Thirdly, you get real-time feedback and notifications, making budgeting far easier. You can tag your purchases, add notes, and scan receipts to add to purchases. This means you can keep track of your spending, check how much you’re spending on travel, food, shopping, etc, and even use it to form the basis of your expenses claim if travelling for business.

In my opinion, Curve is still lacking a few features though. You can’t yet add it to Apple Pay, which would streamline payments even further. You can’t add a single debit or credit card to multiple Curve accounts either. The reason for this is that when travelling as a group or a couple, you could all add some money into a shared pre-paid card (e.g. Monzo), and then add that to your individual Curves. This would solve all those split-payment problems when you’re paying for supplies or travel costs for the whole group. If there’s any “debate” around a certain item being charged to the group card, it’s easy to “go back in time” (as Curve put it), and charge it to a different (i.e. personal) card.

I’m pretty sure the group payment challenge is solvable, either with existing tech, or by someone shortly bringing out fintech joint accounts. Either way, using Curve has definitely made my travelling life a lot easier, and I’m excited to see what they do next.

Shameless promo: sign up for Curve here with code NPWZA and you’ll get £5, and so will I.

Embrace the silence

When i am silent, i have thunder hidden inside me.

Over the years in my career so far, I’ve found that in some (many) situations, my speaking style in meetings doesn’t always “work” effectively.

Some background: when I was young, I was diagnosed with dyspraxia, and had trouble forming sentences and speaking properly. I had speech therapy until the age of around 8 years old. The word “hammer” was a particular challenge for me, apparently. I don’t know why. I can say hammer really well now. Try me.

As a result of this (or maybe it’s just coincidental), I often pause before speaking, particularly when in a larger group, or in a situation where what I say really matters. It’s partly to formulate the content, the idea, the concept, but also to establish the “how” of it; i.e. how to structure the sentences, what phrasing to use, and how the statement is to be delivered.

Now, this pause is useful for everyone. It allows for a more cogent, relevant and useful discussion.

But, people seem to feel the need to fill this audible space. Whether that’s a result of a discomfort with silence, or a desire to be the one speaking and presenting their ideas instead of me, I don’t know. I suspect both, in different scenarios. I don’t really care though, as it gives me more time to build my response anyway.

I guess I could be concerned that some might interpret a pause as a weakness, as some kind of hesitation because I don’t understand the subject matter, but I choose to ignore that concern, and focus instead on being me, and how I function best.

I wonder if we should all try to pause a little more. Think about what we say, how we say it, and how we deliver it. Imagine if meetings were 30% less talk, but with 50% better quality contributions as a result.

Embrace the silence. Embrace your own, and allow others to use theirs.

The OSI model for the cloud

While I was putting together a talk for an introduction to AWS, I was considering how to structure it and thought about the “layers” of cloud technology. I realised that the more time I spend talking about “cloud” technology and how to best exploit it, manage it, develop with it and build business operations using it, the more some of our traditional terminologies and models don’t apply in the same way.

Take the OSI model, for example:

 

When we’re managing our own datacentres, servers, SANS, switches and firewalls, we need to understand this. We need to know what we’re doing at each layer, who’s responsible for physical connectivity, who manages layer three routing and control, and who has access to the upper layers. We use the terms “layer 3” to describe IP-based routing or “layer 7” to describe functions interacting at a software level, and crucially, we all know what each other means when we use these terms.

With virtualisation, we began to abstract layers 3 and above (layer 2? Maybe?) into software defined networks, but we were still in control of the underlying layers, just a little less “concerned” about them.

Now, with cloud tech such as AWS and Azure, even this doesn’t apply any longer. We have different layers of control and access, and it’s not always helpful to try to use the OSI model terms.

We pay AWS or Azure, or someone else, to manage the dull stuff – the cables, the internet connections, power, cooling, disks, redundancy, and physical security. Everything we see is abstract, virtual, and exists only as code. However, we still possess layers of control and management. We may create multiple AWS accounts to separate environments from each other, we’ll create different VPCs for different applications, multiple subnets for different functions, and instances, services, storage units and more. Then we might hand off access to these to developers and testers, to deploy and test applications.

The point is that it seems we don’t yet have a common language, similar to the OSI model, for cloud architecture. Below is a first stab at what this might be. It’s almost certainly wrong, and certainly can be improved.

Let’s start with layer 1 – the physical infrastructure. This is entirely in the hands of the cloud provider such as AWS. Much of the time, we don’t even know where this is, let alone have any visibility of what it looks like or how it works. This is analagous to layer 1 of the OSI model too, but more complex. It’s the physical machines, cabling, cooling, power and utilities present in the various datacentres used by the cloud providers.

Layer 2 is the hypervisor. The software that allows the underlying hardware to be utilised – this is the abstraction between the true hardware and the virtualised “hardware” that we see. AWS uses Xen, Azure uses a modified Hyper-V, and others use KVM. Again, we don’t have access to this layer, but a GUI or CLI layered on top. For those of us who started our IT careers managing physical machines, then adopted virtualisation, we’ll be familiar with how layer 2 allowed us to create and modify servers far quicker and easier than ever before.

Layer 3 is where we get our hands dirty. The Software Defined Data Centre(SDDC). From here, we create our cloud accounts and start building stuff. This is accessed via a web GUI, command line tools, APIs or other platforms and integrations. This is essentially a management layer, not a workload layer, in that it allows us to govern our resources, control access, manage costs, security, scale, redundancy and availability. It is here that “infrastructure as code” becomes a reality.

Layer 4. The Native Service (such as S3, Lambda, or RDS) or machine instance (such as EC2) layer. This is where we create actual workloads, store data, process information and make things happen. At this level, we could create an instance inside a VPC, set up the security groups and NACLs, and provide access to a developer or administrator via RDP, SSH, or other protocol. At this layer, humans that require access don’t need Layer 3 (SDDC) access in order to do their job. In many ways, this is the actual IaaS (Infrastructure as a Service) layer.

Layer 5. I’m not convinced this is all that different to layer 4, but it’s useful to distinguish it for the purpose of defining *who* has access. This layer is analogous to layer 7 of the OSI, that is, it’s end-user-facing, such as the front end of a web application, the interactions taking place on a mobile app, or the connectivity to IoT devices. Potentially, this is also analogous to SaaS (Software as a Service), if you consider it from the user’s perspective. Layer 5 applications exist as a function of the full stack underneath it – the physical resources in datacentres, the hypervisor, the management layer, virtual machines and services, and the code that runs on or interacts with the services.

Whether something like an OSI model for cloud becomes adopted or not, we’re beginning to transition into a new realm of terminology, and the old definitions no longer apply.

I hope you found this useful, and I’d love to hear your feedback and improvements on this model. Take a look at ISO/IEC 17788 if you’d like to read more about cloud computing terms and definitions.

Finally, if you’d like me to speak and present at your event or your business, or provide consultation and advice, please get in touch. 

tom@ec2-34-242-84-40.eu-west-1.compute.amazonaws.com

@tomgeraghty

https://www.linkedin.com/in/geraghtytom/

The Three Ways

The three ways are one of the underlying principles of what some people call DevOps (and what other people call “doing stuff right”). Read on for a description of each approach, which when combined, will help you drive performance improvements, higher quality services, and reduce operational costs.

1. Systems thinking.

Systems thinking involves taking into account the entire flow of a system. This means that when you’re establishing requirements or designing improvements to a structure, process, or function, you don’t focus on a single silo, department, or element. This principle is reflected in the “Toyota way” and in the excellent book “The Goal” by Eliyahu M. Goldratt and Jeff Cox. By utilising systems thinking, you should never pass a defect downstream, or increase the speed of a non-bottleneck function. In order to properly utilise this principle, you need to seek to achieve a profound understanding of the complete system.

It is also necessary to avoid 100% utilisation of any role in a process; in fact it’s important to bring utilisation below 80% in order to keep wait times acceptable. See the graph below.

2. Amplification of feedback loops.

Any (good) process has feedback loops – loops that allow corrections to be made, improvements to be identified and implemented, and those improvements to be measured, checked and re-iterated. For example, in a busy restaurant kitchen, delivering meatballs and pasta, if the guy making the tomato sauce has added too much salt, it’ll be picked up by someone tasting the dish before it gets taken away by the waiter, but by then the dish is ruined. Maybe it should be picked up by the chef making the meatballs, before it’s added to the pasta? Maybe it should be picked up at hand-off between the two chefs? How about checking it before it even leaves the tomato sauce-guy’s station? By shortening the feedback loop, mistakes are found faster, rectified easier, and the impact on the whole system – and the product – is lower.

3. Continuous Improvement.

A culture of continual experimentation, improvement, taking risks and learning from failure will trump a culture of tradition and safety every time. It is only by mastering skills and taking ownership of mistakes that we can take those risks without incurring costly failures.

Repetition and practice is the key to mastery, and by considering every process as an evolutionary stage rather than a defined method, it is possible to continuously improve and adapt to even very dramatic change.

It is important to allocate time to improvement, which could be a function of the 20% “idle” time of resources if you’ve properly managed the utilisation of a role. Without allocating time to actually focus on improvement, inefficiencies and flaws will continue and amplify well beyond the “impact” of reducing utilisation of said resource.

By utilising the three ways as above, by introducing faults into systems to increase resilience, and by fostering a culture that rewards risk taking while owning mistakes, you’ll drive higher quality outcomes, higher performance, lower costs and lower stress!

For my presentation on the Three Ways, click here. Feel free to use, adapt, and feed back to me 🙂

The IT hardware lifecycle explained

In our service desk, where a device is reported as being slow, broken, malfunctioning, or for any other reason the user wishes to have it replaced, we first determine the age of the device. If the device is outside of the standard hardware lifecycle, it will be replaced, because the maintenance and TCO (Total Cost of Ownership) of devices older than the standard lifecycle is more costly than the replacement costs. If it’s within the life cycle, it will either be repaired, or we’ll evaluate if the user actually needs a more capable machine to carry out their role.

TCO vs age:

hardware total cost of ownership

 

 

 

 

 

 

 

In very general, cumulative terms, the TCO of a device increases over time. When the annual TCO exceeds the cost of a new device, it is overdue to be replaced.

TCO includes:

  • Increased support resource costs.
  • Cost of replacement components.
  • Loss of productivity of the employee using the device.
  • Added complexity from maintaining an older (less uniform) fleet.
  • Security concerns due to older devices.
  • Power usage.
  • Staff morale.

An example of a standard hardware lifecycle is:

  • Laptops – 3 years
  • Desktops – 4 years
  • Monitors – 5 years
  • Servers and network hardware – 5 years
  • Mobile phones – 2 years
  • Printers – 3 years (but using a managed service lease contract)

This is standard across the IT industry, although many science/tech firms may have dramatically shorter lifecycles due to the higher workloads that devices are expected to handle.

The above lifecycle means that we will maintain a life cycle of replacing 33% of our laptops each year, 25% of our desktops, 20% of our monitors, and so on. This is the staggered approach; some firms employ the forklift approach which means replacing (e.g) the entire laptop fleet once every three years. This impacts cash flow harder, and can be more disruptive during the change, but has the advantage of delivering a perfectly uniform fleet of hardware each time. Many contact centre-style businesses employ this approach.

The only time I’ve modified this life cycle is when the company I’ve worked for has gone through cash flow difficulties, and we’ve extended the replacement period with a “promise” to pull it back in-line when cash allows. Of course, the promise is rarely fulfilled…

Q. How do you know that you could improve as a leader?

Q. How do you know that you could improve as a leader?

A. You’re still breathing.

Check out Jenifer Richmond and  find out more about her excellent executive coaching services. I’ve been working with Jenifer for some time now, and she has helped me hugely in identifying my career goals and, through questioning and challenging, helped me to make difficult decisions and changes of direction where necessary. I really can’t recommend her enough.