Part 1: The Problem
I’ve been working with CI and CD for some years now, in the context of building and delivering enterprise software for the financial and telco industries. During these years i've seen a "less than ideal" pattern repeat itself.The pattern starts with identifying one amazing individual in the company.
This guy, lets call him Jack (Jack Of All Trades, no similarities intended) is your average dedicated developer, but with a touch of flaming dedication and curiosity.
Jack is also an Ops guy, mainly due to his great curiosity and sometimes he is actually able to teach your Ops team a few things.
So Jack is a DevOps guy.
On a lucky day Jack spends as much as 25% of his time developing business software and the other 75% automating/maintaining the company’s CI/CD pipelines.Think of Jack as the transition between you and the DevOps people we will all become somewhere in promising the future.
You might already have spotted the problem here. Jack is a great guy, he’s a DevOps guy, so he takes responsibility for and understands the whole software delivery process, which is a good thing.
But he is most likely the only one at your company. Jack knows about:
- environments
- servers
- firewalls
- DNS
- load balancer
- cloud stuff
- CI/CD
- compilers
- frameworks
- strange quirks of Microsoft, etc.
The Ops people love him because finally there's a developer who understands some of the pains they have to work with on the MS platform.
The other developers finds his tasks uninteresting or scary, but they are extremely dependent on his skills and work, every single day.
So to be clear, the problem here is you have Jack, one guy, configuring, maintaining and advancing the whole CI/CD process, more or less alone.
What do you do when this guy quits?Despite all Jack's best efforts to automate everything he touches, he is now the ONLY one who really knows how the automation and CI/CD processes works in the company. Everybody loves his work as it makes they're day less painful and most have good suggestions for things that could be even better. However, no one but Jack possesses the knowledge to understand the complexity of the systems/pipelines/toolchains he maintains.
Originally all Jack really wanted, was more time to develop and less time building/deploying, since these are both very error prone processes. Now he finds that his CI/CD work has taken over his main function, which was developing business software.
So now he quits.Leaving behind a couple of 100 scripts, 100 repos, different build setups (your company has some 100+ different apps), different technologies such as bash scripts, powershell, cmd, TeamCity, CruiseControl, integration services, etc, all configured by him and probably a good many servers with dedicated users (with specific rights of course) and specific setups which needs to be maintained (new version of Visual Studio, TeamCity/CruiseControl, service packs, databases, etc).
Maybe he even did a few writings in the company wiki on how the setup works, but come on, who in their right mind can document a complex setup like that. The wise guys scripts it, but there's bound to be gaps as the domains are simply to big for just one guy, Jack.
Boom!Now you wish that every developer had better understanding of what DevOps is about and were capable of taking responsibility for the entire software delivery process from idea, to implementation, to build, to deploy, to test, to staging, to production, to monitoring, etc.
But they don't, at least not yet, and you're screwed, or at best one of the other developers is screwed and needs to immediately start understand the work that the guy did over the last 3 years.
Ultimately this could lead to frustration and more developers quit.You are left at dangerous crossroad of just keeping everything running vs. investing heavily in the CI/CD setup (again) which will cost you money right here, right now.
Seriously, people does this all the time...
Next: Part 2: The Domains
No comments:
Post a Comment