If you were lured onto this article by the title, it did its job. What you will discover here is how a Junior Developer became part of a team that practices Devops by preparing to take on on-call duties.

I joined Shutl almost two years ago as my first Software Developer job. A major part of our culture is that Everybody is Devops! Since I was still a relatively inexperienced developer at the time, I came up with a plan on how to transfer to Devops like the rest of our team. I made it one of my yearly goals to join the on-call rotation since it seemed like the perfect opportunity to get into Devops.

I’ll start with some words on our on-call system. The duty is divided into three levels.

• We have the Level 1 person, available 24 hours a day, responsible for answering technical support requests as well as resolving monitoring alerts on our platform.
• If Level 1 needs help or they don’t respond to an alert after 15 minutes it is escalated to Level 2.
• In the rare cases where this is not enough, you escalate to Level 3 which is everybody in the team.

Level 1 is quite a big responsibility. If a server goes down in the middle of the night, you are responsible to get it back up and running. The question now was, how would I build up the confidence and knowledge to deal with this?

What beast are we working with?

The first step was to use my mentoring sessions with one of our Senior Developers to get a clearer picture of what our infrastructure looks like. Which servers are there, how do they interact, where do the databases sit and how is it all monitored? Where can I look for answers when I get a production alert that says the server is on fire? Where is the bucket of water or wait, maybe the fire extinguisher? I didn’t know. It was time to ask a lot of questions and take notes.
I also informed the rest of the team about my challenge and asked them to prepare for me stalking them whenever an alert came in or a support request had to be taken care of. I basically tried to grab every bit of information I could get on the Devops work we do.

How do you tame it?

To my advantage it turned out we had a gap analysis for when we initially transformed to Devops. The list included abilities like logging into certain services, parse logs, reboot servers, applications, databases as well as familiarity with particular services. I have gathered some questions that can help you to come up with a gap analysis for yourself or your team at the end of this article.
So I added myself to the gap analysis, went through each task and checked if I was able to perform them. If I wasn’t I would do whatever it took to do it, which was usually asking colleagues.
To remember all the commands, locations and setups I simply wrote them in my notes app. Usually I added some search words to the entries so I could quickly find the desired information.

Mastery.

After about three months or so of intensely doing all of the above, an opportunity arose and I took on-call duty three months earlier than planned. I still remember my first alert. My bicycle was broken at the time and I was walking to work when the first alert came in. Basically the worst time possible so I was a little scared when my phone rang. To my surprise I looked at the error and knew what it was. I knew how to handle it!
When I was woken up in the middle of the night for the first time I had to fiddle around a bit before knowing what had happened but did not need to escalate to Level 2. In fact I’ve never had to escalate so far. My preparation had paid off. I’ve done plenty of on-call duty by now and am going to have my first Level 2 support this month.

Lessons.

Is there anything I would do different or that I wouldn’t recommend? I was probably making a bigger thing of it than it was. I remember when I talked to my manager about doing on-call, he said that it’s mostly about confidence. In retrospect I have to agree, there was no reason to be scared. However doing the preparation helped me to get to a point where I felt comfortable to take the next step. Even more than that, I felt quite competent at it right away.

You can’t spell fun without Devops!

On a side note – it even turns out that I really like working on our infrastructure. I would’ve never suspected it but maintaining and improving the infrastructure is really fun. My recommendation would be for anybody to try it out and don’t be too scared. If you are, take the measures above and dive in anyway!