Memories Of On-Call Support
Blogging is like the opposite of sex. You miss a few days, and suddenly you don't feel like doing it anymore. Anyway.
As you may know, in my old job, I did Tier 2 on-call support for six years. I thought I'd never forget this harrowing experience. But I'm starting to. So I'm going to write down at least the parts that I remember.
First, I will hook you guys with a story.
Story
One time, four of us went on a business trip to Toronto. Because the company was strapped for cash, it was required that each two people share a rental car. My teammate Scott and I came in car #1. Anna and Matt were in car #2.
Now, I didn't like Matt, but I didn't know why. Matt was weird. Matt smelled, though this is rather a minor offense in IT. Matt went around telling on people behind their backs, then returned saying: "I just told so-and-so on you". Matt was a nice guy, yet he creeped me out, for reasons yet unknown.
Let me also tell you about Anna. Anna was a very nice woman in her sixties. Sleepless nights and job stress were even harder on her than they were on us.
So, I wasn't really surprised when, on her way to work from our hotel the first morning, Anna made a turn in front of a MAC truck and it hit Anna's rental. It was in a parking lot, so the truck merely made a dent in Anna's passenger door. Matt, however, was in the passenger seat when it happened, and he was not happy.
That evening, Scott, Anna and I went to Chinatown for dinner. If you haven't been to Toronto's Chinatown, go visit there now. It was very different and exciting. We walked around for a while, then ducked into a restaurant that didn't have English menus. Anna and Scott translated the menu for me. The food was delicious, nothing like the greasy, MSG-laden Chinese-American food we had at home. It was very cool.
Over dinner, Anna said:
"Guess what Matt told me today. He said: 'when we're going back home, I won't be riding with you. I'm going to ride with Scott.'"
She paused.
"And then he said: 'Goldie will be riding with you'".
Scott and I just stared. I had been measured, weighed, and found worthless. Matt put a price on my life, and it turned out to be lower than his own. Of course, I didn't for a minute believe that riding with Anna would get one killed, but Matt did. And he did not hesitate to hide behind my back from this perceived death.
"I'll ride with ya," I agreed at once, but Scott didn't like the idea.
"I don't want to ride with Matt. He scares me."
In the end, I went home with Scott, Anna drove alone, and Matt hitched a ride from someone who hadn't heard that monologue of his.
What am I trying to tell you here?
Being on call is a lot like riding in a rental car with an aging, tired woman who is operating on three hours of sleep.
To be efficient at on-call support, you should know your stuff. You should be able to think fast and make split-second decisions. You should be able, on occasion, to hack your data and your file system without breaking everything.
But there is more. You have to work with people.
You get your call from a Tier 1 guy, also known as helpdesk. You talk to the user, who placed the call. You may need to page your teammates, or people on other teams like your database admins and your network admins, to help you fix the problem.
More often than not, it's three in the morning, and the user says they cannot have the machines down for longer than ten minutes. The pressure is incredible. You are all on the front line, together.
This is when you stick for each other and cover each other's back. Or you can be like Matt, but then, you won't last.
More on-call stories to follow, as soon as I dig them out of my memory.
As you may know, in my old job, I did Tier 2 on-call support for six years. I thought I'd never forget this harrowing experience. But I'm starting to. So I'm going to write down at least the parts that I remember.
First, I will hook you guys with a story.
Story
One time, four of us went on a business trip to Toronto. Because the company was strapped for cash, it was required that each two people share a rental car. My teammate Scott and I came in car #1. Anna and Matt were in car #2.
Now, I didn't like Matt, but I didn't know why. Matt was weird. Matt smelled, though this is rather a minor offense in IT. Matt went around telling on people behind their backs, then returned saying: "I just told so-and-so on you". Matt was a nice guy, yet he creeped me out, for reasons yet unknown.
Let me also tell you about Anna. Anna was a very nice woman in her sixties. Sleepless nights and job stress were even harder on her than they were on us.
So, I wasn't really surprised when, on her way to work from our hotel the first morning, Anna made a turn in front of a MAC truck and it hit Anna's rental. It was in a parking lot, so the truck merely made a dent in Anna's passenger door. Matt, however, was in the passenger seat when it happened, and he was not happy.
That evening, Scott, Anna and I went to Chinatown for dinner. If you haven't been to Toronto's Chinatown, go visit there now. It was very different and exciting. We walked around for a while, then ducked into a restaurant that didn't have English menus. Anna and Scott translated the menu for me. The food was delicious, nothing like the greasy, MSG-laden Chinese-American food we had at home. It was very cool.
Over dinner, Anna said:
"Guess what Matt told me today. He said: 'when we're going back home, I won't be riding with you. I'm going to ride with Scott.'"
She paused.
"And then he said: 'Goldie will be riding with you'".
Scott and I just stared. I had been measured, weighed, and found worthless. Matt put a price on my life, and it turned out to be lower than his own. Of course, I didn't for a minute believe that riding with Anna would get one killed, but Matt did. And he did not hesitate to hide behind my back from this perceived death.
"I'll ride with ya," I agreed at once, but Scott didn't like the idea.
"I don't want to ride with Matt. He scares me."
In the end, I went home with Scott, Anna drove alone, and Matt hitched a ride from someone who hadn't heard that monologue of his.
What am I trying to tell you here?
Being on call is a lot like riding in a rental car with an aging, tired woman who is operating on three hours of sleep.
To be efficient at on-call support, you should know your stuff. You should be able to think fast and make split-second decisions. You should be able, on occasion, to hack your data and your file system without breaking everything.
But there is more. You have to work with people.
You get your call from a Tier 1 guy, also known as helpdesk. You talk to the user, who placed the call. You may need to page your teammates, or people on other teams like your database admins and your network admins, to help you fix the problem.
More often than not, it's three in the morning, and the user says they cannot have the machines down for longer than ten minutes. The pressure is incredible. You are all on the front line, together.
This is when you stick for each other and cover each other's back. Or you can be like Matt, but then, you won't last.
More on-call stories to follow, as soon as I dig them out of my memory.
