Several people asked this. It's a good question! I will share my thoughts, but I am certainly not religious about this. You should do what works for you and your teams and their workflows. 📈🥂☺️
https://twitter.com/annabunches/status/1491191325018009604
1) "assuming you have good deduplication"... can a pretty big assumption. You never want to be in a situation where you spend more time tweaking dupe, retry, re-alert thresholds than fixing the problem.
2) having to remember to go futz with a ticket after every little thing feels like a lot of busywork. You've already committed some code, mentioned it in #ops or wherever, and now you have to go paste all that information into a task (or many tasks) too?
3) who is the consumer of all of the tasks? When people want to subscribe to real time updates of an issue, I assume they are in #ops or a channel where people are debugging in real time. You need this anyway, for eng to collaborate and do handoffs and retros.
4) if a problem is persistent or out of scope for an on call engineer, then yes, absolutely create a task and start tracking it. But tasks will be higher quality if a human creates them as needed than if a robot creates them and humans have to constantly prune and batch them.
5) I hate machine generated tasks in general. Worst case, they lead to people using them as a proxy for productivity; best case they're just a lot of spam to deal with.
IME, people get pretty quickly desensitized to tasks from bots. I'd prefer people to be sensitive to tasks.
IME, people get pretty quickly desensitized to tasks from bots. I'd prefer people to be sensitive to tasks.
6) Keeping feedback loops tight is absolutely core to being an effective engineer. Every time you have to switch back and forth to yet another tool or interface is a chance to get distracted.
And as liz says,
And as liz says,
https://twitter.com/lizthegrey/status/1491191718091247619
7) worst of all is the pathological case, where people can't keep up with the spamminess of tasks (or email) and convince themselves they need "AIOps" to tell them which alerts they should look at. 🥺☠️
I'm not saying the "warnings go to slack" solution is elegant, or particularly satisfying to anyone, but it's the most lightweight, integrated, collaborative solution I've seen yet.
Most attempted solutions make things radically more complicated, and therefore much worse.
Most attempted solutions make things radically more complicated, and therefore much worse.
Like it or not, this is the unholy grey area of complex systems. Is it up, is it down? Is it degrading, or a harbinger of doom, or just a blip? How important is it?
On call engineers make these calls many times a day. This is where your sixth sense gets tested and honed.
On call engineers make these calls many times a day. This is where your sixth sense gets tested and honed.
There is only so much discipline you can bring to it by very definition -- this is the land of unknowns and entropy. You can't "solve" it any more than you can mow your lawn once and be done with it.
Tasks seem to give people a certain illusion of orderliness and control. 🙃
Tasks seem to give people a certain illusion of orderliness and control. 🙃
Anyway, in the end it doesn't really matter if you use tasks or slack. It matters that you keep the alert load manageable and actionable, share knowledge, and see that on call engineers are given time to fix things instead of deliver product work.
https://twitter.com/brian_scanlan/status/1491201575909720065?s=21
• • •
Missing some Tweet in this thread? You can try to
force a refresh
