What is the difference between an engineering manager and a data science manager? It's a question I find myself ruminating over almost constantly. There's tons of good thinking and writing about eng management out there, but I don't find that it always translates to the DS world.
Granted, "Data Science" is still a broad cover term, so depending on your flavor of data science, the leap is shorter. One way of segmenting the DS world that relate to is the type A vs. type B DS, where A is for "analysis" and B is for "building"). medium.com/@rchang/my-two…
My suspicion is that most eng management advice can apply to Type B DS management pretty readily. Their work is more purely engineering. But what does that mean for Type A DS management? What makes it different and thus hard to apply the same advice to? I have some thoughts.
To start, DS is not essential like engineering is. Yes, a product will be worse without analytics on how it's performing, but it can still ship. Everyone loves to talk about being data driven, but many people still operate on intuition and feel.
It's easy to find ways to a feature launch a success, especially if basic metrics are neutral to positive. Who cares if the differences are statistically significant? We shipped fast! We paved the way for future innovation! And it is fun to use and looks great!
A DS can do tons of analysis to support a product area or launch, but that work could be largely ignored. As a DS manager, you are stuck wondering--was that because the DS failed to understand their customer's needs, or did their customer not use that work for some other reason?
This happens to engineers too, but it's not as common. Not everyone sees a DS's time as precious. It's not that they don't appreciate it it, but rather that it's a value-add. If DS work doesn't pan out, oh well! We didn't REALLY need it.
And for Type A DS managers, this makes measuring the contributions and progress of your team extremely difficult. The value your team adds is often realized by influencing someone else to act. It's latent, a second order effect. It takes time to see.
A consequence of this is that Type A DS work (and management) tends to be more relationship-based. This is a good path forward, since folks are more likely to trust someone they have a good relationship with, but it also means that Type A DS work is less modular.
Because DS is not essential on its own, the title of DS doesn't automatically convey authority. Customers often trust the individual to advise them, not the office, and this makes rotating people between projects on Type A DS teams difficult to do productively.
Siloing is a big problem for Type A DS, where one person has gone deep on an area and no one else can fill in. As a DS manager, who's supposed to keep the plates spinning regardless of staffing, this makes it stressful when your team is sick, on vacation, etc.
It also makes it harder for DSes to collaborate. A team of two horses can pull more weight together than two horses working alone. Engineering teams, which are inherently more modular, function like teams of horses, and Type A DS teams function like individual horses.
As a type A DS manager, it makes you wonder if you're really managing a team in the same way that an engineering manager is managing a team. There's less of a shared identity. It's hard to say what you're doing as a TEAM, rather than as individuals.
Now, this all might sound like a lament, but I'm not necessarily saying that it's a bad thing. Type A DS teams are viewed as important (maybe even essential) partners much of the time, and an embedded, verticalized team structure usually generates good outcomes for a company.
It's not bad, but it is different. There are some days where I what the hell my job as a DS manager is even supposed to be, but thinking through the differences helps bring me some clarity.
