,
23 tweets,
3 min read
Read on Twitter
"Our AWS bill is through the roof, we need to do something about it!" Answering this question is my entire consulting business; I have some thoughts on the cultural bits.
"Every engineer needs to think about cost in every part of the design!" Well, maybe. I'm not saying spend money on AWS like it's going out of style, but it's often not practical to have engineers focus on it.
AWS bills are vast and deep; spending time and energy getting every engineer to understand the nuances is almost never worth pursuing.
The basics are fine. "This instance costs 5¢ an hour, this one costs $26 an hour" style lessons make sense, as does "we pay for data transfer out to the internet." But RI discussions? ALB billing dimensions? Nobody got time for that!
Further, "your dev environment costs us $400 a month, we need you to spend time getting that under $100" is a generally dumb request to an engineer whose loaded cost is $250K.
(To that end, I like a pattern of "spin up anything you want in a dev account, but it automatically gets whacked after 14 days." Engineers can be creative all they want, and they're unlikely to do too much damage.)
"You need to go through a manual approval process to spin up resources." That was the case with on-prem hardware, and led directly to the rise of cloud / "shadow IT."
Remember, you're only "a corporate credit card" away from someone spinning up a new AWS account without your onerous controls. Aim to build guardrails, not impermeable gates, or you'll hate what happens next.
Companies are always more excited about speeding time to market than they are about saving money. It's in their nature, and it's the right answer. Save them money, meet their CFO. Speed delivery, meet their entire board.
And now, some terrible patterns based upon a wide variety of experience with Enterprise Cloud.
"We're going to build a 'Cloud Center of Excellence'" is a statement only uttered by people powerful enough to avoid being punched in the face for how condescending it presents.
(Implying your existing staff are mediocre morons isn't a page from any business book I ever read, but who am I to judge? I don't have an MBA, just a heart.)
"We're going to build a single dashboard to view the cost for our cloud spend enterprise-wide!" The most depressing thing you will ever see if you build such a thing is the usage stats after the first month. Dirt roads up mountains see more traffic.
"We can build costing tools without talking to finance" goes about as well as finance attempting to build webapps without talking to engineering. "That doesn't seem hard" is about the dumbest thing you can say about an entire department's field of work.
"We don't count X since that's a different cost center" is a formalized way of saying "not my problem," and is indicative of a fractured culture.
"We let every division do their own thing and then we analyze in aggregate." Great in theory, but you've got a serious planning / RI purchasing problem if you go this route.
"Our new cost optimization tool fixes--" I'm going to stop you there. This is a culture problem, and people don't have APIs. If you can fix that, why are you wasting your time on cloud costing?
"Our consultancy has a methodology that--" ...requires 500 billable consultants for 18 months to descend upon a company? My hat's off to you if you can sell it, but I have my doubts.
"AWS themselves will come in and suggest--" ...a number of things that are absolutely spot on, but "we're optimized because the vendor we pay large sums of money to says we are" is surprisingly unconvincing.
"What if we just ignore the entire problem for a while?" Join the club; this is my only serious competitor in this space, for better or worse. "That's a problem best saved for tomorrow" is compelling.
Any meaningful answer has to address all of these points and more. There's no silver bullet. I don't pretend to have one of those--just a few ideas.
To wit, get the stakeholders on board early. Ensure this is a problem the business actually cares about at a strategic level. Lay groundwork for future efforts as early as you can. Get the big easy wins, maybe let the complex 10% slide for a time.
And understand that nobody has done it perfectly yet. Sometimes, "good enough" is closer than you think it is.
