Thread by @QuinnyPig

No, @awscloud. *THIS* is how you launch Amazon Detective to GA:

I stared at the stack of bills on one side of my desk, the half empty bottle of bourbon on the other. No matter which I picked, I'd be drowning.

https://twitter.com/jeffbarr/status/1245022107362152453

The door to my office swung open, and a DevOp, DBE walked in. That was the last instant of normalcy I'd see for the next few days.

I'm Billie the Platypus, Amazon Detective.

The DevOp was borderline hysterical, but I wasn't laughing.

"Billie, it's my--
(Poll)

"I came home late last night and my bill was just astronomical!" they sobbed. Their tears burst forth like a Kinesis Data Firehouse without the ominous billing implications.

"I don't know what I did wrong!"

"There there, let's take a look at your bill" I said consolingly.

A quick glance showed me that their single biggest spend item was: (poll)

I was staring at my phone number, except it was the amount due for data transfer on the DevOp's bill. I didn't like it; if I'd wanted calls I'd have taken out a personals ad.

I turned to Cost Explorer. Unfortunate naming--it always felt like Cost Spelunker instead. I started trying to figure out what it was that was driving the costs, but "Cost Allocation Tags" were as new a phrase to this DevOp as "business hours only."

I didn't have a choice. I turned on VPC Flow Logs, mindful of the impact that saving these logs to S3 would have on the bill itself.

I dialed up my old pal `awk` down at the precinct; he still owed me a favor.

"It basically looks like everything is talking to everything; I can't make heads nor tales out of this nonsense. You've gotta go talk to the folks who built this."

You question the witnesses! Do you start with:
(poll)

The systems architect was chowing down on her spaghetti like Hacker News devouring a new Javascript framework. I struggled to tell the difference between her dinner and her architecture.

"Word on the street is that your application keeps cross-talking in some ridiculous way we can't untangle."

She looked up at me.

"Do I come down to where you work and slap the Excel file out of your idiot duckbill? No? Then go bother someone else!"

"Look," I said, straining for patience like an autoscaling instance straining to scale before crashing. "Something you built is causing massive transfer between AZs; we can't figure out what it is."

She starts to sneer a response, but I see something flash in her eyes.

Fear.

"I can't tell you!" she pleaded. "They'll kill me!"

"Who?! Who wants you silenced?"

(poll)

"Balderdash!" I snapped. "Oracle is the freaking devil, but they don't cause cross-AZ data transfer spikes no matter how badly you misconfigure it."

"That's where you're wrong" replied the systems architect.

And suddenly the spaghetti on her plate wound up on the whiteboard.

"With Oracle RDS, replication traffic in a multi-AZ environment is of course free--but there's only one primary database endpoint. All writes have to traverse the AZ boundary to get there."

"Sure, sure," I said, my head spinning. "But your application is read bound."

"I... didn't configure different database endpoints when we were designing things."

She hung her head in shame, as only the namer of Systems Manager Session Manager could.