Thread by @a2d2

This post is getting a lot of surprise/derision from SV VC and tech.

That's a problem. Not only is this normal, it highlights most of the community don't understand #enterprise SW and infrastructure.

So let's dive in. Why do Citi and other banks have huge engineering teams? 🧵

https://twitter.com/MarceloPLima/status/1526562846787633152

First, most of the big banks are actually a ton of smaller banks or finserv firms (e.g.: a trading desk, a retail bank, an investment bank, etc.) strung together.

Some of these firms have roles that are unsurprisingly complicated and require SW engineering.

Citi is no exception. They host one of the larger algo/quant trading desks, which means they need a lot of SW engineers to be writing fast code to execute trades and perform analysis.

In fact, Citi's MQA group which serves that desk is pretty fire.

citibank.com/emeaservicecen…

Unsurprising groups aside, most engineers at a big bank are focused on building/maintaining infrastructure.

Modern banking is technologically hard. Executing a trade on Schwab from your phone (or even transferring $5 from one acct to another) safely/reliable requires a lot.

Big banks have historically been some of the largest, most technologically progressive enterprise IT orgs in the world.

eg: Capital One was one of the first of the Global 2000 to move mission critical workloads to the cloud (AWS) and embrace multicloud.

hashicorp.com/resources/how-…

Managing and deploying such infrastructure - infrastructure that frankly can't get hacked and can never go down lest the wrath of the Treasury, SEC, and much more be incurred - requires herculean effort and automation.

This is where the bulk of most banks' engineering orgs live

The amount of engineers focused on building banking infrastructure highlights a different approach to SW engineering that most people (esp. non SW engineers) see if they only know FANGs.

We don't really have a term for this in tech, so I'm going to borrow something from the military: Tooth to Tail Ratio (T3R).

The T3R in is basically the ratio of combat personnel to support personnel.

en.wikipedia.org/wiki/Tooth-to-…

Hopefully your job as an engineer at Google or Goldman Sachs doesn't involve 40 mile hikes or shooting a rifle. So instead of being something like mechanized infantry to motorized support, the T3R here refers to the ratio of:

software engineers to engineering support

What's engineering support? Well that's stuff like:
- DevOps
- IT
- Security
- Whole mess of other folks who ensure that novel code is not only written well, it's maintained in a high state in perpetuity

In most SV/West Coast tech there's a high T3R ratio. There's very few explicit software engineers, and lots of folks supporting them.

Example: SDEs at MSFT are supported by a whole host of SDETs and more, SW engineers at Meta are vastly outnumbered by support roles, etc.

Why? It's simple: for most orgs there's a higher penchant for "move fast and break things."

Certainly this doesn't hold true for everyone. VMware and Cisco aren't exactly 2010-era Facebook. But in general, novel engineering is a revenue-generating component of the company.

This is not the case on Wall Street, where the code written needs to be just as/more performant and secure than its SV counterparts but is NOT revenue generating for the business.

As a result the T3R is lower, and potentially closer to 1:1.

Culture has a huge role to play here too. The division between "who's an engineer" and "who is QA" is far less important at a big bank than it is at Facebook or Apple.

Engineering culture/org structure at these orgs tends to be more flat and mixed with eng support.

So yes, big banks have huge teams of software engineers. But given that this includes eng support roles, saying Citi has 30k engs while Facebook has 20k isn't that surprising.

What is surprising: why people (esp. tech or VC people) in Silicon Valley think this is fucking weird

If you work in enterprise IT/infratech in the US, you have worked with/in the engineering org of a big bank.

It's unavoidable. The big banks influence everything from reference architectures to cloud deployments to the roadmaps of major OSS projects like Kubernetes and MongoDB.

This is why most serious infrastructure investors haunt stuff like Goldman Sachs' Tech and Infrastructure Conf and JPMC's TMC conf and awards show.

Seeing what the banks use is a good indicator of where the rest of enterprise tech is going.

goldmansachs.com/insights/pages…

Banks don't have time to fuck around. They make serious infrastructure choices where the cost of fucking up is the entire economy crashing.

So if they're willing to trust their mission critical data to some new database, some cloud provider, or some new OSS software it's big.

Why I think this lack of understanding from the SV tech/vc folks is particularly interesting is one word: blockchain.

There's a **strong** correlation between people I've noticed mocking big eng teams at big banks and folks pushing blockchain as the future of banking/defi.

Full stop: banks exist to make money. If blockchain was supposedly the next better iteration of the internet, banks would vet and then embrace it just like they've embraced the last 3 decades of infra tech.

In reality: they can't. Blockchains suck.

Most of the big banks have been researching things like Solana or Ethereum for years. But they haven't moved any mission critical systems over because they're not performant, scalable, or secure enough.

finextra.com/blogposting/18…

I think it's super telling that most of the crowd mocking big banks for having large software engineering teams are simultaneously the ones saying that big bank fintech sucks and blockchain defi is the only answer....

Basically it highlights a key point: the overwhelming number of people pushing blockchain defi don't understand infrastructure tech (especially infrastructure fintech) at scale.

That is some serious 🚩🚩🚩🚩🚩🚩🚩

TL;DR:

Big banks have big engineering teams. But when you factor for their flatter SW eng infrastructure and their need to move fast while not breaking everything, those numbers are comparable to most tech companies.

Also this is super normal. And if anyone says it isn't, run.

PS: Vibes of the GOAT MongoDB skit

"I will never have to witness the collapse of the world economy because NoSQL radicals talked financial institutions into abandoning perfectly good datastores because they didn't support distributed fucking map/reduce."