Thread by @thdxr

It's easy to make the same old jokes about NoSQL but there's a much more important lesson to learn of how it tragically became a punchline in the first place.

Story time...

https://twitter.com/supabase/status/1394688110160596994

Back in the early 2010s a PAAS company called 10gen decided to pivot. They were going to narrow their focus and develop a new kind of database that they named MongoDB

At this point most software engineers only had experience building with SQL databases. The cloud was new and exciting with a whole lot of companies being founded. People were open to a paradigm shift.

10gen needed a strategy to position MongoDB at the forefront of this new wave

They decided to deploy a massive marketing campaign evangelizing NoSQL and MongoDB. Suddenly everyone was wearing a MongoDB shirt and attending 10gen sponsored classes and hackathons.

It felt like the next big thing.

The critical mistake they made that echoes to today was focusing the pitch on MongoDB being easier because it was schemaless.

No pesky schema to define before writing data and no columns to add when things change. Maximum flexibility for your fast moving startup. Also web scale

This lead to engineers expecting an easy experience and conflating not having a schema with not having to spend time data modeling. Data was thrown into MongoDB without thinking too hard about it.

And at first, it definitely felt easier...

...but then reality hit.

NoSQL's flexibility actually requires more attention to data and access patterns - not less.

Additionally engineers were not thinking in NoSQL, they were replicating SQL patterns in MongoDB and expecting them to work well.

As these systems grew they created some of the most painful technical problems I have seen in my career.

Everywhere I looked teams were dealing with poor data integrity, deep debt in their data models, and system outages. Not to mention some data loss bugs in Mongo itself

Across the industry, expensive migrations were initiated to get these systems off of Mongo and back to traditional SQL databases. A few of those efforts did not fail.

This era of pain led to Mongo becoming a meme and poisoned the well for NoSQL at large.

The educational failure by 10gen lead to misconceptions that are still repeated today:

1. NoSQL lacks schemas and data integrity
2. NoSQL is for niche non-relational use cases
3. NoSQL is unwisely optimizing for scale most companies won't see

Frankly these misconceptions are coming from those without experience deploying a well designed NoSQL database like @dynamodb. It can handle nearly every use case that a sql database can.

The most that can be said is some things are easier while some things are harder.

Additionally, teams are choosing @dynamodb not because Postgres can't scale to their needs (it definitely usually can) but because of what the scaling experience looks like.

SQL databases require continuous scaling work feature by feature. Tables get too big, queries get too slow, need to manage connections, etc. All manageable

In NoSQL, if you make the mental shift to learn the patterns, features are scalable on day 1 and won't need work on day 100

These are two subjective styles of building comes down to what feels right to you. There is no good vs evil.

If this has rekindled some curiosity around NoSQL I highly recommend watching talks by @houlihan_rick and checking out The DyanmoDB Book by @alexbdebrie

The referenced tweet was deleted so attaching here. It was good natured joking, definitely didn't think it needed to be deleted