The book @intensivedata has got to be the most information-packed one I've read. Summary of all major DB storage techniques, explained in 35 pages in the book. Thread.
1. "Plain old" key-value store in a textfile
2. Indexing a key-value store (e.g. a CSV) with hash indexes (1/6)
1. "Plain old" key-value store in a textfile
2. Indexing a key-value store (e.g. a CSV) with hash indexes (1/6)
3. Segmenting files as they grow via compaction
4. SSTables - sorted string tables (sentence of key-value pairs sorted by keys).
5. LSM-trees (Log-Structured Merge Tree)
6. B-trees: standard storage in many relational/non-relational databases (2/6)
4. SSTables - sorted string tables (sentence of key-value pairs sorted by keys).
5. LSM-trees (Log-Structured Merge Tree)
6. B-trees: standard storage in many relational/non-relational databases (2/6)
6.1 B-tree reliability & optimization (write-ahead-logs, latches, copy-on-write)
6.2 B-trees vs LSM trees
7. Other indexing approaches: clustered indexes, covering indexes, fuzzy indexing, in-memory DBs (3/6)
6.2 B-trees vs LSM trees
7. Other indexing approaches: clustered indexes, covering indexes, fuzzy indexing, in-memory DBs (3/6)
8. OLTP (online transaction processing) & OLAP (online analytics processing) (3/X)
8.1 Data warehousing and ETL (extract, transform, load)
8.2 OLTP vs data warehousing
8.3 Star schemas / dimensional modeling (4/6)
8.1 Data warehousing and ETL (extract, transform, load)
8.2 OLTP vs data warehousing
8.3 Star schemas / dimensional modeling (4/6)
9. Column-oriented storage
9.1. Compressing columns: bitmap encoding, vectorized processing
9.2 Importance of sort order. C-Store.
9.2 Writing to column storage: B-trees & LSM trees (hello again!)
10. Data cubes & materialized views (5/6)
9.1. Compressing columns: bitmap encoding, vectorized processing
9.2 Importance of sort order. C-Store.
9.2 Writing to column storage: B-trees & LSM trees (hello again!)
10. Data cubes & materialized views (5/6)
This information (and explanations) are about 1/20th of the book.
I thought I knew data systems at decent detail. I keep learning new things, and having "aha" moments on concepts I used, but didn't know in detail, or their tradeoffs.
Kudos to @martinkl for the great work. (6/6)
I thought I knew data systems at decent detail. I keep learning new things, and having "aha" moments on concepts I used, but didn't know in detail, or their tradeoffs.
Kudos to @martinkl for the great work. (6/6)
• • •
Missing some Tweet in this thread? You can try to
force a refresh
