Aurimas Griciลซnas Profile picture
Dec 23 โ€ข 9 tweets โ€ข 4 min read
If I could only choose 5 books to read in 2023 as an aspiring Data Engineer these would be them in a specific order:

Read on in the Thread ๐Ÿ‘‡

--------

Follow me and hit ๐Ÿ”” to ๐—Ÿ๐—ฒ๐˜ƒ๐—ฒ๐—น ๐—จ๐—ฝ in #MLOps, #MachineLearning, #DataEngineering, #DataScience and overall #Data space!
1๏ธโƒฃ โ€๐—™๐˜‚๐—ป๐—ฑ๐—ฎ๐—บ๐—ฒ๐—ป๐˜๐—ฎ๐—น๐˜€ ๐—ผ๐—ณ ๐——๐—ฎ๐˜๐—ฎ ๐—˜๐—ป๐—ด๐—ถ๐—ป๐—ฒ๐—ฒ๐—ฟ๐—ถ๐—ป๐—ดโ€ - A book that I wish I had 5 years ago. After reading it you will understand the entire Data Engineering workflow. It will prepare you for further deep dives.

๐Ÿ‘‡
2๏ธโƒฃ โ€๐—”๐—ฐ๐—ฐ๐—ฒ๐—น๐—ฒ๐—ฟ๐—ฎ๐˜๐—ฒโ€ - Data Engineers should follow the same practices that Software Engineers do and more. After reading this book you will understand DevOps practices in and out.

๐Ÿ‘‡
3๏ธโƒฃ โ€œ๐——๐—ฒ๐˜€๐—ถ๐—ด๐—ป๐—ถ๐—ป๐—ด ๐——๐—ฎ๐˜๐—ฎ-๐—œ๐—ป๐˜๐—ฒ๐—ป๐˜€๐—ถ๐˜ƒ๐—ฒ ๐—”๐—ฝ๐—ฝ๐—น๐—ถ๐—ฐ๐—ฎ๐˜๐—ถ๐—ผ๐—ป๐˜€โ€ - Delve deeper into Data Engineering Fundamentals. After reading the book you will understand Storage Formats, Distributed Technologies, Distributed Consensus algorithms and more.

๐Ÿ‘‡
4๏ธโƒฃ โ€œ๐—ง๐—ฒ๐—ฎ๐—บ ๐—ง๐—ผ๐—ฝ๐—ผ๐—น๐—ผ๐—ด๐—ถ๐—ฒ๐˜€โ€ - Sometimes you might get confused about why a certain communication pattern is in place in the company you work for. After reading this book you will learn the Team Topologies model of organizational structure for fast flow.

๐Ÿ‘‡
5๏ธโƒฃ โ€œ๐——๐—ฎ๐˜๐—ฎ ๐— ๐—ฒ๐˜€๐—ตโ€ - Data Mesh has become an extremely popular buzzword in recent years. By reading this book you will understand the intent by the author of the term herself. Donโ€™t be the one to throw around the term without understanding its meaning deeply.

๐Ÿ‘‡
[๐—ก๐—ข๐—ง๐—˜]: All of the books above are talking about Fundamental concepts, even if you read all of them and decide that Data Engineering is not for you - you will be able to reuse the knowledge in any other Tech Role.

๐Ÿ‘‡
[๐—”๐——๐——๐—œ๐—ง๐—œ๐—ข๐—ก๐—”๐—Ÿ ๐—ก๐—ข๐—ง๐—˜]: I did not include any Fundamental Classics in the list as these can be picked up after you have already established yourself in the role.

๐Ÿ‘‡
I will help you Level Up in #MLOps, #MachineLearning, #DataEngineering, #DataScience and overall #Data space.

๐—™๐—ผ๐—น๐—น๐—ผ๐˜„ ๐—บ๐—ฒ and hit ๐Ÿ””

Join a growing community of 3000+ Data Enthusiasts by subscribing to my ๐—ก๐—ฒ๐˜„๐˜€๐—น๐—ฒ๐˜๐˜๐—ฒ๐—ฟ: swirlai.substack.com

โ€ข โ€ข โ€ข

Missing some Tweet in this thread? You can try to force a refresh
ใ€€

Keep Current with Aurimas Griciลซnas

Aurimas Griciลซnas Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @Aurimas_Gr

Dec 22
What is a ๐—™๐—ฒ๐—ฎ๐˜๐˜‚๐—ฟ๐—ฒ ๐—ฆ๐˜๐—ผ๐—ฟ๐—ฒ and why is it such an important element in ๐— ๐—Ÿ๐—ข๐—ฝ๐˜€ ๐—ฆ๐˜๐—ฎ๐—ฐ๐—ธ?

Find out in the Thread ๐Ÿ‘‡

--------

๐—™๐—ผ๐—น๐—น๐—ผ๐˜„ ๐—บ๐—ฒ and hit ๐Ÿ”” to ๐—Ÿ๐—ฒ๐˜ƒ๐—ฒ๐—น ๐—จ๐—ฝ in #MLOps, #MachineLearning, #DataEngineering, #DataScience and overall #Data space! Image
Feature Store System sits between Data Engineering and Machine Learning Pipelines and it solves the following issues:

โžก๏ธ Eliminates Training/Serving skew by syncing Batch and Online Serving Storages (5)

๐Ÿ‘‡
โžก๏ธ Enables Feature Sharing and Discoverability through the Metadata Layer - you define the Feature Transformations once, enable discoverability through the Feature Catalog and then serve Feature Sets for training and inference purposes trough unified interface (4๏ธ,3).

๐Ÿ‘‡
Read 15 tweets
Dec 21
Do you know what CDC(Change Data Capture) is and that there are multiple ways to implement it?

Find out in the Thread ๐Ÿ‘‡

--------

๐—™๐—ผ๐—น๐—น๐—ผ๐˜„ ๐—บ๐—ฒ and hit ๐Ÿ”” to ๐—Ÿ๐—ฒ๐˜ƒ๐—ฒ๐—น ๐—จ๐—ฝ in #MLOps, #MachineLearning, #DataEngineering, #DataScience and overall #Data space! Image
๐—–๐—ต๐—ฎ๐—ป๐—ด๐—ฒ ๐——๐—ฎ๐˜๐—ฎ ๐—–๐—ฎ๐—ฝ๐˜๐˜‚๐—ฟ๐—ฒ is a software process used to replicate actions performed against Operational Databases for use in downstream applications.

๐—ง๐—ต๐—ฒ๐—ฟ๐—ฒ ๐—ฎ๐—ฟ๐—ฒ ๐˜€๐—ฒ๐˜ƒ๐—ฒ๐—ฟ๐—ฎ๐—น ๐˜‚๐˜€๐—ฒ ๐—ฐ๐—ฎ๐˜€๐—ฒ๐˜€ ๐—ณ๐—ผ๐—ฟ CDC. ๐—ง๐˜„๐—ผ ๐—ผ๐—ณ ๐˜๐—ต๐—ฒ ๐—บ๐—ฎ๐—ถ๐—ป ๐—ผ๐—ป๐—ฒ๐˜€:

๐Ÿ‘‡
โžก๏ธ ๐——๐—ฎ๐˜๐—ฎ๐—ฏ๐—ฎ๐˜€๐—ฒ ๐—ฅ๐—ฒ๐—ฝ๐—น๐—ถ๐—ฐ๐—ฎ๐˜๐—ถ๐—ผ๐—ป (refer to 3๏ธโƒฃ in the Diagram).

๐Ÿ‘‰ CDC can be used for moving transactions performed against Source Database to a Target DB. If each transaction is replicated - it is possible to retain all ACID guarantees when performing replication.

๐Ÿ‘‡
Read 15 tweets
Dec 21
What does good Model Tracking System look like?

Find out in the Thread ๐Ÿ‘‡

--------

๐—™๐—ผ๐—น๐—น๐—ผ๐˜„ ๐—บ๐—ฒ and hit ๐Ÿ”” to ๐—Ÿ๐—ฒ๐˜ƒ๐—ฒ๐—น ๐—จ๐—ฝ in #MLOps, #MachineLearning, #DataEngineering, #DataScience and overall #Data space! Image
It should be composed of two integrated parts: Experiment Tracking System and a Model Registry.

From where you track ML Pipeline metadata will depend on MLOps maturity in your company.

If you are at the beginning of the ML journey you might be:

๐Ÿ‘‡
1๏ธโƒฃ Training and Serving your Models from experimentation environment - you run ML Pipelines inside of your Notebook and do that manually at each retraining.

If you are beyond Notebooks you will be running ML Pipelines from CI/CD Pipelines and on Orchestrator triggers.

๐Ÿ‘‡
Read 14 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(