What is data lineage and why is it important when building ML systems?
From @chipro’s new book, Designing Machine Learning Systems:
1/5
From @chipro’s new book, Designing Machine Learning Systems:
1/5
Data lineage is the process of keeping track of the origin of your data and tracking versions of it over time.
This is important if your data changes and you want to keep track of which model was trained using which data and how the model performance is affected.
2/5
This is important if your data changes and you want to keep track of which model was trained using which data and how the model performance is affected.
2/5
You could track data versions yourself but it'll likely be as error prone as "model_latest_latest_actual_latest_2021.pth" is when tracking models.
@weights_biases Artifacts is one way you can track the data you used to train your models with a few lines of code.
3/5
@weights_biases Artifacts is one way you can track the data you used to train your models with a few lines of code.
3/5
Here’s @charles_irl showing you how to use Artifacts:
and here’s some guides and docs if you’d like to explore it for yourself.
docs.wandb.ai/guides/artifac…
docs.wandb.ai/guides/artifac…
• • •
Missing some Tweet in this thread? You can try to
force a refresh
