๐Ÿฆ‰DVC Profile picture
Dec 12 โ€ข 5 tweets โ€ข 3 min read
๐Ÿฆ‰ DVC is designed to improve upon past solutions to make the life of ML teams easier. Hereโ€™s how it differentiates from other related technologies:

๐Ÿฑ Git
๐Ÿ“ฆ Git-LFS (Large File Storage)
๐Ÿ—„ Git-annex

@Iterativeai @DVCorg
#mlOps #data #git #lfs

๐Ÿงต[1/5]
๐Ÿฑ Git vs DVC

DVC builds upon Git by the concept of data files โ€“ large files that should not be stored in a Git repository, but need to be tracked and versioned.

It leverages Git's features to enable managing different versions of data, data pipelines, and experiments.

๐Ÿงต[2/5]
๐Ÿ“ฆ Git-LFS vs DVC

DVC does not require special servers like Git-LFS demands. Any cloud storage like S3, Google Cloud Storage, or even an SSH server can be used as a remote storage.

No additional databases, servers, or infrastructure are required.

๐Ÿงต[3/5]
๐Ÿ—„ Git-annex vs DVC

Git-annex is a datafile-centric system whereas DVC focuses on providing a workflow for machine learning and reproducible experiments.

Moreover, DVC can use โ€œreflinks or hardlinksโ€ instead of โ€œsymlinksโ€ to improve performance.

๐Ÿงต[4/5]
โค๏ธ Thanks for the time

Check out dvc.org/doc/user-guideโ€ฆ for a more detailed comparison.

๐Ÿงต[5/5]

โ€ข โ€ข โ€ข

Missing some Tweet in this thread? You can try to force a refresh
ใ€€

Keep Current with ๐Ÿฆ‰DVC

๐Ÿฆ‰DVC Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @DVCorg

Dec 14
โœจ Visualize your pipelines as beautiful graphs of connected stages using โ€œdvc dagโ€ command.

Here's different types of graphs you can explore ๐Ÿ—บ๏ธ

โœ… Full DAG
โžก๏ธ DAG of outputs
๐Ÿงœโ€โ™€๏ธ Mermaid flowchart
โšช๏ธ DOT format
๐Ÿ“ฃ Verbose

@Iterativeai @DVCorg
#data #pipeline #dag

๐Ÿงต[1/7]
โœ… Full DAG

โ€œdvc dag --fullโ€ shows the full DAG that the target stage belongs to, instead of showing only its ancestors.

๐Ÿงต[2/7]
โžก๏ธ DAG of outputs

โ€œdvc dag --outsโ€ shows a DAG of chained dependencies and outputs instead of the stages themselves.

๐Ÿงต[3/7]
Read 7 tweets
Dec 3
๐Ÿฆ‰ Hereโ€™s a quick brief from Iterative Community๐Ÿ‘‡

๐Ÿช GitHub Universe
๐Ÿค Hacktoberfest Contributions
๐Ÿ“น Closure Session 2 event
โ™พ๏ธ CML at NeurIPS
๐Ÿ‡ฌ๐Ÿ‡ง MLOps Summit London
๐Ÿ‡บ๐Ÿ‡ธ ODSC West

@Iterativeai @DVCorg
#mlOps #data #pipeline #dag

๐Ÿงต[1/8]
๐Ÿช Dmitry at GitHub Universe

Dmitry Petrov (@FullStackML) spoke at #GitHubUniverse on โ€œML with Git: experiment tracking in Codespacesโ€ on 8th November.

He presented how to use the DVC extension for VS Code and Codespaces to streamline ML experimentation process.

๐Ÿงต[2/8]
๐Ÿค Hacktoberfest Contributions

Huge thanks to @francesco086, vvssttkk, and @DeepyamanDatta for their contributions!

We are sending personalized shirts to them.

Special thanks to @mertbozkirr for leading the Hacktoberfest charge!

๐Ÿงต[3/8]
Read 8 tweets
Nov 16
๐Ÿฆ‰Wanna get started with DVC on your own machine? Here's how you can get started๐Ÿ‘‡

๐ŸŽ MacOS
๐ŸชŸ Windows
๐Ÿ–ฅ๏ธ Linux
๐Ÿ Python Package
๐Ÿ‘จ๐Ÿปโ€๐Ÿ’ป VS Code Extension

@Iterativeai @DVCorg
#mac #windows #linux #code

๐Ÿงต [1/7]
๐ŸŽ MacOS

The recommended way to install dvc on Mac is using @MacHomebrew with command โ€œbrew install dvcโ€ โ˜•๏ธ

๐Ÿงต [2/7]
๐ŸชŸ Windows

If you are on Windows, we recommend installing Chocolatey. Then using the command โ€œchoco install dvcโ€ ๐Ÿซ

๐Ÿงต [3/7]
Read 7 tweets
Nov 7
๐Ÿฆ‰ Hereโ€™s some exciting news brief this month from Iterative Community ๐Ÿ‘‡

๐Ÿค October Meetup
๐ŸŽ‰ Hacktoberfest
๐Ÿฒ DVC-Hydra Integration
โ™พ๏ธ CML in Research
๐Ÿ“„ DVC Wikipedia Page

@Iterativeai @DVCorg
#mlOps #data #pipeline #dag

๐Ÿงต [1/7]
๐Ÿค October Meetup

@NadiaIIT presented her team's research on Collaboration Challenges in Building ML-Enabled Systems in the October Meetup. If you missed it, visit .

๐Ÿงต [2/7]
๐ŸŽ‰ Hacktoberfest

Contributions are pouring in, as Hacktoberfest proceeds. Find out all the issues with #hacktoberfest tags in our repos.

๐Ÿงต [3/7]
Read 7 tweets
Sep 2
For all the devs out there willing to contribute to DVC, here is a quick guide to contributing to iterative/dvc repo
๐Ÿž Open a new issue
๐Ÿ’ป Set up a dev environment
๐Ÿด Fork iterative/dvc
๐Ÿงช Add tests and run them locally
โฌ†๏ธ Submit a pull request
@iterativeai

๐Ÿงต [1/7]
๐Ÿž Open a new issue

Open a new issue in the issue tracker, whether it be a bug report or a feature request. ๐Ÿ‘‡๐Ÿฝ
github.com/iterative/dvc/โ€ฆ

๐Ÿงต[2/7]
๐Ÿด Fork iterative/dvc

Fork iterative/dvc and then clone it into your local computer to start contributing.

๐Ÿงต[3/7]
Read 8 tweets
Aug 23
๐Ÿฆ‰ Here are some of the cool commands you can try out right now in the DVC command line interface!

๐Ÿ’ป dvc dag
๐ŸงŠ dvc freeze
๐Ÿ“ฆ dvc move
๐Ÿ“Š dvc metrics show
๐Ÿงน dvc gc

๐Ÿงต [1/7]
๐Ÿ’ป dvc dag

๐š๐šŸ๐šŒ ๐š๐šŠ๐š is very helpful in quickly checking out the stages of a pipeline up to the target stage in a simple visual representation. If the target is omitted, it will show the full project DAG.

๐Ÿงต [2/7]
๐ŸงŠ dvc freeze

๐š๐šŸ๐šŒ ๐š๐š›๐šŽ๐šŽ๐šฃ๐šŽ helps us to freeze stages until ๐š๐šŸ๐šŒ ๐šž๐š—๐š๐š›๐šŽ๐šŽ๐šฃ๐šŽ is used on them. Frozen stages are never executed by ๐š๐šŸ๐šŒ ๐š›๐šŽ๐š™๐š›๐š˜.

๐Ÿงต [3/7]
Read 7 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(