neptune.ai Profile picture
Dec 14 5 tweets 2 min read
Greensteam subscribed to the idea of doing #MLOps at a reasonable scale.

Seeing the quickly growing number of customers (= ML experiments), they decided to build their MLOps stack from 0 and solve all core problems around it.

Here are some of the issues → solutions:
- 1000s of Jupyter notebooks → git
- Managing dependencies and reproducibility → @Docker
- Dealing with unit tests (in some parts of the model code) that don’t test → running smoke tests
- Different linter versions showing different results locally and in Jenkins → code checks moved into Docker
- Finding parts of the code that unit tests didn’t cover → mypy
- Testing models for multiple datasets of different clients in different scenarios → @argoproj
- Monitoring the results of models, trained on multiple datasets, with different parameters and metrics (and comparing all those model versions) → neptune.ai
- Training a separate model for each vessel type with constantly growing time-series datasets → @FastAPI
Full MLOps tool stack: buff.ly/3UnBr0i

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with neptune.ai

neptune.ai Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @neptune_ai

Dec 14
Models aren’t intelligent enough to adjust to a changing world unless they’re constantly retrained & updated

You need to monitor them, detect data drift & update the data

To detect data drift, do distribution tests by measuring distribution changes using these distance metrics:
> Basic statistical metrics you could use to test drift between historical and current features are:
- mean/average value,
- standard deviation,
- minimum and maximum values comparison,
- and also correlation.
> For continuous features, you can use divergence and distance tests such as:
- Kullback–Leibler divergence,
- Kolmogorov-Smirnov statistics (widely used),
- Population Stability Index (PSI),
- Hellinger distance,
- and so on.
Read 8 tweets
Dec 13
@LukawskiKacper is joining us next week on #MLOps Live to share his experience and advise on implementing vector search – AMA. Image
Kacper has almost 15 years of experience in data engineering, ML and software design. As the founder of @AiEmbassy, he has been also actively taking part in AI discussions, especially on similarity learning, vector search, and solving social issues by applying ML methods.
Jump on a live with us to ask him anything about:
- Using vector search vs neural search to build search engines
- Evaluating and comparing vector search engines (both open-source and paid solutions)
- Optimizing the speed and effectiveness of vector search apps
- And more
Read 4 tweets
Dec 12
3 steps to be more productive doing #ML at a reasonable scale.

1/ Identify all critical problems your team is dealing with
2/ Look for the best solution available
3/ Apply + evaluate

Example with @instadeep
Step 1/ Challenges faced by the BioAI team while building DeepChain (platform for protein design):
> Experiment logs all over the place

With logs scattered across documents & files, experiments become difficult to manage. Engineers & researchers would take a long time looking for the results, rather than doing the actual research.
–––
Read 8 tweets
Dec 7
Great @pytorchlightnin + Hydra (clean and scalable) template to kickstart any deep learning project by @ukashxukash (and some other contributors).

Main ideas behind it:
-Predefined structure: clean & scalable so that work can easily be extended
-Rapid Experimentation: thanks to hydra command line superpowrs
-Little Boilerplate: thanks to automating pipelines with config instantiation
-Main Configs: specify default training configuration
-Experiment Configs: override chosen hyperparameters
-Workflow: comes down to 4 simple steps
-Experiment Tracking: @TensorBoard, @weights_biases, neptune.ai, @Cometml, @MLflow, @CSVLogger
-Logs: all logs are stored in a dynamically generated folder structure
-& more
Read 4 tweets
Dec 6
#MLOps standard industry best practices” don’t apply to most #ML teams’ reality.

Why?

Those who write and share best practices are doing ML at a hyper scale.

Those who read and re-share them are doing ML at a reasonable scale.
Companies like Google, Netflix, Uber, and Airbnb are doing an awesome job for the community by sharing their blogs, white papers, and open-sourcing their tools.

But whatever they do, it is shaped (and biased) by THEIR MLOps problems.

Most companies don’t have their problems.
They would love to have their problems, but they don’t.

They operate on a smaller scale & have different (& other) challenges.

And they are the biggest part of the ML industry.

They want to know what’s the best way to do MLOps at their scale, with their resources & limitations
Read 6 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(