Tweet

@replicateai

@weights_biases

@MLFlow

More from @full_stack_dl

Full Stack Deep Learning

@full_stack_dl

16 Feb

@huggingface

1/🛠Tooling Tuesday🛠

Let's talk about an open source absolute unit: 🤗

@huggingface provides open-source implementations of most Transformer architectures, in both PyTorch and TF.

Better yet, they have many pre-trained NLP models and datasets.

And recently, an inference API.

2/ The 🤗 transformers library implements many architectures: BERT, GPT, XLNet, etc.

It works for both PyTorch and Tensorflow, and provides three main modules, all nicely interoperable:

- Data configuration and tokenization
- Model architecture
- Training

3/ All of the transformers library code is compatible with pre-trained model weights available on Hugging Face Hub.

This enables rapid progress: no matter what your NLP task/language is, there is probably a pre-trained model for you to start fine-tuning with.

Read 5 tweets

Full Stack Deep Learning

@full_stack_dl

9 Feb

@ApacheTVM

🛠️ Tooling Tuesday 🛠️

This week: @ApacheTVM (tvm.apache.org)

TVM describes themselves as a an "end to end machine learning compiler framework for cpus, gpus, and accelerators".

Let's talk about what that means and why it might be useful for you:

Under the hood, deep learning frameworks are built on "kernel operator" libraries like cuDNN. These are the primitives that help run your model fast on a GPU or other accelerator.

The problem is: these aren’t open source (not extensible) and only work on specific platforms.

So what happens if you the platform you want to deploy to isn't supported or performance is bad?

That's where TVM comes in. It has importers for all the major frameworks and tutorials to compile optimized versions of your models for common CPUs and GPUs.

Read 8 tweets

Full Stack Deep Learning

@full_stack_dl

26 Jan

🛠️Tooling Tuesday🛠️

Good training data is like half of the task when it comes to machine learning.

There are many great tools and services for annotating your data. Today, we'll talk about labelstud.io

1/6

2/6 Many kinds of data can be annotated with Label Studio: images, audio, text, time series, and any combination of the above.

You can easily define your own annotation interface with a simple HTML-like markup.

Check out many pre-made templates at labelstud.io/playground/

3/6 Label Studio is open-source and is under active development.

It has 4K stars on Github, the latest major release is just a couple of weeks ago, and pull requests are successfuly getting merged :)

You can try it yourself locally with a single Docker command.

Read 6 tweets

Full Stack Deep Learning

@full_stack_dl

20 Jan

https://twitter.com/josh_tobin_/status/1351662321240707073

🛠️ tooling tuesday 🛠️

In honor of our first lecture at Berkeley this evening, here's our remote teaching stack:

https://twitter.com/josh_tobin_/status/1351662321240707073

@zoom_us

1/ @zoom_us. Duh.

One thing that makes it better is to have a good audio/video setup. Here's a good guide:

https://twitter.com/garrytan/status/1240649924212961280?s=20

Fujifilm cameras work too and avoid the need for the Camlink.

@SlackHQ

2/ @SlackHQ for question management.

Zoom chat is unthreaded and hard to react to. Asking questions live is chaos. Instead, students post questions in a slack channel.

Instructors can answer them directly in slack, or summarize and answer aloud at a break in the lecture.

Read 7 tweets

Full Stack Deep Learning

@full_stack_dl

13 Jan

@GoogleColab

🛠️Tooling Tuesday🛠️

Today, we share a @GoogleColab notebook implementing a Transformer with @PyTorch, trained using @PyTorchLightnin.

We show both encoder and decoder, train with teacher forcing, and implement greedy decoding for inference.

colab.research.google.com/drive/1swXWW5s…

👇1/N

2/N Transformers are a game changer.

This architecture has superseded RNNs for NLP tasks, and is likely to do the same to CNNs for vision tasks.

PyTorch provides Transformer modules since 1.2, but the docs are lacking:

- No explanation of inference
- Tutorial is encoder-only

3/N Our notebook shows both. Let's get started with simple data.

Our output will be number sequences like [2, 5, 3].

Our input will be the same as output, but with each element repeated twice, e.g. [2, 2, 5, 5, 3, 3]

We start each sequence with 0 and end each sequence with 1.

Read 10 tweets

Full Stack Deep Learning

@full_stack_dl

5 Jan

🛠️Tooling Tuesday🛠️

Let's talk about setting up our Python/CUDA environment!

Our goals:

- Easily specify exact Python and CUDA versions
- Humans should not be responsible for finding mutually-compatible package versions
- Production and dev requirements should be separate

1/N

Here's a good way to achieve these goals:

- Use `conda` to install Python/CUDA as specified in `environment.yml`

- Use `pip-tools` to lock in mutually compatbile versions from `requirements/prod.in` and `requirements/dev.in`

- Simply run `make` to update everything!

2/N

Here's our `environment.yml` file.

It specifies Python 3.8, CUDA 10.2, CUDNN 7.6.

To create an environment from this, install Miniconda (docs.conda.io/en/latest/mini…) and run `conda env create`.

Activate the environment with `conda activate conda-piptools-sample-project`

3/N

Read 7 tweets

Share this page!

Full Stack Deep Learning

Try unrolling a thread yourself!

More from @full_stack_dl

Full Stack Deep Learning

Full Stack Deep Learning

Full Stack Deep Learning

Full Stack Deep Learning

Full Stack Deep Learning

Full Stack Deep Learning

Did Thread Reader help you today?

Like this author's thread?