Hugging Face Profile picture
May 10 6 tweets 3 min read Twitter logo Read on Twitter
We just released Transformers' boldest feature: Transformers Agents.

This removes the barrier of entry to machine learning

Control 100,000+ HF models by talking to Transformers and Diffusers

Fully multimodal agent: text, images, video, audio, docs...🌎… A diagram of a HuggingFace ...
Create an agent using LLMs (OpenAssistant, StarCoder, OpenAI ...) and start talking to transformers and diffusers

It responds to complex queries and offers a chat mode. Create images using your words, have the agent read the summary of websites out loud, read through a PDF Image
How does it work in practice?

It's straightforward prompt-building:
• Tell the agent what it aims to do
• Give it tools
• Show examples
• Give it a task

The agent uses chain-of-thought reasoning to identify its task and outputs Python code using the tools. Image
It comes with built-in tools:

• Document QA
• Speech-to-text and Text-to-speech
• Text {classification, summarization, translation, download, QA}
• Image {generation, transforms, captioning, segmentation, upscaling, QA}
• Text to video

It is EXTENSIBLE by design. Image
Tools are elementary: a name, a description, a function.

Designing a tool and pushing it to the Hub can be done in a few lines of code.

The toolkit of the agent serves as a base: extend it with your tools, or with other community-contributed tools:… Image
Please play with it, add your tools, and let's create *super-powerful agents* together.

Here's a notebook to get started:…

• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Hugging Face

Hugging Face Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @huggingface

Apr 20
SAM, the groundbreaking segmentation model from @Meta is now in available in 🤗 Transformers!
What does this mean?

1. One line of code to load it, one line to run it
2. Efficient batching support to generate multiple masks
3. pipeline support for easier usage

More details: 🧵 Image
You can first read more about the model, and learn how to use it on our documentation page:…
Let's check all the features we support below!
Automatic mask generation pipeline!

With one line of code predict automatically the segmentation masks of a given image (similar as the examples above)

Check out the example notebook:… Image
Read 7 tweets
Dec 31, 2022
It's been an exciting year for 🤗Transformers. We tripled the number of weekly active users over 2022, with over 1M users most weeks now and 300k daily pip installs on average🤯 Image
We doubled the number of architectures (89 to 167🤯) with new models in audio🔊, text📚, vision🖼️, multiple modalities or even time series📈and protein folding🧬

Here are a few highlights in the most used of those new models👇
Swin Transformer is a vision model from @MSFTResearch added back in January, which can be used as backbone for a variety of tasks such as image classification, object detection or semantic segmentation.… Image
Read 10 tweets
Mar 31, 2022
📊4 challenging speech tasks, 102 spoken languages: can one model solve them all? 🤯

Introducing @GoogleAI's XTREME-S🏂 - the first multilingual speech benchmark that is both diverse, fully accessible, and reproducible!


XTREME-S covers
- automatic speech recognition (ASR),
- speech translation (ST),
- speech classification, and
- speech retrieval.

ASR is based on the novel FLEURS dataset, the multilingual LibriSpeech dataset, and the VoxPopuli dataset.

Over 100 languages and varying domains, including audiobooks & parliament speeches are covered.

Read 9 tweets
Nov 4, 2021
[THREAD] Following the public release of Spaces, here is a showcase of a few ones we like. Let’s start with this surprising Draw-to-Search demo by @osanseviero and powered by CLIP.…
Next, listen to the tacotron2 voice reading you how to bake cookies (in Mandarin) with this Coqui Text-to-Speech demo by @eugene_siow.…
What a time to be alive! You can finally decode your doctor's prescription with this cool OCR demo by @NielsRogge - using the Microsoft TrOCR encoder-decoder model.…
Read 6 tweets
Nov 3, 2021
Part 1 of the course focused on text classification, part 2 will focus on all other common NLP tasks. @mervenoyann has made videos to introduce you to each of them!
Let's start with Token Classification (giving a label to some/each word in a sentence):

Then there is question answering: finding the answer to a question in some context.

Next is Causal Language Modeling: guessing the next word in a sentence. This is how GPT-2 and its descendants were pretrained.

Read 6 tweets
Oct 20, 2021
Transformers can read and write, but how well can they listen and speak 🗣️?

Find out by pitting your models against the SUPERB Challenge 📊!

SUPERB tests pretrained models on a wide range of speech processing tasks & datasets

Submit here 👉:
SUPERB aims to accelerate the development of *universal* speech representations 🤯

You can find all the details about the challenge here 👉:
Participants of the SUPERB Challenge can submit their results to the AAAI 2022 Workshop: The 2nd Self-supervised Learning for Audio and Speech Processing🤖

The winners will be invited to present their methods 🏅🏅🏅!

Workshop details here 👉:
Read 6 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!


0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy


3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!