Hugging Face Profile picture
May 10, 2023 6 tweets 3 min read Read on X
We just released Transformers' boldest feature: Transformers Agents.

This removes the barrier of entry to machine learning

Control 100,000+ HF models by talking to Transformers and Diffusers

Fully multimodal agent: text, images, video, audio, docs...🌎

huggingface.co/docs/transform… A diagram of a HuggingFace ...
Create an agent using LLMs (OpenAssistant, StarCoder, OpenAI ...) and start talking to transformers and diffusers

It responds to complex queries and offers a chat mode. Create images using your words, have the agent read the summary of websites out loud, read through a PDF Image
How does it work in practice?

It's straightforward prompt-building:
• Tell the agent what it aims to do
• Give it tools
• Show examples
• Give it a task

The agent uses chain-of-thought reasoning to identify its task and outputs Python code using the tools. Image
It comes with built-in tools:

• Document QA
• Speech-to-text and Text-to-speech
• Text {classification, summarization, translation, download, QA}
• Image {generation, transforms, captioning, segmentation, upscaling, QA}
• Text to video

It is EXTENSIBLE by design. Image
Tools are elementary: a name, a description, a function.

Designing a tool and pushing it to the Hub can be done in a few lines of code.

The toolkit of the agent serves as a base: extend it with your tools, or with other community-contributed tools:

huggingface.co/docs/transform… Image
Please play with it, add your tools, and let's create *super-powerful agents* together.

Here's a notebook to get started: colab.research.google.com/drive/1c7MHD-T…

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Hugging Face

Hugging Face Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @huggingface

Oct 11, 2023
What you may have missed from the 🤗 open source community gathering in Paris last week: demos 🕹️

The ~2k folks who attended, while mingling, stumbled upon demos from the community: companies, researchers, artists and tinkerers ✨

A bit more about each of them 👇🧵 Image
1 - Rewyse: a platform that uses ML and NLP AI to provide insights and analysis on product reviews at scale rewyse.ai
2 - Augmented Remote Sensing: Louis Ulmer showcased a pipeline that integrates smart mapping, news and social media for disaster response 🚒👨‍🚒

- Project:
- Open-source application for the Morrocan earthquake of Sep 2023: lulmer.github.io/augmented-remo…
huggingface.co/spaces/nt3awno…
Read 20 tweets
Apr 20, 2023
SAM, the groundbreaking segmentation model from @Meta is now in available in 🤗 Transformers!
What does this mean?

1. One line of code to load it, one line to run it
2. Efficient batching support to generate multiple masks
3. pipeline support for easier usage

More details: 🧵 Image
You can first read more about the model, and learn how to use it on our documentation page: huggingface.co/docs/transform…
Let's check all the features we support below!
Automatic mask generation pipeline!

With one line of code predict automatically the segmentation masks of a given image (similar as the examples above)

Check out the example notebook: github.com/huggingface/no… Image
Read 7 tweets
Dec 31, 2022
It's been an exciting year for 🤗Transformers. We tripled the number of weekly active users over 2022, with over 1M users most weeks now and 300k daily pip installs on average🤯 Image
We doubled the number of architectures (89 to 167🤯) with new models in audio🔊, text📚, vision🖼️, multiple modalities or even time series📈and protein folding🧬

Here are a few highlights in the most used of those new models👇
Swin Transformer is a vision model from @MSFTResearch added back in January, which can be used as backbone for a variety of tasks such as image classification, object detection or semantic segmentation.

huggingface.co/docs/transform… Image
Read 10 tweets
Mar 31, 2022
📊4 challenging speech tasks, 102 spoken languages: can one model solve them all? 🤯

Introducing @GoogleAI's XTREME-S🏂 - the first multilingual speech benchmark that is both diverse, fully accessible, and reproducible!

👉huggingface.co/datasets/googl…

1/9
XTREME-S covers
- automatic speech recognition (ASR),
- speech translation (ST),
- speech classification, and
- speech retrieval.

2/9
ASR is based on the novel FLEURS dataset, the multilingual LibriSpeech dataset, and the VoxPopuli dataset.

Over 100 languages and varying domains, including audiobooks & parliament speeches are covered.

3/9
Read 9 tweets
Nov 4, 2021
[THREAD] Following the public release of Spaces, here is a showcase of a few ones we like. Let’s start with this surprising Draw-to-Search demo by @osanseviero and powered by CLIP. huggingface.co/spaces/osansev…
Next, listen to the tacotron2 voice reading you how to bake cookies (in Mandarin) with this Coqui Text-to-Speech demo by @eugene_siow. huggingface.co/spaces/eugenes…
What a time to be alive! You can finally decode your doctor's prescription with this cool OCR demo by @NielsRogge - using the Microsoft TrOCR encoder-decoder model. huggingface.co/spaces/nielsr/…
Read 6 tweets
Nov 3, 2021
Part 1 of the course focused on text classification, part 2 will focus on all other common NLP tasks. @mervenoyann has made videos to introduce you to each of them!
Let's start with Token Classification (giving a label to some/each word in a sentence):

Then there is question answering: finding the answer to a question in some context.

Next is Causal Language Modeling: guessing the next word in a sentence. This is how GPT-2 and its descendants were pretrained.

Read 6 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(