Thread by @dicksonneoh7

Deploying GPT-like language models on a chatbot is tricky.

You might wonder
• How to access the model?
• Where to host the bot?

In this 🧵I walk you through how easily I deployed a GPT-J-6B model by #EleutherAI on a #Telegram bot with @huggingface and @Gradio.

For FREE 🚀

By the end of this🧵, you’ll have your very own Telegram bot that can query the GPT-J model with any text you send it 👇

🤖Token From the BotFather

To create a bot, you must have a Telegram account.

Next, get a TOKEN from the BotFather. This TOKEN allows you to access the bot.

Keep this TOKEN private🤫. Anyone with this TOKEN can access your bot.

🐍 Python Telegram Bot

Next, install a Telegram bot wrapper library👉python-telegram-bot (PTB).

Now, we can code our bot using Python!

github.com/python-telegra…

🤯With PTB, you can run your own bot with only 8 lines of code!

👋 Running the above code results in a simple bot that replies hello to the user

Now, how do we access the GPT-J model?

Enter👉 Hugging Face Hub.

Hugging Face Hub is a central place where anyone can share their models, dataset, and app demos.

The GPT-J-6B model is on the Hub! Anyone can use it.

huggingface.co/EleutherAI/gpt…

Let's create a @gradio demo on the Hub to interact with the GPT model.

You can create your own or use my demo. Play around with some text as the input to the GPT-J model.

Here's the demo app 👇
huggingface.co/spaces/dnth/gp…

With the @gradio app, we also gain access to a HTTP endpoint that allows us to query the GPT model from elsewhere!

I've used this feature to deploy large models on an Android app.

Now, all we have to do here is call the endpoint from the Telegram bot!

For example 👇

🕹Putting everything together, we now have a Telegram bot that can query the GPT model!

Just one problem 👉 we must keep this script running 24/7 on a computer to keep the bot alive.

Is there a better way?🤔

Yes! I recently discovered you can host your bot on Hugging Face Space! 🤫

All you have to do is create a @Gradio app, make a requirements.txt file and upload the above script!

Here's my repo for reference 👇
huggingface.co/spaces/dnth/pt…

You do not want to expose your token in your source code in the repo.

For that, create an environment variable to hide the token as a secret.

The end result 👉 a 24/7 working Telegram bot that has access to the GPT-J-6B model 🥳

For FREE 🚀

😍 That’s a wrap!

In this 🧵 I showed you how you can utilize any model on the @huggingface Hub and deploy it on a Telegram bot.

There are tons of models on the Hub. What models are you thinking of? #dalle? CLIP? Try it!

Link to my bot 👉 t.me/ptbgptbot

Or, just for fun, replace the GPT-J-6B model with a GPT-Neo model that paraphrases texts.

huggingface.co/spaces/dnth/gp…

🙏 I hope you’ve learned something from this 🧵. I'm curious what other bots you'd create with this. Tag me in your Tweets!

💡 If you find value in this post, consider following me for more bite-size deployment tips like this!

🖥 Details in the blog post
dicksonneoh.com/portfolio/depl…

🤒 Lastly, I need your help.

During the pandemic, my day job was impacted.

I’m now seeking remote employment opportunities as a Data Scientist or Machine Learning Engineer.

If you know of any openings, please tag or connect me. I'll be forever grateful 🙏

Or help me re-tweet this post so that it will reach the right hiring manager.

Sincerely, thank you.

https://twitter.com/dicksonneoh7/status/1527512946603020288