Teknium Profile picture
May 2 4 tweets 1 min read Twitter logo Read on Twitter
I'm working on a LORA right now for 7B Llama. Its training consists of a combined/shuffled dataset of WizardLM + GPT4-LLM (GPT4Alpaca+Unnatural Instructions) + GPTeacher(General-Instruct+Roleplay-Instruct) and my unreleased Roleplay-Instruct v2.
Epoch 1: huggingface.co/teknium/Llama-…
I havent tested it at all yet fyi. Could be completely terrible. Who knows. Will upload epoch 2 and the final 3rd epoch when done.
I've uploaded a merged full model to HF, in the link above in it's own folder.
Epoch 2 is uploaded now

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Teknium

Teknium Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @Teknium1

May 3
Finally getting around to cleaning up the GPT4 / GPTeacher's code-instruct dataset to add to the repo, wish me luck xD
I'm not liking the progress so far... I may have made such a mess making this dataset it has to be cleaned manually 🥲
eyyy breakthrough got it now
Read 4 tweets
Apr 19
New LLM: @StabilityAI has released a 3B and 7B parameter LLM trained on (at least a large portion of) The Pile v2 - its unclear if it is 800B tokens or 1.5T tokens, I'm hearing conflicting reports on that - they also released fine tuned alpaca versions: huggingface.co/stabilityai/st… Image
It's confirmed atm only 800B Tokens - but *will be 1.5T* when done
Image
Read 5 tweets
Mar 14
Can you bring back Sydney with ChatGPT4?
Read 5 tweets
Mar 3
Okay, so, people are having issues with running LLaMa on their home PC. I'm using a 3090, thanks to some help by a few people, I have got updated code for LLaMa's example inference code to run it.
First, download this repo and the models: github.com/facebookresear… 1/
Place each model folder in the root of your project, install the requirements, make sure you get pytorch with cuda installed. Replace `example .py` with pastebin.com/fG2J7CHf
Then replace model .py in the llama folder with this
pastebin.com/1qUU3sW5
Read 6 tweets
Mar 2
LLaMa models torrent.. If FB or someone requests I take it down I will. eee cdn.discordapp.com/attachments/10…
If you download it this is the only inference code I know of atm github.com/facebookresear…
Read 4 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(