I'm working on a LORA right now for 7B Llama. Its training consists of a combined/shuffled dataset of WizardLM + GPT4-LLM (GPT4Alpaca+Unnatural Instructions) + GPTeacher(General-Instruct+Roleplay-Instruct) and my unreleased Roleplay-Instruct v2.
Epoch 1: huggingface.co/teknium/Llama-…
I havent tested it at all yet fyi. Could be completely terrible. Who knows. Will upload epoch 2 and the final 3rd epoch when done.
I've uploaded a merged full model to HF, in the link above in it's own folder.
Epoch 2 is uploaded now
• • •
Missing some Tweet in this thread? You can try to
force a refresh
New LLM: @StabilityAI has released a 3B and 7B parameter LLM trained on (at least a large portion of) The Pile v2 - its unclear if it is 800B tokens or 1.5T tokens, I'm hearing conflicting reports on that - they also released fine tuned alpaca versions: huggingface.co/stabilityai/st…
It's confirmed atm only 800B Tokens - but *will be 1.5T* when done
Okay, so, people are having issues with running LLaMa on their home PC. I'm using a 3090, thanks to some help by a few people, I have got updated code for LLaMa's example inference code to run it.
First, download this repo and the models: github.com/facebookresear… 1/
Place each model folder in the root of your project, install the requirements, make sure you get pytorch with cuda installed. Replace `example .py` with pastebin.com/fG2J7CHf