Discover and read the best of Twitter Threads about #glm130b
Most recents (1)
I have successfully compiled and run GLM-130b on a local machine! It's now running in `int4` quantization mode and answering my queries.
I'll explain the installation below; if you have any questions, feel free to ask!
github.com/THUDM/GLM-130B
I'll explain the installation below; if you have any questions, feel free to ask!
github.com/THUDM/GLM-130B
130B parameters on 4x 3090s is impressive. GPT-3 for reference is 175B parameters, but it's possible that it's over capacity for the data & compute it was trained on...
I feel like a #mlops hacker having got this to work! (Though it should be much easier than it was.)
I feel like a #mlops hacker having got this to work! (Though it should be much easier than it was.)
To get GLM to work, the hardest part was CMake from the FasterTransformer fork. I'm not a fan of CMake, I don't think anyone is.
I had to install cudnn libraries manually into my conda environment, then hack CMakeCache.txt to point to those...
I had to install cudnn libraries manually into my conda environment, then hack CMakeCache.txt to point to those...
