Discover and read the best of Twitter Threads about #JeanZay
Most recents (1)
🌸 The @BigScienceLLM BLOOM 176B parameters model training has just passed 230B tokens: that’s more than a million books in two months!
🤔 But how did we decide what model to train with our one million GPU hours?
⬇️ Thread time! #acl2022nlp
🤔 But how did we decide what model to train with our one million GPU hours?
⬇️ Thread time! #acl2022nlp
🏅 We had five main considerations: it needed to be proven, scalable, efficient, multilingual, and to exhibit emergent capabilities (e.g. zero-shot generalization)
⏰ At the >100B scale, every inefficiency matters! We can’t afford an unoptimized setup…
⏰ At the >100B scale, every inefficiency matters! We can’t afford an unoptimized setup…
