Discover and read the best of Twitter Threads about #PALM2
Most recents (2)
1/ 🚀 Google now offers FREE courses on Generative AI! Master the future of AI and unlock new possibilities in various applications. #AI #GenerativeAI #Google #courses
2/ 🧠 Get introduced to Generative AI Studio, a powerful tool within Vertex AI that allows you to create and personalize generative AI models for your applications.
#Vertex #AI #Google
#Vertex #AI #Google
3/ 🌐 At Google I/O 2023, Google unveiled a suite of generative AI features for Gmail, PaLM 2 language model, Med-PaLM 2 for medical applications, and Bard for #developers.
#GoogleIO #PaLM2 #MedPaLM2 #Bard #MedTwitter
#GoogleIO #PaLM2 #MedPaLM2 #Bard #MedTwitter
#NewPaperAlert When and where does pretraining (PT) data matter?
We conduct the largest published PT data study, varying:
1⃣ Corpus age
2⃣ Quality/toxicity filters
3⃣ Domain composition
We have several recs for model creators…
📜: bit.ly/3WxsxyY
1/ 🧵
We conduct the largest published PT data study, varying:
1⃣ Corpus age
2⃣ Quality/toxicity filters
3⃣ Domain composition
We have several recs for model creators…
📜: bit.ly/3WxsxyY
1/ 🧵
First, PT data selection is mired in mysticism.
1⃣ Documentation Debt: #PALM2 & #GPT4 don't document their data
2⃣ PT is expensive ➡️ experiments are sparse
3⃣ So public data choices are largely guided by ⚡️intuition, rumors, and partial info⚡️
2/
1⃣ Documentation Debt: #PALM2 & #GPT4 don't document their data
2⃣ PT is expensive ➡️ experiments are sparse
3⃣ So public data choices are largely guided by ⚡️intuition, rumors, and partial info⚡️
2/
PT is the foundation of data-centric and modern LMs. This research was expensive but important to shed light on open questions in training data design.
Here are our main findings:
3/
Here are our main findings:
3/
