TimeGPT is the first foundation model specifically designed for time series analysis.

It excels at generating precise forecasts across a diverse range of datasets and domains.

Here's what you need to know about it:

1/8 The model leverages a Transformer-based architecture, optimized for time series data, with self-attention mechanisms that facilitate the handling of temporal dependencies and patterns across varied frequencies and characteristics.

2/8

8 Free Courses to Master Large Language Models:

1. @cohere LLM University
2. @huggingface NLP course
3. @databricks courses
and more!

🧵





1. @cohere LLM University

The course offers insights into how LLMs work, and their practical applications, and guides participants on using LLMs to build and deploy applications.docs.cohere.com/docs/llmu

DoRA (Weight-Decomposed Low-Rank Adaptation) sets a new standard for optimizing AI models.

It combines the benefits of full model fine-tuning and LoRA.

How does it do that? Let's see 👇🏼

1/7 The genius of DoRA lies in its unique handling of pre-trained weights.

It separates these weights into two parts:

1. one that determines the size (magnitude)
2. one that determines the orientation (direction) of the weight vectors

2/7

Want to understand foundation models, generative AI models, and transformers?

Here is your FREE list of 15+ resources to do that: 1. Efficient Transformers: A Survey explores the evolution of Transformer models in various domains. It provides a comprehensive overview of different Transformer variants (X-formers) to guide researchers.
arxiv.org/abs/2009.06732

7 resources to master prompt engineering:

1. Prompt Engineering Guide
2. Learn Prompting course
3. ChatGPT Prompt Engineering for Developers course
...

🧵





1. Zero-shot, few-shot, and chain-of-thought lineage prompting techniques explained.

Check our article detailing various prompt engineering techniques at: turingpost.com/p/cot

8 free courses to master large language models:

- Cohere LLM University
- Hugging Face NLP course
- DeepLearning AI courses
- Weights & Biases course
- Introduction to LLMs course by Google Cloud
...

🧵





1. @cohere LLM University

The course offers insights into how LLMs work, and their practical applications, and guides participants on using LLMs to build and deploy applications.

docs.cohere.com/docs/llmu

Why Vector Embeddings Are Key for LLMs?

Vector embeddings turn complex data into numerical forms, a crucial step for Foundation Models & LLMs.

Let's dive into how they redefine AI’s capabilities: 1. Semantic Information Capture:

Vector embeddings are adept at encoding both semantic & syntactic information. This allows models to grasp context and meaning, a fundamental aspect for understanding natural language.

10 surveys about transfer learning and domain adaptation you need to read.

Domain: Computer vision

🧵





1. A Survey on Transfer Learning (2010)

The survey categorizes and reviews transfer learning's progress for classification, regression, and clustering, discussing its relationship with domain adaptation, multitask learning, and co-variate shift.

cse.ust.hk/~qyang/Docs/20…

Adapting LLMs to your specific needs is key to making the most out of these powerful tools in your business.

Ways to do this:

▪️ Prompting techniques
▪️ Retrieval-Augmented Generation (RAG)
▪️ Fine-Tuning

Let's dive into each of them: 1. Prompt engineering

Designing specific input prompts that guide the model to apply its general knowledge in a way that's relevant to the task.

Quantization is a technique used to reduce the size and increase the efficiency of deep learning models.

Here is a list of 23 LLM quantization techniques you need to know about: 1. LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models arxiv.org/abs/2206.09557

Understanding how large models can be optimized into smaller yet efficient versions is key to AI advancements.

Let’s delve into the critical characteristics of a model.

These can be tweaked to maintain performance while reducing size and resource demand: 1. Number of Parameters: This is the total count of learnable weights in a model. Generally, more parameters mean greater expressiveness, but they also demand more computational resources and memory during both training and inference phases.

Hugging Face's Chief of Science @Thom_Wolf shared the resources he used to join the fields of NLP, AI, and ML!

Here is the list with the links he shared. 🧵 1. "Deep Learning" by @goodfellow_ian, Yoshua Bengio and Aaron Courville

Provides a great overview of current deep learning techniques.

deeplearningbook.org

5 open-source datasets used to train LLMs

1. YT-Temporal-1B (video)
2. Muffin (text)
3. LAION-400M (text)
4. HumanEval (code)
5. WebVid-2M (text and video)

Links 🧵 YT-Temporal-1B (video)

crfm.stanford.edu/ecosystem-grap…

Automatic prompting is one of the hot topics.

4 research papers on auto-prompting and one bonus

▪️ AutoPrompt
▪️ LLMs are Human-Level Prompt Engineers
▪️ Automatic Chain-of-Thought prompting in LLMs
▪️ LLMs are Zero-Shot Reasoners
🎁 Bonus!

🧵 1. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts

Proposes AutoPrompt, a method to automate the creation of prompts based on a gradient-guided search.

arxiv.org/pdf/2010.15980…

5 open-source datasets used to train LLMs

1. MineDojo (videos and text)
2. ROOTS (text)
3. NaturalInstructions-v2 (text)
4. Anthropic Helpfulness dataset (text)
5. LAION-115M(text and image)

Links 🧵 MineDojo (videos and text)

crfm.stanford.edu/ecosystem-grap…

1/5

OpenAI CEO, Sam Altman, testified before Congress yesterday for 4 hours.

Don't have the time to watch it all?

We've collected the highlights that are already resonating with people 🧵 A super-edit video summarizing the full conversation in 27 minutes


1/10

LangChain: the popular tool that's taking over.

Main components

▪️ Schema
▪️ Models
▪️ Prompts
▪️ Indexes
▪️ Memory
▪️ Chains
▪️ Agents

Let's review them in more detail 🧵 Schema

LLMs work with what is usually called text.

It includes
▪️ ChatMessages consist of text + user. Users can be: System, Human, and AI
▪️ Examples: input/output pairs
▪️ Document: a piece of unstructured data

1/8

Some key LLMs in China

1. CPM (2020, 2.6B) @Tsinghua_Uni
2. M6 (2021, 100B) @AlibabaGroup, @Tsinghua_Uni
3. PLUG (2021, 27B) @AlibabaGroup
4. PanGu-α (2021, 207B) @Huawei, Recurrent AI, @PKU1898, Peng Cheng Lab
5. ConSERT (2021, 345B) @meituan

Links to papers 🧵 WuDao-Wen Yuan (2020, 2.6B)

Paper: arxiv.org/pdf/2012.00413…
Code: github.com/TsinghuaAI/CPM…

1/5

5 open-source datasets used to train LLMs

1. SA-1B (image)
2. OIG-moderation (text)
3. OIG-43M (text)
4. Flan Collection (text)
5. LAION-5B (text and image)

Links 🧵 SA-1B (image)

crfm.stanford.edu/ecosystem-grap…

1/5

5 open-source LLMs (save the list)

1. NLLB
2. GLM-130B
3. RWKV (RNN with Transformer-level performance)
4. Flan-T5
5. Galatica

All repos in 🧵 1. NLLB

github.com/facebookresear…

5 free ML courses recommended by @OpenAI professionals.

Don't miss out - save the thread

▪️ Linear Algebra @Stanford
▪️ Neural Networks for ML @geoffreyhinton
▪️ Neural Nets @cs231n
▪️ Advanced Robotics @pabbeel
▪️ Deep RL @johnschulman2

All links in 🧵 1. EE263: Introduction to Linear Dynamical Systems by Stephen Boyd, Stanford University

ee263.stanford.edu/archive/