Joris de Jong Profile picture
Jun 21 12 tweets 6 min read Twitter logo Read on Twitter
Did you know you can use ChatGPT to summarize your (online) meetings?

With @LangChainAI, and a couple of lines of code, you can!

Let me show you how in 6 simple steps🧵

#AI Image
Before we dive in, this is day 2 of my '7 days of LangChain'.

Every day, I'll introduce you to a simple project that will guide you through the basics of LangChain.

Follow @JorisTechTalk to stay up-to-date.

If there's anything you'd like to see, let me know!

Let's dive in:
High level overview of what's happening:

1️⃣ Load your audio file
2️⃣ Speech-to-text with Whisper
3️⃣ Split the transcript into chunks
4️⃣ Summarize the transcript

Let's dive into the code ⬇️ Image
1. Open your audio file.

Whisper works on files with a maximum duration of approximately 20 minutes.

Longer file?

Split it up by using PyTube and handle each chunk seperately. Image
2. Call the Whisper API

Of course, you can use any speech-to-text API you prefer. I like Whisper for its accuracy and ease of use.

Any open-source alternatives with quality output? Image
3. Splitting the transcript into chunks

With the new OpenAI GPT 16k model, you can fit a large amount of context into one chunk. This is amazing for the model to 'understand' the full context and make connections.

Use some overlap in order for context to not be lost. Image
4. Prompting

Prompting is key. It determines your output more than anything else.

Be as concise as you can be. Instruct the model on how you want the output to look. You could include bullet points, main takeaways, follow-up actions and much more. ImageImage
5. Initialize and run the summary chain

I'm using the refine summarization chain. This is great when you're working with large files.

It generates an initial summary based on the first chunk and updates it with the subsequent chunks. Image
6. Export the summary to a text file

Your meeting is summarized and you're ready to take action!

Once again, try to play around with the prompts you're using. This will greatly impact the resulting summarization. Image
7. Possible future implementations

You can go wild with this. You could use a Zapier integration to send the summarized meeting in an email, create appointments in your schedule and much more.

Let me know what you're going to add.
That concludes day 2 of '7 days of @LangChainAI'

Tomorrow's project: Creating mindmaps with @LangChainAI and @XmindHQ.

Follow @JorisTechTalk to stay up-to-date.

What else would you like to see?
@LangChainAI @XmindHQ Day 2 of '7 days of @LangChainAI' ✅

Tomorrow's project will be exciting: generating mindmaps for studying!

What else do you want to see?

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Joris de Jong

Joris de Jong Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @JorisTechTalk

Jun 21
Meta entered the chat...

Meta AI has unveiled Voicebox, a groundbreaking generative model for voice synthesis tasks.

This model can generate speech from text and perform tasks like editing, noise removal, and style transfer.

Let's dive into the details! 🧵
Voicebox is a generative model that can synthesize speech in six languages.

It has been trained on a general task of mapping voice audio samples to their transcripts, enabling it to perform various text-guided speech generation tasks seamlessly.
🔬 The researchers at Meta developed a unique training method called "Flow Matching" for Voicebox.

This technique allows the model to learn from diverse speech data without the need for careful labeling.

Trained on 50,000 hours of speech and transcripts from audiobooks. Image
Read 10 tweets
Jun 20
ChatGPT is great for creating plans.

But it can't use YouTube videos as a knowledge base.

With @LangChainAI, you can!

I've used the @thedankoe's YouTube video on '4-hour workdays' and let AI create a detailed plan.

Let me show you how you can do it too, in just 8 steps.

#AI Image
Before we dive in, this is day 1 of my '7 days of LangChain'.

Every day, I'll introduce you to a simple project that will guide you through the basics of LangChain.

Follow @JorisTechTalk to stay up-to-date.

If there's anything you'd like to see, let me know!

Let's dive in:
A high-level overview:

1️⃣ Load the YouTube transcript
2️⃣ Split the transcript into chunks
3️⃣ Use a summarization chain to create a strategy based on the content of the video
4️⃣ Use a simple LLM Chain to create a detailed plan based on the strategy.

And now for the code ⬇️ Image
Read 16 tweets
Jun 20
Finance 🤝 AI

Language models have transformed natural language processing across industries, and now they're making waves in finance.

Enter FinGPT: An open-source Financial Large Language Model

Let's dive in 🧵 Image
Extracting financial data can be daunting, spanning web platforms to PDFs.

While proprietary models like BloombergGPT have specialized data, the need for an open and inclusive alternative is clear.

Introducing FinGPT:
Developed by researchers from Columbia University and NYU Shanghai, FinGPT is an end-to-end open-source framework for economical large language models (FinLLMs).

Its mission: democratize financial data access and foster open finance. 📈
Read 8 tweets
Jun 19
One step closer to human-level intelligence in AI:

A year ago, Meta's Chief AI Scientist, Yann LeCun, proposed a groundbreaking architecture that could revolutionize AI systems as we know them.

Today, the first implementation is here: I-JEPA.

A deeper dive 🧵 Image
1/13 The goal?

To create machines that can learn internal models of how the world works, enabling them to learn faster, plan complex tasks, and adapt to new situations.

Let's dive into the details! 👇
2/13 📚 Introducing the Image Joint Embedding Predictive Architecture (I-JEPA).

The first AI model based on LeCun's vision. I-JEPA learns by creating an internal model of the world, comparing abstract representations of images instead of pixels themselves. 🖼️
Read 14 tweets
Jun 18
AI and Safety:

@owasp has released a list of the top 10 most critical vulnerabilities found in artificial intelligence applications based on large language models (LLMs).

These vulnerabilities include prompt injections, data leakage, and unauthorized code execution.

A 🧵

#AI Photo by Pixabay from Pexels
1. Prompt injections:

This involves bypassing filters or manipulating the LLM using carefully crafted prompts that make the model ignore previous instructions or perform unintended actions.
2. Data Leakage:

Data leakage occurs when an LLM accidentally reveals sensitive information through its responses. #cybersecurity
Read 12 tweets
Jun 17
The power of natural language interaction is taking over!

Companies are bringing AI applications to life with large language models (LLMs). The adoption of language model APIs is creating a new tech stack in its wake.

Key takeaways from research by @sequoia

🧵 Image
1/ Nearly every company in the Sequoia network is building language models into their products.

From code to data science, chatbots to sales, and even grocery shopping and travel planning, the possibilities are endless.
2/ The new stack for these applications centers on language model APIs, retrieval, and orchestration, but open source usage is also growing.

Companies are interested in customizing models to their unique context, and the stack is becoming increasingly developer-friendly.
Read 7 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us on Twitter!

:(