step 4: the filtering system (this is the game-changer)
the biggest problem with AI research tools is they give you the same insights over and over
"people want clear skin" - yeah, we know that already
you need a system that only shows you NEW insights
this is where embeddings come in
embeddings turn text into numbers that capture meaning
"I love food" and "I enjoy eating" use different words
but mean the same thing embeddings see they're similar because the numbers are nearly identical
step 5: duplicate detection process
when an AI finds a new insight, embeddings help narrow down your database to the top 5-10most similar existing insights
then an LLM compares the new insight to just those 5 and determines if it's truly new or a duplicate
this is way more accurate than asking AI to compare against your entire database
step 6: clean daily reporting
instead of drowning in repetitive information, you get reports with only genuinely new insights
each insight comes with actual quotes from Reddit users as supporting evidence
your research stays valuable instead of becoming noise
what this creates:
a clean, growing database of customer insights that doesn't get cluttered with duplicates
daily reports that actually teach you something new about your customers
real customer language you can use in ads and copy
to recap how to automate customer research with AI:
> target relevant subreddits where customers hang out
> scrape top daily threads automatically
> extract insights in specific categories with focused prompts
> use embeddings and LLMs to filter out duplicate insights
> get clean daily reports with only new findings
• • •
Missing some Tweet in this thread? You can try to
force a refresh
how to give an AI agent memory to make it smarter over time:
most AI system treats every conversation like it's meeting you for the first time
for example a customer first complains about a broken chair, then the AI responds like it's a new issue
but the same customer complains about another broken chair 2 months later
but in most cases an AI system still treats it like a new convo and acts like it's never happened before
what an AI with memory would do:
"Hello David, following up on your previous case from August where you experienced issues with an ergonomic chair, I sincerely apologize this is happening again. We appreciate your continued trust since 2023..."
it references past interactions and treats customers like actual humans
here's everything you need to make money with AI (the only 2 paths that actually work):
there are only two ways to make money with AI
most people get confused because they see all the time on twitter about how people are making six figures with this automation or using this tool and shit like that
but when you strip away all the noise, it comes down to just these two paths
path 1: use AI to amplify what you're already good at
this is for 90% of you reading this
if you're decent at copywriting, sales, paid ads, whatever - don't abandon that to become an "AI expert"
instead, use AI to make you 10x better at what you already do
here's everything you need to build and deploy AI apps in hours (complete beginner's guide):
most people think you need years of coding experience to build apps
but with AI tools doing the heavy lifting, you can go from idea to deployed app in a single day
the key is using the right stack and approach
here's the vibe coding stack I usually recommend for beginners:
> planning with ChatGPT/Claude
> coding with Cursor and Sonnet 4
> storing data with Supabase
> handling user logins with Clerk
> deploying with Vercel or Railway