It's the new customer, stupid. I spent years at Netflix testing only new members and ignoring existing members. If you want to become a great and big product/company you need to focus on new members. Figure out what delights them, and how to retain them over time.
Existing members don't notice most of the changes you make. When they do, they often complain. But they don't quit the service. There's a tendency to listen too carefully to these customers because they have loud voices and a similar product sensibility as yours.
I call these customers "Silicon Valley Freaks" and it describes most of us -- folks who are on Twitter on a Sunday morning. We are not normal, and if you over-indulge these freaks (us) you end up creating a very complicated product that works extremely poorly for new members.
(I don't mean to offend product leaders outside the Bay Area. There are freaks all over the world-- Toronto Freaks, Berlin Freaks, Singapore Freaks, NYC Freaks. "Silicon Valley Freaks" broadly refers to non-normal people who spend lots of time using new tools and technology.)
At Netflix, we stopped doing qualitative (focus groups, usability) in the Bay Area. We traveled to far-off places like Providence or Memphis to get more in touch with non-freaks. And we focused most of our time/effort on new members.
Then we would A/B test with new members to see if it changed behavior -- converted more new members into the service and/or retained them better during the critical first few months.
Occasionally we would do a sanity test of a new feature with existing members before we rolled it out to all. But in most cases, the existing members didn't notice it, with exception of a few freaks who complained mightily, but didn't quit. (No one likes change.)
Why am I writing this on a Sunday morning? I have been using lots of new tools in my transition online (Slido, Teachable, Zoom, Loom, Google Slides, Miro). The complicated issues for a new member are obvious.
If product leaders spent more time looking over the shoulders of brand new, "normal" customers they would build better products. Products that grow faster and retain better in the long-run.
And spend less time listening to "Silicon Valley Freaks" who complain mightily about things like Netflix's auto-play feature (which likely makes it easier for new members to find/watch a movie). These are loud voices that are likely wrong. (Netflix, give me a peek at this data!)
And please don't listen to me. With all of the new products I use, I dedicate the time to figure out the absurdly complicated features that normal people don't find. I do crazy things like find the product leader building the feature to learn how to use it. I am a freak.
The takeaway: It's the new customer, stupid. As you build a product, focus on normal, non-Silicon Valley freaks. Do lots of qualitative outside the Bay Area. Engage in lots of A/B testing with new members and occasionally indulge your existing members with a back-test.
Focus on new customers. You'll build better, simpler products that both convert and retain better in the long run and you'll spend far less time reading rants from people like me.
