Thread by @RichMillington

1 - I want to share some data from a client we worked with last year where we did something that I think was a little clever to definitively prove the incredible value of their community.

But first, some background....

#CMGR

2 - The client was a mature community in the tech sector. Despite the community's size and scale, the community team had been downsized the previous year and their budget was being squeezed each year because they couldn't clearly prove the value of the community.

3 - Yes, they had used the call deflection templates you can find online, but the senior execs simply didn't believe the metrics were real (I've never believed them either tbh).

It's almost impossible to prove if someone a) got an answer and b) would've called support).

4 - The only way to really prove call deflection is to shut the community down and see what happens. But that pretty much kills your superuser program and everything else too.

So we worked with them on the next best thing..

We deindexed the community from search for 4 months.

5 - Deindexing is quite clever because 80%+ of traffic came via search so you see a huge drop, but it doesn't kill the community for loyal members / regular visitors (although we had to explain what we were doing to the superusers).

6 - So we removed the community from search and spent a few nervous months seeing what would happen (more on how we spend this time later).

THE RESULTS WERE AMAZING!

You can see them below (the yellow line represents support tickets across all support channels).

BIG IMPACT !

7 - During the four months, the following things happened:

1) YoY community visitors dropped by 83%.

2) YoY total support tickets increased by 58%(!!!).

3) Customer satisfaction (not shown) plunged because support staff couldn't handle the increase in support tickets.

8 - This data showed that for every 207 visitors to the community, 1 would've contacted support (note this is a LOT lower than the 1 in 20 or 1 in 5 you often hear about).

However, the community gets 33.8m+ visitors a year. That's around 163k+ deflected tickets per year.

9 - Given each support call (according to the VP of support) is $18, we can now say the community saves our client $2.94m per year.

On a budget of $855k in 2019, that is an ROI of around 244% (which feels about right).

(*excludes the initial setup fees).

10 - So, yes, by noindexing the community for a few months we were able to fairly definitively prove the value of the community. Three more interesting points to note about this experiment.

11 - Interestingly, execs seemed less concerned about the dollar value savings than the rapid decline in customer support satisfaction as a result of hiding the community.

They had zero desire to hire more support staff, but some to shift more work to the community.

12 - While $3m in savings per year isn't huge for a major tech brand, we showed it's $13 cheaper to solve a ticket in the community than a support agent. This shifted the conversation from 'is community worth it?' to 'how do we make people visit the community first?'

13 - This only assess value by ticket deflection, it doesn't account for retention, loyalty, advocacy, feedback/ideation etc - which probably do deliver tremendous value but are FAR more difficult to prove. We picked the lowest hanging fruit here and kept the experiment simple.

14 - I've waited years to run an experiment like this. No other client was allowed (or brave enough) to do it. Be warned, the results give you nowhere to hide. Either you show a clear impact or you accept the community doesn't have the impact you thought.

15 - Also don't waste this time. If you reduce tickets by 80% for an experiment like this, use that time to improve tagging/update old, popular, questions, remove/archive outdated info etc...This is rare breathing space to make big improvements to the community.

Good luck :-)