Thread by @jugander

"An Experimental Study of Structural Diversity in Social Networks," new paper with @jessicatysu, @krishna_kamath, @aneeshs, and @5harad, to appear at ICWSM 2020: arxiv.org/abs/1909.03543 Several years in the making, I'm really excited to share this work! 1/n

In adoption decisions based on social referrals, what about graph structure? Classic modeling of such decisions study "influence response functions", adoption as a function of # of adopted friends, see e.g. this classic by @duncanjwatts/@peterdodds:
jstor.org/stable/10.1086… 2/n

In 2010-12 I had the opportunity to look very closely at the adoption of Facebook. With that rich data we saw significant differences in adoption probability as a function of graph structure. Paper, "Structural diversity in social contagion", in PNAS: pnas.org/content/109/16… 3/n

The paper examined both Facebook adoption and engagement as a function of contact neighborhood structure. In both cases, "diverse" networks *predicted* (ahem, foreshadowing) much more likely adoption/engagement. Adoption left, engagement right: 4/n

For adoption, "diversity" could be easily operationalized in terms of the number of connected components (gray bands in figure above). For engagement, where neighborhoods were much larger, we looked at the number of "large" components. 5/n

This was all super interesting, and adoption was consistent with causal mechanisms like learning on networks: you discount information from connected sources as redundant. Two isolated sources of info > two redundant sources, etc. But the analysis was far from causal. 6/n

In 2015 I joined Stanford and @5harad and I quickly started talking about the causal story that may/may not underly my structural diversity results. @jessicatysu was headed to Twitter and together with @krishna_kamath & @aneeshs we started to sketch experimental designs 7/n

It's a tricky experiment because you need to randomize people to different contact neighborhoods. One obviously can't randomize actual connections, but with the recommendation system we could randomize what connections were suggested. 8/n

Twitter was already experimenting with recs regularly. There is lots of work on pros/cons of diversity in rec systems; one interpretation of our new work is as a causal study of the long-run effects of diverse recommendations (engagement, not just acceptance of recs). 9/n

There are important details of the design, but essentially we had three separate recommendation algorithms running in parallel, serving low/medium/high diversity recommendations based on the similarity+component structure of the set of recommendations 10/n

Most people pick-and-choose from recs, but it turns out a sizable fraction of users "accept all", taking our slate of 20 recommendations with them (left). Thus, we were able to induce a lasting difference (right) in the diversity of the accounts they follow. 11/n

Interlude: diversity on Twitter means something slightly different than diversity on FB. Diversity in twitter recs often means topical diversity (mixing sports, arts, and professional recs) vs. FB which is more about social circles. 12/n

So what happened to these users who accepted our high/medium/low diversity (l/m/h pairwise similarity) recommendations? After 3 months, their retention rates were nearly identical. No causal effect of structural diversity. 13/n

Now, a very important kink in the design: for many users it was _not possible_ to form high diversity recs. The three treatment populations we studied were a randomization of only those users (12%) that were eligible for all recs (high diversity was the bottleneck) 14/n

Without this conditioning (conditioning on it being possible to serve high diversity recs), across all new users you see a positive relationship between engagement and diversity, consistent with the earlier FB work. 15/n

Going back to the FB findings, basically, the types of people who have high diversity networks are different than the types of people who have low diversity networks. But based on the Twitter findings, if you could randomize their network "keeping them the same", no effect. 16/n

High diversity definitely *predicts* higher adoption/engagement in all these contexts (the Twitter study replicated this aspect of the FB study), but the new twitter experiment suggests that diversity doesn't "cause" engagement. 17/n

This work involved many lengthy discussions about what is being causally tested in this experiment, among us co-authors and with colleagues who were very generous with their time. See the paper for our best efforts to report our findings: arxiv.org/abs/1909.03543 18/n