Thread by @shreyas

There's a lot of confusion on evaluating a candidate’s Product Sense in interviews.

So let’s analyze bad, good, great answers to an interview question:

"Why do threads on Twitter typically get more engagement than a tweet that links to a blog post with the same content?"

👇🏾

First:
I don’t actually use this specific question when I interview candidates, but you do need a question like this *as a part of* your overall Product Sense interview. This question mainly assesses a candidate’s cognitive empathy & ability to analyze product-user interactions.

If you want to learn more about Product Sense & its components, check out this article afterwards:
blog.tryexponent.com/improve-your-p…

Second:
Don’t take this stuff literally. The idea here is to share guidance via an example. Do not misconstrue this stuff as hard rules. Your context matters more than anything else. Also, it’s a good idea to augment your interviews with a practical product exercise or two.

With that let’s review Bad, Good, Great answers to our question.

Again, your initial prompt is:

Why do threads on Twitter typically get more engagement than a tweet that links to a blog post with the same content?

Bad answer:

"The Twitter algo prioritizes threads more than tweets with external links"

Remember that good PM interviews should mirror the types of discussions that you are likely to actually have in the job. In that sense, this is not a bad start. Most candidates will take 2-3 minutes to basically say some version of this answer. Your job is now to prompt further.

Example prompts:

Why might the Twitter algo do that?

Let’s assume a strictly chronological timeline in which the algo doesn’t play any favorites & the same holds true there. What might explain that?

You’re looking for a number of things when you prompt:

Is the candidate able to look at this issue from multiple perspectives, especially in response to your prompts?

Is the candidate asking you reasonable questions to explore the topic?

Candidate's communication skills

Etc.

But after a number of prompts, if the candidate's core message doesn’t move beyond the observation that threads get greater engagement because that’s what the Twitter algo is set up to do, it is almost certainly a bad answer to this question. Let’s now move on to a good answer.

Good answer:

"In aggregate, for a given topic written by a given creator on Twitter, users don’t click on external links as much as they do threads"

This is a decent starting point for a discussion. You might prompt the candidate on why this might be the case.

Note: this discussion will usually require meaningful support from you, but the main ideas should be discovered by the candidate.

Here's what a really good discussion on this "why" might reveal:

1)

"Users know from prior experience that a blog post or an article will take a few seconds to load, so the bar for clicking it is higher for an article than a thread."

2)

"The reading experience is unpredictable with an article (ads / paywalls / badly formatted site), but very predictable with a thread, without any of these downsides. There's also the disparity in the experience between a webview vs. native."

3)

"Related to the above two points, with a thread you can decide very quickly if it’s worth more of your time (e.g. by reading the first 2-3 tweets) than you can with an article. So the *exit cost* for a thread is much lower, and therefore the willingness to explore is higher."

This is good stuff, because it demonstrates strong cognitive empathy and a decent analysis of user-product interactions.

Your job is now to continue prompting on whether the candidate can make this good answer great.

Here's what a great follow up observation might look like:

"Both of these are related to the observation that deep-down, people don’t want to feel foolish. That's why for a given topic, investing up to 10 seconds in opening an article requires a much higher degree of excitement & motivation than investing 2-3 seconds to tap on a thread."

Very few candidates (1 in 5, if you have a decent candidate pipeline) will reach the great level. The point isn't to disqualify anyone who doesn't reach the great level. Rather it's to intentionally identify via your conversation those candidates who *can* reach this level.

To state the obvious here, we aren't looking for just this specific answer. I like to tell candidates that there is no right answer & that I am not looking for them to share what I already have in mind.

So if it helps you, here's another good answer that's different from above:

"Threads have 2 interesting properties:

1) In many threads, each tweet is a self-contained fragment within the overall content. This greatly assists users with comprehension of the overall message (and retention too). The visual separation between tweets provides implicit satisfaction as you finish reading each tweet.

2) In web articles, it is difficult for a user to bookmark (for oneself) & amplify (for others) a specific fragment that particularly resonated with him/her. With threads, these actions are natively built into the tweet itself. So this leads to greater amplification of threads."

Now, this is a very good answer already. How might this answer become great in the ensuing discussion?

Here's an example of what a great follow up discussion might look like:

"Now, stepping back from all of this, there’s some selection bias at play here. The very fact that you are on Twitter, implies that you are already more predisposed to consuming tweets (and therefore also a collection of tweets i.e. a thread) than the average web user."

"So even if articles are a superior, higher engagement content format on the web overall (let’s just make this assumption for now), it is still possible that users on Twitter are more likely to consume & amplify a relevant thread than that same content as an article."

Note: The hypothesis generation above is more important than facts that you may know about threads vs. articles. In other words, the answer above can actually be “wrong”, but that doesn’t make it “bad”. Because in real life you can do analysis to confirm/deny your hypotheses.

What’s important is that a product manager has the user insight, open mindedness, and communication skills to explore a wide range of hypotheses with you, and his/her insights are backed by a solid understanding of user psychology & behavior (i.e. the insights are not random).

Before I end this thread, let's look at another great answer, to drive home the point that the quality of the discussion matters more than any specific answer.

Perhaps a candidate mentions one or more of the reasons we already covered above, and then follows it up with this:

"Okay, besides all this, there’s also the factor of relative usefulness of content on Twitter. Let’s take a user who’s using Twitter mainly for professional learning. Such a user likely sees a ton of tweets that are somewhere between Slightly Relevant to Completely Useless."

"Now, in the midst of a bunch of such tweets, the user comes across a thread on how Zapier acquired customers by getting very creative with content/SEO. This thread is going *appear* higher quality because of its relative usefulness vs. the adjacent tweets in the user’s timeline"

"So, while this point doesn’t directly get to engagement on threads vs. articles, creators who don’t care very much about directing traffic to their website should prolly consider writing threads to take advantage of this disparity in relative usefulness of tweets in a timeline."

Again, the hypothesis presented in this answer can be evaluated via qualitative & quantitative research, so the point isn’t about how accurate it is. The point is that this person made a cogent and non-obvious observation about how users might perceive content in their timeline.

There’s a lot more I can say about good interview practices, questions for other aspects of Product Sense (creativity, design taste, domain knowledge), evaluation criteria, etc. but I’ll stop this experimental thread here. Let me know if this type of content & style is useful.

Back to the top of this thread:

https://twitter.com/shreyas/status/1409247659613835265

I know I've already said that there should not be "one right answer" for Product Sense questions and the interviewer needs to be skilled at prompting, but it's important to beat this dead horse some more.

So I am listing examples of interesting answers that some of you shared👇🏾

#1

My follow up prompt to this answer would be:

What do you think contributes more to the preference for threads - the relatively lower friction to consume them or the expectation of social proof & interactions?

https://twitter.com/awks888/status/1409303483665108992

#2

A couple of folks mentioned this.

My follow up prompt:

Given this, if you were a blogging platform (say Medium), do you see an opportunity to partner somehow with Twitter? How might such a partnership manifest in the Medium & the Twitter products?

https://twitter.com/gummadi/status/1409252845505835009

#3

My follow up prompt:

How much of a role might replies (and the ensuing conversation) play in the eventual engagement that a thread receives vs. other factors such as low consumption friction?

https://twitter.com/Rohanlfcforlife/status/1409261065922244608

#4

My follow up prompt:

Given this, what's your instinct on level of engagement for threads on pseudonymous accounts? Same or less?

(the answer below is a great example of how there are so many different places that a good Product Sense question can go)

https://twitter.com/Rohanlfcforlife/status/1409262056281231365