Thread by @getpidlit

This map from @chefspencil declaring the "Top Cities for BBQ in the U.S.” has many folks on Twitter upset.

How can Seattle make the list when no cities from Tennessee or Texas grace the map?

🧵Lets take a look at the #data!

#BBQ #dataliteracy #MissingMemphis #Thread

https://twitter.com/JoeBriggsEsq/status/1392813007147880453

There are two key components you should always be critical of when you are evaluating someone’s interpretation of data:
1.Measurement
2.Sample

Link to details about the origin on the map:
chefspencil.com/americas-best-…

Measurement:
Q. First, when @chefspencil created this map how were they measuring the quality of BBQ?

A. TripAdvisor Restaurant Reviews Averages

Sample:
Q. Second, which reviews made it into their sample?

A. Restaurants “offering BBQ on their menu” that had at least 5 reviews in the 75 largest U.S. cities (they then removed restaurants in any city with less than 10 BBQ places).

🧵Reasons why @chefspencil interpretation of this data might be problematic:

1. Measurement Issues: Are Trip Advisor Restaurant Reviews a good way to measure the best BBQ?

2. Sample Issues: Is this a representative sample?

Note: measurement & sample issues can often overlap.

▪️Restaurant reviews are not always about the taste of the food. Folks may rate a restaurant based on their service, price, and even how easy it is to find parking.

▪️Who leaves restaurant reviews on TripAdvisor? Are they the typical BBQ eater?

▪️Could those living in a BBQ-rich cultures have higher standards when it comes to rating BBQ restaurants – skewing their average ratings?

▪️They only included large cities. Maybe the best BBQ locations are further away from a city center?

▪️Does a minimum of 5 reviews for a restaurant count as a good representation of those who have eaten at the restaurant?

▪️Restaurants were included if they offered BBQ on their menu. Does having BBQ as part of a larger menu really make a restaurant a BBQ joint?

To @chefspencil’s credit they did look at other ways of measuring the best BBQ such as cities with the most BBQ places per capita and the cities with largest number of high-rated BBQ places – cities in both Tennessee and Texas made the top-ten on these lists.

All of this is good reminder that people can use the exact same dataset to tell very different stories.