Thread by @max_mick

A thread on my approach to using data, solving limited formats, and improving technical gameplay
(1/too many)

In my consumption of content and browsing of social media, I have seen many people use data to draw what I feel are unjustified conclusions. Any statistic you choose will have flaws and the key to gaining knowledge from data is understanding statistics and using them properly.

The obvious example is the popular GIH% (game in hand win%). The big flaw here is that cards which go in slower decks which will be biased high. Slower decks see more cards in games they win, making it more likely to draw the card in a game they won than in a game they didn't.

An example is Devious Cover-Up. Despite being a clearly weak card, my GIH win rate must be high. I will only play it if I'm worried about decking, and in those decks, games I win will see every card. While games I lose will be over by turn 8 making it <50% I drew a Cover-Up

I used my data in that example, which leads me to another big flaw: Data is tracked using the average Arena player. If you are taking the time to read this long and poorly written thread you are likely stronger than the average player.

Imagine a hypothetical graph of a card's strength (y) based on skill (x). Different cards will have different graphs, and those with lower slopes will bias high in data, while those with higher slopes will bias low.

This is shown in MID through the high WR of UB data. The archetype has a high skill floor as it mostly involves basic sequencing and using above rate cards. There is also a low skill ceiling, as you have less control than other archetypes over flood, screw, and getting run over.

Ok I've made it clear that I think data fallible, now lets get to the good stuff. What do I use data for and how do I solve formats without relying heavily on data? Data is an excellent jumping off point and way to generate questions to ask yourself. You just have to be careful.

Early on in a format I will use data to try the apparently strong things and get a baseline. Later, if I feel my learning has stagnated, or if I'm having a different experience than the data, I will revisit it to ask myself questions and try to draw general conclusions.

This is how I came to my opinion on UB. I noticed that I wasn't doing well with UB, which struck me as odd considering the data. I asked myself why was I losing? At first, it just appeared to be variance. "Oh I just got mana screwed", "My opponent got super lucky and curved out"

But as I thought more I realized I wasn't having as many of these experiences with my other decks. So I again asked myself why and revisited the data. It turned out that it was the blue cards which were strong, and black was merely the highest skill floor color to pair with it.

I continue to analyze and reflect during the following drafts. They confirm my feelings and theory. I lose to variance more with UB, when the things I'm doing with other Ux archetypes are just as strong in the best case while also being more consistent.

This happens often, and my most profound insights will often come from a process like this. To summarize: play to learn, and ask yourself questions. Use data to generate these questions, or hypothesis to answer them. Don't use only one statistic or rely too heavily on data.

You can also get highly specific and useful information from data, but it takes work and expertise. I don't do this because I find it more efficient and enjoyable to just play magic. However, there are many people post this for free. Use these opportunities for efficient leaning.

This is a simple example I love from @rcsaxe. The graph shows how the data on Festival Crasher is misleading, and creates some questions. Why is this graph that shape it, and what can we learn from it?

https://twitter.com/rcsaxe/status/1445085615658315789?s=20

Two thoughts come to my mind seeing a graph shaped like this. The first is that this is a card which gets much stronger in multiples but is weak on its own. Cards like Mind Sculpt or Squadron Hawk. This clearly seems false which leads us to my second theory:

Festival Crasher is a card with a high skill cap and low skill ceiling. We see this initially negative trend because it is a weak card when not played or drafted around correctly. The data is polluted at low copies by players who do this.

But at high numbers we see a significant upwards trend. The only people who will play that many copies are the ones who are taking the card early and value it highly. These are likely also the players who know how to play with and build around the card correctly.

Of course none of that is provable, I could be wrong on all accounts. Such is the weakness of relying on data and drawing conclusions without justification. But it displays a process for learning from data. You just have to use my theories as a starting point, not a fact.

This thread is way too long, so I'm going to save my process for improving technical play and solving formats for another time. The short version is too ask yourself why you won or lost a game after each game, and ask why your deck performed and felt the way it did.

Essentially, you try to learn what caused a game/deck's result, the way a match/draft felt, or what cards over/under performed. Then you sit on this information until the next time a similar scenario occurs, or you need to answer a question.

This allows you to learn while the information is still fresh, and to answer questions with less time spent. Perhaps I'll make a video with examples, because it isn't easy to describe a process like this. It's somewhat similar to the Socratic Method.

To those who made it to the end, thank you. I didn't plan it to be this long but the words were flowing and suddenly it was 3:40 am. I hope you found this useful and I would love to answer questions or elaborate, so please tell me if something could be more clear.

Some other great follows for this type of content are @Sierkovitz and @carlgoodtoseeu1.