The win percentage of a deck or player is a topic we see discussed with extreme frequency in the world of competitive hearthstone.
Whether it’s on Twitch, Reddit or Hearthpwn, we see cries for legitimacy founded upon appeals to the sacred statistic. “Wow!” we exclaim, "your deck MUST be good if it can put out numbers THAT big, against THOSE opponents, in THIS meta!" We assemble for ourselves this earth-shattering decklist, in newly-crafted golden cards (surely a risk-free investment), take to the ladder and… lose. Or at least, we don’t win as much as we expected to – as much as we were led to believe.
Why? In this article I’d like to examine the various factors – relating to variance, skill and environment – that make up any given win percentage, to uncover the selection biases you might have been missing and to encourage a more informed analysis of the ever-ubiquitous win percentage.
Let’s start with the obvious complaint – that of sample size. It’s pretty obvious that a 100% win rate over 1 game is going to give us very little information about a player’s effectiveness with the deck. At the other end of the spectrum, we have Mryagut’s fairly extraordinary 64.8% winrate over 1200 games with oil rogue. As the sample size gets higher and higher, the empirical win percentage will generally get closer and closer to the “true” value – i.e the player’s real chance of winning a given game.
This is, of course, fairly uncontroversial stuff. No one’s going to disagree with the idea that win percentage becomes more accurate with more games played. I feel like the major problem is that the samples are typically just way too small.
Try flipping 50 coins. A coinflip is directly analogous to a deck with 50% winrate – so you can let heads be your wins and tails be your losses. Try flipping 50 coins a few times, and you’ll see how woefully inadequate a sample size of 50 games actually is. You see wild fluctuations in the percentages. How could a 50 game sample ever be accurate when we see fluctuations between 40% and 60% winrates purely due to chance? Try raising the sample size to 100 coins – better, but there’s still a lot of variance here, with 5-10% deviations from the true likelihood of heads (50%) still not uncommon.
Consider how deck-guides come to be. Take the subreddit CompetitiveHS for example, browsed typically by aspirational types ranked between rank 5 and legend. If you’re frequenting a subreddit of this nature, you probably play the game quite a lot. You also probably experiment quite a bit with your decks. You come up with your own builds of the mainstream decks – some are successful, some less so. This is probably a story that applies to many of the thousands of people who browse CompetitiveHS.
Now try to imagine all those people playing Hearthstone in relation to the coin flipping simulator – thousands of people flipping thousands of coins, some winning more and some winning less than others. It is a statistical fact that some of these players will do well, some extremely so. I’m not even assuming a 50% winrate here – most of the players who do achieve these extraordinary winrates over small time periods will be good players with well optimised decks. However, even accounting for their skill, it is inescapable that their performances were the exception, not the rule.
There’s a fairly heavy selection bias going on here – only those with exceptional winrates are going to be sufficiently motivated to make a deck guide. This is fairly obvious. The guy who got rank 1 legend is more likely to write a guide than the guy who only got to top 20, for example. And if your winrate is exceptional, then – yes – you are in all likelihood something of an outlier. You are overachieving, winning way more games than you are supposed to. An exceptionally high winrate, the sort that allows someone to climb to rank 1 legend, is very much the perfect storm. A strong player, a strong deck and – most importantly – a huge amount of luck.
It’s also important to understand that one of these factors in abundance can compensate for a smaller amount of the other two. Not everyone who reaches rank 1 legend is as good as Kolento. In fact, some players who’ve reached the very top are considerably worse than Kolento. Maybe they just got extremely lucky, or perhaps stumbled upon the absolute perfect deck for the meta. Or, as is much more likely, their success was down to some combination of these two factors.
I have it on fairly good authority that the highest ever recorded win percentage in the legend games of any one player over the course of a season is Ostkaka’s 68%. Last season, no one even broke 65%. We must assume here that Blizzard is only looking at players over a certain threshold of games, else the person who went 1-0 in legend just before the season ended would have 100%. It’s fairly telling, though, that for a decent sample size over the course of a season (although variance could not have been eliminated entirely) no one is even coming close to breaking 70%. Yet, community sites are awash with guides boasting 75+% win rates from 3 to legend, or even higher win rates for the player who shot to rank 1 legend in a few hours. The fact is, these win rates are snapshots taken at convenient moments – a near-flawless upswing on a player’s perfect day.
It’s possible the tone of this article was overly cynical. Do I mean to say a win percentage over just 50 games is completely worthless? No. Higher win rates over any sample size are still strictly better than lower ones, and for a true win rate of 50% to fluctuate to 75% over 50 games would be some pretty extreme variance. I would at no point deny that decks that produce extremely high win rates generally have something about them, and I’m no different to anyone else in my desire to explore this and try them out for myself. What I am encouraging, however, is a more holistic understanding of the selection process that leads to these deck guides being published. You should understand that you would be pretty fortunate to reproduce the writers’ streak, and that this is for reasons other than just their supposed higher skill, or a shift in the meta.
So, that new deck you just crafted. Is it good? Maybe, who knows. Are you about to go 80% to legend? Almost certainly not.
