How to determine the win rate percentage of a deck?
General forum
Posted on Oct. 27, 2023, 11:42 a.m. by Elekkos
Hi, everyone!
Recently, I and my friend have created a couple of decks that we want to playtest.
There are 6 decks in total, and what we want to do is get an average of which deck plays better than the other one in certain matchups.
For example: Deck 1 vs Deck 2, Deck 3 vs Deck 4, Deck 5 vs Deck 6, Deck 2 vs Deck 4, and so on...
So, when playtesting two decks, how many matches should we play to determine an average of which one has a higher percentage of winning than the other one?
Should we do something like a Best of 5 / First to 3 number of rounds? Bo7/FT4? Bo9/FT5? More?
Or is there a better way to do it?
Gidgetimer says... #3
It kinda depends on the composition of the decks and if they have sideboards.
The first to 2 match structure is predicated on sideboarding. If these are 60 card 4-of limit decks with sideboards I agree with legendof that playing a match of each player playing against each other deck would give you quick and dirty feel for relative power of each deck. And you could do additional matches for more detailed results and to minimize the impact of variance.
If the decks are 60 card 4-of without sideboards I would start with the same structure for basic testing, but to get more detail you could just play more games so I'd say like first to 5 instead of first to 2.
For larger decks or singleton formats, and especially for formats that are both, You probably need to do a minimum of first to 5. Or, if they include sideboards, first to 3 matched of first to 2 games which is much more time consuming to test.
Most of the time when people talk about match-ups as expected win percentages it is analysis of meta decks that have been played thousands of times against each other with minimal or no changes. Or a rogue brew against a meta deck with some level of theoretical analysis. Two brews against each other is going to take a lot of testing. Unfortunately computers suck at Magic (currently), so there is no way to really automate the testing.
October 27, 2023 8:48 p.m.
wallisface says... #4
Personally to get the bare-minimum (imo) as far as deck win ratios I'd be doing the following for each matchup:
- 5 games pre-sideboard with Deck A on the play and you piloting Deck A
- 5 games pre-sideboard with Deck B on the play and you piloting Deck A
- 5 games post-sideboard with Deck A on the play and you piloting Deck A
- 5 games post-sideboard with Deck B on the play and you piloting Deck A
- 5 games pre-sideboard with Deck A on the play and you piloting Deck B
- 5 games pre-sideboard with Deck B on the play and you piloting Deck B
- 5 games post-sideboard with Deck A on the play and you piloting Deck B
- 5 games post-sideboard with Deck B on the play and you piloting Deck B
This should give a fairly good/definitive representation of how well the decks match up against each other, and ensures the results aren't skewed by differing player skill/piloting between you and your friend. However, that's 40 games per matchup (so 600 games total), which is a LOT and probably more than you'd want to do between two people. You can probably drop each of the above bulletpoints down to 3 games (meaning only 24 games per matchup and 360 total), though I can't see how you'd get very accurate results doing anything less than that.
It would help if you both sideboarded face-up and agreed on what the optimal sideboard plan for each deck was together - as using varying/incorrect sideboard plans will heavily mess with the results.
October 27, 2023 10:59 p.m.
wallisface says... #5
What might be an easier/more-practical option is to add the 6 decklists onto the site and then post the links up to get feedback on. While it won’t be nearly as scientific, a bunch of us here have a good eye for deck viability/practicality, and can provide thoughts on what matchups look about even versus favoured for a particular deck.
The main bonus of this approach is that it’ll save you from spending countless hours playing hundreds of games.
October 28, 2023 1:07 a.m.
Thanks for the reply everyone, and ouch, that's certainly a lot of time investment. Still, it's understandable if you want to get statistically accurate results.
But since we're playtesting 6 decks, which totals to 15 different matchups, playing 100+ matches without some sort of simulator would be a considerable time investment.
In your opinion, what would be a good middle ground of how many matches we should play that is not overly time-consuming, provides some statistical results, and at the same time reduces the luck/randomness factor?
October 28, 2023 11:08 a.m.
Also, since I can't edit my previous comment, I forgot to mention that we would not be side-decking, just playing one deck vs another one, with alternating who's on the play/draw each subsequent match.
October 28, 2023 11:10 a.m.
wallisface says... #8
Elekkos do the decks not have sideboards? Usually the sideboards of decks become one of the larger determining factors in which deck prevails over the other, so i’m surprised you’re not including this. It would make more sense imo to only play post-sideboard, than not at all.
legendofa says... #2
Most matches are best of 3, so I would start there. If you want more tests between two decks, do multiple best of 3 matches.
A starting outline could be to test two decks against each other in a best of 3, then trade decks and play another best of 3. Switching decks removes any issues of skill or strategy preference. Repeat this for every combinations of decks. This will let both players use every deck against every other deck, but it's going to take 30 matches to finish.
More matches equals more information, but also more time and dedication, and something like a deck's 52% win rate against another deck can be hard to tell without dozens of matches between the two decks. If you're just looking for quick and dirty general trends and don't need to be too precise, a best of three round robin will probably get you what you need.
If you want to get really detailed, take notes after each match--were the games swingy, or stalemated? Did one deck take the lead from the beginning? Was there a specific play (or misplay) that decided the match? Did either player ever feel like they could have won if they drew different cards, and would that have actually made a difference?
October 27, 2023 4:19 p.m.