No One Saw Zombies Coming: Community Meta Predictions vs Pro Tour AKH Top Decks

About a month ago I asked the Magic community to join a survey contest to predict the top cards from Amonkhet to show up in Pro Tour. $50 in credit from Cardhoarder was on the line. Notably, the survey rules specified that the more people picked a card, the fewer points it would be worth, so there was strong incentive to speculate on fringe overlooked cards - even if those cards weren't huge at the PT, by being one of the few to pick it you could still rake in points.

(The contest is obviously now closed, but you can review the contest rules here)

I asked everyone to choose 5 monocolored cards. After over 450 submissions, how did the community do? Pretty much a massive punt:

Underperformers

None of the most popular cards - Glorybringer, Cast Out, Censor, Manglehorn, Gideon, or Rhonas - saw enough PT play to justify their popularity on the survey. We need to go down all the way to Magma Spray to find a card that actually overperformed its survey expectations. To be fair we should point out that many survey results came in before the surprise banning of Felidar Guardian, which certainly shook up the expected metagame.

Overperformers

What did overperform? Zombies massively overperformed early expectations at the PT, as the most popular Zombie on the survey, Dread Wanderer, barely had 1/3rd the number of picks of Rhonas. Note that I excluded cards with fewer than 30 survey picks unless they showed up at the PT, so there was an extremely long tail of cards with the same level of popularity as something like Lord of the Accursed.



In fact, only TWO (!!!!!) participants predicted Liliana's Mastery would show up at the PT. As one of the core cards in Zombies decks the card did put up results in a big way. So basically winning the contest came down to whether or not you were one of the only two people who correctly identified that Liliana's Mastery is a constructed-playable card.

I should also mention that the survey closed before the SCG Open on launch weekend began. That tournament showed us that Zombies might actually be a real deck, but prior to that tournament Zombies wasn't really hyped at all.

So if someone tells you that they totally saw the Zombies deck coming... there's a good chance that's just hindsight bias. Very few people actually thought that this casual tribal deck would be pushed to tier 1 with Amonkhet cards - or at least, not enough of them to place a marker and pick up some free store credit from Cardhoarder!

 

Dissenter's Deliverance and Dispossess were other solid playable cards that the survey completely missed, with only ONE participant identifying that Dispossess is actually a pretty solid sideboard option. Even with SB-only appearances being worth only 25% according to contest rules, Dispossess was a ubiquitous enough SB card that it was worth a lot of points to the lone participant who picked it.



Another notable overperformer is Never//Return, considering how down people were on Aftermath cards when it was first spoiled - to the extent that MtGGoldfish published an article panning the mechanic before the complete spoiler was even out! Granted, at least some of that criticism was directed at the visual aesthetic, but aftermath is definitely the sort of design that is stronger than it looks.

I also asked everyone to pick one gold card. Cut//Ribbons performed almost exactly to expectation here, but the notable overperfomers are Wayward Servant, from the WB variant of the Zombies deck, and Bounty of the Luxa, which showed up in some of the successful Marvel builds as a card advantage and ramp engine. Nissa is completely overhyped, but again to be fair the late banning may have impacted the survey here.

What did we learn?

It's easy to feel smug from the peanut gallery, watching the community survey whiff once again on meaningfully predicting the PT metagame. While it is amusing to repeatedly observe just how poorly the community is at evaluating cards and card sets on first impressions, if you are feeling smug, or if you do detect any condescension in my own tone, I assure you that's not my point at all. After all, I can hardly claim to have identified the Zombies deck back when the survey was open either - the ballot I filled out myself for fun was itself full of over-hyped cards like Glorybringer.

Rather my point is this: we should all have more epistemic humility when it comes to initial card impressions. Instead of bashing new sets or cards as under- or over-powered before even playing with them, moderate your evaluations a little bit. Because as we've seen, evaluating cards based on pure theorycrafting, with zero empirical playtesting or tournament results, is really hard! No one is very good at this - not WoTC R&D, sure, but not their critics or the Pros either.

One of the most revealing moments of the PT broadcast for me was the brief interview with Dave Williams, who was in the testing house with eventual PT AKH winner Gerry Thompson, about how their team landed on the Zombies deck. As Dave tells it, they dismissed the deck at first. It was only over the course of their week of testing together that people were gradually "infected" with the knowledge that hey, this deck full of casual tribal rares is actually pretty good!

So if you totally missed on the Zombies deck and bought into the Glorybringer and Gideon hype instead, don't feel too bad. On first impression, the best players in the game were just as ignorant about Liliana's Mastery as you were. But the successful Pros knew not to trust their first impressions, but to subject their assumptions to rigorous testing.

Just something to keep in mind before bashing a card as OP or weak the next time you see a spoiler. Unless you're one of the two people who picked Liliana's Mastery. In that case you can trash talk as much as you want! =P

Wrap-up

Well, that's it for the PT AKH metagame prediction game. I hope everyone who participated had fun. The winners have been identified and contacted by email to collect their prizes.

If you'd like to sign up for future Pro Tour survey contests, leave your email address here!

Pro Tour Amonkhet Fantasy Draft is Live. Up to $80 Credit from Cardhoarder in Prizes!

UPDATE: This contest is now closed and the winner will be contacted soon. If you missed out and want to be notified about the next Pro Tour prediction game, leave your email address. I'll only use email addresses collected this way to notify you about new Pro Tour Prediction Games.

Pro Tour Amonkhet is almost here, and in addition to the usual individual prizes, this year the established pros have the added incentive of playing for team prizes in the Pro Tour Team Series.

In this Pro Tour Fantasy Draft, you will build your own roster of 6 players, each of whom will earn points for winning matches at the Pro Tour. But pay attention - high-ranked players will earn fewer points per win than those who haven't been playing as well lately. Owen Turtenwald may be safe pick, but the rewards would be far greater if a relative unknown - such as first-time PT competitor and 15-year-old Team Cardhoarder associate Jack Kiefer - puts up a breakout performance.

The highest-scoring fantasy roster wins $40 in credit from Cardhoarder/Isle of Cards, and the runner-up will win $10. But thanks to our amazing sponsors at Cardhoarder, there's a bonus! If Team Cardhoarder puts a player in the Top 8, Cardhoarder's going to celebrate by potentially DOUBLING the prize pool!

Read the complete rules and join the contest here!

Game Theory Explains Why WoTC Hasn’t Banned Anything (Yet)

Update the morning after: Well, would you look at that, they went and did it. The community mystification now seems centered around "Why would R&D not ban something only to ban it two days later? Why does 'Look at me, I'm the DCI' seem more and more like an accurate representation of R&D's ban process?" 

And all of the analysis below is just as applicable to explaining why WoTC would not ban on Monday and then ban on Wednesday as it is to explaining why WoTC would not ban on Monday. The key factor, then and now, comes down to their certainty in their read on the metagame, in this case coming from MtGO data. As long as the metagame is uncertain, WoTC acts in fear of landing in the bottom right quadrant. With some certainty that the format would improve from bannings, WoTC can balance the cost-benefit ratio of banning cards against improving the format.

---

Despite being on record as an apologist for WoTC R&D (though my faith was recently shaken by their admitted failure to catch the Saheeli-Cat Combo), I must acknowledge what pretty much everybody knows: Standard is really bad right now. So bad, that people seem can’t seem to comprehend how WoTC could fail to remove some of the problem cards from Standard in Monday’s B+R announcement.

Reddit and social media are filling up with commentary from Pros and average players alike wondering the same thing: what is WoTC thinking? After all, standard is definitely in a worse place today than it was preceding the last Standard banning in January. What gives?

In fact WoTC’s behavior makes perfect sense if we just game out some of the scenarios they’re faced with. Here’s the approximate payoff matrix:

		Would Hypothetical Bannings Create a Healthy Format?
		Yes	No
WOTC ACTION	Don’t Ban	Customers can play with all their cards. Format isn’t as good as it could be. Poor outcome	Customers can play with all their cards. Format is good. Best outcome
	Ban	Customers feel bad about their cards being taken away. Format becomes good. Poor outcome	Customers feel bad about their cards being taken away. Format stays bad. Worst outcome

Granted this is a simplification of the forces at play here, and format health/customer confidence are dimensions that are more of a spectrum and can’t be reduced to binary good/bad. But it still expresses the essential insight: the “Ban” row has a lot of red in it. The absolute worst outcome, from WoTC’s perspective, is to end up in the bottom right quadrant - to incur the very real costs of banning cards while not even getting an improved format out of it. In fact, given sufficient uncertainty about whether or not a format will improve post-bannings, the dominant strategy is to not ban. 

In order for WoTC to be able to ban cards, they need to be confident in their predictions of how the format will change, else they risk ending up in the worst-case scenario. If WoTC has strong belief that hypothetical bannings would create a healthy format, then they will be willing to do calculus of figuring out if the non-ideal outcome of creating a better format while shaking customer confidence is preferable to the non-ideal outcome of tolerating a bad format and preserving customer confidence. Absent this strong belief, the clear choice is to not ban.

Under this model, we can predict that WoTC will be more ban-happy in those formats wherein they can confidently project the future. Non-rotating formats where the format grows by 2% or less with each set are far more stable and predictable than standard, which grows by ~15% per set, and so just as the model predicts, we do see that WoTC is far more assertive with bans in non-rotating formats.

Worst-Case Scenario Realized: The Pre-PT Aether Revolt Ban Announcement

Which brings us to WoTC’s most recent standard banning, the consequences of which we are probably still feeling today: the January announcement that removed Emrakul, Smuggler’s Copter, and Reflector Mage from Standard prior to PT AER.
  
In retrospect, this banning was an unmitigated disaster. Yes, the banned cards were problematic. But WoTC took on the significant cost of banning cards, and then the metagame that followed the banning was somehow worse than the one before it. After Sam Stoddard polled Twitter to rate standard this past winter, it became fashionable to deride KLD as the “4/10 standard.” How fondly we look back now at a standard mediocre enough to be a 4!

While a bad format definitely hurts WoTC’s bottom line, it’s important to emphasize that banning cards has a similar impact, especially in the gateway format of Standard. People who spent their budget for Magic for the next few months on a playset of Emrakuls are just not going to play Magic for a while. Not all of those players will come back. If this were offset by the format becoming more fun and attractive for players, that would be acceptable. But that didn’t happen, and Wizards was stuck with a scenario where they lost both the customer that was turned off by an un-fun format and the one that was turned off by the bannings. At least if they didn't touch the format they'd still be making money from one of those two. This experience has certainly made WoTC more gun-shy about future bans.

What to Expect from Standard B+R Announcements Going Forward

With all of this in mind, we can make some predictions about WoTC’s ban behavior in the future:

1) WoTC will continue to only ban when there is low uncertainty about how the format will evolve. Which means we can expect there to be fewer bannings in standard than in non-rotating formats.
2) WoTC will be more likely to ban cards from standard after there is a clear picture of what the metagame looks like. This means they’re much more likely to ban Standard cards after the the Pro Tour than before it.
3) When it comes to the present situation specifically, I predict a ban will occur in two months if the new metagame remains as clearly unhealthy as it is today. But if it's at all plausible for the format to be interpreted as healthy, WoTC will continue to err on the side of not banning.
4) This last one’s not really a prediction, but I’d bet Goyfs to Seances that MtG’s designers and developers are consumed with jealousy when they look at how digital games can just patch in their latest balance tweaks every week.

Think you can predict what's going to happen in the metagame? If you're right, you could win $40 in Cardhoarder/Isle of Cards credit by filling out the Amonkhet metagame prediction survey.

Picking Against the Crowd - Win the Free Amonkhet Metagame Prediction Game for $40 in Cardhoarder/Isle of Cards Credit

UPDATE: This contest is now closed and the winner will be contacted soon. If you missed out and want to be notified about the next Pro Tour prediction game, leave your email address. I'll only use email addresses collected this way to notify you about new Pro Tour Prediction Games.

Another new set is out and that means it's time for another metagame prediction game. Once again, we are being sponsored by the great team over at Cardhoarder/Isle of Cards, who will be handing out $40 in credit to the person who best predicts which new cards will be seen in top Pro Tour Amonkhet decks, and $10 to the runner up.

This time, I'm adding a wrinkle to the game - instead of simply being rewarded for picking the most popular cards, participants will be rewarding for picking the most undervalued cards. Perhaps the best way to explain this concept will be to take a quick look at the most popular picks in the PT Aether Revolt metagame prediction game.

PT AER - The "Obvious" Picks

Aether Revolt had a lot of hyped cards that seemed obviously good, but as we see in the chart below, the community's card evaluations didn't line up that well with the PT.



Still, the clearly powerful and hyped cards such as Felidar Guardian, Fatal Push, and Shock all did reasonably well, even if in some cases the community overvalued them. The result of that contest was a funny situation wherein the "popular" picks all turned out to be somewhat overvalued by the survey but still did quite well points-wise. Meanwhile, let's consider a less-hyped card:



That Release the Gremlins is a powerful sideboard card in an artifact-heavy metagame seems obvious now with the benefit of hindsight bias, but prior to the PT this card had no hype. In fact, only one person in the entire survey picked it to do well at the PT so it doesn't even appear in the graph above.

Finding the Undervalued Cards

Which brings us to the PT AKH Metagame Prediction game, and how we're going to score it. This time, the points each card is worth will be divided among everyone who picks that card. This means that if you're the only one to identify an unhyped constructed playable card - such as Release the Gremlins - you will reap the rewards. Here's the nitty gritty:

• Amonkhet cards will earn points for appearing in top-performing PT AKH decks (defined as 7 wins/21 points or higher)
• Every deck in which that card appears maindeck is worth 1,000 points.
• Every deck in which that card appears in the sideboard only is worth 250 points.
• To capture the impact of cards that may be format-defining despite not being 4-ofs in their decks (such as Emrakul), a card will receive the same point value whether it is a 1-of, 2-of, 3-of, or 4-of in its decks.
• The points each Amonkhet card earns will then be DIVIDED by all the players who picked that card.
• At the end of the survey you can REVIEW THE SURVEY RESULTS and then GO BACK AND EDIT YOUR ENTRY if you think you've found undervalued cards.
• You can always come back to the survey later to revisit your entry (anytime before it closes at the end on April 29)

This scoring structure means you will need to consider just how strong the "obviously good" cards will be. Sure, Magma Spray is probably going to be a popular card at PT AKH... but just how much more popular will it be than, say, Forsake the Wordly?

  
Think you'll be able to value the cards of Amonkhet better than the collected wisdom of the Magic community? Prove it! =D

UPDATE: This contest is now closed and the winner will be contacted soon. If you missed out and want to be notified about the next Pro Tour prediction game, leave your email address. I'll only use email addresses collected this way to notify you about new Pro Tour Prediction Games.

The Speed of the Metagame - Tracking the Fall of BG Via Metagame Prediction Survey

The Magic Online Championships is in the books, and so is the Prediction Game. How well did the community predict the metagame at the Magic Online Championships when $50 in Cardhoarder bot credit was on the line? Um… not so well.

You can check out an in-depth breakdown of the submissions in my previous blog post, but in short, the community overwhelming picked the BG decks to both be most popular and to win the tournament. Mardu Vehicles came in second, and Saheeli a distant third. And at the actual tournament? The full decklists are available on the mothership, but here’s the archetype breakdown:

Ouch. BG was actually a distant third, at 2 decks split between 2 archetypes, coming behind a solid showing from 4-color Saheeli and an even more solid showing from Mardu Vehicles, which comprised fully half of the field. Mardu was also the choice of all three of the players tied for best standard record: Lee Shi Tian, Marcio Carvalho, and Piotr Glogowski, all at 6-2.

All in all a pretty poor showing from our community prediction survey. It would be one thing if the numbers were close and BG was just barely edged out by Mardu. But Mardu was fully half the field, and even second-most-common Saheeli had double the metagame share of BG. So what happened? Well, let’s look at the breakdown of predicted metagame share of each of the top decks by date of survey submission. Note that this is a 100% stacked column chart, so even though the vast majority of submissions came in over the weekend I’ve normalized the subsequent days to show how the proportions compare over time.

As I mentioned in my previous post, it does appear as if the community was just a step behind the metagame. Most of the submissions came in over the weekend, when GP Utrecht was still in progress and the results from the GP Pittsburgh, at which BG did dominate, were fresh on people’s minds. However, once the results from Utrecht became public on Monday, there was an immediate metagame shift: the “Mardu Ballista” or “Big Mardu” technology demonstrated that Mardu had fresh legs. BG previously was popular because it feasted on a favorable matchup against the “best deck” coming out of the Pro Tour, Mardu. But if Mardu Ballista tech could reverse that matchup - and it appeared from the Utrecht results that it did - then that would leave BG without a place in the metagame, and room for Saheeli to rise to number 2.

As the community digested the results from GP Utrecht, there were quite a few articles publicly available that spelled out the metagame implications. For example, at Channel Fireball, Josh Silvestri wrote an article headlined “B/G is Out, Saheeli is In.” A slightly less high-profile example is my aforementioned blog post, in which I expressed my belief that the survey was a few days behind the metagame’s shift away from B/G and towards bigger Mardu.

The submissions over time graph seems to confirm that it took time for people to recognize the shift. Granted, the sample sizes on some of the later days of the survey are quite small, as we only got a trickle of submissions later in the week. Even so, for what it’s worth we do see a trend towards greater predicted metagame share from Mardu, and decreasing predicted metagame share from BG.

Honestly, dynamics like this hammer home just how quickly the competitive metagame can shift. If you’re still jamming the same BG deck you built after the Pro Tour and GP Pittsburg because it seemed strong then, at this point I think we can safely say you’re a step behind the field. This constant churn even intra-rotation is what makes Standard so interesting from a pure gameplay point of view - but we must admit that this grind can become exhausting or cost-prohibitive for more casual players.

Since this is the MTGO Championships though, I think it’s worth pointing out that the ability to quickly switch decks is one reason why, for all its flaws, MTGO is nevertheless the most cost- and time-efficient outlet for a competitive-oriented player hoping to hone their skills. Swapping cards in real life is process that carries high transaction costs. Either you sell to your local store at half price (to be fair, they need these margins in order to pay the rent), or you binder-grind constantly to recoup as much value as you can from trade partners. In contrast, the online economy makes transactions far less painful as the vendors (such as our contest sponsor, the all-around cool dudes at Cardhoarder) can be far more efficient in their operations. If you wanted to audible from one tier 1 deck to another the day before the PTQ, you can totally do this online and lose just 10% of your deck’s equity just by conveniently selling and buying cards from a vendor. That's setting aside the fact that decks are typically a flat 25% cheaper online to begin with.

I hope everyone enjoyed taking part in the Magic Online Prediction Game, and that the results were instructive. The winners of the contest have been contacted by email.

If you're interested in staying in touch and participating in future free games like this one, leave your email address. I'll only use email addresses collected this way to notify you when I create a Magic Pro Tour Prediction Game.

Community Predictions for the Magic Online Championships

Earlier this week, I invited the paper and online MtG communities to fill out a metagame prediction survey for the Magic Online Championship being held this weekend, with $50 in Cardhoarder/Isle of Cards credit on the line for the most accurate submissions. With over 200 submissions collected so far, let’s take a look at how the community predicts this event to go down.

(Update: some additional submissions came in after the post, but did not significantly affect the numbers below. You can scroll all the way down for a link to a live dashboard with the complete data set.)

Picking Winners

The MOC has an elite 16-player invitation list: the defending champion, the Magic World Champion, 2 players from each MOCS Quarterly Playoff, and 8 players from the MOCS points leaderboard. Note however that in 2016, Gold and Platinum level pros received significant edges in the form of invites to Monthly and Quarterly MOC Events. This system means that the field is a mix of 11 established pros and 5 comparatively unknown online players. The “tiebreaker” questions on the survey asked for predictions on the makeup of the Top 4 and the eventual winner.

Our survey respondents clearly considered strong pros to be favorites to win the tournament, with the 5 non-pro players together combining for less than 1/3rd of the responses compared to incumbent Player of the Year Owen Turtenwald. Brian Braun-Duin also gets a lot of love from the community. This makes a lot of sense as the structure of the MOC, with its small yet elite field, actually closely resembles the Worlds tournament which BBD took down just a few months ago. There was a little bit of drama in the Pro community both about the “legitimacy” of BBD’s invite to Worlds - he earned the “grinder’s invite” as Grand Prix Master - and whether or not BBD was truly the “best” player of the tournament or the beneficiary of some variance, but it appears that in the community’s eyes, at least, BBD is a legit powerhouse.

The position of Marcio Carvalho in the middle of the pack merits additional discussion. Carvalho has been on an absolute tear lately, being the runner-up at Worlds and the current leader in the 2017 Player of the Year race. Judging purely from his results we would expect Carvalho to be a more popular pick. Unfortunately, we need to acknowledge that Carvalho has long been under a cloud of suspicion for sketchy play. Although Marcio’s only serious past sanctions have been a 6-month ban a decade ago and a recent Worlds DQ, there are a number of Pros that continue to “call out” Marcio to this day and publicly accuse him of continuing to cheat.

Of course even his detractors will admit that Carvalho is very good at magic, so why the low ranking? Perhaps the community buys the unproven allegations and is discounting Carvalho’s recent performances, particularly as the Magic Online engine will completely negate any possible cheating. Or perhaps people just don’t feel right picking a suspected cheater to win it all. The sad truth is that Carvalho is either an honest player with some past indiscretions that is unfairly being maligned, or an unrepentant and continuing cheater that is actively stealing from his fellow competitors, and neither scenario is ideal.

Predicting the Standard Metagame

One of the more interesting questions to what extent is the current Standard metagame a “3-deck-meta” of Mardu, Black-Green, and Saheeli Combo as the best Aggro, Midrange, and Combo decks in the format. Here’s how many rogue, ie not-Mardu/BG/Saheeli decks the survey expected to show up:

With a median of 4 or 25% rogue decks, there were slightly less “Standard-Optimists” expecting greater than 4 rogue decks, than there were “Standard-Pessimists” expecting fewer than 4 rogue decks.

As for the decks themselves, people were very taken with Black-Green, with Mardu and Saheeli Combo the picks for 2nd and 3rd most popular:

Not only was BG Midrange the consensus pick to be the “most popular” deck in the tournament, it was also the majority choice for the “most successful” deck of the tournament. These are not at all synonymous, and often the “best” deck of the tournament is the one that “next-levels” and preys on the “most popular” deck. But here, many people thought that - for this weekend at least - the “best” and “most popular” would be one and the same.

That said, I think the survey respondents are at least a day or two behind the latest meta. The survey opened (and the bulk of the submissions came in) over the weekend while GP Ultrecht was still in progress, so they were largely informed by the results of the previous major event, GP Pittsburg. There, the Black-Green decks put up strong performances thanks to a favorable Mardu Vehicles matchup; the GB decks were just a little bigger than Mardu and so could keep up with the fast aggressive vehicles decks and then eventually turn the corner with more powerful spells. But the latest tech out of GP Ultrecht was “Mardu Ballista,” a vehicles variant that itself went slightly bigger still, including an anti-Black-Green sideboard plan of planeswalkers and Fumigate. The latest consensus, from what I can gather, is that this new tech even flips the matchup in Mardu’s favor.

At this point it’s worth reminding anyone who’s filled out the survey that you can go back and edit your submission as long as you log into your google account again. It’s very possible that these responses will change dramatically as people come to appreciate the implications of GP Ultrecht’s results.

We also see some interesting things when we narrow the charts to display only those responses from “Standard Optimists” and “Standard Pessimists,” as defined by their outlook on “number of rogue decks” as mentioned in our earlier chart. Here’s what the optimists think:

The makeup of the “most popular” decks doesn’t actually change all that much - the optimists are actually even more confident that Black-Green will be the most popular deck - but the standard optimists are more hopeful that a non-meta deck such as Dynavolt Tower, Emerge, or some other archetype will be able to attack the format and win.

What about how the pessimists think?

At least from what I see here, I must admit the pessimists appear savvier than the optimists in their metagame predictions. Mardu - which as we cover above is the latest tech - is still second but pessimists almost twice as likely to have picked it to be most popular, and slightly more likely to have picked it to be the winningest.

As for the moderates, no one cares what the moderates picked. Their picks were dull in how closely they matched the survey as a whole. Pick a side, moderates, you’re boring! (Ok, if you’re really interested in how the moderates picked you can scroll down and investigate the complete interactive dashboard for yourself.)

That’s what the community expects from the MOC at least. My personal prediction: the Mardu Vehicles’ “Ballista” tech is very real. I’ve actually seen it at work already in some quite high-level events, and the matchup against Black Green truly seems to be as good as advertised, while the matchup against slower combo and control decks remains quite good. I think it’s about to show up in a big way on one of Magic’s biggest stages. I'm less confident about which deck will actually be the most successful - it could be Mardu Ballista, but there are a lot of very talented players competing this weekend for a large prize pool. The rewards would be huge for any player who successfully identifies new tech. For the most winning Standard deck, I wouldn't be surprised to see some real innovations, either in the form of another novel re-tweaking of one of the big existing archetypes, or a rogue build.

Of course, we’ll need to watch the event itself to see what happens for real. Until then, fill out the survey if you haven’t already to be in the running for the prize, and check out the live dashboard if you want to do a deep dive into the survey data. I will update the dashboard with live data at least once or twice until the event begins and further entries are locked. Hope everyone has fun playing the MOC Metagame Prediction Game!

Predict the Magic Online Championships Metagame and Win $40 in Cardhoarder/Isle of Cards Credit!

The Magic Online Championship being held at WoTC HQ on March 3-5 is a unique event which invites 16 players - some top players from the Pro Tour and some successful online qualifiers - to 14 rounds of grueling Swiss play for a total prize pool $116,000. The select nature of the competition means that making the right metagame call for the 8 Standard rounds in this tournament will give you a huge advantage in taking down the first place prize of $40,000 and instant Platinum status.

Think you've got a read on the meta but somehow missed getting an invite to this 16-competitor tournament? Well, you might not be in the running for the 40 grand, but nail the questions in the FREE metagame prediction survey and you can still get bragging rights and earn $40 credit at Cardhoarder/Isle of Cards! Runner up gets $10 in credit.

Fill out the survey here. Good luck!

Lessons Learned about Running the Pro Tour Aether Revolt Metagame Prediction Game

The PT Aether Revolt Metagame Prediction Game has concluded and our winner has been identified. The response has been quite positive so it’s likely I’ll run a similar game again in the future (though I am running out of random unwanted sealed product to put up as prizes). So with that in mind, I’m going to take a moment to review how this game went from an operational perspective, to guide how I iterate and improve to deliver an even better experience for participants for PT Amonkhet.

1. Picking cards seems far more popular than picking players… why not both?

The prediction game for PT AER was actually the second such survey I ran; the first one was for PT EMN when I ran a fantasy draft of Pro Players. Now the PT EMN Fantasy Draft had a number of advantages over the PT AER Metagame Prediction Game—I put up a more valuable product as a prize for the winner, because the survey was picking players and not cards I could schedule it to be closer to the actual event and ride more of the PT hype, and my posts and content leading up the the game were much higher visibility than the ones for this PT (one of the interactive dashboards I made to provide context for people’s fantasy drafts got a shout-out from the mothership.) That’s a lot of people seeing my invitation to fill out the survey. Nevertheless, even with less visibility, being timed well before the Pro Tour, and a less valuable prize, the PT AER game ended up with over twice the participation of the PT EMN Fantasy Draft.

Ultimately I think the reason here is just that most of the Magic community has a stronger relationship to the cards than to the pro players. Sure, the biggest names on the PT have pretty broad name recognition - your Jon Finkels and Luis Scott Vargas’s. But something I heard many times from my personal friends that I invited to join the PT EMN Fantasy Draft, even ones who were quite into Magic, was this: “I’m not sure if I even know 8 Pros to pick for my fantasy roster!” The fact is, your average Magic player is probably not going to have a very strong opinion on how well some mid-tier Gold-level Pro is going to do in the next PT. On the other hand, everyone who plays Magic can have an opinion on whether or not that sweet card in the spoiler is going to be OP or not. Ultimately even with all its relative handicaps, especially in visibility, the PT AER prediction game it just more accessible to a bigger potential audience.

That said, I don’t want to completely stop running Pro Player Fantasy drafts, because personally I love them. I enjoy following the Pro scene and I still find it more emotionally engaging to root for a player than to root for specific cards. And another advantage of the player fantasy drafts is, unlike the metagame prediction game, it gives me something to run in the week immediately prior to the PT, so it doesn’t necessarily even need to conflict with the metagame prediction game. Honestly, I wanted to do it again this PT, and just couldn’t find the time to organize it.

Ideally for PT Amonkhet there will be two games. One, to be run immediately after set release, for contestants to predict the PT meta. The second, to be run the week prior to the PT, for contestants to predict Pro Player performance. This will be the best of both worlds and keep people engaged.

2. Scoring Structure Could be Tweaked to Reward Non-Obvious Picks

Unlike the PT EMN Fantasy Draft, scoring of the PT AER Metagame Prediction Game was done pretty much straight-up. There were no odds, no handicaps, not many limitations other than the segregation of colored/colorless/gold cards. I didn’t want to discourage people too much from making the “obvious” choices because remember, one of my objectives was to answer the question of just how “obvious” development’s “obvious” mistakes truly are if we don’t have the benefit of weeks and weeks of metagame iteration.

I think this did make the game a little less interesting as a consequence, because there was little incentive to deviate away from these obvious picks and no payoff for correctly identifying an under-hyped card. The way the scoring worked out, you still got enough points for picking Felidar Guardian—a hyped card that actually significantly underperformed its expectation—that it didn’t really hurt your chance of winning. I wish there were some way to reward the lone submission which correctly identified Release the Gremlins as a constructed-worthy card. Over 300 submissions, and only one person saw that Manic Vandal might be constructed-playable in an artifact set! Finding the hidden gem is an accomplishment and for future iterations of these games I’ll try to tweak the scoring to reflect the difficulty of that.

3. Getting Visibility/Accessibility Tricky.

Speaking of visibility and accessibility, both are key in letting people know about the game.

My primary avenue for advertising was just making posts on this blog, which I would then post links to on reddit. There were three posts that I made in all about the PT AER metagame prediction game: one post announcing the game, one post reviewing the entries so far after a few days, and one post after the PT was over wrapping everything up.

The funny thing is how the first two posts both struggled to get any traction at all, with 0 (!) and 6 net upvotes respectively. The most popular post, by far, was the wrap-up post after the PT, which shot straight to the top of the front page the Monday after the Pro Tour and is sitting up 350 net upvotes and counting. Of course, by this time it was too late for people to actually participate and join the game.

So what happened here? Shouldn’t the posts where people can actually, you know, play the game and win a free duel deck, be more popular than the post where I just throw up some charts about an event that has already concluded? Well I think a few things are at work here.

First, there’s just the element of luck. Just like in Magic, sometimes when it comes to social media things just break your way. If you get a few early upvotes that can give you enough momentum to get on the main page and grow from there. If you get a few early downvotes you never make it off the new tab and are buried. I do think there was a bit of a snowball effect here. The third post got a cluster of upvotes pretty much immediately after I submitted it, which definitely helped give it that nudge in the right direction to get things rolling.

Second, though, there’s definitely that element of accessibility again. It’s widely been observed that the Reddit model tends to reward easily digestible content that garners quick upvotes. My recap post was a bunch of charts you can scroll through, go “huh that’s interesting,” and then immediately upvote. This is much more accessible than the initial posts, which are entire surveys which you need to click through, consider your options, and then make selections. By the time you’ve done all that you might not be interested in going back to reddit to give me an upvote.

Here I may personally be somewhat to blame. You may notice that my post announcing the game doesn’t just announce the game - I have to first go on a rant wherein I disparage the collective card-evaluation prowess of the Magic community and defend WoTC RND. Now, this wasn’t a non-sequitor. The point of all that was to advance a thesis I do genuinely believe, and establish the concepts of prediction commitment and testing for which the prediction game could serve as a functional exercise. And I actually thought that framing the game this way would give it a “hook” that would encourage participation—let’s prove this random guy on the internet wrong and show him that I actually am smarter than Wizards RND! But still, this made the entire announcement more difficult to digest.

Third, timing was probably an issue. Because I wanted participants in the metagame prediction game to make their predictions without the benefit of any major tournament results, I needed to schedule it for the week immediately after prerelease, prior to any SCG Opens. At this point, the PT is on the horizon, but the Magic zeitgeist is still very much revolving around cool things that happened at prereleases and release events, not the PT. The recap post had the advantage of going up the Monday after the PT, when the conversation in the Magic community was all about… the PT. So the recap post was far more topical for its time than the game announcements posts.

Fourth, and I don’t want to over-emphasize this as a potential factor because I do believe it’s the least important of the four, there was perhaps a bit of gamesmanship from the game participants themselves seeking to prevent broader participation. Posters of giveaways in other subreddits have often observed how often early voting patterns seem to skew negative on posts where they are giving something away (ahem, like a free duel deck) because lurkers are trying to improve their own odds of winning by reducing the visibility of the giveaway and driving down participation. Magic players can be value fiends with a solid grasp of game theory, so this kind of behavior on /magictcg wouldn’t be inconceivable to me. This theory is made more convincing by the observation that my unpopular 6-point second blog post was actually very similar in content to the eventual popular 350-point recap post; both posts go through charts looking at survey results, but the recap post went up after there was no longer anything on the line.

Still on the whole I think the effect of this is minor, and comes only from a small minority of the community. If a post doesn’t get initial traction, though, that small minority could become the “vocal” minority as the more positive-minded broader community never gets a chance to see and appreciate a giveaway. And in any case, maybe my first post just got a bunch of downvotes because I was ranting about how stupid the community was - that’s not an unreasonable theory either.

TL;DR, My Plans for PT Amonkhet

1. Time permitting, run both a prediction game about picking cards and one about picking Pro players.
2. Tweak the scoring model to reward people who correctly identify non-obvious/under-hyped choices.
3. Keep the lessons learned about getting visibility in mind. Getting traction on reddit is never going to be an exact science, but I can emphasize accessibility and timing to try and do better. Also, maybe don’t announce the contest with an extended rant this time.

Well, I do hope to see everyone to participated this time back next time for PT Amonkhet prediction games. Now I can’t guarantee 100% that I will run them, as this is of course a hobby for all of us, and I just happened to have a lot of spare time at the appropriate periods this time around, but the positive response to the event this time definitely encourages me to continue doing this. (Shoutout to the random stranger who bought me reddit gold!)

As you've seen, it can be difficult to get the word out about these free prediction games. If you're interested in being notified about the next Pro Tour prediction game, leave your email address. I'll only use email addresses collected this way to notify you about new Pro Tour Prediction Games.