The PT Aether Revolt Metagame Prediction Game has concluded and our winner has been identified. The response has been quite positive so it’s likely I’ll run a similar game again in the future (though I am running out of random unwanted sealed product to put up as prizes). So with that in mind, I’m going to take a moment to review how this game went from an operational perspective, to guide how I iterate and improve to deliver an even better experience for participants for PT Amonkhet.
1. Picking cards seems far more popular than picking players… why not both?
The prediction game for PT AER was actually the second such survey I ran; the first one was for PT EMN when I ran a fantasy draft of Pro Players. Now the PT EMN Fantasy Draft had a number of advantages over the PT AER Metagame Prediction Game—I put up a more valuable product as a prize for the winner, because the survey was picking players and not cards I could schedule it to be closer to the actual event and ride more of the PT hype, and my posts and content leading up the the game were much higher visibility than the ones for this PT (one of the interactive dashboards I made to provide context for people’s fantasy drafts got a shout-out from the mothership.) That’s a lot of people seeing my invitation to fill out the survey. Nevertheless, even with less visibility, being timed well before the Pro Tour, and a less valuable prize, the PT AER game ended up with over twice the participation of the PT EMN Fantasy Draft.
Ultimately I think the reason here is just that most of the Magic community has a stronger relationship to the cards than to the pro players. Sure, the biggest names on the PT have pretty broad name recognition - your Jon Finkels and Luis Scott Vargas’s. But something I heard many times from my personal friends that I invited to join the PT EMN Fantasy Draft, even ones who were quite into Magic, was this: “I’m not sure if I even know 8 Pros to pick for my fantasy roster!” The fact is, your average Magic player is probably not going to have a very strong opinion on how well some mid-tier Gold-level Pro is going to do in the next PT. On the other hand, everyone who plays Magic can have an opinion on whether or not that sweet card in the spoiler is going to be OP or not. Ultimately even with all its relative handicaps, especially in visibility, the PT AER prediction game it just more accessible to a bigger potential audience.
That said, I don’t want to completely stop running Pro Player Fantasy drafts, because personally I love them. I enjoy following the Pro scene and I still find it more emotionally engaging to root for a player than to root for specific cards. And another advantage of the player fantasy drafts is, unlike the metagame prediction game, it gives me something to run in the week immediately prior to the PT, so it doesn’t necessarily even need to conflict with the metagame prediction game. Honestly, I wanted to do it again this PT, and just couldn’t find the time to organize it.
Ideally for PT Amonkhet there will be two games. One, to be run immediately after set release, for contestants to predict the PT meta. The second, to be run the week prior to the PT, for contestants to predict Pro Player performance. This will be the best of both worlds and keep people engaged.
2. Scoring Structure Could be Tweaked to Reward Non-Obvious Picks
Unlike the PT EMN Fantasy Draft, scoring of the PT AER Metagame Prediction Game was done pretty much straight-up. There were no odds, no handicaps, not many limitations other than the segregation of colored/colorless/gold cards. I didn’t want to discourage people too much from making the “obvious” choices because remember, one of my objectives was to answer the question of just how “obvious” development’s “obvious” mistakes truly are if we don’t have the benefit of weeks and weeks of metagame iteration.
I think this did make the game a little less interesting as a consequence, because there was little incentive to deviate away from these obvious picks and no payoff for correctly identifying an under-hyped card. The way the scoring worked out, you still got enough points for picking Felidar Guardian—a hyped card that actually significantly underperformed its expectation—that it didn’t really hurt your chance of winning. I wish there were some way to reward the lone submission which correctly identified Release the Gremlins as a constructed-worthy card. Over 300 submissions, and only one person saw that Manic Vandal might be constructed-playable in an artifact set! Finding the hidden gem is an accomplishment and for future iterations of these games I’ll try to tweak the scoring to reflect the difficulty of that.
3. Getting Visibility/Accessibility Tricky.
Speaking of visibility and accessibility, both are key in letting people know about the game.
My primary avenue for advertising was just making posts on this blog, which I would then post links to on reddit. There were three posts that I made in all about the PT AER metagame prediction game: one post announcing the game, one post reviewing the entries so far after a few days, and one post after the PT was over wrapping everything up.
The funny thing is how the first two posts both struggled to get any traction at all, with 0 (!) and 6 net upvotes respectively. The most popular post, by far, was the wrap-up post after the PT, which shot straight to the top of the front page the Monday after the Pro Tour and is sitting up 350 net upvotes and counting. Of course, by this time it was too late for people to actually participate and join the game.
So what happened here? Shouldn’t the posts where people can actually, you know, play the game and win a free duel deck, be more popular than the post where I just throw up some charts about an event that has already concluded? Well I think a few things are at work here.
First, there’s just the element of luck. Just like in Magic, sometimes when it comes to social media things just break your way. If you get a few early upvotes that can give you enough momentum to get on the main page and grow from there. If you get a few early downvotes you never make it off the new tab and are buried. I do think there was a bit of a snowball effect here. The third post got a cluster of upvotes pretty much immediately after I submitted it, which definitely helped give it that nudge in the right direction to get things rolling.
Second, though, there’s definitely that element of accessibility again. It’s widely been observed that the Reddit model tends to reward easily digestible content that garners quick upvotes. My recap post was a bunch of charts you can scroll through, go “huh that’s interesting,” and then immediately upvote. This is much more accessible than the initial posts, which are entire surveys which you need to click through, consider your options, and then make selections. By the time you’ve done all that you might not be interested in going back to reddit to give me an upvote.
Here I may personally be somewhat to blame. You may notice that my post announcing the game doesn’t just announce the game - I have to first go on a rant wherein I disparage the collective card-evaluation prowess of the Magic community and defend WoTC RND. Now, this wasn’t a non-sequitor. The point of all that was to advance a thesis I do genuinely believe, and establish the concepts of prediction commitment and testing for which the prediction game could serve as a functional exercise. And I actually thought that framing the game this way would give it a “hook” that would encourage participation—let’s prove this random guy on the internet wrong and show him that I actually am smarter than Wizards RND! But still, this made the entire announcement more difficult to digest.
Third, timing was probably an issue. Because I wanted participants in the metagame prediction game to make their predictions without the benefit of any major tournament results, I needed to schedule it for the week immediately after prerelease, prior to any SCG Opens. At this point, the PT is on the horizon, but the Magic zeitgeist is still very much revolving around cool things that happened at prereleases and release events, not the PT. The recap post had the advantage of going up the Monday after the PT, when the conversation in the Magic community was all about… the PT. So the recap post was far more topical for its time than the game announcements posts.
Fourth, and I don’t want to over-emphasize this as a potential factor because I do believe it’s the least important of the four, there was perhaps a bit of gamesmanship from the game participants themselves seeking to prevent broader participation. Posters of giveaways in other subreddits have often observed how often early voting patterns seem to skew negative on posts where they are giving something away (ahem, like a free duel deck) because lurkers are trying to improve their own odds of winning by reducing the visibility of the giveaway and driving down participation. Magic players can be value fiends with a solid grasp of game theory, so this kind of behavior on /magictcg wouldn’t be inconceivable to me. This theory is made more convincing by the observation that my unpopular 6-point second blog post was actually very similar in content to the eventual popular 350-point recap post; both posts go through charts looking at survey results, but the recap post went up after there was no longer anything on the line.
Still on the whole I think the effect of this is minor, and comes only from a small minority of the community. If a post doesn’t get initial traction, though, that small minority could become the “vocal” minority as the more positive-minded broader community never gets a chance to see and appreciate a giveaway. And in any case, maybe my first post just got a bunch of downvotes because I was ranting about how stupid the community was - that’s not an unreasonable theory either.
TL;DR, My Plans for PT Amonkhet
- Time permitting, run both a prediction game about picking cards and one about picking Pro players.
- Tweak the scoring model to reward people who correctly identify non-obvious/under-hyped choices.
- Keep the lessons learned about getting visibility in mind. Getting traction on reddit is never going to be an exact science, but I can emphasize accessibility and timing to try and do better. Also, maybe don’t announce the contest with an extended rant this time.
Well, I do hope to see everyone to participated this time back next time for PT Amonkhet prediction games. Now I can’t guarantee 100% that I will run them, as this is of course a hobby for all of us, and I just happened to have a lot of spare time at the appropriate periods this time around, but the positive response to the event this time definitely encourages me to continue doing this. (Shoutout to the random stranger who bought me reddit gold!)
As you've seen, it can be difficult to get the word out about these free prediction games. If you're interested in being notified about the next Pro Tour prediction game, leave your email address. I'll only use email addresses collected this way to notify you about new Pro Tour Prediction Games.
No comments:
Post a Comment