Bayonetta 2, review scores, and market meddling

Call me crazy, and I’m sure some will, but that 7.5 Polygon review for Bayonetta 2 (and to a lesser extent, the 6.5 for Tropico 5 for much the same reason) on — by the writer’s own admittance — the exclusive grounds of the depiction of the titular character struck me as odd. I wasn’t angered over the review, especially as I’ve never been a Bayonetta fan and I haven’t played the game in question yet to have judged it myself, but it just seemed odd for reasons I couldn’t quite place.

Well, this morning it struck me. I’d heard that exact score mentioned by games journalist Liana K in a speech hosted by IGDA Toronto. It’s important to note I’m not indicting Liana K in anything, here. I link the video to highlight a named and discussed phenomenon — review scores influence developers’ pay:

…people are basing bonuses on Metacritic…I stopped giving out 7.5’s when I found out it needed an 8 to get your bonus…in that environment, giving a 7.5 is just a dick move.

If you don’t want to take her words for it, it’s a well-documented phenomenon elsewhere and has been for years:

Make of the phenomenon itself what you will — companies in a free-market economy incentivizing the production of a quality product, guilt-tripping critics into inflating reviews — what is important is this phenomenon exists, and is clearly elucidated.

This engenders a certain number of questions. Are certain critics, if they are docking points simply for finding the game’s content distasteful, trying to punish game developers and their publishers by denying them review-based bonuses? If so, is this emblematic of an effort on the part of critics to influence what games are produced and how they produced, contrary to critics’ claims? Is this a valid, or ethical, form of protest against content deemed objectionable by critics and those who produce it?

This, among other reasons, is why I find myself highly skeptical of game critics and their claims with regards to their influence and the gaming industry throughout this controversy, and the circumstances that preceded it. I’m not suggesting Bayonetta 2‘s developers were under such a bonus agreement — I don’t know, and have no information one way or the other — I’m merely illustrating a trend at this point, and my opinion on how we as critical consumers should approach it.

Then, there’s the question do review scores impact what games are made? If so, how do review scores impact what games are made? The games industry is fundamentally capitalist; they do what they do to make money. They make money by selling their products; the more products they sell, the more money they make, and more importantly (given games require massive investments of time, money, and human resources) vice versa.

Whether or not review scores influence sales is a topic that’s been subject to controversy for years; some say yes, some say no, and the industry clearly hedges its bets by saying “yes” regardless (at least enough to incentivize good review scores). Universally accepted is that brand and marketing matter, arguably more than review scores, which I’ll address shortly.

What is more interesting is actually looking at the data.

Those who argue for no correlation base their argument, more or less, on the fact most games (96% as cited by the Venture Beat article) sell less than 500,000 copies, and that games that sell more are statistical outliers. Fair point. This bears out in arguments for a weak correlation as well. Those who argue for correlation point to average sales by rating and general industry trends, and assert the correlation does exist — beyond the 500,000-unit sales threshold.

Before anyone gets up in arms this is “old” data, this Ars Technica article by Kyle Orland written just this year illustrates these trends persist.

What this ultimately tells us is review scores and sales do correlate…but not for the entire games industry, just the bigger players (i.e. double- and triple-A companies). This is easily explained through intervening variables — exposure, brand, marketing, the extreme amounts of resources dedicated to each by double- and triple-A companies (incidentally enabling multi-platform releases, a major driver of sales figures), but more importantly the lack of resources on the part of indies which acts as a limiting factor.

Did I mention, as per the aforementioned articles, the breakpoint for exponential sales increase is 80%? One article even identifies it as “the 80% divide”, and Orland’s article even demonstrates a sales “dead zone” for games that receive ratings in the 70’s.

Suddenly, that Bayonetta 2 — a triple-A game and therefore subject to the correlation between review score and sales — rating by Polygon seems awfully targeted.