We should talk about Pokémon Go's loot boxes

UPDATE 13/8/20: Detailed player research for Pokémon Go's most recent Dragon Week event is now available, and it backs up fan collective Silph Road's early findings which Eurogamer previously reported on below. As expected, the event's most sought-after Pokémon, Deino, seems to have been extremely hard to find until an unannounced mid-week change in the game's odds made it slightly less rare.

"Deino's hatch rate was likely less than one per cent for the first three days of the event, but after the change researchers recorded a hatch rate above three per cent," Silph Road wrote.

Over on reddit, the game's community is organising a large-scale effort to contact developer Niantic and request it disclose loot box odds to make the game's monetisation more transparent. We'll be keeping an eye on how things go.

ORIGINAL STORY 7/8/20: Pokémon has always been a game of chance: the chance you'll find a particular creature, the chance that creature might be strong in a particular attribute, the chance it might be an alternate ("Shiny") colour. These systems of chance have existed in every Pokémon game. The trouble comes when you start being able to pay for them.

As a free-to-play game, Pokémon Go is largely funded by paying for Pokémon, and over the past four years it has gotten particularly good at enticing fans to spend. It is now the model of a successful live-service game, accruing billions of dollars from its millions of daily players. Last month, several million people bought a £15 ticket to the virtual Go Fest event, raising $10m for charity in the process. Pokémon Go profits for the first half of 2020 were the highest of any year so far.

Day to day, Pokémon Go's two biggest money earners are raid passes (tokens to battle a specific creature you then have an oppurtunity to catch) and egg incubators (a way to hatch or unlock a random creature from a pool of possible candidates). Both items offer chances at strong or Shiny versions of specific creatures. Some creatures are only available from these methods. And both raid passes and incubators are offered freely in small quantities, but can also be bought to receive more goes, more chances.

For now, I want to focus on Pokémon Go's egg and incubator system, which has once again come under fire this week for offering low odds at sought-after creatures - and for not stating these odds openly. In the past few days, fans have been struggling to find a Deino - the headline new creature in Dragon Week, Pokémon Go's latest in-event - and they have widely branded its slim-to-none chance of turning up as the latest example of something feeling unfair with incubators in general.

Pokémon eggs are free to acquire but the creatures within them need to be hatched, or unlocked, by walking. You can hatch one egg at a time using the game's "infinite" incubator, or buy limited-use incubators to stack up to eight more. Finally, the top tier of paid incubator additionally offers the ability to hatch Pokémon at a faster rate.

Niantic has recently begun listing the full contents of each egg pool for particular egg-centric events - seemingly in response for clarity after earlier complaints it had advertised certain rare species as being available, only for the majority of hatched Pokémon to be something else instead. But there's still little to no indication how many eggs you might need for the thing you really want. Fans have to find this out for themselves, likely paying in the process, to pool personal experiences in communities - of which the largest and most reliable is respected reddit fan hub The Silph Road.

For Dragon Week, Niantic notified players what would be in eggs, and suggested some Pokémon might be rarer than others: "The following Pokémon will be hatching from 7km Eggs: Horsea, Dratini, Trapinch, Swablu, and Bagon. Gible and Deino will also be hatching from 7km Eggs, but only if you're lucky!" The release of Deino's Shiny version was also announced. Available for the first time, this alternate-coloured version quickly became the number one thing fans wanted to find during the event.

Niantic's description of something only being possible "if you're lucky" or if you're a "lucky Trainer" is the company's shorthand for it being low odds, though even this is variable (Community Day events which offer vastly boosted chances of finding Shiny Pokémon also state you will be "lucky" to find one). To date, Niantic has never disclosed any numerical odds values.

On Monday, after 266 hatches of 7km Pokémon eggs, Silph Road staff reported a 25 per cent hatch rate for Dratini, and an average of 15 per cent for Horsea, Trapinch, Bagon and Swablu. All of these are common spawns in the wild during the event or common spawns all year round. Just 7.9 per cent of hatches were reported as Gible, a rare event spawn, while zero Deino had been hatched - the event's main spotlight, and not boosted beyond their standard ultra rare appearance in the wild.

Mid-week, around the time Eurogamer contacted Niantic for comment on this article, players suddenly began to report more Deino hatches than before, suggesting odds may have been altered. The current Silph Road data is cumulative - meaning any change to the odds will likely be muddied, stacked on top of the earlier results, but it does appear to support a change being made. (More detailed data is typically published after an event concludes, analysing hatches from before and after a suspected change in the odds - so we're likely to get a more definitive picture over time.)

Today, after 676 hatches of 7km Pokémon eggs, Silph Road reported a slightly lower odds rate for Dratini, a relatively consistent average of 15 per cent for other species, with a slightly higher 10 per cent for the rarer Gible, and 1.8 per cent of hatches as Deino (12 out of 676 eggs total).

As for the Shiny variant of Deino - well, there's no data for its odds other than a few examples it simply exists. Shiny rates for the species are yet to be determined by fans, but from the odds given to other rare spawns it is likely to be at least another 1/80 chance on top of getting a Deino in the first place.

Before we get onto discussing how this fits into a wider definition of loot boxes, it's worth adding more context to the event in general. Dragon Week was billed as a reward for players having completed a certain number of tasks during the ticketed Pokémon Go Fest (though Niantic's global challenge targets have never been set to a level where players failed to complete them).

Deino, an incredibly rare spawn under regular conditions, was chosen as this week's star - and so fan expectations were high that this would finally be a chance to own several of their own, or hunt down its Shiny version. (Though it's worth noting, two Deino were given out to all players via simple tasks at the start of the event so you could at least fill in your Pokédex entry and not have to use incubators to do that.)

Finally, its worth addressing Shiny Pokémon. Yes, these are Pokémon with only a cosmetic difference and no additional benefit to PVP or PVE gameplay. But as a collection game, Pokémon Go has turned the hunt for shinies into gameplay itself. As with Dragon Week, new Shiny creatures are often touted as the standout thing to play for in time-limited events.

Pokémon Go's egg and incubator system has long been the subject of debate. It's not a new gameplay model - it is regularly compared to a loot box and key system - nor are its low odds anything new to Pokémon Go. Looking back at Silph Road research from a couple of years ago, now-common species were offered at rare odds upon their introduction to the game back then, only to become less rare over time. Pokémon Go is driven by the hunt for its latest new and Shiny thing, and while its communities are welcoming, the feeling of a friend getting something incredible only fuels the FOMO further.

Do Pokémon Go's eggs and incubators fit the definition of loot boxes? They certainly fit most standard descriptions. They are consumable virtual items which can be redeemed to receive a randomised selection of further virtual items. You can pay for more chances or to speed things along. Their opening animations, like many other loot boxes, are meant to spark that same dopamine hit.

Players might know the pool of species which can hatch from an egg at any given time, but there are several layers of randomness involved to determine strength, species, or whether something is Shiny, all with differing odds. Finally, fans have pointed to Nintendo mobile games like Mario Kart Tour and Fire Emblem Heroes which openly show their loot box odds and wondered - why doesn't Niantic's Pokémon Go do the same?

The frustration this week has been all about Deino, but there have been several previous fan flare-ups sparked by similar events featuring a time-limited window to hatch other rare creatures - such as Unown, or Shiny-possible Pokémon normally only available in specific regions.

In each of these cases, fans have called on Niantic to publish the odds of hatching specific rare creatures, as other games do which operate similar mechanics. Until this occurs, fans have taken to contacting Apple and Google to report the game - which they say contravenes each store's own policies by not openly stating the likelihood of random chances.

Since December 2017, Apple's developer guidelines have stated the following: "Apps offering 'loot boxes' or other mechanisms that provide randomised virtual items for purchase must disclose the odds of receiving each type of item to customers prior to purchase." Google added a similar statement to its own Play store guidelines in May 2019: "Apps offering mechanisms to receive randomised virtual items from a purchase (i.e. 'loot boxes') must clearly disclose the odds of receiving those items in advance of purchase."

Fans have pointed to countries with stricter policies on in-game gambling, such as Belgium and the Netherlands. In 2018, both declared in-game loot boxes were gambling and therefore illegal, leading to several games removing loot box systems from local versions. Pokémon Go operates as normal in these countries.

Here in the UK, the debate about how we spend money in video games continues to grow. In July of this year, the House of Lords Gambling Committee said video game loot boxes should be classified as "games of chance", and regulated under the Gambling Act 2005 to protect young people from "gambling and gambling-like products". While not a change to the law itself, it was the clearest sign yet that the UK will act in this area.

"If a product looks like gambling and feels like gambling, it should be regulated as gambling," the report concluded. "The government must act immediately to bring loot boxes within the remit of gambling legislation and regulation."

Niantic declined an interview with Eurogamer for this article, and did not reply to a list of follow-up questions.

However, it's my understanding Niantic has noticed the recent fan discussion, and that its developers are now discussing internally how to improve future events in response. (You can use this page to send further feedback to Niantic on this issue, if you feel so inclined.)

Four years in, Pokémon Go has fostered a large and welcoming player community which is clearly happy to pay money to play. But whenever you pay, it should feel fair - and events like these erode satisfaction with the game and trust in its developer, while the continued lack of openness around odds feels like an issue that will never go away until properly addressed. My hope is that Niantic takes this step itself, before it is forced to, for the long-term health of game and to respect its community's call for transparency.