In a normal competition, calling it a draw might seem like a bit of a non-result. But Tom's Guide's latest speed test of the iPhone 7 and Galaxy S8, which uses GameBench to conclude that the two phones are evenly matched for gaming performance, actually represents a surprising win for Android users. Here's why...
We recently heard some interesting stats for game studios, which came to us by way of a contact at Google:
Improving your Play Store rating from 3.3 stars to 4.1 stars is correlated, on average, with a gain of 150 percent in revenues.
Moreover, one of the best ways to achieve this improved rating is by optimising the speed and fluidity of your game, because 60 percent of four-star reviews mention this particular aspect of user experience.
There are many ways to test a mobile game, from compatibility testing through to subjective testing (i.e., determining whether it’s fun or not). But one aspect of the QA process that is still quite new, and hence doesn't always get the attention it deserves, is performance testing.
The need for performance testing has arisen in response to recent demand for more premium mobile game experiences -- in other words, games that deliver high levels of visual quality and fluidity, and which increasingly do so in combination with other intensive tasks (such as AR or VR, physics simulation or sophisticated AI).
Given the newness of this discipline, I think it’s worth pinning down an essential checklist of six common pain-points which we frequently encounter here at GameBench, and which any meaningful performance test should take into account.
1. Slow or jerky animation
When a studio sets a target frame rate for a game's animation, usually at either 30 or 60 frames per second (fps), it's essential that this target matches the game's genre and that it is achievable on popular devices.
When you’re on a long flight, have you stopped yourself from playing a mobile game in order to conserve your battery? Have you experienced too many stutters when you play a visually intensive title? Do you think it’d be helpful if you could choose a phone to suit your specific needs, based on public data about how well each model handles the best games and apps?
Smartphone and tablet reviewers try to distill their sense of how well a phone performs, but no matter how good their intentions, their conclusions are inevitably subjective and anecdotal. That’s why a market has developed around benchmarking apps like GFX, CPUBench, AnTuTu etc.
Is there such thing as an ‘uncheatable’ benchmark? Cheating isn’t new in benchmarking, as seen with SPEC (for PCs) or Dhyrstone (for embedded processors) from a decade ago. More recently, benchmark wars have resurfaced, given the news around how certain smartphone manufacturers like Samsung and HTC have been rigging the results on particular top end models. It appears that they detect when a benchmark that is running (such as GFX, Basemark, or AnTuTu), and then increase the chipset frequency and temperature constraints to give higher results.