Mobile game performance pitfalls that studios and QA teams often overlook

There are many ways to test a mobile game, from compatibility testing through to subjective testing (i.e., determining whether it’s fun or not). But one aspect of the QA process that is still quite new, and hence doesn't always get the attention it deserves, is performance testing.

The need for performance testing has arisen in response to recent demand for more premium mobile game experiences -- in other words, games that deliver high levels of visual quality and fluidity, and which increasingly do so in combination with other intensive tasks (such as AR or VR, physics simulation or sophisticated AI).

Given the newness of this discipline, I think it’s worth pinning down an essential checklist of six common pain-points which we frequently encounter here at GameBench, and which any meaningful performance test should take into account.

1. Slow or jerky animation

When a studio sets a target frame rate for a game's animation, usually at either 30 or 60 frames per second (fps), it's essential that this target matches the game's genre and that it is achievable on popular devices.

Read More

The market’s first ‘usability’ benchmark

When you’re on a long flight, have you stopped yourself from playing a mobile game in order to conserve your battery? Have you experienced too many stutters when you play a visually intensive title? Do you think it’d be helpful if you could choose a phone to suit your specific needs, based on public data about how well each model handles the best games and apps?

Smartphone and tablet reviewers try to distill their sense of how well a phone performs, but no matter how good their intentions, their conclusions are inevitably subjective and anecdotal. That’s why a market has developed around benchmarking apps like GFX, CPUBench, AnTuTu etc.

Read More

How to make an ‘uncheatable’ benchmark

Is there such thing as an ‘uncheatable’ benchmark? Cheating isn’t new in benchmarking, as seen with SPEC (for PCs) or Dhyrstone (for embedded processors) from a decade ago. More recently, benchmark wars have resurfaced, given the news around how certain smartphone manufacturers like Samsung and HTC have been rigging the results on particular top end models. It appears that they detect when a benchmark that is running (such as GFX, Basemark, or AnTuTu), and then increase the chipset frequency and temperature constraints to give higher results.

Read More