Special Guest post by Gyujin Sim, Mobile QA for netmarble.
There are many uncomfortable experiences users have with mobile games. Frame drops, crashes, overheating and more. These issues deny the user a good experience and naturally result in a decrease in DAUs (Daily Active Users).
Netmarble is committed to identifying and analyzing the causes of these problems for better user experience, and GameBench tools are an important part of this effort.
How do we find the problem?
It’s not hard to discover game performance issues. Our testers will uncover problems just by playing in the same manner and conditions as actual users. But how can we report all the discovered issues during the testing process, along with detailed reproduction steps?
This consumed a lot of testing resources, so we looked for possible solutions and that’s why we started using GameBench.
What’s new?
Using GameBench, we've had a very different experience in rooting out consistent game performance issues.
In the past, we had to profile the engine to try to analyze the cause of the problems found on a game developer’s build, but from the moment we started using GameBench, our testers themselves could identify the likely cause of the problem of during testing.
What caused our problems?
GameBench analysis has shown that memory utilization increases dramatically prior to when some of our games crash. So, in addition to other aspects of performance, we also focused on memory-related optimizations. In this way, we were able to reduce the crash rate by up to 90% after continuing to think about how we manage memory in the game. However, optimization is not yet over. After solving the major problem, other problems were revealed.
New devices are being introduced in the market from a variety of manufacturers. Therefore, when thinking about how to optimize game performance, it is now necessary to optimize based on device characteristics.
What was our other breakthrough?
As we all know, smartphones have been optimized differently by each manufacturer and device, especially in the Android ecosystem. This makes it hard to narrow down the range of hardware that must be taken into account during game optimization.
We’ve been accumulating data by using GameBench across our various games.
In turn, these data are helping us to analyze the characteristics of certain manufacturers, devices, and OS versions separately, as well as providing us with a clue on how to set up better internal testing strategies.
Any producer will agree on the goal - that all can enjoy their products confidently.
Ways to further improve performance
We should do our best to improve user satisfaction with our games, of course. But we should also prepare for a lot of risks that our games won’t work the way we think. Also, depending on the characteristics of the product, we need to think from the user’s perspective and try to identify more issues while testing. If you find an issue, record and analyze using the right tools. Things will vary depending on the game, but if you collect and analyze enough clues, you’ll eventually see which underlying elements need to be optimized to improve performance.
Most important: Never be afraid if you find a problem. Discovering it, logging it and analyzing it is better than letting it show up later.
Netmarble is still trying to optimize game performance and we still run into many problems. There’s nothing wrong with admitting it!
. 