- GameBench Staff
- 25. September 2019
-
0 Comments
It's flagship season here at GameBench Labs, when we get to test the real-world gaming performance of all the latest and greatest smartphones. Huawei has supplied us with the Mate 30 and Mate 30 Pro, and our iPhone 11 Pro Max pre-order has also just arrived, so we're keen to find out which phone is more gamer-friendly in terms of objective metrics. We're additionally including the Samsung Galaxy Note 10 in our benchmarks, even though we've already published our numbers for this phone, because it's another recent flagship that deserves to be in the comparison. For our sample of test games, we've picked some of the world's most popular Tencent titles that should appeal to Chinese and Western gamers alike.
Introducing Touch Latency
As usual, we're going to provide our usual colour-coded badges to highlight our top-level findings and then move on to some hard numbers. What's different this time though, is that once we're done with our normal analysis of visual fluidity (i.e., frame rate), we're also going to rate a whole new aspect of mobile gaming performance: touch latency. Touch latency represents how quickly a phone responds to tap gestures such as "shoot". Frame rate and touch latency together play a big part in determining whether a game feels laggy or responsive, so we're proud to be the first to benchmark both of these aspects of user experience on mobile. If you'd like to learn more about how we benchmark touch latency, you can read more here.
Game Frame Rate Benchmarks
Let's start with our familiar frame rate badges, which highlight the first clear result: both the Mate 30 and Mate 30 Pro stand out as the only devices that achieve Ultra-grade frame rates across all the test games. Both devices maintained a solid 60fps on all titles, producing maximum visual fluidity and minimal graphical lag (i.e., lag caused by the frames failing to render on time). This is a strong result, especially since we put a considerable burden on the devices by testing the games at max available settings.
We tested the hugely popular battle royale shooter, Game for Peace (和平精英), which struggled to maintain 60fps on the Galaxy Note 10 and iPhone 11 Pro Max. We also tested the top MOBAs, Honor of Kings (王者荣耀) and its Western counterpart Arena of Valor, plus the Mario Kart-style racer, QQ Speed (QQ飞车), known as Speed Drifters in the West. These other titles played equally well across the Mate 30-Series and the iPhone 11 Pro Max. The Galaxy Note 10 fell short of an Ultra badge not only Game for Peace but also on Arena of Valor, which is surprising because we've repeatedly highlighted poor performance in this game on Samsung phones and the company has had a lot of time to fix it.
On the other hand, we're aware that our test games are all published by Tencent and this potentially gives Huawei an advantage over Samsung. After all, Huawei has specifically tailored its GPU Turbo technology for top MOBA and battle royale games that straddle Western and Chinese audiences, which inevitably produces a focus on Tencent titles. We'd argue that it's fair to expect good optimisation from all the top manufacturers across all the most popular games (which certainly includes Tencent's), so we don't think our game sample is unfair, but we do urgently need to test more games, including Fortnite and Mobile Legends when Huawei's devices officially arrive in the European market.
Ultra | Smooth | Basic | Poor |
Represents a 60fps experience (or very close to it) with no sacrifices on visual quality. | Represents an experience that at least sticks to the 30fps threshold for good playability; | Represents an experience of at least 20fps | Represents everything below 20fps |
Huawei Mate 30 Pro | Huawei Mate 30 | Galaxy Note 10* | iPhone 11 Pro Max | |
---|---|---|---|---|
和平精英
|
||||
Arena of Valor
|
||||
Honor of Kings
|
||||
QQ Speed
|
(*Note: We tested the Chinese version of the Note 10 in China, and the European / Exynos version of the Note 10 in the UK).
Next, we've got some charts for those who want numbers rather than badges. The first two charts are for Game for Peace specifically, because this game showed the most severe performance differences between devices. One chart shows the median and minimum frame rates (higher is better), while the other shows frame rate variability, which is the typical "jumpiness" of the frame rate (lower is better).
Next we have our aggregated charts, which show why both Mate 30 devices deserve their Ultra badges across our game sample as a whole. In our test sample, they deliver 60fps with fewer and smaller frame rate drops than their competitors:
Does performance matter to your development team?GameBench knows that performance matters to you, that is why we offer an end-to-end testing platform that you can use to gather these same performance metrics for your game and even compare against competing titles. Find out more about how GameBench's performance tools can make your entire team more efficient. |
Game Touch Latency Benchmarks
Huawei recently told us it has evolved its GPU Turbo technology to boost not only frame rates, but also the touch responsiveness of its devices. We've been keen to put this claim to the test using our new touch latency test rig and in general our results lend weight to Huawei's claim -- although we'd caution that only the most competitive gamers are likely to notice the improvement.
We award an Ultra rating for touch latency that is less than 83ms (i.e., up to four whole frames of lag on a 60Hz display). This target is based on current-gen PC and console games like Call of Duty and Battlefield, which consistently deliver this level of responsiveness. We think mobile gaming should aim just as high. So long as a game experience achieves this latency threshold, then we think its responsiveness will satisfy the vast majority of gamers.
However, this isn't easy: the iPhone 11 Pro Max and the Galaxy Note 10 often respond to our test games after five or even six frames of lag. This is illustrated by our touch latency results for Game for Peace, shown in the chart below. The Note 10 took between 88 and 103ms to respond to our shoot gesture (averaging 93ms, or five frames), while the iPhone took between 97ms and 104ms (averaging 101ms, or six frames):
The Huawei Mate 30 devices stand out as being much quicker to respond on Game for Peace, and they consistently manage to achieve our Ultra threshold. The Mate 30 Pro is actually the first device we've tested that goes beyond our Ultra criteria, achieving 51 to 64ms (averaging 57ms) in this game, which is just three frames of lag.
This pattern of quicker touch latency continues across our sample of four test games, as shown in the badge table below and the aggregated numerical results in the chart below that (which shows average latency across all games) :
Ultra | Smooth | Basic | Poor |
Touch latency <83 ms | Touch latency 83-133ms | Touch latency 133-167ms | Touch latency >167ms |
Huawei Mate 30 Pro | Huawei Mate 30 | Galaxy Note 10* | iPhone 11 Pro Max | |
---|---|---|---|---|
和平精英
|
||||
PUBG Mobile
|
||||
Arena of Valor
|
||||
QQ飞车
|
(Note: We had to do all our touch latency tests in the UK. When testing Honor of Kings outside of its native market we found very inconsistent touch latency results-- conceivably because network lag was interfering with touch latency for certain game actions. For this reason, we dropped Honor of Kings from our test sample and replaced it with PUBG Mobile, which produced very consistent results. We tested all games at max available quality at 60fps. In the case of PUBG Mobile, this meant "Smooth" quality and "Extreme" frame rate. In the case of Game for Peace, we were able to test with "HDR" quality and "Extreme" frame rate on all devices except the iPhone, for which this setting wasn't available -- so we tested it on the "Smooth" setting. The iPhone therefore had a slight headstart in its game, but it still fell behind in the end.)
Wrap-up and additional data
Overall, our benchmarks show that the Mate 30 and Mate 30 Pro offer Ultra-grade gaming in terms of both visual fluidity (frame rates) and responsiveness (touch latency). They're the only devices in our sample to achieve this. The iPhone 11 Pro Max comes very close on frame rates, but falls down on latency. The Galaxy Note 10 comes in last for frame rates due to reduced performance in Arena of Valor, but makes up for it with iPhone-beating touch latency, so it still delivers a good experience overall, especially if you're a mainstream gamer who isn't obsessed with having a rock-solid 60 frames per second.
In fact, while our benchmarks reveal some significant differences for hardcore gamers, they also show that all the phones tested here are generally good for playing our test games, with no Basic or Poor badges anywhere to be seen. That's real progress compared to the bad old days of when Android gaming performance lagged far behind iOS.
Finally, it's also important to remember that these are preview benchmarks. Unlike the iPhone and Note 10, our Huawei devices were supplied direct from the OEM because they're not officially available in Europe. As soon as possible, we'll be able to test a public OS build across a wider range of Western games, which should paint a more comprehensive picture. If you register your interest using the form below, we'll send you new benchmark results as soon as they're available and also send you the game-specific results that underpin the charts and tables above.