We've just completed a thorough round of game testing on the iPhone 6s, iPhone 6s Plus, and a number of other iPhones and Android phones. For the first time, we've even tested some Chinese Android phones (i.e., phones intended primarily for the Chinese market), to see how they compare.
Some of our key findings are presented below, including charts comparing average frame rates for each device as measured across a sample of popular games based on the Unity engine. Frame rate metrics are widely considered to be the best objective proxy for smooth gameplay and hence a good user experience, and our charts compare three such metrics: the median average, the stability around this average, and minimum (i.e. worst) frame rate observed during gameplay. For more information, you can register to receive our full, free-of-charge report in PDF format:
Download "The performance gap: comparing Unity-based games on iOS and Android"
Top-level charts
First, here are those charts. Please note that a wrench symbol signifies that a device had compatibility issues with at least one game in our sample, such that its average scores are based on a reduced sample of games. The full report provides further clarification.
Comparing iPhones
As you can see, the iPhone 6s and 6s don't much improve median frame rates versus Apple's previous generation. This is because frame rates in iOS games tend to be carefully optimised by developers, such that they stick to either 30fps or 60fps on all current iPhones going back to the 5s and even further. (30fps is a wise choice for certain genres of game, whereas 60fps is often used in games that require maximum fluidity, such as first-person shooters and physics-based titles.) Realistically, this means there simply isn't a huge amount of scope for improvement in new iPhones unless developers start to build their games differently (e.g. always running at 60fps) just for most recent models.
In fact, the 6s devices show a slightly a worse median frame rate than the previous generation. This is largely due to the distorting affect of one game, Lara Croft: GO, which is capped at 30fps on the 6s and 6s Plus, and yet is somehow allowed to run at 60fps on last year's devices. But this is a freak result and overall we don't think the variation in median fps is significant across any of the iPhones -- they all perform extremely well.
On the other hand, the 6s generation does show significant progress in our secondary metrics, with higher minimum frame rates (i.e. less severe bottlenecks) and superior stability. In particualar, the 6s Plus compensates for some of the slow-downs we've observed on the original 6 Plus when playing graphically intensive titles such as Unkilled and Hitman: Sniper.
Chinese Android
The Chinese devices in our sample suffer quite badly from compatibility problems with our sample of games. This is to be expected, since there's not much commercial pressure on Chinese manufacturers to support Google Play Store games, or on Play Store developers to support Chinese phones. However, even on games where there were no compatibility issues, performance was far below that of the Galaxy S6 and HTC One M9, which just goes to show that raw hardware specs aren't everything and optimisation between software and hardware is critical.
Extremes: LG G4 vs. Galaxy S6
The LG G4 has no compatibility issues, but its performance still puts it amidst the Chinese devices towards the bottom of the ranking. This corroborates previous tests we'd done on this phone using a different sample of games, which also showed poor performance.
The Samsung Galaxy S6 was the only Android device to deliver acceptable performance (above 25fps) across all games in our sample, which means that the it remains the best Android phone for gaming that we've tested so far.
Reminder: You can see a full game-by-game breakdown of the results, plus analysis of image quality on different devices, by downloading our full PDF report. You can also download raw screenshots from a number of our games and devices in order to compare image quality for yourself.
Causes and solutions
Our aim in gathering these data wasn't solely to update our previous iPhone vs. Android ranking. We also wanted to provide the industry with a useful snapshot of how healthy mobile gaming is right now in terms of performance optimisation, including explanations of why performance gaps exist between devices and platforms and potential solutions that might help to close those gaps and improve experiences for gamers. To this end, our report also includes key insights from GameBench's recent open(London) event, which brought together hardware and software players (including Samsung, Google, ARM, Facebook, Thumbstar, Natural Motion and many others) specifically to discuss the thorny issue of cross-platform optimisation.
We hope you enjoy the report. And please don't forget -- you can measure frame rates for yourself, using your own devices and your own games, by means of our free Android app. You can also register for beta access to our Desktop Profiler, which allows frame rates and other performance metrics on both Android and iOS devices to be profiled in exactly the same way.
Update - As of early 2017, you can now download our desktop application by visiting - https://pro.gamebench.net/