There are plenty of tools available that let app creators check whether poor connectivity is hurting their users. However, such app-centric solutions have rarely been tailored to the needs of game studios or gamer-focused OEMs, especially now that network-heavy games (including streamed games) have become more popular.
At GameBench Labs, we've now worked on numerous projects for our clients that involve monitoring and rating the network resilience of gaming products, so we'd like to share a few things we've learned along the way.
What does 'network resilience' mean?
Let's start with a bit of necessary semantics. We chose the label network resilience to refer to the ability of a real-world gaming experience to withstand poor connectivity. It's this resilience that needs to be monitored and optimized in order to protect a product from being abandoned by users who have less than perfect connectivity. .
Our choice of label is a little reminder that we are not here to test networks. We are not even trying to test games or phones. Instead, we're all about testing experiences, which are produced by a multitude of elements working together; hardware and software, local and remote.
It's of course essential to break down these different variables and diagnose where problems lie, but this only makes sense once you've captured real-world data and determined whether a real UX problem exists,
Finally, we recognized that network resilience is not a single test, but a broad category that can include many different types of test with many different parameters and thresholds. This is because there are many different ways in which a gaming experience may or may not be network resilient.
Let's take a look at some of these types of network resilience, broken down by where they sit in the stack:
Network Resilience in Games
Tencent's game roster includes some of the most widely played PvP online games in the world. But not all Tencent games are equal when it comes to network resilience. In our tests for example, we've found that PUBG Mobile is more frugal with the amount of network uploads and downloads it requires during gameplay. All else being equal (such as remote server performance), PUBG can survive even the most extreme "bad coffee shop" WiFi conditions. Tencent's newer shooter, Call of Duty Mobile, requires three times as much data and so is more vulnerable to connectivity bottlenecks.
How big is your game's appetite for network data during gameplay? How much of this is under your control, rather than down to the engine you're using? This is something we've been helping our clients to discover in two ways:
- GameBench Pro is our DIY solution that let's you quickly highlight a section of gameplay and look at its network demands and other key metrics such as CPU activity. You can then do the same for a competitor game, or for different builds of your game, to spot important differences.
Is your game's launch time excessive due to slow local CPU processing, or due to network, or both? Is a competitor title more frugal? Does it show a different pattern of network activity that makes it more resilient, perhaps downloading data in smaller, more frequent chunks, rather than making the user wait for one big download? GameBench Pro helps you to answer these questions.
|
TRY GAMEBENCH PRO FREE FOR 7 DAYS |
- GameBench Labs is our service for studios who prefer to outsource the testing and analysis to us. As part of this service, we can test real-world gameplay on your game and benchmark games in a wide range of different simulated network conditions (e.g. 3G, 4G, 5G, WiFi, with different levels of quality etc.), across a range of different iOS and Android devices, to see how different aspects of your game's UX change. We don't just send you raw data. We summarise the data using our color-coded badges, using our crowdsourced data to rate the likelihood that the network resilience of your game will cause users to abandon it.
Network Resilience in Phones, Chipsets and Networks
It's surprising how network resilience changes across different device tiers. For example, with lower-spec phones, the combo of slow CPUs with slow connectivity can cause such long launch times that users are immediately put off. GameBench data is great at revealing such vulnerabilities.
With higher-spec phones, 5G is widely expected to improve gaming not only by enabling game streaming, but also by improving locally-run games through higher bandwidth that allows shorter launch times as well as reduced latency to make these games more responsive. Our own test data suggests that the 5G picture is rather more complicated than that -- we'll share some data when we're ready! -- but the principle that networks can be better or worse for gaming is absolutely true.
A gaming-optimized phone can also show good network resiliency if it is able to switch intelligently to the best available network. Let's say you're playing on a public WiFi network that starts to lose packets. A phone with effective smart-switching will detect that gameplay is being affected and, if the user has turned on this function in the settings, will automatically switch to LTE or 5G or back to WiFi depending on which connection will deliver the lowest latency. GameBench Labs has helped OEMs to test the real-world impact of smart-switching technology, and we are also well-placed to help others, such as chipset vendors and carriers, to demonstrate the power of their own gaming-related technologies.
Network Resilience in the Wild
Finally, while network simulation is the often the only viable option when testing pre-release products, a whole new range of testing and monitoring options are unlocked when you think about post-release testing in the wild. This is something we facilitate through our SDK-based solution: an end-to-end monitoring solution that uses the same underlying GameBench technology and intelligence as our Pro and Labs offerings, but let's you get actionable data at a much greater scale from real-world sessions in real-world environments. If you're interested in learning more about our SDK (currently in closed beta), please let us know!
 Want to get live player data on how well your game runs straight from their devices?Explore GameBench SDK |