Using VoC To Drive Live Ops Iterations

The games development landscape has changed radically since the days of its inception. The methodology used to be fairly contained and linear. The roadmap began with ideation and ended with the final build, leading to a shipped product. It was very much a closed practice, with any feedback only influencing the next game, not the current, in-development one.

However, the current modern trend of “live” games–where development is a continuous process and new, iterative builds keep being added to the active experience–integrates player feedback into the development pipeline. Yet it isn’t necessarily the vocal player feedback, collected in forms and forums, that influences the iterative design of a live game. Rather, the most important feedback is generated invisibly by players and seen only by developers.

The development shift from static to dynamic has in some ways redirected the balance of power from developer to player base. In no other time have game creators been under tighter scrutiny, as the Internet makes near-realtime reactions a force to be reckoned with. This has resulted in a kind of sea change to the traditional top-down nature of game development, where players were simply fed what the creators and publishers wanted them to experience.

Instead, the planned course of a game’s lifecycle is now partially subject to what players want. Regardless of the ideas planned, the goal is always to keep players playing, so the smart developer pays attention to feedback.

Off-the-cuff

However, this feedback comes in many forms, and not all of them are equally important. Obviously, the first form is social media. What players say about a live game online is typically unfiltered, spontaneous reactions. As such, it can be difficult to winnow out what’s meaningful from what has no value. Generally, an aggregated sentiment can be discerned, a sense of what people like and dislike on a macro scale. This is useful for the first couple of days following a new feature rollout, even if it doesn’t allow for in-depth descriptions of what works and what doesn’t.

Yet not all voices carry equal weight. The range of players runs from first-timer to seasoned veteran and all shades in-between. So, what the newbie says about a game might not be as meaningful as what the hardcore says. It’s incumbent upon the Community Manager to know what remarks are actionable and worth paying attention to.

Careful Consideration

A second form of feedback is what’s communicated directly in a game’s official channels. These can be email forms on a webpage, surveys that players can take, or a moderated forum. The benefit of this form of feedback is that because it takes a little more effort for the player to commit, the sentiment tends to be more detailed and involved, getting to a specific point. Casual players typically don’t leave sentiment here, so its perusal is worth the time.

At the same time, the messages found in these channels can also include technical issues, or problems relating not to the content of a game, but how it plays. Communication of this nature is valuable not only to the developers, who use it to pinpoint bugs, but to other players, who might be having similar issues. A good forum thread, containing both problems and answers, can do the work of a hundred direct phone calls to a support line.

The Invisible Feedback

There’s a third form of feedback that occurs without any direct communication on the player’s part. Arguably the most valuable to a developer, it’s called telemetrics, and it’s collected automatically by the game itself. Telemetrics track every action a player performs, from the UI elements they interact with, to the items they use, places they spend time, and a host of other in-game factors. Perhaps more than any other aspect, telemetrics helps developers understand what’s working and what isn’t.

For example, telemetrics allow a designer to see where players spend the most time on a map. There might be a specific, ideal pathway from one area to another, but telemetrics will show where players might be stuck, lingering in areas instead of pushing through them. This might tell the designer that there needs to be a better visual guidance that shows players where they should be going. This then gets put into the next update.

This is merely one example out of hundreds of ways that telemetrics aid the design and production process. So much data is collected in any one game session that a Data Analyst is required to pore over it all to make sense of what’s valuable and what isn’t. And here we come to the heart of the power of player feedback.

The Tightrope

Design and production choices are vitally important to live games. These choices affect the next iteration of the game, which in turn affects the player experience, and thus, player satisfaction. Player satisfaction can make or break a live game. In some ways, it’s very simple: keep the players happy and your game will continue to make money.

However, these decisions must be backed up by careful consideration of the data; they can never be based on emotions. Players have the leeway to make emotional pronouncements, but developers can’t trust that input. Decisions must be supported by what the data shows the players are doing in a game, versus what they say they want. It may surprise some to learn that it’s not the volume of what players say in forums that causes changes to games, but what the data shows is happening. This discrepancy stems from the fact that players may know that they’re not having fun with a game, but they may not always know why.

It’s akin to walking a tightrope. Players must know that their feedback is being heard, but developers must iterate the design in ways that improves the experience. Simply giving players what they say they want isn’t the answer, because what one player desires may be directly opposed to what another asks for.

In the best-case scenario, a popular player request involves cosmetic changes. A particular skin might be asked for, or a change to the colors of the user interface, or any of a hundred things that only affects the look of a game and not its actual functional content. These changes are relatively simple to design and implement and there is a huge variety of ways to go about it.

However, if it’s a change to the actual mechanical underpinnings of a title that’s required, that is a much greater issue. Such changes can have wide-ranging consequences, such that the very gameplay is affected. This risks alienating the audience, who may prefer the original feel of a game over the newly-implemented changes. There are well-documented stories of games that have fallen by the wayside because the developers altered mechanics too much and lost their existing customer base.

In cases like these, it’s likely that the original design of the game was flawed, or the game was merely mimicking the success of an existing, more popular game, but not adding enough difference for players to make the jump from older to newer. Regardless, proper attention to the telemetrics would have helped inform the right decisions, allowing the game to thrive amongst its competitors.

Conclusion

Modern game developers wield a double-edged sword with their live games. The potential for success, measured by ongoing revenue, is huge, but equally vast is the amount of data that such games generate. Regardless of the content and genre of the game, it’s vital for a company to collect and analyze this continuously-growing volume of data to truly understand what players are doing, to properly manage future updates. A dedicated staff of Data Analysts is the key to whether a live game rises or falls.

As an external developer, having Development, Player Support, and QA all under one roof enables us to drive Live Ops Iteration pretty efficiently. QA teams are the knowledge powerhouse for post- and pre-release problems, workarounds and usability advice for software products. Harnessing such benefits produces a more informed, reactive and efficient PS team. In turn, this complements the ongoing iterations in the development process.

Live games are tricky, and nailing the feedback loop can be the key to the success or failure of your game. We love getting great games to market and making sure they stay fun and relevant post-release. If we can help craft your VoC, Live Ops, or Player Support strategy, please get in touch!