In-Game Surveys: A better way to get feedback

I've taken some cold medicine, and I anticipate some meandering thoughts ahead, so please forgive me. And go easy.

1. They are accessible to everyone with an active account. Due to a myriad of reasons, not all players can or choose to post on the official forums to voice their opinions (ahem, players with "forum bans" or players who simply do not utilize the forums).

Totally agree with you there, the forums, fansites, front page, and even gaming-websites as a whole is just one slice of the playerbase. Most players don't learn about new patches until they log in when it's released. Which probably seems crazy to everyone here, but that's because you're the type of person that would come to a gaming forum, read the posts, and even less likely actually log in and post(!). Most or all of your friends are probably like you as well, because (unsurprisingly) we tend to make friends with people that are like us. It's that projection bias that makes it difficult to understand that there are people out there that are not like us. We love you for being who you are, passionate people coming and giving us feedback is a big reason for why we exist, but we all have to understand (and account for the fact that) most people just don't do that.

Still, I don't think surveys would be as useful as you may think.

05/09/2013 12:46 PMPosted by Celista

2. Surveys allows players to "vote" once per active account on features they would like to see implemented. No more players jumping on alts to voice their opinion for or against a given feature in forum threads!

Surveys can be great on getting individual's perspectives, but it can also be dangerous in creating expectations. As much as some people say they won't get their hopes up, asking people which of three new features they want to see will absolutely incline them to expect the feature they chose to be implemented. Some more than others. And - not to get into it too much, you can find all sorts of papers and articles and such on it - what people say they want and what they actually want are two different things. Sometimes it's interpreted as game developers telling players what they want, and to some degree that's true. It's important as designers to be able to filter things down to their root causes. Sure someone says they want Dance Studio, but in fact what they're really after is ... I don't know, more character customization options, and in fact the Dance Studio is probably a pretty poor way of achieving that, and the development time investment to do it would compromise our ability to ... have unique raid bosses, item tiers, have to cut an entire zone, or etc. etc. Most people don't care to think about dev resource management, and that's good, because it's not very fun.

05/09/2013 12:46 PMPosted by Celista

3. Surveys tied to accounts allows developers a way to analyze a lot of player opinion data very quickly (who does and doesn't like x feature, their pve/pvp participation, subscription history, etc). This way, Blizzard can see what type of player likes which particular features, and can adjust accordingly.

We do have a lot of data already. We know what PvE/PvP participation is, we know how often people are using their farms, we know often often people take talents or glyphs, we know how often people are using specific abilities, we know subscription history, we know how often groups are clearing content, we know queue times, etc. We can anticipate what the majority of players are looking to do, and develop that content. Not too long ago Heroic dungeons were a big thing and a lot of people were upset and wanted more in this expansion, which isn't possible to pull off before the end of this expansion, but we did hurry to put together some challenging small-group content with decent rewards to see if we could scratch that itch with Heroic Scenarios. It will be interesting to see... well first what people think of them, and second how close it gets for people to filling that role, but also just how many people actually try them, considering how loud of an issue that was.

Ultimately though there is no right single way to gather feedback. We use a variety of methods and venues to see and experience how people are playing the game, what they think, and what they want. We're always looking for more, and certainly surveys could be a part of that, but we get so much feedback and data already I'm not sure the pros for surveys outweigh the cons.

We do have a lot of data already. We know what PvE/PvP participation is, we know how often people are using their farms, we know often often people take talents or glyphs, we know how often people are using specific abilities, we know subscription history, we know how often groups are clearing content, we know queue times, etc. We can anticipate what the majority of players are looking to do, and develop that content. Not too long ago Heroic dungeons were a big thing and a lot of people were upset and wanted more in this expansion, which isn't possible to pull off before the end of this expansion, but we did hurry to put together some challenging small-group content with decent rewards to see if we could scratch that itch with Heroic Scenarios. It will be interesting to see... well first what people think of them, and second how close it gets for people to filling that role, but also just how many people actually try them, considering how loud of an issue that was.

I'm sorry, but know what people do (game data) is not the same thing as knowing what the want to do, or what they think of the stuff they're doing. They could be doing something they hate, knowing that it's the only way to progress in the game or something.

BTW: Heroic Scenarios would be fine if you'd let us queue for them, not require us to have pre-made groups. As long as you can't queue for them, they are not even in the same ballpark as Heroic Dungeons.

Cheers.

I was hoping the last couple sentences would kind of cover that. We don't just look at game data. And, in fact, while it's super helpful, a static list of numbers and percentages isn't nearly as useful as hearing how people feel about something. How something feels is far more important that statistical balance.

We use a variety of methods and venues to see and experience how people are playing the game, what they think, and what they want. We're always looking for more, and certainly surveys could be a part of that, but we get so much feedback and data already I'm not sure the pros for surveys outweigh the cons.

Back in BC developers created an entire new raid that was never meant to come to fruition because players surprisingly began clearing Black Temple and were screaming they were bored.

Interesting.

Sunwell Plateau was absolutely in development from the start. Black Temple was supposed to be in the game when BC launched. To keep from delaying it any further quite a few things didn't make the launch game that were supposed to, and were then added in major content patches. Sunwell was always on the feature list from before the expansion was released, and was always an intended raid.

We've gotten a lot tighter with our dev cycles, obviously, but that doesn't change that the vast majority of content you see throughout the life of an expansion is in some stage of development before an expansion ships - or in some cases just couldn't make the initial release. This go around I'd say the one big exception is the Battlefield Barrens event coming in 5.3, which wasn't something that was concepted until around 5.2 when we thought we really needed a bigger story lead up and tie-in to 5.4 as we didn't have one planned. Quests and Scenarios that use existing locations are pretty quick to turn around.