|
|
 |
TREKCORE >
GAMING >
ST ONLINE >
DESIGNER DIARY > Customer Survey Results
PUBLISHED: Nov 29, 2005
AUTHOR: Daron
Stinnett
INTRODUCTION
In September 2005 we contracted with
Nielsen
Media Research to conducted an extensive survey in order to learn
more about our opportunity, characterize our target audience, and test
specific design decisions. Using web-ads placed on sites oriented
towards MMO and PC gamers, Nielsen selected over 600 active gamers who
expressed interest in both Star Trek and MMOs. They then used additional
filter questions sorted participants into three groups of about 200
each:
- MMO gamers: Those who play MMOs frequently
with various levels of Star Trek enthusiasm.
- Star Trek enthusiasts: Those who expressed a
high level of interest in Star Trek and had watched several series and
many episodes.
- Gamers: Those with at least a minimal
interest in Star Trek and MMOs who don't fall into the MMO or Star
Trek enthusiast categories.
The purpose of selecting people who fit into one of these
three groups was to ensure that we were getting feedback from people who
might be interested in Star Trek Online and to understand our
opportunity within various target audiences. We also wanted to ensure
that we were hearing from people beyond those that identify themselves
as Star Trek enthusiasts since we assume their opinion is under
represented within the existing Star Trek Online community forums.
We asked this sample of our potential audience over fifty questions that
covered everything from general game playing habits to demographic data,
with a focus on MMO preferences and feature guidance. Below I'll cover
some of the highlights from the results.
TARGET AUDIENCE
 We
were pleased to learn that MMO gamers with low-to-moderate levels of
enthusiasm in Star Trek, express a strong interest in Star Trek Online.
This tells us that, in general, players believe Star Trek will make for
an interesting MMO experience even if they are not hard core fans of the
franchise. They were followed closely by Star Trek Enthusiasts. This is
very good news because it shows that if we do our job well, Star Trek
Online has the opportunity to capture a significant segment of the
existing MMO audience and has strong potential to bring in new MMO
players from the vast, world-wide Star Trek Fan community. As expected,
general PC gamers came in third place at 30% expressing moderate
interest in a Star Trek MMO. Which given the size of that audience,
again represents a significant opportunity. I should add that these
numbers capture initial interest levels that were assessed early in the
questionnaire before any details about the STO's feature set were
revealed. Overall, we are very pleased with the level of interest
expressed by all three groups and especially pleased that the idea of a
Star Trek MMO scored so well within the existing MMO player community.
FACTIONS
 The idea of Factions for STO has long been a controversial issue within
the team; Starfleet alone makes for a full development plate. On the
other hand, we all recognize the value that factions bring to an MMO. So
while we haven't ruled out the possibility of factions at release, at
this time, they remain off the table. This decision has been in place
for quite a while and hasn't moved. However, we were still curious to
measure the perceived value of factions with this survey. We knew that
given a choice, most players would choose factions. But we wanted to
know if the lack of factions would turn people off. So we tested the
faction concept against the non-faction concept by randomly alternating
the two game concepts in the survey.
-
Non-Factions
Boldly go where no one has gone before!
Join Starfleet and explore the greatest wonders from our star system and
beyond. Find incredible discoveries in some realms - and dangerous
threats in others! Team up with thousands of players as you graduate
from Starfleet Academy, master your skills aboard a starship, explore
strange new worlds, and seek out new life and new civilizations.
-
Factions
Experience Star Trek as never before - as a
Starfleet officer, a Romulan, or a Klingon!
The time has come to choose sides: Defend the Federation as a Starfleet
recruit, outwit your enemies as a cunning Romulan officer, or pursue
honor in battle as a merciless Klingon warrior. Can these ancestral
enemies unite to combat a re-emerging ancient threat, or will the galaxy
plummet into chaos? Join with thousands of players to command starships
in epic space combat, explore strange new worlds, and determine the fate
of the galaxy!
On average, the introduction of factions lead to an 8%
increase in purchase intent for MMO gamers, and 6% for Star Trek fans.
That the difference was so marginal, was a surprise to the team. One
explanation for surprising result is that a Star Trek MMO is appealing
regardless of factions. The other is that factions are just not valued,
which I find hard to believe. IMHO, factions would have a greater effect
than is indicated in the survey, but these numbers certainly don't
support an argument that we should find a way to make factions happen
even if it means sacrificing other goals.
Another surprise was that the Borg is the most appealing opposing player
faction. We knew the Borg would be a popular enemy, but we didn't expect
that actually playing as a Borg drone would be so appealing. Though it
was a surprise, I think we can understand why they would be a popular
player faction. In fact, these results would appear to contradict the
overall conclusion that faction value is marginal. One explanation is in
the wording of the question. We asked people to select their favorite
opposing faction. I can only guess that this wording had the effect of
swaying people away from the Federation. Still, this point clearly
deserves more research.
FEATURES
 We took the opportunity to test interest in some high-level features to
gauge both value and relative importance. We had a series of questions
that asked players to choose their favorite characteristics or features
of a Star Trek MMO game. Most of the features we asked about are planned
in some form or another, faction related questions being the most
obvious exception.
The first question asked responders to pick their five favorite
high-level characteristics of Star Trek Online. We asked this question
after presenting several one-sheet descriptions of STO that each took a
different approach to describing the game. As you can see, starship
related activities dominate the top of the list. We were surprised to
see ground combat rank so low. At first we thought this might be because
it is a feature that people take for granted, but its low rank was
confirmed by the follow-up question that asked people to select their
least favorites with the same list. Again ground combat showed up as a
marginal feature. The away mission is a core component of Star Trek so
it is a bit of a mystery why it ranked so low. My thought is that the
answer lies in the how the feature is expressed. Perhaps if we had said
"Away Missions", the feature would have ranked higher. Of course, this
theory illustrates one of the challenges we face in making a Star Trek
MMO: combat is a basic activity that is central to a fun MMO. Yet combat
is on the periphery of most people's conception of Star Trek. This of
course has been a challenge faced by all Star Trek games. Even with an
MMO, a genre that is ultimately more about community than combat, we
will always struggle to balance the needs of the interactive medium with
the ethics of Star Trek.
 The results' second general feature question also supports the notion
that the space experience is our priority. This doesn't mean that we'll
drop ground combat. In fact, even though ground combat doesn't get a
high ranking on its own, we feel that giving players a chance to engage
in combat as an individual on the ground and to combat in space with
starships is a compelling feature. We think that having both modes of
combat, without making two different games, will go a long ways towards
breaking up the typical monotony of MMO combat. However, these results
make it clear that above all else, making a brilliant space experience
is critical to Star Trek Online's success.
 We also asked potential buyers for their feelings about the solo vs.
crew starship experience, and about combat preference for action vs. RPG
style combat. As you can see from the results, we are on the right track
to offer a starship experience that allows people to choose whether to
captain a crew, crew for a captain, or go it alone, controlling the
entire ship. When it comes to combat style, we were reasonably confident
that we'd made the right decision to move away from FPS style combat,
and the results support that decision. When asked to choose between
action style combat or RPG style combat, 70% chose RPG style combat.
PLAYER RACE
 Another decision we wanted to test was our choice of playable races. We
know that players will want to differentiate themselves and the more
options we can offer, the better. Our current slate of playable
Federation races is Vulcan, Bolian, Human, Trill, Bajoran, Andorian, and
Tellarite. We not only wanted to know which of those would be most
popular (though it wasn't hard to guess), but we wanted to make sure
that we weren't making a mistake to exclude others. As you can see,
players overwhelmingly want to play Humans, Vulcans, and Klingons when
playing as a Starfleet officer. In retrospect, I wish we would have
included ex-Borg and Android as choices in the survey. I'm guessing both
would have been more popular than several of our current choices.
PLAYER CLASSES
 We gave our panel a list of specialties that doesn't exactly correspond
to the choices we plan to offer. As I alluded to in an earlier blog
post, Glen has re-factored the character classes (professions) to give
players the chance to evolve their character's specialty over time
rather than making all the choices up front. I'll let Glen describe the
new system in detail when he's ready, but I will say that the general
idea is to have a hybrid system that combines some up-front choices with
the ability to specialize over time. We'd say that these results tell us
that we're on the right track. More on this topic later...
CONCLUSION
It is not often that a development team has early access to focused
feedback from a wide range of players who might eventually play their
game. Usually, the first time a team is exposed to player feedback is
after it is too late to respond. So we're very pleased with the results
since we're in a great position to shore up our best choices and
consider areas where we don't align with expectations.
And I should add that while consumer research is valuable, we shouldn't
necessarily follow it blindly. In other words, it often provides useful
validation and direcional input but it also should be used as just one
part of a larger decision-making matrix in combination with development
team experience and feedback from the community.
We're looking forward to reading your analysis in the forums.
- Daron
|
