Game Design Music and Art

Science of Play – HanClinto

HanClinto

Administrator

Posts: 1828
From: Indiana
Registered: 10-11-2004
How Microsoft Labs Invented a New Science of Play

quote:
Sitting in an office chair and frowning slightly, Randy Pagulayan peers through a one-way mirror. The scene on the other side looks like the game room in a typical suburban house: There's a large flat-panel TV hooked up to an Xbox 360, and a 34-year-old woman is sprawled in a comfy chair, blasting away at huge Sasquatchian aliens. It's June, and the woman is among the luckier geeks on the planet. She's playing Halo 3, the latest sequel to one of the most innovative and beloved videogames of all time, months before its September 25 release.

The designers at Bungie Studios, creators of the Halo series, have been tweaking this installment for the past three years. Now it's crunch time, and they need to know: Does Halo 3 rock?


I think that this is extremely cool -- doing case studies, not only on your own game, but also studying the competing games and trying to make yours better. Amazing stuff.

We did a bit of playtesting for our 2-week game, but not nearly anything quite this extensive.

What are your guys' thoughts on this? Do you think this is a smart thing to do? What would we have to gain from doing similar things with our games? If there was a study done on existing Christian-based games, what would you want it to test?

SSquared

Member

Posts: 654
From: Pacific Northwest
Registered: 03-22-2005
I'll apologize now. This subject is a very passionate and heartfelt topic, so I may get a little carried away here. What the article said is SO RIGHT ON. I'm glad you brought this up.

I'm actually a little surprised this is some new fandangled approach. I would have figured companies were already doing this. This is something I wanted to cover at CGDC if I had given a talk there.

As you can probably tell, I think this kind of testing is so important and I kind of hinted at it in some thread awhile back. This type of approach has been going on for years in the software development industry and I have found it extremely successful in creating easy to use and functional GUI. I recall one time we had a group just look at icons and ask them what they thought each one would do.

Looking at the competition is always a good idea. I did just that recently with a competitor's product. Although our product is better in its results/output acheived, we lose sales because the other product is flashier. I wanted to understand what we can add to give a little more pizazz.

> What are your guys' thoughts on this?

It's an extremely important aspect of the development process. It helps to solidfy the game. To make it more cohesive. To make it easier and (hopefully) more enjoyable.

> Do you think this is a smart thing to do?

Yes.

> What would we have to gain from doing similar things with our games?

By 'our' do you mean our contest games or 'our' as in general Christian-based games? I don't think there is much time in two weeks to get a full understanding of what to do. But as far as games in general, I think we can gain some good gameplay, ease of use in the interface, professional quality, and plain/simple to understand and pick up.

> If there was a study done on existing Christian-based games, what would you want it to test?

The interface. Are things laid out nicely? Does it follow standards? Do people keep hitting "Quit" because it is normally where the "Play" button is? Stuff like that. How do the controls work? Are the controls standard? Are areas too dark to see? What if someone does not have a mousewheel? Can a non-gamer easily get comfortable with it? Is there variety? Is it unique? Is it too unique it requires a large learning curve? The list is endless.

Perhaps if there is interest, I can give some examples by using a specific game and areas where I think a focus group could have made a game even better.

Again, I'm very passionate about this area, so please excuse me if I sound over the top, forceful, or anything like that. Very important topic if you ask me. This is why I was interested in hearing what was discussed during the QA session at CGDC.

samw3

Member

Posts: 542
From: Toccoa, GA, USA
Registered: 08-15-2006
Well, of course we couldn't easily get a one-way-mirror going over the web.. maybe a web cam?

But still, I think this would be a great idea. In fact Microsoft does this for all of its interfaces. Its always good to get a user's hands on the software as early as possible in the process.

One thing that I think is critical is brutal honesty... in both directions. If something really rocks, say so, if its mediocre, say that too, and any recommendations. If it down right stinks.. then you really have to say that as well and try to say why.

On an emotional note, its easy to take things personally as an indy developer. The big game companies simply look at the bottom line and if people turn up there nose, out comes the axe. We, for the most part, pour our souls into our creations. And it can really hurt when what we thought was a good thing is torn to shreds by critics.

Perhaps a way to solve this problem of over-criticism could be having rules where people can ask for comments on only certain areas of the program, and whether they want recommendations or just a critique.

All in all I think its a great idea!

God Bless!

Sam

------------------
Sam Washburn

Check out my CCN SpeedGame 2 Blog

[This message has been edited by samw3 (edited August 22, 2007).]

SSquared

Member

Posts: 654
From: Pacific Northwest
Registered: 03-22-2005
Yes, being honest is really key. Even more reason why the upfront approach is necessary. Why wait for the negative comments/reviews AFTER the game is released? Use the resources you have early on. Get feedback early on. Listen to the feedback. Using a closed, internal group of testers is good, but is not enough. And they are probably generally focused on finding bugs.
CPUFreak91

Member

Posts: 2337
From:
Registered: 02-01-2005
quote:
Originally posted by HanClinto:
If there was a study done on existing Christian-based games, what would you want it to test?

I'll be a bit bold.
I would want to test the "funness", the acceptance by non-christians (and their rating of fun), and the message style: whether or not it throws scripture in your face or does it less blatantly. I'm not saying throwing scripture in one's face is wrong or not fun niether am I saying (EDIT that being blatant about it is wrong.

------------------
All Your Base Are Belong To Us!!! chown -r us ./base
"After three days without programming, life becomes meaningless.'' -- Tao of Programming Book 2

"Oh, bother," said the Borg. "We've assimilated Pooh."

"Socialism works great... if there are no people involved." -- Pastor David Ginter, Union Church of Guatemala.

My Programming and Hacker/Geek related Blog

[This message has been edited by CPUFreak91 (edited August 22, 2007).]

HanClinto

Administrator

Posts: 1828
From: Indiana
Registered: 10-11-2004
Thanks for the discussion -- this has been good.

CPUFreak -- I too am interested in those very things as well. I think that the gospel by itself is offensive enough to those who are against it -- we don't need to make it any more so through a poor presentation.

I wonder what kinds of tests we could do to evaluate the effectiveness of the spiritual communication in a game. Perhaps create a series of cutscenes/levels, and ask volunteers to write what impression each one left them with?

HeardTheWord

Member

Posts: 224
From: Des Moines, IA
Registered: 08-16-2004
This is definitely something that all game companies should be doing. Honestly, I see it as a cheap way to get instant feedback on how your game will compete in the market place. Obviously everyone is going to have their own opinion and in the end it comes down to your best judgement but if most of your testers seem to be in agreement then the "feature" should be axed or kept.

The "hands-off" approach is probably the best way to accomplish this (what Bungie is doing). If you monitor them like a hawk and give them hints then you will never get true feedback.

I also agree that we should be getting non-christians to play our games. A simple survey after play testing could be adequate to get feedback from them and see if anything immediately made them want to quit the game. It might be fun to try this with some of the existing Christian games.

dartsman

Member

Posts: 484
From: Queensland, Australia
Registered: 03-16-2006
hmm... just sounds like a dedicated QA (Quality Assurance) Company. Any decent games company will have a QA Team and gets focus groups to play test the game to get their opinion on the game from non-developers. Still sounds good, but didn't sound like anything more then what most major games companies currently do and have been doing in the past.

I've been doing QA for Rebel Planet Creations whenever they need it for the past year now, I did Beta testing (and QA support) for "The Axys Adventures" and have done QA testing on "Monster Island" (recently announced at CGDC'07 in July).

The main thing I'm looking at is 'is this game fun?', can I pick this game up and play it without difficulty, do I get what I'm meant to do in the game? If it's not fun or sections are difficult I provide feedback on why it isn't fun and suggestions on how we could make it better.

Even though the games have a Christian message, I don't initially focus too much on how the Christian Content comes across. I do keep it in mind however when I play the game, but if the game isn't fun how is the game supposed to get noticed and purchased by players for them to even get a chance at hearing the message.

------------------
jonwarner.net

HeardTheWord

Member

Posts: 224
From: Des Moines, IA
Registered: 08-16-2004
I agree that this is very similar to what a QA team would do. However, most of the team lives and breathes the game on a regular basis. Bringing in other people and asking them to play the game without over analyzing the game is a huge benefit. First off, the players can play the game as they normally would. It sounds like they've tried to create a comfortable atmosphere for testing and monitor the game from a one-way mirror as well as recording video footage of the game.

Another benefit of this approach is seeing what objectives aren't obvious to everyone. The example of the woman who couldn't see the grenades is a great example of the benefits of this kind of testing. Where a QA team who has played the game several times already knows what to expect.

I'm by no means discounting a QA team because they are extremely important in working out bugs and any show stoppers that come along. But unfortunately testing how "fun" the game is becomes a whole different strategy.

dartsman

Member

Posts: 484
From: Queensland, Australia
Registered: 03-16-2006
quote:
Originally posted by heardtheword:
I agree that this is very similar to what a QA team would do. However, most of the team lives and breathes the game on a regular basis. Bringing in other people and asking them to play the game without over analyzing the game is a huge benefit.

Did you miss what I said??

"Any decent games company will have a QA Team and gets focus groups to play test the game to get their opinion on the game from non-developers."

Another point to note is Alpha and Beta testing stages, however these are at a later stage in development.

------------------
jonwarner.net

HeardTheWord

Member

Posts: 224
From: Des Moines, IA
Registered: 08-16-2004
Sorry, I must have overlooked that. I'm sure there are several game companies that do this well and bring in focus groups. Although Microsoft seemed to take it a step further and created a dedicated testing facility.

Sorry if I misunderstood your post.

dartsman

Member

Posts: 484
From: Queensland, Australia
Registered: 03-16-2006
yeah, just funny cause there are plenty of software quality assurance testing facilities around the world already. Seems more of another marketing ploy for Halo 3...

------------------
jonwarner.net

samw3

Member

Posts: 542
From: Toccoa, GA, USA
Registered: 08-15-2006
Micro$oft always seems to "invent" things other people have done for years.

------------------
Sam Washburn

Check out my CCN SpeedGame 2 Blog

SSquared

Member

Posts: 654
From: Pacific Northwest
Registered: 03-22-2005
Hahaha. OK, so most everyone seems to agree. This isn't some new idea.

Now we are talking about two different levels of examination. There is the GUI/Interface which is what I was previously mentioning. Then there is the actual content, whether it is biblical, historical, science, educational, whatever.

To me, without the first one, the content really doesn't matter.

> Perhaps create a series of cutscenes/levels, and
> ask volunteers to write what impression each one left them with?

That is a good idea. But I would take note on how many people are actually watching the cutscenes. I know many gamers who simply don't care about storyline or cutscenes. One friend really liked "Knights of the Old Republic" and yet complained to me about all the reading. And that's an RPG where story is important. So how else do you get that content into the game? Note, I am not necessarily looking for an answer to that. I'm just throwing it out there as a question to ponder. If the cutscenes contain the game's main messages, and people are skipping them, well... I am not trying to diminish your idea. In fact, I think finding out how they react to the cutscenes is actually part of answering your question.

> I wonder what kinds of tests we could do to evaluate
> the effectiveness of the spiritual communication in a game.

I think you may just need to ask probing questions. For example, take "Victory at Hebron", for those who know it. You can ask, "Did you read the cards?" If not, "why not?" If yes, "what did you learn?" "Who did you read about?" "Which cards do you remember?" "Were any of the names new to you?" "Which ones?" "Did you read the good cards?" "Did you read the evil cards?" "Why did you read only the good cards?" if they only read good.

SSquared

Member

Posts: 654
From: Pacific Northwest
Registered: 03-22-2005
I heard of an example of this today on the radio. Thomas Nelson apparently placed three potential bookcovers on their blog and asked people to rate them and give input.

By the way, I also want to point out something. Even though I believe this is a good practice, I also realize the reality of a small Christian developer putting together questions, setting aside a day (or more), perusing through the results, determining the results...it all takes a lot of time and money which may be considered precious.

I actually really enjoy meeting customers and finding out their needs and desires. One of my favorite things is to add a new feature for a customer and seeing (or hearing about) the smile on their face when they try the new feature.