Friday, September 28, 2007

Virtual user research in Second Life

Just reading a write-up User testing in Second Life by Lisa Herrod at Sitepoint, of a presenattion at the recent OZ-IA confernece in Sydney, User Research in Virtual Worlds from Gary Bunker and Gabriele Hermansson, both from Hyro.

Interesting indeed.

I am a slight cynic of the whole Second Life virtual world model especially when users are flocking to webpage models of social interaction like facebook, mySpace and leaving second life to a slightly faddish geek set at the moment.

I am currently playing around with a few online card sorting tools and think they are great tools for IA validation and wide spread quantative research but not replacements for good live person exercises.

I do not believe good focus groups can be run in a virtual environment as one can not view the side looks, or comments, the physical reponse to a comment from other participants where real value can be produced. It's not what they say but what they do. Valid for actually user testing but equally valid for focus groups and interviews.

The whole point of focus groups is person to person interaction and discussion. The presentation of avatars and digital represenation of discussion items does not truly do this justice.

Also its ridiculous to think that second life represents a wide enough user base for most websites. Second life is well overrepresented by narrow audiences and age groups. We don't even know that an avatar is actually a 45 year old white female, or a 15 year old etc etc.

Real people skills are the key the success in user research. The need to talk and engage with actual users represented as they truly are. We need to move away from technology itself to make technolofgy more friendly and intuitive for users not the other way round.

Labels: , ,


At 12:39 PM, Blogger Gary said...

Hi Nick,

good article. First of all a disclaimer, I am Gary Bunker, subject of the blog.

I appreciate your perspective, as it mirrors (somewhat) my own when we first began to look into this field. I was curious to know whether this could ever work, and so we undertook a measurable research phase in both on and off line forums.

I'd just like to point out that this research was not undertaken with any expectation of a virtual forum replacing real-world testing or groups - it was undertaken to look at extending our toolkit into new arenas. As you quite rightly say, there is no substitute for being in the same room as the user, but there are times when that can't happen, and that's when we need a new tool.

We are often being asked to run research that involves international audiences, and are often limited by the client in International travel - meaning that telephone interviews and/or online meetings/sharing are the only current choices. I was interested in finding out whether virtual forums could provide a richer level of feedback.

The answer was a clear 'yes' - they can, although they are subject to many of the same restrictions and limitations as any offline research, and a few more due to the medium. What was more interesting though, was the high level of feedback we did receive - in fact almost identical to that received in our offline session. This included emotive, interactive and requirement feedback, to quite a high level (although it did take around 1.5 times as long).

If you manage to view my presentation, you'll also see that SecondLife was only a temporary home for our lab, as I personally believe a more open virtual forum for the web will exist sometime soon - SecondLife itself is only a single and limited platform.

Finally, again your point about knowing who is who is well made - but I'll point out the same thing can easily happen in real life too. Although you're unlikely to get a 45 year old man turning up to a real world workshop pretending to be a 21 year old female, you can easily end up with people saying they are expert web users who in reality struggle to open a browser. I have personally sat in on workshops were users who had said they were regular users of a service turned out to have hardly used it at all.

The key to solving this online is the same as solving it offline - blind recruitment processes that ensure the participant has no idea who or what you are looking for. If they cannot see who you are looking for they have nothing to gain by changing their details.

We've found for two recent client-facing projects that SecondLife has, in fact, provided a wealth of participants across quite a wide range of the potential user spectrum, so although I don't believe the future of virtual world testing lies within SecondLife, I do believe it offers a useful option for our toolkit today.

Thanks - again, I enjoyed reading the blog.


At 4:39 PM, Blogger Nick Besseling said...

Thanks Gary.Great reply.

We should definitely be utlising all the tools we can to ensure we are capturing the right amount of quantative and qualatative information to make good decisions and recommendations.

I guess my approach was around Second Life itself which I find overly hyped in many regards.

I think your approach and research into utlising it (and vitual reserach in general) though is something to be commended and I'm sure it will add to many a usability professional's list of 'must reads'.


Post a Comment

<< Home