peterme.com

A friend pointed me to a survey that eBay is conducting. Because that survey will likely expire soon, I grabbed screen shots of the survey, because it was a fascinating example of how not to understand user behavior.

The first page of the survey is straightforward enough:

Click to see at full size

They want to get a sense of general satisfaction. Probably good for calibrating responses that follow.

But the second page of the survey is bizarre.

Click to see at full size

They are asking you to
- pretend to be someone you are not
- pretend you're interested in something you are not
- pretend that you did something you have not

And then, after you look at the information shown,
- estimate your likelihood to engage in an action

Hell, even if you WERE interested in digital cameras, this survey is meaningless, because you haven't actually seen these "different models."

There is no way this survey can generate valuable insight. I worry as to what eBay ends up making of the responses.

Don't get me wrong... I love surveys. But surveys are best at uncovering demographic information, and pretty spotty when it comes to predicting behavior. And, if you are trying to predict behavior, you at least need to:

• survey with people who are in the midst of the process which you're interested in understanding (so, in this case, anyone not actively in the market for a digital camera would be screened out)
  
• provide enough context so that reactions can be meaningful (as in -- show those other pages)

Posted by peterme at June 25, 2005 10:54 AM