Experiment One | Invitation to join the online experiment
February 1st, 2007 by Kevin Capota
The time has come to present the online version of my first experiment. In the mean time I’m planning the experiment in a lab setting for a second group of psychology students from the University of Twente. This experiment will start next Tuesday (the 6th).
I’d like to invite you in joining the experiment. It will take approximately 20/25 minutes of your time and you have the opportunity to win a copy of the Designing Interactions book by Bill Moggridge. It’s a great book with interesting interviews and nice pictures; definitely worthwhile.
I hope you will take the time to participate in the Dutch or English version of the test.
Thanks in advance!
Oh, and in case you missed the link to the test… HERE it is again.
Hi Kevin,
some feedback on the test:
a) Aligning the two first scales would help the test subjects.
b) The last web page (after the bule/yellow page) was not shown in my test
Cheers and good luck!
_Wolf
Hi Wolf,
Thanks for your comments and participating - the problems have been fixed.
You were shown one extra screen (a blank screen), so you haven’t missed a thing.
Regards,
Kevin
Hi Kevin,
First of all I want to thank you for sharing your process of research.
A couple of loose thoughts.
I don´t know if you have but if you haven´t I recommend you read “Nisbett R E, Wilson T D 1977 Telling more than we can know: Verbal reports on mental processes Psych. Rev. 84 : 231-259.”
I am not sure about asking the participant to complete statements like “I feel…”
Have you thought about using some kind of variant of the “Think aloud protocol”? Or maybe something like the Google Image Labeller http://images.google.com/imagelabeler/ or the similar ESP Game http://www.espgame.org in this way you can prevent the participant from elaborating too much.
Also I wonder if the introspection required from the participant in order to select the option that fits the way she feels about the website’s aesthetics influences the very process of appraisal. I think it might. Take a look at “Thinking too much” by Wilson & Schooler, I know it is not the same case, but it might be relevant to your study http://content.apa.org/journals/psp/60/2/181
Good luck and keep up the good work!
Hi Ariel,
Thanks a lot for these comments.
Concerning having the participants evaluate their emotions with the question “I feel…”, you are probably right that it may influence the appraisal process but we still made the decision to include it in the test. We needed to find out if there were emotion words ‘missing’. Perhaps there are indeed better methods to do so, but for this first experiment we thought it should work well.
The other options you’re providing seem very interesting and I will surely take them into account when developing the actual LEMtool.
Kevin
Hi Kevin,
Interesting test! A couple of commments.
Some of the emotion words might not be understood by all the people doing the test. Not everyone knows what “avaricious” and “quiescent” mean, for example.
During the “I feel” questions, I wasn’t sure whether you were asking me how I felt, or what I felt the website was projecting… maybe it’s just me…
Have you considered the fact that images are being compared to one another? After going through all the sites, I wanted to go back and change some of my answers, as I had begun the test by rating things somewhat conservatively (I think its called “edge avoidance”). Presenting the sites in a random sequence would help adjust for this.
A comment: I know you were probably trying to capture the “first impression” but I wanted to be able to enlarge the small image of the website to look again.
Good work!
Hi Trevor,
Thanks for your questions. I really appreciate it that you (all) take the effort to provide feedback. It’s the first experiment I performed, so I guess there are some points of discussion. But then again… there always will be.
Now, to try and answer your questions.
Some words may indeed be less familiar, but one goal of the test is to filter the emotion words that are less used or relevant for website measurements. If people do not know the word they will probably rate it as ‘not at all’.
The “I feel” questions are always tricky and exactly for the reason you said: are they object related or feeling related? I meant them to be feeling related and tried to ask the question in such a way that I hoped it would be clear. But don’t worry… it’s not only you, I’ve had simular questions from the participants in the lab setting
Yes, the impact of the images can be very significant in multiple ways. I recall a test where they even took a five minute break after each picture was shown to make sure the image is ‘erased from memory’. I didn’t think that would be necessary and presented the sites in random order to account for these problems.
I’m not really trying to capture the ‘first impression’. If I wanted to do that I wouldn’t have presented the small image of the website next to the questions. My goal is to measure the emotional appeal of the aesthetics of the website. The first impression is shaped within milliseconds, as you can read in my post about Lindgaard’s research. But emotions may occur only after a reflective evaluation, that’s why the site is shown for longer than milliseconds (10 sec). I don’t think you can speak of a first impression after that amount of time.
The small image is meant to remind you of the colours and layout of the website (part of the aesthetics experience). If I offer the opportunity to look longer (or again) at the full-size webpage I think participants will tend to read the text on the site and perhaps shape their opinion. I wanted to leave the influence of this content out of the research, that’s also why they’re all university websites. The influence of content seems very interesting and I hope I can say more about this after testing the actual LEMTool (somewhere in April) which will include an evaluation about content.
Kevin
btw thanks for linking the study on your site
Hi Kevin,
interesting experiment. I had a hard time answering the ‘I feel…’ question. Also, the rating scales you use seem to be very non-innovative in measuring emotional responses to visual design of websites.
What I do like was the approach of only showing the image for 10 seconds. That way, people do not get too tied up in looking what is on the page as opposed to how the page as a whole looks.
-Arne