User Testing

 

I’ve recently been asked how to do user testing, and I thought that this would be a great topic for a blog post. Here are a few things to bear in mind when preparing for and then doing user testing. User testing is definitely a skill you refine with practice. The purpose of user testing is to find what influences participants as they test your design.

The great point about doing user testing face-to-face with participants is that you can find out why your participants have behaved and responded a certain way, you can find out their reasoning. It allows you to delve deeper into the thinking and behaviour of your participants and ask follow-up questions. This is the opposite of using on-line testing services where you send through your list of questions/tasks to be tried out on participants that they find for you.

 

Background Research

 

• Gather relevant background information

Gather background information on the topic and people you’ll be interviewing. Make sure you understand both the people who will use your designs and the situations in which they will be used. Also, find analytics providing evidence as to how the site is being used, for example data as to who are the site users (the audience), where site users leave the current site (if there is one) to find possible problems with the site design, and to find out how they use the current site.

 

 

Preparation for the User Testing

 

• Ascertain the goals for the user testing

Create a test plan – you’ll need to ascertain the goals and from these create the tasks and questions to ask the participants. The goals should include what you want site users to achieve when using your design. It’ll be helpful for your later analysis to have a rating for the participants possible answers to the tasks and questions.

• Be flexible with your test plan

It’s good to be flexible with your test plan as it’s likely the participant will highlight areas of further exploration of which you were unaware. Be flexible with the goals/areas you want to cover. You’ll get more from this flexibility as it’ll let you ask participants questions on areas they raise that are missing from your test plan. This flexibility will also aid you by not having to spend time covering an area if you’ve already found the participant can clearly understand that aspect of the design.

• Carefully word your questions

Avoid ‘target’ words so as not to influence the participants answers. Also, don’t give too much detail that you influence the answer. You’ll also need to make sure you don’t focus too narrowly on their response.

• The context of the user testing

At the start of the sessions I ask basic factual or demographic questions e.g. age, location, occupation etc so there’s a context for the interview. These questions will vary depending on the purpose of the interview.

 

Carrying out the User Testing

 

• Icebreaker questions

I ask a couple of icebreaker or warm-up questions so the participant feels at ease and starts talking. If they feel at ease then you’re more likely to have answers that are accurate and less likely to be about giving a good impression.

• Be aware of …

You’re not just listening to the participant’s answers, you’re also observing their body language such as their facial expressions, as well as their pauses.

• This is not the opportunity to give your own views

Be an active listener. Avoid saying what you would do, you want to know their views and thoughts and you don’t want to affect their answers.

• The way to approach the user testing

Be friendly. You want to make the participant feel at ease. At the start I ask whether they have any questions about the process to alleviate any concerns they may have.

I briefly tell them the purpose of the study. You don’t want to give them too much information that you provide them with clues which will affect their behaviour, directing them as to their actions. It is a skill as to how to find out how they use your design, without impacting on their use.

The participant will want to say what they think you want to hear. I explain to them that the purpose is not to test them. I usually say “we’re not testing you, we’re trying to investigate how best to design this and so we’re looking for your help. There’s no right or wrong answer, we just want to see whether we should change the design”.

• How will the results be used

Explain where the information from their involvement will be used. I get the participants to sign a user consent form and give them at least 24hours to read the consent form so they can understand the purpose of the study, what is expected of them and so they understand where the information from their involvement will be used. It’ll give them the opportunity to consult with family and friends.

• The opportunity for participants to ask questions

Give participants the opportunity to ask questions about anything they are unsure about throughout the sessions.

• Avoid leading questions, closed yes/no questions and target words

Avoid leading questions and closed yes/no questions. Ask follow-up questions from what you’ve found to investigate further. Ask open-ended questions that encourage the subject to talk and not closed questions that can be answered with a simple “yes” or “no.” If you use target words in your questions, you’ll be influencing the participants behaviour. For example, if you have a menu label called ‘register’ then a question concerning seeing whether participants can register on the site should not include the target word ‘register’ as the participant will simply recognise that label wording and click onto it. With your questions you want to provide sufficient information to set the scene and you don’t want to influence their response.

• Give the participant reassurance

Give the participant reassurance. I make comments like “thank you for your help with this”.

• Be relaxed as the interviewer

You want the session to be natural, so be relaxed to reassure the participant so they’ll be calm and their response will be natural.

 

 

• Be nearly invisible

Keep the session natural by being nearly invisible. You should insert yourself when necessary to get them back on topic, for further clarification or to continue onto the next area. You want to find their response, behaviour and thoughts on the design.

• Get participants to tell you their thoughts as they use your design

I find it very useful to video the sessions. It helps with subsequent analysis and provides concrete points of reference when discussing your results with clients.

Get the participants to talk through their thinking out aloud as they’re using your design to give you their needs, priorities, their language, themes and information on their mental models.

• Ask the user to expand on information

I note phrases they use to explain their thoughts and ask them to “explain more about that”. I avoid asking “why” as it can seem that you are being negative, questioning their answers.

If the participant cannot do a task, don’t let them struggle on. It’ll be obvious if they can’t complete a task. You want them to feel that you’re not testing them and that they are helping you find out whether changes need to be made to the design.

• The wrap-up

I usually ask participants to reflect on the tasks they’ve completed during sessions and ask whether there’s anything they’d like to mention. This provides them with the opportunity to highlight any background and reasoning concerning the tasks, which is useful.

I have found useful information from participants at the end of sessions.

I always thank participants for their time and let them know what the following steps will be in the project.

 

Analysis of the Results

 

For ease of analysis, I create two spreadsheets, one showing the quantitative results and one with the qualitative results.

Use the testing sessions to find the participants needs, priorities, their language, themes, information on their mental models, their behaviour and design problems they’ve raised. If you’re not in a position to continue doing user testing, one can use the information already gathered from the user testing to create personas for use in the remaining design process.
Hope you enjoyed this blog post.

For a future blog post again from my own UX experience, we’ll look at considerations to bear in mind when interviewing older adults. When devising the user test plan and undertaking the user testing, consideration should be given to differences they’ll have to younger people in terms of cognitive, visual and physical differences.