Full Length Guide | Usability Testing
- Gregory Schmidt
This Full Length Guide will help you conduct a usability test.
Consider printing it out and following it closely. It is OK, and even recommended by some, to read from the guide word-for word.
If you make changes to this guide, and find you have discovered better ways to phrase lines, please contact the EHR Design Lab so that we can improve this version.
There are five sections to this usability interview:
1. Friendly Welcome
The person will be a bit nervous. Make them comfortable.
Thanks for joining us today and participating in our usability research.
We’re constantly trying to make your work easier by improving the electronic medical record, (insert name of your EHR).
Getting your honest feedback is a really important.
To help other team members benefit from your comments today, it is customary for us to record these sessions. Please read this form, and sign if you agree with the statement. (Link to: Consent Form)
If they sign the sheet, start the recording, and then say: Thank you, I am now turning the recording on.
The most important part of this sessions is to remember that I am not testing you. I'll ask a a lot of questions, but there is no right or wrong answer.
My goal is to test these prototypes, and understand how you react to them. Tell me your reactions outlaid as you think of things during the tasks.
If you are confused or don't understand something, please tell me. If you see things you like, tell me that too.
Since I didn't design the prototype, you won't hurt my feelings or flatter me. In fact, honest, candid feedback is most helpful.
Again, I'm not testing you, we're testing this prototype to see what we need to fix.
If you do get stuck, I'm going to try to not help you get unstuck. It's very helpful for me to see how you try to get unstuck - as if you were using the product on your own. But don't worry - I'll help you if you get completely stuck.
Do you have any questions before we begin?
2. Context Questions
Understand more about the user and their background. The goal here is several fold. First, it is to help gather backgrounder information about the user and their behaviours. (These are Who? Where? When ? What? Why? How? type questions). Second, it is to help put them more at rest, by being small talk. Spend a few minutes here, based on the time you have.
Examples of questions
Job
What type of work do you do here?
EMR
Have you ever used an electronic medical record before?
What do you like about the electronic medical record you work with?
What do you find frustrating about the electronic medical record you work with?
** How could the electronic medical record make your work easier? **
Are there things about your electronic medical record that confuse you?
How has your ability to care for patients change because of the medical record?
Are there parts of your work that are still on paper?
Technology
Outside of work, do you use a mobile phone, tablet or computer?
How comfortable are you using these devices?
What type of apps do you use the most?
Transition to tasks sentence:
Now I’d like to show you some rough prototypes of ideas we’re experimenting with. Again, we're testing the prototype, not you.
Because this is just a prototype, some links or buttons or features may not work quite right. You can still click anywhere you'd like. I'll let you know when you run into something that's not working.
Don't worry, you cannot break anything on the prototype. And, all patient information here is also not real.
To begin, please just take a look at [this screen]. What are your first impressions of what this is?
3. Tasks
Tasks are specific things the user is asked to accomplish. Watching how a user is able to accomplish the tasks provides valuable feedback.
Specific tasks will be provided for each prototype.
(Insert usability tasks here)
Follow-up questions
The goal of follow-up questions is to help encourage the user to talk. After each task, or after a series of similar tasks, it may be helpful to ask some follow-up questions. Especially if the user is being quiet. Follow up questions are not meant to quiz or grill the user.
Some examples of follow-up questions:
So what happened there?
Was that what you expected? Why or why not?
So what goes through your mind as you look at this?
Can you describe to me what you see on this page?
Did you find what you were looking for?
What is this? What is it for?
What did you think of that?
What would you do next? Why?
Is there anything else you would do at this point?
Is there any other way to do that?
In what ways would you want this changed to make it better for you?
What additional info would have helped?
What do you think this [point to UI element] might do?
What does this [point to UI element] mean?
Going off script
When performing a task, the user may click on the wrong thing. This is ok. Allow them to do so, and watch to see where they click after that. As you mentioned in the introduction, your goal is to try and see how they trouble shoot the issue themselves.
If the user looks quite distraught and frustrated and in a dead end. Help provide a hint or direct them to the right next step.
Alternative way to do the same thing
The user may complete a task using an alternative method than anticipated.
For instance, the task may be for the user to return to the overview of the patient chart. Instead of using the application's navigation to get there, they may use the back button on the browser. This is ok, let them do so. After the user has completed the task using their first method. Have the user return to the page of interest, and ask, "Is there another way you would do __________"
Don't say the word on screen
The task question should not include the word written on the screen.
For instance:
Bad: "Click the Help button"
Good: "What would you do if you needed assistance?"
Bad: Register a new patient. (if the screen has a button labeled "Register new patient")
Good: How would you create a new patient record?
4. Debrief
The debrief provides an opportunity to comment more broadly on the prototype and tasks they have performed.
Examples of debrief questions include:
Which parts of this page are most or least important to you?
What do you like or dislike about this?
If you had three wishes to make this better for you, what would you wish for? Why?
If you wanted to _______, how would you…?
Under what circumstances would you use this? Why?
How would you describe this to a friend?
If you’re testing two or more prototypes in your interviews, you can use these questions:
How would you compare those different versions? What are the pros and cons?
Which parts of each prototype would you combine to create a new, better version?
Which one worked better for you? Why?
How is A different from B?
What does each of these do well? Poorly?
What types of people does each of these versions seem to be designed for?
Don't Lead
Questions should be objective, and now influence how the user will respond.
Michael Margolis gives examples of leading questions:
Bad: "Is this good?
Bad: You program your home thermostat to save energy, right?
Good: Do you program your home thermostat?
Bad: Would you rather use the old version or this new, improved design?
Bad: Is this version better?
Good: How would you compare these two?
Good: What are the pros and cons of these prototypes?
5. Cool Down & End
Summarize the session briefly for the user.
This has been incredibly helpful.
[Briefly summarize key points.]
Your input is really valuable for me and the team as we think about the next steps for these ideas.
I really appreciate your taking the time to come in, and answering all of my questions.
Thank you
-
Resources
See the OpenMRS Wiki Usability Testing Links & Resources page for all resources.
This script draws upon, and directly mirrors in areas, the work of Michael Margolis in his Google Venture's Usability Testing work
More Context / Discovery Question examples by Michael Margolis
from his PDF document on UX Research Workshop
Types of Questions for Discovery Research
Specific Examples: Who did you call from your cell phone yesterday?
Complete List: What are all the payment apps on your phone? Are there any others?
Activities: What do you typically do to get ready for a trip?
Reenactment: Please show me exactly how you do that.
Sequence: Please walk me through a typical day. How do you start? And then what do you do next?
Inputs and Outputs: What information do you need to gather before you can do x? How and where do you get that information? What do you do with it when you’re done?
Guided Tours: Can we take a look at your email account together?
Projection: What do you think would happen if. . .?
Changes over Time: How does that compare to the way you did that a year ago?
Exceptions: Under what circumstances do you do that differently?
Suggestive Opinion: Some people have very negative feelings about using cell phones in cars while others don’t. How do you feel about it?
Identification: Who do you think would use something like that? Who wouldn’t?
Outsider Perspective: How would you describe <feature or activity> to someone who hadn’t done that before? What advice would you give to somebody who was going to try it?
Comparisons: What’s the difference between Tweeting and sending an email? How do you do that differently when you’re at home vs. at work?
Successes and Failures: What would be the worst case scenario? Can you tell me about a time when this didn’t work?
Fill in the blank: So in that situation, you. . . [pregnant pause]?
3 wishes: If you had 3 wishes to make this better for you, what would they be?
Follow-up Questions for discovery research
Why?
Point to participant’s reactions contradictions, paradoxes, non sequiturs, unexpected reactions, or laughter. Why do you roll your eyes when you say that?
Clarification: When you say “her” you mean your daughter, right?
Reflecting Back: So, what I hear you saying is______. Is that right?
Native Language: Why do you call your computer “my brain”?
Silence: Trust your question and wait for participants to fill in the gaps. Or try leaving pregnant pauses: “When that happened, you felt. . . ?“