This Full Length Guide will help you conduct a usability test.
Consider printing it out and following it closely. It is OK, and even recommended, to read from the guide word-for word.
If you make changes to this guide, and find you have discovered better ways to phrase lines, please contact the EHR Design Lab so that we can improve this version.
There are five sections to this usability interview:
1. Friendly Welcome
The person will be a bit nervous. Make them comfortable.
Thanks for joining us today and participating in our usability research.
We’re constantly trying to make your work easier by improving the electronic medical record, (insert name of your EHR).
Getting your honest feedback is a really important.
To help other team members benefit from your comments today, it is customary for us to record these sessions. Please read this form, and sign if you agree with the statement. (Link to: Consent Form)
If they sign the sheet, start the recording, and then say: Thank you, I am now turning the recording on.
The most important part of this sessions is to remember that I am not testing you. I'll ask a a lot of questions, but there is no right or wrong answer.
My goal is to test these prototypes, and understand how you react to them. Tell me your reactions outlaid as you think of things during the tasks.
If you are confused or don't understand something, please tell me. If you see things you like, tell me that too.
Since I didn't design the prototype, you won't hurt my feelings or flatter me. In fact, honest, candid feedback is most helpful.
Again, I'm not testing you, we're testing this prototype to see what we need to fix.
If you do get stuck, I'm going to try to not help you get unstuck. It's very helpful for me to see how you try to get unstuck - as if you were using the product on your own. But don't worry - I'll help you if you get completely stuck.
Do you have any questions before we begin?
2. Context Questions
Understand more about the user and their background. The goal here is several fold. First, it is to help gather backgrounder information about the user and their behaviours. (These are Who? Where? When ? What? Why? How? type questions). Second, it is to help put them more at rest, by being small talk. Spend a few minutes here, based on the time you have.
Examples of questions
Job
What type of work do you do here?
EMR
Have you ever used an electronic medical record before?
What do you like about the electronic medical record you work with?
What do you find frustrating about the electronic medical record you work with?
** How could the electronic medical record make your work easier? **
Are there things about your electronic medical record that confuse you?
How has your ability to care for patients change because of the medical record?
Are there parts of your work that are still on paper?
Technology
Outside of work, do you use a mobile phone, tablet or computer?
How comfortable are you using these devices?
What type of apps do you use the most?
Transition to tasks sentence:
Now I’d like to show you some rough prototypes of ideas we’re experimenting with. Again, we're testing the prototype, not you.
Because this is just a prototype, some links or buttons or features may not work quite right. You can still click anywhere you'd like. I'll let you know when you run into something that's not working.
Don't worry, you cannot break anything on the prototype. And, all patient information here is also not real.
To begin, please just take a look at [this screen]. What are your first impressions of what this is?
3. Tasks
Tasks are specific things the user is asked to accomplish. Watching how a user is able to accomplish the tasks provides valuable feedback.
Specific tasks will be provided for each usability study.
(Insert usability tasks here)
Going off script
When performing a task, the user may click on the wrong thing. This is ok. Allow them to do so, and watch to see where they click after that. As you mentioned in the introduction, your goal is to try and see how they trouble shoot the issue themselves.
If the user looks quite distraught and frustrated and in a dead end. Help provide a hint or direct them to the right next step.
Alternative way to do the same thing
The user may complete a task using an alternative method than anticipated.
For instance, the task may be for the user to return to the overview of the patient chart. Instead of using the application's navigation to get there, they may use the back button on the browser. This is ok, let them do so. After the user has completed the task using their first method. Have the user return to the page of interest, and ask, "Is there another way you would do __________"
Follow-up questions
The goal of follow-up questions is to help encourage the user to talk.
After each task, or after a series of similar tasks, it may be helpful to ask some follow-up questions. Especially if the user is being quiet.
Follow up questions are not meant to quiz or grill the user.
Some examples of follow-up questions
So what happened there?
Was that what you expected? Why or why not?
So what goes through your mind as you look at this?
Did you find what you were looking for?
What is this? What is it for?
What did you think of that?
What would you do next? Why?
Is there anything else you would do at this point?
Is there any other way to do that?
In what ways would you want this changed to make it better for you?
What additional info would have helped?
4. Debrief
The debrief provides an opportunity to comment more broadly on the prototype and tasks they have performed.
Examples of debrief questions include:
Can you describe to me what you see on this page?
If you had three wishes to make this better for you, what would you wish for? Why?
Which parts of this page are most or least important to you?
What do you think this [point to UI element] might do?
What does this [point to UI element] mean?
If you wanted to _______, how would you…?
Under what circumstances would you use this? Why?
What do you like or dislike about this?
How would you describe this to a friend?
If you’re testing two or more prototypes in your interviews, you can use these questions:
How would you compare those different versions? What are the pros and cons?
Which parts of each prototype would you combine to create a new, better version?
Which one worked better for you? Why?
How is A different from B?
What does each of these do well? Poorly?
What types of people does each of these versions seem to be designed for?
5. Cool Down & End
Summarize the session briefly for the user.
This has been incredibly helpful.
[Briefly summarize key points.]
Your input is really valuable for me and the team as we think about the next steps for these ideas.
I really appreciate your taking the time to come in, and answering all of my questions.
Thank you
-
Resources
This is strongly based upon the format suggested by Michael Margolis, with adaption for our local context.
See Usability Testing Links & Resources for further details.