Brief Guide to Usability Testing

Hi Guys, the following article indicates a brief steps approach on how to do usability testing for websites and mobile apps and also highlighting some important concepts of UT.

What is the purpose of usability testing?

It not only serves as a method for discovering usability issues but also indicates what works. You might have to pay close attention because really intuitive and self-explanatory steps, interactions, and attributions might not even get noticed or mentioned at all by the participant during a usability test.

Typical scenarios:

• Identifying main issues in the usability of a product
• Checking if users understand the steps to carry out a task and the navigation
• Observing how easily and quickly they accomplish tasks
• Validating the value proposition of an app or website — do your potential customers understand it?
• Testing competitors’ solutions. (Even without a test-ready prototype or a website in the initial phase of product development, you can get ahead by testing competitors’ solutions with the target group to gain insights on what to do better. Check how we did exactly that in one of our case studies.)

Benefits of UT

1. Getting direct feedback from the users.
2. Getting actual data to cross correct design assumptions.
3. Saves development time by avoiding rework on later stages of product cycle.
4. Builds a work team that values UCD process.

2 Main types of UT

Formative Testing: It is conducted during product or site development and it is used to determine specific design objectives are met and to look for design bugs and design validation.

Summative Testing: It is conducted at the end of the development and use to measure whether the product has accomplished the desired goals.

Roles in Usability Testing

Participants: Actual potential users to participate in the UT as per user personas created during design research.

Facilitator: Leads sessions with participants and facilitates the complete process planning to testing and reporting.

Observer: Collaborates with the facilitator to observe the UT sessions and records events as they occur, does not have interaction with participants.

Qualitative and quantitative usability testing

All usability-testing studies involve a participant performing some assigned tasks on one or more designs. There are, however, two types of data that can be collected in a user-testing study:

• Qualitative (qual) data, consisting of observational findings that identify design features easy or hard to use. Qualitative data offer a direct assessment of the usability of a system: researchers will observe participants struggle with specific UI elements and infer which aspects of the design are problematic and which work well. They can always ask participants followup questions and change the course of the study to get insights into the specific issue that the participant experiences. Then, based on their own UX knowledge and possibly on observing other participants encounter (or not) the same difficulty, researchers will determine whether the respective UI element is indeed poorly designed.
• Quantitative (quant) data, in form of one or more metrics (such as task completion rates or task times) that reflect whether the tasks were easy to perform. Quantitative data offer an indirect assessment of the usability of a design. They can be based on users’ performance on a given task (e.g., task-completion times, success rates, number of errors) or can reflect participants’ perception of usability (e.g., satisfaction ratings). Quantitative metrics are simply numbers, and as such, they can be hard to interpret in the absence of a reference point. For example, if 60% of the participants in a study were able to complete a task, is that good or bad? It’s hard to say in the absolute. That is why many quant studies usually aim not so much to describe the usability of a site, but rather to compare it with a known standard or with the usability of a competitor or a previous design.

How many test users in an usability test?

• Testing with 5 people lets you find almost as many usability problems as you’d find using many more test participants. With 5 users, you almost always get close to user testing’s maximum benefit-cost ratio. Aiming at collecting insights to drive your design, not numbers to impress people in PPT.
• For really low-overhead projects, it’s often optimal to test as few as 2 users per study. For some other projects, 8 users — or sometimes even more — might be better. For most projects, however, you should stay with the tried-and-true: 5 users per usability test.

Getting the right participants

1. Select participants based on the user group
2. Applicability to site or application being tested
3. Current and Potential Users
4. Level of domain expertise

Recruiting Requirements

Preparation for recruiting

Decide optimal sample needed for test, recruit users closest to selected user personas

Document the recruiting requirements

1. User groups needed
2. Characteristics
3. No of participants in each group.

Get approval before writing the screener.

Recruiting Screener

Prep for recruiting

1. Screener is based on recruiting requirements.
2. Screener will guide recruiters, it contains screening questions, confirmation of qualified participants.

Step by Step guide to UT

Preparing a test plan

The first step is to create a test plan. It is a very important step to determine what we are testing, what we are going to give to our users. Before preparing a test plan we need to:

• Define what areas to concentrate on
• Determine potential usability issues
• Write a test plan
• Determine what tasks will be tested

A Test plan should include

1. Objective
2. Test task (scenario)
3. End state (answers)

Interact with participants before test starts

• Explain to your participants that the objective is to test the software and not the participants’ intelligence. This is important as the participants may feel that their ability is being tested.
• Explain how the test material and record will be used
• Encourage participants to think aloud while using the product
• Prepare a pre-test questionnaire and a post-test questionnaire

Prepare a checklist for Usability Testing

• Success rates
• Time on task
• Errors made in performing the task
• Confusion(unexpected user actions)
• System features used / not used
• System bugs or failure

During the test

• Record techniques and search patterns participants employ when attempting to work through a difficulty.
• If participants are thinking aloud, record assumptions and inferences being made.
• Do not interrupt participants unless absolutely necessary
• If participants need help, provide some responses

After the test

Hold a final interview with the participants and tell them what has been learned in the test, and if you noticed any discomfort in users or they gave a signal of confusion through “think aloud” during the test ask them more about such things.

Provide a follow-up questionnaire that asks participants to evaluate the product or tasks performed. The post-test questionnaire is generally used to gauge what users think of their performance. We should also include the questions like “What is the one thing that you would change…?”.

If video recording is required then get a written permission and respect participant’s privacy

Do not generalize the opinion unless 5–8 users indicate the same problem

Analyzing the collected data

1. Tabulate data : Consolidate the data from the UT sessions into a spreadsheet, this helps you review each sessions in detail and see the big picture.
2. Analyze data and list findings
3. Prioritize findings
4. Create redesign recommendations
5. Create a management presentations or report
6. Follow up with the technical team on UT results corrections.

I hope you find it useful. Let me know if you have any queries by dropping your comments.