UX Testing for the Masses: Keep It Simple and Cost-effective
UX testing is a crucial step in the design process but is often overlooked for tight budgets or deadlines. You do not need a formal lab to do quality research—just the right tools and a clear plan.
UX testing is a crucial step in the design process but is often overlooked for tight budgets or deadlines. You do not need a formal lab to do quality research—just the right tools and a clear plan.
Ruaridh is an award-winning UX/UI designer and creative director with 14 years of experience designing for a variety of well-known brands.
Expertise
User experience design (UX design) is the practice of design that focuses on the user and a product’s usability in order to deliver the most satisfying experience. While some aspects of user-centered design can be intuitive or rely on UX best practices, delivering great user experience requires deep insights into the user’s needs, goals, and pain points. Uncovering these insights requires UX testing.
User experience testing is one of the most misunderstood yet crucial steps in UX design. Sometimes treated as an afterthought or an unaffordable luxury, user research and UX testing should inform every design decision.
In order for UX testing to be effective, it must involve the most important people—your users/customers. For UX designers, running tests with actual users may seem difficult and overwhelming—not to mention expensive. Deciding what aspects of the product to test and how is a major task on its own.
You do not need a formal lab or an entire research department to do quality UX testing. There are many types of usability testing methods out there as well as tried and true guidelines on when and how to conduct effective “lean” testing without breaking the bank or torpedoing deadlines.
Understanding User-centered Design (UCD)
Before we proceed to testing, let’s start by explaining the basic concept behind user-centered design (UCD). UCD places the user first in the design and development cycle of an application/website. Practicing UCD is based around developing an understanding of the product’s purpose and what a user is using a site or an app for—considering the complete user experience as a whole.
In order to ensure the end product meets the user’s needs, as well as the product’s initial brief requirements as fully as possible, the concept of UCD means that the entire design process must involve real users throughout.
There are many ways to approach UCD, but the most basic form of the process should include the following:
- Context of use: Identify who will use the product, what they will use it for, and under what conditions they intend to use it.
- Requirements: Identify any business requirements or user goals that must be met for the product to be successful.
- Design solutions: This part of the process may be done in stages, building from a rough concept to a complete design through a number of iterations.
- Evaluation of designs: Ideally through usability testing with actual users. Usability analysis is just as important for UCD as quality testing is for good software development.
To build the data foundation for these phases of user-centered design, designers must put together a usability test plan that incorporates some or all of the following UX testing tools.
UX Testing Methodologies
Before putting together a user experience or usability test plan, begin with the assumption that needs testing. This will help inform what type of testing should be done. For example, when trying to understand why a sign-up form is not converting users as expected, it makes sense to compare analytical data to results from a usability study. It may not be necessary to run the usability test with dozens of users—five may be enough. By being clear about the focus and scope of what’s being tested, you can make the most of your time and budget.
There is a wide variety of traditional usability testing methods that can be conducted according to the research questions and needs. For most design initiatives, a combination of quantitative and qualitative research methods should be applied in order to get the best insight into the target user, their needs, and how to address them.
Card Sorting
Card sorting offers useful insights for information architecture and user experience design experts. Understanding how users organize their own ideas, tasks, and concerns can help designers build the basis for a website sitemap, establish a hierarchy of modules on a homepage, or inform the structure of an app’s menu.
Card sorting involves participants being given an unsorted group of cards, with each card having a label on it relating to a page or section of the website. The participants are then asked to sort the cards into groups and name them in a way that makes sense.
Capture results from a few users and look for patterns in responses. Card sorting is an inexpensive UX testing method for learning what your website navigation and content structure should look like and how they should work in a way that’s logical to your intended user base.
Usability Testing Session
What is the purpose of a usability test? A usability testing session involves collecting data from a group as they use the website/interactive prototypes. By watching how real users engage with a product or interface, designers can surface usability problems, expose gaps in user flows, and gain other valuable insights that might have otherwise gone unnoticed.
Usability testing sometimes comes at a relatively high cost because it involves a lot of human interaction and legwork. That cost can go up with the number of users being tested. Nevertheless, the benefits of usability testing can translate into saved time and expense in redesigns and development fixes later on.
What does a usability testing session look like? People are invited to attend a session during which they will be asked to perform a series of tasks on a website or app while you or the moderator takes notes. The user will often be asked to fill in a questionnaire at the end of the test to ascertain how difficult it was to perform certain tasks, such as buy a product on an eCommerce site from a specific category page and proceed to checkout.
Usability testing does not necessarily have to be performed in person. App and website usability testing can be done remotely, showing users prototypes through platforms such as InVision or Marvel and then conducting the tests through a UX testing platform like UserTesting.com. Conducting usability testing remotely is a great way to save time and cost during usability research.
Be sure to begin with a usability test plan that includes a clear understanding of scope. Decide in advance what assumptions need to be tested and what tasks the user needs to complete in order to surface useful insights. Keep the number of participants in the usability testing to a sensible number—many suggest that five users is enough. Usability testing is a great way of gathering data on the most common issues real-world users will encounter.
Focus Groups
Focus group testing involves asking focus group members (which could be product users or the intended target audience) a series of questions related to the app or website and encouraging them to share their thoughts and feelings on different related areas of the site design/wireframes.
One of the benefits of focus groups is that the shared discussion gets multiple people talking, making it easier to see commonalities in the users’ expressed needs and pain points. Often, one person will volunteer an idea or complaint, which inspires others to build upon it and surfaces more valuable information.
It’s normally a good idea to have an experienced moderator during such a group session to keep participants focused as well as to ensure accurate notes are taken. Additionally, a good moderator should be able to identify the telltale signs of groupthink and make sure that the whole process is not negatively affected by group dynamics.
User Questionnaires and Surveys
Another simple way to get information from potential users or existing customers is to solicit feedback with a questionnaire or survey. Questionnaires can be a great way to generate invaluable, solid statistical data—provided the right questions are asked.
When drafting questions, it is important to go deeper than just asking users if they enjoy the app or website—try to find out why. Ask if the user would recommend the site or app to a friend. Probe users about how using the product in question fits into their lives and habits.
Questionnaires usually include rating systems and yes/no questions; be sure to add space for the user to give written feedback and comments to get some qualitative data as well.
A questionnaire can be particularly useful when you want to collect a larger, more varied cross-section of data than could be achieved through a small focus group. Questionnaires can also present an opportunity to collect valuable demographic information that can help inform design decisions.
It can also be argued that people tend to be more honest without the immediate pressure of a small user group. The risk of groupthink is averted, so individuals will make their own decisions.
UX Testing Tools and Software
As the user-centered design disciple becomes more sophisticated, so do the available tools and platforms to facilitate good UX testing. There is a wide variety of quality testing platforms and tools that help design teams collect and analyze quantitative data and usability analysis information. Depending on your team’s needs, many of these tools are either relatively inexpensive or offer a free trial.
Google Analytics
A service that hardly needs an introduction, Google Analytics is one of the most in-depth tools available and currently the most widely used web analytics service in the world. In a nutshell, Google Analytics allows you to analyze demographic and device data as well as click testing and drop off points.
Integrated with AdWords, it allows you to track landing page conversions (from ad clicks) and view in-depth information about your website’s traffic.
Google Analytics is part of a suite of interconnected tools to help interpret data on your site’s visitors including Data Studio, a powerful data-visualization tool, and Google Optimize, for running and analyzing dynamic A/B testing.
Quantitative data from analytics platforms should ideally be balanced with qualitative insights gathered from other UX testing methods, such as focus groups or usability testing. The analytical data will show patterns that may be useful for deciding what assumptions to test further.
Crazy Egg
Crazy Egg offers a suite of tools for website usability testing. The platform outputs mouse-tracking heatmaps to show you where all the action takes place on your site. They also offer A/B testing tools for trying out designs to see what works and what doesn’t.
The platform allows a 30-day free trial of their services, which means it can be used briefly even on the tightest of budgets. With careful, clear planning, you can get a lot of valuable data before having to pay for a subscription.
Again, it makes sense to look at data such as heatmaps alongside other testing results. Heatmaps can show what users are looking at but may not provide enough information to explain why.
Optimizely
A/B testing involves comparing two versions of a page to find out which one performs better. Two pages are shown to similar visitors at the same time, the page with the better conversion rate being the more effective page. A/B testing can sometimes mean the difference between a campaign’s success or failure.
Optimizely allows you to conduct effective, in-depth A/B testing, yet it’s relatively easy to use. Versions of a design can be tested side by side without the need for separated development. The platform also includes great data and usability analysis tools.
Optimizely is not free, although you can use the Starter plan free of charge. However, the free plan lacks a lot of features compared to the Enterprise plan. Pricing may be a problem for independent designers and small outfits.
When planning an A/B test, it is essential to be clear about the assumption or theory the design should validate. Make sure to pair this type of analysis with complementary UX testing and usability analysis that fit with the question to be answered.
Usertesting.com
Conducting usability research in person can be time-consuming and costly. Usertesting.com is a great solution for conducting remote testing with real users.
The platform records testers’ screens as they use a website or prototype, complete tasks, and answer questions. It also includes handy tools for making clips of recorded video and taking time-tagged notes on user responses.
Usertesting.com is not free, but researchers can get a lot of value from the tool, especially by carefully designing and validating tests before launching them to minimize mistakes or wasted tests.
Testing on a Tight Budget or Timescale
There are dozens of UX testing tools available to practitioners of user-centered design. User research is essential, even when a project must be operated on a tight budget or a hard deadline. There are plenty of clever ways to streamline the process without sacrificing too much.
Devise a usability test plan with a clear scope and list of questions the team needs to answer to avoid wasted time or resources. Look for opportunities to leverage existing data, such as customer support emails and reviews. Use that data to help inform and focus the plan for further research and UX testing.
Every designer and researcher would love to have unfettered access to unlimited testing resources, but that’s not always realistic. What is important is that you are actively seeking involvement, feedback, and constructive criticism from other people.
If your budget and schedule won’t allow you to do everything you had in mind, you need to think outside the box and come up with new ways of obtaining usable test results. Be creative. Test users early with low-fidelity wireframes. Launch a survey direction on your website. You don’t need a lab to send out an email questionnaire or conduct guerilla testing in a coffee shop.
Measuring Success
Design decisions based on good data and usability research can mean the difference between success and failure. Well-founded design decisions can lead to increased site traffic, more conversions, better sales and performance, improved customer satisfaction, and a higher retention rate. Poorly-informed design, however, can be disastrous and costly.
No single UX testing methodology will entirely answer all user experience questions. With an effective user-centered design approach, synthesizing results from one or more of the above tools and approaches will allow you to set realistic goals in order to improve your app, website and user experience.
Before you start, be clear about what is being tested and what metric will help determine the result. If the team is trying to determine how button color and placement affect conversions, look at existing analytics and decide what the threshold is for data significance. Then run an A/B test with two or more variations to see which performs better. Conduct a usability test on the designs, watch what users notice, and get them thinking aloud as they browse the page.
Good design comes from more than intuitive creativity—it should be informed in part by data. Gathering that data does not have to be costly or time-consuming if well planned. Document the questions to be answered and the metric by which to assess success. Develop a clear usability test plan that balances qualitative and quantitative UX testing methods.
UX testing does not end when a new product or feature launches. Conduct UX testing early and often. Synthesize user feedback with findings from quantitative analytics to look for problems and opportunities for improvement. Using the right tools strategically will yield benefits of usability testing that outweigh the initial costs.
Further Reading on the Toptal Blog:
- E-commerce UX: An Overview of Best Practices (with Infographic)
- Empty States: The Most Overlooked Aspect of UX
- Anticipatory Design: How to Create Magical User Experiences
- Heuristic Principles for Mobile Interfaces
- The Importance of Human-centered Design in Product Design
- Pragmatic Pixel Perfection: A Manifesto for Balancing Design Quality and Speed
Understanding the basics
What is the difference between UCD and UX?
UCD, or user-centered design, is a design philosophy which puts the target user in the center of the design process. UX, or user experience, is the design process aimed at delivering a quality experience to users of a product, app, site, or service.
What is a human-centered design?
Human-centered design is a philosophy of design practice that relies on research and observation of human behavior in order to deliver engaging user experience that meets users’ needs.
What is card sorting in web design?
Card sorting is an information architecture exercise in which users are prompted to label and organize index cards in a way that represents their ideal or intuitive thought process. Card sorting is often used to design sitemaps and navigation.
What is the meaning of usability testing?
Usability testing is a testing methodology that invites users to test out a product, wireframe, or prototype under observation with the purpose of uncovering flaws in user flows and optimizing the user experience.
How do you perform usability testing?
Usability testing can be performed in person or remotely using a testing platform. Participants are given tasks to complete using the product or site in question while being observed.
What is the purpose of a usability test?
The purpose of a usability test is to see what users do with a product, website, app, or prototype. UX researchers are looking for problems in user flows as well as opportunities to improve the user experience.
Edinburgh, United Kingdom
Member since March 24, 2016
About the author
Ruaridh is an award-winning UX/UI designer and creative director with 14 years of experience designing for a variety of well-known brands.