Design Process
9 minute read

UX Myths – Prototyping, User Testing, and UX Deliverables

James is a problem-solver who fuses cognitive science techniques with interaction design to make products people love.

Myths. Tall tales. Misconceptions. These are things we hear and believe but aren’t necessarily true. In nearly every field, including UX design, myths can change the perception of those that practice as well as those that benefit from the practice.

For example, a common UX myth that pervades design projects is, “We came from this industry, so we know what the user wants.” As it turns out, this is rarely true, and we find that it’s a good idea to challenge this statement for the greater good of the end user and a higher chance of success.

As UX designers, we aim to create better user experiences through rigorous and methodical design processes. Due to common misconceptions, it can often be challenging to convince clients of the value these processes can bring. Addressing these myths will help UX designers overcome roadblocks in the design process.

UX myths pervade the design process and challenge project success
Tall tales and myths make it hard to prove the value of specific design processes like prototyping and user testing. ( Jerry Lofaro)

UX Myths: Prototypes

Prototypes are essential design assets that contribute to rapid feedback, user insights, and a more efficient design process. They can, unfortunately, be a hard sell due to several UX myths.

Many UX prototyping myths make it a hard sell to clients
Myths surrounding prototyping can make it hard to prove the value of such a critical part of the design process. (ergomania)

Myth 1 – Prototypes Take Too Much Time: False

Throughout the design process, UX designers might create rapid prototypes when they have clearly defined objectives and some existing data on which to base task flows. This can result in prototypes with as little as three screens:

  1. Where the user came from (this could be a notification or login)
  2. Where the user is going (main screen in most cases)
  3. What they need to do (task destination)

Here are a few design tools that facilitate a rapid prototyping process:

  • Paper Prototypes: In some cases, it might be easier to get out some grid paper and a few colored markers and create a paper prototype. These can be detailed, semi-functional, and useful for user feedback.
  • Native Mobile Products: InVision and Marvel work well in a native mobile environment where a realistic full-screen mode is required for user testing. Other prototype tools like Figma use a “mirror” approach which requires the laptop to be always on, a stable wifi connection, and their mirror app on a single target mobile or tablet device.
  • Forms and Landing Pages: Webflow and Axure are good for forms that need to feel like HTML, allowing users to tab through and add content to fields. These tools are also responsive and will feel natural to the user when tested on their computer or device.
  • SaaS Desktop: There are several choices and some have collaboration features like Atomic, Adobe Experience Design, InVision, Axure, Justinmind, and UXPin. A special mention needs to go to Marvel App, which has user testing features and quick reporting built in.
  • Data-driven: If users need to see live data or integrations, then some coding may be involved. A few tools which facilitate that are Bubble.is, Framer, and Axure.
  • Interaction: When the challenge is trying to find out how a product acts or reacts to user clicks or external events, then tools like Hype Pro, Principle, Flinto, Adobe Animate, and InVision Studio have animations that can improve the way users engage with the product.

Myth 2 – Prototypes Are the Final Designs: False

The term prototype is often falsely associated with a costly design endeavor that can take months to build. However, prototypes are generally used to validate product usability and cost far less than making changes to final designs.

If the particular product does not have a final design approved, prototypes can still be developed using well-established design systems like Material Design, Atomic Design, and Bootstrap. The prototypes aren’t the final design; they are an essential artifact that informs and reduces the chance of final design “failure.

UX prototyping is a part of the process but not the final design.
Prototypes are a part of the design process, but they aren't the final design. (Fredo Tan)

Myth 3 – Five Users Will Uncover 85% of Usability Issues: False

A popular UX myth is that it only takes five users to uncover 85% of usability issues. This is rarely true in the field. In the article titled, “Testing web sites: five users is nowhere near enough,” Jared Spool and his team had this to say, “Our findings differ sharply from rules-of-thumb derived from earlier work by Virzi and Nielsen commonly viewed as ‘industry standards.’ We found that the four sites we studied would need considerably more than five users to find 85% of the problems.”

Though this myth may be true in some cases, such as in lean/iterative design or for small startups getting rapid feedback, it is not the case for large-scale projects that will have significant resources and an appropriate budget dedicated to it.

So how many users does it take?

There are no rules for a specific project type needing a specified number of users to test with. It depends on the scope, budget, time, and design staff who are experienced in doing UX user testing.

UX Myths: User Testing

User testing helps designers gain valuable insights into potential problems before products are finalized and released, saving time, money, and the risk of product failures.

Myth 1 – Users Don’t Know What They Want: False

Users know what they want; they just don’t know how to articulate it. It is the job of the researcher and designer to uncover these insights from user research without showing final designs. User testing is done to find out what works as well as what doesn’t work for the user.

A common practice is to build three or more prototypes and deploy user testing to get, not just knowledge of what works or doesn’t work, but a deeper sense of what the user really wants. When this is done in the spirit of innovation, everyone generally agrees that it makes good business sense.

User testing is a critical part of the design process.
User testing provides insights into the needs, motivations, and behaviors of users to ensure the success of the final product. (James Tucker)

Myth 2 – Analytics and QA Are Equivalent to User Testing: False

Analytics isn’t user testing. It’s a set of data showing user activities that happened in the past, and while it can inform “what” users did, analytics cannot inform “why” they did it. The “why” comes from user testing and is a foundation of improved user experiences.

A good example is the Coke and Pepsi taste challenge. Analytics informs how much Coke or Pepsi has sold in a geographic area over the past year, while user testing blindfolds people in that same geographic area to find which soft drink they like best. Analytics tells us that Coke sold 30% more, but in the taste tests 75/100 people preferred the taste of Pepsi.

Hypotheses can then be tested to validate why people prefer the taste of Pepsi, while Coke sells more. It could be advertising, marketing, packaging, price, shelf placement, or brand reputation. Analytics cannot tell us why, but user testing can.

User testing is done by UX professionals who can understand the subtleties of human behavior.
User testing is a critical part of the design process in order to pick up on such subtleties as non-verbal reactions, microexpressions, and small nuances.

Quality assurance (QA) testing encompasses all of the activities that ensure a design team is providing the best possible usability experience and code for a product and/or product feature set.

QA is essential to the product design process, but it doesn’t use the same scientific methods or produce the same results as user testing.

Historical references (validated user stories, video clips of users interacting with the original problem, and prototypes showing how the feature is supposed to be built) of the feature-set being tested assist developers and QA engineers.

User testing can help validate QA assumptions, but it is not a replacement for QA testing.

Myth 3 – User Testing Can Be Done by Anyone at the Office: False

There are some people who are naturally good at asking questions and observing behaviors; however, when it comes to performing formal user testing, it’s best to have trained, experienced professionals (UX designers) who can pick up on microexpressions and the smallest nuances in the users’ interactions with prototypes.

There is often a split-second action that happens during user testing, and these details can become vital to the success of the project. UX designers and UX researchers with experience will quickly pick up on these actions and document them.

If there were any other reason to have an experienced UX professional conduct user testing, it is because they have a keen ability to read body language, capture unbiased data, reset the test if needed, and get vital data if the prototype does not load or unforeseen situations occur.

UX Myths: Reports and UX Deliverables

UX deliverables (reports) are the result of all the work designers do during each phase of the design process. The value of these deliverables cannot be understated as they inform future improvements as well as provide a historical snapshot of the entire product design project. There are, however, a few myths which are common to UX deliverables.

Myth 1 – Nobody Reads the Reports: True

As valuable as the reports might be, we all know that they are rarely read. Designers put a lot of work into the design process, and documentation is a large part of that work. The final UX deliverable should strike a balance between the scientific backbone of the research any aesthetic emphasis that has been woven in.

Some ideas to draw readers into the report are:

  • Visual emphasis – Create advertising-like spreads to surface the problem statement.
  • Visual contrast – Sketch a comic strip with talking bubbles to show the user’s dialog.
  • Personalization – Quotes from important customers and their photos.
  • Variety of media – Videos of the users interacting with the software highlighting the smoking gun.
  • Typographic hierarchies – Big, bold, quantitative summaries.

Myth 2 – Reports Should Only Come from Outside Consultants: False

There is a misconception that internal UX deliverables and reports on user testing do not hold as much value as the same information delivered by outside consultants who charge large fees. There is also a belief that an outside consultant brings more value due to their specialization or niche in a specific industry.

Internal reports are free and provided by UX designers which tends to deflate their value-add in the minds of the executive team.

Here are some tips UX designers can implement to help “sell” the deliverables and their associated value:

  • Make the report look like it was delivered by a professional consultancy; if it looks the part, it will change perceptions.
  • Reports by experts are some of the most referenced documents out there, so model it like the expert market research reports.
  • Champion the research by sharing the insights and findings at a company all-hands meeting or town hall. Treat the findings and report as benefits to the organization.
UX deliverables are an important part of the UX design process.
UX deliverables are a key part of the UX design process. They hold value in future decision-making as well as providing insights to other teams like marketing and sales.

Myth 3 – Reports Are Dated the Minute They Are Presented: False

This widely believed myth can be made manifest by creating a folder named “Reports” where they all go to die. UX deliverables are the result of a lot of hard work and serve as artifacts throughout all stages of the design process. They can include business and user insights, roadmaps, usability issues, and product marketing gold nuggets.

Instead of leaving deliverables abandoned in a folder to die, use these best practices to keep them alive:

  • Add reports and other UX deliverables into the ecosystem of the company so they are accessed frequently.
  • Add links to the reports in Slack, Trello, Jira, or other places where they will be seen.
  • When there’s an opportunity to reference a finding from the report, include a link so it can be accessed.
  • Turn the deliverables into living documents, action items, or tasks.

UX myths, like any other misconception, have a tendency to turn an important and valuable process into a lot more work than it needs to be. As designers, we are continually working hard to prove the value of this process and its outcomes. Knowing the myths and their truths will go a long way to overcoming doubt, improving the user experience, boosting KPIs, and will help a lot of people involved with the project gain more trust in the process.

• • •

Further reading on the Toptal Design Blog:

Understanding the basics

What is the purpose of user testing?

The purpose of user testing is to discover any potential problems with a product, website, or app. User testing also serves to increase empathy with users.

What is prototyping in design?

Prototyping is the process of creating an early release or model of a product, website, or app. It is used in design to get feedback and to iterate for further design modifications.

What is prototyping and why it is useful?

Prototyping is the act of creating draft or early versions of a product. It’s useful for designers to be able to test out product features without having to build an expensive and complete version of a product.

What is a disadvantage to prototyping?

One disadvantage to prototyping is time. If a particular product needs to be done on a short timeline, then prototyping isn’t going to be advantageous as it will take time to both create the prototype, get feedback, and build the final product.

How do you perform user testing?

User testing is performed by first creating a prototype and then coming up with a test plan. Once a plan has been created, recruit target users and then perform testing while moderating and/or observing the test and documenting the results.