How to get started with Quantitative UX Research

I’m trying to move my career into Quantitative User Experience Research. That’s a lot of big words, so let’s call it Quant UXR for short. If you are in a similar situation, you’ve come to the right place. I’m no expert on Quant UXR, but I know a lot about how to learn about Quant UXR. Over the past year or so, I’ve tried a ton of different resources.

In this article, I’ll share some of my favorites. We’ll start with the easy stuff then move on to more advanced topics. Work your way through the list from top to bottom or mix and match to your heart’s content!

Step 1) Learn the basics of Quant UXR in an hour

If you are familiar with the methods of Quant UXR, skip this section.

If you are like me, you probably aren’t sure where to start. That’s what I’m here for.

The best Quant UXR article for beginners

Start with the NNg article Quantitative User-Research Methodologies: An Overview (article, 14 minute read). This article was fantastic for me when I started digging into quant. It is a fantastic in-depth comparison of the most-common Quant UXR methods.

As you read about each method, write down one or two possible studies you might be able to do for each method. Can you sneak a Quant UXR study into your day job? Can you help out a small business or a non-profit? Get creative.

The best Quant UXR videos for beginners

Sit back and hit play on the Nielsen Norman Group (NNg) playlist Analytics, UX Metrics, & Quantitative Usability Measurement Methods (19 videos, about an hour total). NNg appears to be updating this playlist regularly.

The only downside to this playlist is that it doesn’t seem to be in any particular order. I would have expected fundamental videos like Qualitative vs. Quantitative UX Research (video, 4 minutes) and Analytics vs. Quantitative Usability Testing (video, 2 minutes) to come first in the playlist, but they don’t.

Step 2) Run your own Quant UXR study

“What?!” you say, “I’m not ready to run a study yet! All I’ve done is read an article and/or watch a few videos!”

That’s okay. Your first study is not going to be great, but you’re going to learn so much by digging in and trying things. This first study is going to be the frame that all your future knowledge hangs on.

Pick one of the study ideas that you came up with while reading the NNg Quant UXR methodologies article. Plan the study, run the study, then analyze the data and present the results. As you do this, write down any questions that come up. Some that came up for me included:

• How do I know what kind of sample size I need?
• What’s the best way to visualize quantitative results?
• How certain can I be of my results?
• What’s the difference between a p-value, a confidence interval, and a z-score?
• How can I explain statistical significance to other people without sounding like an idiot or making them feel like idiots?

You’ll be answering your own questions over the next few steps.

Step 3) Learn basic statistics

Do you know what a p value is? A z score? Confidence intervals? Even if you have taken a stats course or two, it can’t hurt to brush up on your skills.

There are a million ways to learn basic statistics. I guarantee that none are as enjoyable as Crash Course Statistics (44 videos, each about ten minutes long). It’s a bit of an investment. But it’s worth it. Adrienne Hill has a great way of explaining things so they’re funny, memorable, and understandable. I’ve watched their ANOVA video (13 minutes) three times already.

These videos should start to answer some of the numbers questions you came up with when you dove in and ran your first Quant UXR study.

Step 4) Learn about Quant UXR in the real world

Now that you have a good foundation, it’s time to get intimidated. Take a moment to read about Quant UXR at Google (PDF, 10 minute read).

An approximate technical level of expertise is this: an applied researcher at a graduate school level in social or natural sciences, with expertise in multivariate statistics and programming.

Chris Chapman, Quantitative UX Researcher

If that made you crawl under a desk and cry, know that you’re not alone. I feel the same way. But we shouldn’t let it discourage us. Even Quant UX Researchers with a decade of experience at Google had to start somewhere. One day, we’ll be like Chris Chapman.

For a more balanced, less intimidating take on Quant UXR in the real world, check out Becoming a Quantitative UX Researcher is Messy (article, 6 minute read).

When people ask me “how do I find a Quant UXR role?” I tell them, don’t look for that specific title. Only a handful of companies even have such a role, the big FAANGs, and it is very difficult to get in without prior experience.

Randy Au

Speaking of FAANGs (Facebook, Amazon, Apple, Netflix, Google), here’s a good article on the difference between data science and Quant UXR. How Facebook differentiates data science and quantitative UX (article, 2 minute read).

Edit: I ended up writing an entire article on this that summarizes some great case studies. See How Quant UXR works in the real world.

The last (but best) way to learn about real-world Quant UXR is to meet people. Reach out to your network, try UX Coffee Hours, find a Slack or Meetup group, and/or subscribe to the UX Research subreddit. Ask questions. Offer your expertise. Learn!

Step 5) Run another study

Go back to Step 2 and repeat it. You’ll know way more this time, but you’ll also uncover more questions. Learn by doing.

Step 6) Go deeper

You’re now at a fork in the road. You know the basics and you fearlessly dove in and ran a couple Quant UXR studies. Use the questions that came up to choose your next topic. Then, once you feel like know a thing or two, try another study. Repeat until you’re a pro!

Option a) Statistics for UX Researchers

So, you like stats, do you? Me too! Let’s get nerdy.

First, read Quantifying the User Experience by Jeff Sauro and James Lewis. It is DENSE, but it is nicely laid out. You might not absorb everything on the first pass, but that’s okay. Thanks to the decision maps in the first chapter, you’ll always know where to look.

Then, subscribe to the MeasuringU blog. They put out about one post per week, and sometimes they get pretty technical, but it’s super relevant stuff.

Some experienced Quant UX Researchers I’ve talked to have also recommended Benchmarking the User Experience by Sauro. I haven’t read it yet, but it’s on my list because it looks to be a practical, step-by-step guide on how to measure your user experience.

Option b) Analytics for UX researchers

Ahhhh, analytics. This is what first got me interested in Quant UXR. Unfortunately, the only serious reading I’ve done on the topic is Web Analytics 2.0 by Avinash Kaushik (book, 475 pages). I don’t recommend this book for people trying to get into Quant UXR. Don’t get me wrong; there was a lot of great, timeless wisdom in the book. But a lot has changed since it was written in 2009. Web apps were barely a thing back then, so Web Analytics 2.0 focused on static web pages and search engine traffic.

Next on my analytics reading list is Researching UX: Analytics, by Luke Hay (book, 176 pages). This book is only a few years old and is written for UXers. After that, I may read yet another Jeff Sauro book, Customer Analytics For Dummies (book, 336 pages). For more analytics books, see 5 books for bringing analytics into the UX process (article, 2 minute read).

Option c) Data visualization for UX researchers

You can’t be good at Quant UXR if you’re not good at presenting your findings. This section has resources for just that.

First, start with an absolute CLASSIC. Edward Tufte may have first written The Visual Display of Quantitative Information (book, 197 pages) almost 40 years ago and updated it about two decades ago, but it still holds true today. Even print-focused advice like “not wasting ink” holds true when you’re building digital graphs. As a bonus, it’s full of beautiful data visualizations.

Then, if you haven’t already, take the ten graph challenge (10-day challenge). I know it was only ten graphs, but it really built my data viz confidence.

Lastly, get involved with Storytelling with Data (online community). There are books, podcasts, a great blog, and a thriving community with monthly challenges. If you subscribe to their newsletter, you’ll be continually reminded of the fundamentals of data visualization as well as some more advanced topics.

Option d) Take an online course

If you’ve made it this far, you’re motivated enough to dive into a serious online course related to Quant UX.

I haven’t taken either of these yet, but they both look promising.

First up, and most relevant, is UX Research at Scale: Surveys, Analytics, Online Testing (Coursera course, about 19 hours to complete). This course is part 5 of a 6-part specialization in UX Research and Design. If you haven’t done much UX and UX research, consider taking all six parts. But if you’re focused on Quant UXR, just take this one part.

Then, consider taking Become a Data Analytics specialist on LinkedIn Learning (course, about 24 hours of content). It is not UX-focused, but data analytics skills are valuable as a Quant UXer. Topics include statistics, Excel, data visualization, and Tableau.

If these aren’t quite what you’re looking for or you’d prefer to learn in person, check out your local college or university to see if they offer relevant programs.

Option e) Learn a tool

If you’re working with any type of statistics, you need to know Excel or Google Sheets at the very least. If you want to be ready for large sample size analysis, try R, a statistics-focused programming language. If you are a student or work for a big company, you may have access to SPSS or Dataiku, which are powerful but expensive programs that don’t require much programming skills.

If you want to try R, the kind folks at the UXResearch subreddit have recommended using swirl, which is a way to “learn R, in R”.

If you want to learn any of the other tools, Google it and ask around. For example, I stumbled upon some great Dataiku training at work after I started hanging out with the company’s data experts.

Good luck!

I know everyone learns in different ways, but I hope that this post was helpful for you. If so (or even if not), leave me a comment below! I’d love to hear how you are learning about Quant UXR.