Duolingo is great, but…

I LOVE Duolingo. I can’t recommend it enough for people who want to start learning a new language.

Ever since I first tweeted about it back in 2015, I’ve been using it pretty much daily. First for a little bit of Spanish, and now for French and even a little bit of Vietnamese!

However, I have a few bones to pick, especially with some of the more recent changes, both as a user and as a UX professional.

Lingots vs. Gems

Back in the day, Duolingo used “lingots” to reward users for things like having a long streak of practicing, allowing them to “buy” things like extra lessons or even costumes for Duolingo’s owl mascot. This was consistent between the web application and the mobile app.

But sometime in 2017, the mobile app switched over from lingots to “gems”. I remember seeing a nice notification after that switch that explained to me why the change was made and what it meant to me. No harm done.

The weird thing: The web app didn’t switch over. I thought this might just be a temporary transition, but several months later and it still hasn’t. So one one device I have gems and on the other I have lingots.

The Shop on mobile: I have 3272 gems and I’ve wagered 50 gems on “Double or Nothing”.

The Shop on web: Even though it’s the same account, I have 2334 lingots and I haven’t made a “Double or Nothing” wager but I can for 5 lingots.

I really have trouble keeping track of the difference between the two. I wish I didn’t care about these fake internet points but when my 650+ day streak is riding on them, I can’t ignore them.

The fix

My suggestion to Duolingo: Follow Jakob Nielsen’s heuristic of “Consistency and standards”.

Users should not have to wonder whether different words, situations, or actions mean the same thing.

Choose one: gems or lingots. Then get rid of the other.

The Double-Tap Bug

It may be because my iPhone 6 is no spring chicken, but I’m regularly frustrated by the following behavior on Duolingo’s iPhone app.

On “listen then type” questions, when I tap the field to start typing, nothing seems to happen. So I tap again. The keyboard then appears (due to my first tap) then immediately disappears (due to my second tap). So I tap once more and all is well.

Confusing? Here’s a gif I made to illustrate it.

duolingo double tap bug.gif

The double-tap bug. Due to a delay in registering the first tap, the first two taps cancel each other out. The third one works though!

The bad thing is that this doesn’t seem to consistently happen. Sometimes when I tap the first time the keyboard immediately appears so there’s no problem. Sometimes I tap, then wait patiently for the keyboard to appear, resisting the urge to tap again, only to be staring at my screen for several seconds like an idiot, apparently because the app didn’t actually notice my first tap.

The fix

The problem seems to be that there’s a delay in registering the first tap, possibly due to the phone working hard to play the audio. I wonder if it’s possible to ignore a tap to close the keyboard if it happens immediately after the keyboard was opened.

Also, perhaps make the tappable area larger on these questions? That would prevent the “staring like an idiot” scenario above.

Condescending Messages

On the iOS app, after my first five questions of the day then again after about ten, I get a message. Instead of letting me continue on in my French flow, that damn owl pops up and gives me an infuriating pat on the head.

Yes, maybe when I start learning Japanese and am getting frustrated with the first few lessons, this might be a welcome ray of light. But not when I’ve devoured all the French content that Duolingo has to offer and I’m just reviewing a few old words. In that case, it’s just annoying.

The fix

As I said in my tweet, maybe these messages should just be for beginners. Or better yet, why not add a “Stop showing these messages” button to allow for opt-out?

But it’s not all bad!

With that being said, I still love Duolingo and will probably continue using it for years to come. And not all of their new stuff is bad.

Stories

I love Duolingo’s new stories feature, which seems to only be available on the web app under the “Labs” tab or at stories.duolingo.com.

These are short, enjoyable stories with words on screen and voices narrating, these are a great way for intermediate to advanced learners to get comfortable with how a language is spoken. They’re interactive to make sure you’re paying attention.

Duolingo’s story. It’s asking me which word means “to stay on the surface of the water”. It’s “flotte”!

And best of all, they’re entertaining. The stories usually have a nice little twist at the end.

The twist: The man that Rose was speaking to at the art gallery was Captain Black Beard himself!

Conclusion

Keep doing what you’re doing Duolingo!

For the gems vs. lingots, I’m guessing that either I’ve run into an edge-case bug or that there is some kind of constraint that I’m not aware of.

For the messages, I know that I’m now turning into an edge case. There are millions more beginners on your platform than people like me who have consumed all content for a given language, but still, with a couple of small changes, you can keep me preaching about how awesome you are without alienating your new customers at all!

For the stories, please make them mainstream! They’re such a great, entertaining way to learn!

Want to talk Duolingo with me? Find me on LinkedIn!

Advertisements

Talking to Mars

Lately I’ve been bingeing on WaitButWhy. From Artificial Intelligence to the history of everything, Tim Urban is an expert at carefully researching complex topics then crafting entertaining, easy-to-read, and informative posts on them.

The other day I read the article How (and Why) SpaceX Will Colonize Mars. It was a great article about space, Mars, and Elon Musk, but what stuck with me was this little tidbit buried in Part 3 of the post:

Earth people and Mars people will be in close touch, emailing and texting and watching each other’s movies and TV shows (no phone calls or Skype convos though—because data transfer is limited by the speed of light, a message sent from one planet takes between three and 22 minutes to get to the other, depending on the planets’ locations)

As well as this footnote:

For 2–4 weeks of each 26-month planet location cycle, the sun is directly between the two planets, and they can’t communicate at all.

In other words, if Earthlings want to talk with those brave few people who head to Mars starting as early as 2024, there’s going to be some interesting challenges.

So how are we going to talk to Mars?

We won’t be able to FaceTime, but we will be able to do video messaging. Kind of like SnapChat but slower. Let’s call it SlowChat. It should be good at making it clear just how slow each message will be, because there’s a big difference between a 6-minute round-trip, a 44-minute round-trip, and a 2-4 week delay!

Inspired by Tim Urban, I’ll illustrate this with a bit of a story.

It’s the year 2030. I’m floating around in 38% gravity and I want to chat with my mom.

wcm0008.png

Okay, maybe Mars won’t be terraformed enough to have beaches and retirees by 2030, but we’ll definitely need to figure out ways to improve communication if we’re sending people up there!

What do you think? Tweet me your thoughts or post a comment below!

Match Debt: When your system and the real world don’t align

Twenty-two years ago, design legend Jakob Nielsen published 10 Heuristics for User Interface Design. These ten tips are just as relevant now as they were in the early days of the internet.

I consider them my ten commandments when it comes to designing software. In fact, this was the one and only thing that I displayed on my cubicle wall at my previous job.

IMG_2947

My ten commandments (Forgive me father Nielsen, for I have sinned)

Anyway, the second heuristic on the list is:

Match between system and the real world

The system should speak the users’ language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

In other words, you want to make your product come as close as possible to the mental models of your users. In a perfect world, all products would closely align with how customers actually think.

1

The heuristic “Match between system and the real world” means that your product should align closely with the mental models of your users

Unfortunately, it’s not a perfect world.

In reality, there’s always a gap between how the product represents things compared to how the user actually thinks about them.

2

In reality, there’s probably a gap between your product and the mental models of your users

Actually, it’s often quite a large gap.

3

There may be a significant gap between your product and the mental models of your users

At my previous job, an established financial software company, the RTFM test revealed a feature that caused a lot of support requests: user access control. After speaking with some customers, we noticed that they were using different words than we were. What the system referred to as Access Areas, they called Functions. Also, the interface for user access control wasn’t consistent with the rest of the system. Our customers were used to using Excel-like tables to configure and view things, but this feature had a series of listboxes and dropdowns that were much less familiar or flexible. For this feature, there was definitely a gap between our product and the mental models of our users.

The Match Debt

Having this gap is not a huge problem (users tend to “satisfice“, settling for “good enough” solutions instead of fully understanding all options), but it is a cost. A cost that shows up in customer support, bug fixes, and worse: lost sales. I call that cost the Match Debt, because much like technical debt, the decisions you make today can make future development more expensive, like paying interest on a loan.

4

The Match Debt is the size of the gap between the system and the real world; the bigger the gap, the more money you’re losing

For our user access system, the Match Debt showed up most visibly in the form of calls to our support team. It also meant that the feature was hard to maintain. It may also have led to lost sales, though that is much harder to mention.

There are really only two ways to pay off Match Debt.

Educating your users

Firstly, you can try to move the users mental models closer to the way the product works via customer training and documentation- let’s call this education.

5

Education attempts to move your users’ mental models closer to the way your product works

Our user access system had thorough, well-written documentation, which I’m sure helped some of our customers, but users don’t read!

Learning about your users

Alternatively, you can try to move the product’s design closer to how the users think. This can be done in many different ways, including researching your users, designing solutions to their problems, testing these solutions, and more. But for simplicity’s sake, let’s just call this User Experience, or UX.

6

UX attempts to change your product closer to your users’ mental models

This is what we did when we redesigned the user access system. The new design uses slightly more familiar language and presents it in a way that’s consistent with the rest of the product. It’s nowhere near perfect, but now the Match Debt is lower, meaning that we’ll probably have fewer support calls and slightly happier customers than before.

It’s not you, it’s me

Both education and UX do the trick in reducing Match Debt, but from a customer’s perspective, UX is a whole lot nicer because it means that the product is doing the work, as opposed to the customer having to do the work. UX puts the onus on the few people who develop the product instead of the many people who use the product.

UX is the product team telling the user “it’s not you, it’s me”, then actually doing something to fix “it”. Isn’t that the way it should be?

Sure, this is a gross oversimplification, but this is why I love UX so much. It pays off Match Debt by aligning the product (and in fact the whole organization) closer to the users’ mental models.

It’s hard work. But it’s worth it.

Quotes from Interviewing Users by Steve Portigal

I just finished reading Interviewing Users. It’s one that has been on my list for a long time, probably because I loved Steve Portigal’s Dollars to Donuts podcast. It did not disappoint.

It was a great read for me because I’ve done enough user research to have some experience to map to, but not so much that I knew everything in the book.

Here’s a collection of quotes from the book that I found valuable, organized by theme.

Preparing for interviews

Portigal first walks the reader through what to do before interviewing users.

“At the outset of a project, make the objectives your initial priority.”

Interview 6-8 stakeholders (yes, you interview the team even before you interview the users!) to see what they want to learn. Sift through the politics and agendas and try to get everyone on the same page as to what the objectives are. Find issues such as

“…hypotheses masquerading as facts, aspirations, and mass hallucinations… resolve those issues as tactfully as possible.”

Prepare a field guide but

“this is not a script. It reads very linearly, but it’s really just a tool to prepare to be flexible.”

Research scheduling

Once you’ve agreed on the objectives and recruited participants, schedule two interviews a day or so. But why not more?

“Quality work doesn’t come from being rushed, exhausted, harried, or overwhelmed. Interviewing is hard work. You need time between sessions to reflect.”

And while it’s tempting to save time or resources by using Skype or a phone call or getting users to come to you, go to where the users are.

How many people should go to each interview?

“I find the ideal size for the field team to be two people: one to lead the interview, and one to back up the other person.”

Documenting the interview

Photography

Prepare a shot list before interviews that lists the photos that you want to take.

“Even if you’re capturing imagery using video, still pictures are essential.”

“Even though they agree to the use of photography when they sign your release, let the interview settle in before you start taking pictures. You can verbally confirm that it’s okay before you take your first picture.”

Taking notes

Note taking is okay but no one writes fast enough to capture everything and

“you must maintain eye contact while writing.”

If taking notes, just write the facts. Not judgements. But if you must interpret,

“it’s easy to lose track of what you were told versus the conclusions you made, so take care in how you document the two.”

As soon as possible after each interview, write a top-of mind summary.

“Allow time for debriefing after each interview… The longer you wait, the less you will remember, and the more jumbled up the different interviews will become.”

Video or audio

Video or audio recording is the only way to capture everything the participant says.

“Keep your eyes and brain in interview mode until you are fully departed.”

“consider keeping your recording device on, even if it’s packed”

because of

“the “doorknob phenomenon,” where crucial information is revealed just as the [therapy] patient is about to depart.”

Interview technique

Portigal offers lots of tips about actually conducting the interview.

They’re the expert

“make it clear to the participant (and to yourself) that they are the expert and you are the novice.”

“Asking that person to play the teacher role not only reinforces the idea that she is the expert here, but it also can make it easier for her to articulate the details you are seeking.”

Invite honest criticism

“if you bring out a concept by saying, “Here’s something I’ve been working on…” you’re activating a natural social instinct that will diminish their comfort in being critical.”

Building rapport

Start with general (easy) questions to get them warmed up. It will feel awkward and uncomfortable but accept and break down the awkwardness by

“accepting, acknowledging, and appreciating her responses.”

“Try to construct each question as a follow-up to a previous answer.”

If you change topic, “signal your lane changes”.

Stories

Once you’ve built rapport, you will reach a tipping point from question/answer to question/story.

“Stories are where the richest insights lie, and your objective is to get to this point in every interview.”

Use their language

“build rapport by accepting the terms the participant was using rather than trying to demonstrate credibility.”

“Don’t correct her perceptions or terminology if the only outcome is “educating” her. Advocate for her, not for your product.”

The interview isn’t about you

“[Only] talk about yourself if doing so gives the other person permission to share something.”

To dodge questions asked by participants,

“Do the Interviewer Sidestep and turn the question back to them: “Is that important to you?” “What would you expect it to be?””

Question Types

Ask about sequence, quantity, examples. Ask for exceptions, lists, relationships. Clarify comments, language/ code words, and emotional cues. Ask them to teach, to compare, etc.

You want to know about their needs. But you can’t directly ask about their needs. So you ask them indirectly.

“There’s a difference between what you want to know and what you ask.”

On participatory design

“My aim with participatory design is to give people a different way to talk about needs, where the solutions stand as proxies for those needs.”

You don’t actually have to implement any of these designs.

Silence

“One way a novice interviewer tries to counteract nervousness is by preemptively filling the silence.”

“Ask your question and let it stand…. After she has given you an answer, continue to be silent.”

Improving as an interviewer

Practice interviewing. Even in daily life.

“Cultivate a style of interacting socially that emphasizes listening, reflecting back the other person’s comments, allowing for silence, and so on.”

Also, review your interview recordings with a critical eye like a football coach would review game tape.

“ask someone to come along to your interviews and get his feedback. Also ask for feedback from the rest of your field team.”

Other tips include teaching others to interview and even taking an improv class, because that will improve your abilities of thinking on your feet, accepting awkwardness, and “going with it”.

 

Analysis and synthesis

After you’ve finished all of your interviews, follow a procedure like this to squeeze as much learning as possible out of your research:

  1. Divide and conquer: “get your data in text form and divide it among the team.” Each team member reviews and summarizes one or more interviews.
  2. Summarize for the team: “The group should then reconvene and present each interview as a case study.”
  3. After all summaries, recap: “Discuss each interview briefly, and then create a sticky note that summarizes the key point of that interview.”
  4. Group findings: “create groupings. You may want to start with the categories from your topline and add to them.”
  5. Prepare the report: “the Presentation of Findings, which is the main research deliverable.”

User research and organizations

Interviewing is a good way to increase a company’s UX Maturity. It…

“starts to drive shifts in the organizational culture”

“position yourself in your organization so that interviewing customers is an integral part of how you work.”

How?

“The most impact for the least effort comes from your colleagues joining you in the field.”

For organizations with high maturity, research informs design but also exposes

“new opportunities for teams to embrace a user-centered approach to their work.”

Conclusion

I really enjoyed this book and I’m sure I’ll continue to refer to it (and these notes) for years to come!

The RTFM Test

As your team improves its UX maturity, it can be easy to be overwhelmed by all of the possible ways that you can improve your product.

One good place to start is the RTFM test.

How to do the RTFM test

It’s simple. Ask your support team:

What area of the product gets the most support requests that don’t end up in bugs being logged?

In other words: Which areas does the support team most often find themselves telling the customers to RTFM (Read The Freakin’ Manual)?

Or worse: Are there any areas that the support team has to keep repeating itself because they’re not even covered by the manual?

That is where you start. That’s the lowest hanging fruit. Those are the usabugs. Fix those and you’ll free up your support team to deal with real bugs.

But how?

Now that you know which area of your product is the biggest RTFM offender, there are many ways you can improve it. Let’s turn to Jakob Nielsen’s classic Ten Usability Heuristics for inspiration. Specifically, number ten:

Help and documentation

Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user’s task, list concrete steps to be carried out, and not be too large.

There’s a lot in there, so let’s break it down:

GROTFM

If at all possible, get rid of the freakin’ manual. “It is better if the system can be used without documentation.” This should be paired with user research such as usability tests to make sure you’re solving the real underlying issue.

For example, maybe the product uses a list of cryptic codes that the user must look up in the manual in order to understand what they mean. If so, provide that information on-screen instead.

Improve your documentation

If you can’t GROTFM, then follow Nielsen’s hints on how to improve your manual.

  • Provide easy-to-use search functionality
  • Center the documentation around user tasks, not system areas
  • Provide step-by-step instructions for accomplishing those tasks
  • Keep it short

As above, use user research to guide and validate your work on the documentation of your system.

Try it out!

It seems obvious in hindsight but before I redesigned the user access system at my previous company, I had dabbled with dozens of other ideas for improving the product. Sure, I was able to find lots of areas of the system that violated usability heuristics or clearly didn’t work as expected, but only when I discovered the RTFM test was I able to find something that was actually valuable to the business and to our users. The RTFM test allowed me to easily prioritize all those opportunities for improvement.

My Own Service Design Mentor

One piece of career advice I keep hearing (for instance in Whitney Hess’s interview on the UX Intern Podcast) is to get a mentor.

Why Get a Mentor?

The value of that is obvious: A smart, experienced person who you meet with regularly and learn the craft from? Sign me up!

Oh wait.

The Hard Part

There’s no actual sign-up sheet? You mean I have to actually go find someone and convince them to spend hours of their free time with me, chatting and teaching me about their career?

That’s the hard part.

It’s a really BIG ask.

So I’d been procrastinating on this for ages.

My Mentor

But then in late-2016 I met Mark Plant. He works at my company, ION, but in a different department. Whereas my team works on a specific product, Mark’s team is more of an internal design agency, working on developing various products, both internally and externally.

Basically, he’s doing my dream job 🙂

Testing the Waters

So I bought him a coffee and peppered him with questions about his career and the field of UX. It also came up that he’s into skiing and mountain biking, much like myself, so I asked him a lot about that as well.

Somehow the barrage of questions didn’t scare him off, so I offered to go for another coffee a week or two later. And by then he was already going through a step-by-step explanation of not just UX, but the wider fields of Service Design, Experience Design, and more. It was like he had a curriculum developed and was just waiting for some young designer like myself to ask him to teach them.

It’s Official!

And before I could even ask him to be my mentor, he offered!

We now meet every week or two to design a new app based on the Explor app from the AngelHack hackathon I did last year.

 

File_000

Some sketchy sketches of possible app interactions

 

He’s given me career advice, he’s shown me specific techniques, and he’s broadened my horizons beyond just pure UX.

For instance, I previously rolled my eyes at marketing. I sort of felt like it was UX but without the cool tech and research aspects. Mark suggested I read On Brand by Wally Olins, and boom. I am now filled with respect for the discipline of marketing.

Thanks Mark!

Giving Back

By the way, I won’t be as slick of a teacher as Mark, but if anyone reading this is just starting out in tech or UX and have a few questions or even just want someone to bounce design ideas off of, let me know! I’d love to have a mentee of my own.

It’s the circle of life!

Save

Organizing Documentation with Card Sorting

We use Confluence at work for internal documentation. It’s kind of like Wikipedia, but only us developers can see and edit it, and the articles are things like How to set up Server x, or How to Test y.

I love it because we previously used long email chains with the answer buried deep within, mixed with dusty old word documents locked into source control. Confluence feels so much more lightweight, nimble, and searchable.

The Problem

Unfortunately over time it’s become a bit of a mess. We have over a hundred pages but there’s not much rhyme or reason as to how to find them. Confluence does have a search function but, believe it or not, people still browse. Also, if you’re adding an article it’s hard to know where to put it.

The Solution

So I decided to run a card sort exercise with my coworkers. My goal was to understand how they thought about these topics then organize the articles in a way that matched that as close as possible.

I scheduled a lunch and learn, invited the whole development team, and…

(Almost) No One Showed Up

1024px-tumbleweed_rolling

A photo taken at my first attempt at a card sort workshop

Just one of my coworkers and my manager came. I should know by now that now all developers are as into UX as I am 🙂

Luckily, my manager agreed that this was important enough to schedule in and make mandatory, so a couple weeks later, the workshop was on!

The Actual Workshop

I hadn’t run card sorts recently, so I brushed up on my basics. The two main rules that stuck out for me:

  1. Run with one person at a time
  2. Never make people sort more than 30-50 cards at a time

I broke both those rules. Because I only had a one-hour workshop, I had everyone do the sorting in three small teams instead of individually. Also, because we had 100 articles, I did the sort with all 100 of them.

F5ABFE81-7E70-4DC5-A4C8-26C55D6FCF67-cropped

Team Pink sorts away. See guys? UX can be fun!

After that workshop, each team ended up with several named groups with several cards each.

Analyzing and Testing the Results

I then used a template from Donna Spencer to analyze the results. I’d definitely recommend this template. It’s free and it did the trick.

Each team came up with remarkably similar groups. Based on those groups, I then came up with proposed categories and tested with a quick tree test using TreeJack from OptimalWorkshop.

Photo 2017-03-31, 2 57 27 PM-edited

My attempt at making sense of the card sort results. Paper still beats Excel sometimes.

Based on the tree test, one of the cards was clearly in the wrong category so I moved it. A couple of the others weren’t decisive so I added cross-links, effectively giving those articles two homes instead of just one.

Reflections

I’m very happy I did this. It was nowhere near perfect but it definitely vastly improved the structure of our documentation, showed everyone which articles were available, and introduced card sorting to the team.

Next Time

If I had more time I probably would have done the sorts one-on-one with my teammates instead of doing everyone at once in teams.

I should have used a smaller representative sample of the articles instead of trying to sort them all.

In other words, I should have followed the basic rules of card sorts!

Oh well! You can’t learn the limits if you don’t go past them every now and then, right?!

Ugly yet organized sketching

As a side project at work, I’m currently slowly chipping away at a troublesome area in a legacy desktop application. I’ve done some group ideation (brainwriting) and some usability testing in this area, but I hadn’t had much of a chance to really “sink my teeth” into the interface.

My favorite way to really get to know an interface?

Sketching

I love sketching because it turns half-formed ideas into tangible things, often exposing flaws that you never knew were there.

I just returned from a trip to Germany and Poland. Anticipating lots of waiting and planes, trains, and buses, I set a goal of 100 sketches over the course of the week.

I ended up with 43.

A little disappointing, but I think what I lacked in quantity I more than made up for in quality.

Now that’s some quality sketchin’!

image 123923953

It ain’t pretty, but it works!

No, not aesthetic quality. They’re definitely not the prettiest of pictures. However, I feel like I made a breakthrough when it comes to how I lay things out.

Number sketches for linking

As you can see in the example, I started every sketch with a number. I only really started doing this because I wanted to know how many sketches I had done, but it had the secondary benefit of allowing me to refer to each idea one at a time and “link” between sketches. It really forced me to keep my sketches and ideas organized.

This was an improvement over my previous sketching binges where I’d just fill up the page with multiple ideas then move on. It also allowed me to branch out and come back to ideas instead of having to choose one at a time.

Pros and cons

Each page starts with a brief intro and often a “Why”. The sad faces show the doubts that I had after creating the sketch. Often, I address these doubts in later sketches. Or at least I try.

Results

Thanks to this slightly more organized way of sketching, I can see myself getting much more value out of my sketches. It’s sort of like writing code. If you don’t put a little effort into making it clear what you’re doing, it’ll probably be incomprehensible to your coworkers- actually, it’ll probably be incomprehensible even to yourself a few months down the road!

Comments?

How do you sketch? Can you recommend any resources for improving my sketches? Bill Buxton is on my list but are there any others that I should know about? Give me a shout out @sgryzko and let me know!

Communicating the Value of User Research

I recently came across a great guide to communicating the value of user research to stakeholders.

It seems to be written more for UX designers brought in on a contract or agency basis to help clients “do UX”. However, as a permanent developer who’s trying to “do UX” on the side, most of the points have rung just as true for me, if not more so.

It can be an uphill battle trying to justify the investment required for User Research, but by keeping things simple (keeping costs low) and clearly communicating the benefit of the research, progress can be made.

This was originally posted by Frauke Seewald on the Toptal blog, but they’ve given me permission to repost it here. Enjoy!


The beginning of a new project: Your client needs help with a redesign of its website or application.

“We want to improve the user experience, it has to be jaw-dropping for our customers, we want them to fall in love with our product.”

Here is the good news: Your client is aware of User Experience (UX), cares about customers’ needs and sees the value in investing in a great user experience. They asked for an expert with UX skills to help, but do theyreally understand what it means to deliver an exceptional user experience?

User research is a vital component of UX design. Don’t let anyone tell you otherwise.

User research is a vital component of UX design. Don’t let anyone tell you otherwise.

UX is more than a bunch of rules and heuristics that you follow in your product design process. UX is subjective, as the name suggests. It is the (subjective) experience that a user gets while using a product. Therefore, we have to understand the needs and goals of potential users (and those are unique for each product), their tasks, and context.

As a UX expert you should already be familiar with the maxim, It all starts with knowing the user.

Now for some bad news; this is the point when you discover your client’s misconceptions about UX.

UX expert: “Ok, let’s start with your users: Who are they? What do they do? What do they want? What are some of their pain points? I would like to talk to them, observe them, learn from them…”

Client: “Oh, we don’t need user research, that’s a waste of time.”

Wrong!

In this post I will try to explain why, and hopefully, help fellow UX specialists in their efforts to convince clients that good UX is next to impossible if it is not preceded by good user research.

No Need For User Research? There Is Always A Need For User Research

You cannot create a great user experience if you don’t know your users or their needs.

Don’t let anyone tell you differently. Don’t simply accept the common argument that there is no time or money to do any user research for your project.

User research should shape your product design and define guidelines that will enable you to make the right UX decisions.

User research should shape your product design and define guidelines that will enable you to make the right UX decisions.

User research will shape your product; it will define the guidelines for creating a product with a good experience. Not spending any time on research, and basing all of your design decisions on best guesses and assumptions, puts you at risk of not meeting your user needs.

This is how senior UX architect Jim Ross UXmatters sees it:

“Creating something without knowing users and their needs is a huge risk that often leads to a poorly designed solution and, ultimately, results in far higher costs and sometimes negative consequences.”

Lack Of User Research Can Lead To Negative Consequences

Skipping user research will often result in “featurities,” decisions that are driven by technical possibilities and not filtered by user goals.

“My wife would really enjoy this feature! Oh, and I heard from this person that they would like to be able to xyz, so let’s add it in there too.”

This leads to things such as overly complex dashboards in cars, where the user’s focus should be on driving, not on figuring out how to navigate an elaborate infotainment system.

Many users find automotive infotainment systems overly complex and distracting. Identifying the target audience is crucial to good UX design.

Many users find automotive infotainment systems overly complex and distracting. Identifying the target audience is crucial to good UX design.

Tesla’s cutting edge infotainment system, based on Nvidia Tegra hardware, employs two oversized displays, one of which replaces traditional dials, while the other one replaces the center console. Yes, it looks good, but it was designed with tech savvy users in mind. In other words, geeks will love it, but it’s clearly not for everyone. It works for Tesla and its target audience, but don’t expect to see such solutions in low-cost vehicles designed with different people in mind.

Poorly designed remote controls are not intuitive, so casual users tend find them overwhelming, resulting in a frustrating user experience.

Old remote controls are another example of hit and miss UX. There is little in the way of standardization, so each one takes time getting used to.

Old remote controls are another example of hit and miss UX. There is little in the way of standardization, so each one takes time getting used to.

But what about the purely digital user experience? Too many fields in a form, or too much information may overwhelm and drive your users away.

Poorly designed digital interfaces can drive users away. Even if they don’t, they will annoy users and feel like a waste time.

Poorly designed digital interfaces can drive users away. Even if they don’t, they will annoy users and feel like a waste time.

Instead of creating the opposite behaviour, poorly designed and implemented interfaces are more likely to scare off potential users.

Start User Research With Sources For Existing Information

Yes, user research will expand the timeline and it won’t come cheap, but both time and costs can be minimized. You can start with existing, and easy accessible, sources of information about user behaviour to gain a better understanding of user needs. These are:

  • Data Analytics
  • User Reviews and Ratings
  • Customer Support
  • Market Research
  • Usability Testing

Quality user research requires time and resources. However, you can start by using existing information to get a sense of what your users need.

Quality user research requires time and resources. However, you can start by using existing information to get a sense of what your users need.

Let’s take a closer look at each of these sources.

Data Analytics

If you are working with an existing product, your client might have some data and insights about its use. Data analytics assist with getting a good overview about general usage: How many visitors are coming to the website, what pages are most visited, where do visitors come from, when do they leave, how much time do they spend where, and so on.

But here is what this data is not telling you: How does the experience feel? What do users think about your service, and why are they spending time on your website? Why do they leave?

For example, your data indicates that users are spending a lot of time on a specific page. What it doesn’t tell you is why. It might be because the content is so interesting, which means users found what they were looking for. On the other hand, it could be an indication that users are looking for something they cannot find.

Data Analytics is a good starting point, but it needs further qualitative data to support the interpretation of the statistics.

User Reviews And Ratings

Your client’s product might have received some user feedback, already. There might be a section for feedback or ratings on the website itself, but external sources may be available as well. People might have talked about it in blog posts or discussion boards, users may have given app reviews in an app store. Check different sources to see what users are saying.

However, be aware of limitations. People tend to leave reviews and ratings about negative experiences. Don’t take this as a reason to shy away from user reviews or to ignore feedback!

“All these complainers… These aren’t the users we want, anyway!”

Instead, try to look for patterns and repetitive comments. Here are a few tips for making the most from user input:

  • Check whether any action has been taken on negative comments.
  • Compare the timing of negative comments to releases and changelogs. Even great apps can suffer from poor updates, leading to a lot of negative comments in the days following the update.
  • Do your best to weed out baseless comments posted by trolls.
  • What are users saying about the competition? Identify positive and negative differentiators.
  • Don’t place too much trust in “professional and independent” reviews because they can be anything but professional and independent.

User reviews are a good source for collecting information on recurrent problems and frustrations, but they won’t give you an entirely objective view of what users think about your product.

Your client might have a customer support hotline or salespeople who are in touch with the user base. This is a good resource to get a better understanding of what customers are struggling with, what kind of questions they have, what features/functionality they are missing.

Setting up a couple of quick interviews with call center agents, and even shadowing some of their calls, will allow you to collect helpful data without investing too much time or money.

Customer support provides you with a good opportunity to learn about potential areas for improvement, but you will have to dive in deeper to get detailed information about problems.

Market Research

Your client may have some basic information about the customer base, such as accurate demographic information, or a good understanding of different market segments. This information is valuable to understand some of the factors behind the buying decision.

It does not offer any information about the usage of the product, though.

Market research is a good source of information if you need a better understanding of how your client thinks, what their marketing goals are, and what their market looks like. However, it won’t reveal all relevant details about user goals or needs.

Usability Testing

If you are lucky, your client might have done some usability tests and gained insights about what users like or dislike about the product. This data will help you understand how people are using the product and what the current experience looks like.

It is not quantitative research, and therefore you won’t get any numbers and statistics, but it helps you identify major problems, and gives you a better understanding about how your user group thinks.

There is also the option to do some quick remote testing session by using services such as usertesting.com.

Usability tests are another good way of identifying key problem areas in a product.

How To Educate Your Client About The Value Of User Research

The budget might be small and the timeline tight, but ignoring user research will eventually bite you. Help your clients avoid pitfalls by making them aware of the benefits of user research.

What’s the ROI of good user experience? Knowledgeable UX experts must be able to communicate the value of user research to clients.

What’s the ROI of good user experience? Knowledgeable UX experts must be able to communicate the value of user research to clients.

Here are some common arguments against user research and how to deal with them:

  • We don’t need user research. We trust in your skills as a UX expert

As a UX designer, you need to view user research as part of your toolkit, just like a hammer or saw for a craftsman. It helps you to apply your expertise in practice. No matter how much expertise you have as a designer, there is no generic solution for every problem. The solutions always depend on the user group and the environment, so they need to be defined and understood for every product.

User research will help get an unbiased view, to learn about users’ natural language, their knowledge and mental models, their life context.

You are the UX design expert, but you are not the user.

  • Just use best practices instead of research

Best practices originate from design decisions in a specific context; the digital industry is evolving at a rapid pace, design trends and recommendations change constantly, there is no fixed book of rules. We need to be able to adjust and adapt. Those decision should be made based on research, not practices employed by others, on different projects.

  • We already know everything about our users

Invite your client to a user needs discovery session to observe how users are using the product. Start with small tests and use remote usability testing tools such as usertesting.com to get some quick insights and videos of users in action.

The outcome might be a user journey map or a user task flow. Aim for a visualized document that identifies outstanding questions so you can define areas that need more research.

  • We have personas, we don’t need more research

Personas are a good tool for making your target group more tangible, and for becoming aware of different needs, key task flows and and how that might vary for different groups. It’s the common ground and a good starting point.

However, to redesign a product you need a better understanding of the usage. You need to know how people work with your product, what they do with it, when they get frustrated.

Ask for further details about user stories and task flows to make use of personas.

  • We don’t have the budget for it

The above list of sources for information about user behaviour should give you a good starting point for sharing ideas with your client on how to gain user information on a (very) tight budget.

Make your client aware of the risks if product design decisions are made without a good understanding of the user.

User Research Is The Basis Of Every Good User Experience

User experience is still a bit of a “mystery” in many circles: Everybody talks about it yet it is hard to define, as a good experience is in the eye of every user.

It is, therefore, key to gaining a sound understanding of the context, the user goals, and the thinking necessary for designing a truly exceptional user experience.

The more transparent you are with your work process, the better your client will understand your tools and the information you need to make good decisions.

While some clients may not be open to the idea of using additional resources on research, it’s necessary for experience specialists to explain the value of user research, and to argue for further research when necessary. To accomplish this, UX designers will require negotiating skills to make their case.

Luckily, proper user research is beneficial to clients and UX designers, so convincing clients to divert more resources towards research should be feasible in most situations. Reluctant clients may be swayed if you manage to devise a cost-effective user-research method, and I hope some of the tips and resources in this article will help boost user research, even if money is tight.

My Name is Shane and I’m a Usabilityholic

 

Yes, it’s a bit cheeky to make drugs and addiction the theme of a business presentation, but I think it paid off!


 

The Setup

I’ve been at my new job as a Software Engineer for Wall Street Systems (Part of ION Trading) for just over two months now. On the side, I’ve been slowly and steadily asking around and getting a feel for how we approach User Experience. Last week, my persistence paid off and I was asked to make a presentation to the company’s User Experience Community of Practice (CoP) on Usability Testing.

The CoP is a group of a few dozen people who have done some great work recently on the visual design of ION’s products. I wasn’t sure how much expertise already existed in the company with respect to user research, so I started my talk by polling the audience. It turned out that I was the most experienced person in the company on Usability Testing. It was a novel and exhilarating feeling to be “the expert” in the room. I guess I earned that though- over the past few years I’ve read and tried as much as possible in the world of UX, gravitating towards User Research such as Usability Testing.

The Sell

After explaining how easy, valuable, eye-opening, and addictive usability testing can be, I concluded my talk by offering to write a draft on best practices for running usability tests and to pilot a usability test.

I wasn’t sure what to expect going in, but I definitely didn’t expect a C-level to ask me to run a usability test the very next week- and that’s exactly what happened!

Sadly, I was on holiday the next week, but we’ll be doing a pilot hopefully before the end of this month!

The Software Speech

Meanwhile, my favorite London meetup, London Software Craftsmanship Community (LSCC), put out a call for speakers to do lightning talks at Skills Matter‘s beautiful new CodeNode facility. Initially, I shied away because I’ve never spoken publicly about software, but because the talk at work went so well, I decided to jump in and do it.

I had to rush through a little to get it down to 5 minutes but the audience was great and I’m glad I did it!

You can see a video of my talk here on the Skillsmatter website.

Just in case that link doesn’t work, the slides are below.