Match Debt: When your system and the real world don’t align

Twenty-two years ago, design legend Jakob Nielsen published 10 Heuristics for User Interface Design. These ten tips are just as relevant now as they were in the early days of the internet.

I consider them my ten commandments when it comes to designing software. In fact, this was the one and only thing that I displayed on my cubicle wall at my previous job.


My ten commandments (Forgive me father Nielsen, for I have sinned)

Anyway, the second heuristic on the list is:

Match between system and the real world

The system should speak the users’ language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

In other words, you want to make your product come as close as possible to the mental models of your users. In a perfect world, all products would closely align with how customers actually think.


The heuristic “Match between system and the real world” means that your product should align closely with the mental models of your users

Unfortunately, it’s not a perfect world.

In reality, there’s always a gap between how the product represents things compared to how the user actually thinks about them.


In reality, there’s probably a gap between your product and the mental models of your users

Actually, it’s often quite a large gap.


There may be a significant gap between your product and the mental models of your users

At my previous job, an established financial software company, the RTFM test revealed a feature that caused a lot of support requests: user access control. After speaking with some customers, we noticed that they were using different words than we were. What the system referred to as Access Areas, they called Functions. Also, the interface for user access control wasn’t consistent with the rest of the system. Our customers were used to using Excel-like tables to configure and view things, but this feature had a series of listboxes and dropdowns that were much less familiar or flexible. For this feature, there was definitely a gap between our product and the mental models of our users.

The Match Debt

Having this gap is not a huge problem (users tend to “satisfice“, settling for “good enough” solutions instead of fully understanding all options), but it is a cost. A cost that shows up in customer support, bug fixes, and worse: lost sales. I call that cost the Match Debt, because much like technical debt, the decisions you make today can make future development more expensive, like paying interest on a loan.


The Match Debt is the size of the gap between the system and the real world; the bigger the gap, the more money you’re losing

For our user access system, the Match Debt showed up most visibly in the form of calls to our support team. It also meant that the feature was hard to maintain. It may also have led to lost sales, though that is much harder to mention.

There are really only two ways to pay off Match Debt.

Educating your users

Firstly, you can try to move the users mental models closer to the way the product works via customer training and documentation- let’s call this education.


Education attempts to move your users’ mental models closer to the way your product works

Our user access system had thorough, well-written documentation, which I’m sure helped some of our customers, but users don’t read!

Learning about your users

Alternatively, you can try to move the product’s design closer to how the users think. This can be done in many different ways, including researching your users, designing solutions to their problems, testing these solutions, and more. But for simplicity’s sake, let’s just call this User Experience, or UX.


UX attempts to change your product closer to your users’ mental models

This is what we did when we redesigned the user access system. The new design uses slightly more familiar language and presents it in a way that’s consistent with the rest of the product. It’s nowhere near perfect, but now the Match Debt is lower, meaning that we’ll probably have fewer support calls and slightly happier customers than before.

It’s not you, it’s me

Both education and UX do the trick in reducing Match Debt, but from a customer’s perspective, UX is a whole lot nicer because it means that the product is doing the work, as opposed to the customer having to do the work. UX puts the onus on the few people who develop the product instead of the many people who use the product.

UX is the product team telling the user “it’s not you, it’s me”, then actually doing something to fix “it”. Isn’t that the way it should be?

Sure, this is a gross oversimplification, but this is why I love UX so much. It pays off Match Debt by aligning the product (and in fact the whole organization) closer to the users’ mental models.

It’s hard work. But it’s worth it.


Quotes from Interviewing Users by Steve Portigal

I just finished reading Interviewing Users. It’s one that has been on my list for a long time, probably because I loved Steve Portigal’s Dollars to Donuts podcast. It did not disappoint.

It was a great read for me because I’ve done enough user research to have some experience to map to, but not so much that I knew everything in the book.

Here’s a collection of quotes from the book that I found valuable, organized by theme.

Preparing for interviews

Portigal first walks the reader through what to do before interviewing users.

“At the outset of a project, make the objectives your initial priority.”

Interview 6-8 stakeholders (yes, you interview the team even before you interview the users!) to see what they want to learn. Sift through the politics and agendas and try to get everyone on the same page as to what the objectives are. Find issues such as

“…hypotheses masquerading as facts, aspirations, and mass hallucinations… resolve those issues as tactfully as possible.”

Prepare a field guide but

“this is not a script. It reads very linearly, but it’s really just a tool to prepare to be flexible.”

Research scheduling

Once you’ve agreed on the objectives and recruited participants, schedule two interviews a day or so. But why not more?

“Quality work doesn’t come from being rushed, exhausted, harried, or overwhelmed. Interviewing is hard work. You need time between sessions to reflect.”

And while it’s tempting to save time or resources by using Skype or a phone call or getting users to come to you, go to where the users are.

How many people should go to each interview?

“I find the ideal size for the field team to be two people: one to lead the interview, and one to back up the other person.”

Documenting the interview


Prepare a shot list before interviews that lists the photos that you want to take.

“Even if you’re capturing imagery using video, still pictures are essential.”

“Even though they agree to the use of photography when they sign your release, let the interview settle in before you start taking pictures. You can verbally confirm that it’s okay before you take your first picture.”

Taking notes

Note taking is okay but no one writes fast enough to capture everything and

“you must maintain eye contact while writing.”

If taking notes, just write the facts. Not judgements. But if you must interpret,

“it’s easy to lose track of what you were told versus the conclusions you made, so take care in how you document the two.”

As soon as possible after each interview, write a top-of mind summary.

“Allow time for debriefing after each interview… The longer you wait, the less you will remember, and the more jumbled up the different interviews will become.”

Video or audio

Video or audio recording is the only way to capture everything the participant says.

“Keep your eyes and brain in interview mode until you are fully departed.”

“consider keeping your recording device on, even if it’s packed”

because of

“the “doorknob phenomenon,” where crucial information is revealed just as the [therapy] patient is about to depart.”

Interview technique

Portigal offers lots of tips about actually conducting the interview.

They’re the expert

“make it clear to the participant (and to yourself) that they are the expert and you are the novice.”

“Asking that person to play the teacher role not only reinforces the idea that she is the expert here, but it also can make it easier for her to articulate the details you are seeking.”

Invite honest criticism

“if you bring out a concept by saying, “Here’s something I’ve been working on…” you’re activating a natural social instinct that will diminish their comfort in being critical.”

Building rapport

Start with general (easy) questions to get them warmed up. It will feel awkward and uncomfortable but accept and break down the awkwardness by

“accepting, acknowledging, and appreciating her responses.”

“Try to construct each question as a follow-up to a previous answer.”

If you change topic, “signal your lane changes”.


Once you’ve built rapport, you will reach a tipping point from question/answer to question/story.

“Stories are where the richest insights lie, and your objective is to get to this point in every interview.”

Use their language

“build rapport by accepting the terms the participant was using rather than trying to demonstrate credibility.”

“Don’t correct her perceptions or terminology if the only outcome is “educating” her. Advocate for her, not for your product.”

The interview isn’t about you

“[Only] talk about yourself if doing so gives the other person permission to share something.”

To dodge questions asked by participants,

“Do the Interviewer Sidestep and turn the question back to them: “Is that important to you?” “What would you expect it to be?””

Question Types

Ask about sequence, quantity, examples. Ask for exceptions, lists, relationships. Clarify comments, language/ code words, and emotional cues. Ask them to teach, to compare, etc.

You want to know about their needs. But you can’t directly ask about their needs. So you ask them indirectly.

“There’s a difference between what you want to know and what you ask.”

On participatory design

“My aim with participatory design is to give people a different way to talk about needs, where the solutions stand as proxies for those needs.”

You don’t actually have to implement any of these designs.


“One way a novice interviewer tries to counteract nervousness is by preemptively filling the silence.”

“Ask your question and let it stand…. After she has given you an answer, continue to be silent.”

Improving as an interviewer

Practice interviewing. Even in daily life.

“Cultivate a style of interacting socially that emphasizes listening, reflecting back the other person’s comments, allowing for silence, and so on.”

Also, review your interview recordings with a critical eye like a football coach would review game tape.

“ask someone to come along to your interviews and get his feedback. Also ask for feedback from the rest of your field team.”

Other tips include teaching others to interview and even taking an improv class, because that will improve your abilities of thinking on your feet, accepting awkwardness, and “going with it”.


Analysis and synthesis

After you’ve finished all of your interviews, follow a procedure like this to squeeze as much learning as possible out of your research:

  1. Divide and conquer: “get your data in text form and divide it among the team.” Each team member reviews and summarizes one or more interviews.
  2. Summarize for the team: “The group should then reconvene and present each interview as a case study.”
  3. After all summaries, recap: “Discuss each interview briefly, and then create a sticky note that summarizes the key point of that interview.”
  4. Group findings: “create groupings. You may want to start with the categories from your topline and add to them.”
  5. Prepare the report: “the Presentation of Findings, which is the main research deliverable.”

User research and organizations

Interviewing is a good way to increase a company’s UX Maturity. It…

“starts to drive shifts in the organizational culture”

“position yourself in your organization so that interviewing customers is an integral part of how you work.”


“The most impact for the least effort comes from your colleagues joining you in the field.”

For organizations with high maturity, research informs design but also exposes

“new opportunities for teams to embrace a user-centered approach to their work.”


I really enjoyed this book and I’m sure I’ll continue to refer to it (and these notes) for years to come!

The RTFM Test

As your team improves its UX maturity, it can be easy to be overwhelmed by all of the possible ways that you can improve your product.

One good place to start is the RTFM test.

How to do the RTFM test

It’s simple. Ask your support team:

What area of the product gets the most support requests that don’t end up in bugs being logged?

In other words: Which areas does the support team most often find themselves telling the customers to RTFM (Read The Freakin’ Manual)?

Or worse: Are there any areas that the support team has to keep repeating itself because they’re not even covered by the manual?

That is where you start. That’s the lowest hanging fruit. Those are the usabugs. Fix those and you’ll free up your support team to deal with real bugs.

But how?

Now that you know which area of your product is the biggest RTFM offender, there are many ways you can improve it. Let’s turn to Jakob Nielsen’s classic Ten Usability Heuristics for inspiration. Specifically, number ten:

Help and documentation

Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user’s task, list concrete steps to be carried out, and not be too large.

There’s a lot in there, so let’s break it down:


If at all possible, get rid of the freakin’ manual. “It is better if the system can be used without documentation.” This should be paired with user research such as usability tests to make sure you’re solving the real underlying issue.

For example, maybe the product uses a list of cryptic codes that the user must look up in the manual in order to understand what they mean. If so, provide that information on-screen instead.

Improve your documentation

If you can’t GROTFM, then follow Nielsen’s hints on how to improve your manual.

  • Provide easy-to-use search functionality
  • Center the documentation around user tasks, not system areas
  • Provide step-by-step instructions for accomplishing those tasks
  • Keep it short

As above, use user research to guide and validate your work on the documentation of your system.

Try it out!

It seems obvious in hindsight but before I redesigned the user access system at my previous company, I had dabbled with dozens of other ideas for improving the product. Sure, I was able to find lots of areas of the system that violated usability heuristics or clearly didn’t work as expected, but only when I discovered the RTFM test was I able to find something that was actually valuable to the business and to our users. The RTFM test allowed me to easily prioritize all those opportunities for improvement.

My Own Service Design Mentor

One piece of career advice I keep hearing (for instance in Whitney Hess’s interview on the UX Intern Podcast) is to get a mentor.

Why Get a Mentor?

The value of that is obvious: A smart, experienced person who you meet with regularly and learn the craft from? Sign me up!

Oh wait.

The Hard Part

There’s no actual sign-up sheet? You mean I have to actually go find someone and convince them to spend hours of their free time with me, chatting and teaching me about their career?

That’s the hard part.

It’s a really BIG ask.

So I’d been procrastinating on this for ages.

My Mentor

But then in late-2016 I met Mark Plant. He works at my company, ION, but in a different department. Whereas my team works on a specific product, Mark’s team is more of an internal design agency, working on developing various products, both internally and externally.

Basically, he’s doing my dream job 🙂

Testing the Waters

So I bought him a coffee and peppered him with questions about his career and the field of UX. It also came up that he’s into skiing and mountain biking, much like myself, so I asked him a lot about that as well.

Somehow the barrage of questions didn’t scare him off, so I offered to go for another coffee a week or two later. And by then he was already going through a step-by-step explanation of not just UX, but the wider fields of Service Design, Experience Design, and more. It was like he had a curriculum developed and was just waiting for some young designer like myself to ask him to teach them.

It’s Official!

And before I could even ask him to be my mentor, he offered!

We now meet every week or two to design a new app based on the Explor app from the AngelHack hackathon I did last year.



Some sketchy sketches of possible app interactions


He’s given me career advice, he’s shown me specific techniques, and he’s broadened my horizons beyond just pure UX.

For instance, I previously rolled my eyes at marketing. I sort of felt like it was UX but without the cool tech and research aspects. Mark suggested I read On Brand by Wally Olins, and boom. I am now filled with respect for the discipline of marketing.

Thanks Mark!

Giving Back

By the way, I won’t be as slick of a teacher as Mark, but if anyone reading this is just starting out in tech or UX and have a few questions or even just want someone to bounce design ideas off of, let me know! I’d love to have a mentee of my own.

It’s the circle of life!


Organizing Documentation with Card Sorting

We use Confluence at work for internal documentation. It’s kind of like Wikipedia, but only us developers can see and edit it, and the articles are things like How to set up Server x, or How to Test y.

I love it because we previously used long email chains with the answer buried deep within, mixed with dusty old word documents locked into source control. Confluence feels so much more lightweight, nimble, and searchable.

The Problem

Unfortunately over time it’s become a bit of a mess. We have over a hundred pages but there’s not much rhyme or reason as to how to find them. Confluence does have a search function but, believe it or not, people still browse. Also, if you’re adding an article it’s hard to know where to put it.

The Solution

So I decided to run a card sort exercise with my coworkers. My goal was to understand how they thought about these topics then organize the articles in a way that matched that as close as possible.

I scheduled a lunch and learn, invited the whole development team, and…

(Almost) No One Showed Up


A photo taken at my first attempt at a card sort workshop

Just one of my coworkers and my manager came. I should know by now that now all developers are as into UX as I am 🙂

Luckily, my manager agreed that this was important enough to schedule in and make mandatory, so a couple weeks later, the workshop was on!

The Actual Workshop

I hadn’t run card sorts recently, so I brushed up on my basics. The two main rules that stuck out for me:

  1. Run with one person at a time
  2. Never make people sort more than 30-50 cards at a time

I broke both those rules. Because I only had a one-hour workshop, I had everyone do the sorting in three small teams instead of individually. Also, because we had 100 articles, I did the sort with all 100 of them.


Team Pink sorts away. See guys? UX can be fun!

After that workshop, each team ended up with several named groups with several cards each.

Analyzing and Testing the Results

I then used a template from Donna Spencer to analyze the results. I’d definitely recommend this template. It’s free and it did the trick.

Each team came up with remarkably similar groups. Based on those groups, I then came up with proposed categories and tested with a quick tree test using TreeJack from OptimalWorkshop.

Photo 2017-03-31, 2 57 27 PM-edited

My attempt at making sense of the card sort results. Paper still beats Excel sometimes.

Based on the tree test, one of the cards was clearly in the wrong category so I moved it. A couple of the others weren’t decisive so I added cross-links, effectively giving those articles two homes instead of just one.


I’m very happy I did this. It was nowhere near perfect but it definitely vastly improved the structure of our documentation, showed everyone which articles were available, and introduced card sorting to the team.

Next Time

If I had more time I probably would have done the sorts one-on-one with my teammates instead of doing everyone at once in teams.

I should have used a smaller representative sample of the articles instead of trying to sort them all.

In other words, I should have followed the basic rules of card sorts!

Oh well! You can’t learn the limits if you don’t go past them every now and then, right?!

Ugly yet organized sketching

As a side project at work, I’m currently slowly chipping away at a troublesome area in a legacy desktop application. I’ve done some group ideation (brainwriting) and some usability testing in this area, but I hadn’t had much of a chance to really “sink my teeth” into the interface.

My favorite way to really get to know an interface?


I love sketching because it turns half-formed ideas into tangible things, often exposing flaws that you never knew were there.

I just returned from a trip to Germany and Poland. Anticipating lots of waiting and planes, trains, and buses, I set a goal of 100 sketches over the course of the week.

I ended up with 43.

A little disappointing, but I think what I lacked in quantity I more than made up for in quality.

Now that’s some quality sketchin’!

No, not aesthetic quality. They’re definitely not the prettiest of pictures. However, I feel like I made a breakthrough when it comes to how I lay things out.

Number sketches for linking

As you can see in the example, I started every sketch with a number. I only really started doing this because I wanted to know how many sketches I had done, but it had the secondary benefit of allowing me to refer to each idea one at a time and “link” between sketches. It really forced me to keep my sketches and ideas organized.

This was an improvement over my previous sketching binges where I’d just fill up the page with multiple ideas then move on. It also allowed me to branch out and come back to ideas instead of having to choose one at a time.

Pros and cons

Each page starts with a brief intro and often a “Why”. The sad faces show the doubts that I had after creating the sketch. Often, I address these doubts in later sketches. Or at least I try.


Thanks to this slightly more organized way of sketching, I can see myself getting much more value out of my sketches. It’s sort of like writing code. If you don’t put a little effort into making it clear what you’re doing, it’ll probably be incomprehensible to your coworkers- actually, it’ll probably be incomprehensible even to yourself a few months down the road!


How do you sketch? Can you recommend any resources for improving my sketches? Bill Buxton is on my list but are there any others that I should know about? Give me a shout out @sgryzko and let me know!

Communicating the Value of User Research

I recently came across a great guide to communicating the value of user research to stakeholders.

It seems to be written more for UX designers brought in on a contract or agency basis to help clients “do UX”. However, as a permanent developer who’s trying to “do UX” on the side, most of the points have rung just as true for me, if not more so.

It can be an uphill battle trying to justify the investment required for User Research, but by keeping things simple (keeping costs low) and clearly communicating the benefit of the research, progress can be made.

This was originally posted by Frauke Seewald on the Toptal blog, but they’ve given me permission to repost it here. Enjoy!

The beginning of a new project: Your client needs help with a redesign of its website or application.

“We want to improve the user experience, it has to be jaw-dropping for our customers, we want them to fall in love with our product.”

Here is the good news: Your client is aware of User Experience (UX), cares about customers’ needs and sees the value in investing in a great user experience. They asked for an expert with UX skills to help, but do theyreally understand what it means to deliver an exceptional user experience?

User research is a vital component of UX design. Don’t let anyone tell you otherwise.

User research is a vital component of UX design. Don’t let anyone tell you otherwise.

UX is more than a bunch of rules and heuristics that you follow in your product design process. UX is subjective, as the name suggests. It is the (subjective) experience that a user gets while using a product. Therefore, we have to understand the needs and goals of potential users (and those are unique for each product), their tasks, and context.

As a UX expert you should already be familiar with the maxim, It all starts with knowing the user.

Now for some bad news; this is the point when you discover your client’s misconceptions about UX.

UX expert: “Ok, let’s start with your users: Who are they? What do they do? What do they want? What are some of their pain points? I would like to talk to them, observe them, learn from them…”

Client: “Oh, we don’t need user research, that’s a waste of time.”


In this post I will try to explain why, and hopefully, help fellow UX specialists in their efforts to convince clients that good UX is next to impossible if it is not preceded by good user research.

No Need For User Research? There Is Always A Need For User Research

You cannot create a great user experience if you don’t know your users or their needs.

Don’t let anyone tell you differently. Don’t simply accept the common argument that there is no time or money to do any user research for your project.

User research should shape your product design and define guidelines that will enable you to make the right UX decisions.

User research should shape your product design and define guidelines that will enable you to make the right UX decisions.

User research will shape your product; it will define the guidelines for creating a product with a good experience. Not spending any time on research, and basing all of your design decisions on best guesses and assumptions, puts you at risk of not meeting your user needs.

This is how senior UX architect Jim Ross UXmatters sees it:

“Creating something without knowing users and their needs is a huge risk that often leads to a poorly designed solution and, ultimately, results in far higher costs and sometimes negative consequences.”

Lack Of User Research Can Lead To Negative Consequences

Skipping user research will often result in “featurities,” decisions that are driven by technical possibilities and not filtered by user goals.

“My wife would really enjoy this feature! Oh, and I heard from this person that they would like to be able to xyz, so let’s add it in there too.”

This leads to things such as overly complex dashboards in cars, where the user’s focus should be on driving, not on figuring out how to navigate an elaborate infotainment system.

Many users find automotive infotainment systems overly complex and distracting. Identifying the target audience is crucial to good UX design.

Many users find automotive infotainment systems overly complex and distracting. Identifying the target audience is crucial to good UX design.

Tesla’s cutting edge infotainment system, based on Nvidia Tegra hardware, employs two oversized displays, one of which replaces traditional dials, while the other one replaces the center console. Yes, it looks good, but it was designed with tech savvy users in mind. In other words, geeks will love it, but it’s clearly not for everyone. It works for Tesla and its target audience, but don’t expect to see such solutions in low-cost vehicles designed with different people in mind.

Poorly designed remote controls are not intuitive, so casual users tend find them overwhelming, resulting in a frustrating user experience.

Old remote controls are another example of hit and miss UX. There is little in the way of standardization, so each one takes time getting used to.

Old remote controls are another example of hit and miss UX. There is little in the way of standardization, so each one takes time getting used to.

But what about the purely digital user experience? Too many fields in a form, or too much information may overwhelm and drive your users away.

Poorly designed digital interfaces can drive users away. Even if they don’t, they will annoy users and feel like a waste time.

Poorly designed digital interfaces can drive users away. Even if they don’t, they will annoy users and feel like a waste time.

Instead of creating the opposite behaviour, poorly designed and implemented interfaces are more likely to scare off potential users.

Start User Research With Sources For Existing Information

Yes, user research will expand the timeline and it won’t come cheap, but both time and costs can be minimized. You can start with existing, and easy accessible, sources of information about user behaviour to gain a better understanding of user needs. These are:

  • Data Analytics
  • User Reviews and Ratings
  • Customer Support
  • Market Research
  • Usability Testing

Quality user research requires time and resources. However, you can start by using existing information to get a sense of what your users need.

Quality user research requires time and resources. However, you can start by using existing information to get a sense of what your users need.

Let’s take a closer look at each of these sources.

Data Analytics

If you are working with an existing product, your client might have some data and insights about its use. Data analytics assist with getting a good overview about general usage: How many visitors are coming to the website, what pages are most visited, where do visitors come from, when do they leave, how much time do they spend where, and so on.

But here is what this data is not telling you: How does the experience feel? What do users think about your service, and why are they spending time on your website? Why do they leave?

For example, your data indicates that users are spending a lot of time on a specific page. What it doesn’t tell you is why. It might be because the content is so interesting, which means users found what they were looking for. On the other hand, it could be an indication that users are looking for something they cannot find.

Data Analytics is a good starting point, but it needs further qualitative data to support the interpretation of the statistics.

User Reviews And Ratings

Your client’s product might have received some user feedback, already. There might be a section for feedback or ratings on the website itself, but external sources may be available as well. People might have talked about it in blog posts or discussion boards, users may have given app reviews in an app store. Check different sources to see what users are saying.

However, be aware of limitations. People tend to leave reviews and ratings about negative experiences. Don’t take this as a reason to shy away from user reviews or to ignore feedback!

“All these complainers… These aren’t the users we want, anyway!”

Instead, try to look for patterns and repetitive comments. Here are a few tips for making the most from user input:

  • Check whether any action has been taken on negative comments.
  • Compare the timing of negative comments to releases and changelogs. Even great apps can suffer from poor updates, leading to a lot of negative comments in the days following the update.
  • Do your best to weed out baseless comments posted by trolls.
  • What are users saying about the competition? Identify positive and negative differentiators.
  • Don’t place too much trust in “professional and independent” reviews because they can be anything but professional and independent.

User reviews are a good source for collecting information on recurrent problems and frustrations, but they won’t give you an entirely objective view of what users think about your product.

Your client might have a customer support hotline or salespeople who are in touch with the user base. This is a good resource to get a better understanding of what customers are struggling with, what kind of questions they have, what features/functionality they are missing.

Setting up a couple of quick interviews with call center agents, and even shadowing some of their calls, will allow you to collect helpful data without investing too much time or money.

Customer support provides you with a good opportunity to learn about potential areas for improvement, but you will have to dive in deeper to get detailed information about problems.

Market Research

Your client may have some basic information about the customer base, such as accurate demographic information, or a good understanding of different market segments. This information is valuable to understand some of the factors behind the buying decision.

It does not offer any information about the usage of the product, though.

Market research is a good source of information if you need a better understanding of how your client thinks, what their marketing goals are, and what their market looks like. However, it won’t reveal all relevant details about user goals or needs.

Usability Testing

If you are lucky, your client might have done some usability tests and gained insights about what users like or dislike about the product. This data will help you understand how people are using the product and what the current experience looks like.

It is not quantitative research, and therefore you won’t get any numbers and statistics, but it helps you identify major problems, and gives you a better understanding about how your user group thinks.

There is also the option to do some quick remote testing session by using services such as

Usability tests are another good way of identifying key problem areas in a product.

How To Educate Your Client About The Value Of User Research

The budget might be small and the timeline tight, but ignoring user research will eventually bite you. Help your clients avoid pitfalls by making them aware of the benefits of user research.

What’s the ROI of good user experience? Knowledgeable UX experts must be able to communicate the value of user research to clients.

What’s the ROI of good user experience? Knowledgeable UX experts must be able to communicate the value of user research to clients.

Here are some common arguments against user research and how to deal with them:

  • We don’t need user research. We trust in your skills as a UX expert

As a UX designer, you need to view user research as part of your toolkit, just like a hammer or saw for a craftsman. It helps you to apply your expertise in practice. No matter how much expertise you have as a designer, there is no generic solution for every problem. The solutions always depend on the user group and the environment, so they need to be defined and understood for every product.

User research will help get an unbiased view, to learn about users’ natural language, their knowledge and mental models, their life context.

You are the UX design expert, but you are not the user.

  • Just use best practices instead of research

Best practices originate from design decisions in a specific context; the digital industry is evolving at a rapid pace, design trends and recommendations change constantly, there is no fixed book of rules. We need to be able to adjust and adapt. Those decision should be made based on research, not practices employed by others, on different projects.

  • We already know everything about our users

Invite your client to a user needs discovery session to observe how users are using the product. Start with small tests and use remote usability testing tools such as to get some quick insights and videos of users in action.

The outcome might be a user journey map or a user task flow. Aim for a visualized document that identifies outstanding questions so you can define areas that need more research.

  • We have personas, we don’t need more research

Personas are a good tool for making your target group more tangible, and for becoming aware of different needs, key task flows and and how that might vary for different groups. It’s the common ground and a good starting point.

However, to redesign a product you need a better understanding of the usage. You need to know how people work with your product, what they do with it, when they get frustrated.

Ask for further details about user stories and task flows to make use of personas.

  • We don’t have the budget for it

The above list of sources for information about user behaviour should give you a good starting point for sharing ideas with your client on how to gain user information on a (very) tight budget.

Make your client aware of the risks if product design decisions are made without a good understanding of the user.

User Research Is The Basis Of Every Good User Experience

User experience is still a bit of a “mystery” in many circles: Everybody talks about it yet it is hard to define, as a good experience is in the eye of every user.

It is, therefore, key to gaining a sound understanding of the context, the user goals, and the thinking necessary for designing a truly exceptional user experience.

The more transparent you are with your work process, the better your client will understand your tools and the information you need to make good decisions.

While some clients may not be open to the idea of using additional resources on research, it’s necessary for experience specialists to explain the value of user research, and to argue for further research when necessary. To accomplish this, UX designers will require negotiating skills to make their case.

Luckily, proper user research is beneficial to clients and UX designers, so convincing clients to divert more resources towards research should be feasible in most situations. Reluctant clients may be swayed if you manage to devise a cost-effective user-research method, and I hope some of the tips and resources in this article will help boost user research, even if money is tight.

My Name is Shane and I’m a Usabilityholic


Yes, it’s a bit cheeky to make drugs and addiction the theme of a business presentation, but I think it paid off!


The Setup

I’ve been at my new job as a Software Engineer for Wall Street Systems (Part of ION Trading) for just over two months now. On the side, I’ve been slowly and steadily asking around and getting a feel for how we approach User Experience. Last week, my persistence paid off and I was asked to make a presentation to the company’s User Experience Community of Practice (CoP) on Usability Testing.

The CoP is a group of a few dozen people who have done some great work recently on the visual design of ION’s products. I wasn’t sure how much expertise already existed in the company with respect to user research, so I started my talk by polling the audience. It turned out that I was the most experienced person in the company on Usability Testing. It was a novel and exhilarating feeling to be “the expert” in the room. I guess I earned that though- over the past few years I’ve read and tried as much as possible in the world of UX, gravitating towards User Research such as Usability Testing.

The Sell

After explaining how easy, valuable, eye-opening, and addictive usability testing can be, I concluded my talk by offering to write a draft on best practices for running usability tests and to pilot a usability test.

I wasn’t sure what to expect going in, but I definitely didn’t expect a C-level to ask me to run a usability test the very next week- and that’s exactly what happened!

Sadly, I was on holiday the next week, but we’ll be doing a pilot hopefully before the end of this month!

The Software Speech

Meanwhile, my favorite London meetup, London Software Craftsmanship Community (LSCC), put out a call for speakers to do lightning talks at Skills Matter‘s beautiful new CodeNode facility. Initially, I shied away because I’ve never spoken publicly about software, but because the talk at work went so well, I decided to jump in and do it.

I had to rush through a little to get it down to 5 minutes but the audience was great and I’m glad I did it!

You can see a video of my talk here on the Skillsmatter website.

Just in case that link doesn’t work, the slides are below.

Screenshot from the entrepreneur first application video I made

Application Video- Entrepreneur First Program

The Program

Entrepreneur First is a program that operates under the theory that if you take a bunch of talented individuals, offer them mentoring and guidance and a small stipend to live off of, they will end up creating brilliant businesses. It appears that their theory is working quite well.

“To date, our founders have built companies with a total valuation of $250m…”

The Application

Naturally, I wanted in. Entrepreneur First has quite a lengthy application process. The first step is a questionnaire. Fortunately, they liked my questionnaire enough that I was able to move onto the next stage. For the next step, I was asked to submit a video answering the following questions:

  1. Tell us a bit more about the most impressive technical thing you’ve built. Why do you think it’s impressive?
  2. Tell us about a time when you’ve been relentless to achieve something you wanted. How did you do this?
  3. What do you think you would like to work on at EF? What skills do you have to build this?

I was very happy with my video, which I created quite easily, just using iMovie for iPhone. Check it out below!

The Result

Sadly I wasn’t selected to proceed to the next round of the application process but I understand- I don’t have any entrepreneurial experience; not yet at least!

But I did enjoy learning about EF and creating this video and I think it really showcases my tenacity when it comes to making things happen.

Customizing Google Forms and Sheets

The Client

Alberta Science Network (ASN) is a charity that is very close to my heart. They have several educational initiatives, including “Scientists And Engineers in the Classroom”. Through this program, from 2011 to 2015, I made over 40 presentations at local schools, teaching over 1500 students about Chemistry using magic and hands-on activities.
Here’s a quick video I whipped up when ASN wanted their volunteers to talk about what they do for the organization.
ASN approached me asking if I knew of any technologies or people who could help smooth out some of their processes. I initially declined because it was a busy time at work, but later approached the organization to see if I could help out.


Here’s a screenshot of the member form that was in use originally.
Original Form Screenshot
More importantly, here’s the original process:
  1. Member fills out form
  2. ASN staff gets an email with form data
  3. ASN staff copies form data into a local spreadsheet
  4. At end of year, all members are sent an email, asking them to fill out the form again

This is quite a labor-intensive process for staff, the data is not backed up, and it does not make it easy for members to renew.


As you can see below, the form was remade using Google Forms. The look and feel wasn’t significantly changed but a few unnecessary fields were removed and the fields were aligned and spaced further apart from each other.
New Form Partial Screenshot

The Process

Going into the project with ASN, I was hoping to be able to use Ruby on Rails (RoR) to make a custom solution because I had recently done the fantastic Rails Tutorial by Michael Hartl. After considering RoR and looking into Customer Relationship Management tools such as the open-source platform CiviCRM, I realized that Google Forms and Sheets could do the trick in considerably less time.
Google automatically inserts form data into a spreadsheet, but there were a few things that weren’t automatically supported. For example, ASN has annual memberships, so every June, a script goes through and sets all members to inactive then invites them to renew their membership. In order to improve the renewal rate, I used the following script to pre-fill the membership form for these renewal emails.
* Use Form API to generate pre-filled form URLs
* Based on
* Note: This script can either be run manually or linked to a trigger such as a time-based trigger.

function GeneratePrefilledURLs()
  var range = GetSpreadsheetDataRange();
  var data = range.getValues();
  var headerRow = data[0];

  // Use form attached to sheet
  var formUrl = SpreadsheetApp.getActive().getFormUrl();
  var form = FormApp.openByUrl(formUrl);
  var formItems = form.getItems();
  // Grab column numbers for later use.
  var urlCol = headerRow.indexOf("Prefilled URL");
  var activeMemberColumn = headerRow.indexOf("Approved Active Member");
  // Skip headers, then build URLs for each row in this sheet.
  for (var row = 1; row < data.length; row++)
    var activeMember = data[row][activeMemberColumn];
    // Only update for active members.
    if (activeMember == "y" || activeMember == "")
      // Create a form response object to prefill.
      var formResponse = form.createResponse();

      // Cycle through all form items that have matching columns.
      for (var i = 0; i < formItems.length; i++) { // Get text of question for item var ques = formItems[i].getTitle(); // Get col index that contains this question var quesCol = headerRow.indexOf(ques); var resp = ques ? data[row][quesCol] : ""; var type = formItems[i].getType().toString(); //Logger.log("Question='"+ques+"', resp='"+resp+"' type:"+type); // Treat Membership Type separately because we always prepopulate the same response (Renewal) regardless of what the data says. if (ques == "Membership Type") { var item = formItems[i].asMultipleChoiceItem(); resp = "Renewal"; } else { // Need to treat every type of answer as its specific type. switch (formItems[i].getType()) { case FormApp.ItemType.TEXT: var item = formItems[i].asTextItem(); break; case FormApp.ItemType.PARAGRAPH_TEXT: item = formItems[i].asParagraphTextItem(); break; case FormApp.ItemType.LIST: item = formItems[i].asListItem(); break; case FormApp.ItemType.MULTIPLE_CHOICE: item = formItems[i].asMultipleChoiceItem(); break; case FormApp.ItemType.CHECKBOX: item = formItems[i].asCheckboxItem(); // Resp is a CSV string. Convert to array of separate choices, ready for createResponse(). resp = resp.split(/ *, */);   // Convert CSV to array break; case FormApp.ItemType.DATE: item = formItems[i].asDateItem(); resp = new Date( resp ); break; case FormApp.ItemType.DATETIME: item = formItems[i].asDateTimeItem(); resp = new Date( resp ); break; default: item = null;  // Not handling DURATION, GRID, IMAGE, PAGE_BREAK, SCALE, SECTION_HEADER, TIME break; } } // Add this answer to our pre-filled URL if (item && resp != "") { try { // Store the item response in formResponse. If it doesn't work, just log the exception and move on. var response = item.createResponse(resp); formResponse.withItemResponse(response); } catch (e) { Logger.log(e.toString()); } } // else if we have any other type of response, we'll skip it else { Logger.log("Skipping i = " + i + ", question = " + ques + " type: " + type); } } // Generate the pre-filled URL for this row. var editResponseUrl = formResponse.toPrefilledUrl(); var shortUrl = UrlShortener.Url.insert({longUrl: editResponseUrl}).id; // If there is a "Prefilled URL" column, update it. if (urlCol >= 0)
        var urlCell = range.getCell(row+1,urlCol+1);

// Helper function that returns the size of an object.
// From
Object.size = function(obj)
  var size = 0, key;
  for (key in obj)
    if (obj.hasOwnProperty(key)) size++;

  return size;
I had never worked with Google Apps Script (GAS) or JavaScript (which GAS is based on) before, so it took a lot of reading, searching, and trial and error, but I made sure to document my code to make it easy for myself or others to be able to modify the code in the future. I’m sure that there are some “rookie mistakes” in there but so far, no problems have been reported and on-time member renewals almost tripled from 2014 to 2015. ASN was quite pleased with my work!