Usability testing is fun

Last week, a colleague and I ran an eight-person usability study with a big customer. We started at 6:30, so it was a super-early, super-long day.

Iterating on the product AND the process

Yes, it was exhausting, but I love this stuff. You take your best guess at a design then you put it in front of a bunch of people. Then they struggle with it and you make it better. That iteration what research is all about.

I also love how we’re iterating on the process itself. November was the first time I’ve traveled for usability testing. Yesterday was the first time I’ve done remote usability testing with clients. It was also the first time we’ve hosted a viewing party where people from all over the company could tune in and see what usability testing all about. Oh, and it was the first time we had a client representative observe all of the tests we ran. So many leaps forward!

What’s next?

When it comes to process, I wonder what is next for us. Maybe a focus on quality? For instance, we could start to critique each other on body language, tone of voice, note taking, and more. Or maybe more logistic stuff, like making scheduling better. Or smoothing out the participant recruitment pipeline.

But for now it’s good to keep enjoying it, keep learning, and keep iterating!

P.S. It was fun last time, too!

I just checked my notes from the November usability testing trip, and I had a similar rush of enthusiasm:

I’m exhausted.

Man is it draining to do this much research. It is not easy to sit in a little meeting room for a full eight hours, actively listening while thinking ahead and trying to be self aware enough not to be biased. Then summarizing findings and prepping for the next day’s tests with customized prototypes and doing it all again. That’s hard too.

But I love it.

-My own notes after a full week of usability testing


Feature photo stolen from @jdorn. Thanks, friend 😈!

Advertisements

Using Airtable to store research nuggets

I started at Solium in January. They did a full day of usability testing for World Usability Day in the previous November but hadn’t yet got through analyzing all of the videos. It’s a small team and other priorities got in the way.

Around that time, I listened to a Tomer Sharon interview on Mixed Methods, a UX Research podcast that I highly recommend.

In that interview, Sharon discussed how UX Research usually ends up in reports that are read once then forgotten, and how non-researchers make observations about users every day that aren’t documented.

The answer?

Track every observation, or nugget in the same place. The nugget is the atomic unit of research, not the report.

This prevents research from being forgotten, it gives transparency to how research is interpreted, and it “democratizes UX” by potentially allowing non-researchers to contribute and search research findings.

Here’s what I did

So I copied Tomer Sharon’s “Polaris UX Nuggets” base.

Screenshot 2018-06-21 06.08.34.png

Tomer Sharon’s Polaris UX Nuggets database- available for you to copy!

I knew that this wouldn’t be adopted by my team if it was too much work, so I simplified. And simplified some more. Including:

  • Removing provocations, journeys, acts, scenes, props, and sets
  • Combining “Experience Vector” and “Magnitude” because they’re two measures of the same thing and because “High Neutral” doesn’t mean anything. Thanks Cong for suggesting this!

Then tried it out.

Then documented.

Then tested it with a teammate.

Then moved our videos from Dropbox to a private YouTube channel and added a function to timestamp the nugget url. Thanks Jason for suggesting this!

Then tested it with another teammate.

Then simplified by creating a simple form with only 3 mandatory fields. Far more efficient and less overwhelming than dealing with a giant database.

Then tested it with my manager. Thanks Stuart!

Then simplified.

Then assigned all remaining videos to be nuggetized by the team.

Meanwhile we’re doing more usability tests and nuggetizing those as we go.

The Results

This has already come in handy twice.

First when my coworker Jason was considering adding a hyperlink in a certain place, I expressed my doubts. Luckily, Jason remembered some recent observations and within a couple minutes he had pulled up Airtable and showed me two clips of real customers trying to click the text that he wanted to hyperlink. Damn you Airtable! I gave you life and you’re already proving me wrong!? Seriously though, these observations would not have been found so easy if they were just buried in a usability report somewhere

Second, when we were having a mini design sprint on one problem area in our dashboard, I was able to quickly pull up several short video clips of real customers running into usability issues in that area. It’s great that this database can essentially provide a highlight reel for anything you need!

Next Steps

Going forward, the UX team at Solium is continuing to evaluate Airtable this will continue to evolve, but so far I’m very grateful that Tomer Sharon was sharin’ his database! #punIntended #dadJoke #badJoke

Appendix: How to generate timestamped Youtube URLs

For all the formulas I used, make a copy of Youtube url generation on airtable. Using that, you should be able to get from a youtube share link to a timestamped start/end youtube url.

Example youtube share link: https://youtu.be/1SXhEw1ijs4

Nugget timestamp: 24:00-24:13

Timestamped youtube url: https://youtube.com/embed/1SXhEw1ijs4?autoplay=1&rel=0&start=1440&end=1453

It’s okay to be slow

This post is #7 in a series of posts about my UX research about UX Maturity. For all other posts, see my UX Maturity Research page.


 

Last week I wrote a long post summarizing my progress so far. At the end, I discussed some of my constraints, including the following:

 

Do TWICE as much with HALF the resources

  • Don’t do more; do it differently
  • Can’t sacrifice speed

 

Which came from numerous quotes in my research, such as:

  • “We [designers] slow things down because we think about them longer. “
  • “Can’t slow down velocity on getting features done”
  • “Product owners need to demonstrate progress. If feels like we’re interfering with their status updates. “

But now I’m starting to have my doubts.

 

Maybe you can sacrifice speed

I recently realized that while it would be great to speed up design, that might not be the best approach to increasing an organization’s UX maturity.

The reality is that good design does slow things down.

But isn’t it better to walk in the right direction than to run in the wrong direction?

Yes, well-designed solutions take longer than just getting features done, but they make your users happy, they are less buggy, more maintainable and extensible, and they differentiate you from your competitors.

My research was focused on mid-maturity companies, where people told me that design has to be fast. But I’m now learning that high-maturity companies don’t think that way. As UX Designer Barbara Shain told me about a past employer:

“There isn’t just a perception that design slows down development– we all accept it as a fact.”

In other words, yes UX is slow, but no that’s not a problem.

Related image

 

But speed is still important

As I was writing this post, I decided to look up some transcripts of the podcast that first introduced me to User Research 3 years ago: Dollars to Donuts. Turns out that many of the mature companies featured in this podcast do still have speed pressures.

I think if we double the size of the research team, I’m not sure that would be in the best interest of the company. I think that might actually slow things down.

-Alex Wright, Etsy

[The research team] got our foot in the door with some product teams and showed what kind of impact we could have and how… rapidly we could do this work… How it could speed up development in some cases.

-Aviva Rosenstein, DocuSign

But again, speed isn’t everything.

You can go too fast if you skip especially those early important steps of focusing on ‘Okay, let’s clarify, what are we doing?’ … if you cut the wrong corners, there’s a “gotcha” at the end.

-Kerry McAleer-Forte, Sears Holdings

Like many things in life, this is a very delicate balancing act.

 

What do I do now?

I’m so confused. But that’s normal in research.

Going forward, I’ll continue exploring ways to incentivize teams to care about design while keeping up the speed. However, I’ll also look into ways of embracing the slowness!

The first diamond

This post is #6 in a series of posts about my UX research about UX Maturity. For all other posts, see my UX Maturity Research page.


The double diamond

The double diamond is a great model for the design process. This Kayla Heffernan blog post describes it in more detail, but it boils down to two phases:

  1. First you figure out which problem to solve,
  2. then you solve it.

In both phases, you first diverge (go broad), then converge (go deep). Hence the diamond shape!

My UX career has really focused on the second diamond (solving problems) as opposed to the first (figuring out which problems to solve). Which is why I’ve focused my work so far on the first diamond (figuring out which problem to solve). It is so valuable to take the time to understand the problem space before diving in and exploring solutions.

Before I jump into the second diamond (fixing the problem), I’m taking the time to recap my progress so far.

My hunt statement

So where did I start this first diamond? With the following hunt statement, which I carefully crafted when I started this research earlier this year.

I want to learn how decision-makers at enterprise software companies understand what UX design is, and how it fits into the success of their business;

So that I might develop more effective approaches for increasing investment in UX and getting the company to move further along the scale of UX maturity.

As I reread this statement, I’m almost embarrassed by it. But that’s a good thing.

“If you’re not embarrassed by the first version of your product, you’ve launched too late.” — Reid Hoffman (source)

The fact that I cringe as I read that means that I’ve learned something since I wrote it!

Like what?

UX growth doesn’t just come from decision makers

Firstly, I really didn’t focus my research on decision makers. I did have formal interviews with one CEO and a couple of managers, but I balanced that with several chats with “worker bees” like myself.

I did this because as I wrote about in The Three ITs of UX Growth, average employees don’t have to waIT for decision makers to force UX growth; they can commIT to UX growth themselves!

Increasing investment may not be the right choice

Fred Brooks said way back in 1975 that doubling the number of programmers on a development team does not double the speed of that team.

The same goes for designers. As one designer who I spoke with put it:

“We don’t need more funding. We need different funding”

Marcos Lopez, CEO of Solium, put it nicely during my talk with him:

“Things work when they’re not forced… organically start to say this is how we build things.”

Increasing investment isn’t just unnecessary for UX growth, but it might actually be a hindrance. I’ve heard stories of multiple conflicting design systems being developed in parallel by different design teams in the same company. That might not have happened if there were fewer resources available and the team had to run a little more ‘lean’.

There’s more than just one “scale”

As you can see in my UX Maturity Models summary, there are dozens of different “scales” for measuring UX growth, so it’s a little ambiguous to say “move along the scale of UX maturity”. Which scale?!

I niched down

I didn’t end up looking at a broad sample of enterprise software companies. Most people who I spoke with were with “mid-maturity” software companies, meaning companies where UX was accepted and encouraged, but where UX was not involved in the overall strategic direction of the company.

This wasn’t really on purpose but was a bit of a happy accident because I ended up seeing a lot of similarities between the companies that I spoke with.

My Methods

So what did I do to address that hunt statement? I did both primary and secondary research.

Primary Research

Armed with tips from Steve Portigal’s great book Interviewing Users, I conducted seven different formal interviews and dug up notes from two recent informal conversations about UX maturity. In these 20-45 minute chats I asked questions including

  • (Show a summary of Neilsen’s 8-stage UX maturity model) Where on this scale is your company? How about when you first started there? If it has changed, why?
  • What would you like to change about UX at your company?
  • Other than Development, which departments should UX collaborate with?

I then summarized all notes in one big spreadsheet then did two rounds of Research synthesis to pull out themes and patterns from these talks. Below is an image of one of those synthesis sessions.

IMG_0407.JPG

Secondary Research

There is a lot of work that has already been published on UX growth. Reading that work is what’s called secondary research.

In addition to collecting and analyzing different UX Maturity Models, I read several case studies, including this paper on Applications of Maturity Models and this UXmatters article on IBM. I also read the very actionable book UX Revolution by Paul Boag.

These sources were great for giving me a broader view of the problem space and provided a bit of a sanity check of my own research.

Speaking

I also recently spoke about my ideas. Signing up for a ten-minute talk at a UX meetup forced me to really distill my thoughts down into something simple (see The Three ITs of UX Growth) and also gave me a few insights and introductions from the audience!

Themes/Findings

The grand-daddy problem

After clustering and analyzing my notes from this research, the main problem I’ve found is that even with a shared vision, UX growth can be stunted by the lack of a shared “how”. To encourage UX growth, we need to change the default “how”.

Even with a shared vision, UX growth can be stunted by the lack of a shared “how”; to encourage UX growth, we need to change the default “how”

Design constraints

To solve this problem, I’m going to keep in mind the following constraints, which were also based on my research:

  1. Do TWICE as much with HALF the resources
    • Don’t do more; do it differently
    • Can’t sacrifice speed
  2. They’ll think it was their idea (INCEPTION)
    • Change can’t be forced
    • Should happen organically
  3. From ANY team to ANY team
    • UX might be a pot calling the kettle black
    • UX needs to be more self-aware

Specific design prompts

I summarized my findings into three possible design prompts.

Firstly, how might we clearly trigger when one team should involve another team?
Software development can be a messy, disjointed process, which makes it hard to join in and act as a team. It can also be hard to know when the design team should be involved (or when the design team should get help from other teams.

Next, how might we improve the design of products without slowing things down? Design needs to be able to jump in without delaying projects.

Lastly, how might we incentivize teams to be fast and produce well-designed solutions? We need to incentivize the right process as opposed to speed only. The design team must be able to keep pace.

 

What’s next?

While each of these prompts has tons of potential, I decided to focus on option 3 for the sake of time:

How might we incentivize teams to be fast and produce well-designed solutions?

Research

Yes, more research. My research thus far has only scraped the surface of how teams are incentivized. So before jumping into fixing the problem, I have to loop back to that first diamond to get a little deeper understanding of it.

Brainstorming

I love this part. Quantity over quantity. I’ll be sketching at least 25 different ideas for addressing this design prompt. After that, I’ll work with my mentor Meaghan to evaluate the ideas and iterate on them. Eventually, I’ll further explore some of the better ideas. Stay tuned!

 

Reflection

So far, this research has been great. It’s always fascinated and frustrated me that for some people and some companies, UX just seems so obvious, whereas for others, users are seen as annoyances. I’ve enjoyed sinking my teeth into this puzzle while learning about research.

Even though the mentorship program that prompted me to start this project ends next month, I could see myself exploring this problem space for years to come.

The Three ITs of UX Growth

This post is #5 in a series of posts about my UX research about UX Maturity. For all other posts, see my UX Maturity Research page.


 

My journey

I used to be a developer. I read as much as I could about UX, then really tried to put those readings to use. There were a lot of successes, but these great UX ideas that everyone was raving about online didn’t always catch on as much as I expected them to.

So I started learning about UX Maturity models. I even compiled a list of models. And I embarked on a research project with the help of my mentor Meaghan, interviewing people about UX and scouring the internet for insights.

That research is still ongoing but I’d like to share a super high-level summary.

The Three ITs of UX Growth

When it comes to leveling up your UX game, there are really only three things you can do.

1) commIT

Some companies up their UX game thanks largely to the efforts of a single person. In fact, my employer Solium is one of them. Not too long ago, we were having problems improving the UX of our products. But then a developer decided to take things into his own hands. He did a prototype of a redesign for one of the company’s main products and showed it to the leadership team. They loved it and built a team around him. That app is being released to the first major wave of customers this week, as shown in the blog post below!

new participant experience

A recent Solium blog post announcing a redesign that started as a side project

Obviously this one developer wasn’t the only person who cared about UX in the company, but his commITment and results led to a tipping point within the company and some very positive changes.

2) waIT

UX is becoming more and more mainstream. From articles on Design Thinking in the Harvard Business Review to reports of enterprise software being rejected by clients due to bad UX, executives are more aware than ever of the value and practices of UX.

Picture a typical customer of an old-skool enterprise software application. Let’s call him Jon. Jon wakes up one day and with two taps on the CityMapper app, he finds the best train route to work. While on the train, he does some leisurely scrolling through Airbnb and books a place for his upcoming vacation. He’s feeling pretty good… until he gets to work and has to try to do his job using a hideous, mind-numbing interface originally developed for Windows 95.

oneofthesethingsisnotliketheother.png

One of these things is not like the other!

Don’t you think that Jon is going to eventually wonder why his work software can’t be a little more like the sweet apps on his phone?

He is.

So if you wait long enough, the executives at your company are probably eventually going to hear complaints about bad UX and be aware of how to fix them!

This is exactly what happened at a financial institution that I spoke with.

“[The change] came top down actually. It wasn’t grassroots. [Prompted by] reality… banks must find unique ways of differentiating or they won’t be able to continue existing for much longer.”

This bank is now undergoing huge UX growth and seeing great results because of it.

3) quIT

If you tried to do IT and you can’t waIT any longer, do what I did: quIT your current job and find a company, like Solium, that really values User Experience.

iquit.png

If you want to work for a company that values UX, go work for a company that values UX!

Obviously no company is perfect, but Solium has gone through its UX breakthrough, which means I can spend more time solving problems and less time trying to convince other people that they are really problems.

The talk

I know this isn’t anything groundbreaking or particularly interesting, but writing and speaking has always helped me clarify my thoughts. I’ll be speaking about this later this week at the Calgary UX meetup in a ten-minute Rapid Fire talk.

Do you agree that UX Growth boils down to these three things? If not, why?

More questions than answers

This post is #4 in a series of posts about my UX research about UX Maturity. For all other posts, see my UX Maturity Research page.


 

I’m now past the halfway point in my six-month mentorship program. Looking back, I’ve done pretty well on the goals I published back in January.

1. In order to learn more about UX maturity and UX research techniques, I will interview at least 5 people from different companies about UX maturity by April 1st, 2018.
2. At that point, I will decide on a specific direction for future research.
3. Throughout the process, I will blog at least twice a month about my process and progress.

Pretty well but not great.

  1. I’ve interviewed not 5, but 6 different people about UX maturity (good) but they have all been from my company (not so good)
  2. I’ve decided on a direction for future research (good- I’ll get to that later)
  3. I have fallen behind on blogging twice a month (not so good)

So why no bloggin’?

I’m still writing at least 10 minutes each week as part of my Thursday Tens habit, so there’s no shortage of draft posts. Yet I’ve been reluctant to publish anything because I haven’t had any obvious insights from my research.

Isn’t research supposed to lead to a bunch of lightbulbs going off in my head?

Not only have I had no aha moments, but I was starting to doubt the whole idea of getting companies to increase UX maturity. Can UX maturity be sped up? If so, should it be sped up?

In my meeting last week with my mentor Meghan, I mentioned this frustration to her. How she responded surprised me:

“Good new questions are just as good of a research output as plain old answers… designers need to be comfortable moving forward despite a total lack of clarity and information”

So this whole process wasn’t fruitless after all! Sure, I have no answers yet, but I now have several questions that have been informed by research. I also have a little bit of practice being in this uncomfortable situation of trusting the process in spite of its haziness.

Oh, and I now have a blog post. The source of my writers block is now a source of strength, thanks to Meaghan’s encouragement!

Next steps

Now that I have a good feel for how UX evolved at my company, I’m going to move on to other companies.

I’m also going to do some serious secondary research, combing the internet for similar research that has already been done. Luckily I already have a good start on that thanks to my initial interest in UX Maturity models back in 2016!

Oh, and I’m going to get back in the habit of bloggin’. Twice a month, of course!

 

Research synthesis

This post is #3 in a series of posts about my UX research about UX Maturity. For all other posts, see my UX Maturity Research page.


 

Over the past few weeks, I’ve been cornering a variety of coworkers and chatting with them about how they relate to User Experience. Even though I’ve only worked there for two months and I’ve only had four interviews, I already have a good feel for the history and the current dynamic of UX at my company.

For each of these chats, I went in with a discussion guide and went out with pages and pages of notes and quotes.

The other day, I worked with my mentor Meaghan on tying to make sense of these notes. By the way, Meaghan’s company Mikata Health was recently selected for the AccelerateAB conference!

Setting up

To prepare for this synthesis activity, I broke up my notes into tweet-sized chunks. I then printed these onto cardstock and ended up with over 60 cards. I didn’t have access to a colour printer so I just marked each card with a highlighter so we knew which card was from which interview.

Sorting

I’ve done card sorts before to help set up a site’s information architecture, but never to synthesize research.

The general idea is the same though: Take one card at a time and put it close to other cards that “feel” similar. Every now and then you’ll need to move one cluster close to another, or split up groups of cards. The fancy term for this is “cluster analysis”.

Eventually you’ll end up with groups of cards. Give these groups a name or a theme. You’ll end up with something like this.

 

And then what?

A lot of interesting themes were revealed when we did this sorting activity. We came up with 14 of them.

There were a lot of insights that I wouldn’t have noticed if I hadn’t done this exercise. It’s strange: I was the one who did all the interviews, yet it wasn’t until I saw everything laid out in front of me that these things clicked.

But even though lots did “click”, I still wasn’t sure where to go. Meaghan explained to me that this “fuzzy” feeling is normal.

My next task: digest it. I’m going to review my notes and my themes, then use this to guide my next round of interviews.

Stay tuned for more on my process and findings!

The Hunt Statement

This post is #2 in a series of posts about my UX research about UX Maturity. For all other posts, see my UX Maturity Research page.


 

As I mentioned in my previous post, I’ve recently started researching how companies relate to UX Maturity.

Since that first post, I’ve been in brainstorming mode. I’ve been jotting down tons of questions, as well as a huge list of people that I might like to ask those questions to. At first it was exciting to get into writing all of the possibilities that were in my mind, but it quickly became a little overwhelming.

Narrowing things down

Enter my fearless mentor Meaghan. When we last sat down together, we had the goal of focusing my list of questions. But how?

The hunt statement

As Meaghan says, the hunt statement is your ‘north star’ that you align all of your research towards to keep your research focused. The format is a what followed by a why. Meaghan suggested filling in the blanks in the following way:

I want to learn ____________

So that I might ____________.

This is going to come in very handy, because I’m really quite curious.

 

Will ferrell meme: curious like a cat

While no one actually calls me Whiskeres, I’m also curious like a cat.

Just this week, when I tried Guerrilla User Research for the first time (in a work project), my research partner kept pointing out how long I was taking. I didn’t understand why he thought it was a problem- as long as the people I was talking to were happy to continue, why not keep asking questions?

But now I realize that some of my questions weren’t especially relevant to the problem that we were investigating. If we did have an explicit hunt statement for that study, we would have been on the same page I wouldn’t have been pulling on threads that my partner wasn’t interested in.

So for this UX Maturity Research project, I’m going to have a hunt statement to keep me in line.

My hunt statement

I want to learn how decision-makers at enterprise software companies understand what UX design is, and how it fits into the success of their business;

So that I might develop more effective approaches for increasing investment in UX and getting the company to move further along the scale of UX maturity.

Next steps

Using this hunt statement, I’m going to create an initial discussion guide, do a couple interviews this week, then refine the guide if necessary and continue the research from there!

Stay tuned for future updates 🙂

Kicking off a UX Research side project with my mentor Meaghan

This post is the first in what will be a series of posts about my UX research about UX Maturity. For all other posts, see my UX Maturity Research page.


 

I’ve long been obsessed with the concept of UX Maturity. Why do some companies rave about the power of User Experience Research and Design while other companies just don’t seem to get it?

Which is why, for at least the next 2 months, I’ll be conducting research on this topic.

How it began

I’ve had great mentors (and unofficial mentors) in the past, so when a friend tweeted about the Chic Geek mentorship program, I jumped at the opportunity.

The program is incredibly well-run. They match mentors with mentees based on a thorough questionnaire about goals, experience, and interests.

My match is Meaghan Nolan, a co-founder of the digital health company Mikata Health and a former lead designer at New York agency Moment. Needless to say I’m happy with my match!

But what do I want?

After I learned that Meaghan was going to be my mentor, the organizers asked me as the mentee to come up with a SMART goal. I really wasn’t sure what to shoot for, so I set a goal to set a goal. (Huh?!)

goal to set a goal.png

My waffly initial goal: To set a goal

Fast-forward to the first official mentorship meeting: When I sheepishly showed Meaghan my somewhat confusing goal, she immediately jumped into researcher mode. After just a few questions, she proposed a WAY better goal.

If you are interested in tech and UX research, why don’t you do UX-style research on tech?

Yes! Brilliant!

We eventually settled on the following:

  1. In order to learn more about UX maturity and UX research techniques, I will interview at least 5 people from different companies about UX maturity by April 1st, 2018.
  2. At that point, I will decide on a specific direction for future research.
  3. Throughout the process, I will blog at least twice a month about my process and progress.

They aren’t especially specific or measurable, but UX rarely ever is. The April 1st deadline is in place in order for me to get some broad initial research in before drilling down into something interesting or unanswered from that research.

Next steps

After this first meeting, Meaghan sent me some relevant links. One of which was a talk SHE GAVE at the Interaction17 conference on Design at Scale. What!? My mentor not only has worked with everything from startups to Fortune 500 companies, but she’s given a talk on design culture?! Amazing.

After I read through the rest of the materials that Meaghan sent, I’ll put together a rough research plan based on the IDEO Field Guide for Human-Centered Design.

Then of course I’ll actually start talking to people and post about it right here later this month! I can’t wait!

Want to talk?

Care to enlighten me on your personal journey through UX maturity? Let me know!

Quotes from Interviewing Users by Steve Portigal

I just finished reading Interviewing Users. It’s one that has been on my list for a long time, probably because I loved Steve Portigal’s Dollars to Donuts podcast. It did not disappoint.

It was a great read for me because I’ve done enough user research to have some experience to map to, but not so much that I knew everything in the book.

Here’s a collection of quotes from the book that I found valuable, organized by theme.

Preparing for interviews

Portigal first walks the reader through what to do before interviewing users.

“At the outset of a project, make the objectives your initial priority.”

Interview 6-8 stakeholders (yes, you interview the team even before you interview the users!) to see what they want to learn. Sift through the politics and agendas and try to get everyone on the same page as to what the objectives are. Find issues such as

“…hypotheses masquerading as facts, aspirations, and mass hallucinations… resolve those issues as tactfully as possible.”

Prepare a field guide but

“this is not a script. It reads very linearly, but it’s really just a tool to prepare to be flexible.”

Research scheduling

Once you’ve agreed on the objectives and recruited participants, schedule two interviews a day or so. But why not more?

“Quality work doesn’t come from being rushed, exhausted, harried, or overwhelmed. Interviewing is hard work. You need time between sessions to reflect.”

And while it’s tempting to save time or resources by using Skype or a phone call or getting users to come to you, go to where the users are.

How many people should go to each interview?

“I find the ideal size for the field team to be two people: one to lead the interview, and one to back up the other person.”

Documenting the interview

Photography

Prepare a shot list before interviews that lists the photos that you want to take.

“Even if you’re capturing imagery using video, still pictures are essential.”

“Even though they agree to the use of photography when they sign your release, let the interview settle in before you start taking pictures. You can verbally confirm that it’s okay before you take your first picture.”

Taking notes

Note taking is okay but no one writes fast enough to capture everything and

“you must maintain eye contact while writing.”

If taking notes, just write the facts. Not judgements. But if you must interpret,

“it’s easy to lose track of what you were told versus the conclusions you made, so take care in how you document the two.”

As soon as possible after each interview, write a top-of mind summary.

“Allow time for debriefing after each interview… The longer you wait, the less you will remember, and the more jumbled up the different interviews will become.”

Video or audio

Video or audio recording is the only way to capture everything the participant says.

“Keep your eyes and brain in interview mode until you are fully departed.”

“consider keeping your recording device on, even if it’s packed”

because of

“the “doorknob phenomenon,” where crucial information is revealed just as the [therapy] patient is about to depart.”

Interview technique

Portigal offers lots of tips about actually conducting the interview.

They’re the expert

“make it clear to the participant (and to yourself) that they are the expert and you are the novice.”

“Asking that person to play the teacher role not only reinforces the idea that she is the expert here, but it also can make it easier for her to articulate the details you are seeking.”

Invite honest criticism

“if you bring out a concept by saying, “Here’s something I’ve been working on…” you’re activating a natural social instinct that will diminish their comfort in being critical.”

Building rapport

Start with general (easy) questions to get them warmed up. It will feel awkward and uncomfortable but accept and break down the awkwardness by

“accepting, acknowledging, and appreciating her responses.”

“Try to construct each question as a follow-up to a previous answer.”

If you change topic, “signal your lane changes”.

Stories

Once you’ve built rapport, you will reach a tipping point from question/answer to question/story.

“Stories are where the richest insights lie, and your objective is to get to this point in every interview.”

Use their language

“build rapport by accepting the terms the participant was using rather than trying to demonstrate credibility.”

“Don’t correct her perceptions or terminology if the only outcome is “educating” her. Advocate for her, not for your product.”

The interview isn’t about you

“[Only] talk about yourself if doing so gives the other person permission to share something.”

To dodge questions asked by participants,

“Do the Interviewer Sidestep and turn the question back to them: “Is that important to you?” “What would you expect it to be?””

Question Types

Ask about sequence, quantity, examples. Ask for exceptions, lists, relationships. Clarify comments, language/ code words, and emotional cues. Ask them to teach, to compare, etc.

You want to know about their needs. But you can’t directly ask about their needs. So you ask them indirectly.

“There’s a difference between what you want to know and what you ask.”

On participatory design

“My aim with participatory design is to give people a different way to talk about needs, where the solutions stand as proxies for those needs.”

You don’t actually have to implement any of these designs.

Silence

“One way a novice interviewer tries to counteract nervousness is by preemptively filling the silence.”

“Ask your question and let it stand…. After she has given you an answer, continue to be silent.”

Improving as an interviewer

Practice interviewing. Even in daily life.

“Cultivate a style of interacting socially that emphasizes listening, reflecting back the other person’s comments, allowing for silence, and so on.”

Also, review your interview recordings with a critical eye like a football coach would review game tape.

“ask someone to come along to your interviews and get his feedback. Also ask for feedback from the rest of your field team.”

Other tips include teaching others to interview and even taking an improv class, because that will improve your abilities of thinking on your feet, accepting awkwardness, and “going with it”.

 

Analysis and synthesis

After you’ve finished all of your interviews, follow a procedure like this to squeeze as much learning as possible out of your research:

  1. Divide and conquer: “get your data in text form and divide it among the team.” Each team member reviews and summarizes one or more interviews.
  2. Summarize for the team: “The group should then reconvene and present each interview as a case study.”
  3. After all summaries, recap: “Discuss each interview briefly, and then create a sticky note that summarizes the key point of that interview.”
  4. Group findings: “create groupings. You may want to start with the categories from your topline and add to them.”
  5. Prepare the report: “the Presentation of Findings, which is the main research deliverable.”

User research and organizations

Interviewing is a good way to increase a company’s UX Maturity. It…

“starts to drive shifts in the organizational culture”

“position yourself in your organization so that interviewing customers is an integral part of how you work.”

How?

“The most impact for the least effort comes from your colleagues joining you in the field.”

For organizations with high maturity, research informs design but also exposes

“new opportunities for teams to embrace a user-centered approach to their work.”

Conclusion

I really enjoyed this book and I’m sure I’ll continue to refer to it (and these notes) for years to come!