How to do detailed first click analysis on UsabilityHub data

What is UsabilityHub?

UsabilityHub is a user research tool where you can build surveys and various simple tests.

The story

I recently used UsabilityHub to evaluate four variations of a fundamental design system change. I wrote about this in Expert Quant UXR advice from Sauro and Lewis.

The study was set up as a Variation Set, meaning that we ran the same test on four different designs. There were several questions, including a couple of First Click Tests and Short Text Questions. These all had clear right or wrong answers.

After we received all of our responses, I played around briefly in UsabilityHub. I highlighted the correct answers on the First Click Tests and tagged some of the Short Text Question responses as “correct”.

Then I exported my results as CSV.

The problem

When I opened the CSV, I saw my tags from the Short Text Questions, but my highlights weren’t included in the export.

The workaround

When I got in touch with UsabilityHub support, they suggested a workaround.

Filter your results to only include correct First Click responses
Export the filtered results

The problem with this workaround is that I wanted to analyze all the responses; not just the responses with correct First Click answers. So to work around this, I could have done the following:

Filter the results to only include incorrect First Click responses
Export the filtered results
Combine the two exported files, adding a column to show whether the First Click Test was correct for that response.

The problem with this workaround is that I had two First Click Tests and four design variations. Multiply that by the two types of responses (correct or incorrect) and you get 16 different files that you have to stitch together. Also, the four design variations had identical “correct” hit areas, so I wanted their highlight dimensions to be identical, which is hard when you’re just clicking and dragging on an image.

The solution

While UsabilityHub doesn’t export highlights, it does export First Click Test coordinates. The exported coordinates are fractional, meaning that (0,0) is the top-left of the image and (1,1) is the bottom-right.

Knowing this, you can convert your “correct” area to a fractional bounding box, then use Excel or Google Sheets to calculate whether or not a given pair of coordinates are correct.

Here’s how:

View the Click Map for your First Click Results then Export click map as image
Open the image in your favorite image editor, such as Sketch, and draw the “correct answer” box on top of the click map
Take note of the dimensions (W and H) of the image, making sure that X and Y are both 0

Example Click Map dimensions from Sketch’s Inspector panel

Take note of the pixel values for x_min, x_max, y_min, and y_max of the box you drew

Example “Correct” rectangle dimensions from Sketch’s Inspector panel

Using the numbers in the example above,

x_min = X = 493
x_max = x_min + W = 493 + 86 = 579
y_min = Y = 359
y_max = y_min + H = 359 + 19 = 378

Convert these dimensions to fractional by dividing by the overall image dimensions

xFrac_min = x_min / W = 493 / 932 = 0.529
xFrac_max = x_max / W = 579 / 932 = 0.621
yFrac_min = y_min / H = 359 / 557 = 0.645
yFrac_max = y_max / H = 378 / 557 = 0.679

Using Google Sheets or Excel, add a column to your CSV file with the following formula

=IF(AND(A1>=xFracMin, A1<=xFracMax, B1>=yFracMin, B1<=yFracMax),1,0)

where A is the column containing the X coordinate of the click and B is the column with the Y coordinate.

And there you have it! Now you can calculate whether or not a First Click Test response is correct or not!

How was that?

On second thought, that was quite a bit of work. Maybe I should have used the workaround that the experts suggested! Oh well. I found it fun. And if you got this far, I hope you did too. If you didn’t find it fun, let me know what questions you have or how I can help! If there’s enough interest, I’ll add a Google Sheets template to this post that you can just fill in the blanks on.