Why I wait for research to prove something is needed before it’s added to the UI

Last week, a colleague gave feedback on a prototype I’m working on - a flow that helps users review a case.

The UI has an accordion of documents. You can select text within each document to annotate it:

Here’s what the page looks like when the accordions are collapsed:

The suggestion was to indicate which documents have annotations in the closed state.

It’s a solid idea - one my content designer and I had previously discussed - because users would otherwise have to open each document and scroll down to see if it has annotations.

But we didn’t want to do that without seeing users struggle without it. Because there’s limited space and adding more information on screen increases cognitive load.

So we put it aside as something to consider later if research shows it’s needed.

Because in testing it might be that users add annotations and rarely need to return to them.

Or users might check their annotations at the end of the review flow using the ‘check answers’ page:

The best way to find out is to leave the indicators out and see if that causes a problem because:

It’s harder to include something and prove it’s unnecessary, than it is to leave something out and prove that it is.

It boils down to what a good usability test looks like.

You shouldn’t start by asking the user what they think of the interface.

You should start by giving them a task to carry out and watching quietly to see how they get on.

If they struggle, you can probe, and later consider what to add or change.

If they don’t struggle, you don’t need to add or change anything.

We haven’t observed users opening each document and scrolling down to see if it has annotations. But we’ll keep an eye on this as we’re in the middle of our second round of research.

Now imagine what would’ve happened if we had added the indicators and run the same tests.

We’d probably have seen the exact same thing. No struggle.

But that wouldn’t tell us it was needed.

When a user doesn’t interact with something, there are a few possible reasons:

1. They saw it and didn’t need it
2. They saw it, it helped a tiny bit, but not necessarily enough to justify it given the trade offs
3. They never even saw it

The problem is you don’t know which it is.

The second option is especially tricky. A user can be mildly distracted by something without it stopping them completing the task - so it still looks like it worked well.

Whereas if you see users expanding documents and scrolling down to find all the annotations on a repeated basis, that’s a clear signal.

This is why it’s so important to be protective about what gets added to your interface.

Because if you only add things based on hunches, not only do you miss out on learning about how users actually behave, you end up with an interface that increases cognitive load and degrades usability.

I run a course called Form Design Mastery. It contains patterns built in the same protective way: every pattern is there because research proved it’s needed, not just because of a hunch:

https://formdesignmastery.com

Cheers,
Adam