Little Language Models

User research–not the discipline, but the term that refers to the activity–has a terrible reputation.

Bringing up user research at work events often results in hand-wavy indifference from the other person. We’ve already got it covered, they’ll state. When I inquire further, I hear about marketing segmentation data. Or that different teams share responsibility for the UX quality of the end product.

Discussing user research during initial meetings with potential clients often results in reassurances about its importance, without clarifying available research findings (if any) or committing to objectives and methodologies. Or concerns about timelines and budgets. Or excitement.

From 30+ projects, I share what makes research exciting or dreaded, and how to decide if your team should invest.

I’ll share the patterns I’ve noticed to help you determine if, when, and how much research to conduct for meaningful results.

Don’t say user research

The term is associated with hundreds of interviews of the same questions that little meaning can be derived to.

Focus on areas that tend to provoke disagreement or subjective opinions

Focus on experiences where human judgment is paramount, where having clear understanding of what users need can minimize taste-based opinionated meetings with little takeaways.

Introduce specific methods

Most people think user research consists of the same surface-level interview questions. How likely are you to recommend this to a friend? Would you use this? Do you like this? They are mostly ineffective for product improvement purposes. Enter the rich world of UXR and the different types of research: card sorting! tree testing! unmoderated user tests!

Establish a fixed, mutually agreed-upon goal

Express a clear goal and give folks opportunities to define the goal and share their concerns and desires.

Complete only the minimum necessary to meet the goal

I’m a fan of doing less. This is a mix of all the other points, where ruthless prioritization gets us the results we want by having a clear:

• definition of goals
• structure of the research
• understanding of how the result will affect business goals

Research Return on Investment (ROI) example

Just 8 user tests transformed the funding process for thousands of PhD students.

About 4 hours and $250 in recruitment and tools costs gave us invaluable, actionable feedback on page structure and language.

• reduced service support requests (clearer web experience = less user confusion = fewer emails and customer support requests)
• increased the number of eligible applications received (clearer web experience = less user confusion = fewer applications by people who don’t meet requirements)

The new structure successfully guided users to answers, reducing friction and enabling the team to focus on their core mission of supporting qualified fellows.

Another example of Research Return on Investment (ROI)

Another client received thousands of email inquiries every week. After a technology replatform, questions were being sent to the wrong departments, causing delays, inefficiencies, and customer frustration.

This user research didn’t involve talking to a single user, but it was still able to produce results like:

• Time on task reduced by 20% (contact page → question submitted) 
• Improved response times
• Decreased email volume
• Increased customer satisfaction as measured by increased exit rates in the Contact page by reorganizing content hierarchy to address key user needs preemptively, reducing the need for email inquiries

I hypothesized about potential causes of confusion, then used Hotjar ($99/mo) and Google Analytics (free), looking at screen recordings and path exploration data to understand what real users were experiencing on the site.

• Tab navigation confusion

Users defaulted to the first tab and submitted forms there, unaware that 7 department-specific tabs existed for faster routing. The expanded default state didn’t signal alternatives. The controls were supposed to guide users, but had the opposite effect.

Further reading: How much control should the user have over a workflow?

• Inconsistent labeling

Tab names mixed two categorization systems, organizational roles and topics. This created ambiguity about which tab matched the user inquiry.

• Unclear page identity

The page title didn’t clearly communicate that users were already on the Contact page. This caused some to click the contact link in the main navigation, suggesting they thought they were elsewhere.

After this MVP research, I designed low-fidelity content wireframes, solving for the pain points identified:

• content hierarchy
• clarified tab labels with consistent naming conventions
• improved page identity to signal to users that they were in the right place
• incorporated frequent questions into the page, reducing the need for email inquiries

Good research hits the right balance of -whelm

When people hear user research, they imagine someone asking users what they like about the product, how often they use it, and what they would change about it. This doesn’t reflect the method, meticulousness, or the quality of findings of many skilled user researchers I’ve met. It’s not what we, research-driven consultants committed to building exceptional products, are talking about when we suggest user research.

I suggest doing the exact amount of research that will get your team:

• engaged without getting overwhelmed by adding more work on top of their day job (in case it wasn’t clear, this article is referring to teams without dedicated user researchers)
• excited without getting underwhelmed (fixed scope for research prevents overpromising and reduces the risk of the team perceiving the findings as insignificant/underwhelming)

The research goal should be simple enough to explain to colleagues in the office kitchen.

Suggesting user research as a generic activity without tying it to a specific business context or user need is unlikely to be effective.

The next time someone suggests user research, explore with curiosity if there’s a specific, valuable question that can be answered, and if so, consider using targeted, goal-driven methods to find the answer quickly and affordably.

AI Policy: I personally write each draft and final copy on this website. All content reflects my own thinking, ideas, style, and craft.  I do not use AI such as ChatGPT or other LLMs to generate articles. Occasionally, I ask AI (such as Formalizer or Equativ) to summarize or re-state my own ideas and may restructure sections based on the response.