A structured framework and empathetic background knowledge provide fruitful prototype testing

CLIENT Technology firm

SERVICES Problem Discovery, Concept Creation, Prototyping, Data Analysis

METHODS In-depth interviews, user personas, data synthesis, desk research, surveys

BACKGROUND AND BUSINESS NEEDS The client wanted a way to improve the search results on its company intranet, which rarely returns the information employees are looking for. The company intranet contains information such as policies, historical data, and technical instructions as well as other project and team-related content. The client also requested a quick and light-weight solution since they had limited resources and time to devote to a long-term effort.

PROBLEM DISCOVERY In order to know what and how to improve with the client’s intranet, I needed to understand how its users behave on the platform, but also what were their expectations and motivations when interacting with the service. I conducted user interviews with employees from different parts of the organization. Common user complaints included irrelevant search results, misleading page titles, key information buried in dense-pages, outdated information, and incorrect content. Based on my findings, I created user personas as a reference guide that illuminated the motivations, behaviors, and expectations of users when they use the intranet as well as how their current experience on the intranet impacts their daily work experience.

PROTOTYPE TESTING After confirming with the intranet admin that the search functionality worked as expected – strictly “out of the box” as my research underscored – I researched search add-ons and enhancements that would work with the client’s intranet and bridge the gap between the intranet’s current search behavior and users’ expectations.

I chose two add-ons and created a third solution – an area on the intranet homepage that would present links to specific pages and allow users to bypass the search function altogether. I then created my hypothesis, measurements, and success metrics.

Hypothesis: If the top 3 search results contain the answer to users’ most searched for queries, user satisfaction will increase. Measurement: User conversion Success metric: 20% increase

For testing, I reached out to staff within four segments – content experts, content managers, new hires, and long-time employees – to participate in the experiment. Each tester would perform a search twice – once on our production instance (the “control”) and once on the experimental instance in stage. Each user would use a query that appeared in the top 20 sets of queries based on Google Analytics. Each user would look at the first 3 results and assign a score from 1 - 5 based on the link title and summary if the page would contain the answer to their question.

IMPACT Two out of three experiments received an improvement rating of 24% (20% was the threshold), so I implemented those two in our production instance. After four months, I conducted a survey that showed a 42% increase in user satisfaction after the search enhancements were implemented and a 25% increase in active user sessions.