MY RESEARCH PROCESS

I was tasked with conducting user research on Project Shield’s website as a demonstration of my UX Research skills. Follow along and learn how I approach user research initiatives.

OVERVIEW

Image of Project Shield's landing page. This image links to the Project Shield webpage.

As part of a demonstration of my skills, I was tasked with understanding why users were not signing up for Project Shield and recommending strategies for boosting conversion rates. Project Shield offers DDoS protection services underpinned by affiliations with industry leaders like Google.

My research uncovered several user experience challenges, including trust issues, a perceived lack of clarity in the value proposition, and unengaging design. Through this study, I aimed to inspire a transformation of the user experience and reinforce Project Shield’s position as a premier web security platform.

As I outline the steps I took during this initiative, I explain what was involved at each phase of research. I also share how each phase unfolds beyond the written strategy. So follow along to learn more about my process of conducting research for people, providing you with a blueprint of my approach to transforming user insights into actionable improvements.

TEAM MEMBERS

1 researcher (me)

TIMELINE

1 WEEK

TOOLS

Coda, Dovetail, Figma, Zoom

THE PROCESS

Figure 1: My research process is not always linear and phases overlap. I believe that research should not simply end; there’s always room for iteration.

KICKOFF

At the start of any initiative, I collaborate with both direct and indirect stakeholders to grasp the core 'why' behind the initiative. This ensures that I align the research outcomes with stakeholder expectations. In turn, the findings will not only be insightful but also actionable. This collaborative approach at the kickoff phase ensures that both research and business objectives are seamlessly aligned.

For this specific project, the goal was to evaluate the website's effectiveness for non-technical users—specifically, how well the site conveyed essential information and facilitated informed decision-making. Prior research had made it clear that non-technical users dropped from the site rather than converting by signing up for Project Shield’s protection. The question, though, was why?

RESEARCH PLAN

At this point in the process, I draft a research plan based on the conversation I had with stakeholders during kickoff. In my research plans, I outline the scope, methodology, assumptions and hypotheses, research objectives, purpose, roles, anticipated timeline, related research, and next steps. This plan is iteratively refined in collaboration with stakeholders, ensuring that their insights and concerns are thoroughly integrated into the research approach. Therefore, a research plan can evolve over multiple iterations given stakeholder feedback and business needs, so it is important to use a tool that allows for dynamic growth. That’s why I used Coda.io to create the research plan for this project.

Once the research plan was finalized, it acted as the roadmap guiding me and the stakeholders along our journey. I crafted 3 main objectives to keep myself on course throughout the study:

Determine the ease with which non-technical users could comprehend the site's content.
Uncover any user journey obstacles for non-technical visitors.
Unveil the clarity of the site's communicated next steps once users decided to engage with Project Shield.

The overarching goal was to ensure that even non-technical users could understand what Project Shield was, what it offered them, and how it could increase their website’s security.

METHODOLOGY

Crafting a research methodology isn't simply about selecting tools and techniques; it's about architecting a robust strategy that brings clarity and direction to the investigative process. For Project Shield, the methodology was meticulously chosen to meet the unique needs of the study, aiming to illuminate the user experience from various angles:

Usability Testing: Assessing the site's usability with a representative user following the framework of ISO 9241.
Pre- and Post-test Questionnaires: Gathering initial and concluding user data, including a System Usability Scale (SUS) assessment.
User Interviews: Conducting structured conversations to gain qualitative insights.

Each method was chosen not only for its individual strengths but also for how they complement each other, ensuring a comprehensive understanding of the user experience.

A screenshot of the usability data capture doc which is housed in the research plan in Coda.

Figure 2: Using Coda.io, I’m able to create and organize materials needed for testing given the chosen research methods.

PREPARATION

With the methodology set, the focus shifted to meticulous preparation. For remote sessions like those in this study, I leveraged digital collaboration tools—Google Meet for interaction, Google Forms for structured data collection, and Coda.io for dynamic documentation and real-time data analysis. This preparation ensured that the study would run smoothly and that the data collected would be both rich and reliable.

In the spirit of preparation, it's worth noting that stakeholders played an active role throughout this phase. Regular check-ins and reviews of the research instruments not only refined the approach but also maintained alignment with the project goals. Stakeholders were able to contribute their insights and validate the direction of the research, ensuring that the execution would yield results resonant with both the user needs and business objectives.

The result was a well-oiled machine, ready to launch into the execution phase with precision and purpose.

EXECUTION

Execution. The heart of user research. This is the phase of research that I get most excited about. Here, I launch the study, capturing data such as recordings, transcripts, usability metrics, survey responses, or other data, depending on the research methods. Stakeholder engagement continues during the execution phase, where their presence as observers can offer immediate insights and foster a deeper understanding of the findings as they emerge.

When running unmoderated studies, I may capture data asynchronously and automatically. When running moderated studies, such as this project, I run live sessions with the participant(s) and capture data manually. Executing research with manually recruited participants requires that I manage the logistics of:

Finding/selecting participants
Scheduling the sessions
Ad-communication with participants
Procuring and distributing incentives, as necessary.

Image of user persona which represents target user of Project Shield.

Figure 3: Recruiting participants who meet the target user base is essential in ensuring that my recommendations satisfy their Jobs-to-be-Done.

For this initiative, I met with a single participant who stepped into the shoes of a local newspaper reporter on the hunt for web security after a recent cyber scare. This individual was a perfect fit for the kind of user Project Shield aimed to cater to.

While feedback from this single participant provided valuable user experience insights, a broader sample of 5-10 users, as recommended by Nielsen Norman Group, would yield more comprehensive findings and robust recommendations.

ANALYSIS & IMPACT

Analysis is when I delve into the data collected from participants. For this study, I used Dovetail to code qualitative data and extract nuanced user sentiments and patterns. Meanwhile, quantitative data provided objective measures of effectiveness, efficiency, and usability through metrics such as:

Task completion rate
Time on task
Task error rate
User confidence score

These metrics painted a backdrop for the more subjective qualitative insights, grounding our findings in a comprehensive evidence base.

Figure 4: Dovetail makes unearthing insights from qualitative data easy. My approach is rooted in Grounded Theory, limiting the amount of research bias I introduce into the data.

Based on participant feedback and the results from the usability test, I noted significant areas for improvement, including:

Clarity of Value Proposition: Both qualitative anecdotes and quantitative SUS scores highlighted confusion among users regarding the service and its advantages.
Usability Barriers: Navigation challenges and visual design issues, evidenced by poor completion rates, prolonged task times, and accessible design violations, signaled critical areas for design intervention.
Trustworthiness: Hesitations captured in interview transcripts, along with lower confidence scores, pointed towards a need for stronger credibility signals.

All milestones were met, with all qualitative data meticulously coded and quantitative data analyzed. These findings were ready to be leveraged for enhancing Project Shield's user experience and strengthening its market position. Stakeholders were encouraged to engage in a dialogue over these insights to refine our approach and recommendations.

SYNTHESIZE & SHARE-OUT

This project highlighted the importance of aligning the user experience with expectations, illustrating the growth potential when feedback is properly leveraged. It also showed that ongoing user research is crucial, even after designs have been published. By addressing these insights, Project Shield can become a trusted DDoS solution for non-technical users, leading them to confidently choose the service.

With any research initiative, sharing the insights is paramount. I created clear visualizations, user personas, and video highlights to help stakeholders understand the user experience. For detailed insights, I developed a comprehensive deliverable using Figma. In addition to the embedded view of the findings report below, you can also find the full presentation in Figma.

REFLECTION

A diverse group of professionals sitting around a table carrying on a discussion.

Stakeholder collaboration is crucial to ensuring an impactful deliverable. Throughout the research process, I find value in continually checking in with other teams or stakeholders to ensure that the project remains on-track while also maximizing the studies exposure and creating a sense of excitement around the insights.

A pen sitting on top of an open notebook that has been sketched in.

Future research would benefit from incorporating a broader range of stakeholder inputs, tailoring presentations to specific decision-maker needs, and by facilitating workshops post-research to turn insights into action. Moreover, building in continuous loops of feedback even post-delivery ensures that research findings evolve with the project and remain relevant.