Blog

Pros and cons of remote moderated testing: Considerations for ongoing research during COVID-19

|
March 10, 2020

It does not matter what you do for a living, where you do it, or who you do it for – the global spread of COVID-19 has impacted how we work in one way or another. UX research is no different. At a time when travel and face-to-face communication are viewed more as risks than benefits, Bold Insight is taking active measures to ensure that we can continue to conduct the research necessary to inform our clients’ development of products that provide their customers with exceptional experiences. One way we are doing that is to encourage remote moderated user research whenever appropriate. Of course, there are advantages and disadvantages to remote testing independent of the current travel environment, but below are some of the key points to consider when deciding whether to shift to a remote-first mindset for your research needs during the next couple months.

Remote testing pros:

  • Avoid cost and risk associated with travel
  • Typically more cost effective than in-person research
  • Can still observe research live via any number of remote-viewing platforms
  • Works well for usability testing of websites and mobile applications
  • Works well for exploratory interviews
  • Can even work well to investigate current practices when paired with digital ethnography
  • Higher sample sizes easier to achieve due to cost-effectiveness

Remote testing cons:

  • Qualitative data often not as rich
  • Participant engagement and adherence can be reduced
  • Does not work well for simulated-use testing of physical products
  • Professional research participants harder to control for
  • Less efficient in terms of higher no-show rates and data loss due to technical issues

Considerations for conducting remote sessions:

  • Plan for more sessions than you need
  • Plan for more time between sessions and fewer sessions per day to allow for technical issues (on your side or the participant’s)
  • Be as diligent about the preparation for your remote research as you are for any research
  • Plan for obtaining and documenting informed consent remotely
  • Take advantage of time zones to capture outside working hour participation
  • Unless your remote session is unmoderated, use a note taker just as you would for an in-person session

COVID-19 general considerations for ongoing in-person research:

While we are advocating for a remote-first mindset in general given COVID-19, we also work with many physical products and research objectives that don’t lend themselves well to such an approach. Where possible, we are counseling our clients to prioritize research that can be conducted remotely, but in many cases that flexibility just does not exist.

We are committed to supporting our clients even in difficult times. Bold Insight will continue to conduct in-person user research with a number of common-sense adjustments:

  • Avoid highly affected metro locations
  • Plan for more sessions that you need (higher no-show rates)
  • Plan for even easy recruits to be more difficult (increase compensation and lead time)
  • Build in COVID-19 screening questions into pre-screeners as well as on-site protocols
  • Consider remote viewing for your stakeholders to minimize the need for travel
  • Take appropriate measures to prevent the spread of the virus in the event screening measures are not 100% effective (e.g., sanitize the test environment before and after every session, consider providing gloves and/or masks for researchers and participants)
  • If considering research with HCPs, further prioritize remote methods in recognition that our healthcare providers are very busy combating the spread of the virus and remote sessions will be easier to attend

 

By making these adjustments, we ensure our research can continue while keeping our team, our clients, and our vendors safe.