14 Research panels

Image by kvector/shutterstock.com ^[1]

What is a research panel?

A research panel is a group of people who have agreed to participate in future research studies after having been pre-screened and pre-profiled, then given clear expectations about the types of research to be conducted, the frequency, and the types and amounts of compensation or incentives to be offered. These days most of the research panels are based online. “Proprietary panels” refers to those research panels which are owned by a panel provider. This organisation is responsible for building and maintaining the panel.  The panel owner is responsible for the ‘quality’ of the panel, ensuring that the panel is regularly validated and ‘refreshed’.

The main benefit of using a research panel is the ease and speed of accessing potential survey respondents. The overall per head cost of surveying people can be low in comparison to other methods. Moreover, researchers have the ability to recruit potential respondents from all across the world. This can be especially useful when marketers are planning to operate in new regions. Finally, online panels offer greater privacy and confidentiality to respondents. Therefore, it is easier to discuss sensitive topics (e.g., a medical condition) or recruit hidden populations (such as members of the LGBTQ+ community). ^[2]

Challenges in using research panels

^[3]

There are four main issues that have been identified by researchers and practitioners in using research panels.

The first one refers to ‘professional survey-takers’ who intentionally participate in surveys frequently. Such survey-takers are bound to respond differently to survey questions than other respondent groups. Panel providers are encouraged to participation-rates in their panel. Similarly, the reimbursement rate (such as a movie ticket or $10 or donation to favourite charity) to panel members needs to be made available so clients can evaluate the incentive behind respondent participation.

The second issue relates to the ‘representativeness’ of an online panel.  While this used to be an issue during the early days of the internet, it is useful to still give it some consideration. With widespread access to online data, the digital divide is becoming smaller. In Australia, 99% of adults have access to the internet ^[4]. Researchers are now more confident of finding greater representation of the overall population in online panels since internet access is no longer bound by age, education, income or cultural/linguistic background. However, researchers may still want to examine whether an online panel has sufficient representation of a specific population. For example, if a study requires the recruitment of “housewives living in Sydney AND who were born in Sudan” may need to be discussed with the panel provider.

The next point relates to the ‘health’ of the online panel As with all databases, it is not enough to simply have a list of people willing to respond to surveys and interview questions. A constant review and monitoring of the panel is required. If this is not done, then there is a risk of possessing a panel with outdated member information. Panel members’ profiles change all the time. People move away, switch jobs, get married, have children, develop new opinions, and alter previously held beliefs. Unless there is a mechanism to refresh and update information about panel members, there is a greater chance of ‘non-response’, as the panel will not fit with the required respondent profile. There are some practitioners who feel regular recruiting of new panel members may solve the problem. While increasing the panel size is a good selling point, it is more cost-effective to update the information of existing members than recruit new ones^[5].

Finally, there is a concern about the ‘quality of data’ generated from online panels as recruited panel members may not fully engage with the survey content, not make an effort in thinking about survey questions or have a general lack of inattentiveness. While this is an important issue, it is also easy to identify such errant survey takers. Panel providers can discuss criteria with clients and delete responses from ‘speeders’ (i.e., those respondents who complete a survey in an impossibly quick time) and ‘straight-liners’ (i.e., those respondents who repetitively select the same option for all survey questions. This is especially common for a survey with a Likert-scale type of questions. Any buyers of online panels want to be assured that the integrity of the panel is maintained by removing any members who are found to display such behaviour repeatedly.

Management of Online Research Panels

The four stages of the panel-management process are^[6]:

• Recruitment of panellists
• Selection of a sample
• Panel monitoring and maintenance
• Panel engagement

Recruitment of panellists:

The main recruitment method used to build a panel is referred to as the ‘opt-in method’ (Goritz 2007, as cited in Kahn 2012).  With ‘opt-in’ or ‘volunteer’ panels there is usually no restriction on who can participate.  Ideally, a panel should be constructed by using a range of methods (both online and offline) used to recruit participants.  Usually, interested individuals are directed toward the panel organisation’s website where panel-related terms and conditions can be found.  Potential respondents are asked to fill in a registration form that automatically generates a socio-demographic database.  By applying different recruitment methods, perhaps with collaboration with diverse affiliate partners, the resulting panel is bound to be more diverse and representative than the one based on a limited source of member recruitment.

Selection of a sample:

A sample is selected as per the requirements of the research project. It is always useful to discuss the respondent profile with the panel provider beforehand. For instance, if the researcher specifically wants to survey “Sydney-based Indonesian-born mothers with school-going children”, not all online panels might comprise this population group.

Quota sampling is often undertaken while selecting an online panel sample. The researcher may decide that he/she wants an equal split between males and females. Even if more female panel members are available to respond to a survey, they will not be included in case the quota limit has been reached.

Panel monitoring and maintenance:

One of the signs of a good panel provider is its strategies regarding the monitoring and maintenance of its online panel. To ensure the quality of the panel, it needs to be ensured that members are ‘active’. Moreover, any members found to be ‘speeders’ or ‘straight liners’ are given warnings and removed from the panel, if such behavior continues. The panel provider must also have policies to prevent ‘over-use’ of panel members. For instance, no respondent should be attempting a survey more than once a month. Similarly, if a panel is already responding to a survey they should not be asked to do another survey simultaneously.

Over a period of time, panel members may become inactive because of a loss of interest, a lack of appreciation for the amount of work involved, or a change in household circumstances.  Other reasons like natural mortality, invalid email addresses, and member concerns with data security may also contribute to members’ inactivity. It is useful for the service provider to monitor the panel for any such signs and to explore options for keeping the panel alive and healthy.

Panel engagement:

Similar to the way companies engage with their employees, engaging with panel members is also important. While the respondents are often provided with a (financial or other) incentive, it is recognised that strong respondent cooperation needed for data quality is something that cannot be paid for.  Online panel providers need to manage their panels by building trust and commitment through multi-channel levels of communication such as via personal contact, newsletters and telephone calls. This is also an opportunity to ‘train’ the respondents in good practice by clearly communicating the researcher’s expectations and the importance of generating good quality data.

ESOMAR Guidelines for Research Panels

ESOMAR (European Society for Opinion and Marketing Research) is a not-for-profit membership organisation, established in 1948. ESOMAR is headquartered in Amsterdam, the Netherlands, and is present in over 130 countries. ESOMAR members form a community due to their interest in data analytics, research, and insights to help improve the lives of individuals in societies and organisations. Being a lead organisation representing researchers worldwide, ESOMAR has published a guide on how to recruit online samples by using panels. The guide is available at Esomar.org. 

Video: Example of a Consumer Panel

Source: Camden BRI ^[7]