Maximizing survey sample quality in 2020

Online surveys have been a go-to source of information for marketers for the last 20 years. Although survey collection approaches continue to evolve and access to online surveys continues to get easier, the importance of ensuring the quality of our samples has never been more important.

As insights professionals, we have an obligation to ensure accurate and valid sample data. Given the emphasis on using data in business decision-making today, we need to continually revisit and update our best practices for making sure that our respondents are of good quality.

One of the biggest challenges with online panels is low quality respondents. We consistently see 10-15% of respondents are low quality on short or easy surveys. That number jumps to 20 or even above 30% on longer or more complex surveys.

These dishonest or unengaged respondents are nothing more than noise that can lead to less definitive or even misleading findings. For example, we’ve found it’s almost impossible to get a good consumer segmentation if you don’t clean out these low-quality respondents.

At Magid, we’ve identified ways to eliminate these respondents from our surveys. In every survey we do, we eliminate respondents based on issues we see such as the following:

• Speeders. These respondents complete the survey in a time that would not have allowed for serious consideration to give real responses. We eliminate these people based on how they compare to people who take the survey seriously based on the length of time they take and the type of responses they give.
• Nonsense Spewers. We review all open-end responses and remove respondents who answer with gibberish or responses that don’t address the question asked.
• Contradictors. We examine the pattern of responses at the individual level. Unengaged respondents often don’t vary their responses or will contradict their own responses so we can see they aren’t paying attention while they answer.
• Deceivers. Deceivers give us patterns of responses that simply don’t make sense (based on awareness, where they shop, product usage, etc.).

Today’s businesses need insights they can trust. Eliminating low quality respondents is essential to making that happen. Knowing which respondents to remove requires a deep understanding of nuanced survey behavior and an unwavering approach to quality.

Magid’s proprietary approach for removing these low-quality respondents means you can trust the learning you get.

Want results you can count on? Let’s talk.