7 most important considerations for quantitative researchers

Over the decades I’ve spent in quantitative research, I’ve learned it is much more about the story consumers are telling us and less about the numbers – although it’s something I still need to remind myself of, from time to time. 

A quantitative researcher is the impartial voice and advocate for the consumer, and some skills and considerations are fundamental to fulfill that role and find success in this field. 

1. Finding the right sample

It’s of critical importance that the survey sample is reflective of the desired consumer base, as the goal of quantitative research is to be representative of the overall population of the desired consumers. For example, if the targeted consumers are more likely to be young men, then the sample needs to reflect that reality. The findings will be unreliable if the sample is not on target. 

Sometimes, we may need to widen the aperture to ensure that we’re not limiting our view of the target population. It’s about challenging assumptions of who should be included without opening the aperture so wide that the study’s costs exceed its benefits – which brings us to the sample size.

2. Choosing the sample size

I have been asked hundreds of times, “How many people do you want to talk to?” And my answer is always the same, “Everyone.” 

However, that’s just not possible. There is always a trade-off between sample size and costs. The quantitative researcher needs to look for a sample size that minimizes the margin of error, while staying within budget. Although having an error rate is inevitable, a small sample size leads to a higher margin of error, hence questioning the results.

3. Designing the survey in consumer speak and lifestyle

In addition to avoiding double-barreled questions and ambiguity, a survey should always consider the voice of the consumer, both in tone and language. Doing so will increase comprehension and receptivity.

4. Respect consumers’ time

Consumers’ time and effort should be respected, meaning that questions added to the survey should always have a planned analysis associated with it. Asking questions and not reporting the result is a waste of time for both the researchers and the consumers. If there isn’t a vision for what we’re seeking to answer in our findings, the question should be removed from the survey.

Another aspect to keep in mind is creating mobile-friendly surveys. Most recently many participants, often 50%+, started a survey on a mobile device. An unfriendly mobile survey would compromise the sample, skewing the results, as the researcher would be designing a survey that is “self-selecting” respondents.

Overall, an engaging survey ensures consumers have an equal chance to respond. If the survey is unfocused, long, and has poor design, only certain participants will finish it, skewing the results. Respect the consumers’ time by building a reliable, clear, succinct and interesting survey.

5. Honesty above all

Clients seek honesty. Companies hire quantitative researchers to examine the data, determine the findings, and reflect the consumers’ thoughts and opinions. A quantitative researcher should never deter from the truth to agree with the initial client hypothesis. Always focus on consumers’ attitudes without letting outside voices influence the findings.

6. Control fraud

While never a pleasant subject, fraudulent consumers are a growing problem. Having fraudulent participants included in the final sample will skew the results and lead to inaccurate findings.

However, there are some steps that can be taken to control fraud:

• Bot question – Bots are problematic, but they can be avoided by adding a type of verification, e.g., captcha verification, at the beginning of the survey.
• Straight lining – Be on the lookout for consumers who “straight line” through the survey, meaning they answer with the same foil/answer on grid questions. These are easy to identify and remove.
• Length – Remove the participants who complete the survey too fast.
• Keyboard smash – You can identify fraud on open questions when consumers type nonsense. They smash the home button or have an irrelevant or nonsensical answer.
• Zip code – Always include a zip code question when asking geography and remove consumers who answer inaccurately.

Our preferred panel partners are always receptive to removing these consumers from their database.

7. Keeping the analysis at the forefront

Although the story comes together during the analysis portion of the research, it really starts being built as the instrument is designed. Therefore, data analysis is always at the forefront of our researchers’ decisions. For example, while writing the survey, we’re considering the questions that may “hang together” through Factor Analysis and the specific scales we’ll use in building our narrative. 

Crosstabs, multivariate, max-diff, conjoint, and line optimization are some of the methods our quantitative researchers employ at Magid. However, no matter the method, the analysis is always planned beforehand. This ensures we create the most accurate and reliable sample data that will yield the right insights to allow you to make better decisions.

The future of quantitative research

Human behavior remains challenging to predict using past data alone – particularly the big data driving so much of today’s decision-making – however, by carefully addressing the above considerations to integrate insights on consumer attitudes and perceptions alongside that behavioral data, we can stay ahead of the curve.

As we continue to evolve our best practices in quantitative research, adapting to new technologies in the field and addressing the increasingly complex world of business and consumers, what will remain is our role as storytellers. We’ll be here for you offering an impartial voice for the consumer – grounded in data.