What is the drawback of collecting less than the recommended sample size?

The implication of having lower sample size would be the robustness of the data (i.e. wider confidence intervals).

The extent to which survey answers/recommendations would potentially be affected is dependent on the survey itself (whether respondents have strong and clear preferences or not).

Our platform displays the recommended sample size for your experiment on the left-hand panel (please see below).

Viewing the recomemnded sample size

If you foresee that the number of responses you will be collecting is lower than the current recommended sample size, you can make the following changes, which will lower the recommended sample size:

  • Selecting the “Identify and zoom in on top claims/variants” setting (for Claims Test / Product Variant Selector only):

Identifying and zooming in on top claims/variants

  • Increasing the number of choice sets per respondent (which will make the survey longer). With this setting, we would also recommend forcing respondents to stay for a few seconds in the choice-based exercise (you can do so from the same tab: “Advanced settings” –> “Response quality warnings”)

Increasing the number of choice sets per respondent

This question from our users was answered on 26 September 2019. If there is anything else you'd like to know, please do not hesitate to contact us.