Different Recruitment Processes for an Online-Survey Among German Livestock Farmers - Impacts on Sampling Bi-ases and Data Quality

Abstract

This paper discusses the methodological challenges of conducting online surveys for agricultural social science research, with a focus on the potential biases introduced by sampling processes. Online surveys often use convenience or quota sampling, which can lead to self-selection biases that compromise the validity, reliability, and generalizability of research findings. Response quality issues, such as straight-lining and response fatigue, further exacerbate these challenges. The study examines data from a survey of more than 600 German livestock farmers who were recruited through three different methods: informally through social media channels, formally through professional networks, and commercially through a market research service provider. Results show that the commercial sub-sample had shorter response times and higher rates of incorrect responses for a control question. The sub-samples also showed demographic variations and differences in personality traits, agricultural land areas, and farm characteristics. The role of quota requirements in shaping differences within the commercial sub-sample is emphasized in the discussion. This reveals limitations in online surveys' ability to represent objectively defined target populations. The interpretation of results is complicated by the absence of consistent theoretical concepts and clear ex-ante hypotheses. The paper concludes by recommending that agricultural social science researchers transparently disclose their sampling procedures. This will enable readers to make informed judgments on the reliability and validity of their findings.