Title Targeting the bias – the impact of mass media attention on sample composition and representativeness of volunteer web-surveys in Germany
Workshop Workshop 2014
Year 2014

Purpose of the study: The increasing use of volunteer web-surveys has often been criticized because the results are not representative for the population of interest. While the discussion has mainly focused on different weighting techniques to correct for sample biases, the question remains whether ‘pre-survey adjustments’ (e.g. different recruitment strategies of volunteer websurveys) might provide an alternative tool for reducing sample biases. As the knowledge of how to reach a large and heterogeneous group via web-marketing has increased tremendously, our study aims to explore whether selection bias in a volunteer web-survey might be tackled through media attention. Four sub-questions are addressed: How long does the effect of media attention last on average? Does the sample composition change in the periods of massive media attention? Which sample characteristics are subject to changes? What are the differences in the observed effects comparing two different media measures? Design/methodology/approach: Data from the continuous, volunteer German WageIndicator web-survey (Lohnspiegel) are used for 2013. Two media events will be analysed: first a ‘mass media event’ based on a first page publication of the top-hundred highest earning occupations in the over-regional newspaper ‘Bild Zeitung’. Second, a ‘tailored media event’ based on a press release of an occupational-specific income analysis which has been advertised through different popular off- and online channels. For both events, we first analyse the change in the number of daily web visitors (prior and posterior to the event). Second, we divide the sample between the period of increased and regular web visits and compare the two samples for socio-demographic and work-related characteristics. Findings: The preliminary findings of the mass media attention analysis show that such an event has a strong positive effect on number of visits (doubled) and questionnaire submission (fourfold). However, for the composition of the sample we find support that the sample is biased towards highly educated, prime aged, male respondents as well as high earning workers in professional occupations. This increases the already existing bias of the volunteer sample. Moreover, we also observe a long-lasting effect of such an event. Even after 4 weeks of the event, women and older people are less represented in our sample in comparison to the sample composition before the event. This implies that groups so far underrepresented in the web-sample are hardly motivated to take part in a survey through mass media events. Originality/value: This topic has not been addressed in the scientific discussion about volunteer web-surveys. Therefore, it will offer new insights and an alternative approach to deal with selection biases in volunteer web-surveys. In particular, it will clarify whether arising problems can be tackled by specific web marketing actions. Research limitations/implications: This is an attempt to approach the sample biases caused by the non-probability nature of the sampling procedure in an innovative and rather ‘unconventional’ way. However, it is an exploratory study which needs further testing with different data sets and types of media attention. Practical implications: An increased understanding of the effects of media events is particularly useful for organisations expressing a need to attract media attention because they want their mission to be heard by a wider audience. In marketing studies the effects of offline media events have been studied largely. However, the effects of online activities are monitored predominantly by looking at volumes (page views, clicks, followers, etceteras), but hardly by analyzing the socio-demographic characteristics of the sample. They may benefit from further insights into these effects.