Assessing the Representativeness of Nonprobability Online Panels. The Italian Case.

Online panels are increasingly used in social research. The advantages of online panels are undisputed (i.e., fast data collection, lower costs). However, online panels – and in particular the nonprobability ones – have a number of limitations, due to coverage error and self-selection into the panel. The key issues is that online panels may not be representative of the population that they intend to represent and this may have important repercussions on the quality of the estimates produced (AAPOR, 2010; Callegaro et al., 2014). Despite the relevance of these issues, there are few studies in this field (see below); these studies have found strong evidence for bias and have shown that adjustments strategies may not be effective. The overall aim of the work is to assess the representativeness of an Italian nonprobability online panel and investigate the effectiveness of different post-survey adjustment strategies (i.e., post-stratification, propensity score adjustments) in reducing the bias. We compare the demographic and socio-economic composition of the panel and estimates from a set of selected survey items (e. g., consumption) to those of the general population computing percentage differences and appropriate statistical tests. When appropriate, we use regression analysis. We then compare estimates from the online panel and the “gold standard” data, obtained by implementing different post-survey adjustment strategies. We use data from the Italian online panel Opinione.net, and from the Multiscopo survey. The former is a non-probability online panel, established in 2011 (see also http://opinione.net/). The latter (considered as a “gold standard”) is a probability based survey that collects a wealth of information on the socio-economic characteristics and opinions of the Italian population (N=44.974; RR: 78.9%). Preliminary analysis shows evidence for sample selection bias; compared to the general population, the panellists are more educated, are more likely to be young adults, and employed. We found no major differences in area of residence and sex