Dr. Gregor Čehovin, a member of the CDI, presented an extensive comparative analysis of the bias of estimates in 18 surveys at the Open Science Session at the Faculty of Social Sciences on January 28th. The surveys included 2,600 variables and were conducted in parallel traditionally (e.g., surveys by the Statistical Office of the Republic of Slovenia and the National Institute of Public Health), in a probability online panel (1KA panel) and a non-probabilistic online panel (Valicon).
The bias of survey estimates (e.g., a survey estimate of a party’s election result is 20%, while the actual result is 30%) was compared across different areas of social science surveys. It turned out that survey estimates in traditional surveys are less biased than in probability online panels, where the bias is smaller than in non-probability online panels but still present. On the other hand, costs increase inversely with bias. The key factor influencing the extent of bias is the research topic. The characteristics (sociodemographic profile, attitudes, behavioral patterns) of the people included in the probability and non-probability panels were also presented.
More information
- Slides: Comparison of online panels and traditional surveys: Analysis of estimates biases (Gregor Čehovin, Vasja Vehovar, Andreja Praček, Luka Štrlekar, Kaja Forte).
- Link to news and gallery.