Probability Online Panel 1KA

The 1KA Panel is a pool of adult Slovenian residents who were recruited from the population register via probability sampling (a.k.a. random sampling) and who have consented in advance to participate in online surveys (details on sample, recruitment and participation are in the Methodology).

In 2024, the 1KA Panel consists of around 3,400 panelists. It is primarily used for surveys in the academic and public sectors, where a higher quality of results is required. Due to the probability sample, relatively high participation rate and rigorous implementation of surveys, the panel provides survey estimates with low biases.

The 1KA Panel surveys are conducted online but can also be supplemented with mixed-mode survey methods (e.g., postal or telephone surveys).

The 1KA Panel is managed by the Centre for Social Informatics (CSI) at the Faculty of Social Sciences, University of Ljubljana, which also develops the online survey tool 1KA, after which the panel is named. The panel is financed by funds from the ARIS agency, European projects and funds from clients.

In case of interest, the 1KA Panel infrastructure can also be used to conduct a survey based on a new probability sample, i.e. outside the 1KA Panel, or in the context of a new recruitment wave where participation is particularly high. This is especially applicable considering that probability samples of the general population are nowadays predominantly conducted online.

Online Panels

Online survey modes have revolutionised survey research and the survey industry (see Callegaro et al., 2015). Online surveys have become the dominant survey mode, while at the same time making surveys much simpler, faster, cheaper and more accessible. Today, virtually any internet user can create an online survey (not just specialised agencies as in face-to-face or telephone surveys). As a result, the number of surveys has increased exponentially.

On the other hand, recruiting people to participate in online surveys is becoming increasingly difficult, mainly due to declining participation, rising costs (respondents need to be rewarded) and growing privacy concerns. In this context, online panels have emerged where people agree to regularly participate in various online surveys in exchange for certain rewards. This eliminates the need to recruit respondents for each individual survey.

The vast majority of online panels do not control the probability of people being included in such panels. In this case, we are talking about commercial or non-probability online panels (a.k.a. access panels), which recruit people to the panel in a variety of ad hoc ways (e.g. online, through another survey).

However, for more complex surveys in the public, non-profit and academic sectors, probability online panels have been set up in most developed countries, which select respondents probabilistically/randomly (e.g. from a population register) and therefore do not involve any volunteers. For details, see Vehovar & Čehovin, 2022.

Probability Samples

Probability online panels, such as the 1KA Panel, are based on a probability sample, where each unit in the population has a known and positive probability of being included in the sample (Kalton & Vehovar, 2021, p. 11). Probability samples typically require a list of all units; in the case of the general population, this is usually the population register or a list of all postal addresses in the country.

Probability samples are a prerequisite for generalisation or statistical inference from a sample to a population, i.e. for calculating confidence intervals and testing assumptions. If the sample is not probability-based, then generalisation to the population is, in principle, not possible, and the associated risk is unknown.

Non-probability samples (and also non-probability panels), where we do not know the probability of a unit’s inclusion, are faster and cheaper than probability samples, but give less accurate estimates. Many comparative studies have been carried out (e.g. Cornesse et al., 2020; Mercer & Lau, 2023; Dutwin & Buskirk, 2017), which have uniformly shown that estimates derived from probability panels are significantly more accurate than those from non-probability panels.

The Mission of the 1KA Panel

The mission of the 1KA Panel is to provide a low-cost (non-profit) infrastructure for conducting online surveys based on a probability sample and the highest methodological standards.

The 1KA Panel was piloted in 2022 for the Probability online panels in national statistics for persons and households​ project, which is testing the potential of a probability online panel for national statistics, and the Evaluating probability and non-probability online panels​ project, which is developing an integrated approach for comparing probability and non-probability online panels.

Why Choose the 1KA Panel?

The 1KA Panel is based on probability sampling and has the following advantages compared to probability samples using traditional survey methods (postal, face-to-face and telephone surveys):

• Significantly lower costs.
• Faster data collection process, because recruitment (including respondent consent) has already been done.
• Faster data processing, because each completed survey is automatically entered into the database.
• Lower social desirability bias, leading to more honest answers on sensitive topics (e.g. drugs, health, sexual activity, finance, politics).
• As the panel is relatively large, it enables the selection of the target population based on the data obtained, e.g. only units of a certain age, region, income group, etc.

Compared to commercial or non-probability online panels, the main advantages of the 1KA Panel are:

• Significantly better representativeness of the sample, as the sample is (probabilistically) recruited from the entire adult population in Slovenia (1.7 million) and not just from the few percent of the Slovenian population willing to join commercial non-probability online panels.
• This is also associated with lower bias and thus higher accuracy of estimates, which stems from the probability nature of unit selection. A comprehensive comparison of probability and non-probability panels (TAP) has shown that probability panel estimates are significantly closer to the population value.
• For academic research where data are modeled, an important advantage is the significantly greater diversity of respondents. This also increases the variance that the models can explain. In non-probability panels, respondents are often very similar and most responses may be primarily at one value, e.g. 4 (on a scale of 1-5), with minimal occurrence of other responses (3, 5). In a probability panel, however, the variation is significantly greater because units are recruited from the entire population.
• In particular, probability samples – also called scientific samples (Kalton & Vehovar, 2001, p. 11) – provide a formal basis for statistical inference, so that the risks of sampling error can be calculated in the first place.
• The scientific nature of probability samples can, in the case of academic research, significantly increase the likelihood of results being published in the most reputable scientific journals.
• An important advantage is also the reduced burden on the respondent. This is because panellists receive an invitation to take part in a survey at most once a month, which is very different from commercial panels where invitations are much more frequent. This avoids the risk of panellists becoming so-called ‘professional respondents.’
• The topics covered by the surveys in the 1KA Panel are generally relevant social issues, thus avoiding the extremely broad spectrum of commercial sector content. This general relevance may discourage panellists from responding in a disengaged and routinised manner and especially discourages the over-involvement of respondents interested in a particular topic.
• All questionnaires are systematically reviewed for compliance with the 1KA Panel’s ethical and professional standards, ensuring a high level of quality.
• The participation rate of the panellists is very high (around 80% after two years) for all the reasons mentioned above, which contributes significantly to the quality of the results.

It is also worth highlighting the advantages of the 1KA platform, which the 1KA Panel uses for its surveys:

• Users can develop, create and test the questionnaire themselves in the 1KA tool, which is a very user-friendly, simple and intuitive online survey tool that has been used by more than 100.000 users in Slovenia. This way, they save time and resources while at the same time developing a hands-on relationship with their questionnaire. Of course, external methodologists or content experts can also be involved in the development of the questionnaire.
• Proper use of the 1KA tool eliminates the time-consuming pre-preparation of data, especially in terms of missing values and non-responses (e.g. Don’t know, Prefer not to answer).
• Users can (if they wish) easily weight the data themselves and play with the different options, as a user-friendly 1KA weighting application has been developed for this purpose.
• Both the 1KA platform as well as the 1KA Panel are managed by the Centre for Social Informatics, so the integration of IT support for each step of the survey process is particularly efficient.