SLOMethodology
Sample
The units for the 1KA Panel sample are selected from the Central Register of the Population. Units are included based on a probability mechanism. The selection procedure is a simple random sample (SRS), which is improved upon in terms of a systematic sample during the implementation process, where implicit stratification according to the 12 statistical regions and 5 types of settlements is taken into account (Kalton & Vehovar, 2023).
The target population consists of all residents of the Republic of Slovenia aged 18 years or older who have a permanent residence or an official resident status in Slovenia. There is no upper age limit.
Participation and Non-Responses
Recruitment to the panel is based on a postal invitation sent to all randomly selected individuals from the population register. The selected individuals receive a 10€ voucher if they participate in the survey. Non-respondents will receive up to three more reminders via post and/or e-mail. Experiments are also being carried out to offer 5€ upfront unconditionally instead of the conditional 1€0, or even both (5€ upfront and 10€ after completing the survey). The initial recruitment of units to the 1KA Panel in 2022 had a response rate of 33%. In addition, the initial recruitment also took the form of a follow-up survey of two other probability samples. For details, see the report of the Probability web panels in national statistics for persons and households project.
In total, there were 1,626 panellists in 2023, who were selected from the population register and agreed to participate in the panel. The panel has been further renewed and expanded in 2024. In total, it includes 3,029 panellists.
The overall recruitment rate (ORR) - the proportion of invitees who agree to take part in the 1KA Panel - is 20 %, as a third of those who take part in the recruitment survey then decide not to continue taking part in the panel. This places the 1KA Panel very favourably in an international context - indeed, Kocar & Kaczmirek (2023) report that probability panels have an average ORR of 17 %. It is worth bearing in mind that the 1KA Panel uses a simplified calculation of response rates and ORRs that does not filter out non-respondents in the denominator, which means that participation is underestimated by 3 - 5 %. In the case of letter canvassing, response rates can be increased by up to about 5 percentage points, and with additional effort (cost) the recruitment rate can be further increased, but it is difficult to exceed 40 % with letter canvassing. This is in principle only possible for a one-off survey with very topical content (e.g. national health).
If we take into account the non-response rate in a given wave, where around 90 % of invited panellists in the 1KA Panel respond (which is extremely high in domestic and international comparisons) the overall response rate in a given wave of the 1KA Panel is around 18%. In principle, the overall response rate can be further increased, but the costs increase disproportionately. The CRONOS online panel, which had considerably higher costs, did achieve an overall response rate of 23% in Slovenia a few years ago (see Lugtig & Maslovskaya, 2020), which is probably the upper limit of participation for such panels in Slovenia.
It should be noted, however, that the above-mentioned participation rate in the recruitment survey (33 - 35%) can represent the response rate in the case of a newly recruited sample (outside the 1KA Panel).
Panel Wave Distribution
Panel waves currently planned for 2024:
- WAVE 6
Past panel waves:
- WAVE 0': Slovenian public opinion 2022/2
- October 2022
- N = 2,500, response = 1,041 (42%)
- WAVE 0: Current aspects of health and education
- December 2022
- N invited = 4,340, response = 1,681 (39%)
- N overall = 6,600, response = 1,681 (25%)
- WAVE 1: Living conditions
- February 2023
- N invited = 1,628, response = 1,517 (93%)
- N overall = 9,100, response = 1,517 (17%)
- WAVE 2: Travel habits, use of information and communication technologies (ICT) and risky behaviours
- April 2023
- N invited = 1,629, response = 1,486 (91%)
- N overall = 9,100, response = 1,486 (16%)
- WAVE 3: Views on current social problems in Slovenia
- May 2023
- N invited = 1,627, response = 1,454 (89%)
- N overall = 9,100, response = 1,454 (16%)
- WAVE 4: Work activity, mental health and safe use of the Internet
- June 2023
- N invited = 1,620, response = 1,439 (89%)
- N overall = 9,100, response = 1,439 (16%)
- WAVE 5: Internet use and current social topics
- June 2024
- Existing panelists:
- N invited = 1,620, response = 1,341 (83%)
- N overall = 9,100, response = 1,341 (15%)
- New recruitment:
- N = 7,300, response = 2,425 (33%)
- Total:
- N invited = 8,917, response = 3,766 (42%)
- N overall = 16,400, response = 3,766 (23%)
Survey Process
In terms of the survey process, the following points are worth highlighting:
- The surveys are primarily conducted online. If necessary, combinations of methods may be used (e.g. a postal survey for respondents who are unable or unwilling to respond online).
- As a rule, panellists participate in the survey once every two months; while there are some variations, no panellist responds more than once a month.
- If they are selected for a particular survey, panellists will receive a 5€ gift voucher by post in advance (regardless of participation), together with a written invitation and a link to the survey.
- Each survey usually takes about 20 minutes.
- Non-respondents are sent up to three more reminders to their e-mail address (usually at weekly intervals).
- Typically, 90 % of all invited panellists participate in the surveys; more than 80 % of panellists have participated in all panel waves so far.
- For most of the surveys, however, not all panellists need to be recruited. In fact, for questions on ordinal 1-to-5 scales, subsamples of a few hundred units are sufficient, or even less if the general population is being studied.
Representativeness and Weighting
In statistical terms, representativeness refers to a variable rather than a sample, and the risk of achieving the expected precision (e.g. confidence interval) must be defined for that variable. A common statistical standard in the social sciences is a risk of 5 %, and precision is defined by the relative error, where the coefficient of variation cv is less than 0.1 (Kalton & Vehovar, 2003, page 126). This means that the confidence interval - with a normal risk level of 5 % - can be as wide as ± 20 % of the estimate, e.g. for an estimate of 10, this means 10 ± 2. The 1KA Panel provides this kind of representativeness for practically all variables in samples of around 1,000 units. The exception are very small groups of units, where the number of units is close to or below 10. The 1KA Panel ensures this type of representativeness through probabilistic selection of units and high quality of implementation. It is worth adding, however, that the concept of 'representativeness' can be interpreted in other ways, which are often unclear or even misleading (see the discussion of the concept of 'representativeness').
In addition, the representativeness of individual control variables (region, settlement type, gender, age and education) can be further improved through weighting. The Slovenian general population is slightly under-represented in the 1KA Panel in terms of male respondents, respondents over 70 years of age and respondents with a lower level of education. As these are small deviations, weighting usually has a mere cosmetic effect - the sample is perfectly matched to the population in the control variables after weighting. This gives a good impression, and the survey estimates themselves rarely change by more than a few percentage points.
The data is weighted using the raking method (sample structure is iteratively adjusted to the population structure) using the 1KA open-source weighting tool SurveyWeightingGUI. The tool is user-friendly so that users can also weight the data themselves. The weighting defaults to cutting the weights at 5, full convergence and normalising the weights to the base sample size.