Publication - Statistics Publication
Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS): Technical Report 2015
Detailed information on the fieldwork and data processing for the 2015 Scottish Schools Adolescent Lifestyle and Substance Use Survey (SALSUS).
96 page PDF
1.5MB
96 page PDF
1.5MB
Precision of results and measurement of change
Survey respondents represent only a sample of the total population of 13 and 15 year old pupils in Scotland, and results are therefore subject to sampling error. The sampling error is the amount by which the value of a sample estimate for a particular parameter is expected to differ from its true value in the population sampled from. This means that observed differences between sub-groups may have occurred by chance. Throughout the report only differences that are statistically significant, where we can be 95% confident that such a difference has not occurred by chance (p<0.05), are commented upon.
The formula for calculating the sampling error ( SE) of a simple random sample is:
where p = the estimate of the parameter and n=sample size.
The formula for calculating the sampling error for the differences between two estimators (p1 and p2) derived from two independent samples (assuming a simple random sample) is:
Rather than using a simple random sample, whole classes were sampled within the schools that agree to participate. Therefore, classes were clustered within those schools, and pupils are clustered within those classes. Pupils within the same class and within the same school were more likely to be similar to each other, and therefore values cannot be assumed to be independent of one another. Further details on the calculation of standard errors and design effects are provided below.
It is important to recognise that sampling error is only one of the sources of error that affect the accuracy of survey results. Other sources of error include response bias (previously discussed) and over/under reporting, both of which are difficult to quantify.
Calculating standard errors and design effects
The sample design of SALSUS is complex, involving stratification by local authority and school type (state or independent), as well as clustering within schools. In addition, weights were applied when obtaining survey estimates.
Complex design and weighting affects standard errors for survey estimates, which are generally higher than the standard errors that would be derived from an unweighted simple random sample ( SRS) of the same size. For example, clustering reduces the precision of estimates, whereas stratification can increase precision. Weighting can also reduce the precision of estimates.
The ratio of the standard error of the complex sample to that of a simple random sample of the same size is known as the design factor. Put another way, the design factor (or 'Deft') is the factor by which the standard error of an estimate from a simple random sample has to be multiplied to give the true standard error of the complex design.
A Design Factor greater than 1.0 indicates a confidence interval wider than it would be with simple random sampling, meaning that the precision of estimates is reduced. A Design Factor of less than 1.0 indicates a narrower confidence interval and greater precision.
The true standard errors and Defts for SALSUS 2015 have been calculated in STATA using Linearization (Taylor series method), also known as the propagation of variance by Kish, and was the method used from 2006 to 2013. Thirty-six strata were included for the calculation of standard errors and Defts, one for each combination of local authority and school type (state or independent sector). For five local authorities: Dundee City, Fife, Inverclyde, Moray and Stirling, the variance between clusters could not be calculated for independent schools because there was only one independent school sampled in each of these local authorities. The independent schools in these ten local authorities were combined into one stratum to calculate sampling errors and Defts. There were 851 clusters used in the calculations, one for each class that participated in the survey ^{ [4] }.
The Deft values applied and adjusted true standard errors (which are themselves estimates subject to random sampling error) are shown in Tables 5 to 10 for six key variables from the survey along with 95% confidence intervals.
When comparing the 2015 key variables with the 2013 key variables, significance tests were applied based on the 'pooled standard error' for each variable (a weighted sum of the true standard errors ^{ [5] } for each year). Table 11 shows the six key variables for 2013 and 2015, with the true standard errors for each year and whether or not the difference is significant at the 5% level.
Table 5: Linearised standard errors and 95% confidence intervals for the proportion of pupils who are regular smokers, by age and gender: Scotland 2015
Sample Size | Proportion | Linearised Standard error | Binomial Wald 95% CI | Defts | ||
---|---|---|---|---|---|---|
Lower CI | Upper CI | |||||
13 year olds | ||||||
Boys | 6439 | 1.51% | 0.19% | 1.15% | 1.88% | 1.19 |
Girls | 6712 | 1.65% | 0.21% | 1.24% | 2.07% | 1.29 |
15 year olds | ||||||
Boys | 5657 | 7.39% | 0.48% | 6.45% | 8.33% | 1.45 |
Girls | 5595 | 7.21% | 0.48% | 6.27% | 8.14% | 1.44 |
Table 6: Linearised standard errors and 95% confidence intervals for the proportion of pupils who drink alcohol at least once a week, by age and gender: Scotland 2015
Sample Size | Proportion | Linearised Standard error | Binomial Wald 95% CI | Defts | ||
---|---|---|---|---|---|---|
Lower CI | Upper CI | |||||
13 year olds | ||||||
Boys | 6522 | 2.25% | 0.24% | 1.78% | 2.71% | 1.24 |
Girls | 6743 | 2.38% | 0.24% | 1.91% | 2.85% | 1.22 |
15 year olds | ||||||
Boys | 5714 | 11.96% | 0.59% | 10.80% | 13.12% | 1.45 |
Girls | 5607 | 12.98% | 0.68% | 11.65% | 14.30% | 1.58 |
Table 7: Linearised standard errors and 95% confidence intervals for the proportion of pupils who drank alcohol in the last week, by age and gender: Scotland 2015
Sample Size | Proportion | Linearised Standard error | Binomial Wald 95% CI | Defts | ||
---|---|---|---|---|---|---|
Lower CI | Upper CI | |||||
13 year olds | ||||||
Boys | 6166 | 3.83% | 0.34% | 3.16% | 4.51% | 1.35 |
Girls | 6431 | 4.22% | 0.33% | 3.56% | 4.87% | 1.27 |
15 year olds | ||||||
Boys | 5485 | 15.89% | 0.67% | 14.57% | 17.21% | 1.44 |
Girls | 5479 | 18.91% | 0.78% | 17.38% | 20.44% | 1.54 |
Table 8: Linearised standard errors and 95% confidence intervals for the proportion of pupils who have ever used drugs, by age and gender: Scotland 2015
Sample Size | Proportion | Linearised Standard error | Binomial Wald 95% CI | Defts | ||
---|---|---|---|---|---|---|
Lower CI | Upper CI | |||||
13 year olds | ||||||
Boys | 6193 | 5.80% | 0.39% | 5.03% | 6.57% | 1.28 |
Girls | 6531 | 4.92% | 0.37% | 4.20% | 5.65% | 1.31 |
15 year olds | ||||||
Boys | 5476 | 20.89% | 0.80% | 19.33% | 22.45% | 1.53 |
Girls | 5490 | 15.94% | 0.69% | 14.58% | 17.30% | 1.47 |
Table 9: Linearised standard errors and 95% confidence intervals for the proportion of pupils who have used drugs in the last year, by age and gender: Scotland 2015
Sample Size | Proportion | Linearised Standard error | Binomial Wald 95% CI | Defts | ||
---|---|---|---|---|---|---|
Lower CI | Upper CI | |||||
13 year olds | ||||||
Boys | 6193 | 4.92% | 0.35% | 4.23% | 5.62% | 1.24 |
Girls | 6531 | 4.15% | 0.34% | 3.50% | 4.81% | 1.29 |
15 year olds | ||||||
Boys | 5476 | 18.78% | 0.76% | 17.28% | 20.27% | 1.52 |
Girls | 5490 | 14.20% | 0.64% | 12.93% | 15.46% | 1.43 |
Table 10: Linearised standard errors and 95% confidence intervals for the proportion of pupils who have used drugs in the last month, by age and gender: Scotland 2015
Sample Size | Proportion | Linearised Standard error | Binomial Wald 95% CI | Defts | ||
---|---|---|---|---|---|---|
Lower CI | Upper CI | |||||
13 year olds | ||||||
Boys | 6193 | 3.14% | 0.29% | 2.58% | 3.70% | 1.24 |
Girls | 6531 | 2.88% | 0.27% | 2.35% | 3.41% | 1.23 |
15 year olds | ||||||
Boys | 5476 | 13.44% | 0.67% | 12.13% | 14.75% | 1.52 |
Girls | 5490 | 8.65% | 0.51% | 7.65% | 9.65% | 1.41 |
Table 11: Statistical significance of comparisons between 2013 and 2015 results for key variables
2013 SALSUS | 2015 SALSUS | T test (2 sided) | P - Value | Significant at the 5% level? | |||||
---|---|---|---|---|---|---|---|---|---|
% | True standard error | Sample size | % | True standard error | Sample size | ||||
Regular smokers | |||||||||
13 yr old boys | 1.82 | 0.199 | 8518 | 1.52 | 0.188 | 6439 | 1.111 | 0.267 | No |
13 yr. old girls | 1.71 | 0.166 | 8546 | 1.65 | 0.211 | 6712 | 0.208 | 0.835 | No |
15 yr. old boys | 8.27 | 0.48 | 8080 | 7.39 | 0.478 | 5657 | 1.296 | 0.195 | No |
15 yr. old girls | 9.00 | 0.467 | 7969 | 7.21 | 0.478 | 5595 | 2.648 | 0.008 | Yes |
Drink at least once a week | |||||||||
13 yr. old boys | 1.90 | 0.212 | 8664 | 2.25 | 0.237 | 6522 | -1.097 | 0.273 | No |
13 yr. old girls | 1.82 | 0.177 | 8607 | 2.38 | 0.239 | 6743 | -1.932 | 0.053 | No |
15 yr. old boys | 11.86 | 0.513 | 8156 | 11.96 | 0.593 | 5714 | -0.128 | 0.899 | No |
15 yr. old girls | 11.25 | 0.512 | 8020 | 12.98 | 0.676 | 5607 | -2.061 | 0.039 | Yes |
Drank in the last week | |||||||||
13 yr. old boys | 4.08 | 0.255 | 8671 | 3.83 | 0.342 | 6166 | 0.571 | 0.568 | No |
13 yr. old girls | 4.22 | 0.294 | 8617 | 4.22 | 0.334 | 6431 | 0.005 | 0.996 | No |
15 yr. old boys | 18.37 | 0.634 | 8178 | 15.89 | 0.674 | 5485 | 2.661 | 0.008 | Yes |
15 yr. old girls | 19.06 | 0.629 | 8027 | 18.91 | 0.780 | 5479 | 0.146 | 0.884 | No |
Taken drugs in last month | |||||||||
13 yr. old boys | 2.34 | 0.198 | 8293 | 3.14 | 0.286 | 6193 | -2.376 | 0.018 | Yes |
13 yr. old girls | 1.75 | 0.166 | 8425 | 2.88 | 0.269 | 6531 | -3.778 | 0.000 | Yes |
15 yr. old boys | 10.92 | 0.494 | 7925 | 13.44 | 0.667 | 5476 | -3.092 | 0.002 | Yes |
15 yr. old girls | 7.83 | 0.405 | 7897 | 8.65 | 0.510 | 5490 | -1.276 | 0.202 | No |
Taken drugs in last year | |||||||||
13 yr. old boys | 3.81 | 0.289 | 8293 | 4.92 | 0.354 | 6193 | -2.460 | 0.014 | Yes |
13 yr. old girls | 2.82 | 0.222 | 8425 | 4.15 | 0.335 | 6531 | -3.455 | 0.001 | Yes |
15 yr. old boys | 16.68 | 0.682 | 7925 | 18.78 | 0.762 | 5476 | -2.056 | 0.040 | Yes |
15 yr. old girls | 14.30 | 0.559 | 7897 | 14.20 | 0.644 | 5490 | 0.122 | 0.903 | No |
Taken drugs ever | |||||||||
13 yr. old boys | 4.79 | 0.322 | 8293 | 5.80 | 0.393 | 4 | -1.998 | 0.046 | Yes |
13 yr. old girls | 3.52 | 0.247 | 8425 | 4.92 | 0.369 | 6531 | -3.281 | 0.001 | Yes |
15 yr. old boys | 19.24 | 0.763 | 7925 | 20.89 | 0.797 | 5476 | -1.496 | 0.135 | No |
15 yr. old girls | 16.34 | 0.651 | 7897 | 15.94 | 0.695 | 5490 | 0.418 | 0.676 | No |