Early learning and childcare at age five: comparing two cohorts

2 Methods

2.1. Sample and data overview

This report uses data collected from the two GUS birth cohorts ( BC1 and BC2). The majority of the analysis draws on data collected at the time the children were just under 5 years old. ^[7] In some instances (such as details about ELC use), data collected when the child was aged 4 were also used. For BC1, this means the analysis draws primarily on data collected in 2009/10, with some data collected in 2008/09. For BC2, the analysis draws primarily on data collected in 2015, with some data collected in 2014.

The cohorts are both comprised of nationally representative samples of children living in Scotland at age 10 months (their age at the first sweep of data collection) and who were born over a specific time period: all children in BC1 were born between June 2004 and May 2005; all children in BC2 were born between March 2010 and February 2011. For simplicity, comparisons in the report refer to BC1 and BC2. However, the results for each cohort should be understood to represent all children of the respective age living in Scotland at the time point in question and any statistically significant differences noted in the report should be taken to reflect actual differences in the circumstances and experiences of children of the relevant age and their families. They have not occurred because of differences in the research design for the two cohorts.

For example, the results presented for children in BC2 at the time they were aged 5 are representative of all children of this age living in Scotland in 2015 (when the GUS data were collected) who had also been living in Scotland when they were 10 months old.

The majority of GUS data have been collected through face-to-face interviews with children and parents in their homes. In addition, on three occasions (twice with families in BC1 and once with families in BC2) data were collected through the use of short online questionnaires which could also be completed over the phone. This report draws on data collected both as part of in-home interviewer visits and data collected through the use of online/telephone questionnaires (see Table 2-1).

Table 2 ‑ 1 Data overview

Child's age	4 years		5 years
Cohort	BC1	BC2	BC1	BC2
Year of data collection	2008/09	2014	2009/10	2015
Data collection mode	In-home visit	Online/telephone survey with child's main carer	In-home visit	In-home visit
Number of interviews achieved*	3994	3237	3833	4434

*The focus of the analysis is on the outcomes of children at the age of 5. Therefore, only children whose families took part in the age 5 interview were included in the analysis.

In addition to the GUS survey data, the analysis also draws on administrative data from the Care Inspectorate which was 'linked' to the GUS data. Further information is provided below.

2.2. Survey data on ELC

Around the time the child was aged 4, parents in both cohorts were asked whether the child was attending a 'pre-school' or early education place. If so, further information was collected, including information about the number of hours the child would usually spend in this setting on a weekly basis and details of the provider. ^[8]

As outlined in section 1.3, since the introduction of the CYP Act 2014 the term 'Early Learning and Childcare' ( ELC) has largely replaced that of 'pre-school education' among policy makers and practitioners in Scotland. Accordingly, in this report ' ELC' is used to refer to what was previously referred to as pre-school education. The focus of the report is on ELC provided by the child's main ELC provider (including both funded and unfunded hours). Thus, figures outlining details of a child's ELC attendance refer to any time he or she spent at their main ELC provider (i.e. their main pre-school provider), but does not include any time spent with a different provider. ^[9] Any childcare the child may have received from other providers is treated separately and is referred to as 'childcare', rather than ELC. ^[10] Further to this, note that only ELC use after the child's third birthday is included in the GUS data.

Data on ELC were collected slightly differently in the two cohorts. In BC1, ELC data were collected through interviewer-administered questionnaires, the majority as part of the age 4 interview though some parents were asked as part of the age 5 interview. In this report BC1 age 4 data were used as far as possible to ensure the highest possible level of comparability with BC2 data (see below). Age 5 ELC data were used in cases where age 4 information was incomplete or non-existent. In total, ELC data were collected for 3731 children in BC1 - for 3459 of these, ELC data were obtained at the time the child was aged 4; for the remaining 272 the data were obtained when they were aged 5.

For some children in BC2 ELC data were collected through an online or telephone questionnaire when the child was aged 4. For others it was collected through an interviewer-led questionnaire when the child was aged 5. At age 4, information was collected about the child's current ELC attendance and provider. The ELC data collected from BC2 families as part of the age 5 interview were collected about current ELC use or, if the child was not currently using any ELC, about ELC use since the child's 3 ^rd birthday. In total, ELC data were collected for 4314 children in BC2 ^[11] - for 2908 of these, ELC data were obtained at the time the child was aged 4 ^[12] ; for 1406 the data were obtained when they were aged 5. ^[13]

The data on ELC attendance and ELC provider type used in the analysis are described in more detail in sections 3.3 and 3.4.

2.3. Linked admin data from the Care Inspectorate

The report draws on linked admin data from the Care Inspectorate. As noted in section 1.3.2, the Care Inspectorate regulates care services for children and adults in Scotland. This includes a wide range of services for children and young people, including fostering, adoption and residential care along with child-minding and day care services. This latter category includes settings which provide ELC.

During inspections, the Care Inspectorate assesses the quality of the provider against four quality themes: care and support, environment, staffing, and management and leadership. Until 2016, ^[14] these quality themes each had accompanying quality statements - these are shown in Table 2-2. For each quality theme, services were awarded a grade between 1 and 6 (Care Inspectorate, 2016) indicating the following: 6 - excellent; 5 - very good; 4 - good; 3 - adequate; 2 - weak; 1 - unsatisfactory.

Table 2 ‑ 2 Care Inspectorate quality themes and statements

Quality theme	Accompanying quality statements
Quality of care and support	Ensures that parents and families participate in assessing and improving the quality of care and support provided by the service. Enables service users to make individual choices and ensures that every service user can be supported to achieve their potential. Ensures that service users' health and wellbeing needs are met. Uses a range of communication methods to ensure we meet the needs of service users.
Quality of environment	Ensures service users and carers participate in assessing and improving the quality of the environment within the service. Ensures the environment is safe and service users are protected. The environment allows service users to have as positive a quality of life as possible. The accommodation and resources are suitable for the needs of the service users.
Quality of staffing	Ensures that service users and carers participate in assessing and improving the quality of staffing in the service. Staff have been recruited and inducted in a safe and robust manner to protect service users and staff. Has a professional, trained and motivated workforce which operates to National Care Standards legislation and best practice. Ensures that everyone working in the service has an ethos of respect towards service users and each other.
Quality of management and leadership	Ensures service users and carers participate in assessing and improving quality of the management and leadership of the service. Involves the workforce in determining the direction and future objectives of the service. To encourage good quality care, promotes leadership values throughout the workforce. Uses quality assurance systems and processes which involve service users, carers, staff and stakeholders to assess the quality of services provided.

2.3.1. Matching the admin data to the survey data

The administrative data on ELC settings used in this report are produced using data from inspection reports. Data on the grades awarded following inspection are held electronically and published on the Care Inspectorate website.

Children in BC1 were born between 1 June 2004 and 31 July 2005. As such, the first children eligible to attend their statutory ELC place in this cohort would have done so from the autumn term of 2007. Most of these children would have continued to attend until June 2009 before starting school in autumn 2009. However, across the cohort as a whole, children would have started ELC sometime between August 2007 and August 2008 and would have potentially attended up to June 2010. Some children may have moved between providers or received their ELC entitlement from two different providers. However, it is not possible to identify which children did so from Care Inspectorate data. In addition, some settings closed or merged or moved and new settings were established and registered.

To reflect the full period of attendance for all children in the cohort, Care Inspectorate data were requested covering the entire period from 2007 to 2010. Where data on a single provider were available for multiple years, for example, quality ratings from multiple inspections, an average figure was calculated.

ELC details provided by parents as part of the GUS interview were successfully matched to Care Inspectorate information for 3658 children following the age 4 interview (99%). This corresponded to 1296 ELC centres meaning that, on average, each centre was attended by two to three children in BC1. ^[15] Grades on quality themes from Care Inspectorate inspections were available for between 2878 and 3013 BC1 children. ^[16] For the small number of BC1 cases where Care Inspectorate quality data had not been successfully matched based on the information provided at the age 4 interview, another attempt was made using ELC provider details obtained at the age 5 interview. This resulted in the addition of Care Inspectorate quality information for between 38 and 47 cases, depending on the quality measure.

Children in BC2 were born between 1 March 2010 and 28 February 2011, with the first children becoming eligible to attend their statutory ELC place from the autumn term of 2013. The majority of these children would have continued to attend until June 2015 before starting school in autumn 2015. Across the cohort as a whole, children would have started ELC sometime between August 2013 and August 2014, and would have potentially attended until June 2016. Care Inspectorate data were therefore requested covering the entire period from 2013 to 2016.

4426 children in BC2 attended ELC at some point after turning 3 years old. As in BC1, parents were asked for details about the child's main ELC provider. Rather than manually linking survey records to Care Inspectorate data in the office (as was done for BC1), interviewers were able to use an electronic lookup table to select the ELC provider from a pre-defined list provided by the Care Inspectorate. The list included all ELC services that were operating at the point immediately before fieldwork commenced and incorporated a unique Care Inspectorate identifier. This meant the Care Inspectorate identifier for the provider was already appended to the survey data, allowing subsequent matching of other data from the Care Inspectorate.

Survey data were successfully matched to Care Inspectorate quality information for 3941 BC2 children who took part in the age 5 interview (91% of those who attended ELC). This corresponded to 1401 ELC centres meaning that, on average, each centre was attended by two to three children in BC2.

2.4. Child outcomes considered in the report

This report considers three different types of child outcomes, all measured at the time the cohort children were aged just under 5:

• Adjustment to primary school: Information was collected from the child's parent or carer in cases where the child had started primary school at time of interview
• Social, emotional and behavioural development: Information was collected from the child's parent or carer using the Strengths and Difficulties Questionnaire ( SDQ), for all children
• Cognitive development: Objective assessments were carried out with all children where consent was obtained.

Details about each type of outcome measure are provided below.

2.4.1. Adjustment to primary school

To assess how well children in BC1 and BC2 had transitioned from ELC to primary school, questions about adjustment to school were asked of the child's main carer as part of the age 5 interview ^[17] where the child had already started Primary 1. This applied to 1227 children in BC1 and 2750 children in BC2.

Two composite measures of the child's adjustment to primary school were developed based on responses to six questions (shown in Table 2-3). Each measure is described in turn below.

Table 2 ‑ 3 Questions on adjustment to primary school

Question	Response categories	Most positive response
How often has your child complained about school?	More than once a week Once a week or less Not at all	Not at all
How often has your child said good things about school?	More than once a week Once a week or less Not at all	More than once a week
How often does your child look forward to going to school?	More than once a week Once a week or less Not at all	More than once a week
How often has your child been upset or reluctant to go to school?	More than once a week Once a week or less Not at all	Not at all
Is your child finding it hard to sit still and listen?	Agree strongly Agree Neither agree nor disagree Disagree Disagree strongly	Disagree/disagree strongly*
Overall, has your child adjusted well to school?	Agree strongly Agree Neither agree nor disagree Disagree Disagree strongly	Agree/Agree strongly*

*Categories were combined as shown.

The first composite measure identifies three levels of adjustment to primary school: 'excellent', 'good/average' and 'poor'. Definitions are provided in Table 2-4 below.

Table 2 ‑ 4 Adjustment to primary school: composite measure (3 categories)

Level of adjustment	Definition
Excellent	Most positive response given for all six items
Good/average	Most positive response given for at least one, but not for all six items
Poor	Most positive response not given for any of the six items

The second composite measure identifies scores which were 'below average' versus those which were 'average and above'. This measure was derived by adding up standardised scores for the six individual items to create a total score. ^[18] Those with a score below the mean were classified as 'below average' while the remainder were classified as 'average or above'.

Table 2 ‑ 5 Adjustment to primary school: composite measure (2 categories)

Level of adjustment	Definition
Average or above	The child's total score summed across all six items was at or above the average (mean) for all children in the cohort whose parent/carer answered the questions
Below average	The child's total score summed across all six items was below the average (mean) for all children in the cohort whose parent/carer answered the questions

2.4.2. Social, emotional and behavioural development

On GUS, measures of social, emotional and behavioural development are routinely obtained through the use of items from the Strengths and Difficulties Questionnaire ( SDQ) (Goodman, 1997). A parent report version of the SDQ was included in the self-completion section of the age 5 interview in both BC1 and BC2. ^[19]

The SDQ is a commonly used behavioural screening questionnaire designed for use with children aged between 3 and 16. It consists of 25 questions about a child's behaviour, to which the respondent can answer 'not true', 'somewhat true' or 'certainly true'. Responses can be combined to form five different measures of the child's development, namely emotional symptoms (e.g. excessive worrying), conduct problems (e.g. often fighting with other children), hyperactivity/inattention (for example, constantly fidgeting), peer relationship problems (e.g. not having close friends), and pro-social behaviour (e.g. being kind to others). Furthermore, the first four measures can be combined into a 'total difficulties' scale. Higher scores imply greater evidence of difficulties on each of the scales, with the exception of the pro-social behaviour scale where the reverse is true.

In this report, both mean scores and banded versions of the scales have been used. Mean scores are useful for comparing overall levels of social development across groups of children or over time, while the banded versions provide more nuanced and more easily interpretable measures. Previously, SDQ scores were most commonly divided into 'normal', 'borderline' and 'abnormal' scores. These bandings were reviewed in 2016 and it is now recommended that SDQ scores on each of the scales are divided into the following categories: 'close to average', 'slightly raised', 'high' and 'very high', with 'very high' indicating multiple problems identified. The exception is the pro-social scale, which is divided into 'close to average', 'slightly lowered', 'low' and 'very low', with 'very low' indicating very little pro-social behaviour. Across the different scales, the further above (or below) average the score is, the greater the potential cause for concern.

Additionally, the definitions of the scores which constitute each of the categories have been re-standardised to reflect a more recent, British sample. ^[20] Scores for BC1 and BC2 were derived using the same cut-off points to allow comparison.

2.4.3. Cognitive ability

At age 5, the cognitive ability of children in BC1 and BC2 was assessed using the British Ability Scales ( BAS). The BAS is a cognitive assessment battery designed for children aged between 2 years and 6 months and 17 years and 11 months. Though numerous tests of ability and intelligence exist, the BAS are particularly suitable for administration in a social survey like GUS. Two subtests of the BAS were used - the naming vocabulary and picture similarities subtests - which were each administered individually.

The naming vocabulary assessment measures a child's language development. The test requires the child to name a series of pictures of everyday items in order to assess their expressive language ability. Conversely, the picture similarities test measures a child's problem-solving skills (or non-verbal reasoning ability). In this assessment children are shown a row of four pictures on a page and asked to identify a further congruent picture. The child is therefore required to recognise a relationship based upon a common concept or element. The child must perceive various possibly relevant features of the pictures and engage in hypothesis testing to select the correct elements of commonality. The relationships become increasingly complex as the exercise progresses.

There are 36 items in the naming vocabulary assessment and 33 items in the pictures similarities assessment. However, to reduce burden and avoid children being upset by the experience of repeatedly failing items within the scale, the number of items administered to each child is dependent on their performance. For example, one of the criteria for terminating the naming vocabulary assessment is if five successive items are answered incorrectly. Children in both cohorts were asked to complete these assessments when they were just under 5 years old. As such, the GUS BAS assessment scores offer a snapshot of children's ability in expressive vocabulary and problem solving around the time they start primary school. ^[21]

On completion, the child's raw score (the count of the number of items they answered correctly) is converted into an ability score. The ability score reflects the range and the difficulty of the specific items a child is asked. Lookup tables for the transformation from raw scores to ability scores are provided in the BAS testing materials. The ability scores are then adjusted for the child's age at the time of assessment using scores from a 'norming' sample which are also supplied with the assessment materials. Adjusting the scores in this way avoids older children obtaining higher scores due to their more advanced stage of cognitive development and greater educational experience, rather than their ability.

While the same BAS assessments - naming vocabulary and picture similarities - were used for both cohorts at the same age, the edition of BAS was different. For BC1, the 2 ^nd edition assessment was used ( BAS-II), whereas for BC2 the third edition was used ( BAS-3). While the assessments are almost identical across BAS-II and BAS-3 in terms of administration protocols, a number of minor changes were made which would introduce caveats when making a straightforward comparison of ability scores. To allow this, the assessment authors provided a calibration formula which permitted comparison of the standardised ability scores (t-scores). Note that because of this adjustment, it is not possible to express differences in average cognitive ability scores in terms of developmental age in months (as has been done in a previous GUS report, cf. Bradshaw, 2011) for comparison between cohorts.

2.5. Analytic approach and interpreting the findings

Much of this report is concerned with exploring changes between the two cohorts both at an overall level and for different groups of children according to a number of socio-economic characteristics and location (annual equivalised household income; highest level of parental education in the household; area deprivation ( SIMD); and urban/rural location). Definitions of these measures are provided in Appendix A. For each outcome considered in the report (e.g. cognitive ability), the relationship between the outcome and measures of socio-economic status or location was examined separately for each cohort. This allowed us to identify any noteworthy differences in outcomes - within each cohort - between children in different circumstances. By then comparing the results for BC1 and BC2 it was possible to assess if there had been any change in the nature of the relationship between the outcome variable and the socio-economic or location variable across the cohorts. For example, whether there had been a narrowing or widening of the gap between children in different socio-economic sub-groups. Further details about this analysis and how to interpret tables are provided in Appendix B.

Throughout the report, unless otherwise stated, only differences which were statistically significant at the 95% level or above are commented on in the text.

Not all families who initially took part in GUS did so for all of the subsequent sweeps. There are a number of reasons why respondents drop out from longitudinal surveys and such attrition is not random. Therefore, the data were weighted using specifically designed weights which adjust for non-response and sample selection. All results have been calculated using weighted data and all comparisons take into account the complex clustered and stratified sample structures. Note that because results were calculated using weighted data, the results and bases presented cannot be used to calculate how many respondents gave a certain answer.

2.5.1. Multivariable analysis

A key point of interest in this report is to examine whether there is an association between ELC use and characteristics and child outcomes. When examining such associations it is important to consider the influence of underlying factors and the inter-relationships between such factors and other things we are interested in. For example, the number of hours children spend in ELC is likely to be related to whether their parent or carer is in paid work, as well as their income level. Simple analysis may identify an association between weekly ELC attendance and children's social development - for example, that children who attend ELC for between 12.5 and 16 hours per week have higher levels of social difficulties than children who attend ELC for less than 12.5 hours per week. However, this association may be occurring simply because of an underlying association between ELC attendance and household income. Thus, rather than ELC attendance being associated with children's social development in its own right, the relationship found in the analysis may be due to the influence of other factors. To 'control' for the influence of other factors (e.g. household income) multivariable regression analysis was used. This form of analysis allows the examination of the relationships between an outcome variable (e.g. social development) and multiple explanatory variables (e.g. weekly ELC attendance, household income) whilst controlling for the inter-relationships between each of the explanatory variables. This means it is possible to identify whether there is an association between any single explanatory variable and the outcome variable also when other relevant variables have been controlled for. For example, to look at whether there is a relationship between duration of time spent in ELC and children's social development that does not simply occur because ELC attendance and household income are related. Note, though, that the identification of associations between one or more explanatory variables and an outcome variable does not necessarily imply that the explanatory variable(s) causes the outcome.

For certain analyses - for example, to consider whether an association between ELC quality and social development was different for children in higher and lower income households - 'interactions' were included in the multivariable models. Where an interaction is statistically significant this indicates that the relationship between the explanatory variable (e.g. ELC quality grade) and the outcome variable (e.g. social development) is different according to the value of another explanatory variable (e.g. household income). This may suggest, for example, that whilst ELC quality is generally associated with children's social development, the relationship is stronger among children in lower income households than among those in higher income households.

The multivariable analysis used both linear and (binary) logistic regression models. Full results of the models are included in the Technical Annex along with notes on how to interpret them.

Note that the statistical analysis and approach used in this report represents one of many available techniques capable of exploring this data. Other analytical approaches may produce different results from those reported here.