Annual canvass reform proposals: consultation paper

Annex 1 - Summary of the 2016 & 2017 pilots

In 2016 and 2017, electoral administrators from twenty-four Local Authorities and Valuation Joint Boards ( VJBs), working alongside their Electoral Management Software ( EMS) suppliers and the Cabinet Office’s Modern Electoral Registration Programme, delivered pilots to test whether or not alternatives to the legislated annual canvass exist that are more efficient and at least as effective.

Four alternative canvassing models were tested using randomised controlled trials: the Household Notification Letter ( HNL) model, the email model, the telephone model and the discernment model. The HNL model issued a HNL (household notification letter) that listed the details of everyone registered to vote in a household, and only required a response if there had been a change to the details listed. The email and telephone models piloted different ways of contacting households. Under the email model, if an email address was held for a household they would be contacted twice by email, once by post and then receive a household visit. Under the telephone model, if a telephone number was held for the household, a phone call would replace the household visit stage of the canvass. Under these models, if an email address or telephone number was not held, the household followed a more typical canvass process of two contacts by post and a household visit.

The Discernment model also issued a HNL by post. However, the key difference here was that the ERO would only issue a HNL if the individuals in the household could be matched using local data; the ERO could therefore be more confident there had been no change to the details listed. Households that couldn't be matched using local data followed the same process as under the email model, where they were contacted by email (if an email was held), by post and finally with a household visit.

Each LA or VJB completed the usual legislated canvass in their control group and their chosen alternative canvass model in their intervention group. By comparing the results of two approaches, delivered at the same time and in the same area, we can show that the difference in outcomes was driven by the alternative approach tested. Through our analysis we found that while each model was successfully implemented, only the telephone and email canvass models were as effective as the legislated canvass at a lower cost. On average the telephone model cost 30% less than the usual canvass and the email model cost an average 22% less.

The HNL and discernment models made larger savings of 65% and 37% respectively but were not as effective as the legislated canvass. This can be largely attributed to the HNL being less effective in capturing the same volume of information as the usual canvass, and the quality of the data that routed households to the HNL as part of the discernment model.

However, while the discernment model as tested was not as effective as the usual canvass we found there were benefits to the processes it introduced. For example we found that, while the quality of data could be improved, using data was effective in targeting households that did not need to report a change in household composition. With 57% to 83% of households across the pilot sites reporting that there had been no change to their household composition. A data driven approach that targets resources appears both an effective process and cost saving solution. Equally, through the email and discernment models, we found that using emails alongside one posted HEF and a household visit was just as effective as the legislated canvass and less expensive.

While the legislated canvass implemented in each control group was used to assess each alternative model, EROs also emphasised their reservations about returning to the legislated canvass approach. A collective belief that the usual process is costly and repetitive reiterates clear support for modernisation attempts - and the canvass pilots’ evaluation process has set a robust evidence-base to inform this.