The 5 Step Approach to Evaluation: Designing and Evaluating Interventions to Reduce Reoffending

Appendices

Appendix 1: Example proposal for an evidence-based intervention

Why is this service needed?

There are 2 clear reasons for providing this service

1. There is a gap in current provision. The Scottish Prison Service have identified a gap in services aimed at improving prisoners' social, communication and emotion-management skills. There is currently no prison-based service available to address these needs in prisoners.

2. There is strong and consistent evidence that this intervention should help contribute to reducing reoffending.

We have drawn on robust research evidence to demonstrate how our planned activities should lead to prisoners developing better communication and emotional skills which should help them deal more effectively with people, achieve positive goals (such as finding employment) and desist from crime after liberation. We have drawn from reviews of the evidence rather than single studies which can be biased or unreliable. We have drawn on the following sources:-

• Systematic and literature reviews on 'what works' to reduce reoffending to provide justification for throughcare and the needs the service will address .
• A range of qualitative research studies which suggest how the service should be implemented to achieve outcomes and data provided by the Scottish Government on prisoners' reconviction rates.

What outcomes do we want to achieve?

Evidence - what factors contribute to reducing reoffending?
The findings from international systematic reviews of 'what works' to reduce reoffending provides strong and consistent evidence services need to target criminogenic needs to reduce reoffending.

A systematic review published by the Ministry of found a clear association between poor social and emotion management skills with reoffending, therefore they are both justifiable targets for interventions aimed at reducing reoffending. Other dynamic criminogenic needs have been found to be criminal peers, drug use and criminal attitudes.

There is fairly strong evidence that that better outcomes are achieved if services are holistic, well-structured and continue after release (reference).

What outcomes will we be trying to achieve?
This intervention will focus mainly on improving offenders social and emotion management skills. To ensure that this intervention forms part of a wider holistic service which will address multiple needs, this intervention will link into existing throughcare processes as one of the range of services that prisoners can be referred to in prison and in the community.

The community-based part of the intervention will also encourage liberated prisoners to join sports and leisure groups which should help ex-prisoners build pro-social networks.

Therefore this intervention will focus on addressing 3 needs that have been found to be associated with reoffending - improve social skills, improve emotion management and develop social networks

Which activities will achieve outcomes?

Evidence for linking this intervention with existing throughcare processes
Although there is a lack of controlled experimental studies which show that throughcare reduces reoffending, there is a growing body of evidence that throughcare plays an important role in helping short term prisoners reintegrate back into the community and that throughcare can enhance the effectiveness of prison-based interventions (add reference). With particular relevance to social skills and emotion management, some research studies have shown that throughcare can enable prisoners to practice skills they have learned once they return back to their communities which will help them deal with real life situations, seek work, control aggression and engage positively with non-offending peers (add reference).

As there are existing throughcare processes, we have designed this intervention to align with the processes and with the main tenets of effective throughcare which are listed below. According to an international review of throughcare published in 2013 by SCCJR,

• Throughcare should start as early as possible following sentencing
• Pre-release planning is important and prisoners should be involved in devising these plans
• There should be continuity of provision through the gate
• Supporting prisoners to practice their skills in the community embeds more positive behaviour.

How will our service link with throughcare?
In accordance with this evidence, this service will engage prisoners who are referred to us as early as possible and we will also offer one-to-one support available to prisoners on an appointment basis. Once liberated, prisoners will be able continue to improve and practice their social and emotional management skills in the community through the provision of a community-based structured programme.

As there is strong evidence that criminal peers place offenders at a high risk of reoffending, once released the users will be encouraged to find work, join local sports or social clubs of interest to the users so they can forge relationships with non-offending peers. Where possible, links to clubs and employers will be made prior to release to smooth the transition. The community-based service will support prisoners for at least 6 months after release or until a time they are confident in using positive social skills without further support.

Evidence on effective techniques to improve social skills and emotional management
There is strong and consistent evidence from systematic reviews that show CBT is the most effective technique at improving social skills, managing aggression and reducing reoffending (add reference).

What approach will we use in our sessions?
Specialist workers who are trained and experienced in motivating offenders using CBT techniques will run structured interventions within the prison and in the community. These will improve communication and emotion management skills though using CBT approaches, participatory role play within peer groups.

Within the prison, peer groups will be used to practice holding conversations, allowing prisoners to learn from each other how to express ideas and develop effective techniques which help them deal with anger and frustration. As stated above, it is important that once released, prisoners also start to build relationships with pro-social peers.

Evidence on required intensity of support
Risk, Needs and Responsivity principals show that the intensity or 'dosage' of supervisions needs to be relative to the risk of reoffending (add reference).

Our approach to intensity of support
As this group are at a high risk of reoffending (see participants section), this project will provide intensive support. This will take the form of one-to-one support on an on-going appointment basis and two, one hour sessions per week designed to improve social and emotion management skills. This frequency will continue in the community as appropriate to individual users.

Evidence on what increases motivation
Research also shows that targeting this group may present problems with lack of motivation and engagement as the target group may be extremely resistant to receiving support. However research has found there may be rational reasons for a lack of motivation (e.g. poor experiences of interventions in the past, chaotic life-styles which act as a barrier to participation, being forced to own up to their offending, guilt, fear of ridicule from others and a lack of self-efficacy etc).

Our approach to motivating prisoners
Therefore our staff are trained in evidence-based methods to motivate reluctant offenders by developing trusting relationships, showing willingness to focus on and discuss life goals and discuss how they could achieved, agreeing shorter term treatment goals, explaining processes and activities that will be fully explained and by using enjoyable tasks. This should help prisoners to feel motivated and sustain engagement with the programme.

Who will we reach?

Evidence on selecting appropriate target groups
RNR principals suggest that offenders with a medium-high risk of reoffending are most likely to benefit from structured interventions compared with those who are low-risk or very high risk.

Offenders who at a high risk of reoffending tend to have the following characteristics,

• They have a number of prior convictions
• They are serving short term sentences
• They have multiple criminogenic needs

Data on the Scottish prison population provided by the Scottish Government confirms that prisoners serving short term sentences (6 months or less) who have multiple prior convictions have the highest reconviction rate out of all prisoner groups. Approximately 70% of prisoners under 30 serving under 6 months with more than 10 prior reconvictions were reconvicted within a year compared with 50% of all prisoners serving short term sentences.

Which target group will we reach?
This project will therefore target the most prolific male prisoners who are

• Serving sentences of under 6 months
• Have over 10 prior convictions

The total number of prisoners across the whole estate who should be eligible for the service per annum is based on figures for 2010-11 cohort which suggests 1264 prisoners would fall into this category. As this is an intensive service, which requires considerable time spent with prisoners we aim to target 30 offenders per annum.

Appendix 2: An example data collection framework

Example data collection framework for a criminal justice intervention

Part of logic model analysed	Indicators	Data collected from (data source)	Data recorded in…	Data entered into database as…
INPUTS
Were there sufficient resources to run the intervention and how were they deployed? Unless there were sufficient resources, then the quality of the intervention could have been compromised. The evaluation should show what resources were required to run the intervention and whether they were sufficient to deliver the intervention as intended.	The total cost of the intervention. Average £ spent on each user. What were funds spent on? How many staff, were required, staff, staff case loads, costs of running sessions, cost of materials, venues etc. Gather views on whether resources were sufficient. To what extent was the evidence base embedded into the intervention?	Manager and staff Annual accounts	Intervention level database	Costs, values and views Costs can be reviewed periodically (e.g. annually)

Part of logic model analysed	Indicators	Data collected from (data source)	Data recorded in…	Data entered into database as…
PARTICIPANTS Collect information on your users to check that you reach your intended target group. Set up the database so you can collect data on each user. Data can then be aggregated to provide important quantitative data on users e.g. percentages, averages etc. You can also see whether the intervention worked for some users but not others by breaking down outcome data into different types of users (e.g. different ages, offence types) Numbers have to be large and data has to be quantified to do this though.	User ID number	N/A	User level database	Entered as 01, 02, 03 etc
	Name	User	User level database	Name
	Date of birth	User	User level database	Date of birth
	Age at start of programme	User	User level database	Age
	S number (if possible)	Police/CJSW	User level database	S number
	Gender	User	User level database	Column - Gender Male= 1 Female= 2
	Local Authority	User	User level database	Assign code for each LA area
	Current index offence	SOI	User level database	Code crime types e.g. Sex offence = 1 Crime of dishonesty = 2
	Needs e.g.: Substance use Criminal peers Criminal attitudes Housing Employment Education Risk Sentence length Number of previous convictions and LSC/MI score Gender	LSC/MI	User level database	LSC/MI scores and record of needs for each user

Part of logic model analysed	Indicators	Data collected from (data source)	Data recorded in…	Data entered into database as…
ACTIVITES
What did users experience? Information on activities is important because if activities didn't happen or were poorly delivered, then it is unlikely that outcomes will occur, or if they did, something external to the intervention might be responsible. Work out the number and % of users who complete and not complete the intervention as a whole and whether each user completed the activities that were designed to meet their needs.	Date of intervention start/completion	Case record	User level database	Enter dates in xx/xx/xxxx format
	How many users had their needs assessed?	Case record	User level database	For each user Needs assessment 1 = Yes 2 = No
	Describe intervention sessions that each user should have completed. For each user, record the type of modules, activities or sessions that should have been delivered, the number of sessions they should have completed, and the duration of these sessions. Report the content of sessions and mode of delivery (e.g. group sessions, role play, CBT etc)	Intervention supervisors	User level database	For each user, create columns for each module or activity that was supposed to be undertaken by the individual. For each activity record intended number of sessions, session duration, content etc. Create a column for if they completed the intervention as a whole e.g. Completed Yes = 1 No = 2
	For each user, record what modules, sessions or activities they did complete.	Case record	User level database	For each user, assign a code for whether each activity was completed or not. For example: Pro-social skills Completed = 1 Not completed = 2
How did users experience the intervention? Gather user accounts of what they actually did. The extent to which users valued the content and their views on the way the intervention was delivered is important	User perspectives on what happened in the sessions, the length of sessions, the format, quality of relationships with supervisors, what they learned and skills they developed. What were the most useful aspects what were the least useful? If they did not complete, why not? How satisfied were they with the intervention?	User	User level database	Assign codes to closed responses For example, user views on session length - Less than 10mins =1 10-20mins = 2 20-40mins =3 or relationship quality - Poor = 1 Good = 2 and enter into database. or usefulness Very useful = 1 Quite useful = 2 Not very useful = 3 Analyse qualitative open questions (not entered into data base)

Part of logic model analysed	Indicators	Data collected from	Data recorded in…	Data entered into database as…
SHORT- and MEDIUM-TERM OUTCOMES
Did change happen? Quantitative measures of change Obtain a pre-intervention base line and post-intervention assessment end line. Short term outcomes tend to be changes to attitudes, knowledge, learning, motivation or skills. Medium term outcomes show evidence of individual behaviour change. Measure the same outcomes at the start and exit point to see if change occurred.	The difference between the baseline situation and the end situation is the measure of whether change happened. Record the following at the start and end of the intervention For example, record the score on psychometrics tests (if appropriate). Record where users started from on key outcome variables such as attitudes, needs, skills, knowledge, awareness, status, views , feelings, behaviours or competencies.	User and intervention supervisor	User level database	Create two columns-one for the outcome variable before and one column for after the intervention . For example Communication skills 'able to express ideas' (before): Yes =1 No =2 and 'able to express ideas' (after): Yes= 1 No = 2 Or Housing Has stable accommodation (before): Yes = 1 No= 2 Has stable accommodation (after): Yes = 1 No = 2
Qualitative measures of change As well as scales, ask the users, supervisors and family as to whether they think users have changed and in what way. If there is no control group but you want to explore attribution you could elicit views on the relative impact of the intervention by asking users and family about perceived impact the other interventions or support has had. Could also observe sessions at the exit to see if progress has been achieved.	Did offenders friends and family think progress has been made ? Clinical judgement of progress. User and family views on the contribution of external factors to offender outcomes relative to intervention.	Users friends and family Intervention Supervisor User and family	User level database	Could code answers e.g. Supervisor views of change 'Made progress' Yes = 1 No = 2 And/or transcribe interviews for more depth information
LONG-TERM OUTCOMES
Did the intervention reduce reoffending?	Statistical differences in outcomes between a control group and the treatment (user) group	Reconviction data	Separate analysis conducted