Evaluating your health and wellbeing programme

Planning your evaluation

Evaluation can show where things are working well and where there is room for improvement. It can help to move from providing a good service to an excellent one. Robust evaluation can help to make a difference, demonstrate impact, improve your work and offer value for money.

An important starting point is to think about evaluation right from the beginning of an initiative – even before it has been implemented if possible. This will allow you to clearly prioritise your goals and outcomes, gather internal support and develop a clear evaluation strategy. A logic model may be a good way to help you with this. 

It is important to match your evaluation tools to the type of intervention. In broad terms, evaluation can take place at three levels: individual, organisational and system. In general, you will be dealing with individual or organisational-level interventions, while an integrated care board (ICB) may look to do more interventions at system level.

An individual-level evaluation can look at the impact on the health of individual members of staff. For this, many organisations use feedback forms, which give a snapshot in time about how people feel and whether they enjoyed and valued the intervention. What this doesn’t measure however is the impact it had on the individual's behaviour, and if they have made a positive shift as a result of taking part. Measuring behaviour change can help you to demonstrate where your health and wellbeing strategy was effective and help you to improve your initiatives as a result.

There are several standardised health questionnaire tools (including mental health measures) which are widely used in healthcare. It is recommended that when seeking to measure impact, you use one of these tools to enable comparability. It is also important that when evaluating health impact, you consider the health status of individuals taking part in the programme and that there is usually an element of positivity bias in self-reported surveys. You should compare results across all those signed up for the intervention rather than just those who complete the course. It is generally not possible in health interventions in NHS organisations to ensure a random sample of those who take up the offer. A control group of similar size which is not covered by the intervention may be helpful in identifying impact, but may not be practicable. In general, information will be a before and after study, which is a valid form of assessing impact provided you are aware of its limitations.

Process evaluations should also be considered, as they look at how the intervention was implemented. Undertaking a process evaluation may provide a greater understanding as to why some wellbeing interventions may be more effective than others in terms of staff uptake. It is important to gain a greater understanding of any barriers to participating in current wellbeing interventions, so new interventions can learn as to what employees will use, and where spend in interventions will be more worthwhile.

This can generate useful learning which can be applied to ongoing interventions and be useful for other organisations considering the same intervention. In this field, it is clear that it is not just what you do that has an impact, but how you go about doing it. 

When you start planning your evaluation, consider the following key steps.

Define your objectives

Objectives should set out in your evaluation plan what your overall health and wellbeing programme/initiative aims to achieve, and the overall goals that will be used to measure performance.

The objectives should always be clear, provable (if you will not be able to prove an objective has been met, then think about revising it) and relevant to your role and the remit of your strategy.

Health and wellbeing initiatives often involve partners or participation from staff, and success can be dependent upon a number of external factors beyond your control. Make sure your objectives are not impossible for you and your team to achieve. You may find the following questions and points helpful when setting objectives:

• What will success look like? Make sure this is definable and achievable.
• How will you measure the impact of this intervention/programme? For example, via surveys or interviews.
• What would you ideally like the outcome to be? This could be improved mental health, reduced absenteeism or fewer occupational health referrals.
• Have you built in the opportunity for staff feedback on the overall process? Ensuring staff are engaged is essential and incorporating feedback could improve the effectiveness of an intervention. Staff input can make a crucial difference in ensuring staff feel engaged with the process. Line managers generally have a key role in the success of an initiative and should therefore be involved with planning and delivery. NHS England has developed a useful guide for line managers on supporting health and wellbeing.
• Will you measure the impact both quantitatively and qualitatively? A range of different metrics may offer a more robust measure. A quantitative measure looks at numerical indicators such as the number of contacts with an occupational health service. A qualitative measure focuses more on how staff feel about the usefulness of the support. Metrics can also be a mix of direct measures (staff in contact with the service) and indirect ones (staff absence levels). It is important to set realistic expectations on metrics, especially on indirect measures such as absenteeism, which may be affected by other factors. NHS Staff Survey health and wellbeing questions are an indirect metric for the overall perception of health and wellbeing support and can be supplemented by local health and wellbeing questions in the People Pulse survey. Focus groups can provide additional and richer data than surveys.
• Do you have the resource within your role/team to do this effectively? Make sure you are mapping out your evaluation activity relative to your capacity. 
• Surveys should be relatively easy to analyse if standard tools are used. Focus groups will be more intensive. You may be able to find additional support by linking evaluations to student projects but would need to balance your needs and those of the student. Collaborations between organisations on evaluation may also help and some ICS may be developing evaluation capacity to support initiatives on priority issues such as violence reduction.

Your health and wellbeing strategy may involve a range of tasks, activities or initiatives, for example, a toolkit, staff physiotherapy service and a health and wellbeing roadshow. Together these contribute to the overall objectives of the programme. 

You may choose to define specific initiative objectives within your evaluation plan, or for large programmes you may have individual evaluation plans for specific activities within the overall programme evaluation plan.

When setting activity objectives, the same rules apply as for programme objectives. You should make sure they fit within the context of the overall programme objectives and ensure you aim for consistency for similar types of activity or product.

NHS England has produced a range of materials on evaluation which you may find useful and NHS Employers has also identified the evidence base for some widely used approaches. It is still important to evaluate implementation.

Identify target audiences

It is really important to identify all of your target audiences. Are you trying to engage all staff or specific staff groups? Do you have different audiences for different interventions? Identifying your audiences will help you tailor your interventions and measure how successful they have been.

Interventions can be universal for all staff or targeted. If you decide to target the intervention, review the evidence to assess where it will have the most impact. Do you have any data on the health status of staff? The Model Hospital site can give you benchmark information on relative absence levels, whilst local level absence data can provide information on absence by area and occupation. Look at the equality dimension to see which groups would benefit most from an intervention. If an intervention is being offered universally, it is recommended to collect data on who is accessing it to ensure underserved groups get access. Review when and where it is offered and how it is promoted to ensure equity of access in practice.

Mapping tasks and activities

The next stage in creating your evaluation plan is to map your tasks and activities. This will help you demonstrate the full impact of your work by ensuring you choose the right performance measures to track and evidence impact. This is useful as we don't often include everything we do and the time it takes to run a health and wellbeing programme.

Begin to think about how to collect evaluation data and information as activities proceed:

• List everything you have done so far and how it has contributed towards your overall strategy.
• Develop a logic model or theory of change to help you clearly map out your future activities and intended impact.
• Consider how your objectives and activities are likely to influence behaviour.

Consider use of real-time data

A key finding from the 2021 NHS England report The Future of NHS Human Resources and Organisational Development highlights the need for real-time data. The findings suggest that while the NHS Staff Survey is very good for benchmarking scores and data, it does not provide a real-time, truly representative reflection.

Capturing frequent snapshots of health and wellbeing data could be hugely beneficial to help identify key individuals, teams, departments or cohorts that may need additional health and wellbeing support.

Examples include regular NHS People Pulse surveys to identify health and wellbeing information over time and online tools that enable tracking of how staff are feeling at intervals.

Real-time data can help measure staff perception and can also be an indicator of behaviour change. Behaviour change is likely to take some time and may take three to six months to become embedded. Readily available technology such as health fitness trackers that monitor and measure heart rate and steps can provide more feedback on health status than has been the case in the past. Sentiment tracking is now possible through easy-to-use technology. But it is only useful if you have the capacity to assess it and respond.

Define performance measures

When looking at what to measure to build your evaluation, it is helpful to think about the following:

• Inputs – these are the activities that you will be doing, for instance, how many roadshows are you running?
• Outputs - these are the number of people who have engaged in or been exposed to your intervention, for instance, web hits on your intranet or attendees.
• Out-takes – this means the immediate impact on the audience after your intervention. This could be the number of people who intend to change as a result of your intervention, for instance, a health check or a mindfulness session.
• Changed behaviours – this is how many people have changed something as a result of the intervention for instance, in the health check example, has the person made changes to their lifestyle as a result of the conversation?

The NHS Health and Wellbeing Framework implementation guide demonstrates the factors you may want to consider as part of your evaluation framework.

There are recognised measures of mental health and wellbeing that can be used as useful metrics to gauge the impact of interventions. For example, the Office of National Statistics has standard measures of overall wellbeing, and the NHS Staff Survey has standardised questions on burnout and stress. There are also widely used health and wellbeing status surveys and questions used in a large-scale health and wellbeing survey undertaken by RAND corporation in the NHS: The Healthiest Workplace survey.

Identify data sources

How will you gather data? Will you create a survey or measure the impact indirectly on overall health and wellbeing data? Do you already have access to this data?

Some key places to look for data are:

• NHS Staff Survey
• NHS People Pulse survey
• NHS England Model Hospital
• sickness absence
• data dashboards - such as the NHS health and wellbeing framework diagnostic tool
• occupational health referrals
• employee assistance programmes.

You could also try and collect your own data where resource and capacity allows from interviews and/or focus groups.

Or you could get external support to gain deeper insight from pre-existing resources, for example, using in-depth health and wellbeing surveys like Britain’s Healthiest Workplace which is used by some NHS trusts. External support can offer a level of expertise (expert data analysis techniques) and independence that internal support may not be able to.

Risks and constraints

For the final step in your evaluation planning, it's important to review and consider the following questions:

• Will budget be a problem?
• Do you have all the resources you need?
• Will people get survey fatigue?
• Do you have enough time to do everything?
• What do you need to prioritise?

You can then start to implement your evaluation plan.

Once you have gathered your findings, it’s important to share these with your colleagues and act on them. It can be a great way of demonstrating the value of your interventions and deciding where to invest money in new initiatives.

Furthermore, demonstrating that you have taken into consideration the feedback from your sources can motivate them to give more feedback in the future.

Your organisation may want you to be able to show a Return on Investment (ROI). This would require you to show that the cost of the intervention has resulted in a reduction in spending elsewhere in the organisation, for example by demonstrating that it has helped an individual return to work sooner than expected. You can also show an ROI if overall interventions help reduce absence and therefore save money on costly bank or agency spending. It is hard to demonstrate ROI in advance, although some widely used interventions can show average ROI.

Further resources to guide your evaluation

• In partnership with RAND Europe, NHS Employers ran evaluation workshops to support health and wellbeing leads in their roles and help them understand how to effectively evaluate their strategies or initiatives. See a recording from a workshop and view the slides. 
• Evolve Workplace Wellbeing website has free, evidence-informed workplace wellbeing resources developed and curated by the Workplace Wellbeing team at University of East Anglia and RAND Europe. There is also a free cost effectiveness calculator to help make a business case for workplace wellbeing initiatives.
• The developing and evaluating workplace health interventions: employer toolkit, developed by Public Health England and Northumbria Healthcare NHS Foundation Trust, sets out a four-stage strategy for developing and evaluating workplace interventions.
• The What Works Centre for Wellbeing provides a practical, eight-step guide for how to evaluate wellbeing interventions. The guide was designed for people who may have lots of different functions in an organisation, with busy working days. The guide allows you to dip in and out of the content as needed, at your own pace.
• The What Works Centre for Wellbeing in partnership with the Economic and Social Research Council,  University of East Anglia and University of Sheffield have created a cost effective calculator and how to guide to  help you evaluate and compare cost effectiveness of workplace wellbeing activities.
• The What Works Centre for Wellbeing hosts a wellbeing data usage library which aims to increase the accessibility of wellbeing data for researchers by speeding up dataset extraction. In his blog, Interim Head of Evidence at What Works Centre for Wellbeing and Professor of Public Policy at KCL Michael Sanders explains more.
• The Society of Occupational Medicine (SOM) Evaluating and supporting Neurodifferences at work guide is aimed at occupational health (OH) practitioners, human resources professionals and employers who are considering referring their staff for a diagnostic assessment of or services to support ADHD, Autism, Dyslexia, Dyspraxia, Tourette's Syndrome and/or similar. It outlines what to look out for in staff, different options available for support and legal duties of employers. The guide presents recommendations informed by research evidence, the latest guidance from regulatory bodies, current practice and case law.