Part 5: Measuring and Evaluating
the Effectiveness of the Training
The Commission could not set performance targets
for the project
5.1 The Electoral Commission did not set any specific
performance targets for the training of electoral staff for the
European Parliamentary Elections. Part of the reason for this
is that performance management has not featured strongly in the
culture of electoral services departments of local authorities.
There is a lack of robust performance data on the management and
control of electoral processes and so it is difficult to set measurable
specific performance targets.
The Commission decided to evaluate progress through
a report on the whole project
5.2 In June 2004, the Electoral Commission commissioned
Deloitte and Touche LLP (Deloitte) to carry out an evaluation
of the training project[17].
The Deloitte report assessed whether the Electoral Commission
achieved its four key training objectives to ensure that:
- professional elections staff
based in local authorities across Great Britain and in the Electoral
Office for Northern Ireland were provided with clear guidance
on the procedures and legal framework for managing the elections,
including substantial new requirements arising from legislation
and case law since the last European Parliamentary elections;
- polling station staff were equipped to deal accurately
with the most common queries from electors and understand their
duties. This should include providing polling staff with the necessary
knowledge to explain concisely the specific voting system used
in the election;
- all staff involved had a clear understanding
of the equal access and 'quality of service' principles that should
underpin the management of the election process; and
- all training provided was carefully targeted,
high quality, produced on time and disseminated effectively.
The project largely met its four key objectives
5.3 The Deloitte Report concluded that the training
project largely delivered on its four key objectives. However
there were concerns about the level of understanding of the contractors.
The main findings of the Deloitte Report are set out in Figure
5.
Training materials were of high quality but could
not be reused
5.4 Polling station staff received training in a
number of different ways, with presentations being the most popular
method (see Figure 6).
5.5 Overall the training materials were of a high
quality and electoral staff considered the content to be accurate
and with the right tone. More than 90 per cent of polling staff
who used the materials found them useful or very useful although
Deloitte noted some electoral administrators had felt overwhelmed
by the amount of choice.
5.6 The Electoral Commission intended to use a modular
approach to training where electoral staff could identify areas
of learning need and avoid repetition of areas where they already
had expertise. Although the learning materials were divided into
modules, these 'modules' represented sub-headings or sub-sections,
meaning electoral staff could not easily identify training modules
that would address their individual learning needs.
5.7 Electoral staff provided a mixed response as
to the reusability of training materials. The Deloitte report
examined them in detail and found that a significant amount of
work would be required to adapt these materials for future training
use. Project training contractors were not primarily concerned
with reusability during the creation of the materials.
There was a broad difference between what was
expected by the Regional Returning Officers at the training seminars
and what was delivered
5.8 As part of the contract requirement, Solace Enterprises
Limited commissioned Insight Social Research Limited (Insight)
to evaluate whether the initial training seminars met the objectives
of the project. Insight distributed questionnaires to all electoral
staff who participated in the training seminars and to a sample
of polling station staff who received training through the cascade
process. Insight also interviewed voters in traditional and all
postal ballots, and representatives from political parties (agents
and candidates). The Insight report evaluated the feedback received
from both electoral staff and voters, and the Deloitte report
used the research from the Insight report as one of the evidence
streams.
5.9 There was a broad difference between what was
expected by the Regional Returning Officers and what was delivered
at the one national seminar in March 2004. This difference may
have been because Regional Returning Officers were not clear whether
the seminar was providing training or information on training.
Solace Enterprises did not make the objectives of the seminar
clear to the Regional Returning Officers.
5.10 The Insight report found that Regional Returning
Officers were generally happy with the arrangements for the training
seminar held in March 2004, but not with the content of the training
provided by Solace Enterprises for the Electoral Commission. Ninety
per cent of Regional Returning Officers felt that their expectations
were not met and 84 per cent felt that the content of the training
was incomplete. Solace Enterprises did not prepare the training
materials on time but they were dispatched by the middle of May
2004, less than one month before the elections. Figure 7
provides a breakdown of the feedback from Local Returning Officers
and Electoral Administrators on the delivery of the initial training
seminars for both traditional and all-postal voting.
There were regional differences in the responses
of Local Returning Officers and Electoral Administrators to the
training seminars
5.11 As with the Regional Returning Officers' training,
Solace Enterprises did not prepare the training materials in time
to be ready before the initial seminars were held. Local Returning
Officers and Electoral Administrators were more positive about
the training itself, but fewer than fifty percent of respondents
considered delivery of the training seminars to be good or very
good. More Local Returning Officers and Electoral Administrators
who received training in the all-postal regions found the training
seminars to be good or very good compared to those in the traditional
voting regions.
Feedback from Polling Station Staff showed that
training had contributed to the service provided to voters
5.12 Polling Station staff (Presiding Officers and
Poll Clerks) received training through the cascade approach, so
local training sessions depended on the ability and knowledge
of those delivering the training. The Electoral Commission did
not provide any 'train the trainer' sessions.
5.13 Local trainers provided training sessions over
a one month period from the beginning of May to the beginning
of June 2004. Insight asked whether respondents considered the
training had contributed to the service provided to voters. Fifty
four per cent considered that the training had contributed a lot.
Seventy eight per cent considered that their own previous experience
had contributed. The respondents felt that there should have been
support from a project team member who had experience in the electoral
processes.
5.14 The training materials were not ready in time
for some of the training sessions, which were held up to a month
before the election. Electoral staff indicated that they would
have preferred to go through the materials themselves and identify
how to use them for local training. It was also felt that the
training sessions should have been divided to target those with
no experience and those with many years experience of elections.
London Elects' approach demonstrated good practice
and should be replicated
5.15 Running concurrently with the National training
project was a project partly funded by the Commission in the London
region. This was to be delivered by the Greater London Returning
Officer (London Elects) in conjunction with training for London
mayoral and Assembly elections. This reflects the different role
of the Greater London Returning Officer in delivering London elections
and the differing level of funding obtained.
5.16 The Electoral Commission provided London Elects
with £70,000 towards training electoral staff for the elections
held in London in June 2004. Deloitte's evaluation noted the following
examples of best practice:
- London Elects consulted with
experienced AEA trainers employed by London authorities and other
stakeholders to identify a list of training needs. Using this
list, London Elects put together a training programme and launched
it at an Away Day for senior electoral staff from a selection
of London boroughs. This programme was passed to the Electoral
Commission for approval.
- When developing the training materials, London
Elects considered the differing requirements of each London borough
and need for flexible learning materials plus any skills gaps
using feedback from the electoral community.
- External consultancy provided training skills
and "train the trainer" sessions, which helped to boost
the confidence of those "cascading" the training later.
- Trainees were provided with a 'toolkit' and handbook
for the purpose of delivering the training to the remaining electoral
staff in their areas. The 'train the trainer' sessions gave trainees
the chance to go through the toolkit materials and structure their
own local courses.
3 
