|
Types of Training Evaluation
There are many ways that managers and staff can evaluate training. This Family Planning Manager focuses on one level of impact evaluation—individual job performance as a result of training.
|
|
Type of Evaluation |
Purpose |
|
Needs assessment |
To identify the knowledge, attitudes, and skills (KAS) needed for acceptable job performance |
|
Baseline evaluation |
To determine the trainees’ levels of KAS before training |
|
Input evaluation |
To assess the elements associated with the training: costs, selection of trainers and trainees, curriculum plans, venue, materials |
|
Process evaluation |
To conduct assessments periodically during the training, and adapt the schedule, content, or approach accordingly |
|
Outcome evaluation |
To assess new or improved KAS after training |
|
Impact evaluation |
To determine the effect of the training on:
•individual job performance (on-the-job application of new knowledge, attitudes, and skills; work that meets or surpasses professional or organizational requirements)
•organizational performance (quality of services, client satisfaction)
•program performance (use of modern contraceptive methods, couple-years-of-protection)
•regional or national demographic and health indicators (decreases in birth, fertility, infant mortality, and maternal mortality rates) |
Using Internal Resources to Conduct the Evaluation
When organizations consider conducting an impact evaluation, they often turn to outside experts in the belief that only outsiders can provide the unbiased, objective expertise that produces credible results. But many managers have learned that external evaluation does not guarantee appropriate recommendations, partly because outsiders cannot always grasp the realities of the workplace and the related training needs of a particular organization in the short period of time that they are given to conduct the evaluation.
There is growing support in the evaluation community for participatory evaluation which allows stakeholders to assess their own achievements [Feuerstein], drawing on special external expertise (for example in sampling) only when necessary. In the training business, the key stakeholders include the managers and staff of the training organization or program, and the managers and staff of the organizations that send their staff to be trained. Evaluating a training program through a participatory process that includes representatives from both organizations has important benefits:
- Internal evaluation costs less than hiring outside experts. As funding agencies cut back on their support, organizations will have to draw on their own funds for evaluation.
- When trainers participate in impact evaluations, they become familiar with their trainees’ working conditions. Training organizations can also use the results from the TIE to design future training courses.
- TIE can be incorporated into routine supervision and monitoring. Service delivery organizations can institute regular, focused, small-scale evaluations to explore reports of problems encountered in the field or to confirm that specific elements of a course have succeeded and should be duplicated in other courses.
- Organizations that send staff for training gain a better understanding of what training can and cannot do. They can use this information to clarify their expectations for future training events.
Forming an Internal Evaluation Team
The evaluation team should include representatives from the training organization or training program and the service delivery organization (or organization receiving the training). If possible, one or more trainees should be on the team so that their perspective and experience can be taken into account during the process. If you decide to include an expert whose skills are considered essential to a particular aspect of the TIE process (for example, a statistician), that person’s role and responsibilities should be clearly spelled out to the satisfaction of every team member. This clarification will ensure that the process remains in the control of the participating organizations.
This cross-organizational team is particularly valuable when the evaluation uncovers the need for improvements in either organization. If both the training and service delivery organizations have participated in the entire TIE process and agreed on the objectives and eventual recommendations, they will be far more likely to accept the suggestions and make worthwhile changes. If time or distance do not allow a close working relationship between the two organizations, either organization may have to conduct the TIE on its own. This presents some risk in that the non-participating organization might not fully accept the findings and recommendations. But if every step of the process is well-documented and the other organization is kept informed, the risk can be minimized.
|
