Training is one of the best investments your organization can make. It can drive productivity, boost revenue and help you meet your strategic goals. But telling the difference between effective training and total flops is far from easy. That’s why using a training evaluation model can be helpful. With a multitude of such models to choose from, which one should you choose?
Here at Kodosurvey, we believe that there are many valuable approaches to training evaluation. Instead of there being a ‘right’ or ‘wrong’ approach to take, we recognize that every organization has its own unique set of circumstances, options and limitations. That’s why we wanted to put together this complete guide to training evaluation models and give a broad overview of the available options.
In this complete guide to training evaluation models, we’ll give you an overview of some of the best frameworks in use today. We’ll highlight the similarities and differences between these models and illustrate the benefits and disadvantages of each approach. Hopefully, this will help you choose the right approach for your organization.
Contents
1. What are training evaluation models?
2. The Kirkpatrick Model
3. The CIRO Model
4. The Phillips ROI Model
5. The Brinkerhoff model
6. Kaufman's Model of Learning Evaluation
7. Anderson Model of Learning Evaluation
1. What are training evaluation models?
Training evaluation models are systematic frameworks for investigating and analyzing the effectiveness of training or learning journeys. Different models target different things but in general, they look at things such as:
- Was the training successful?
- What did the participants learn?
- Did the participants use what they learned on-the-job?
- What was the impact on the organization?
- Was the training a good investment?
- Did the training offer value for money?
- Could the training be improved?
As we’ll see, each training evaluation model takes a slightly different approach and may or may not answer some or all of these questions.
2. The Kirkpatrick Model
The Kirkpatrick Model is by far the most popular and widely-used training evaluation model in use today. It was developed and introduced by Don Kirkpatrick in 1959 through a series of articles that were published in the Journal of the ASTD. Kirkpatrick’s innovative model provided a way for any organization to evaluate any course or training program with ease. The model quickly achieved global recognition and its use became widespread among organizations of all sizes throughout the 1970s and 1980s.
Kirkpatrick continued to refine his model, leading to the publication of Evaluating Training Programs: The Four Levels, in 1993. This book represented the first time that a training evaluation model had been made available in an easy-to-use reference guide, and use of Kirkpatrick’s model grew rapidly.
The four levels of the Kirkpatrick model are:
Level 1: Reaction
Level 2: Learning
Level 3: Behavior
Level 4: Results
Here’s how each level works:
Level 1: Reaction
This level helps you determine how the participants responded to the training. This helps identify whether the conditions for learning were present in the training.
Level 2: Learning
Through the use of short quizzes or practical tests, this stage helps you determine what, if anything, the participants learned from the training.
Level 3: Behavior
The third stage takes place a while after the training has finished. Using various assessment methods, you try to assess whether the course participants put what they learned into practice on-the-job.
Level 4: Results
The Kirkpatrick model measures results against stakeholder’s expectations. This is known as ROE (Return on Expectations).
In summary, the Kirkpatrick Model is the most common training evaluation model in use worldwide today. As we’ll see, it has provided the base for many other training evaluation models.
3. The CIRO Model
In 1970, Peter Warr, Michael Bird, and Neil Rackham and published their book, Evaluation of management training. Their framework for evaluating training became known as the ‘CIRO model’ and offers businesses a way of evaluating training needs and results.
Unlike other models such as Kirkpatrick’s Model that can be applied to a broad range of training and evaluation programs, the CIRO model is specifically aimed at evaluating management training.
CIRO is an acronym that stands for the four levels which make up this approach to learning evaluation. They are:
- Context
- Input
- Reaction
- Output
The CIRO model is hierarchical, meaning that practitioners must start by studying ‘Context’, before moving through ‘Input’, ‘Reaction’ and ‘Output’.
Here’s a brief overview of how it works:
Stage 1: Context Evaluation
In the CIRO Model, you must first collect performance-deficiency information, ie. what is the organization lacking? This stage assesses the operational situation that an organization finds itself in.
Stage 2: Input Evaluation
During the second stage of the CIRO Model, practitioners must gather information about possible training techniques and methods.
Stage 3: Reaction Evaluation
The third stage of the CIRO Model involves gathering participant views and recording any suggestions they make about the training they received.
Stage 4: Outcome
This stage of the CIRO Model involves presenting information about the results of the training.
The CIRO Model is a practical way of evaluating management training and has found favor with many organizations the world over.
4. The Phillips ROI Model
As the Kirkpatrick Model grew in popularity during the 1970s, many academics and business practitioners wanted to build and expand on it. Among them was Jack Phillips, who published his own book, Return on Investment in Training and Performance, in 1980. Phillips wanted to build on Don Kirkpatrick’s work and address several of what he perceived to be its shortcomings. Over time, Phillip’s approach to training evaluation became known as the Phillips ROI Model.
The most commonly quoted aspects of the Phillips ROI Model is the addition of a fifth level. This expands upon the Kirkpatrick Model and offers organizations a way of calculating the ROI of their training. The Phillips model has five levels that broadly follow the scope and sequence of the Kirkpatrick model.
The five levels of the Phillips ROI Methodology are as follows:
Level 1: Reaction
In common with the Kirkpatrick Model, the Phillips ROI Model begins by evaluating the participants’ reaction to the training they received.
Level 2: Learning
The second level of the Philips ROI Model evaluates what, if any, learning took place during the training.
Level 3: Application and Implementation
Like the Kirkpatrick Model, the Phillips ROI Model looks at whether the participants used what they learned during the training when they returned to the workplace. However, Phillips approach helps an organization determine whether an issue (if there is one) lies with the application of the learning or its implementation. This represents a subtle but crucial improvement over the Kirkpatrick Model.
Level 4: Impact
While the fourth level of the Kirkpatrick taxonomy focuses purely on results, the Phillips ROI model is much broader and looks at the impact of the training. This helps identify whether factors other than training were responsible for delivering the outcomes.
Level 5: Return on investment (ROI
Unlike the Kirkpatrick Model that simply measures training results again stakeholder expectations (ROE), the Phillips ROI model contains a fifth level. This is designed to measure ‘return on investment’, or ROI. This level uses cost-benefit analysis to determine the value of training programs.
The Phillips ROI Model found favor with organizations who wanted to assign a monetary value to the results of the training. Given the cost and complexity of applying an ROI analysis, however, only around five to ten percent of training courses actually require a full, five-level evaluation.
5. The Brinkerhoff model
In 2003, Robert O. Brinkerhoff introduced an entirely new way of evaluating training effectiveness, called the Success Case Method (SCM). The SCM is a methodology that helps an organization understand how a training or coaching program works well, or why it is not working. However, there are two big differences between the SCM and other training evaluation programs.
Firstly, the SCM isn’t simply limited to evaluating training and can be applied to a wide variety of events or activities. For example, it could be applied to a new type of machinery or a new facility in which a company has invested.
Secondly, unlike other training evaluation models (such as the Kirkpatrick Model) SCM isn’t concerned with finding the average performance of training participants. Instead, it deliberately studies the most successful participants and the least successful ones. In other words, it looks at the extreme cases.
The two most important questions asked by the SCM are:
- “How well does a program work in a best-case scenario?”
- “When a program doesn’t work, what’s the reason for this?”
The SCM has five different steps that an organization must go through to evaluate a training course.
- Plan a Success Case study.
- Write an “Impact Model” that defines what success should look like.
- Write a survey that identifies best-case and worst-case scenarios.
- Document success cases and conduct interviews.
- Draw conclusions, make recommendations and communicate findings to stakeholders.
As we’ve noted, what sets SCM apart from all other training evaluation models is that it can be applied to a range of situations and events. In general, the SCM is simpler and cheaper to apply than other types of training evaluation models such as the Kirkpatrick Model, adding greatly to its appeal.
6. Kaufman's Model of Learning Evaluation
Roger Kaufman and John M. Keller published Levels of evaluation: Beyond Kirkpatrick in the winter 1994 edition of Human Resource Development Quarterly. This work became known as Kaufman’s Five Levels of Evaluation and is commonly referred to as Kaufman’s Model of Learning Evaluation.
Like the Phillips ROI Model, Kaufman’s model was closely based on the Kirkpatrick Model and made a few changes and modifications.
These are:
- Kaufman divided Kirkpatrick’s Level 1 (Reaction) into two parts: 1a, “Input”, and 1b, “Process.”
- Kaufman introduced a fifth level that evaluates the impact of the training on both the organization’s customers and on society.
Kaufman’s Model takes the following approach:
- Level 1a: Input
Were the training resources and materials suitable and appropriate?
- Level 1b: Process
Was the training well delivered?
- Level 2: Acquisition
To what extent did participants acquire new knowledge and skills?
- Level 3: Application
To what degree did participants utilize their learning or training in their on-the-job roles?
- Level 4: Organizational Results
In what ways did the organization benefit from the training?
- Level 5: Societal/customer consequences
How did the training impact on the organization’s customers and society at large?
Kaufman’s model is often praised for separating ‘input’ and ‘process’ as this makes it simpler to assess whether the training materials or the delivery were the cause of the success or failure of a training course. However, the fifth level of Kaufman’s Model – Societal/customer consequence – is often deemed infeasible to implement for the majority of businesses.
7. Anderson Model of Learning Evaluation
The last training evaluation model that we’ll discuss is the Anderson model of learning evaluation. This unique model was first published by the Chartered Institute of Personnel and Development in 2006 as the ‘Anderson’s Value of Learning Model’. This model differs from other training evaluation models in two significant respects.
Firstly, the primary focus of the Value of Learning Model is on aligning an organization’s training programs with its strategic priorities. Whereas other training evaluation models focus closely on the learner and whether they derived benefit from the training, the Anderson Model is designed to be implemented at a management level and cover the whole organization.
Secondly, this model is unique as it doesn’t focus on the outcome of individual programs. Instead, it looks at an organization's strategic goals and examines whether the training programs are aligned or whether they should shift focus.
Anderson’s model is a three-stage cycle that helps an organization determine the best training strategy for their needs.
The three stages are:
Stage 1: Determine the current alignment of training against strategic priorities for the organization.
Stage 2: Use a range of methods to assess and evaluate the contribution of learning.
Stage 3: Establish the most relevant approaches for your organization.
The Anderson model of learning evaluation is harder to compare and contrast with other types of training evaluation models as it takes such a unique approach. While the cost and complexity of data collection are lower using this model, the second stage – evaluation of learning – still requires care and expertise to implement effectively.
8. Conclusion
As we’ve seen, there is a wide range of training evaluation models available, each with their own unique advantages and disadvantages. One crucial point is that none of these models need to be implemented in their entirety; they can always be adapted to suit the needs and requirements of a specific organization. With careful consideration, an organization can find the most suitable approach for their situation and circumstances.