It is important to know whether the training that you have given to a team has been effective or not, right? Is your audience getting enough out of your training? And most importantly, does your training have a positive impact on their role and on the organization? These are the kinds of questions that are well answered with the help of Kirkpatrick's 4-Level Training Evaluation Model. That could be a huge sigh of relief for you because for most people, calculating the effectiveness and ROI of training is often a daunting task.
The four levels of Kirkpatrick Model
For the first time, Donald Kirkpatrick got his model published in 1959, during the time he was a professor at University of Wisconsin. The first update to that came in 1975 and then in 1993 which included his most renowned contribution, “Evaluating Training Programs”. The interesting thing about the Kirkpatrick Model is every level offers a more accurate measure of effectiveness of training. Initially, Donald with his son James prepared the model. Later on, it was developed by Donald and Wendy Kirkpatrick, his wife.
It wasn’t until 2016 when James and his mother Wendy reviewed and updated the actual theory by introducing “New World Kirkpatrick Model” as an update. The book is called Four Levels of Training Evaluation. One notable change is the focus on significance of making the training applicable to everyone’s day jobs nowadays.
Let’s take a detailed look at all the four levels namely: Reaction, Learning, Behavior and Results.
Reaction (Level 1):
What’s the number one goal of your training? Most probably it’s that you want them to know that the training is useful. On a simpler level, there are three metrics that help you decide how well they received it:
- How engaged the audience was during your training?
- How much contribution did they put in and how actively did they do that?
- How was their overall reaction to your training?
Not only you can get feedback on the training, it would also help you greatly improve your next training. This is because you can easily identify significant topics that you might have missed by any chance. There are a lot of questions that you can ask from the audience. Such as:
- Do you think the training was successful?
- Would you recommend this training to others in your unit?
- If anything, what would you change about this class, and why?
- In your opinions, what are the strong and weak points of this class?
- Did the training activities/exercises engage you?
- What could be the 3 most valuable stuff that you learned by attending this training?
- How good/bad were the venue?
- How god/bad was the presentation style, in your opinion?
- From this training program, how much do you think you can apply at your job?
- How much and what kind of support might you need when applying the learned knowledge at your job?
Learning (Level 2):
This is a level that emphasizes what the trainees have or have not learned from your training. There is one new and innovative addition in the New World version. It also measures how the trainees think they can do things differently, how confidently they feel in that aspect and what the source of their motivation is. As a result, you can assess how the training has imparted skills and knowledge to them, their self-confidence and determination.
The first step in line is to identify what you want to evaluate. This will streamline the process of measuring how much the trainees have learned. Ensure that there are enough learning objectives in your training, that the they are defined in a measurable way.
Measuring learning effectively
Of course, there are a number of ways to measure learning and it would depend on the objectives. However, as a rule of thumb, it is good to evaluate these areas before and after the training. A good way to do that is to have a conversation or quick question answer session with your trainees. This would help you assess their knowledge level, attitudes and skill level as well.
Behavior (Level 3):
This is a really important level which helps you comprehend how well people make of their training. In other words, how much they apply the training they have received. Another benefit of this level is that it explores which parts the trainees may need assistance. However, for the behavior to change, the conditions have to be favorable.
Let’s elaborate this part in more detail. Imagine a classroom where you are giving a training to your colleagues. You would assess them afterwards, right? In one scenario, it is quite possible that they show little change, helping you reach a conclusion that the people learned almost nothing. This renders your training ineffective.
Behavioral change
In another scenario, it is also likely that the colleagues of yours learned a lot. It might be the case that they learned but the company or organizational culture hinders behavioral change. With the help of existing processes, you may assess there is little to no room to apply new knowledge. What happens as a result? People don’t feel comfortable when they are in a position to use the new knowledge or see few chances to do so. In some cases, people may not even find sufficient time to put the new skills to work.
Required drivers
If you are facing such negative factors and developmental blocks, Kirkpatrick model presents a solution. Focus on developing processes that boost, encourage and reward positive behavioral changes. Such processes, as per New Kirkpatrick Model, are known as “required drivers”. Simply put, if a colleague or trainee uses the newly learned skill effectively, highlight them and appreciate them for it.
- It is to be noted that efficiently measuring behavior is a long process that demands patience. Our recommendation is to do it three months after the training. You want to do it as early as possible but not too early sine the trainee’s need to have had the opportunity to apply their learning as well.
Conducting observations and interviews are two of the most effective ways to measure behavior. However, since they are quite resource demanding only about 15% of those we speak with are doing it. Using a digital tool that automates the evaluation can reduce the workload and release the resources to work with e.g. continuous improvement of the training portfolio
Results (Level 4):
This part is usually regarded as the primary goal of the program. At this level, you determine the overall success of the training based on a number of critical business objectives. These include decreased spending, greater ROI (return on investments), better quality of products and more work efficiency in employees etc.
It is important to note that the level 4 is definitely the most time consuming and costly process. One of the trickiest challenges is to predict which advantages, outcomes and results are associated with the training. Then you also have to think of an effective way to evaluate the outcomes in the longer run.
Using Kirkpatrick model in reverse order
It is interesting to see how the modern trainers use the Kirkpatrick model in the reverse order. They first define the final results, breaks this down to behavior that will drive the results and finally break those down to learning objectives that will drive . This help in prioritizing the goals and ultimately, the training becomes more effective.
Last but not the least, all the results would not mean a thing if not communicated to the stakeholders.
There is a list of outcomes that one may consider, as long as they are relevant to the training objectives:
- Higher level of motivation
- Reduced work lags and procrastination
- More sales
- Higher level of customer service/customer satisfaction
- Greater and more efficient production
- Higher workers retention and reduced workers complaints
Measuring results (level 4) might take long time. Perhaps you’ve already managed to run all your employees through training before you realize nothing happens with the results. So, make sure to measure and track behavioral changes (level 3) as well. This will give you more prompt indicators if you’re, at least creating the behaviors that you believe will drive the results. You should even measure learning (level 2) since this will tell you whether you have succeeded creating the learning you believe will drive the behavior
Possible risks of Kirkpatrick’s Model
This model has remained very popular since its arrival and is now over 60 years old. However, there has been several updates to it and the way people learn and organizations work has changed the way it works. This is the primary reason why it should be used with care. For example, even the term “learning” has been changed to “learning and development”.
Tackling Real-World challenges of evaluating learning success with Kirkpatrick’s Model
There’s a few challenges of evaluating training effectiveness using Kirkpatrick’s model. Let’s take a look at the two main ones:
First Challenge:
As you know the four levels of Kirkpatrick model: Reaction, Learning, Behavior and Results. Just as you start applying the model to real-life scenarios and problems, you will observe that not all levels of evaluation are always required. Just as an example, sexual harassment training is almost always mandatory. This is mandatory regardless of the ROI, which is why you might want to disregard that level completely. In other scenarios, compliance training is also necessary in order to know that the trainees retain the information and also apply at work. In such scenarios, you should definitely focus on level 2 and 3.
Second Challenge:
The second challenge in today’s world is called “Choosing the right metrics” as required in entry level of this model. For example, emotional response can be measured by the “reaction”. Similarly, the level of information retention may be measured by the “learning level” and the other two levels measure how the new knowledge is transferred to everyday job and measuring ROI respectively. However, there may be hundreds of metrics to use among these 4 levels. It can be a difficult task to choose the right metrics and get the right result. The below tipscan help you choose the right metrics and ultimately get deeper insights:
- Asking the stakeholders about the metric is always a good idea. Although, the metric should drive business decisions and also determine the value of program for those who funded them.
- Check the relevancy of the metric and see if the performance indicators are inside the scope of the project.
- Carefully review how the metric can be connected to the key performance indicators (KPI).
- Always ensure that there is enough data available to draw conclusions. Data volume is important for data integrity. Needless to say, the more data you have for a metric, more likely you are to get accurate insights.
The key point, at the end, is to bring your training process out of the theory and into the reality of what data and evaluation offers. This would be of great help to your learning & development project. Also, you can make strong conclusions to assist the business decisions and of course, develop effective training programs.
