Don Kirkpatrick’s 1993 book, the Four Levels of Training Evaluation, revolutionized how businesses use and evaluate training programs. If you're looking for ways to measure the impact of corporate training and development, here is the complete Noob guide to understanding one of the world’s most popular learning evaluation frameworks, the Kirkpatrick Model.
Why evaluate training?
Ineffective training is a huge waste of time and money for organizations. Over five decades ago, Don Kirkpatrick devised his learning evaluation model to help measure whether a particular training has met its goals and to find ways to improve the experience for the learners. As all training materials can be constantly improved and refined, Kirkpatrick’s framework helps to continually improve training courses.
The evolution of the Kirkpatrick Learning Evaluation Model
if you’re just getting started with this evaluation system, it’s important to know that it has been revised and updated over the years.
Here’s the deal:
Over time, a number of faulty practices crept into the way people were using this model. Kirkpatrick’s son James and his wife Wendy noticed this problem and decided to co-author the updated book, Kirkpatrick's Four Levels of Training Evaluation, to help set the record straight. This book should be considered the definitive guide to using the system. It addresses many of the incorrect ways in which people have used and interpreted the system and how to do things correctly.
But it gets better:
You should also be aware that the model has been further updated and clarified and is now known as the New World Kirkpatrick Model. This is the current definitive blueprint for evaluating training and maximizing business results. It contains the latest advice and guidance as to how a company or organization can start using the model to evaluate their training.
What are the four levels of training evaluation?
Kirkpatrick’s Training Evaluation model has four levels:
-
Reaction
-
Learning
-
Behavior
-
Results
Here’s a brief guide to the four levels of evaluation:
Level 1: Reaction
The first level of evaluation is reaction. The trainees are asked to rate the instructor, the materials, the presentation, and the venue. This measures how favorably the trainees reacted to the training and gives you an idea of how the training was received and what you could do to improve the user experience.
Level 2: Learning
The second level is learning. This level measures what the trainees actually learned from the training. The trainer would measure these parameters against the learning objectives to ascertain whether the training was effective.
Level 3: Behavior
Level 3 is all about behavior. At this level, the trainer evaluates the trainees to see whether they have changed their behavior, based on the training, when they are back on the job. It’s quite common for trainees to learn the correct procedure for doing something and then go back to their jobs and continue doing things the same way that they used to. Kirkpatrick’s third level of training evaluation helps organizations avoid this by measuring to what extent the material presented during the training was used and implemented in the workplace.
Level 4: Results
Level 4 is results. At this level, the trainer analyzes the final results for the training by measuring the outcomes. These outcomes are measured against the criteria that the organization has determined will be good for business, the employees and for the bottom line.
How to implement the Kirkpatrick Evaluation Model
As a newcomer, implementing the Kirkpatrick Evaluation Model can seem daunting. However, the system is so well established and widely adopted that many organizations routinely conduct similar forms of evaluation, even if they don’t knowingly follow the Kirkpatrick model.
That’s right:
You may already be familiar with many of the types of assessments used in the Kirkpatrick model.
Here, we’ll explain everything you need to know about implementing the Kirkpatrick model in your business or organization. If you're pressed for time, you may like to check out our brief guide to mastering the Kirkpatrick Model in six simple steps.
How to implement a Level 1 evaluation
The simplest way to start evaluating your training is to ask trainees to complete a post-training questionnaire and analyze their responses. Ask them how they found the course and try to find recurring themes in their answers.
For example, if you ran a training course that aimed to improve health and safety in your company, you could ask your staff to complete a questionnaire. Popular online tools such as SurveyMonkey make this simple, and free, to conduct.
Following the Kirkpatrick system, questions you may like to ask your staff could include:
-
How helpful did you find the training?
-
How receptive was the instructor to your needs?
-
How clear were the training materials?
-
How happy were you with the speed of the course?
-
How likely would you be to recommend this course of a colleague?
The respondent could rate each question on a sliding scale from 0 to 10.
You could then collect the responses and try to draw conclusions from the given answers. By finding similarities or common threads in the responses, you may be able to determine which aspect of the training worked well and which area needs improvement.
What does this look like in practice?
If, for example, many participants of a particular training sessions indicated that they didn’t feel the instructor was receptive to their needs, you may need to re-train the instructor or hire a new one for future training sessions.
Similarly, if many trainees indicated that they didn’t feel the course was helpful, you may need to revise the materials covered and find ways of making it more helpful and relevant.
How to implement a Level 2 evaluation
Level 2 is perhaps the most straightforward type of evolution. A common approach is to assess the trainee’s knowledge and understanding with two quizzes; one before and one after the training. If participants can correctly answer more questions after the course, this suggests that learning took place and that the training was effective.
To implement Level 2 evaluation, most organizations rely on the instructor to administer the quizzes. Alternatively, you can email the quizzes to the trainees so that they can complete them in their own time.
Here’s the kicker:
One limitation of Level 2 evaluation is that quizzes taken immediately after training may give overly positive results with regards to how much information the trainees have remembered. It’s common for participants to score very highly in post-training tests, only to quickly forget the information soon afterward.
To address this, you could implement a follow-up quiz a while after the training has finished. This helps you identify how much of the training material has been retained. The results from this quiz may help you decide whether to offer other ways to help cement the learning that took place. For example, you could offer a refresher course or send daily messages or emails to help the trainees keep the knowledge fresh in their minds. We covered this in more detail in our post about people's expectations vs. reality when they start using the Kirkpatrick model.
How to implement a Level 3 evaluation
Level 3 evaluation is one of the most difficult aspects of implementing the Kirkpatrick Training Evaluation Model. New behaviors take time to settle in and be adopted. It’s very common for staff to learn new techniques or gain new knowledge during training and then fail to use what they’ve learned to improve their work. For a variety of reasons, the tools and techniques introduced during the training never make it to the workplace and are therefore wasted.
Reasons could include:
-
The supervisors or managers prevent staff from working in new ways.
-
Tools or resources aren’t available for staff to use.
-
The staff doesn’t fully understand the content of the training (this will be identified during level two evaluation).
-
The training was too radical or difficult to adopt.
To properly implement Level 3 evaluation, you should wait a while after the training has finished. One of the most helpful approaches is to use 360 Degree feedback. This type of evaluation asks the trainee, their colleagues and their superiors for feedback as to whether their behavior has changed after the training.
Let’s say your organization conducted training to introduce staff to a new piece of productivity software that helped them go paperless, such as Evernote. The goal was to reduce the amount of paper being used in the office and to get the staff used to scanning and uploading notes, documents and files to a cloud-based provider such as Google Drive or Dropbox.
To conduct 360 Degree feedback, you would want to evaluate multiple members of your organization from different levels and different departments.
Here are some suggestions:
-
You could ask an employee’s supervisor whether their paperless communications such as notes and documents had increased, decreased or stayed the same since the training.
-
You could survey an employee’s colleagues about their habits. Was their colleague scanning more notes?
-
The resource department may be able to give you data about a department’s paper usage. A decrease would indicate that a member of staff was scanning more documents and using less paper following the training.
This approach gives you a full picture of the situation, including any reasons why changes may not be apparent. If your evaluation indicated that a trainee wasn’t using less paper, 360 Degree feedback would flag up the reasons.
For instance:
A boss could be preventing their employees from applying their new knowledge, by insisting on the use of paper documents.
There could be a missing tool or system that isn’t in place, perhaps the staff lack access to scanners or the mobile devices necessary to use the new app.
Level 3 will help you identify issues and remedy the situation. If you're looking for more help on ways to implement a Level 3 evaluation, check out our recent post about Kirkpatrick Level 3 - Free Evaluation Examples.
How to implement a Level 4 evaluation
To implement this level of evaluation you need a clear understanding of how success is defined. What changes are the managers hoping to see? You need to start with a clear definition of success and then work backward to determine whether the training has been successful.
Imagine that you’re aiming to reduce the number and frequency of workplace accidents and have run training with the aim of addressing this. What would success look like?
Fewer accidents?
Fewer serious accidents?
Faster response times?
The most common way to get started is to collect and analyze data that is relevant to your goals.
To conduct level 4 evaluation, you’d need to collect data from any accidents or incidents both before and after the training. What does the data indicate? If there are fewer accidents after the training, this would suggest that the training had been effective in terms of reducing the number and frequency of accidents.
If accidents have increased since the training, you would need to dig deeper into the level 3 evaluation to pinpoint the reason. There could have been an increase in job difficulty or an increase in the amount of work that employees are being asked to carry out.
Another example of level 4 evaluation in action would be to consider a case where you are aiming to improve the return on investment (ROI) of your organization, or a specific department or project. One of the key pieces of data you’d need to collect would be the financial statements from your business or organization. What do the numbers show?
If profits have increased since the training, this may indicate that the training effectively boosted ROI. You could analyze the financial statements to pinpoint where the profits were made, or where expenditures were cut. These details would inform any future training that you offered and help to make it more effective. We explained how to measure your ROI using the Kirpatrick model in greater detail in a recent post.
Kirkpatrick’s learning evaluation model provides a route to more effective training
By following each of these four levels of evaluation, any trainer or organization can gain an understanding of how effective their training is and how it can be improved in the future. No training is set in stone and it can always be refined, modified and realigned to make it more effective. A training evaluation model such as the Kirkpatrick model will give you an overall perspective on any training course or program.
If the training didn’t meet expectations, you can find out what can be done to improve it in the future. If the training was successful, you can find ways to increase its impact even further and accentuate its benefits.
In this brief guide, we’ve covered the nuts and bolts of the Kirkpatrick learning evaluation model.
But there’s more!
If you want to go from noob to pro, consider downloading our free white paper about how to maximize business impact through training and development. This helpful resource will turn you into a Kirkpatrick whizz in no time!
So, what are you waiting for? Discover for yourself how to effectively implement the Kirkpatrick learning evaluation model today!
Related posts:
How to Master Kirkpatrick model of training evaluation in 6 Simple Steps
Kirkpatrick levels of evaluation: Expectation vs reality
Kirkpatrick level 3: Free evaluation examples
Kirkpatrick level 4: learn How to measure your ROI training