One of the most challenging aspects of any training process is the task of evaluating the effectiveness of the training. You have to determine to what extent the training met not only your own goals but those of the management and shareholders too. One of the most widely-used evaluation methodologies is the Kirkpatrick Model, introduced by Don Kirkpatrick in 1959. In this post, we’ll show you exactly how to master the Kirkpatrick Model in six simple steps.
Step #1: Learn about or review the Four Kirkpatrick Levels
The first step to mastering the Kirkpatrick model is to learn about the four levels of evaluation.
These are:
-
Reaction
-
Learning
-
Behavior
-
Results
While these levels have remained the same over the years, people haven’t always applied the Kirkpatrick model correctly. To address this problem, Don Kirkpatrick’s son James and James’ wife Wendy released a revised and updated edition of the book: Kirkpatrick's Four Levels of Training Evaluation.
This book aimed to set the record straight and iron out any faults in the ways that people were using the model. The latest version of the Kirkpatrick model is known as the New World Kirkpatrick Model. This is the current definitive blueprint for evaluating any training process.
Whether you’re new to Kirkpatrick or have read about it before, it’s important to fully understand the four levels.
Here’s a brief overview of each of the four Kirkpatrick levels of evaluation:
Level 1: Reaction
The first level focuses on learner reaction. How did the learners feel about the course? Was the material relevant? Did they feel engaged?
Level 2: Learning
The second level looks at what learning took place during the course. Did the trainees learn new content or skills?
Level 3: Behavior
The third level of evaluation takes place a while after the course has finished. Are the skills and techniques presented in the training being used in the workplace? Why? Why not?
Level 4: Results
The fourth level looks at the impact the training had on the business. Can the financial outcome be measured? What impact did the training have on metrics such as customer satisfaction?
Step #2. Find and use a Kirkpatrick implementation planner
Your second step to mastering the Kirkpatrick model is to find and use an implementation planner. Kirkpatrick is such a widely used model that many companies and individuals have created their own fill-in-the-blank Kirkpatrick planning templates.
Why?
Templates make it easy to understand the whole process from start to finish. They help you decide upon the best evaluation methods for each of the four levels. Templates help you break down the objectives, questions, and goals of each level. They also help you see which aspects of the model you can automate, through a Learning Management System (LMS).
You can find many free Kirkpatrick implementation planning templates online. They are simple to use and will help you identify areas to consider when preparing to implement the Kirkpatrick model.
Most planning templates include the heading for each level and then spaces where you can write down the goals you want to set for each section. This can help you understand which areas need more attention before you implement them.
Step #3: Choose the best ways to implement each level
Your third step to mastering Kirkpatrick is to decide upon the best ways to implement each level of evaluation.
Let’s briefly look at each level and see what you need to consider:
How to implement Level 1: Reaction
An implementation planner will help you decide the best ways to evaluate Level 1 – Reaction.
You need to find out how the trainees reacted or responded to the training.
Your options include:
-
Verbal feedback: Simply ask the trainees and record their responses.
-
Paper-based questionnaires: Ask the participants to fill in questionnaires.
-
Online survey tools: Popular options such as Kodo Survey or SurveyMonkey let you send the trainees surveys via email.
How to implement Level 2: Learning
Next, you need to decide how you’ll evaluate the learning.
Your options include:
-
Oral assessments before and after the training.
-
Written assessments, such as quizzes or tests.
-
Practical tasks such as demonstrating a process or creating something new.
Where possible, the way you evaluate learning should reflect the contents of the training. For example, if the training is focused on helping the trainees develop a new creative skill such as programming, an evaluation that asked them to put their skills into practice would be most suitable. You should also avoid asking them how good they are at being creative since that would merely render you a subjective measure, i.e. their opinion. You should always try to come up with questions that will give you as objective results as possible.
How to implement Level 3: Behavior
Choosing the best way to implement the third level of evaluation – behavior – is one of the most difficult aspects of using the Kirkpatrick model.
Level three aims to evaluate whether the training has had an impact on workplace practices. In other words, have the trainees applied their knowledge to their jobs?
Your main options are:
-
Peer observations: Ask employees to observe each other and assess which tasks or strategies they see being implemented.
-
Job checklists: Ask employees to indicate which skills they use on a daily basis.
-
Pre- and post-training assessments: These assessments are conducted by managers or supervisors and indicate which skills they see being used in the workplace.
-
Pre- and post-training self-assessments: You could also ask which skills they are using and their competency in these skills, but again, that would give a more subjective measure. To drive objectivity, ask the employees to self-assess how they behave in certain situations critical to their job (and covered in the training), or how often they behave in a certain way in a specific situation.
How to implement Level 4: Results
Lastly, you need to decide how to measure the results.
Which business metrics do you wish to impact?
How often do you plan on assessing these metrics? Annually or biannually?
Once you’ve completed this step, move onto step four.
Step #4: Decide the content for each level of evaluation
You now have a good idea of how you’ll conduct each level of evaluation. Your next step is to decide specifically how to phrase the questions or the evaluations.
Let’s look at each level to see how this would work in practice.
Deciding the content for Level 1 – Reaction
In the last step, you decided how you would evaluate the trainee's reaction to the training – whether through a verbal, paper-based or online method. You now need to decide on the questions you want to ask and how you’ll collect the responses.
There are many different ways to phrase questions and invite a response. The three main ways are:
-
Open-ended
-
Closed
-
Hybrid
Each option has benefits and drawbacks, so we’ll briefly discuss each option separately.
Open-ended
If you take an open-ended approach, you could just ask the questions and leave spaces for a written response. This gives you a greater insight into the trainee’s mindset but it will take you far longer to process and interpret the results.
Closed
A simpler way is to provide a check-box or sliding scale such as:
-
Strongly disagree
-
Disagree
-
Neutral
-
Agree
-
Strongly agree
This type of tick-box questionnaire is far faster to process but gives you limited insight into why the trainees chose the responses they did.
Hybrid
The third option is to provide a mixture and closed and open-ended options for responses. For example, you could ask a question and provide a check-box scale for responses. Underneath you can provide space for written responses. This would be optional and only employees who were interested in going into greater detail would complete these sections. Kodo Survey uses a hybrid approach with both open-ended and closed questions.
For more inspiration on what questions to ask and how to shape them take a look at our whitepaper for determining and optimizing the impact of your training and development.
Which questions should you ask?
Once you’ve decided on the types of responses you want to invite, you need to start writing the questions. These could include the following:
-
How did you feel about the training?
-
Was the training useful?
-
How did you feel about the instructor?
-
Did you find the pacing of the course good?
Deciding the content for Level 2: Learning
In the last step, you already decided how you plan to evaluate the learning that took place. You now need to write the questions or tasks that you’ll include in this stage.
You need to match the type of questions you ask with the learning objectives of the training.
If the training is focused on learning knowledge and understanding, the types of questions you ask should reflect this. For instance, you don’t want to run training about customer service techniques and then ask the trainees to complete a higher-level skill such as interpreting or designing their own customer service strategies.
Likewise, if your training is focused on higher-level skills such as analyzing and debating techniques or processes, you don’t want to ask participants questions about basic knowledge and understanding of key terms and concepts.
The questions you ask in Level 2 should reflect the taxonomies of training being offered.
Deciding the content for Level 3: Behavior
When choosing how to implement Level 3 evaluation, you must aware of a wide range of reasons why knowledge or skills learned during training may never be used in the workplace.
For example:
-
The training could be too difficult to implement.
-
The trainees could lack full knowledge or understanding of the training material.
-
The staff could lack the resources necessary to implement what they learned.
-
The management could be preventing staff from changing their working habits.
Once you’ve chosen how to evaluate level three, you need to look at the right types of questions to ask.
These will vary depending on how to plan to conduct the assessment. If you chose peer observations, you want to know whether the employees that complete the observation can identify the skills or behaviors that the training focused on.
If you are using a job checklist, you need to ensure that you are asking the employees to focus on an aspect of their work that they developed during the training.
Kodo Survey mainly uses critical incident questions since we think this gives the most objective results to the least effort, which is important when evaluating the effectiveness of training for a high volume of learners.
Deciding the content for Level 4: Results
In the last step, you decided which business metrics you wished to measure and how often you planned to measure them. You now need to decide what questions will you need to ask in order to evaluate these metrics. If you ask the wrong questions, it will be difficult to tell whether the training has helped you meet your metrics.
Step #5. Consider whether you could set up an LMS or automate the processes
The fifth step of mastering the Kirkpatrick model is to see whether you can digitize the entire evaluation process. An digital platform for evaluating effectiveness of training and development could help you automate some or all of this. This will make the process far simpler to implement and the results will be more accurate.
Here’s a brief overview of the types of processes you may wish to automate.
Level 1 – Reaction
A digital platform for evaluating effectiveness of training and development could help you implement level 1 evaluation in the following ways.
You could:
-
Automatically email trainees with a post-training survey
-
Make the post-training survey mandatory if trainees wish to complete the course.
-
Set up email reminders to prompt trainees to leave feedback
-
Email reports to relevant stakeholders
Level 2: Learning
A digital platform for evaluating effectiveness of training and development could help you evaluate the learning as follows:
You could:
-
Automatically email evaluation materials to the learning supervisor(s)
-
Automatically assign pre- and post-training assessments to trainees
Level 3: Behavior
When it comes to evaluating level three, a digital platform for evaluating effectiveness of training and development offers clear benefits.
You could:
-
Deliver pre-training and on-the-job training assessments via email
-
Automatically email trainees with job checklists
-
Automatically set up job checklists according to the training
Level 4: Results
You will not identify Level 4 metrics in a digital platform for evaluating effectiveness of training and development. Those would be found in your other systems. Increased sales revenue is likely found in your ERP. Reduced absence due to work-related accidents is likely found in your HRIS. However, the data from those systems could be compared to the data in your digital platform for evaluating effectiveness of training and development as an attempt to find out if any of the learning and development is more likely to drive changes in those level 4 metrics.
Step #6. Test out your program
The sixth and final step of mastering the Kirkpatrick model of evaluation is to test your program. This will help you identify areas where you could make improvements or modifications. Try out your evaluations in real life and see how they go.
In this brief guide, we’ve explained how to master the Kirkpatrick learning evaluation model in six simple steps.
If you want to learn more about determining and optimizing the impact of your training download our free eBook today!
Related posts:
The Noob Guide to Understanding Kirkpatrick model evaluation
Kirkpatrick levels of evaluation: Expectation vs reality
Kirkpatrick level 3: Free evaluation examples
Kirkpatrick level 4: learn How to measure your ROI training