If you’re interested in evaluating the effectiveness of a training course, you’ll know that there are a number of training evaluation models to choose from. One of the lesser-known models is the CIRO Model, and that’s what we’ll be discussing today. In this definitive guide to the CIRO Model, we’ll show you exactly what it is and how it differs from other training evaluation models. We’ll explain how you should apply it and offer examples of how it looks in practice.
This tutorial will cover the following areas:
- What is the CIRO Model?
- What are the levels of the CIRO Model?
- How does the CIRO Model work?
- Key differences between the CIRO Model and other models
- Example of the CIRO Model in action
- Common criticisms of the CIRO Model
- Conclusion
1. What is the CIRO Model?
The CIRO Model is a training evaluation model that is used to evaluate the effectiveness of management training course. It was developed by Peter Warr, Michael Bird, and Neil Rackham, who published the book, Evaluation of management training, in 1970. The CIRO Models offers businesses an effective way of evaluating their management training needs and results.
In common with other training evaluation models, the CIRO Model is hierarchical meaning that practitioners must start at the first of its four levels, before progressing sequentially through the following levels. In this respect, the CIRO Model is similar to other training evaluation models that we’ve discussed in the past such as the Kirkpatrick Model and the Phillips ROI Model.
However, unlike other learning evaluation models, the CIRO model is specifically aimed at evaluating management training. This sets it apart from other models that can be applied broadly to a number of different roles and positions within an organization.
To understand more about the CIRO Model it is necessary to study its four levels in greater detail.
2. What are the levels of the CIRO Model?
‘CIRO’ is an acronym that stands for the four levels which make up this approach to learning evaluation:
- Context
- Input
- Reaction
- Output
As a hierarchical model, you must start by studying ‘Context’, before moving through ‘Input’, ‘Reaction’ and ‘Output’. The first three levels of the CIRO model are ‘Evaluations’ and the fourth level is the ‘Outcome’, or results. According to the CIRO model, prior to assessing reactions and outcomes, there must be an analysis of the context and possible inputs.
A fuller description of the CIRO Model is as follows:
Stage 1: Context Evaluation
Stage 2: Input Evaluation
Stage 3: Reaction Evaluation
Stage 4: Outcome
Let's look at this in more detail.
3. How does the CIRO Model work?
Now that we’ve looked at what the CIRO Model is, it’s time to look at how this four-level approach works.
Stage 1: Context Evaluation
At this stage, the CIRO Model is used to assess the operational situation that a business or organization finds itself in. This provides useful information that can be used to determine the training needs and objectives.
During this first stage, a training-needs analysis is conducted, based upon the conditions in the organization. The context evaluation helps to flag up any factors that may have an impact on the effect of the training.
The context evaluation also helps identify and evaluate the training needs. In the CIRO Model, needs are based on collecting performance-deficiency information, ie. what the organization is lacking. The identified needs are set at the following three levels:
The ultimate objective
The goal of the ultimate objective is to eliminate an organizational deficiency such as low sales figures, low productivity or poor customer service.
Intermediate objectives
Intermediate objectives are those that aim to achieve an ultimate objective but may require a change in employee’s work behavior.
Immediate Objectives
Immediate objectives cover things such as acquiring new skills and knowledge as a result of the training. It can also include changing employee’s attitudes, which leads to them changing their behavior.
Stage 2: Input Evaluation
During the second stage of the CIRO Model, practitioners must gather information about possible training techniques and methods. This is known as the ‘input evaluation’ and helps identify the best choice of training intervention.
This stage also addresses design, planning, management and delivery of the training course. It analyzes the organization’s resources and determines how these resources can best be used to achieve the desired objectives.
Stage 3: Reaction Evaluation
The third stage of the CIRO Model involves gathering the views of the participants and collecting suggestions about the training they received. The trainees are asked to give their reactions to the following aspects:
- Program content
- Approach
- Value-added
Information gathered at this stage is used to find ways to improve the training program. As this evaluation is subjective, it must be collected in an objective way.
Stage 4: Outcome
This stage of the CIRO Model involves presenting information about the results of the training. The results are presented at three different levels;
- Immediate
- Intermediate
- Ultimate level
The immediate results include how the trainees got on and whether they managed to complete the training successfully. The intermediate outcomes are those that may take some time to implement, such as changes to the course design, or acquiring new training resource. Outcomes at the ultimate level are the main goals for the organization, ones that have a far-reaching impact on the organization.
As outcomes are evaluated in terms of what happened as a direct result of training, they can be measured on the following four levels:
- The learner level
- The workplace level
- The team or department level
- The business level
The chosen level will depend on the purpose of the evaluation and the available resources.
4. Key differences between the CIRO Model and other models
There are a number of key differences between the CIRO Model and other training evaluation models. It has most in common with the Kirkpatrick Model which, as you may know, was developed by Don Kirkpatrick in the 1950s. The Kirkpatrick Model is a four-step strategy for evaluating a course or training program.
The four levels of the Kirkpatrick Model are:
Level 1: Reaction
Level 2: Learning
Level 3: Behavior
Level 4: Results
You’ll notice that Level 1 (Reaction) in the Kirkpatrick Model is similar to stage 3 of the CIRO Model: Reaction Evaluation. However, there are a number of key differences between the way that each Model assesses participant reaction and what it does with that information.
With the Kirkpatrick Model, you are gauging the participants’ reaction to the training for the sole purpose of identifying whether the conditions for learning were met. You’d want to find out how the trainees reacted to aspects of the training such as the trainer, the venue and the resources provided.
In the CIRO Model, a greater emphasis is placed on gathering suggestions for how to change aspects of the training. The goal is to find out whether the format could be changed, and which parts of the training the participants thought could be improved.
Another key difference between the CIRO Model and the Kirkpatrick Model is the ‘Outcome’ stage. In the Kirkpatrick Model, you will receive outcomes in three of the four levels: Learning, Behavior and Results.
For example: let’s say you are running customer service training for your employees. Using the Kirkpatrick Model, you would find the following:
Level 1: Reaction
This level would show you how your employees responded to the training that took place. What did they think of the instructor? How did they find the venue? Were the right conditions for learning in place?
Level 2: Learning
This level would give you results about whether learning took place during the customer service training course. Managers or the course instructor could give the participants a short quiz or test before and after the training. Comparing the two sets of results would show what, if any, learning took place during the training.
Level 3: Behavior
This level would show you whether the trainees were using their new skills and knowledge at the workplace. Using a variety of techniques such as peer observations or manager observations, you could look at the employees’ work and assess to what extent they were using the skills and knowledge that they gained from the training. For example, you could look at how they were answering the phone, whether they were confidently answering customer’s questions and whether they were using the techniques there were taught during the training during their day-to-day work.
Level 4: Results
The fourth Kirkpatrick level would show you whether the training met the stakeholders’ expectations. For instance, if the stakeholders expected to see improved levels of customer satisfaction, you could conduct a customer survey to see whether the company’s responses had improved. You could present this data to the management to show the value that the training had added.
As you can see, the CIRO Model Stage 4, “Outcome”, straddles these three levels in the Kirkpatrick Model; learning, behavior and results.
The other main difference between the CIRO and the Kirkpatrick Model is that the CIRO focuses on the measurements taken before and after the training has been carried out, whereas the Kirkpatrick model focuses mainly on summative training.
5. Example of the CIRO Model in action
No definitive guide would be complete with an example of the CIRO Model in action. For this example, let’s imagine that a software company specializing in management software is looking to find ways to boost their sales. They decide to use the CIRO Model to develop their plans, set their goals and choose the right training course.
Stage 1: Context Evaluation
The management would begin by determining the training needs and then developing objectives that help the company meet its goal of improving sales. By conducting a training needs analysis, the company identifies that the product knowledge of their sales team is lacking. They set their objective as follows:
The ultimate objective: To help improve software sales.
Intermediate objectives: To encourage sales staff to research the latest software updates and developments.
Immediate Objectives: To boost product knowledge among sales team members.
Stage 2: Input Evaluation
During this stage, the company would analyze various training courses and weigh up the costs and length of each one. They will determine which course would best achieve their desired objective, based on their budgetary constraints and other requirements.
Stage 3: Reaction Evaluation
After conducting the training, the company would ask the sales team members to complete a survey or questionnaire. Data from this evaluation would be used to help the company decide whether the training offered value for money and what changes may be made to future training sessions. The CIRO Model’s Reaction evaluation shows you whether the conditions for learning were present in the training.
Stage 4: Outcome
The fourth stage of the CIRO Model looks at what learning took place and whether this learning made an impact on the employee’s work. You would be looking at what happened as a direct result of the training and whether the managers were implementing the tools and techniques they learned during their working hours. At a senior management level, managers could investigate an organization’s revenue to see whether the sales figures had improved.
Lastly, the managers would present results from the training and determine whether the ultimate level, intermediate and immediate goals were met.
6. Conclusion
Overall, the CIRO Model has many unique advantages over other types of learning evaluation models such as the Kirkpatrick Model. The data collection requirements are simpler and therefore faster and most cost-effective. However, the CIRO Model is specifically aimed at evaluating management training courses and is not designed to evaluate the effectiveness of other types of training or coaching programs. If you're ready to find out how Kodo Survey can help you evaluate the effectiveness of your training programs and learning journeys, request a demo with one of our experts today.
Related Posts