Knowing Bloom’s Taxonomy is great.
Knowing how to evaluate training effectiveness is great.
But what if you combine them both?
Today, we have something that will make you feel as though you have training evaluation superpowers:
We’ll show you exactly how to use the three hierarchical models from Bloom’s Taxonomy to succeed in evaluating training effectiveness.
If you've read our ultimate guide to understanding Bloom's taxonomy, you'll know that its hierarchical structures offer a way to classify the objectives and skills that educators set for their students. If you’re not familiar with Bloom’s, don’t worry, we’ll give you a brief overview later.
But in our experience, Bloom’s Taxonomy is useful for a wide range of other tasks too. That’s why today, we’ll be looking at how to use Bloom’s three hierarchical structures to evaluate any training course.
You see, just as the six levels of Bloom’s Taxonomy can be used to structure the learning objectives, assessments, and lessons of a course, they can also be used to structure training evaluation.
What’s Bloom’s Taxonomy?
Benjamin Bloom, an educational psychology at the University of Chicago, first proposed this hierarchical structure in 1956. It has since been updated to cover six levels of learning.
There are:
Level 1: Remembering
What can someone recall or recognize from their memory?
Level 2: Understanding
Can someone construct meaning from a message, image or text?
Level 3: Applying
Can someone implement their knowledge and understanding? Can they actually do something?
Level 4: Analyzing
Can someone see how one piece of materials relates to another?
Level 5: Evaluation
Can someone check and critique something?
Level 6: Creating
Can someone reorganize or create something new?
As we mentioned, this structure is hierarchical. Level one is the easiest type of learning while Levels five and six are the most demanding. Someone can only learn at the higher levels if they’ve mastered the knowledge and skills at the lower levels.
That’s why Bloom’s Taxonomy is often presented as a pyramid. Levels one and two provides a wide base, like a foundation. The subsequent levels build upon the knowledge and understanding gained in the first two levels. Below is a visual representation of Bloom's levels, courtesy of Vanderbilt University's Center for Teaching:
How does Bloom’s hierarchical structure help inform course design?
Bloom’s hierarchy helps educators plan their courses because it explains how people learn. It’s widely used in schools and universities throughout the world to help educators understand the process of learning, and we think it should be used in the world of corporate training as well.
It’s simple:
-
You must remember a concept before you can understand it
-
You must understand a concept before you can apply it
-
You must have analyzed a concept before you can evaluate it
-
To create something new, you must have evaluated other concepts.
Bloom’s hierarchy gives educators a way to set out their courses.
But there’s more:
You can’t just design every course in the same way. That would be boring. Instead, you need to consider the needs of the students and the type of course you are designing. All educators who do well with Bloom’s Taxonomy ask themselves the same two questions:
-
Who are your students?
-
What type of course are you designing?
Identifying your students will help you understand their needs. If your students are freshmen, you may target the lower level skills on Bloom’s hierarchy. They need to build their foundational knowledge and understanding. If you strive for too many learning objectives related to applying and analyzing, you may be reaching too far ahead.
If most of your students are juniors, seniors or graduates, you can work out how much foundational knowledge they already have. How much terminology and processes do you need to teach? How much can you assume they know?
The second question is about the type of course you plan to design. If you’re putting together an “Introduction to…” course, you’ll target the lower levels on Bloom’s taxonomy. If you’re teaching those senior in the area, you may not need to introduce any radically new concepts and you can focus on higher-order learning objectives such as applying and analyzing.
What is training evaluation?
Training evaluation is the practice of measuring the impact of corporate training and development to determine the impact it had on employee performance and the return on investment that the training delivered. One of the most widely used evaluation models is the Kirkpatrick Learning Evaluation Model.
This system was introduced by Don Kirkpatrick in 1993 when he released a book called the Four Levels of Training Evaluation. It offers organizations a way to evaluate the effectiveness of their training courses. In a recent blog post, we walked our readers through the process of mastering the Kirkpatrick model of training evaluation.
The Kirkpatrick Model has been further updated and clarified over the years and is currently known as the New World Kirkpatrick Model. It is still one of the most popular learning evaluation models out there.
If you’re not familiar with the system, here’s the deal:
In the 1950s, Don Kirkpatrick devised a system for measuring whether a particular training course had met its goals. His data-driven approach finds ways to improve the experience for learners and it also finds ways to improve and refine courses. Kirkpatrick’s model helps to identify problems in training courses. This helps organizations increase the quality of training that they provide for their staff.
Kirkpatrick’s Training Evaluation model has four levels:
-
Reaction: How did trainees react to the training?
-
Learning: What did trainees learn?
-
Behavior: Did trainees change their behavior afterward?
-
Results: Did the training meet the goals of the company?
Now, you may be wondering:
“How do you measure these things?”
This training evaluation model is simple to implement.
-
Reaction
You can send the trainees a survey, they can award different aspects of the course on a scale of 1-5. -
Learning
You can send the trainees two quizzes: one before and one after the course. -
Behavior
You can use on-the-job evaluations to investigate whether the trainees have started to use their training in the workplace. Psst... this can be automated with Kodo Survey. -
Results
You can test the outcomes of the training against the goals set by the organization or management.
How to apply Bloom’s hierarchy when evaluating training effectiveness
To apply Bloom’s hierarchy to the Kirkpatrick model, you need to use a ‘verb table’ that helps you check which verbs are aligned with each of the six levels of learning. Some verbs appear in more than one level and are called ‘multilevel-verbs’, but for the most part, each verb belongs to one level.
The following table is a simplified version of a Bloom’s verb table. It helps you to look at the learning objective of a training course and find out which of Bloom’s level the objective is aiming at. You can then see whether the course is working from level one upwards, or if it is putting higher-level objectives before the trainees have mastered lower-level skills such as remembering and understanding.
Let’s look at the table.
If you look at the verbs in the table, you’ll be able to evaluate the effectiveness of your training in a way that takes account of how the trainees learn. Bloom’s Taxonomy helps you analyze whether a particular training course or program is approaching learning appropriately.
Consider these scenarios:
Scenario 1: A training course could be targeting the lower-levels unnecessarily; the trainees may already have the basic knowledge and understanding.
The result?
The training is a waste of time and money.
Scenario 2: A training course could be targeting higher-levels of learning without the trainees having good knowledge and understanding.
The result?
The training is a waste of time and money.
You get the idea.
How Bloom’s hierarchy works with the Kirkpatrick Learning Evaluation Model
Bloom’s hierarchical structure helps with all four levels of the Kirkpatrick learning evaluation model. We’ll guide you through each of the four levels and show you exactly how to use Bloom’s to improve the Kirkpatrick training evaluations.
Level 1: Reaction
The first level of the Kirkpatrick model is reaction; how did the trainees react to the training? Most companies implement this by sending out a quick survey. The trainees respond with written answers or by choosing a number of a sliding scale from 0 to 10.
Surveys could ask questions such as:
-
How did you find the instructor?
-
How difficult did you find the materials?
-
How easy was it to manage your time effectively?
You then need to analyze the results and thread together common answers.
If most trainees indicated that the training was too difficult, you could infer that there wasn’t enough focus on the lower levels of Bloom’s taxonomy – remembering and understanding.
On the other hand, if trainees indicated that the instruction was too simple, you could improve the course by targeting higher levels of learning such as analyzing or applying.
Bloom’s hierarchy helps you understand whether the training course was pitched at the right level, or whether it needs to be refined and improved.
Level 2: Learning
Level 2 is probably the easiest of the Kirkpatrick levels to relate to Bloom’s Taxonomy.
Why?
Because most organizations use simple quizzes or tests to evaluate this level. Some companies use dedicated platforms designed to measure the impact of corporate training, such as KodoSurvey that helps generate tests and collect the data for you. The most common approach is conducting level two evaluations is to give trainees two tests; one before and one after training. This helps you understand whether the training was effective.
To apply Bloom’s to Kirkpatrick’s second level of evaluation, simply look through the questions being asked.
Pick out the verbs and see which of Bloom’s levels they relate to.
-
If the questions are asking trainees to ‘recall’ or ‘describe’ facts or information, the quiz is targeting the lowest of Bloom’s levels.
-
If trainees are being asked to ‘justify’, ‘compose’ or ‘develop’ information, the quiz is targeting the highest levels of Bloom’s Taxonomy.
You then need to think:
Does the type of question being asked match the type of training being offered?
In other words, is it a fair quiz?
If you’re just running an “Introduction to..” course with plenty of level one or level two content, but the quiz has questions that are level five or six on Bloom’s taxonomy, the trainees may not do so well.
Taking this approach helps you understand that low quiz scores don’t mean the course was bad. You were just assessing it incorrectly.
But:
If you’re running a course that targets level five or six content, like teaching programmers how to create new code, you’d expect a quiz that targets these areas of learning.
It would be easy for the instructor to give a low-level quiz that just targets knowledge and understanding and claim the course was successful.
But would this be a fair assessment?
No.
Bloom’s Taxonomy helps you understand whether the way you are evaluating a course is fair and appropriate. It can help you set appropriate assessments that take account of course content.
Level 3: Behavior
Much of what we’ve discussed apply equally to level three. With level three, you are looking for whether the training has translated to changes in the workplace. Here, you can look at which of Bloom’s levels the course content offered and then which of the levels you are assessing in the workplace evaluation.
For instance:
Let’s say you were running a training course to improve cleanliness in the workplace. If your course focused on Bloom’s first three levels of learning – remembering, understanding and applying – you’d need to conduct an evaluation that took this into account. When you carried out workplace assessments, the most you could hope for would be that employees could remember the hygiene rules and carry out their duties accordingly.
But if your training covered the higher levels, such as level six – create – you’d have to expect more from your staff. An appropriate evaluation could be asking your staff to create or refine their own set of rules and implement them.
For more ideas about how to conduct a Level 3 evaluation, be sure to check out our range of free evaluation examples.
Level 4: Results
People that succeed with Bloom’s and Kirkpatrick do one thing exceptionally well:
They set goals that are achievable.
To evaluate training at level four, you need to look at how success is defined. This level is perhaps the most similar to the original purpose of Bloom’s hierarchy; to help educators set learning objectives. Training goals are similar to learning objectives. If a training course fails to meet its goals, it may be the goals that are incorrect, not the training.
Sound crazy?
Well, a training goal may be targeting a higher-level of learning on Bloom’s hierarchy, without the trainees having mastered the lower levels first. If you weren’t aware of Bloom’s you may miss this important point and think that the training course was the problem.
Let’s use the verb table above to see how we’d use Bloom’s to analyze a training goal for a programming course.
Training goal #1: “Trainees should be able to recall basic coding procedures”
A quick check above shows that “recall” is a Level one word – so the course is targeting the lowest-level of learning; remembering.
Training goal #2: “Trainees should be able to create new code”.
This second goal has “create” so it’s targeting level six.
Using Bloom’s hierarchy, you can see whether the training goals were appropriate. Did the trainees have enough mastery of the lower levels to be able to complete the course? This will help you understand whether your training goals need to be refined. You should match the goals with the needs of the trainees and design the course from there.
And – bada boom – you’re all set!
Conclusion
If you were wondering how to use Bloom's hierarchy to succeed in evaluating training effectiveness, that’s about it.
You’re good to go!
If you want to get more examples, download our free guide here and get a head start on your competition!
Related posts: