CallSource^® Aim Higher

The Four Levels of Training Evaluation

by Dr. Ann Kwinn, July 12 2010

Evaluation is a big word that has many meanings. What does successful training look like to you? We would all privately admit that we want our students to be happy, to learn and to have such a good buzz about the class that more people want to take the class.

With today's performance measurement trend, some organizations are asking training staff to prove that learning occurred and that the learning is measurably beneficial to the organization. 

Author Donald Kirkpatrick proposed four levels of evaluation. This hierarchy has become a popular way to think of training evaluation. Each level is described below. The lowest levels are easier to do and are more commonly done.

Level 1 - "Reaction" - asks the question: "Did students like the course?" Many companies have a standard end of course evaluation that is the same for all courses and is distributed at the end of each course. It contains items such as: "This seminar met my needs: Strongly agree, agree, disagree, strongly disagree. " These forms are sometimes pejoratively referred to as "smile sheets," since the information gathered is subjective. However, they do give you information about customer satisfaction. And, if there is a write in section on the form, you can possibly find out areas of the course that students find confusing.

Level 2 - "Learning" - asks the question: "Did students learn what was intended for them to learn?" In this case, the evaluation is a test of the course content and therefore is specific to the course. The items in the test should assess the students' achievement of the learning objectives established for the course. If the course materials state the students will be able to bake a cherry pie by the end of the training, the assessment should ask them to bake the pie.

One interesting note is that one study found no correlation between level one and level two evaluation - or - just because you like a course, it doesn't necessarily mean you learned something.

More companies are asking training departments to include tests in their courses. At CallSource, we also include a pre-test so you can see how students' knowledge increased as a direct result of training. Tests and other types of practice activities can also serve as a learning opportunity if the student is given feedback on their answers.

Level 3 - "Behavior" - asks the question: "Did job productivity measures improve?" To perform a level three evaluation, you must determine what job performance measures are important to you such as - number of appointments set - and measure this before and some period of time after the training. If the number is higher, we say that the new skills have transferred to the job.

Notice that this type of measure - number of appointments set - is hard, objective information - something really happened - your employee did something. Compare this to a subjective measure such as asking either the employee or their supervisor: Do you think more appointments were set this month than last month? 

At CallSource, our Telephone Performance Analysis product assesses and scores prospect calls so that change over time can be measured - with improvements attributed to training or incentive programs.

Level 4 - "Results" - asks the question - Did the organization's bottom line improve? Most organizations are in business to make money and have level four measurements such as sales, reduced turnover, and customer satisfaction. What is the bottom line for your organization and how can it be measured? If your level four measure is number of leases, you can train people to ask for an appointment on every prospect call and test after the training that they know how to do it. (That's level two). Then see if their performance on the phone improves (that's level three) and if that improvement increases the number of leases written. (That's level four.)

For key training initiatives and key bottom line measures, a level four evaluation can show the true effectiveness of training and prove your return on training investment.

About CallSource:

CallSource, an integrated marketing, sales, training and management company, provides online call-tracking, recording and analytic solutions that allow companies to measure advertising effectiveness, improve return on marketing investment, increase sales and enhance customer service. CallSource tracks more than 500,000 advertisements and 7,000,000 million phone calls every month for multi-family communities, media companies and advertising agencies, and a wide range of businesses throughout North America.

CallSource also offers expert performance analysis, and self-paced and instructor-led training to help clients improve marketing and call-handling and convert more callers into customers. Since 1994, CallSource has provided innovative solutions and services that improve organizational performance across marketing, operations, sales, training and customer service.

TRAINING - Blurb

Learn how outcome-oriented training can help your ROI. Check out the article by training authority Dr. Ann Kwinn.