Measuring Training Impact with the Kirkpatrick Model (Beyond Smile Sheets)
Completion rates and satisfaction scores tell you people showed up and didn't hate it. The Kirkpatrick model gives you a way to measure whether behavior and results actually changed. Here is how I plan evaluation before building anything.
The Kirkpatrick model is a four-level framework for evaluating training: Level 1 Reaction (did learners find it relevant and engaging), Level 2 Learning (did they gain the knowledge and skills), Level 3 Behavior (are they applying it on the job), and Level 4 Results (did the targeted business outcomes improve). Donald Kirkpatrick first described the levels in a series of articles in 1959, and they remain the most common vocabulary for training evaluation.
Most organizations measure Levels 1 and 2 well and Levels 3 and 4 barely at all. That's backwards, because Levels 3 and 4 are the reason the training exists.
Start at Level 4 and work down
The modern Kirkpatrick approach (the "New World Kirkpatrick Model" from Kirkpatrick Partners) recommends planning in reverse. Begin with the result the organization wants, define the critical behaviors that drive it, then design learning that builds those behaviors. This lines up neatly with action mapping.
If no one can name a Level 4 result, that's a warning sign about the training request itself.
Level 1: Reaction, kept useful
Smile sheets aren't useless; they're just often poorly written. Skip "How would you rate the instructor?" and ask questions that predict application:
- How relevant was this to your current role? (1–5)
- How confident are you that you can apply this next week? (1–5)
- What might stop you from applying it?
That last open question is gold. It surfaces environmental barriers, which training alone can't fix.
Level 2: Learning, measured with performance tasks
Measure what the objectives say learners will do. For most workplace skills, that means scenario-based questions or practical tasks rather than recall quizzes. A pre-test and post-test comparison is helpful if you can afford the extra time. If you can't, strong measurable objectives with matching assessments are the minimum.
Level 3: Behavior, the level everyone skips
Behavior change happens on the job, weeks after the course, so you need to plan how you'll see it:
- Manager observations with a short checklist of the critical behaviors.
- System data: CRM fields completed correctly, tickets resolved at first contact, phishing reports submitted.
- Self-reports at 30 and 60 days: "In the past month, how often did you…?" with a specific example.
- Work samples: reviewing real outputs such as emails, reports or call recordings against a rubric.
Kirkpatrick Partners also emphasize "required drivers": the reinforcement, coaching and accountability that make behavior stick. If managers don't support the new behavior, Level 3 results will be weak no matter how good the course is.
Level 4: Results, and leading indicators
Business results often take months to show and are affected by many factors besides training. To keep evaluation practical, pair each Level 4 goal with a leading indicator you can track sooner:
- Goal: fewer security incidents. Leading indicator: phishing report rate and time-to-report.
- Goal: faster time to productivity. Leading indicator: first-week task completion.
- Goal: higher customer satisfaction. Leading indicator: quality scores on reviewed calls.
In the Cyber Shield Academy project, phishing simulation click-through and threat reporting speed served as those faster signals.
A note on ROI
Jack Phillips later added a fifth level, return on investment, which converts results into financial terms. It's useful for large programs where leaders need a monetary case. For most projects, a credible chain of evidence from Level 2 through Level 4 persuades stakeholders more than a precise-looking ROI figure built on many assumptions.
An evaluation plan on one page
Before development starts, I write a one-page plan with four rows, one per level: what we'll measure, how, when and who owns it. It takes an hour and saves the scramble to "prove impact" six months after launch.
Sources
- Kirkpatrick Partners, The Kirkpatrick Model
- James D. Kirkpatrick and Wendy Kayser Kirkpatrick, Kirkpatrick's Four Levels of Training Evaluation (ATD Press, 2016)
- Jack J. Phillips, Return on Investment in Training and Performance Improvement Programs, 2nd ed. (Butterworth-Heinemann, 2003)