When a client or potential client asks me a question about measurement, I get excited. It tells me they see the learning solution we will create for them as connected to an important outcome. Clients focused on outcomes find it much easier to partner with us when we “begin with the end in mind” in the design phase of their project.
Measurement is critical to justify investments in training, help organizations plan future programs, and earn valuable recognition for the learning professionals who plan and oversee learning programs. You’re nodding your head, right?
But if we believe these statements are true, why aren’t more learning professionals prioritizing a measurement strategy when they create new learning initiatives?
In some cases, learning professionals simply do not know what they should be measuring. Attendance, participation, and feedback surveys are seductive because the information is readily available. We can ask learners whether they thought our workshop or course was effective, get a quick score, and feel like we measured something. The box has been checked.
Unfortunately, these forms of measurement do not tell us how much employees actually learned, whether they can or will perform the desired behavior on the job, and whether the business will achieve its desired outcome.
Even the most well designed, creative learning programs have a hard time getting funding if we cannot show how they help learners improve their performance, and how this improved performance impacts the business in a positive way. It’s no wonder many executives see training as a cost center instead of a strategic investment.
Learning Measurement Demystified
Thankfully, tools exist to demystify learning measurement and evaluation. My favorite learning evaluation tool was released in 2018 by Dr. Will Thalheimer. It’s called the Learning Transfer Evaluation Model (LTEM).
If you’d like to take a deep dive into the research and thinking behind the LTEM model, I highly recommend browsing Dr. Thalheimer’s website. I have included relevant links here:
The Takeaways
Here are my top takeaways from reading the report and attending Will’s keynote at eXLearn 2018:
1. The goal of workplace learning is to influence a non-learning outcome, such as increased sales, reduced costs, decreased turnover, or improved customer loyalty. (This is not unique to Will’s model. The Katzell-Kirkpatrick model introduced this concept in the 1950s).
2. While goals like these are important, a measurement that only focuses on the final end goal fails to show us why the learning intervention did or did not help the organization meet its goal. (Will makes a great point here! It is worthwhile to measure competence and learning transfer as a complement to non-learning outcomes measurement.)
3. Likewise, a measurement strategy that focuses on attendance, activity, learners’ thoughts/feelings about the training, and how the training is perceived tells us nothing about the training’s effectiveness.
4. Measurement strategies should evaluate learners’ ability to make the right decisions and competently perform the right tasks based on those decisions, both immediately after and at least several days after training has occurred.
5. Learning professionals must devise ways to measure whether learners are using knowledge and skills successfully on the job, and learners’ managers are often critical to this process.
6. Create the measurement strategy as part of the up-front design process. The measurement strategy should inform and drive design decisions.
The LTEM model provides ways to measure the impact of a learning intervention over the short term and the long term. It also guides us away from thinking of learner feedback and participation as evidence of short-term impact.
A simplified measurement strategy based on the LTEM might look like this:
Measurement of Short-Term Impact:
- The post-training assessment focuses on learners’ decision-making and task competence.
- Managers use coaching and evaluation guides to assess learners’ decision-making and task competence after some time has passed.
Measurement of Long-Term Impact
- Managers are enabled to evaluate and gather data on employees’ success at performing desired behaviors on the job over time.
- Business outcomes are tracked and reported on alongside learning measurement data. Other initiatives and external factors that may have also impacted business outcomes are clearly noted to avoid inaccurate correlations.
The LTEM model provides organizations with meaningful, quantifiable evidence of their learning solution’s effectiveness early on. Plus, it pushes learning professionals to connect their learning solutions to non-learning outcomes. It also de-emphasizes the measurement of unhelpful metrics.
Effective learning measurement enables us to prove that what we do matters to the business. It also helps learning professionals continuously improve their craft. Simply put, the creation of a measurement strategy is a learning design step you can’t afford to skip.
