Corporate learning and development (L&D) can take a significant step forward toward meaningful return on investment (ROI) measures by moving beyond mere transactional data of how many people take a course. Thanks to adaptive learning platforms and the data they automatically generate, it’s possible to assess improvements in proficiency, competence and confidence without the additional effort usually associated with measurement.Practitioners of L&D analytics usually start with a lot of good intentions for gaining insights into performance improvements and business impact. Too often, though, analytics fail to reach that goal and, instead, fall back to the lowest common denominator. They focus on what is easy to measure instead of what they should measure. Often the root cause is a simple lack of time or money to support the effort required.
As we’ve seen, including in recent Donald H. Taylor surveys, the desire to capture and analyze more learner data has been a primary goal in L&D. It’s a commendable objective because determining meaningful ROI for L&D initiatives will help link corporate training to specific business outcomes.
The Holy Grail of L&D would be to show how corporate learning is supporting business strategies; for example that interventions are actually helping people learn—and that learning is driving desired behavioral changes that positively impact business performance. Ultimately, the goal would be to show hard-dollar payback numbers for L&D to justify the cost of corporate education. In most cases, I would argue, these attempts to calculate ROI are not worthwhile.
There are two fundamental problems. First, business outcomes depend on many more variables than human performance, and it is remarkably easy for other parts of the value chain to claim that benefits achieved had little or nothing to do with learning intervention(s). Second, typical learning management systems (LMS) or traditional e-learning platforms have not been able to capture in-depth learner data, such as where individuals struggled and what they eventually mastered. Instead, LMS and e-learning have been able only to measure activity—what we might call “transactional metrics,” such as how many courses were completed. This makes it difficult, if not impossible, to demonstrate that people even learned anything, let alone that behavior was changed, and business outcomes were improved.
This is a long-standing frustration for chief learning officers (CLOs). Getting anything more than the basic data has been very difficult and costly or else very speculative.
Systems such as xAPI (a standard that vendors can implement into their software) hold out the possibility of being able to collect much richer data. However, the adoption of xAPI has been slow, arguably because the data it generates requires an unrealistic amount of time and effort to analyze. Moreover, xAPI only addresses data gathering, not the bigger issue of how best to turn that data into insights. Without the means to derive meaningful analyses from learners’ experiences and outcomes, a proliferation of raw data is nearly as unhelpful as no data at all.
The Adaptive Learning Solution
Enter adaptive learning. An adaptive learning platform collects the same transactional data that is captured in an LMS system; that is, which learners took which courses and how many were completed. But it doesn’t stop there. With its inherently personalized approach, adaptive learning captures feedback from the learning process, minute by minute.
The adaptive learning engine uses that data to create better outcomes for learners and does so without human intervention. The data needed for personalization in essence becomes a “free byproduct” for analysis of the learner experience.
Adaptive learning data also provides insights into both the knowledge and skills that learners already possessed, as well as the knowledge and skills they needed to acquire. Most important, an adaptive system generates feedback on what the learning engine did to meet learners’ needs, and the knowledge and skills they acquired as a consequence. After all, far better than merely knowing something needs to be done is having the assurance that something was actually done.
These insights also carry a metacognitive aspect for learners. As learners become more self-aware of their strengths, what they mastered, and the misconceptions they held and now corrected, they adopt a growth mindset. This encourages ongoing, lifelong learning.
Approaching the Ultimate ROI
Adaptive learning systems can also bring L&D to the threshold of the ultimate ROI: showing not only what was learned but also the consequence of that learning. For example, as part of the personalized approach to developing learners, adaptive learning systems provide insight into learners’ improvements in both competence and confidence—in other words, evidence of what they’ve learned and how sure they are of their knowledge. This gives an indication as to how newly acquired knowledge and skills might translate into new behaviors. Advanced tools, such as the Rhapsode SKILLS™ module within Area9 Rhapsode™, are capable of taking this a step further to measure actual on-the-job performance and link that directly back to learning interventions.
While we’re not at the ultimate ROI as yet, it is possible that data insights driven by artificial intelligence (AI) will one day bring us closer to more easily linking L&D to specific business outcomes. The fact remains, however, that many things can affect workplace behavior, from relationships with bosses and coworkers to how motivated people are to do their jobs. For now, though, concrete evidence that learners are developing competence, confidence and self-awareness should provide reassurance to the business that L&D investment is having an impact, and that business benefits should follow. That, with granular learner data to back it up, would be a huge step forward in any learning strategy.
Nick Howe is the Chief Learning Officer at Area9 Lyceum.
You can reach him at nick.howe@area9.dk