The Evidence Problem in L&D — and How AI Finally Solves It
· 9 min read · OneRange Team
L&D has spent two decades being asked to prove its impact and not quite being able to. The reason isn't lack of effort. It's lack of evidence. That's now changing
There's a quiet conversation that happens in almost every L&D budget review. A finance leader asks, in some polite form, what the company got for the money. The L&D leader presents completion rates, satisfaction scores, and a few stories of programs that went well. The finance leader nods, the budget gets approved at roughly the same level as last year, and both sides walk away feeling like the question wasn't really answered.
It wasn't. Not because L&D leaders aren't trying, and not because the work doesn't matter. The honest reason is that the field has been asked to prove its impact with the wrong kind of evidence. Surveys are vibes. Completion is attendance. Satisfaction is a customer feedback metric, not a capability metric. The actual question — did people get better at the things the business needed them to be better at? — has been almost impossible to answer at scale.
Why the evidence has been so weak
Three structural reasons, none of them anyone's fault.
- Capability is hidden. You can see whether someone showed up and clicked through. You can't easily see whether they can now do the thing better. That requires assessment, and assessment at scale has historically meant either expensive human evaluation or shallow multiple-choice tests
- Skills are messy. Real-world capability is a combination of knowledge, reasoning, and judgment. Most assessment instruments measure recognition — can you pick the right answer from four options — which is almost the easiest version of capability and the least correlated with on-the-job performance
- The unit of analysis is wrong. Most L&D measurement happens at the program level — was this course good? — when the unit that actually matters to the business is the skill: are we more capable at X than we were last quarter, and is that movement attributable to anything we did?
What AI changes about evidence
Conversational AI assessment doesn't just digitize the old multiple-choice quiz. It changes what's measurable. When a learner has a real exchange with a training system — explaining their reasoning, working through a scenario, defending a recommendation — the system captures evidence of capability that no quiz format can produce. And it does it at a cost per learner that makes it practical to assess everyone, not just a sample.
That has compounding effects. Because every learner is assessed, the data is dense enough to support real analysis. Because the assessment is tied to a defined skill model, movement can be tracked at the skill level, not just the program level. And because the assessment is continuous — every training session is also a measurement — the data stays current instead of degrading between annual reviews.
The four-question test for L&D evidence
An L&D function that has solved its evidence problem can answer four questions about any program, in any quarter, without scrambling.
- What specific skills was this program designed to move, and at what proficiency level?
- What was the proficiency profile of participants before, and what is it now?
- How does that movement compare to people who didn't participate?
- Has the movement translated into observable outcomes — output quality, time to ramp, customer impact, retention — at a level the business cares about?
Five years ago, answering all four required a research budget and a year of work. Today, with the right instrumentation, the first three are continuous outputs of the platform and the fourth is a join with HRIS, CRM, or product analytics that any analyst can run.
What this changes about the L&D conversation
When evidence becomes available, the conversation with finance changes shape. It stops being defensive — justifying last year's spend — and starts being strategic: where should we invest more, where should we stop, and what's the marginal return on the next dollar? L&D moves from a cost center asked to prove it's not wasteful to a function asked which capabilities to build next.
It also changes the conversation with the business. A sales leader who used to hear 'we ran the negotiation program for your team' now hears 'your team's proficiency on price discovery moved from capable to proficient, and reps in the proficient band are closing 18% larger deals on average.' That's a fundamentally different sentence. It connects training to outcomes in a way that doesn't require anyone to take L&D's word for it.
What L&D leaders should do now
- Build, or borrow, a real skills model for the domains that matter most to the business — not a wishlist taxonomy
- Replace at least one major assessment instrument with conversational AI assessment, and compare the data quality
- Tie training programs to specific proficiency movements as the success metric, not completion or satisfaction
- Connect skill data to one business outcome — ramp time, quality, retention — and report on the join, not the silos
- Stop apologizing for measurement gaps that are about to be solved
The bigger picture
L&D's evidence problem was never about effort or intent. It was about the cost of producing capability data at scale. AI changes that cost by an order of magnitude. The functions that adopt this early aren't just going to win their budget conversations more easily — they're going to be invited into strategy conversations they've historically been left out of, because for the first time they'll be able to answer the questions the business has always wanted answered.
Tags: L&D, Analytics, ROI, Skills Assessment