How to Qualify AI Use Cases in L&D Before a Pilot
AI use cases in L&D should begin with an observable problem. A stakeholder may ask the team to use AI to improve onboarding or speed up content production, but the request itself does not explain what is causing the underlying performance problem. The next question is whether improving the proposed task would materially affect that problem. If the connection is unclear, the use case needs further definition before testing begins.
Start with the performance problem
The first step is to understand what is actually causing the problem. For example, an onboarding delay may appear to be a learning-content issue. Further investigation could show that system access or opportunities to practise are contributing more to the delay. AI might help with one part of that situation while leaving the main constraint untouched.
This is familiar territory for an L&D function that works upstream from training requests. In Why Your Learning and Development Team Must Evolve, we explain why teams need to understand the underlying cause before deciding on a learning response. The same principle applies to AI. A request for an AI tool is still a proposed response, so the problem needs to be understood before deciding whether AI is the right way to address it.
Current research supports that caution. CIPD’s 2026 BridgeAI work includes eight workplace case studies involving organisations at different levels of digital maturity. The programme considers skills, governance and work design alongside the technology being introduced. CIPD’s BridgeAI research provides the wider organisational context.
RAND examined the issue from a different type of AI work. Its 2024 study interviewed 65 experienced data scientists and engineers about why AI and machine-learning projects fail. Reported causes included poor problem definition, unsuitable data, technology-first decisions and problems that exceeded what the technology could reasonably address. The RAND report provides the full findings.
The scope of that research matters. RAND excluded projects that only used pretrained large language models through prompting. Its findings should not be treated as a failure forecast for ordinary generative AI experiments within L&D. The relevant lesson here is narrower: define the problem before committing to the technology.
Define the task precisely enough to test
“Use AI to improve our learning content” is too broad to evaluate. AI use cases in L&D become easier to assess when the team can name the task AI would perform. A proposal might ask generative AI to draft an initial module summary from approved source material. An instructional designer would review the draft before publication. This makes the task, source material and responsibility clear. The team should then ask whether faster production of that summary would improve the wider process. If approval delays account for most of the elapsed time, automating the first draft may have little effect on the problem that prompted the pilot.
The same scrutiny should apply to whether AI is needed at all. RAND describes cases where machine learning was applied to problems that simpler rules could have handled. The technology could function while still adding avoidable complexity. A simpler process or existing automation may sometimes address the requirement more predictably. In those circumstances, choosing the simpler method is a sound qualification decision.
Check the operating conditions around the AI use case
AI use cases in L&D also depend on the conditions around the task. Source material may need approval, outputs may require review and access may need restriction. A use case involving personal information, enterprise systems or consequential decisions may require involvement beyond L&D. Information security, legal, IT or the relevant data owner may need to review parts of the proposal. The precise requirements will depend on the organisation and the use case.
The U.S. National Institute of Standards and Technology (NIST) takes a similarly contextual view of risk in its Generative AI Profile. Its guidance reflects organisational requirements, available resources and risk tolerance.
For a small L&D content experiment, proportionate controls may be enough. The level of oversight should reflect the task and the consequences of error. Human responsibility also needs to be clear before testing begins. The team should know who reviews the output, what would make it unacceptable and who owns the final decision. We discuss this boundary further in AI in Learning and Development: Automation vs Human Expertise.
Define what the AI pilot needs to prove
AI use cases in L&D should have a baseline and a decision measure before testing begins. The evidence should relate directly to the problem that justified the pilot. If approved learning content takes too long to produce, the team could compare the time required to reach an approved draft and track subsequent rework. Quality would still need to remain within the organisation’s agreed standard.
A learner-support pilot would require different evidence. Response accuracy or resolution time may matter more than production speed. Tool usage can add context, but it cannot show (by itself) whether the original problem improved. A useful pilot should produce enough evidence to support an investment decision. The organisation needs to know whether the improvement justifies further staff time, implementation effort or technology cost.
The evidence may support continued testing, a narrower use case, further investment or a decision to stop. Defining that decision in advance prevents a convincing demonstration from being mistaken for evidence of business value.
When an AI use case should not proceed
Qualification should sometimes lead to a decision not to proceed. The investigation may show that the problem sits outside learning, or that a simpler intervention could address it more reliably. The organisation may also lack the source material, review capacity or operating controls needed for the proposed task. Stopping or postponing the pilot can preserve budget and staff capacity while leaving the option to reconsider the use case later.
An AI pilot can require substantial staff attention and implementation effort even when no new software licence is needed. Early qualification helps direct those resources towards problems with a clearer route to measurable improvement.
Move from a qualified use case to a pilot
Once the problem, task, operating conditions and decision measures are clear, the team can assess suitable tools and design a proportionate test. At that point, the technology is responding to a defined need rather than driving the project. Qualifying the use case first helps L&D teams avoid spending time and budget on AI that does not solve the problem they set out to address.
If your organisation is exploring how generative AI could support learning, talk to SureSkills about where to start.