In a thought-provoking webinar, Josh Penzell from ELB Learning and Maija Mickols from Docebo explored the transformative power of artificial intelligence (AI) in measuring learning success and driving better outcomes for organizations. The discussion shed light on the shortcomings of traditional learning measurement methods and delved into how AI can be harnessed to develop a more comprehensive, data-driven approach to learning and development.
The presenters questioned the effectiveness of the widely adopted Kirkpatrick model, referencing research that unveiled a surprising absence of correlation between the four levels of evaluation. Jump to 5:54 in the recording below to hear Josh and Maija’s thoughts on the Kirkpatrick model.
Research showed that Kirkpatrick Level 2: “Learning” had no correlation with any of the other levels. The strongest correlation between levels was between Level 1: “Reaction” and Level 4: “Behavior.” If learners liked the training, they were more likely to change their behavior.
Rethink the Way We Measure Learning Success
This discovery emphasizes the necessity for a paradigm shift in measuring learning success. Moving beyond mere knowledge transfer to prioritize learner behaviors and the overall learning experience, we must rethink the way we measure learning success.
As Maija shares, “With AI, we’re leveling up what we can do with existing data points.” This includes:
- Pedagogical data - Focused on the data’s relationship to the learning aims and pedagogical frameworks applied
- Linguistic data - Key information about the language produced by generative AI-powered tools or language produced by the learner
- Interaction data - How the learner interacted with the content and experience, focused on UI and UX
- User data - The profile of the user, including preferences, demographics, learning goals, skills, and job role
To fully capitalize on AI's potential, learning professionals must reimagine its roles in the learning process. The webinar introduced seven crucial roles: AI tutor, mentor, tool, simulator, teammate, student, and coach. Each role presents unique opportunities to collect rich data points, personalize learning experiences, and measure success in innovative ways. By incorporating these AI-driven roles into learning design, organizations can create a cycle of continuous improvement where real-world behaviors and learning behaviors become intertwined.
The presenters stressed the significance of data gathering and organization as a critical first step in implementing AI-powered learning measurement. Learning professionals should evaluate their current data collection methods, identify gaps, and explore new avenues for capturing meaningful data points. This may involve leveraging tools like xAPI, eye-tracking, or even simple audio recordings of live training sessions. The key is to start capturing data now so that organizations are well-equipped to take advantage of AI's capabilities as they become more accessible.
Moreover, the webinar highlighted the importance of considering the diversity of learners and the challenges posed by accessibility issues. AI has the potential to address these concerns by providing personalized learning experiences, real-time translations, and adaptable content. As AI continues to evolve, it will become increasingly crucial for learning professionals to ensure that their AI-powered solutions are inclusive and cater to the needs of all learners, regardless of their background or abilities.
Evolve Roles and Skill Sets
As we embrace this new paradigm, learning professionals must also evolve their roles and skill sets. Instructional designers should focus on changing behaviors and producing measurable results rather than simply creating content. Developing expertise in data analytics and engaging stakeholders with data-driven insights will be crucial in driving organizational buy-in and demonstrating the ROI of learning initiatives.
Furthermore, professionals can create truly transformative solutions that deliver measurable results by thinking outside the box and reimagining how AI can be integrated into learning experiences. The presenters encouraged learning professionals to experiment with generative AI tools, such as chat GPT, to analyze data, generate insights, and explore new possibilities for learning design.
Conclusion
By leveraging the power of data, rethinking the roles of AI, and evolving their own skill sets, learning professionals can create a new paradigm for learning and development—one that is agile, personalized, and focused on driving real-world outcomes. As AI continues to advance, organizations that welcome these opportunities will be well-positioned to unlock the full potential of their learning initiatives and drive sustainable success in the years to come. The future of learning measurement lies in those willing to embrace change, harness the power of AI, and continuously adapt to the evolving landscape of learning and development.
Want to get your team up to speed on using different AI-powered tools? Schedule an AI IQ workshop with one of ELB Learning's AI experts today!
This interactive instructor-led training session will raise your team’s AI understanding of prompt creation and how Large Language Models (LLMs) work. Participants will receive practical AI skills they can start using right away.
This AI training course will be tailored to your team’s skill level, whether foundational or advanced. We’ll ensure your employees have the knowledge to leverage AI to improve productivity and efficiency in their day-to-day jobs.
