This was the great question asked to me by the ex-HR director of Carpetright, Lyn Warren. We had been professing the benefits of bottling greatness, social learning and continuous performance coaching – but does any of it really make any difference to the business, in real performance numbers?
Together with Carpetright (a European retailer specialising in Carpets and beds), and our measurement partners – the AI institute of UCL, which has professors in mathematics and neuroscience – we set about answering the question, one hypothesis at a time, outsourcing the measurement methodology to UCL’s experts.
Spoiler alert: the quick answer was yes.
And whilst it made an impact, that wasn’t the most interesting fact – the most interesting fact is something that should really help change how we think about designing learning forever.
Together, we wanted to take just one modern learning concept and see whether our hypothesis on positive business impact was correct or incorrect. We took the ‘bottling greatness’ concept for our first test. Our joint hypothesis was: if an organisation finds the best person within their organisation doing something (such as selling a certain product better than anyone else), then if we digitally extract that knowledge, post-produce it quickly through agile techniques into an interesting, authentic, passionate way, explained by the expert (but codified by the L&D team)…
…and then distributed in a small number of bitesized three-minute videos frictionlessly via the employees’ own mobile phone – that alone would have a measurable business benefit, likely in the 10% area (as that is what historical data tells us from other organisations).
So we did just that.
UCL guided us on what was the best area to choose, the one that would give us the purest form of measurement. We took an existing product that Carpetright had been selling for a number of years, which gave us the historical data that we needed, and would also provide good evidence of any variables such as seasonality or economic climate. We took a period of nine months to measure learning activity with revenue impact over that period and used the previous 3 years of data to compare against.
The result showed that over that nine-month period there was a 13% difference improvement in revenue due to the codifed knowledge being embedded across the organisation, and in data science terms it was ‘significant difference’ – meaning that the scientists stood behind the results.
There was a plus-or-minus 5% variability so at worst, the results indicated an improvement of around 8%. This statistic alone would suggest that even if nothing else happens, then product training should incorporate this element of bottled greatness and deliver it out in a similar fashion. So the question was answered – BUT there is so much more to consider from the data.
The initial month the content was rolled out showed an increase of almost 30% in sales revenue before it levelled out around 10%. What’s hugely interesting is that the performance curve almost matches the Ebbinghaus forgetting curve, which our industry has been referencing for so long, where Ebbinghaus’s research showed that people forget 50% within an hour if knowledge is not used and 80% in a month.
I find it hugely interesting that the two images of learning and sales performance correlate almost perfectly with each other, which suggests a course is not enough – even one designed in the most modern and impactful way. This learning was delivered to staff’s own mobile phones during a normal working day and they consumed it in downtime, but it’s also likely it was applied really quickly thereafter and therefore the gap between learning and application is often within the hour. We know the importance of the close gap between learning and application, and I believe that is one of the reasons for the success. Our industry has been talking about learning in the flow of work recently, this is a great example – especially when we look at how many searches were done to the content after the initial consumption by the colleagues from their mobiles.
The drop between initial performance and business as normal shows that the potential of performance is at least a 30% increase but as the HR director Rachel Wheeler pointed out: pushing bottling greatness is just the telling of excellence, and not the embedding of new habits and behaviours.
And Rachel is right. Moving forward, Rachel and her team have now enabled social learning functionality; coaching for performance has being introduced on top of the codification and distribution of their best tacit knowledge.
So what insights did we take from this data and what’s next?
1) Firstly – the concept of moving away from traditional online book based eLearning towards authentic, concise videos which capture the best people in the business articulating what they do looks to be something that we will continue to recommend and teach our customers. The results are proven.
2) I would encourage you to take the same scientific approach to learning design and make the hypothesis of performance outcomes prior to making the learning design. In other words, turn the Kirkpatrick on his head and then move beyond his measurement thinking, beyond the course, into a more holistic learning design.
3) I would encourage you to take Carpetright’s advice and move from simply telling and used spaced repetition technology-enabled techniques and people-based performance coaching on top, and use data to prove the benefit of the investment.
4) I would highlight to anyone that the demographics of Carpetright’s team shows that technology-enabled learning – and specifically mobile – is beneficial to any generation, not just millennials and younger.
I would also like to say a huge thank you for the teams at UCL and Carpetright for beginning this journey with us, and we look forward to the next stages of our adventure into insights together.
Where are we going next? The picture below speaks a thousand words …
About the author – Steve Dineen:
Steve’s goal as CEO of Fuse, is to help transform its customers into learning organisations through technology which its learners choose to use every day. Customers of Fuse have achieved unparalleled levels of learning engagement which has been translated into business value. Fuse currently lead the market in the area of next generation learning technology and is the only viable end to end alternative to an LMS.
Connect with Steve on LinkedIn
Leave A Comment