Team leaders, supervisors or training managers often face a situation where they need to justify the costs involved in an eLearning experience by proving that it has a positive impact. One formula that is often used to measure ROI is by calculating the difference between the gain from investment and cost of investment divided by the cost of investment. To explain this in mathematical terms,
Calculating the cost of investment is easy.
It is easy to calculate the value of the total cost of investment by adding the various costs incurred for developing the training program. All you have to do is to add the expenses such as the cost of design and development, administration, technology used, materials, salaries of employees working on the project and so on and you can arrive at a figure that indicates the total cost of investment.
However, the challenging part is to calculate the benefits accrued as a result of training. How do you measure that?
How do you calculate the gains from investment?
Donald L Kirkpatrick, Professor Emeritus, University of Wisconsin came up with 4 levels of training evaluation. His model suggests evaluation at 4 levels:
- Reaction level
- Learning level
- Behavior level and
- Business level
At the Reaction level, the effectiveness of training is assessed immediately after the training program. Evaluation of the eLearning program can be an integral part of the training course, which can be made mandatory for course completion. In addition to the feedback form, learner behavior (in terms of the ease or difficulty) while taking the course can be tracked via the learning management system. Valuable data such as the time taken for the learner to complete the course, percentage of registrations vis-à-vis course completions and such other information can be analyzed to quantify success of a particular training program.
When it comes to the Learning level, the extent to which the learners comprehended the new knowledge or skills can be tested though assessments and quizzes. When administering training through online methods, it is easy to measure the learning outcomes through pre-test and post-test assessments. These tests can be used to check on the immediate reaction of learners with respect to the knowledge and skills gained as a result of the training. With the help of LMS, one can easily generate reports that measure the effectiveness of the training program at the learning level.
However, when it comes to the Behavior level, it gets a bit more complicated. Ideally, a survey needs to be conducted after 3 weeks of the conclusion of training session. The survey takes the feedback of participants, supervisors or managers to assess aspects such as the number of times learners applied newly learnt knowledge or skills to the job and the extent to which it has impacted their job performance. This can be rather subjective and accuracy of the data is questionable. Moreover, there may be other external factors that could influence job performance and attributing solely to training would not be inappropriate.
At the Business level, one needs to measure factors such as improved productivity, quality, sales costs, employee retention or customer satisfaction. Any changes in these factors cannot be totally attributed as a result of training because most often there are other players involved in influencing these results at the business level. Also, such data can be obtained under test conditions or for purely research purposes; but, it is rather impractical for organizations to track collect and collate data accurately in a dynamic environment.
While, it is easy to quantify and monetize results at the reaction and learning level, it is difficult to measure the results at behavior and business level. So, what is the way out? In my next blog, I will share some ideas on how we could calculate the ROI taking cues from the advertising world.