In the third part of this series, we looked at the advantages and disadvantages of the Kirkpatrick Model of Evaluating a training program. I mentioned the Kirkpatrick-Phillips Model of Evaluation in passing, and promised you more on this “evolved” model. Here goes.
I call this an “evolved” model because, the Kirkpatrick-Phillips Model of Evaluating a training program is based on the Kirkpatrick Model of Evaluation. There are four levels of evaluating a training program in the Kirkpatrick model; there is an additional fifth level in the Phillip model.
|Level||Kirkpatrick Model||Phillip ROI MethodologyTM|
|1||Reaction||Reaction, satisfaction, and planned application|
|3||Behavior||Behavior, application, implementation|
|5||Return on Investment (ROI)|
The model is commonly known as the Phillip ROI MethodologyTM, after its inventor, Jack J. Phillips, PhD, Chairman of the ROI Institute and world renowned expert on measurement and evaluation. The model provides a practical solution to accurately measure the return on investment (ROI) of a training program even before any investments are actually committed. According to Jack Phillips, a training program must set off a chain of reactions – starting with employees’ reaction to training, to the level of learning and its application in business, and finally to its impact on business.
In most organizations, evaluation of a training program begins with and ends with the first level (reaction to learning) – when employers/managers informally ask employees how they liked the training program. As a result, the efficacy of a training program is never really understood. According to Training Magazine’s 2014 Industry Report, of the 61.8 billion US dollars spent on training, only 20% yielded the desired ROI.
According to the Phillip ROI MethodologyTM model, organizations cannot afford to skip any of the levels of evaluation. Each of the levels must be evaluated and measured, and the results must be converted into a monetary value. This is because, if a detailed evaluation is not done at each level, it would be difficult to attribute – with the utmost conviction, the success of a business to this training program. And if there is a negative ROI, it would be impossible to pinpoint the missing link(s) – the break in the chain of events – the culprit(s) that prevent a positive return on investment.
I mentioned earlier that according to this model, every measurement at level 4 must be converted into a monetary value so as to put a cost on everything (e.g. cost savings and time savings). But there will always be those intangible measurements that one cannot place a monetary value on. Customer satisfaction, a satisfying work environment, the satisfaction of an employee – are all very important outcomes but it would be hard to put a value on them. These must be made note of as ‘intangibles’ and taken into consideration as well.
The Phillip ROI MethodologyTM lays emphasis on collecting and analyzing data from the first four levels, but it lays special emphasis on the 4th and the 5th levels – business impact and ROI. In part 5 of this series, we will look at the measurable data recommended by the Phillip ROI MethodologyTM and how ROI is calculated.