This blog takes a brief look at the Phillips ROI Model and compares it with the Kirkpatrick model of training evaluation. The Phillips model adds another level to the Kirkpatrick model (which has four levels) – measuring ROI.
In the third part of this series, we looked at the pros and cons of the Kirkpatrick Model of Training Evaluation and concluded that the model continues to be one of the most powerful tools used extensively by the ones who know to evaluate their training programs.
I wanted to talk about the Phillips model and how it builds on the 4 levels of the Kirkpatrick Model. As promised, here it comes.
Any business assesses success by looking at its bottom line, the Return on Investment (ROI), which indicates the growth (or otherwise) of a company. And this also holds true for learning evaluation.
Find out how leveraging learning analytics can maximize training ROI.
According to the 2020 Training Industry report, in most cases, the increase in organizational training budgets was due to the increased scope of training and investment in new technologies and equipment, among others. Given this context, it is crucial for organizations to measure the return on their investments.
The Phillips Model which adds an additional fifth level to the Kirkpatrick Model of evaluation helps address this very issue.
Jack J. Phillips, Chairman of the ROI Institute and a world-renowned expert on measurement and evaluation was the one who devised the Phillips ROI MethodologyTM.
The model offers a practical method to accurately measure the potential return on investment (ROI) of a training program even before any investments are actually made.
So, what does the Phillips model add to the Kirkpatrick model? In addition to the four questions asked by the Kirkpatrick model…
- Did the learners enjoy the training? (Employees’ reaction to training)
- Did knowledge transfer occur? (Skills and knowledge acquired)
- Did the training change learners’ behavior? (Application of the knowledge on-the-job)
- Did the training impact business results? (Impact on the business)
The Phillips model asks a fifth question: Was the training worth the cost? (Business results versus training costs or ROI).
Comparing the Phillips Model and the Kirkpatrick’s Model of Training Evaluation
The Kirkpatrick’s model of training evaluation measures reaction, learning, behavior, and results. The Phillips methodology measures training ROI, in addition to the first four levels of the Kirkpatrick’s model.
The Kirkpatrick Model vs. the Phillips ROI MethodologyTM
| Level | Kirkpatrick Model |
Phillips ROI MethodologyTM |
| 1 | Reaction | Reaction, satisfaction, and planned application |
| 2 | Learning | Learning |
| 3 | Behavior | Behavior, application, implementation |
| 4 | Result | Business impact |
| 5 | N/A | Return on Investment (ROI) |
Level 1: Reaction & Planned Application
Like the Kirkpatrick’s model, the Phillips ROI Model also starts with evaluating learners’ reaction to the training with the help of questionnaires or surveys.
Level 2: Learning
The second level of the Phillips ROI Model, like the Kirkpatrick’s model, evaluates whether learning took place – using tests or quizzes before and after the training.
Level 3: Application and Implementation
While level 3 of the Kirkpatrick Model merely assesses whether participants are applying what they learned on-the-job, the Phillips ROI Model looks at the application of the learning along with its implementation. This will help determine, in case of a problem, where the issue is – in the application of the learning or its implementation.
Level 4: Business Impact
We’ve already seen that the fourth level of the Kirkpatrick model focuses only on results. And as I said in my previous blog, it is very difficult to link a particular training to business results with 100% accuracy. The Phillips ROI model takes a much broader view at the impact of the training on the organization and helps us identify whether factors other than training (a new product, seasonal demand, etc.) were responsible for the improved business results.
Level 5: Return on Investment (ROI)
The fifth level of the Phillips ROI model helps determine the value of training programs through a cost-benefit analysis – whether the money invested in the training has produced any returns, and if so, the nature of the returns.
Why Use the Phillips ROI MethodologyTM?
According to Jack Phillips, an efficient training program must set off a chain of impact – starting with employees’ reaction to training, learning and its application, its impact on business, and finally, to the ROI of the training program. Organizations cannot afford to skip any of the levels of evaluation. Each level must be evaluated and measured, and the results converted into a monetary value. This is because, if a detailed evaluation is not done at each level, it would be difficult to attribute (with conviction), the success of a business to the training program.
But unfortunately, in most organizations, evaluation of a training program starts and ends with the first level (reaction to learning) with supervisors/managers informally asking employees how they liked the training program.
You may ask, “What’s wrong with this?”
What’s wrong is that when we limit evaluation of a training program to offhand reactions from learners, we cannot really gauge the efficacy of the training program and so, cannot make any significant changes to the program.
And if there is a negative ROI, it would be impossible to identify the culprit(s) – the break in the chain of events – that are hindering a positive return on investment.
As I mentioned earlier, this model advocates converting every level 4 measurement into a monetary value to place a cost on everything (e.g., cost savings and time savings). But there will always be those intangible measurements that one cannot place a monetary value on – for instance, customer satisfaction, a good work environment, employee satisfaction – all very important outcomes but hard to put a value on. These must be noted as ‘intangibles’ and taken into consideration as well.
The Phillips model focuses on the collection and analysis of data from the first four levels, with special emphasis on the fourth and the fifth levels – business impact and ROI. To calculate ROI, businesses need to:
- Gather data: Before, during and after the training
- Analyze data for improvement in:
- Processes
- Productivity
- Profits
The cost of the training can then be compared with the monetary benefit, giving an indication of the value of the training, and its impact on the organization’s bottom line.
In part 5 of this series, we will take a deeper look into the Phillips ROI MethodologyTM and how ROI is calculated.
Final Thoughts
To sum it up, adopt a practical method to precisely evaluate the potential return on investment (ROI) of a training program even before any investments are made.
If you wish to know more about ROI and ensure your training yields best results, do check this informative eBook on the Myths and Realities of ROI.
