Phillips ROI Model: The 5 Levels, Real Studies and Templates
The Phillips ROI model explained: Level 0 and the five levels applied to one workplace programme, plus two real evaluations and two free Word templates.

The Phillips ROI model, which ROI Institute calls the ROI Methodology, is Jack Phillips's framework for evaluating a programme from its inputs through reaction, learning, application and business impact to a return on investment, after isolating the programme's own effect.
- Level 0 records the inputs, such as who attended, the hours and the cost, and ROI Institute does not count it as a result.
- Isolating the programme's effect from other influences is the step that makes a Phillips ROI figure credible, so plan it before the course runs.
- ROI Institute suggests taking about 10% of programmes to business impact and 5% to ROI each year, so most stop at learning or application.
- A Portuguese bank's phone-sales course looked like a win until a matched comparison group improved as much or more.
Five questions, about a minute. Pick one programme you are about to run, one where a sponsor requires an ROI evaluation. This is an unvalidated planning aid. The total organises suggestions; it does not assess competence or prove that the method will work. Review each answer.

Go from storyboard to an interactive course in minutes
Imagine a finance director asking the training team a fair question a few months after a new accounts-payable system goes live: did the training pay for itself? By then the course has run in both offices, nobody has kept a group back for comparison, and nobody has written down what the accounts-payable clerks were supposed to do differently. Whatever ROI figure comes back will rest on guesswork, however careful the arithmetic.
I'm Carolina, an instructional designer at Coursebox, where we make instructional design software, so I have a commercial interest in how training gets evaluated. I write down what a course should change before I build it, and the Phillips model turns that habit into a full evaluation plan. Below I explain the model as ROI Institute describes it today, walk one workplace programme through every level, look at two real evaluations from Portugal and the United States, and share two free Word templates.
What is the Phillips ROI model?
The Phillips ROI model, which ROI Institute calls the ROI Methodology, is Jack Phillips's framework for evaluating a programme from its inputs through reaction, learning, application and business impact to a return on investment, after isolating the programme's own effect.

View the full-size the phillips roi model diagram
In January 2026, Patti and Jack Phillips summarised the current shape in one line in an ROI Institute article on evaluation trends: "The input into a learning program is Level 0, followed by five levels of outcomes, reaction, learning, application, impact, and ROI." The method is not limited to courses: the Institute describes it as linking "programs, processes, and initiatives" to bottom-line measures.
Three ideas sit underneath the levels:
- A chain of impact. People attend, react, learn, apply what they learned, and the business measures change. The ROI Methodology page calls the five levels plus intangible benefits "a chain of impact that occurs when participants attend a program, learn, apply their learning, and improve impact measures."
- An isolation step. Before any business result is credited to the programme, the evaluator separates its effect from everything else that changed at the same time. The Institute's question is blunt: "How do you know it was the program?"
- Standards. Twelve guiding principles keep the analysis conservative, for example using only the most credible sources and assuming little or no improvement where no data exist.
The Phillips ROI model levels at a glance
The table gives each level, how ROI Institute describes it, the question it asks of a programme and an example from a workplace setting.
| Level | How ROI Institute describes it | The question it asks of your programme | What it looks like at work |
|---|---|---|---|
| 0. Input | Who took part, how long it took and what it cost | What did the organisation put in? | Enrolments, seat time and the full cost of a leadership programme |
| 1. Reaction and planned action | Satisfaction with the programme plus what people plan to do with it | Did people find it relevant, and what will they do differently? | A new manager rates relevance and names two habits to try with the team |
| 2. Learning | Changes in knowledge and skills, including confidence to apply them | Can people do it by the end of the programme? | A technician passes a practical test on a new diagnostic tool |
| 3. Application and implementation | Whether people use it on the job, and what helps or blocks them | Are people using it at work, and what gets in the way? | A sales manager reviews a sample of recorded calls 30 days later |
| 4. Impact | Changes in business measures of output, quality, cost and time | Which measure moved, and how much of the change was the programme? | Fewer lost-time injuries in the six months after a new site induction |
| 5. ROI | Monetary benefits compared with the fully loaded costs | Were the benefits worth more than the costs? | Net benefits of a customer-service programme divided by its total cost |
The levels are categories of data, not a timetable. ROI Basics puts it plainly: "timing of data collection does not necessarily define the level to which you are evaluating." A supervisor's observation on day one of a new process is still Level 3 data.
Older Phillips workshop material named the levels "Reaction and Perceived Value", "Learning and Confidence" and "Impact and Consequences". Current ROI Institute material uses Reaction and Planned Action, Learning, Application and Implementation, Impact, and ROI, with Level 0 Input in front. The Institute's methodology page still says "Business Impact" for Level 4, and ROI Basics treats the two names as the same level. Intangible benefits are reported beside the ROI, not as an extra level.

Case study: Phone-sales training at a Portuguese bank
A Portuguese bank, anonymised as Bank X, wanted to start evaluating training for ROI after a restructuring. It began with one programme: Make the Call, a 10-hour online course on selling by phone for staff in its commercial network.
- What they did: before the 2024 run, the evaluator wrote a data collection plan and an ROI analysis plan. The 129 people trained were compared with 127 untrained colleagues matched on role, tenure, location and gender, using the bank's quarterly sales-target index as the Level 4 measure.
- Evidence and result: against the same quarters a year earlier, both groups' average target achievement rose by 34.8 percentage points in the second quarter, and the untrained group gained more in the third (20.1 percentage points against 8.1). The master's work project concluded that "both groups performed similarly" and did not calculate an ROI.
- Limitation: this is a master's work project, not a peer-reviewed study. Trainees were not randomly assigned, the sales index covers many products, and the bank withheld the financial data an ROI would have needed.
- Lesson: without the comparison group, the trained group's rise of 34.8 percentage points would have looked like a large return on a 10-hour course. With it, most of the rise looks like the new products and incentives launched in the same half-year, though the author also judged the sales index too blunt to detect a 10-hour course. Reporting no isolatable business effect is a correct Phillips result.
Where the Phillips ROI model came from
In 1973 Jack Phillips was an electrical engineer at Lockheed Aircraft in Marietta, Georgia. His senior leader told him that further funding for a cooperative education programme depended on showing a return on the investment. According to the chapter of ROI Basics that ROI Institute shares for free, he found the four-step evaluation idea of industrial-organisational psychologist Raymond Katzell, which Don Kirkpatrick had written about in 1956. Phillips then "added the economic theory of cost-benefit analysis to the four-step concept and created a model and standards".
His 1983 Handbook of Training Evaluation and Measurement Methods put the five levels and the process model into print. Patti Phillips is co-founder and CEO of ROI Institute, and the two have written the current editions together.
The Phillips books: which to read
Three books cover the method at first hand:
- 2019: ROI Basics, second edition (ATD Press), by Patti and Jack Phillips. Short, practical, and its first chapter is free from ROI Institute.
- 2024: Return on Investment in Training and Performance Improvement Programs, third edition (Routledge), by Patti Phillips, Jack Phillips and Klaas Toes. The full reference, with a chapter each on isolating effects, converting data to money, costs, intangibles and forecasting.
- 1983: Handbook of Training Evaluation and Measurement Methods (Gulf Publishing), the original. Read it for history, not for current practice.
If you only read one, start with ROI Basics. Move to the 2024 edition when you plan your first evaluation that goes to Level 4 or 5.
The Phillips levels, worked through one programme
Here is the hypothetical programme I'll carry through every level. A mid-sized distributor is moving its two offices onto a new three-way invoice-matching system, which checks each supplier invoice against the purchase order and the goods receipt. The new system goes live in both offices on the same day in March. Its 40 accounts-payable clerks, 20 per office, need to learn the matching rules and the new exception codes, because an invoice that fails a match lands in an exception queue and someone has to resolve it.
The training is a 90-minute online course plus a one-hour supervised practice session. Office A's clerks take it in the week before go-live. Office B's clerks take it four weeks later, after the month-end close, and work from the vendor's quick-reference guide until then. That four-week gap gives the evaluation a comparison group. The finance director wants to know whether to fund the same approach for the next system rollout, so the plan runs all the way to Level 5.
The example works for Phillips because the business measures already sit in the finance system and convert cleanly to money. For each level you get a sample objective, how to measure it, and where and when the data come from.
Level 0: Input
Level 0 counts what goes in: people, hours and money. ROI Basics is clear that "Level 0 is not a category of results", but it records how much the organisation is investing, and the cost figures feed the ROI at Level 5. Its four Level 0 questions are who attended, how many, what it cost and how long it took.
- Sample objective: all 40 clerks complete the online course and the practice session in their office's training window.
- How to measure it: completion records, seat time and a running cost log covering design, delivery, supervisors' time and clerks' time.
- Source and timing: the course platform and the project budget, captured as the programme runs rather than reconstructed afterwards.
Level 1: Reaction and planned action
Level 1 "measures participant satisfaction with the program and captures planned action", in the Institute's words. Its five reaction questions ask whether the programme was relevant, important, new, likely to be used and worth recommending. The planned-action part is what separates Level 1 from a smile sheet: each person writes down what they will do differently. Our guide to training survey questions has more wording options.
- Sample objective: an average of at least 4 out of 5 on relevance to daily work, and every clerk names two exception types they will handle differently.
- How to measure it: five short questions at the end of the practice session, plus a planned-action box.
- Source and timing: the clerks, at the end of the practice session. Supervisors receive the planned actions, which become the starting point for Level 3.
Level 2: Learning
Level 2 measures "changes in knowledge and skills", and ROI Basics adds that it "also includes the level of confidence participants have in their ability to apply what they have learned." In a procedural course, the strongest Level 2 evidence is a realistic task, not a recall quiz. A scenario-based check gives each clerk the documents a real exception comes with, so the decision can be judged.
- Sample objective: each clerk resolves at least 8 of 10 exception scenarios with the correct matching rule and exception code, and rates their confidence for each.
- How to measure it: a scenario check where each case supplies the purchase order, goods receipt and invoice, with open answers marked against the approved procedure.
- Source and timing: the clerks, at the end of the practice session, marked by their supervisor.
Level 3: Application and implementation
Level 3 asks whether people use what they learned at work, and what helps or blocks them. The Institute calls it "the richest source of data". It shows where a programme breaks down. A clerk who passed the scenario check but cannot find an exception code in the live system tells you something a test score cannot.
- Sample objective: 30 days after training, a supervisor's review of 20 exceptions from each clerk's queue finds the correct code on at least 18, and each clerk reports progress on both planned actions.
- How to measure it: a queue sample review against a short checklist, plus a 30-day action plan in which each clerk records what they changed, what got in the way and their estimate of how much the course contributed.
- Source and timing: supervisors, the finance system's exception queue and the clerks, at day 30 for each office.
Level 4: Impact
Level 4 measures are "the consequence of applying skills and knowledge from the program", typically output, quality, cost and time. This is also where the isolation step happens. ROI Basics says "a step to isolate the program's effects on the improvement in these measures from other influences is always taken." In this example, Office B is the comparison for the first four weeks.
- Sample objective: in weeks 1 to 4 after go-live, Office A's exceptions per 100 invoices and its rework hours are at least 25% lower than Office B's, and late-payment penalties fall over the 12 weeks after go-live.
- How to measure it: exception rate, rework hours and late-payment penalties from finance system reports, with clerk and supervisor estimates of the course's contribution collected as a fallback.
- Source and timing: the finance system, weekly for 12 weeks from go-live.
Level 5: ROI
Level 5 "compares the monetary benefits to the costs". The Institute lists six moves to get there: identify the change in the Level 4 measures, isolate the programme's share, convert it to money, total the fully loaded costs, list the intangible benefits, and compare benefits with costs. Level 5 is the only level that depends on another: without Level 4 data there is nothing to convert.
- Sample objective: break-even or better on first-year benefits, a target agreed with the finance director before launch.
- How to measure it: convert the isolated change using finance's standard values and compare it with fully loaded costs. Count rework savings separately only if they are not already included in the per-exception value.
- Source and timing: the finance director and HR cost data, at week 12, separating observed benefits from any first-year forecast. Recheck savings after both offices have trained before projecting beyond the comparison period.
If you cannot write a Level 4 objective with a number in it, the programme is not ready for a Level 4 evaluation, and possibly not ready to be built. ROI Institute's planning model pairs each kind of need with an objective at the matching level, starting from the payoff and business needs and working down to the reaction you want.
How the ROI formula works
The Phillips method reports two figures. ROI Basics gives them as:
- Benefit-cost ratio (BCR): programme benefits divided by programme costs.
- ROI (%): net programme benefits (benefits minus costs) divided by programme costs, multiplied by 100.
The two describe the same result in different ways. As the book says, "A BCR of 2:1 means that for every $1 invested, you receive $2 back. This translates into an ROI of 100 percent." Benefits are the isolated, converted Level 4 improvements, and costs are fully loaded, from needs analysis to evaluation.
The arithmetic is simple; the inputs are the hard part. For a worked calculation with a sensitivity range and the common calculation errors, use our training ROI guide.
How the levels connect: the chain of impact
The Phillips levels describe a chain that has to hold at every link for a business result to be credited to a programme. In the 2019 version of the chain, the order is input, reaction, learning, application, isolation, impact, ROI, and then intangible benefits beside the ROI. The isolation step sits between application and impact: before a change in a business measure is credited to the programme, everything else that changed in the same period is accounted for.
The chain is not a promise that each level causes the next. Good reactions do not guarantee learning, and learning does not guarantee use on the job. What the chain gives you is a diagnosis. In the invoice-matching programme, if exceptions fall in Office A but the Level 3 queue review shows clerks still choosing the wrong codes, something other than the course is likely behind the drop. The Portuguese bank found the reverse pattern: positive results at the first three levels and no isolatable effect at Level 4.
Two of the guiding principles in ROI Basics set how much work each level needs. The first says "When conducting a higher-level evaluation, collect data at lower levels." The second says the lower level "is not required to be comprehensive." So a study aimed at Level 4 still runs a short reaction survey and a learning check, though neither needs to be elaborate.
Which programmes should go to Level 4 or Level 5?
Only a few. ROI Institute's methodology page says comprehensive evaluation is for "very expensive programs, programs that have a long life cycle, programs that have a broad reach, comprehensive programs, and programs that instill significant change in the organization." ROI Basics adds programmes that have "the attention of senior managers" or are "highly visible".
The 2023 session slides also recommend reaction data for every programme, learning data for 80 to 90% and application data for 30%. Older Institute material set the bar a little higher: the 2009 government workshop suggested 10 to 20% at impact and 5 to 10% at ROI.
The invoice-matching programme is small, with 40 clerks, so it qualifies for Level 5 on executive interest rather than size: the finance director asked, and the answer will decide how the next rollout is funded. If nobody with a budget decision is waiting for the number, stop at Level 4 and report the business measures and intangibles.
How to use the Phillips ROI model to plan an evaluation
ROI Basics groups the process into four phases: plan the evaluation, collect data, analyse data and optimise results. Other Institute material counts 10 or 12 steps, so treat the count as a convenience rather than a rule. Here is how I sequence it, from the Institute's four phases:
- Start with the business need. Name the measure the programme should move and what is happening to it now.
- Check that training is the right solution. A matching problem caused by bad supplier data needs a supplier fix, not a course.
- Write an objective at every level you plan to evaluate, from the business measure down to the reaction you want.
- Write the data collection plan: for each level, the measure, the method or instrument, the source, the timing and who is responsible.
- Write the ROI analysis plan: for each Level 4 measure, the isolation method, the conversion method, the cost categories, the intangible benefits and who receives the results.
- Collect Level 0 to 2 data while the programme runs, and Level 3 and 4 data afterwards on the dates in the plan.
- Analyse the results: isolate the programme's effect, convert it to money, total the costs, list the intangibles and calculate the ROI.
- Report to every stakeholder group, and use what the evaluation found to change the programme.
Steps 4 and 5 produce the two documents ROI Basics names: the data collection plan and the ROI analysis plan. Both should be signed off before the first learner enrols, because a comparison group cannot be added later.
Choosing an isolation method before launch
ROI Institute lists several ways to separate a programme's effect from other influences. They differ most in how much has to be arranged before the programme starts.
| Method | How it works | Use it when | Watch for |
|---|---|---|---|
| Control group | Compare trained people with similar untrained people over the same period | You can stagger a rollout or hold a group back | Groups that differ in role, tenure, location or targets |
| Trend line | Project the measure's pre-programme trend and compare actual results with it | The measure has a stable history and nothing else changed | New products, prices or policies launched in the same period |
| Forecasting | Model the measure from the factors known to drive it | A known relationship exists, such as sales and market size | Models that leave out a major driver |
| Participant estimates | People estimate the programme's share of the change and their confidence in that estimate | No comparison is possible after the event | Estimates must be adjusted for confidence and reported as estimates |
| Manager or expert estimates | Supervisors or experts estimate the share, using what they know of other factors | Managers see the work closely | The same risk of over-crediting, from a different angle |
ROI Institute's 2020 benchmarking summary recommends that estimates "be collected during every study, but only analyzed and included as an isolation method when more credible methods cannot be used." In the invoice-matching plan, Office B is the control group for four weeks, and the clerks' and supervisors' estimates are collected anyway as the fallback.
A 2021 study of a sales course at a Portuguese cosmetics distributor shows what happens without this step. Curado and Sousa credited the whole year-on-year rise in product sales, EUR 138,590 of revenue, to a two-day course for 53 staff. They valued both years at the earlier year's prices and noted there were no marketing campaigns, but there was no comparison group, and one of the course's stated goals was to clear old stock, which can lift sales by itself.
Once everyone has been trained, the only isolation methods left are trend lines, models and estimates. If a staggered rollout is possible, decide who goes first while the schedule is still open, and record how the two groups compare on the factors that drive the measure.

Case study: Compliance officer training at Idaho National Laboratory, United States
Idaho National Laboratory, a US Department of Energy laboratory, appoints compliance officers and coordinators as an extra duty to screen condition reports for regulatory noncompliance. The programme manager revised their eight-hour instructor-led course and wanted evidence to support more recognition for the role.
- What they did: the team followed the ROI Methodology from Level 0 to Level 5 with a data collection plan, an ROI analysis plan and the 12 guiding principles. Benefits came from comparing the laboratory's regulatory penalties and investigation costs for 2022 to 2024 with six similar laboratories. Isolation used participant estimates: 95% of the screening process involves only these officers, so 95% of the benefit was attributed to the role, then adjusted by participants' 83% confidence.
- Evidence and result: adjusted benefits of $472,985 against fully loaded costs of $404,976 gave a benefit-cost ratio of 1.168 and an ROI of 16.8%, against a target of 0%, as the team presented to the Energy Facility Contractors Group's training working group in March 2025.
- Limitation: the source is a conference slide deck, not a peer-reviewed paper, and it does not say how many officers were trained. Role salaries make up about 86% of the costs, so the figure values the role as much as the course, and having no penalties while the two most similar laboratories averaged four could have other causes.
- Lesson: decide whether you are evaluating a course or a job before you build either side of the calculation. The team's own advice was to plan the isolation method earlier, because a late ROI analysis plan forced weaker data collection.
Choosing a conversion method
Conversion only happens when the evaluation goes to Level 5. ROI Institute's methodology page lists the options: standard values for output, quality and employee time; historical costs and records; internal and external experts; external databases; estimates from participants, managers or HR staff; and links to other measures that already have a value.
The 2009 workshop gives a quick test for whether to convert at all: "Can we convince our executive in two minutes that the value is credible?" If not, report the measure as an intangible benefit. In the invoice-matching programme:
- Exceptions: finance already has a standard cost for handling one exception, so the conversion is a lookup.
- Rework hours: each hour is valued at the clerks' hourly pay plus benefits, from HR.
- Late-payment penalties: already money, so no conversion is needed.
- Supplier relationships and clerk confidence: no credible value in two minutes, so they are reported as intangibles.
The guiding principles as planning rules
The 12 guiding principles in ROI Basics are usually printed as a list. They are more useful as decisions you make while planning. Six of them shape the invoice-matching plan directly.
| Principle | What it means when planning | In the invoice-matching plan |
|---|---|---|
| Use at least one method to isolate the effects | Choose the isolation method before launch | Office B as the comparison group, estimates as a fallback |
| Use only the most credible sources | Name the source for each measure in advance | Finance system reports, not clerks' impressions of workload |
| If no improvement data are available, assume little or no improvement | Missing data count as zero, so plan to collect them | If a clerk returns no estimate in the fallback, that clerk's share counts as zero |
| Adjust estimates for potential errors of estimation | Ask for a confidence figure with every estimate | The action plan asks clerks for a contribution and a confidence percentage |
| Use only the first year of annual benefits for short-term solutions | Do not project savings over the system's lifetime | Benefits counted for year one only |
| Fully load all costs | List every cost category before the programme runs | Includes needs analysis, supervisors' time and clerks' time away from the queue |
The evaluation plan, filled in for the invoice-matching programme
Here is the data collection plan for the hypothetical programme, using the objectives from each level.
| Level | Objective | Measure and method | Source | Timing | Responsible |
|---|---|---|---|---|---|
| 0. Input | All 40 clerks complete the course and practice session in their training window | Completion records, seat time, running cost log | Course platform and project budget | As the programme runs | Training coordinator |
| 1. Reaction and planned action | At least 4 out of 5 on relevance; two planned actions per clerk | Five end-of-session questions and a planned-action box | Clerks | End of the practice session | Training coordinator |
| 2. Learning | At least 8 of 10 exception scenarios resolved correctly; confidence rated | Scenario check with the documents supplied | Clerks, marked by supervisors | End of the practice session | Office supervisors |
| 3. Application and implementation | Correct code on at least 18 of 20 sampled exceptions; progress on both planned actions | Queue sample review and 30-day action plan | Supervisors, exception queue, clerks | Day 30 after training | Office supervisors |
| 4. Impact | Office A's exceptions and rework at least 25% lower than Office B's in weeks 1 to 4; fewer late-payment penalties over the 12 weeks after go-live | Finance system reports | Finance system | Weekly for 12 weeks from go-live | Accounts-payable manager |
| 5. ROI | Break-even or better on first-year benefits | Isolated benefits compared with fully loaded costs | Finance director and HR cost data | at week 12, separating observed benefits from any first-year forecast. Recheck savings after both offices have trained before projecting beyond the comparison period | Finance director |
And the ROI analysis plan for the three Level 4 measures:
| Data item | Isolation method | Conversion method | Other influences to record |
|---|---|---|---|
| Exceptions per 100 invoices | Office B as comparison group in weeks 1 to 4; clerk and supervisor estimates as a fallback | Finance's standard cost per exception | Invoice volume, supplier mix, system fixes released by the vendor |
| Rework hours | Office B as comparison group in weeks 1 to 4 | Clerks' hourly pay plus benefits | Overtime rules, staff absences |
| Late-payment penalties | Office A against Office B in weeks 1 to 4 where penalties occur; otherwise clerk and supervisor estimates with confidence. Year-ago figures cannot separate the training from the new system. | Already money | Changes in supplier payment terms |
The costs to capture are the needs analysis (prorated), course design and build, supervisors' time running the practice sessions, clerks' time away from the queue and the evaluation itself. The intangibles to report are supplier relationships, clerk confidence and fewer escalations to the accounts-payable manager. The results go to the finance director, the accounts-payable manager, both office supervisors and the clerks.
A Word template with a data collection plan and an ROI analysis plan, filled in for the invoice-matching example, plus blank versions for your own programme. Adapted from ROI Institute's planning documents.
Reporting the results, including a negative ROI
The twelfth principle says to communicate the results "to all key stakeholders", and that includes the people who took the course. Report the levels in order, with the isolation method next to every business figure and the intangibles next to the ROI. ROI Basics also makes this point: "more can be learned through evaluation projects that achieve a negative ROI than those achieving a high, positive ROI." A weak result at Level 3 is a list of changes to the programme, not a reason to hide the study.
How often courses carry a Level 1 or Level 3 instrument
Two Phillips instruments can live inside the course or straight after it: a planned-action question at Level 1 and an action plan someone checks at Level 3. I wanted to know how often those instruments are built into courses, so I looked at 7,940 courses published on Coursebox in the 90 days to 16 November 2026, from 2,650 creators, with staff accounts and bots excluded. For each course I checked whether its latest published version contained a survey block, an assignment or a quiz.
The full picture: 57 courses (0.7%) carried any survey, 1,249 (15.7%) an assignment and 2,883 (36.3%) a quiz, while 4,889 (61.6%) carried none of the three. Quiz blocks, which can carry a Level 2 check, were more common than survey and assignment blocks in these published courses. The count does not show how often teams collected evidence outside Coursebox.
These figures have limits. A block in a course does not mean anyone used it to evaluate the programme, quizzes and assignments may be graded content rather than evaluation, and the survey count may be incomplete. Reaction surveys and follow-ups can also run outside the course, in an LMS, a form tool or a manager's calendar, which this count cannot see. The practical point stands either way: if your plan needs a planned-action question and a 30-day action plan, decide where each one lives and who reads the answers before the course goes live.
Phillips ROI model vs the Kirkpatrick model
The Kirkpatrick model evaluates training at four levels: reaction, learning, behaviour and results. The Phillips ROI model shares that four-level idea and adds a Level 0 for inputs, a fifth level for ROI, an isolation step before business results are credited, and a set of standards for the analysis.
A common error is to call ROI "Kirkpatrick Level 5". The fifth level is Phillips's, and Kirkpatrick Partners take a different view of financial return. Their current model page says "results are rarely caused by a single program or event" and that, when financial return is needed, contributive ROI "offers a more credible approach than trying to isolate training as the sole cause." Our Kirkpatrick model guide covers the four levels and that approach in detail.
| Question | Phillips ROI Methodology | Kirkpatrick Model |
|---|---|---|
| Levels | Level 0 input plus reaction and planned action, learning, application, impact and ROI | Reaction, learning, behaviour and results |
| Best for | Expensive or high-profile programmes where a budget holder wants a monetary answer | Most programmes, where agreeing the expected results with stakeholders is the main job |
| Weak spot | Needs a credible isolation method and cost data, which take planning and time | Says less about separating training from other causes or putting a value on results |
| Use it when | A decision depends on whether benefits exceed costs | You need evidence of learning, behaviour change and results without a monetary figure |
Two other models are useful when neither fits. The CIPP model evaluates context, inputs, process and product, which suits programme reviews and redesigns. Brinkerhoff's Success Case Method examines the people who used the training most and least, which is faster and tells a more human story. For the general evaluation workflow, start with how to evaluate a training programme.
Limitations of the Phillips ROI model
Critics raise four main objections:
- Isolation often rests on estimates. When no comparison group exists, participants estimate the programme's share. A 2017 paper in Acta Commercii notes that the trainee is asked to "(subjectively) estimate" that contribution. Josh Bersin put it bluntly in Chief Learning Officer: "you really don't know if it was the training that caused it."
- Conversion depends on assumptions. Robert Brinkerhoff, in the same Chief Learning Officer piece, said he would be nervous taking ROI data to a board "because so much of it is based on some pretty shaky assumptions".
- Not every benefit converts to money. The Acta Commercii authors warn that managers "focusing only on the financial results may overlook other (difficult to quantify) benefits". Phillips's answer is to report those benefits as intangibles beside the ROI.
- ROI is often left until after the event. Consultant Paul Kearns, writing on TrainingZone in 2007, argued that the five-level model "puts ROI at the end, after the event, not at the beginning". Kearns is a rival consultant, and the Phillips planning model does set ROI objectives at the start, but it describes a risk when the ROI analysis plan is written late.
The Institute's own benchmarking shows how often the strongest isolation method is used in practice.
That figure comes from organisations committed enough to the method to take part in its benchmarking, and the same benchmarking summary adds that estimates are the fallback, with participant estimates used most. So about two in three studies by the method's own users do not use a control group. Read any published ROI with its isolation method next to it.
Using the Phillips model with an AI course tool
I tested the invoice-matching programme in Coursebox. In one prompt I asked for a course for the 40 clerks, 90 minutes online plus the one-hour supervised practice session, with the Level 1, 2 and 3 instruments built in, and asked it to tell me what it could not build. The outline had a page for each: a five-minute reaction check, a scenario check with four complete sets of documents, and an action plan. Before I opened a page, the builder told me it could not schedule or track the 30-day supervisor review, and that it had used provisional exception codes because I had not supplied the company's own list. Both are jobs for a person before delivery.
The reaction check was the weak page. Only the planned-action question was a real answer box. The other four questions were printed as text with blank lines, so a learner could not answer them in the course. I asked the AI editor to turn all five into ungraded survey questions. Its reply said it had removed the grading, but when I reloaded the page the grading criteria were still on the question. A reply saying something changed is not proof that it did, so check the page itself. I added and saved one Star Rating question by hand through a Survey block, which took about a minute. The other ratings would need the same manual setup; I did not verify a completed five-question survey.
The Level 3 action plan was a good Assignment page, with sections for first-week behaviour, the matching steps, escalation and a 30-day supervisor review. It was set to "AI marks instantly" with an 80% pass mark, and 20% of the score sat on the supervisor section, which a clerk cannot fill in before go-live. I would switch it to supervisor review at 30 days. Levels 4 and 5 sit outside any course platform: the finance measures, isolation, conversion and costs belong in the evaluation plan.
A Word pack with planned-action questions for the end of a course and a 30-day action plan with contribution and confidence estimates, filled in for the invoice-matching example, plus blank versions.
Putting the Phillips ROI model to work
The Phillips ROI model is most useful as a planning discipline, and the ROI figure is the smallest part of it. Use it in full for the few programmes where someone with a budget decision is waiting for a monetary answer, and for the rest, write an objective at every level but collect data only as far as the decision needs, usually reaction and learning, and application for the programmes that count. For your next significant programme, write the objective for each level on one page and choose the isolation method before the schedule is fixed. If you cannot hold back a comparison group, plan for estimates with confidence adjustments and say so in the report.
Frequently Asked Questions
The Phillips ROI model, which ROI Institute calls the ROI Methodology, is Jack Phillips's framework for evaluating a programme from its inputs through reaction, learning, application and business impact to a return on investment, after isolating the programme's own effect. Compared with the older four-level idea, it adds a Level 0 for inputs, an isolation step and twelve guiding principles that keep the analysis conservative.
The five levels are reaction and planned action, learning, application and implementation, impact, and return on investment. ROI Institute also records Level 0, the inputs such as attendance, hours and cost, but does not count it as a level of results.
The Kirkpatrick model has four levels: reaction, learning, behaviour and results. The Phillips model builds on the same idea and adds a fifth level for ROI, a Level 0 for inputs, a step that isolates the programme's effect from other influences, and standards for converting results to money.
No. ROI Institute recommends taking about 5% of programmes to ROI and about 10% to business impact each year. Candidates are expensive, long-running, high-profile or wide-reaching programmes, or ones where a senior manager has asked for a monetary answer.
The most credible way is a comparison group of similar people who have not yet been trained, measured over the same period. Where that is impossible, trend line analysis, forecasting models or estimates from participants and managers are used, with estimates adjusted for the estimator's confidence.

Carolina Martin
Carolina is a highly experienced learning designer with tech start-up experience. She is based in Sydney and previously designed courses alongside Coursebox founder Travis in a previous job. She is passionate about delivering engaging learning experiences and brings the same perspective to customer experience too. She manages our customer support documentation and product testing processes.


