Criterion-Referenced Instruction: Mager's Method Explained

Go from storyboard to an interactive course in minutes

Start Free

Take a hypothetical bank. Its new call-centre agents finish a two-day induction, pass the end-of-course quiz and start taking calls. In their first week, a supervisor hears one of them confirm a caller's balance before checking who they are. The agent knew the rule; they had answered the quiz question about it correctly. Nobody had ever checked whether they could follow it on a live call with an impatient customer.

Criterion-referenced instruction (CRI) is Robert Mager's answer to that gap: decide what people must be able to do on the job, define how well, test exactly that, and teach only what they cannot do yet. I build skill checks for courses most weeks as an instructional designer at Coursebox, where we make an AI LMS with pass marks and progress tracking.

What is criterion-referenced instruction?

Criterion-referenced instruction is an approach to training design in which objectives come from job performance, each objective has a clear standard, and learners practise and are tested until they meet that standard. Mager developed it with Peter Pipe out of his work on programmed instruction in the 1960s, starting from the objectives method in his book Preparing Instructional Objectives. A Mager objective for our example reads: given a live inbound call and the approved question list, the agent verifies the caller's identity before disclosing any account information, on every call.

Greg Kearsley's long-standing summary on instructionaldesign.org sets out five principles:

1. Objectives come from job performance and reflect the knowledge and skills people need.
2. Learners study and practise only the skills they have not yet mastered to the level the objectives require.
3. Learners practise each objective and get feedback.
4. Skills used often, or that are hard to learn, get repeated practice.
5. Learners can choose their own sequence within the limits set by prerequisites, and their progress depends on their competence.

In practice that becomes four pieces of design work: a task analysis; objectives that state performance, conditions and criterion; a skill check for each objective; and modules that get people through the checks.

Case study: Garment-technician training at TAL Apparel

The Hong Kong Management Association’s 2016 award entry describes TAL Apparel's use of Mager's CRI to standardise factory skills training. Designers observed production work and consulted managers, senior staff and technical experts.

Its garment-technician programme comprised 28 modules and 200 learning hours, including 85 hours of production-line practice. Pre- and post-training skill checks and a workplace application project assessed the objectives. Senior staff became onsite instructors; materials were translated into six languages.

The programme received the skills-training Bronze Award. The entry describes the design but gives no quantified effectiveness results; the award is not evidence of a measured productivity gain.

Practical lesson: analyse the job, then align objectives, skill checks and workplace application. A classroom quiz alone would not capture this programme's intended performance.

Where CRI came from

The Varian finding is reported in a 1982 Training and Development Journal article, and Mager's dates come from the Learning Guild's obituary. The Mager Consortium describes its CRI workshop as self-paced, taking 5 to 7 full days.

Criterion-referenced vs norm-referenced

A norm-referenced result tells you where someone stands compared with other people. A criterion-referenced result tells you whether they meet a defined standard. Robert Glaser drew this line in 1963, and it is the core of CRI.

In workplace training, the difference is practical. The best agent in a cohort can still disclose an account balance to the wrong caller. Their rank says nothing about whether the call was safe.

CRI step by step, worked through one course

This hypothetical course prepares new bank call-centre agents to verify a caller's identity before discussing an account. Agents have completed a policy quiz, but still miss required checks during calls. The redesign gives them practice calls using the bank's approved verification procedure, followed by a performance check. That gap between quiz results and job performance is why CRI fits this example. The standards below are illustrative, not a bank's actual security rules.

1. Analyse the job, not the topic

Start by watching or listening to the work. What do good agents do on a call before they discuss an account? Where do new agents go wrong? Mager and Pipe also ask whether training is the answer at all: if agents skip verification because the screen is slow, a course will not fix it.

• Task: verify the caller's identity before discussing account information, handle callers who fail verification, and record the result.
• Skill hierarchy: knowing which questions are approved comes before choosing a factor, which comes before handling a failed check.

2. Write objectives with performance, conditions and criterion

Mager's objectives have three parts. The mistake I see most often is mixing up conditions and criterion: a required completion speed or accuracy level is a criterion when it determines acceptable performance. Conditions specify the situation, tools and assistance.

3. Build the skill check before the lesson

The skill check asks learners to do the performance under the stated conditions, and judges it against the criterion. For verification, that is three simulated calls: a routine caller, one who fails verification and one who pushes for details.

• Essential criteria: no account information before verification; only approved questions; a failed caller handled without revealing which answer was wrong. Missing any one means not yet competent, whatever else went well.
• For this illustrative check, optional refinements to tone, greeting and call length receive feedback. They are not universally minor: make any behaviour required by policy, accessibility or the performance objective an essential criterion.
• Calibrate the assessors: before using the check, two assessors judge the same sample calls and fix any criterion they read differently.

Here is how the hypothetical skill check could work. Confirm the exact bank policy before using these illustrative criteria.

A pass on these cases supports the stated decision under the tested conditions; it cannot guarantee correct behaviour on every future call. Select enough varied cases for the task's risk and retain the assessor's evidence. Knowledge quizzes can test knowledge objectives, but cannot substitute for observing a call when the objective is call performance.

Case study: Proficiency-based epidural training in Ireland

An Irish randomised study published in 2018 compared proficiency-based progression with simulation-only training. This is a related approach, not a study of Mager's CRI by name. Experts helped define 74 performance metrics; assessment reliability was 0.88.

Fourteen trainees completed the study: six in the proficiency group and eight in the comparison group. Their first ten clinical procedures each supplied 140 procedure records. Failure occurred in 13.3% versus 28.7%, respectively (p = 0.04). These are procedure outcomes, not percentages of trainees who failed.

The small trainee sample limits generalisation. The comparison was simulation-only training, rather than an untrained group.

Practical lesson: define and calibrate the benchmark before using it to govern progression. Report both the learner sample and the performance opportunities observed.

4. Let people skip what they can already do

Use the skill check as a pre-test. An agent moving from another bank's call centre who passes an equivalent check against this bank's current policy can skip the module if required training rules permit it. Experience at another bank alone is not enough. In my experience it is the step teams skip, and it is the part that saves experienced people the most time.

In our support inbox, this shows up as two opposite requests. About ten training teams asked how to stop learners moving on until they pass; about seven asked how to let advanced learners skip ahead. CRI needs both: a clear standard, and a way past the content for people who already meet it.

5. Practise, give feedback, repeat

Each objective gets practice, with feedback that points to the criterion the learner missed. Skills that are used often or are hard, like handling a caller who fails verification under pressure, get more practice.

Retries need the same care. With three questions and an instant retry, a pass on a repeat attempt does not establish the whole skill or explain why the result improved. If the criterion is important, use more items or a fresh set of cases for the retry, and send the learner back to the matching practice first.

6. Record the result against the criterion

Record competent or not yet competent, which criterion was missed, the date and the version of the check. "Completed module" does not tell a supervisor or auditor what was demonstrated.

Does criterion-referenced training work?

None of these studies tested Mager's model by name. They investigate related mastery or proficiency-based progression approaches. Those share practice-to-standard principles with CRI, but their outcomes are not proof that Mager's complete method caused the gains.

The Mazzone and colleagues review supports the value of proficiency-based progression in its included settings. It does not show that a default quiz pass mark, a CRI-labelled outline or every training programme will produce the same reduction.

Case study: Simulation mastery learning in a Chicago intensive care unit

A 2009 Northwestern teaching-hospital study followed 92 residents who completed simulation-based mastery learning before inserting central venous catheters. This is evidence for a related practice-to-standard approach, not Mager's full method.

Catheter-related bloodstream infections in the intervention unit fell from 3.20 to 0.50 per 1,000 catheter-days (p = 0.001). Another ICU in the same hospital provided an additional comparison; allocation was not randomised.

A 2010 cost analysis estimated net annual savings above US$700,000 and a 7:1 return using historical costs, including infection costs expressed in 2008 dollars. These are setting-specific estimates, not a current forecast.

Practical lesson: assess competence before independent work, then examine the workplace outcome. Keep observational results and estimated financial benefits distinct from causal proof.

Case study: Proficiency-based utility-work training at ReachActive

A 2020 ReachActive study examined proficiency-based simulation training for locating and excavating buried utilities. Workers progressed after meeting performance benchmarks. This related approach shares CRI's emphasis on standards.

The randomised comparison of 12 workers reported 33% fewer performance errors. After wider rollout, the observational strike rate per 10,000 work hours was 35% lower than in 2016 and 61% lower than in 2017. Those are two different baseline years, not an uncertainty range.

The rollout result was not itself a randomised comparison. Several authors worked for ReachActive or the training provider.

Practical lesson: separate skill-check results from workplace results, and use a consistent exposure denominator when comparing safety outcomes.

Building a CRI course in Coursebox

I asked Coursebox, the course builder we make, for a course for new bank call-centre agents on verifying a caller's identity, using criterion-referenced instruction, with a skill check per objective and a pre-test so experienced agents could skip what they already know.

The outline was closer to CRI than I expected: 9 sections and 44 pages, opening with a readiness pre-test, a skill check closing most sections, a live simulation and a final performance check. Its course description even said experienced agents "may test out of demonstrated skills". This was an outline test, not a test of a delivered course or learner performance.

Two finishing steps turn that outline into a working CRI course.

• Let experienced agents skip what they can do. I did not find a per-module test-out setting in the workflow I inspected. The workaround is to split the modules into short separate courses, use the pre-test results to decide each agent's enrolments, and set a pass mark on each skill check. Experienced agents then take only the modules they need, which is the point of a CRI pre-test.
• Make the key checks performance checks. Most of the skill checks were quizzes, which show whether an agent knows the rule. For the most important objectives, I would swap the quiz for an interactive scenario, where the agent handles a simulated caller, or for an observed call, so the check matches the behaviour in the objective.

Limitations of CRI

• Too many tiny objectives. In his critique of criterion-referenced measurement, W. James Popham argues that attempting to secure mastery of hundreds of objectives undermined the behavioural-objectives movement. Keep objectives large enough to represent meaningful performance.
• More learner time. A 2013 review of simulation mastery learning found better skills results than non-mastery simulation in three comparative studies, but training took longer. This is related healthcare evidence; budget for additional practice without assuming a universal time penalty for every CRI course.
• Learner control needs a purpose. A meta-analysis of web-based instruction found benefits varied by the kind of control, outcome and training context. This does not mean self-paced learning generally fails. Offer supported choices within prerequisites and let people bypass only what they can demonstrate.
• Clear criteria can still measure the wrong thing. A checklist can reward every scripted line in a difficult conversation and miss whether the agent listened. For judgement-heavy work, add expert judgement to the criteria.

CRI and related approaches

If you are writing objectives for the first time, our guide to measurable learning outcomes has more examples, and SMART learning objectives covers a different format many teams use for goals. When you check whether your skill checks measure what they claim to, assessment validity is the next thing to read.

Make the performance standard the organising decision

Criterion-referenced instruction is useful when completion needs to mean that a learner can perform a defined task to the required standard. The criterion guides assessment, practice and progress. It also lets experienced learners demonstrate what they already know, where job policy and prerequisites allow, instead of repeating training simply because everyone else is taking it.

Choose a meaningful task, confirm the current performance standard with its owner and build a skill check that reflects the actual conditions. Use the result to assign relevant practice and feedback, then reassess. For caller verification, that means observing the required decisions and actions in a suitable call scenario. A course pass should represent evidence of the skill under those conditions, with workplace follow-up where needed.