The Success Case Method of Evaluating the Impact of Performance Interventions

Overview

What makes training work has more to do with the performance system than with the training design and delivery itself. The difference is the manner in which the company (or business unit) uses the training, as well as the influence of the prevailing systemic factors such as work habits, reward systems, preparedness of learners, measures, and feedback procedures, to name a few.
(Brinkerhoff, 2003, ¶3)

Following any performance project, the impact of the interventions must be evaluated and communicated to clients. It is preferable if this can be done in a way that focuses on meaningful business results and increases support for performance interventions. The Success Case Method (SCM) is an evaluation model that offers the following benefits:

Unlike other forms of evaluation, the SCM does not apply complex statistical analyses to identify average performance, but rather highlights factual accounts of the best results the program is capable of achieving and explains results to clients in meaningful business terms.

SCM follows this basic process:

  1. Define a clear purpose for the study, identify key stakeholders and their interests, design a plan for the study including time limits and availability of other resources.
  2. Define what success should look like by focusing on desired performance results.
  3. Design and administer a brief survey to targeted performers including items related to use of new methods and tools form the intervention and their results.
  4. Analyze survey results and select a random sample of success and non-success cases.
  5. Conduct one-to-one interviews with success cases to determine specifics of intervention use and results, perceived value of the results, and environmental supports that enabled success.
  6. Conduct one-to-one interviews with non-success cases to determine why they weren’t able to apply or benefit from the intervention and identify barriers to success.
  7. Analyze data and select the most compelling successes.
  8. Present conclusions and recommendations to client in the form of detailed stories (Barrington, 2004; Brinkerhoff & Dressler, 2003)

Perhaps the greatest value of this method is that it assists HPT practitioners in linking the results of interventions to the bottom-line through answers to the questions:

Case Studies

Compaq

Background:
The organization offered a two-week training course to help technicians install and service components of a new product. Despite the high demand for the course driven by business needs, rumors were circulating about the inadequacy of the training

Evaluation Plan:
Survey trainees who had completed the course within the past 12 months with a 2-item questionnaire. Randomly sample success and non-success cases for further interview.

Results:
SCM findings showed that the training program was successfully applied to meet customer needs by 60% of the trainees. The other 40% of trainees reported that they had never used what they had learned from the course. This may be due to the fact that many technicians who did not have customers with the new product had completed the course. The course enrollment process was redesigned and ROI and business impact from the course increased.

Global telecommunications company

Background:
In an extremely competitive market, top sales representatives were being recruited to other companies and annual sales growth goals had been set relatively low compared to the actual growth that had been achieved in a very short time.  A sample of sales leadership positions completed an exercise in which they identified what they needed to do to help the company succeed. One-on-one coaching sessions were then provided to every sales leader, focusing on supporting their plans and successes.

Evaluation Plan:
Ninety-five percent of participants responded to a 7-item survey from which a random sample of success and non-success cases were selected for further interview.

Results:
Successes: The coaching process influenced several leaders to remain with the company, including the top performer. The coaching program was supported at the executive level. The company’s rate charges remained competitive. Non-success: Many participants reported a lack of purpose for the coaching program.

The organization made a long-term commitment to staff development. Support for the coaching program increased as stories of its success spread and retention of top sales talent increased.

Consultants and Resources

Robert Brinkerhoff

Selected Publications:

References

Barrington, G. V. (2004). The success case method. Retrieved December 8, 2004 from http://www.barringtonresearchgrp.com/%20docs/Success%20Case%20Method%20Handout.pdf

Brinkerhoff, D. O. (2003, June). The success case method. PerformanceXpress. Retrieved December 6, 2004 from http://www.performancexpress.org/0306/jun03printercopy.html

Brinkerhoff, D. O. & Dressler, D. E. (2003, May). Using the success case impact evaluation method to enhance training value and impact. Handout presented at the American Society of Training and Development international Conference and Exhibition, San Diego, CA. Retrieved December 2, 2004 from http://www1.astd.org/astdInterim0304/pdf/handouts/TU313.pdf