When implementing a training scorecard it is important to track, collect, compile, analyze, and report six different types of training data collected over different time periods. These types of data are indicators, reaction, learning, application, business impact, and return-on-investment.
Indicators. This is the traditional approach to reporting training data. Some examples of indicators are number of employees trained, total training hours, training hours per employee, training investment as a percentage of payroll, cost per participant. Although these measures are necessary, they do not reflect the results of the training program. There are many types of indicators, but it is most important to include in the scorecard the measures of interest to the organization’s top managers.
Level 1: REACTION. At this level, participants’ reactions to and satisfaction with the training program are measured. Some recommended data to capture on Level 1 instruments are:
- Relevance of training to job
- Recommendation of training to others
- Importance of information received
- Intention to use skills / knowledge acquired
Those four items have predictive validity for projecting actual applications and should be compared from one program to another.
Level 2: LEARNING. Learning can be measured informally with self-assessments, team assessments, or facilitator assessments, or formally with objective tests, performance testing, or simulations. Learning self-assessments may ask participants to rate the following items:
- Understanding of the skills/knowledge acquired
- Ability to use the skills/knowledge acquired
- Confidence in the use of skills/knowledge acquired
Level 3: BEHAVIOR APPLICATION. This level measures changes in on-the-job behavior while the training is applied or implemented. This information often is collected through a follow-up survey or questionnaire. Key questions asked concern:
- The importance of the skills/knowledge hack on the job
- The frequency of use of the new skills/knowledge
- The effectiveness of the skills/knowledge when applied on the job
Level 4: BUSINESS IMPACT. At this level the actual business results of the training program are identified. A paper-based or automated follow-up questionnaire can be used to gather this data. Depending on the training programs’ performance and business objectives, data may be gathered on the following:
- Productivity level
- Cost control
- Sales revenue
- Customer satisfaction
Level 5: RETURN ON INVESTMENT. At this level the monetary benefits of the program are compared with the cost of the program. The costs of the program must be fully loaded. The methods used to convert data should be reported. The ROI calculation for a training program is identical to the ROI ratio for any other business investment:
ROI(%) = ((benefits – costs] / costs) x 100
A benefit-cost ratio may also be calculated by dividing costs into benefits.
INTANGIBLE BENEFITS. Intangible benefits are measures that are intentionally not converted to monetary values because the conversion to monetary data would be too subjective. It is important to capture and report intangible benefits of the training program, such as:
- Increased job satisfaction
- Reduced conflicts
- Reduced stress
- Improved teamwork
Source: Jack J. Phillip and Lynn Schmidt, Implementing Training Scorecards, ASTD Publication.