April 17, 1995Minnesota Office of the Legislative Auditor
Program Evaluation Division
A best practices review is valuable insofar as its ideas prove useful to a broad range of practitioners. We hope that the examples presented here give state managers and training coordinators new ideas that they may adopt cost-effectively in their own agencies.
While a best practices review collects and highlights evidence of success in the design and delivery of services, it is not a full-fledged evaluation. It does not, for example, identify deficiencies in program design or implementation. Rather than focusing on the weaknesses in agencies' training activities, our best practices review of state employee training identifies the characteristics of effective training and presents concrete examples of training practices currently being used that are likely to contribute to an effective workforce.
Specifically, this review has two main objectives:
We focused our research on 21 state agencies. Using focus groups, in-depth interviews with agency representatives, and surveys of state employees and agency training coordinators, we gathered information about ideal training methods as well as actual practices.
For the purposes of this review, we defined employee training as follows:
Employee training is the process of identifying, assuring, and developing, through planned activities, the knowledge, skills, and abilities that employees need to help them perform their current and future job responsibilities in state agencies to the greatest extent possible.
This definition recognizes that training is more than a single learning event and focuses on activities that are directly related to agencies' goals and objectives. It also suggests that training may include a variety of activities, such as formal classes, professional conferences, and structured self study, that may be sponsored or provided by agencies in-house, other state agencies, and other public and private vendors.
In Minnesota, training for state employees is provided largely by individual agencies. Although the Department of Employee Relations is responsible for developing and coordinating consistent training policy across state agencies, each appointing authority is primarily responsible for planning, budgeting, conducting, and evaluating training programs. In reviewing how agencies ensure that their employees keep their skills up-to-date, we found that:
Consequently, there is no statewide information on employee training costs or activities. We asked training coordinators in 21 agencies how much money their agency as a whole spent on training their employees during fiscal year 1994. Of the 19 training coordinators who returned our survey, 12 could give us complete spending data. Their responses, which represent about 65 percent of the state employees in our sample, indicate that:
We estimate that some agencies spent around 0.3 percent of their payroll on employee training in fiscal year 1994 while another spent approximately 1.6 percent. Agency spending was from $34 to $582 per employee. We estimate that an average of $287 (or a median of $331) was spent per employee.
The amount of training that employees receive that is paid for or provided by an agency and the helpfulness of that training are also valuable indicators of agencies' commitment to training. We asked a representative sample of state employees in 21 agencies how many hours of training they completed during fiscal year 1994 that was paid for or provided by their agencies. We found that:
Also, we found that:
Survey respondents reported that they took an average of 33 hours of training during fiscal year 1994, with a median response of 26 hours. This is similar to training figures reported by some private businesses. According to the American Society for Training and Development's survey of 19 major U.S. corporations, most companies reported providing between 16 and 56 hours of training per employee per year, with a typical value of 27 hours. However, it is less training than employees in some corporations who are past winners of the prestigious Malcolm Baldrige Award for Quality receive.
Most of the state employees who said that they had taken some training during fiscal year 1994 reported that the training they received was helpful to performing their current job responsibilities. Moreover, most agreed that their training helped their agency achieve its overall goals and objectives.
Training is a dynamic process made up of four major components: planning and support, needs assessment, methods and activities, and evaluation and feedback. Training experts, including many within state agencies, suggest that agencies should continuously plan for and support training that is linked to their mission, goals and objectives; assess current and future training needs of all employees; ensure that appropriate training activities are provided; and evaluate and utilize the results. Our model of training principles emphasizes these four elements, which we discuss below.
1. Develop policies communicating the significance of training.
2. Develop training plans.
3. Set minimum training requirements for all employees.
4. Develop procedures to implement training policies.
5. Communicate training information to all employees.
6. Support training with funds and staff.
7. Ensure employees responsible for training have skills.
First, we think that state agencies should use their training resources to support their mission, goals, and objectives. This requires agencies to have a clear vision of their roles and responsibilities and the human resources necessary to attain them. Although training is only one of many tools available to management, its link to agency performance should be clearly delineated, communicated, and supported at all levels within an organization. Seven principles, which are summarized below, demonstrate management support for training and help lay the necessary foundation for its success.
A Department of Corrections' practice illustrates how one agency has implemented various principles relating to planning and support. One of the department's seven core values states: "We value staff as our most valuable resource in accomplishing our mission, and we are committed to the personal welfare and professional development of each employee." The department demonstrates its commitment to this core value by requiring that all of its 2,900 employees participate in a minimum amount of training each year.
The Department of Natural Resources' Clerical Training Board also illustrates some principles related to planning and support. The board consists of 11 clerical staff (one from each region of the state and five from the central office) who serve two-year terms. The department solicits nominations and the department's training director makes the final selection of board members. The board is responsible for identifying, planning for, and designing training for nearly 500 support staff throughout the department. Each year, it prepares and submits a training plan for the commissioner's approval that establishes goals for the year and the strategy and tasks necessary to achieve them. At the same time, it maintains a prioritized, statewide list of clerical training needs in the department. During fiscal year 1994, the board had an operating budget of $10,000.
8. Identify agency's core competencies.
9. Identify competency gaps.
10. Monitor agency performance.
11. Involve employees in identifying training needs.
12. Develop individual training plans.
13. Identify training opportunities.
14. Set training priorities.
Second, our model of training principles suggests that training efforts should be targeted toward agencies' core skill requirements so that they can achieve their mission, goals, and objectives. To help ensure this, agencies should routinely identify deficiencies by comparing the competencies that they have identified as necessary to achieve their mission, goals, and objectives with employees' knowledge, skills, and abilities. In addition, performance should be routinely monitored to identify problems that may be amenable to training. The figure below summarizes the seven principles related to needs assessment.
The Minnesota Department of Transportation's succession planning project, which is a staffing and development system, illustrates many of these principles. The project aims to create a pool of talented employees who should be capable of moving into key management positions in the department by identifying and developing key competencies early. It compares the knowledge, skills, and abilities of current employees with the core competencies required for 54 key positions and provides opportunities for employees to acquire the competencies that they may be lacking.
In a similar fashion, the Department of Natural Resources' supervisory feedback project compares the core competencies of selected supervisory positions with supervisors' current knowledge, skills, and abilities by obtaining feedback from employees who are being supervised. The primary goal of this project, however, is to improve current performance rather than develop employees for future positions.
15. Ensure that training activities are relevant to agency goals.
16. Use a variety of training methods.
17. Provide training as efficiently as possible.
18. Make training accessible.
Third, according to our model, agencies should identify and provide access to training activities that narrow skill gaps and address performance problems identified in needs assessment. The four training principles summarized below encourage agencies to use diverse and efficient methods to provide training activities that are linked to their mission, goals, and objectives. This includes collaborating with other public agencies and the private sector to provide training as efficiently as possible.
For example, the Bemidji Intergovernmental Training Exchange is a coalition of city, county, and state governments working together to expand training opportunities in the Bemidji area. Through meetings and needs assessment surveys, the exchange identifies common training needs and provides for joint activities to address those needs. The exchange makes more training activities geographically accessible to its members and reduces travel expenses related to training.
The Department of Human Services ensures that training is geographically accessible by using telecommunications to provide training to its nearly 7,000 employees, most of whom work in the department's regional treatment centers located throughout the state. According to the department, alternative technology, such as videoconferencing and satellites, permits it to provide quality training at low cost to a widely-dispersed audience, saving the department both time and money.
19. Maintain agency-wide training records.
20. Link training to employee performance.
21. Evaluate the effectiveness of training.
22. Use evaluation results to modify training methods and activities.
23. Use evaluation results to monitor agency's plans and goals.
Finally, to help ensure the wise expenditure of public funds, we think that agencies should try to determine training's results by conducting systematic evaluations of training activities. Not only should evaluation measure participant reactions to individual training sessions, but it should also try to measure changes in learning, job behavior, and organizational performance whenever possible. Five principles related to evaluation and feedback are summarized in the figure below.
The Pollution Control Agency's hazardous waste inspector training program illustrates principles related to evaluation and feedback. The agency evaluates training results in three ways. It examines participant reactions by requiring trainees to answer a short questionnaire after completing the required reading modules that are a part of its training program. It measures participant learning by administering competency tests at the end of many of the modules that trainees must pass with a minimum score of 90 percent. Finally, it measures subsequent participant behavior by requiring participants to successfully complete a number of on-the-job activities.
Another example that illustrates evaluation and feedback principles is the learner-centered learning program in the Department of Revenue. This program, which provides technical staff with the knowledge, skills, and abilities to be trainers, involves two types of formal evaluations. First, participants complete an evaluation form after completing the training to gauge satisfaction. Then program staff measure changes in on-the-job behavior by resurveying participants one month later to see how well the class prepared them for working as a trainer and to determine if they were able to use what they learned on the job. In addition, staff informally measure learning by providing participants with opportunities to practice what they are being taught during the training.
This best practices review of state employee training has identified the processes that we think agencies should use to identify what training activities are needed, how those opportunities should be provided, and how results should be evaluated. Because we did not evaluate state agency training overall, we do not know how many state agencies may be using techniques consistent with our model.
Also, while we tried to present innovative examples of practices that are consistent with principles from our model, there are undoubtedly other practices occurring throughout state agencies that we did not review. We invite agencies to measure their own training efforts against our model and submit additional examples of best practices to us. If appropriate, we will update this report in the future.
The Program Evaluation Division was directed to conduct this study by the Legislative Audit Commission on June 15, 1994. For a copy of the full report, entitled "State Employee Training: A Best Practices Review," (95-05), 73 pp., published on April 17, 1995, you may use our order page. Alternatively, please call 651/296-4708, e-mail Legislative.Auditor@state.mn.us, or write to the Office of the Legislative Auditor, 658 Cedar St., St. Paul, MN 55155.
Staff who worked on this project were Jo Vos (project manager and chief author) and Susan Von Mosch, with assistance from Connie Reimer and Matt Bower (interns). For more information, contact Jo Vos