DOE-HDBK-1074-95, Training Program Handbook: A Systematic Approach to Training

TS NOT MEASUREMENT SENSITIVE DOE-HDBK-1074-95 January 1995 DOE HANDBOOK ALTERNATIVE SYSTEMATIC APPROACHES TO TRAINING U.S. Department of Energy FSC-6910 Washington, D.C. 20585 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. WELCOME This Portable Document Format (PDF) file contains bookmarks, thumbnails, and hyperlinks to help you navigate through the document. All items listed on the contents page are linked to the corresponding sections. In addition, if you click on a section heading while you are reading the text, you will return to the contents page. Click on the DOE seal below to move to the contents page. This document has been reproduced directly from the best available copy. Available to DOE and DOE contractors from the Office of Scientific and Technical Information, P.O. Box 62, Oak Ridge, TN 37831; (615) 576-8401. Available to the public from the U.S. Department of Commerce, Technology Administration, National Technical Information Service, Springfield, VA 22161; (703) 487-4650. Order No. DE95006851 DOE-HDBK-1074-95 TABLE OF CONTENTS PARAGRAPH PAGE FOREWORD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i SCOPE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii APPLICABLE DOCUMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v DEFINITIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1. GENERAL GUIDANCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 The Need for Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Effective Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.4 Alternative Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.5 Application of the Systematic Approach to Training . . . . . . . . . . . . 8 2. ANALYSIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.2 Selecting an Analysis Technique . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3 Types of Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.4 Selecting Tasks for Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.5 Elements of Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.6 Products of Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.7 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.8 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3. TRAINING DESIGN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.2 Techniques of Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.3 Elements of Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.4 Products of Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.5 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.6 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4. TRAINING DEVELOPMENT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4.2 Techniques for Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4.3 Elements of Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 DOE-HDBK-1074-95 4.4 Products of Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.5 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.6 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 5. TRAINING IMPLEMENTATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5.2 Techniques for Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5.3 Elements of Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 5.4 Products of Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 5.5 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 5.6 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 6. TRAINING EVALUATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 6.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 6.2 Methods of Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 6.3 Elements of Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 6.4 Products of Training Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 6.5 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 6.6 Conducting Training Evaluations . . . . . . . . . . . . . . . . . . . . . . . . . . 37 6.7 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 APPENDIX A FACTORS AFFECTING TECHNIQUE SELECTION . . . . . . . . . . . . . . . . . A-1 APPENDIX B SAMPLE TEMPLATE FOR DETERMINING SYSTEM KNOWLEDGE AND SKILLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1 APPENDIX C ON-THE-JOB TRAINING GUIDANCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1 APPENDIX D MINIMUM ELEMENTS FOR A SYSTEMATIC APPROACH TO TRAINING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-1 DOE-HDBK-1074-95

i FOREWORD The Alternative Systematic Approaches to Training handbook is approved for use by all DOE Components and contractors. The handbook was prepared primarily for DOE nuclear facilities, but the information can be effectively used by any other type of facility. DOE nuclear, DOE non-nuclear, commercial nuclear reactor, fuel fabrication, chemical processing, or other types of facilities may also apply the principles of this approach and find it useful and applicable to local needs. The handbook provides DOE and contractor operating organizations with concepts and guidance regarding the use of alternative techniques to implement a systematic approach to training (SAT). The techniques described in this handbook are endorsed by DOE and use of the guidance in this handbook is appropriate for establishment of technical training programs at DOE nuclear facilities. The use of guidance on selection and implementation of appropriate training approaches after consideration of job complexity, the consequences of error based on risk/hazard potential, and available training media should result in effective and efficient training programs. The information presented in this handbook can be used to grade the level of effort and formality used in developing training programs. The development of training programs by any method is inherently a graded approach. Once a method is chosen the level of activity required to develop training is directly related to the complexity of the job. Full application of traditional techniques without regard for the risk and hazard potential associated with performance may result in significant expense and effort that is not necessary. While selective application of appropriate SAT techniques is important, it should be noted that other factors such as the competency of the instructional developers, availability and quality of subject matter experts, current status of training, availability of operating procedures, and support of line management will impact the success of training programs regardless of the techniques used. DOE-HDBK-1074-95 ii Beneficial comments (recommendations, additions, deletions, and any pertinent supporting data) that may be of use in improving this document should be addressed to John A. Yoder EH-3.3/GTN U.S. Department of Energy Washington, D.C. 20585 Phone (301) 903-5650 Facsimile (301) 903-6172 by using the U.S. Department of Energy Standardization Document Improvement Proposal Form (DOE F 1300.3) appearing at the end of this document or by letter. DOE-HDBK-1074-95 iii SCOPE Alternative Systematic Approaches to Training applies to Department of Energy (DOE) personnel, DOE contractors, and others who conduct technical training for operation and support of DOE nuclear facilities. Selection of appropriate techniques for implementing a systematic approach to training (SAT) is discussed to help match the formality and effort of the training process with the training need(s), primarily on the basis of complexity, consequences of improper task performance, and hazard potential or risk. Involvement of technical experts, line management, and training personnel is necessary to establish effective training programs. DOE-HDBK-1074-95 iv Intentionally Blank DOE-HDBK-1074-95 v APPLICABLE DOCUMENTS 1. DOE/EP-0095, Guidelines for Job and Task Analysis for DOE Nuclear Facilities 2. DOE-HDBK-1078-94, Training Program Handbook: A Systematic Approach to Training.

3. DOE-NE-STD-1001-91, Guide to Good Practices for Training and Qualification of Instructors. 4. DOE-STD-1005-92, Guide to Good Practice for Developing Learning Objectives. 5. DOE-STD-1009-92, Guide to Good Practices for the Development of Test Items. 6. DOE-STD-1011-92, Guide to Good Practices for the Design, Development, and Implementation of Examinations. 7. DOE-STD-1012-92, Guide to Good Practices for On-the-Job Training. 8. DOE-STD-1070-94, Guidelines for Evaluation of Nuclear Facility Training Programs. 9. DOE-HDBK-1076-94, Guide to Good Practices for Table-Top Job Analysis. 10. DOE-HDBK-1086-95, Table-Top Training Program Design. 11. DOE Order 5480.23, Nuclear Safety Analysis Reports, of 4-10-92. 12. TG-17, Nuclear Information and Records Management Association Guidelines for Management of Nuclear Related Training Records. DOE-HDBK-1074-95 vi Intentionally Blank DOE-HDBK-1074-95 vii DEFINITIONS 1. Competency. The ability of a person to perform job responsibilities. 2. Graded Approach. A process by which the level of analysis, documentation, and actions necessary to comply with a requirement are commensurate with: (1) the relative importance to safety, safeguards, and security; (2) the magnitude of any hazard involved; (3) the life cycle stage of a facility; (4) the programmatic mission of a facility; (5) the particular characteristics of a facility; and (6) any other relevant factor. 3. Hazard. A source of danger (i.e., material, energy source, or operation) with the potential to cause illness, injury, or death to personnel or damage to a facility or to the environment (without regard to the likelihood or credibility of accident scenarios or consequence mitigation). 4. Hazard Categories. The consequences of unmitigated releases of radioactive and/or hazardous material are evaluated as required by DOE Order 5480.23 and classified by the following hazard categories: (1) The hazard analysis shows the potential for significant offsite consequences. (2) The hazard analysis shows the potential for significant onsite consequences. (3) The hazard analysis shows the potential for only significant localized consequences. 5. Job Analysis. A systematic method used in obtaining a detailed listing of the tasks of a specific job. 6. Operating Organization. The onsite contractor organization responsible for operation, maintenance, and technical support services related to operations. This organization may include offsite personnel who provide operational support. DOE-HDBK-1074-95 viii 7. Risk. The quantitative or qualitative expression of possible loss that considers both the probability that a hazard will cause harm and the consequences of that event. 8. Task. A well-defined unit of work having an identifiable beginning and end which is a measurable component of the duties and responsibilities of a specific job. 9. Task Analysis. The systematic process of examining a task to identify skills, knowledge, and/or abilities required for successful task performance. 10. Training. Instruction designed to develop or improve job performance. 11. Training Program. A planned, organized sequence of activities designed to prepare individuals to perform their jobs, to meet a specific position or classification need, and to maintain or improve their performance on the job. DOE-HDBK-1074-95 1 INTRODUCTION

This guidance has been developed to help organizations establish training programs that are systematic and effective regardless of the size, nuclear hazard classification level, complexity, or mission of the facility. Using alternatives to the more traditional methods of establishing systematic training programs can significantly reduce the time and effort associated with the training process. Alternative approaches streamline analysis, design, development, implementation, and evaluation of training materials and programs. Alternative delivery mechanisms such as structured self-study, computer-based training (CBT), or interactive video/multi-media should also be considered where appropriate. Techniques range from very simple to elaborate. The least elaborate techniques are typically used for the training of managers, the technical staff, and oversight personnel. For these positions, the training process may only require 1) an evaluation of the job to determine significant job requirements, 2) an evaluation of the education, experience, and prior training of job incumbents to identify deficiencies between job requirements and the individual's current qualifications, and 3) implementation of a plan for the individual to correct the identified deficiencies. The plan may include temporary rotational job assignments, mentoring, required reading, attendance at workshops, seminars, professional society meetings, and training on specific areas that are applicable to the job requirements. More elaborate techniques, typically necessary for higher risk jobs such as fissile material handlers, reactor operators and senior reactor operators, would normally involve some form of job and task analysis followed by development of detailed learning objectives, lesson plans, job performance measures (JPMs), etc. Regardless of the techniques used, a strong evaluation process is necessary to ensure effective implementation, timely updates, and periodic improvements. DOE-HDBK-1074-95 2 Intentionally Blank DOE-HDBK-1074-95 3 1. GENERAL GUIDANCE 1.1 Discussion The Department of Energy has ultimate responsibility for the safe, efficient, and economical operation of its facilities. Therefore, the involvement of all affected DOE organizations is essential in the selection of effective and efficient training techniques and approaches. DOE's early and frequent involvement, particularly at the local level, is necessary to ensure that the techniques chosen by the contractor are acceptable and consistent with the Department's goals, policies, and requirements. Personnel training is the direct responsibility of line management. This handbook discusses alternative techniques that can be incorporated to assist line management in making decisions about resources that are committed to training. The training organization should provide support to line management in implementing the most economical and efficient techniques feasible. Techniques other than those discussed in this handbook can yield satisfactory results and serve the customer's (the line organization) needs. Where possible, alternative approaches that save time and/or money should be used. Grading the more traditional training practices described in DOE- HDBK-1078-94, Training Program Handbook: A Systematic Approach to Training, and DOE/EP-0095, Guidelines for Job and Task Analysis for DOE Nuclear Facilities, contributes to this economy while maintaining a credible and effective training program.

Where alternative techniques have proven to be successful, the Department encourages their continued use and refinement. DOE Headquarters, field organizations, and contractors are encouraged to discuss and share techniques that have proven effective in reducing resource requirements for training program development. 1.2 The Need for Training Before any significant training effort is expended to address a performance problem, it is important that an assessment be made to determine whether or not training is the solution to the problem. Key questions that should be considered include: DOE-HDBK-1074-95 4 C What is the performance problem? C What is the desired performance? C Will training help to bring current performance to the desired level? Management must recognize and consider other factors which impact performance that may not be corrected with training. Factors such as quality of procedures, human factors, management style, and the work environment also affect performance. This assessment is part of the systematic approach to training and is referred to as a training needs analysis. 1.3 Effective Training The goal of training is to develop and maintain a competent work force. A systematically established program which accomplishes this goal meets DOE expectations. A graded approach that uses alternatives to traditional systematic training techniques is acceptable and encouraged. There are several traditional systematic approaches to training including Performance-Based Training (PBT), Instructional Systems Design (ISD), and Criterion Referenced Instruction (CRI). These approaches all have common elements that are: C Job Based. Training focuses on the job (i.e., the tasks and the criteria/standards necessary for proper performance). C Sequential. The program (every lesson and every lesson plan) is logically and sequentially integrated. C Tracked. A tracking system (such as the task to training matrix described in DOE HDBK-1078-94, Training Program Handbook: A Systematic Approach to Training) is established which allows changes and updates to training materials to be accommodated efficiently. C Evaluated. Evaluation and corrective action allows continuous improvement and maintenance of training information that reflects current status and conditions. Grading of training efforts and using alternatives to the more traditional SAT techniques should not be misconstrued to mean a reduction in quality. Rather, the level of detail and formality are tempered by factors such as hazard and risk, cost-benefit, and productivity. Regardless of the hazard DOE-HDBK-1074-95 5 associated with a facility, some jobs and many tasks are low risk. The development of training for ANY job/task should be graded. Within a high- hazard facility, the consequences of inadequate performance of some tasks may pose a low risk to the individual, the workforce, the environment, and the public. Training materials for these tasks do not need to be as detailed or as formally developed and implemented as the training materials for high- consequence tasks in the same job. Line and training management are expected to consider these factors and to make cost-effective training decisions that meet the expectations and the requirements of the job and the cognizant DOE field organization. 1.4 Alternative Techniques

The techniques described in this handbook allow latitude in making training consistent with the job requirements. The fundamental elements of SAT are the key and a fundamental SAT approach is reflected throughout. Alternative techniques streamline the processes that have historically been driven by formal guidance documents. The following alternatives to traditional approaches, used in conjunction with the information in Appendix A, is presented to help management consider and select the most reasonable and cost-effective technique(s) for the specific training and facility needs. 1.4.1 Table-Top The table-top process is facilitated by a person who is familiar with table-top techniques and application of the results. For the table-top technique to be effective, a minimum of one job incumbent and one supervisor are needed to discuss the task(s) or topic(s).The facilitator conducts the session(s) and documents the information. The success of this technique depends primarily on the expertise of the group and the facilitator's ability to extract and summarize information and learning strategies. This process is most useful and effective in analysis, design, and development. 1.4.1.1 Analysis The table-top method of job analysis typically consists of DOE-HDBK-1074-95 6 C Orienting the team. C Reviewing the job. C Identifying the duty areas associated with the job. C Identifying the tasks performed in each duty area and write task statements. C Sequencing the duty areas and task statements. C Selecting tasks for training. The Guide to Good Practices for Table-Top Job Analysis, DOE- HDBK-1076-94 contains detailed guidance on the table-top method for conducting job analyses. 1.4.1.2 Design A table-top approach to design is used to determine and design the content of a training program. The table-top method typically involves the following steps: C Developing a curriculum outline. C Determining the content of each training session and writing learning objectives, and determining the appropriate learning strategy (instructional method and setting). C Determining testing requirements. The Table-Top Training Program Design handbook, DOE- HDBK-1086-95, contains detailed guidance on the table-top method for designing training materials and programs. 1.4.1.3 Development The table-top method can also be used to review and modify existing facility training materials or materials from similar facilities to minimize development efforts. This method is effective in identifying equivalent material and verifying the applicability of the content. Table-top development may also be used to develop OJT materials such as JPMs (or their equivalent) and OJT guides by analyzing various tasks as a group and writing training materials. DOE-HDBK-1074-95 7 1.4.2 Verification This technique allows training program products to be determined based on work at other facilities on the same or similar tasks or topics. This process can save significant effort and cost. Communication with, or benchmarking visits to, both government and private facilities will enable facilities to take advantage of existing experience and materials. Industry analyses that can be adapted to DOE nuclear facility positions are available for many of the reactor operator, reactor supervisor, maintenance, and technician positions. Use of these lists require the help of SMEs and a trained facilitator. These experts use the lists to decide which tasks apply and to identify the tasks that require modification to reflect job requirements. Other sources of information and industry guidelines that may identify job-related training requirements include guides to good practices, DOE technical standards, other DOE facilities, commercial nuclear utilities, and vocational programs.

The verification technique consists of the following steps: C Gathering relevant existing training materials and task information from local and external sources. C Comparing this information to the facility-specific needs. C Modifying the information as needed. C Verifying the accuracy of the information by SMEs. 1.4.3 Document Analysis This technique is especially valuable when accurate procedures and other job-related documents are available. Document analysis is a simplified technique for determining required knowledge and skills directly from operating procedures, administrative procedures, and other job-related documents. An SME and a trainer review each section and step of the procedure or document to determine training program content. Document analysis consists of the following steps: C Review the procedure or document and list the knowledge and DOE-HDBK-1074-95 8 skills required by a worker. C Verify the accuracy of the results. 1.4.4 Templating Training content can be determined by the careful review/analysis of a template (a list of facility systems, theory topics, or a list of generic learning objectives). The template technique uses a simplified process for determining content or developing learning objectives associated with the operation or maintenance of a specific facility system. This technique produces generic and system-specific learning objectives for the training and evaluation of facility personnel. Some facilities have approached the design of training based on the systems an individual operates or maintains. A template containing generic learning objectives is reviewed by subject matter experts for applicability. This approach directly generates system-specific terminal and enabling learning objectives. It is important that the template be carefully reviewed to determine the applicability of each item to the system. If this review is not accomplished, the result can readily become "know everything about everything." Appendix B contains an example template for system knowledge and skills. The template technique includes the following steps: C Develop or modify an existing template to meet facility needs. C Use of a trainer and a subject matter expert(s) to select applicable objectives and/or complete portions of the template for a given system, component, or process. 1.5 Application of the Systematic Approach to Training Traditional applications of SAT encourage application of the process in distinct steps. Analysis, design, development, and implementation are accomplished in sequence and one phase is completed before the next is started. Evaluation is conducted during and after each phase. Alternatives encourage integration of these steps whenever possible. This simply means that there will likely be some analysis that occurs in the design and development phases and some design work can be accomplished in the DOE-HDBK-1074-95 9 analysis phase. For example, in many cases it is possible and reasonable to progress directly from a task statement to a learning objective. Experience with field application of an integrated approach has identified opportunities for streamlining of the process and has resulted in significant time and resource savings. DOE-HDBK-1074-95 10 Intentionally Blank DOE-HDBK-1074-95 11 2. ANALYSIS 2.1 Purpose

Analysis provides the core information that is essential in establishing programs that are job related. There are several different types of analyses that each have value when applied appropriately to reflect the complexity and risk associated with job performance. Analysis is used in program development, program revision, and program improvement. Analysis is cost- effective in the long term because it ensures that training resources are used effectively (only job-related tasks are identified and included in the training program). 2.2 Selecting an Analysis Technique Factors specific to analysis to consider when selecting an analysis technique include: C Availability of similar task lists. C Availability and accuracy of job-related procedures. C Availability of Safety Analysis Reports (SAR). C Quality of existing training materials. C Availability and quality of input from SMEs. These factors supplement the considerations in Appendix A. If the availability of SMEs is limited, a verification analysis (if a task list is available) or an analysis of procedures and other job related documents should be conducted. Lacking the aforementioned resources, job analysis should be conducted using either table-top or traditional techniques. In practice, a combination of table-top, verification, and document analysis techniques has generally proven to be most efficient. 2.3 Types of Analyses 2.3.1 Needs Analysis When a performance deficiency is being addressed, a needs analysis defines deficiencies or problems and identifies causes and solutions. Needs analysis is a systematic search for the deficiencies between DOE-HDBK-1074-95 12 actual and desired job performance and the factors that prevent desired job performance. A needs analysis should be performed whenever new requirements are issued, when job performance is below standards, and when requests for changes to current training or for new training are received. A needs analysis helps assure training is the appropriate solution, identifies the training issue(s), and ensures that instruction on the topic is not already included in an existing training program. Determination of the cause of performance deficiencies and potential solutions is the primary purpose of needs analysis. A needs analysis which addresses a specific performance problem helps management determine whether or not the problem can be resolved by training or if non-training issues must be addressed. Job aids, revised procedures, increased management attention, etc., may be solutions that do not require training. Depending on the application, needs analysis should include: C A review of regulations (RCRA, OSHA, DOE orders, standards, etc.) to determine training requirements. C A review of occurrence reports, performance indicators, and other performance feedback to identify trends and weaknesses. C Interviews/interactions with job incumbents, supervisors, and facility management to identify needs. C SME table-top discussions to identify and define performance problems associated with the job. Needs analysis provides documentation of the determination of needs and recommended solutions. If training is needed, the needs analysis should provide the following information: C Performance deficiencies. C Barriers to desired job performance. C Recommended training solutions. 2.3.2 Functional Analysis When a position that performs a large number of tasks (e.g., DOE-HDBK-1074-95 13

management or engineering) is being analyzed, a technique called functional analysis can be used. Rather than conducting a job analysis to identify specific tasks, major functions within the position are identified. After the competencies necessary to perform the major functions are identified, those competencies can be analyzed to determine objectives for training. Functional analysis can be conducted using the techniques described in this handbook. 2.3.3 Job Analysis Job analysis is a process which develops a list of tasks for a specific job or position. Job analysis provides reasonable assurance that tasks essential to safe operation are identified for training. It also identifies tasks that are vital to facility operation and tasks that are of lesser consequence. This information helps management assign resources where they are most needed. Job analysis involves developing a task list, selecting tasks for training and determining the appropriate level (train and/or overtrain), and validating the results. There are many approaches to accomplishing these three steps. Some techniques (such as those described in DOE/EP-0095, Guidelines for Job and Task Analysis for DOE Nuclear Facilities) can be relatively elaborate, and while they may provide a somewhat higher level of confidence in the validity of their results, these techniques are both time consuming and expensive. Highly formal techniques should only be used for jobs that are exceptionally hazardous and when other less expensive approaches will not work because of complexity of the task or absence of reference sources (e.g., SMEs, procedures, and other technical documentation). More efficient and less complex techniques (e.g. table-top job analysis) should be used whenever possible. Use of the table-top job analysis technique will generate acceptable results in a short period of time. As a general rule, the lower the impact improper performance of a job has on environmental, safety, and economic factors, the less elaborate the analysis technique should be. The following are examples of documents that should be reviewed to determine job scope and/or performance requirements when conducting job analysis. DOE-HDBK-1074-95 14 C Job or task analyses data of similar jobs (e.g., DOE nuclear facilities or commercial nuclear power facilities). C Standard operating procedures. C Emergency/Abnormal operating procedures. C Existing qualification documents. C Safety Analysis Reports (SARs). C Job descriptions. C Group, department, and/or division procedures. C Alarm response procedures. C Consensus-based content guidelines. C Work Authorization Directives (WADs). C Technical Safety Requirements (TSRs). C System design descriptions. C Equipment/System operating/vendor manuals. C Occurrence Reports (ORs). C Investigative Reports. 2.3.4 Task Analysis Task analysis is a process of examining each task, one at a time, to identify the task elements or action steps, knowledge, and skills. Task analysis is conducted after the task list is completed and tasks have been selected for training. It is not necessary to identify task elements or action steps, knowledge, and skills for those tasks that do not require training. Task analysis can be conducted using a table-top approach, verification techniques, surveys, or interviews. These techniques determine: C Knowledge and skill requirements. C Task testing limitations. C Elements of the task to be tested. C Entry-level requirements. C Amplifying conditions and standards. C Evaluation requirements.

Formal, detailed task analysis can be time consuming and expensive to conduct and should be used only when other less detailed techniques do not produce satisfactory results. Information on conducting detailed task analysis is found in DOE/EP-0095, DOE-HDBK-1074-95 15 Guidelines for Job and Task Analysis for DOE Nuclear Facilities. 2.4 Selecting Tasks for Training After the task list is developed, it is necessary to determine which tasks are important to the facility mission and which are not as important and, therefore, do not warrant formal training. This decision has a significant impact on training resources. If the criteria for determining which tasks are important to the facility mission are too broad, a great deal of resources may be needed to design, develop, implement, and maintain training. Conversely, if the selection criteria are too restrictive, training deficiencies may exist and result in costly personnel errors. The key to an effective balance is to ensure that the correct people are involved in the decisions, and that criteria appropriate to the facility's mission are used. The correct people are the people who know the job, the task, and the associated hazards and complexity of performance. There are several techniques for selecting tasks for training. The traditional technique involves determining the difficulty, importance, and frequency of each task and applying the results to a decision tree that contains numerical values for each trait. Other techniques, such as SME discussion and consensus, can produce acceptable results. However, it must be understood that the results of any technique used are not absolute and should be challenged, evaluated, and refined as necessary. In all cases, the results should accurately reflect job requirements and should have the concurrence of job incumbents and management. Regardless of the selection technique used, tasks are identified and grouped as "train," "no-train," and "overtrain/retrain." Train is defined as requiring initial training, no-train is defined as requiring no formal training, and overtrain/retrain is defined as requiring both initial and continuing training. Even though "no-train" tasks do not require formal training, possession of the knowledge and skills associated with the task is still necessary. These knowledge and skills are normally addressed with entry-level requirements or are readily learned as part of working in the position to which they are associated. Some facilities include no-train tasks in their qualification card/checklist. An additional classification of tasks is referred to as "pre-train" or "just-in- DOE-HDBK-1074-95 16 time" training. These are tasks that require training, but will not be taught until that specific knowledge and skill is needed. For example, tasks specifically associated with maintenance of a complex pump could best be taught just prior to scheduled maintenance. 2.5 Elements of Analysis The fundamental elements of the analysis phase are: C Key facility personnel (line management and subject matter experts) are involved in the analysis process and concur with the results. C Needs analysis is used to address performance deficiencies and identify actions necessary to correct them. C Job analysis is conducted to determine the tasks required for job performance. Tasks are identified, documented, and prioritized according to organizational goals.

C Task statements are written. C Tasks are selected for training. C Task analysis is conducted (if necessary) to determine the knowledge and skills that are necessary for the job. C The analysis process and results are documented and maintained current. 2.6 Products of Analysis Products that typically result from analysis include the following: C Needs analysis data (e.g., descriptions of existing training deficiencies, trainee/learner needs, and regulatory requirements). C Tasks list(s). C Task statements. 2.7 Application 2.7.1 Facility Involvement If the correct people are not used, the results of even the most detailed analysis can be less than satisfactory. Participants must be well acquainted with the job and must understand the consequences DOE-HDBK-1074-95 17 of poor or improper performance of tasks. Facility personnel who should be involved in analysis include experienced SMEs, supervisors, management, qualified training staff, and as necessary, facility engineering, safety, and other technical support staff. The technical support staff can be especially helpful when new systems or procedures are involved. Facility personnel involvement in analysis helps ensure training decisions belong to facility personnel since the results of analysis are used to design training that meets the needs of the job incumbents and line management. The job incumbent should be viewed as the customer and the analysis process as a method for meeting customer requirements. 2.7.2 Periodic Needs and Job Analysis Updating Task lists serve as the foundation for initial and continuing training programs and must be kept up to date. The frequency of the update can vary from as-needed to every two years, depending on the number of changes that impact training (expected or actual) and training organization procedures. Program-level needs analyses can also be conducted on a periodic basis as part of program evaluation to detect performance deficiencies before they lead to personnel errors. 2.7.3 Transition to Design Activities The practices described in DOE/EP-0095, Guidelines for Job and Task Analysis for DOE Nuclear Facilities, encourage the completion of task analysis prior to starting the design phase. An alternative approach that combines task analysis with the design and development of training materials has also been found to be successful and efficient. The time and expense of completing the task analysis before design and development is avoided and training material can be developed and ready for use sooner (and therefore more economically). Accordingly, combining task analysis activities with design and development activities is strongly encouraged whenever feasible. 2.8 Documentation Documentation should be maintained that describes the activities of the analysis phase. Written procedures and forms that support this DOE-HDBK-1074-95 18 documentation are readily available within the DOE complex. Task lists and a record of the participants provide adequate documentation in most cases. Reference to existing task lists, guides to good practices, and similar sources of information should be made when such sources form the basis of tasks selected for training. DOE-HDBK-1074-95 19 3. TRAINING DESIGN 3.1 Purpose Training design produces a training development plan that guides the creation of all training materials and strategies. It is important to design an efficient, effective training program that is based on analysis data and reflects job performance requirements.

3.2 Techniques of Design The techniques for determining training content, sequence, and strategies may vary from highly-formal to less-formal and streamlined. Use of a combination of traditional, table-top, verification, and document/template techniques will usually be most efficient. For additional information regarding technique selection, see Appendix A. Regardless of the technique selected, the following activities generally occur during training design: C Terminal learning objectives are written. C Enabling learning objectives are written. C Testing requirements are determined. C Training content for each unit of instruction is determined. C A training program plan is developed. C The design process is documented. 3.3 Elements of Design 3.3.1 Fundamental Elements of Training Design C Analysis results are used to determine the content of the training program. Terminal learning objectives that are observable and measurable are written from task statements. C Job incumbents, their supervisors, and line management are involved in establishing initial and continuing training program content requirements and concur with the resulting design. Training programs are designed to ensure initial training DOE-HDBK-1074-95 20 provides reasonable assurance that the knowledge and skills essential to job performance are mastered by the trainees after training. Continuing training is designed to maintain and enhance job performance. Elements of continuing training should include facility changes, changes to the scope of tasks, procedure changes, facility and industry operating experience, training to correct observed performance deficiencies, changing regulations, over-train tasks from the job analysis, and other information that requires periodic refresher training (e.g., selected fundamental topics). C The design of the courses and lessons in the curriculum promotes the application of knowledge and skills to the applicable job performance level. C Observable and measurable enabling learning objectives which support the terminal learning objectives are written based on job performance requirements. C Training is designed such that actual job performance requirements, conditions, and standards are replicated to the extent practical. C Testing and performance evaluation requirements and limitations are determined. 3.4 Products of Design The products that typically result from training design include the following: C A training program description that explains and documents the elements of the training and qualification program. A curriculum outline (structure) for the sequencing of initial and continuing training is used in developing the training program description. C Requirements (specifications) for each unit of training, including sequencing of learning objectives, designation of training settings and techniques, and associated testing requirements. C A training development plan. 3.5 Application Training design products guide the development of all program-specific training materials and strategies. A curriculum outline is developed to DOE-HDBK-1074-95 21 establish the basis for the development of a training plan or summary. This is a very important series of activities that provides management with the information necessary to ensure the program will meet the needs of the organization. 3.5.1 Establishing a Curriculum Outline

A curriculum outline documents the general structure of a training program. The outline is developed by using information from analysis to design initial and continuing training programs. The curriculum outline helps prioritize training and identifies training materials that must be developed. Requisite knowledge and skills, systems, topics, tasks, and testing requirements are grouped within the training program curriculum. Designers determine the most effective strategy for presenting information by determining the sequencing of content presentation, selecting the most appropriate training setting for the objectives, determining testing methods, and identifying the most appropriate approaches to instruction. Instructional approaches should be based on objectives and the setting selected for conduct of the training. Instructional approaches may include lecture, demonstration and/or practice, discussion/facilitation, oral questioning, role playing, use of case studies, simulation, exercises and walk-throughs, and individualized instruction (structured self-study). The approaches selected will also be influenced by the hazard and/or complexity of the task or job. For example, more structured, formal instructional approaches are appropriate when the hazard or risk consequences of improper performance are greater. Conversely, less formal approaches are sufficient when the consequences are lower. The same holds true with the curriculum outline. Curriculum outlines may vary from a simple one page outline when the hazard or risk consequence is low to a complex multi-page outline when the hazard or risk consequence is high. After the curriculum outline is developed it should be verified by SMEs. The following techniques may be used individually or combined to establish a curriculum outline. DOE-HDBK-1074-95 22 C Table-top. C Verification. C Industry guidelines. C Task analysis. 3.5.2 Writing Learning Objectives It is essential that learning objectives accurately describe the required performance for a task. Effective learning objectives clearly state measurable performance the trainee must be able to demonstrate and include conditions and standards of task performance. In most cases, a training designer develops learning objectives and obtains SME and management concurrence. Objectives provide the framework for development of training materials and determination of testing requirements. For detailed information relative to developing learning objectives, refer to DOE-STD-1005-92, Guide to Good Practice for Developing Learning Objectives. 3.5.3 Establishing Testing Requirements When designing testing specifications for individual units of instruction or for overall qualification, the designer determines performance testing limitations and scoring methods for performance tests. Written and oral test specifications and job performance measures (JPMs), or their equivalent, are then developed. For additional information about testing requirements refer to DOE-STD-1009-92, Guide to Good Practices for the Development of Test Items, and DOE-STD-1011-92, Guide to Good Practices for the Design, Development, and Implementation of Examinations. 3.5.4 Training Program Description A training program description uses the curriculum outline to document training program elements for each position. The information describes training program content (including initial and continuing training) and training program administration requirements.

Details in a training program description may include: DOE-HDBK-1074-95 23 C A plan and schedule for training program development and implementation. C Identification of resource requirements. C Entry-level requirements. C A training schedule (including topics such as General Employee Training (GET), General Employee Radiation Training (GERT), Radiation Worker Training, etc). C Testing requirements. C Criteria for exceptions (waivers) from portions of the training program through prior education, experience, training, and/or testing. The training program description may be integrated into a document such as the Training Implementation Matrix/Plan or an existing administrative manual such as a Training Users Manual or a Training Management Manual. 3.5.5 Facility Involvement Job incumbents (SMEs) and facility management should be actively involved in the design process. Line personnel must accept ownership in the process and concur with the resulting plans and products. Line management is the customer and line management concurrence with the results of the design phase is essential. 3.6 Documentation Processes used and participants involved in determining curriculum and training content should be documented. Documentation is necessary to track decisions and to provide assurance to the customer (line management) and DOE that the training program design is based on reasonable decisions by appropriate persons. Several documentation formats can be used, including data bases and computer programs, to provide the linkages needed. At most facilities, the training program description should provide sufficient documentation. DOE-HDBK-1074-95 24 Intentionally Blank DOE-HDBK-1074-95 25 4. TRAINING DEVELOPMENT 4.1 Purpose During development, all training, documentation, and evaluation materials are revised, written, or otherwise produced. As training materials are developed, an evaluation to determine effectiveness, cost efficiency, and applicability to the training needs should be made. Training materials should be piloted (tried on a group of trainees or SMEs) and modified to correct any deficiencies. Final materials used in training should be reviewed and approved by line management. 4.2 Techniques for Development Development of training materials represents a considerable investment and it is important to review and use existing material to minimize development efforts whenever possible. Development may include adopting other materials, revising other materials, or creating new materials. Materials from similar facilities or from facilities that have positions performing similar jobs can often be made facility-specific relatively easily, and can result in considerable savings in time and effort. The table-top and verification techniques are effective in identifying equivalent material and in verifying the applicability of content. The degree of formality and complexity of training materials should be tempered or influenced by the complexity and hazard of the tasks. For lower- hazard tasks, the training material used need not be as formal or complex as those for higher-hazard tasks. Rather, these materials may only need to include the key points that support the learning objectives. For higher- hazard tasks, more formal training materials may be necessary. Regardless of the complexity or hazard, training materials should contain sufficient detail to ensure consistency. 4.3 Elements of Development

The fundamental elements of training development are: C Training materials are developed or modified using learning objectives DOE-HDBK-1074-95 26 derived from analysis information that reflects job performance requirements. C Review and approval requirements are established and implemented for all training materials prior to their use. C Training materials are developed with guidance and structure that ensures consistent presentation and evaluation. 4.4 Products of Development Products of training development may include: C Classroom lesson plans. C Laboratory and/or simulator guides and evaluation materials. C Student materials and training aids. C Self study materials, including workbooks, software, handouts, reading guides, and evaluation instruments. C OJT materials such as student guides, checklists, qualification standards, and JPMs, or their equivalent. C Initial and continuing training schedules. C Test items and examinations. C Program evaluation materials such as course, instructor, lesson, post-training, etc., evaluation forms. C Training documentation system, including record systems, attendance forms, and course documentation requirements. 4.5 Application The curriculum outline created in the design phase (which supports the training plan) identifies materials that need to be developed. The training setting and learning objectives that are identified in the design phase determine what training strategies are used. If alternative approaches suggested thus far have been used, the volume and complexity of training materials that require development for lower-hazard tasks should be significantly less than those required for higher-hazard tasks. 4.6 Documentation The training organization should describe the process(es) used for control DOE-HDBK-1074-95 27 and development of materials in a procedure or similar document. This document should also describe how changes to training materials are documented, approved, and tracked. Program records should be maintained that include (as necessary): C Master files of lesson plans/guides, etc. C Student materials, checklists, etc. C Examination banks. DOE-HDBK-1074-95 28 Intentionally Blank DOE-HDBK-1074-95 29 5. TRAINING IMPLEMENTATION 5.1 Purpose During implementation, training is delivered and trainee mastery of the learning objectives is assessed. 5.2 Techniques for Implementation Training should be implemented using written guidance documents (classroom lesson plans, OJT guides, JPMs, written examinations, etc.). Techniques that are used to implement training include: C On-the-job training. C Classroom training. C Laboratory training. C Individualized instruction (self-study, CBT, interactive video, etc.). C Simulator training. Training is conducted by instructors who are trained and qualified for the setting in which they perform. DOE-NE-STD-1001-91, Guide to Good Practices for Training and Qualification of Instructors, and DOE-STD-1012- 92, Guide to Good Practices for On-the-Job Training, contain guidance relative to instructor training and qualification. 5.2.1 On-the-Job Training On-the-job training (OJT) requires instructors who are qualified to perform the task or assignment. The use of formal guidance in conducting OJT and evaluations is necessary to ensure consistent implementation. It may be possible to train 2-4 persons simultaneously without sacrificing safety or training quality, however, trainee evaluations must be conducted one-on-one. OJT can be implemented at three levels.

For high-hazard potential tasks, OJT should be conducted as described in DOE-STD-1012-92, Guide to Good Practices for On-the- Job Training. DOE-HDBK-1074-95 30 For tasks that pose a lower-hazard potential, OJT may be conducted using general instructions and task-specific evaluation materials. General instructions guide the process and a brief JPM (or equivalent) is used. These materials are sufficient to consistently conduct and evaluate OJT. Use of general instructions can significantly reduce training development time and lower costs. OJT for technical staff, oversight personnel, and low-hazard tasks may be conducted and evaluated simultaneously. OJT used for the training of tasks at this level is conducted and evaluated as a one step process (this process is sometimes referred to as structured mentoring). Using general instructions and a JPM (or equivalent), the instructor conducts a qualitative evaluation of the trainee's progress. The instruction and evaluation are repeated until the competency or task is mastered. 5.2.2 Classroom Training During implementation, qualified instructors deliver classroom instruction using lesson plans. Lesson plans should encourage direct trainee participation in the learning process and employ instructional techniques (e.g., briefings, lecture, discussion, case studies, seminars, etc.) that are appropriate to the lesson content and learning objectives. Regular evaluation of the trainee's mastery of the learning objectives is expected during classroom training. Lesson plan detail and format should vary as a function of the complexity and hazard of the task. Lesson plans may consist of less formal guides for low- hazard tasks. These guides should include the key points that support the learning objective(s) and any supporting information determined to be necessary. Lesson plans for high-hazard tasks should contain additional detail and should be consistent in format. The added detail is necessary to address the increased complexity and hazard associated with the task. Regardless of the complexity or hazard, lesson plans should contain sufficient detail to ensure consistent delivery. 5.2.3 Individualized Instruction Although discussed separately, industry standards consider DOE-HDBK-1074-95 31 individualized instruction as a form of classroom training. Individualized instruction is a common technique for training operations, maintenance, and technical staff personnel. For the purposes of this handbook, individualized instruction consists of structured self-study, computer-based training, and interactive video. Structured self-study can be used to supplement or replace traditional lecture-based classroom training. It is effective in improving knowledge of fundamentals, facility systems, components, and procedures. Instructors and supporting training materials should be readily available during self-study. Following self-study, a written examination or an oral checkout/quiz, based on learning objectives, is usually administered. Computer-based training (CBT) and/or interactive video can also be used to supplement or replace classroom training to teach fundamentals, facility systems, components, and procedures. Development of high quality CBT is expensive, but can be cost- effective if large numbers of people are trained on standard topics. To make CBT more effective, assistance (i.e., instructors, proctors, program administrators, etc.) should be available to answer questions.

5.2.4 Laboratory Training Laboratory training can supplement and provide a cost-effective alternative to portions of the classroom and OJT program. Training laboratories often use surplus equipment and repair parts that can, in many cases, be used repeatedly. The laboratory allows one instructor to provide training to a greater number of trainees than is possible with OJT. Laboratory training requires approved training materials such as laboratory guides and JPMs. Laboratory training can be used to provide practice with equipment that is similar or the same as in-plant equipment. It can also be used to provide training in a "job-like" setting without hazards that exist at the job site (e.g., high noise levels, radiation areas, hazardous/toxic environmental conditions, etc.). When laboratory training replicates facility equipment closely enough to provide assurance of task performance, it can serve as a substitute for actual performance on DOE-HDBK-1074-95 32 installed equipment for qualification purposes. 5.2.5 Simulator Training A simulator is an apparatus that generates conditions approximating actual or operational conditions. The most widely recognized simulators are those used in the aircraft and the commercial nuclear power industries. Simulators can be either full-scope or part-task. Full-scope simulators are usually large-scale replications, which create a training environment with a high degree of physical and functional fidelity relative to the actual or "referenced" job. A part-task simulator incorporates detailed modeling of a limited number of components or subsystems and demonstrate expected response. Part-task simulators also add realism to training. Simulator training can be highly effective for normal facility operation, anticipated transients, and accident conditions. Because the simulator reflects actual job conditions, OJT may be completed on the simulator in many cases. In addition, this setting lends itself to effective team training. 5.3 Elements of Implementation Elements of implementation include: C Training is implemented in accordance with current, approved training materials and procedures. C Training activities encourage direct trainee participation in the learning process. C Instructors use instructional techniques that are appropriate to training content and objectives. C Trainee mastery of learning objectives is evaluated using performance examinations (OJT, performance demonstrations, and operational evaluations), written or oral examinations, quizzes, role-play, case studies, and/or group exercises. 5.4 Products of Implementation Products of implementation may include: DOE-HDBK-1074-95 33 C Training schedules. C Trainee records. C Program in-training evaluation records. C Trained employees. 5.5 Application The successful implementation of a training program requires line management involvement. By periodically monitoring on-going training sessions and providing feedback, line management helps ensure training quality. Line management's continued support of training ensures the availability of technically qualified personnel from the operating organizations (e.g., operations, maintenance, technical staff) to the training organization. In addition, the line organization provides in-plant time for qualified instructors to maintain technical qualification. Active involvement is particularly necessary in providing training program funding, facility availability, and support resources for the program.

5.6 Documentation Process(es) and administrative requirements used for training implementation should be described in a procedure or similar document. The guidance in the Nuclear Information and Records Management Association Guideline for Management of Nuclear Related Training Records, TG-17, should be used to help standardize identification, handling, and storage of training records. 5.6.1 The following documentation is associated with for training implementation: C Schedules for initial and continuing training sessions. C Records of training courses or session attendance/ completion. C Individual training records which contain information that reflects mandated requirements (i.e., DOE Order 5480.20A). C Instructor critiques of training. C Trainee critiques of training sessions. C Results of employee performance evaluations (written and oral examinations, performance demonstrations, and operational evaluations). C Line management evaluations of on-going training sessions DOE-HDBK-1074-95 34 (content and instructor). C Copies of examinations and answer keys. C List of personnel qualified to conduct and/or evaluate on-the-job training. C List of required revisions to the training program and its materials. DOE-HDBK-1074-95 35 6. TRAINING EVALUATION 6.1 Purpose The goal of all training programs is to prepare personnel to safely and efficiently operate in accordance with established requirements. A training evaluation program should be implemented at every nuclear facility to determine the program's effectiveness of meeting trainee and management needs. Training activities, employee performance, and subcontractor training should be evaluated on a regular basis to determine the effectiveness of training. Changes to the training program that result from the evaluation process should be approved and documented. Detail in documentation of evaluation can reasonably vary from facility to facility. Post-training evaluation at a low-hazard facility may only require brief interviews with recently qualified job incumbents and their supervisors. Documentation by memorandum to the respective line manager that describes evaluation results and any necessary corrective actions is sufficient. At a high-hazard facility, evaluation may require survey forms or questionnaires from newly-qualified job incumbents and their supervisors. Documentation in this case may include entry of corrective actions into a tracking system, plans and status of training improvements, and approval of all revised training materials. Evaluation activities should be commensurate with the hazard potential, risk, and complexity of job performance. 6.2 Methods of Evaluation There are a number of evaluation activities that provide information about the effectiveness of a training program. Common techniques include: C In-training evaluations (examinations). C Training delivery evaluations. C Post-training evaluations. C Change actions (e.g., job-scope changes, procedure changes, equipment changes, facility modifications, etc.). C Facility and industry operating experience evaluations. C Comprehensive training program evaluations. DOE-HDBK-1074-95 36 6.3 Elements of Evaluation There are several activities associated with evaluation. Preparation for conducting evaluations may include development work, planning, scheduling, and identification of evaluators. The actual implementation of the evaluation process includes preparation, field observation, report preparation, and documentation, including resultant corrective action plans.

Implementation of specific techniques is discussed in section 6.6. A brief explanation of the elements involved in preparation follows. C Development. Some development effort will normally be required to implement a formal and consistent evaluation program. Procedural guidance, standard forms to address specific techniques, and standardized reporting mechanisms should be developed prior to actual implementation of evaluation activities. C Planning. It is important to plan the frequency, scope, and conditions of evaluation. This planning should consider such variables as personnel availability, plant or system conditions, benchmarks in training program implementation, and evaluator qualifications. C Scheduling. A schedule that provides advance notification allows for timely gathering of documentation and personnel to conduct an efficient evaluation activity. There may be circumstances that favor an unannounced evaluation, but advance scheduling is generally more valuable for overall program effectiveness. C Identification of evaluators. Some evaluations will require specific competencies and experience for evaluators. There is little use in conducting evaluations using personnel who are not qualified to make adequate judgements about the quality of activities being reviewed. Highly formal evaluation programs may require special qualifications for evaluators, while less formal programs may only require designation by management, based on an assessment of the evaluator's qualification. 6.4 Products of Training Evaluation Products that typically result from training evaluation include: C Records of evaluation results. DOE-HDBK-1074-95 37 C Records of corrective actions. C Updated analysis data. C Current and accurate training materials. 6.5 Application The frequency and level of detail of training evaluations should reflect the risk, hazard potential, and complexity of the performance that is being addressed by the training program. In any case, it is essential that a formal process be in place to ensure that feedback is provided that allows maintaining training materials which reflect current conditions, equipment, and procedures. The evaluation (feedback) process also serves to improve the training program and respond to line management needs. Generally, as the job complexity and hazards increase, the need for training evaluations also increases. Added frequency and detail of evaluation activities increases data collection and analysis/review efforts. Development of forms (and perhaps data management systems) to simplify this documentation and tracking of each element in the evaluation program may be necessary. 6.6 Conducting Training Evaluations 6.6.1 In-Training Evaluations Employee performance on written, oral, and performance examinations is monitored and evaluated to provide information about the need for refining the training program and testing process. For example, if the trainee failure rate on an examination, JPM, or test item(s) is high, there may be a problem with the training program or the test question. If failure rates are acceptable and other indicators (on-the-job performance, the rate of personnel-caused Occurrence Reports, etc.) are satisfactory, the training program may be considered effective. 6.6.2 Training Delivery Evaluations Training delivery is monitored in all instructional settings to evaluate content accuracy, adequacy of support materials, and instructor DOE-HDBK-1074-95 38

performance. Training delivery evaluations include the use of trainee evaluation/reaction forms, instructor critiques, and periodic assessment of instructor performance by qualified individuals. If necessary, instructors are upgraded and corrections are made to training materials. When changes are made to training materials, the changes should be approved and documented. 6.6.3 Post-Training Evaluations Feedback from former trainees and their supervisors describes how well the training program prepared employees for a task/job. This feedback should be obtained long enough after training to ensure familiarity with the job requirements and should identify the adequacy of training in preparing them to perform job-related tasks. Feedback from supervisors should focus on trainee performance. Information gathering should also include data on training-related performance problems, modifications to systems and components, or changes to procedures that may have effected the relevancy of the training. 6.6.4 Change Actions Job scope changes, procedure changes, equipment changes, and facility modifications are monitored to identify impacts on the training program. The training organization(s) should be notified when modifications are made to assess the need for making changes to training materials. Revisions to training materials should be made and training conducted prior to the change being effected. A formal system for maintaining training materials current is essential to ensure that training is relevant to existing facility conditions. 6.6.5 Evaluating Facility and Industry Operating Experience Training on facility and industry operating experience allows learning from the experience of others and helps prevent similar situations from occurring. This information should be regularly reviewed to allow inclusion of lessons learned from these events in training. Information of this type can be obtained from facility Occurrence Reports and from the Occurrence Reporting and Processing System (ORPS). DOE-HDBK-1074-95 39 6.6.6 Comprehensive Training Program Evaluation A comprehensive training program evaluation should be conducted periodically to identify significant program strengths and weaknesses. This evaluation should cover all aspects of the training program, including subcontracted training, and should be used to identify deficiencies and/or strengths in specific training programs and the overall training program infrastructure. DOE-STD-1070-94, Guidelines for Evaluation of Nuclear Facility Training Programs, provides objectives and criteria for conducting comprehensive training program evaluations. 6.7 Documentation The facility should describe the process(es) used for evaluation and initiation of corrective actions in a procedure or similar document. This document should identify areas of evaluation, the frequency of evaluations, and the process by which required changes are determined, approved, tracked, and implemented. The results of training evaluations should be documented. This documentation may range from a letter to line management at a low- hazard facility to detailed survey forms and analyses of evaluation results at higher-hazard facilities. 6.7.1 Approval and Tracking of Changes/Improvements Changes which result from evaluations should be reviewed, approved, and documented. This process adds assurance that only required changes are made to the program and that the changes are approved by training and line management.

6.7.2 Updating Analysis Data To maintain the effectiveness of training programs, job analysis data must be kept current. During evaluations, training deficiencies, additional learner needs, or additional tasks required for job performance may be identified. Analysis data should be revised to reflect new information that is gained from evaluation activities. DOE-HDBK-1074-95 40 Intentionally Blank DOE-HDBK-1074-95 A-1 APPENDIX A FACTORS AFFECTING TECHNIQUE SELECTION Technique selection should be made after considering a number of variables. These variables should be addressed specifically for facility situations that exist or will exist. This appendix provides general information that will assist line management in making this decision. Key factors to consider include the facility's hazard classification and other general considerations which address local and internal issues that impact the level of formality and effort necessary for training programs. The most appropriate technique may be simple or highly formal, depending on the hazard potential, risk, or complexity of post-training performance. Technique selection also depends on factors such as the quantity and quality of existing materials (procedures, system description manuals, training media, etc.). Determination of the appropriate technique must be made after conservative application of expert judgement. A graded approach to training encourages application of techniques that allow the most efficient use of personnel and resources in training activities. Under no circumstances should health or safety concerns be compromised in the name of cost savings or expediency. The graded approach should be implemented with the concurrence of line management, the training organization, and the DOE field organization. Hazards associated with a facility are significant factors in the selection of appropriate techniques for analysis, design, and development. This handbook encourages and endorses the grading of training efforts after considering the nuclear hazard classification of a DOE nuclear facility. DOE Order 5480.23, "Nuclear Safety Analysis Reports" contains additional information relative to nuclear hazard classifications. This Appendix provides additional considerations that the facility should address in choosing the appropriate techniques for systematically establishing training programs. General Guidance - Grading Based On Hazard Nuclear Hazard Category 1 (High-Hazard) and 2 (Moderate-Hazard) Facilities Elaborate, paper-intensive training programs are not necessarily required because DOE-HDBK-1074-95 A-2 a nuclear facility is designated as nuclear hazard category 1 or 2. Furthermore, facilities with programs that are subject to accreditation need not apply accreditation standards to training programs for job positions that are not identified as accreditable. Alternative techniques can and should be used when designing training programs. A study by Honeywell in 1978 revealed that at least 29 techniques are available and used for analyzing job requirements. This study concluded that (1) no one technique was any better than any other, and (2) regardless of the technique used, analysis results must be used for systematic approaches to training to succeed.

Normally, the level of formality and effort required to develop training is directly related to task complexity and hazards associated with the consequences of inadequate task performance. In addition, competency of the instructional developers, availability and quality of subject matter experts, availability of facility procedures, support of line management, and other factors are important in selection of the appropriate technique(s). When the complexity of the job and consequences of errors are considered with the factors mentioned above, training programs can be developed effectively and efficiently by grading the effort to fit the circumstances. Nuclear Hazard Category 3 (Low-Hazard) Facilities From a practical perspective, training programs at low-hazard nuclear facilities may consist primarily of regulatory-driven training (e.g., OSHA training, industrial safety and hygiene training, radiation worker training, etc.). The formality and detail associated with the training that supports the facility-specific mission may be minimal. The resource expenditure on training at a category 3 facility should be commensurate with the low level of risk. At nuclear hazard category 3 facilities, analysis may be simplified, learning objectives may be written at a level of detail not necessarily related to rigid standards or quantitative testing, and instructional material may consist of less-formal guides. Guides should include key points that support learning objective(s) and supporting information should be included as necessary. This approach differs from nuclear hazard category 1 and category 2 facilities, at which additional detail and consistency in format is more appropriate and necessary to address the increased complexity and hazard associated with many tasks that are performed. Evaluation of trainee competency is expected at low-hazard facilities. Qualitative DOE-HDBK-1074-95 A-3 evaluations may be conducted to check mastery of learning objectives. Mastery of learning objectives may be assessed from response during discussions, actions during exercises, or behavior during role-play. General Guidance - Technique Selection Considerations Key Factors The key factors and considerations for determining techniques presented below are consistent with other graded applications in DOE. Example questions are provided under each key factor. These questions should be considered to relate the training need to an appropriate level of formality and detail in training development activities. If the answers to these questions indicates a low level of concern, the appropriate techniques will likely be those that require minimum formality and effort. If the answers to these questions identify a strong potential for consequences of post-training performance, the appropriate techniques should be more formal and detailed. If there is an obvious concern for safety of the public, site, and environment, the most formal technique may be the appropriate choice. The questions are only examples and are not intended to be all-inclusive. This Appendix must be used cautiously. Competent, safe workers is the goal in all training efforts. Relative Importance to Safety This factor considers the potential impact of post-training performance on safety as it pertains to the public, environment, facility, equipment, and personnel. Some questions to consider are: 1. Is safety analysis information available that addresses the required performance of employees related to this training need?

2. What is the safety-related risk of improper post-training performance by employees? Life Cycle Stage of the Facility The life cycle stage of the facility considers the impact that age, mission transition, and operational status of the facility may have on the employee's DOE-HDBK-1074-95 A-4 post-training performance. Questions to consider include: 1. Will aging equipment or systems contribute to the impact on the facility if an employee performs improperly? 2. Will operational transition impact the ability of the employee to perform properly after training? 3. Does an operational, shutdown, or standby condition of the facility impact the ability of the employee to perform properly after training? Programmatic Mission This factor addresses the concerns related to the impact training has on accomplishing the primary mission of the facility. Potential considerations include: 1. Does post-training performance impact the facility mission? 2. Could improper post-training performance cause the facility to lose its ability to fulfill its mission? 3. If the facility is scheduled to undergo a "mission transition," what impacts will the transition have on the post-training performance of the employee? Characteristics of the Facility (Complexity) The term "complexity" refers to the integral operational, maintenance, and technical support aspects of the facility considered from an engineering, human factors, and safety perspective. The inter-relationship of these elements with employee performance is of significant consequence to technique selection. Considerations include: 1. After training, will the employee need to apply only basic skills, or will synthesis and reasoning be required for proper performance? 2. Are there backup safety systems, system duplicity, or human factors that impact operator performance or the level of formality and effort DOE-HDBK-1074-95 A-5 that should be applied to the training programs? 3. Do procedures, automated operations, personnel oversight, and operator backup impact the level of formality and effort that must be applied in developing and implementing the training program? Magnitude of the Hazards Involved This factor considers the hazard(s) and the magnitude of the risk that the hazard poses. Normally, hazard potential is related to a source, but site-, activity-, or source-specific determinations of the hazard potential should be examined. Considerations include: 1. Has the facility or operation been formally classified by DOE as nuclear hazard category 1, 2, 3 (high, moderate, or low) hazard? 2. Is there a source that poses a potential hazard to the public, environment, personnel, equipment, or facility? 3. Could poor post-training performance cause violation of a technical safety requirement, breach containment, impact the operation of protective systems, cause an unplanned or uncontrolled nuclear criticality, or result in a release of hazardous substances to the environment? Other Factors "Management considerations" are other factors that may impact the determination of the technique(s) for training program development. Considerations include: 1. What is the availability of resources (training personnel, SMEs, funding)? 2. What is the immediacy of the need for the training? 3. What is the anticipated impact of the training and how many people will it affect? DOE-HDBK-1074-95 A-6 Intentionally Blank DOE-HDBK-1074-95 B-1 APPENDIX B SAMPLE TEMPLATE FOR DETERMINING SYSTEM KNOWLEDGE AND SKILLS

The template that is presented in this appendix is an example of but one of numerous templates that may be used. For example, the DOE handbook entitled Training Program Handbook: A Systematic Approach to Training, DOE-HDBK-1078- 94, contains an example template for developing generic learning objectives in Attachment 11, Guidelines for Writing Learning Objectives, that may also be useful when developing a template. Other template models may serve the user equally well. 1. Theoretical Knowledge 1.1 Demonstrate a satisfactory knowledge of each listed theory area, including its practical application(s) as related to the job, during an oral checkout. (List of theory topics) 2. Systems and Component Knowledge 2.1 Demonstrate a satisfactory detailed knowledge of each of the items listed below during an oral checkout. A satisfactory knowledge should include the individual's capability to accomplish the following as appropriate. 2.1.1 State the function and location of the system/component. 2.1.2 Explain how the system/component impacts overall facility operation. 2.1.3 Draw a functional one-line or block diagram of the system showing major components. 2.1.4 Explain how the system works and its operating characteristics. This includes the inter-relationship of flow, temperature, and pressure within the system. DOE-HDBK-1074-95 B-2 2.1.5 State parameters that are monitored, normal readings, instrument locations, reasons for monitoring, and system/component limitations. 2.1.6 Identify power supplies to major components, system and equipment interlocks, and automatic features. 2.1.7 State the capacity of the system/component. 2.1.8 Explain the effects of a system/component failure. (List of facility systems/components) 3. Operating Knowledge and Skills 3.1 Demonstrate competency in each of the practical factors listed below. Requirements for practical factor signoffs are identified on the qualification card cover sheet (definitions of perform, simulate, observe, and discuss) and in the facility training program manual. (List of facility systems/components) 3.2 A satisfactory knowledge should include the following as appropriate: 3.2.1 Discuss pre-startup steps required for the system and/or component and any associated precautions and operating limitations. 3.2.2 Discuss the startup procedure for the system and/or component and any associated precautions and operating limitations. 3.2.3 Discuss the normal operating readings required and manipulations necessary for the system and/or component and any associated precautions and operating limitations. 3.2.4 Discuss the shutdown procedure for the system and/or component and any associated precautions and operating limitations. DOE-HDBK-1074-95 B-3 3.2.5 Discuss the theory of equipment operation as it pertains to the procedure. 3.2.6 Discuss the major steps, sequence, and precautions of infrequent procedures or tests. 4. Emergency/Abnormal Knowledge 4.1 Demonstrate a satisfactory knowledge of the emergency/abnormal situations listed below during an oral checkout. A satisfactory knowledge should include the following, as appropriate: 4.1.1 Describe reasons for the action(s) taken and the effect on the facility. Include the consequences of failure to perform the procedure or mitigate the abnormal or emergency situation. 4.1.2 State immediate actions. 4.1.3 Describe notification requirements for each situation. (List of facility systems)

5. Administrative Knowledge 5.1 Demonstrate a satisfactory knowledge of the items listed below during an oral checkout. 5.1.1 Describe each document's applicability to job responsibilities. 5.1.2 Describe administrative controls associated with each document. 5.1.3 Describe document locations. (List of facility documents) DOE-HDBK-1074-95 B-4 6. Operational Evaluation 6.1 This operational evaluation assesses the job performance capabilities of the candidate which cannot be evaluated by either written or oral examination. A Shift Manager/Supervisor or Shift Foreman will conduct the evaluation by closely observing the candidate perform the following: 6.1.1 Two normal operating procedures. 6.1.2 One emergency/abnormal procedure. 6.1.3 One process related drill. 6.2 Each item must be assigned by the Shift Manager/Supervisor/Foreman. A signature indicates that the candidate has demonstrated satisfactory knowledge and performance skills for the item. The comments section should be used to document strengths and/or weaknesses. (List the 2 normal procedures selected) (List the emergency/abnormal procedure selected) (List the facility drill selected) 6.3 Evaluator comments. DOE-HDBK-1074-95 C-1 APPENDIX C ON-THE-JOB TRAINING GUIDANCE This appendix contains an example of general instructions that could be used in conjunction with a well written procedure and a simple JPM or equivalent (that contains task-specific learning objectives, standards of acceptable performance, and questions to assess trainee knowledge). These documents should be sufficient to guide training and trainee evaluation in the formal OJT setting. These general instructions may be modified to meet facility-specific needs. OJT INSTRUCTOR GENERAL GUIDANCE Conduct training using techniques from OJT instructor training and in accordance with the task's procedure and JPM. Do not count a successful practice as an OJT evaluation. The higher levels of OJT require evaluation to be a separate activity. Preparation 1. Ensure that all materials are available, plant conditions are appropriate for OJT, and the trainee has completed prerequisite training. Conduct 1. Review the initial conditions for the procedure/JPM with the trainee. Explain how to perform the task. Emphasize the knowledge requirements contained in the JPM. 2. Demonstrate and explain the task until the trainee understands what to do, why to do it, and how to do it. 3. Allow trainee practice under direct supervision until proficiency is achieved. 4. Review the trainee's performance and answer any questions. Document completion of the training. DOE-HDBK-1074-95 C-2 Note the level of performance required for the task as specified in the qualification card/checklist or JPM. Levels of performance are perform (P), simulate (S), observe (O), or discuss (D). These levels are defined as: P - Perform the specified task using approved procedures and observing all applicable safety and administrative requirements. Include a thorough discussion (usually prior to performing the task) that addresses safety implications, critical steps, the elements involved, the effects on associated equipment or systems, and abnormal situations that may arise while performing the task.

S - Simulate performance of the specified task. Using approved procedures, "walk through" the task and simulate all actual manipulations (valves, switches, tools, etc.). Describe applicable safety and administrative requirements and the parameters (meters, charts, measurements, etc.) that would be observed/monitored during actual task performance. Conduct the same discussion as required for a perform level signature. O - Observe an individual performing the specified task. Conduct the same discussion as required for a perform level signature. D - Discuss the specified task using applicable procedures, piping and instrumentation drawings, blueprints, etc., including the discussion as required for a perform level signature. Demonstrate knowledge of the task by describing the manipulations required and the parameters that may be expected to change. Perform is the highest and preferred level of task performance. However, facility conditions may require that the next most appropriate level be selected to conduct the training. When the level of accomplishment is specified on the qualification card or JPM, conduct the training at that level. If conditions are not appropriate to perform a task, the training must be rescheduled or training and line management must agree to reduce the required level of accomplishment. Any reduction in level of accomplishment should be approved and documented. CONDUCT OF OPERATIONS GUIDELINES 1. Trainees are not allowed to operate any equipment except under the direct supervision of a qualified OJT Instructor. DOE-HDBK-1074-95 C-3 2. All equipment operation is conducted in accordance with approved facility procedures. 3. Trainee operation of equipment shall be immediately suspended during unanticipated or abnormal events, accident conditions, or whenever the OJT Instructor determines that suspension is necessary to ensure safe and reliable facility operation. 4. During abnormal or accident conditions, the trainee may provide assistance at the discretion and direction of the qualified operator. DOE-HDBK-1074-95 C-4 OJT INSTRUCTIONS TO THE TRAINEE Prepare for OJT by participating in prerequisite training and by reviewing the task's JPM or equivalent. Study reference(s) and become familiar with all requirements for passing the JPM. Ask questions while the instructor is training you to perform the task. Make sure you understand. Do not practice or perform any task or operate any equipment unless directed to do so by your instructor. 1. The purpose of the OJT session is to provide you with task-specific knowledge and skills in your job/work area. The knowledge and skills presented during on-the-job training are directly related to those you will perform on the job. 2. Before starting OJT, the OJT Instructor will state and discuss the learning objectives and answer any questions you may have. The OJT instructor will discuss the JPM with you so that you know the acceptable standards of knowledge and performance. 3. Before starting the training, the instructor will review the initial conditions for the procedure/JPM. The instructor will explain how to perform the task. 4. The instructor will demonstrate correct performance of the task. The instructor will answer questions and continue to explain and/or demonstrate performance until you understand what to do, when and why to do it, and how to do it. 5. You will be allowed time to practice the task until you can safely perform at the desired level of proficiency.

6. When you believe that you can perform the task safely and proficiently, you may request a performance evaluation. (Facility policy should specify whether the same OJT instructor can conduct the performance evaluation or if a separate evaluator or evaluation in necessary.) DOE-HDBK-1074-95 C-5 EVALUATOR INSTRUCTIONS After the trainee has practiced performing the task, a performance evaluation is conducted. A successful practice does not count as an OJT evaluation. The OJT evaluation is a separate activity. 1. Ensure the trainee has completed all prerequisite training. 2. Ensure that the materials required to perform the procedure/JPM (or equivalent) are on hand and available for use. 3. Before administering the performance evaluation, discuss the "Performance Evaluation Instructions" with the trainee. Provide a copy of the evaluation instructions as a reference. Review the initial conditions for the procedure/JPM with the trainee. 4. Review the criteria for passing the performance evaluation. The evaluator and the trainee must clearly understand what is expected. 5. The standard of acceptable performance may be written in the JPM, implied in the action statement, or stated in the procedure. The trainee is expected to perform the steps of the task in sequence unless the JPM indicates differently. Document trainee performance deficiencies on the JPM. 6. If a knowledge question is linked directly to a procedural step, ask the question before moving on to the next step. If it is not linked, ask the listed questions prior to or at the completion of the procedure/skills portion of the JPM. Compare the trainee's answer with the expected answer from the JPM, mark each question as "Satisfactory" or "Unsatisfactory" and briefly record the trainee's answer if the response was unsatisfactory. 7. Terminate the performance evaluation if the trainee's actions may result in danger to personnel or damage to equipment or the environment. Do not allow the trainee to commit an unsafe act. Intervene immediately to prevent or stop unsafe performance. The evaluator is fully responsible for the trainee and is required to take control of systems/equipment during an emergency. DOE-HDBK-1074-95 C-6 8. Evaluate only one trainee at a time. Do not "coach" the trainee during the performance evaluation. Coaching is providing help, prompts, or suggestions that would not be available during normal task performance. 9. The OJT performance evaluation is conducted on a pass/fail basis. If the trainee does not perform according to the JPM requirements, he/she fails. The trainee is expected to satisfactorily answer 80% of the knowledge questions. 10. Review the performance evaluation (covering strengths and weaknesses) with the trainee. 11. Document JPM results. 12. Provide feedback to appropriate management regarding any difficulties encountered using the procedure/JPM and generic instructions for the performance evaluation. DOE-HDBK-1074-95 C-7 PERFORMANCE EVALUATION INSTRUCTIONS TO THE TRAINEE During the OJT performance evaluation, you will not be allowed to ask questions, and the OJT instructor/evaluator is not allowed to give hints or help. The OJT evaluation is scored on a pass-fail basis. To pass, you must perform the task according to the procedure/JPM (or equivalent). In addition, you must pass the knowledge portion by satisfactorily answering the questions in the JPM with 80% accuracy.

1. The purpose of this JPM is to evaluate your knowledge and operating ability. The knowledge and skills evaluated by this JPM are directly related to those required on the job. 2. Before starting the JPM, the evaluator will state the objective, provide you with required information, and answer any questions. 3. At any point, the evaluator may stop the evaluation and ask questions regarding the steps, sequence, acceptance criteria, or the effects of actions. 4. If any critical steps (a procedural/JPM step or action that, if performed improperly during the actual performance of a task, could cause damage to equipment, the environment, or injury to personnel) or any two non-critical steps are not performed properly, or you perform a step out of the proper sequence when required, you will fail the JPM. If you fail to achieve a minimum score of 80% of the knowledge questions asked, you will also fail the JPM. 5. If you violate a Technical Safety Requirement or compromise facility safety or personnel safety, the evolution will be terminated. If the evaluation is terminated, you will be given remedial training and must retake the evaluation. 6. When instructed to by the evaluator, you will perform the task according to the procedure/JPM. DOE-HDBK-1074-95 C-8 Intentionally Blank DOE-HDBK-1074-95 D-1 APPENDIX D MINIMUM ELEMENTS FOR A SYSTEMATIC APPROACH TO TRAINING A systematic approach to training requires certain minimum activities, products, and elements. In order to substantiate a systematic approach and to provide an adequate level of assurance the training is consistent and job related, specific products, documentation, and processes must exist. This appendix identifies minimum products and documentation for formal systematic training and discusses the content of these minimum needs. The minimums identified in this appendix apply only to development and implementation of a systematic approach to training. DOE orders and regulations generally contain minimum prescriptive requirements that may exceed the requirements that result from systematically established training programs. The following information does not reflect all DOE training requirements. Users of this appendix must ensure that other applicable requirements are identified and implemented. PROCEDURES AND ADMINISTRATIVE GUIDANCE A level of consistency and guidance is necessary for the development and conduct of any formal training program. Even the facilities with minimum risk and hazard potential should have guidance that provides management with assurance of formality, quality, and consistency. Procedural or administrative guidance should be available in the following areas. A separate procedure is not required for each of these areas. 1. Roles and Responsibilities It is essential that persons involved in the development, presentation, and approval of training and training materials be clearly identified and understand the responsibilities, authority, and expected interface activities associated with their position. This may be accomplished with a simple designation of responsibility and authority by line management. The use of position descriptions is acceptable, provided these documents are current and adequately reflect the information necessary, commensurate with the organizational structure and need for formal definition. DOE-HDBK-1074-95 D-2 2. Training Development Process

Training activities should be guided by formal requirements. This guidance need not be lengthy or detailed for many facilities and organizations. If a description of the process is endorsed by management and the level of detail is sufficient for participants to understand and follow, the guidance is acceptable. 3. Conduct of Training There should be an organizational definition of the local requirements for conduct of training. This may include on-the-job training guidance, classroom training guidance, structured self-study guidance, or guidance in other areas that are important for the student and instructional staff to understand. A level that defines basic requirements (for example, qualified classroom instructors will use an approved lesson plan) is sufficient. 4. Training Program Evaluation All training programs should have some level of evaluation to assess the effectiveness of the training. This level will vary, but clear guidance is necessary regardless of the activities that are needed. Adequate evaluation guidance may consist only of documentation which designates areas subject to evaluation and required periodicity of evaluations. 5. Feedback and Corrective Actions Maintenance of the training materials and program activities is essential. Formal guidance should be available to ensure changes are identified and incorporated in the training in a timely manner. This guidance should also define application and use of industry and facility lessons learned information. 6. Records and Recordkeeping Any training program requires the maintenance of some records. To ensure consistency and credibility of records and recordkeeping systems, administrative or procedural guidance should be available. The guidance does not need to be complex or lengthy as long as it ensures compliance DOE-HDBK-1074-95 D-3 with DOE order requirements. Identification of retention, storage, disposition, and access/change authority should be sufficient for many programs. MINIMUM PRODUCTS AND DOCUMENTATION The products and documentation necessary to maintain a systematic approach are dependent on the techniques that are selected and applied in the program development and implementation. The traditional approach to SAT identifies efforts in distinct phases which are sequential. Alternative approaches integrate activities associated with phases and do not specifically require activities and phases to match the traditional approach. For ease of understanding, the following minimum products and documentation reflect products by specific phases. The minimum should be met, but the products do not necessarily have to be developed in the order described. 1. Analysis 1.1 Minimum Products C An approved, verified task list (or equivalent) which identifies train, no-train, and overtrain tasks. Equivalent is used to allow for the inclusion of regulatory or other requirements that may not necessarily be identified as job-specific in analysis. 1.2 Minimum Documentation C A description of techniques and participants. A short memorandum which simply states the technique(s) used and the function and name of the participants is sufficient for low-hazard level efforts. 2. Design 2.1 Minimum Products DOE-HDBK-1074-95 D-4 C An approved training summary or plan. C Learning objectives (or equivalent). C Examination, evaluation, and test-out requirements (may be included in the training summary or plan). C A task to training matrix (or equivalent).

The training summary or plan reflects the decisions made regarding the content, structure, and implementation of the training based on the train or overtrain tasks. This summary need only be at a level of detail and comprehension that assures line management that the needs for training will be met for this program. Equivalents for learning objectives are included for regulatory or local requirements that are content-specific or do not allow clear objectives to be developed. A task to training matrix to ensure that all tasks are included in the training program and for modification purposes when changes occur. An equivalent may be any system that tracks the training of tasks and allows identification of the impact changes have on the program elements. 2.2 Minimum Documentation C Approved summary or plan. C Evidence of subject matter expert involvement. Evidence of subject matter expert involvement may only require an SME signature on the plan submittal or some other method of identifying involvement. 3. Development 3.1 Minimum Products C Approved training materials. 3.2 Minimum Documentation DOE-HDBK-1074-95 D-5 C Evidence of subject matter expert involvement and line management involvement. 4. Implementation 4.1 Minimum Product C Trained employees. 4.2 Minimum Documentation C Attendance records. C Student evaluation results. C Evidence of appropriate instructor qualification. C Individual training records. 5. Evaluation 5.1 Minimum Products C Revised and current training materials, strategies, or settings. 5.2 Minimum Documentation C Records of evaluation results. C Records of approved changes made to training and a description of the considerations or drivers for the changes. The information must be sufficient to assure management that the proper revisions are being made for appropriate reasons. DOE-HDBK-1074-95 D-6 Intentionally Blank CONCLUDING MATERIAL Review activities: Preparing activity: DOE EH-33 Project Number 6910-0032 DOE Program Offices DP, EH, EM, ER, NE. DOE Operations Offices AL, CH, ID, NV, OR, RL, SR, OAK, RF. DOE Contractors ANL-W, BNL, LITCo, SNL EG&G Rocky Flats, LANL, LLNL, MMES, ORAU, REECo, WHC, FERMCo, WSRC. FOREWORD