DOE-HDBK-1122-99 Technician Training, Fundamental Academic Training Instructor's Guide Phase I; Radiological Control Technician Training

TS METRIC DOE-HDBK-1122-99 July 1999 DOE HANDBOOK RADIOLOGICAL CONTROL TECHNICIAN TRAINING U.S. Department of Energy AREA TRNG Washington, D.C. 20585 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. This document has been reproduced from the best available copy. Available to DOE and DOE contractors from ES&H Technical Information Services, U.S. Department of Energy, (800) 473-4375, fax: (301) 903-9823. Available to the public from the U.S. Department of Commerce, Technology Administration, National Technical Information Service, Springfield, VA 22161; (703) 605-6000. DOE-HDBK-1122-99 iii Foreword This Handbook describes an implementation process for core training as recommended in DOE Guide G441.1-1, Management and Administration of Radiation Protection Programs and as outlined in DOE’s Radiological Control Standard (RCS). The Handbook is meant to assist those individuals within the Department of Energy, Managing and Operating contractors, and Managing and Integrating contractors identified as having responsibility for implementing core training recommended by the RCS. While this Handbook addresses the training requirements of 10 CFR 835.103 for Radiological Control Technicians, it must be supplemented with facility- specific information to achieve full compliance. This Handbook contains recommended training materials consistent with other DOE core radiological training materials. The training material consists of the following documents: Program Management Guide - This document contains detailed information on how to use the Handbook material. Qualification Standard Fundamental Academic Lesson Plans Fundamental Academic Study Guides Site Academic Lesson Plans Site Academic Study Guides Site Practical Training Manual Oral Examination Board Manual Facility Practical Training Attachment This Handbook was produced in WordPerfect 8.0 and has been formatted for printing on an HP 4M (or higher) LaserJet printer. Copies of this Handbook may be obtained from either the DOE Radiation Safety Training Home Page Internet site (http://tis.eh.doe.gov/wpphm/rst/rst.htm) or the DOE Technical Standards Program Internet site (http://tis.eh.doe.gov/techstds/). Documents downloaded from the DOE Radiation Safety Training Home Page Internet site may be manipulated using the software noted above. DOE-HDBK-1122-99 iv This page intentionally left blank. DOE-HDBK-1122-99 Part 1 of 9 Radiological Control Technician Training Program Management Guide Coordinated and Conducted for Office of Environment, Safety & Health U.S. Department of Energy DOE-HDBK-1122-99 This page intentionally left blank. DOE-HDBK-1122-99 iii Course Developers William Egbert Lawrence Livermore National Laboratory Dave Lent Coleman Research Michael McNaughton Los Alamos National Laboratory Bobby Oliver Lockheed Martin Energy Systems Richard Cooke Argonne National Laboratory Brian Thomson Sandia National Laboratory Michael McGough Westinghouse Savannah River Company Brian Killand Fluor Daniel Hanford Corporation Course Reviewers Technical Standards Managers U.S. Department of Energy Peter O’Connell U.S. Department of Energy William D. Ulicny ATL International, Inc. DOE-HDBK-1122-99 iv This page intentionally left blank. DOE-HDBK-1122-99 v Table of Contents Page

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Purpose and Scope of Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Compliance with 10 CFR 835 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Purpose of RCT Core Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Organizational Relationships and Reporting Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Training and Qualification Program Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Overview of RCT Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Description of Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Initial Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Additional Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Continuing Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Requalification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Instructor Training and Qualifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 RCT First-level Supervisors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Site Training Program Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Job Evaluations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Site Academics Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Practical Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Qualification Standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Training Aids, Reference Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Evaluations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Academic Examinations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Final Comprehensive Written Examinations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Challenge Examinations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Practical Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Oral Examination Boards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Disqualification Procedures and Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Training and Qualification Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Training Program Development/Change Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 RCT Training Program Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Training for Subcontracted RCTs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Attachment A - Evaluating the Effectiveness of Radiological Training . . . . . . . . . . . . . . A-1 DOE-HDBK-1122-99 vi This page intentionally left blank. DOE-HDBK-1122-99 1 Introduction Purpose and Scope of Guide This guide defines and describes the DOE Radiological Control Technician (RCT) Core Training & Qualification Program, material development requirements, standards and policies, and administration. The guide applies to RCT Training & Qualification Programs at all DOE sites. Compliance with 10 CFR 835 The DOE core training materials for RCT Training reflect the requirements identified in 10 CFR 835, recommendations identified in DOE Guide G441.1-1, Management and Administration of Radiation Protection Programs, and in DOE’s Radiological Control Standard (RCS). When implemented in its entirety and supplemented as noted with appropriate facility-specific information, this handbook will generally meet the requirements of 10 CFR 835.103 as they pertain to RCT training. However, it is incumbent on management of each facility to review the content of this course to ensure that the training content is sufficient to ensure the RCT can fulfill his or her responsibilities for developing and implementing radiation protection measures. The core training described in this guide does not eliminate or reduce the need for additional site-specific training identified in the program and/or through a job evaluation. Site-specific learning objectives are flagged by a "�" in the left margin of the lesson plan. Sites should develop and implement the materials for flagged objectives as they apply to their sites. Each site is responsible for developing site- specific training as identified through a site-specific job evaluation. Purpose of RCT Core Training The purpose of this program is to provide guidance concerning DOE’s expectations for proficiency of RCTs across the DOE complex consistent with the requirements of 10 CFR 835 and the guidance in the RCS. Organizational Relationships and Reporting Structure DOE/EH is responsible for approving and maintaining the core course training materials associated with the RCT training program. The establishment of a comprehensive and effective site RCT Training & Qualification program is the responsibility of line management and their subordinates. The training function can be performed by a separate training organization but the responsibility for quality and effectiveness rests with the line management. Training and Qualification Program Description Next DOE-HDBK-1122-99 2 Training and Qualification Program Description

Overview of RCT Program Final approval of RCT qualification should be the responsibility of the Radiological Control Manager. RCT qualification should be granted only after assuring that all the requirements of training, examinations and work performance have been satisfied. By RCT qualification, Radiological Control management assures that the individual is capable of performing aspects of the tasks for which qualification was given. The initial qualification process should require: � successful completion of academic lesson examinations � successful completion of required job performance measures � successful completion of final comprehensive written examination � successful completion of an oral examination � final approval by Radiological Control Manager � compliance with experience requirements as identified in DOE Order 5480.20A The biennial requalification process should require: � participation in continuing training � successful completion of a final comprehensive written examination � successful completion of selected practical training � successful completion of an oral examination Description of Program The training program is based on RCS recommendations, industry guidelines, and the Guide to Good Practices in Radiation Protection Training. Entry-level prerequisites should be established by each site to ensure that RCTs meet standards for physical condition and education. At a minimum, these standards should include: � high school education or equivalency � fundamentals of mathematics, physics, chemistry, and science � systems and fundamentals of process, operations, and maintenance � reading and comprehension level sufficient to follow procedures, write permits, prepare survey maps, write reports, and prepare shipping and transfer permits � ability to work in the support role including communicating verbal instructions to others Continued Next Page Training and Qualification Program Description (continued) DOE-HDBK-1122-99 3 Description of Program (continued) • physical requirements to handle personal protective equipment, other equipment, and assist others in work location commensurate with assignment Initial Training The initial training program ensures that trainees are trained to meet performance requirements using a systematic approach to training. The initial training program contains four segments: Phase I, Phase II, Phase III, and Phase IV. Classroom instruction times may vary based on technician entry level qualifications and are not mandated. Phase I - Academic Training: Phase I is divided into two sections: Fundamental Academics and Site Academics. The lessons in these sections may be taught in any order provided that prerequisite lessons are completed first. Typically, the academic phase is approximately 240 hours in length. a. The Fundamental Academics section includes the following 13 lessons: 1.01 Basic Mathematics and Algebra 1.02 Unit Analysis and Conversion 1.03 Physical Sciences 1.04 Nuclear Physics 1.05 Sources of Radiation 1.06 Radioactivity and Radioactive Decay 1.07 Interaction of Radiation with Matter 1.08 Biological Effects of Radiation 1.09 Radiological Protection Standards 1.10 ALARA 1.11 External Exposure Control 1.12 Internal Exposure Control 1.13 Radiation Detector Theory These 13 lessons contain generic fundamental theory and do not contain any site-specific information. They represent the minimal standard information recommended to be taught to an RCT. These lessons should be taught by all DOE sites.

b. The Site Academics section contains the following 19 lessons: 2.01 Radiological Documentation 2.02 Communication Systems 2.03 Counting Errors and Statistics Continued Next Page Training and Qualification Program Description (continued) DOE-HDBK-1122-99 4 Initial Training (continued) 2.04 Dosimetry 2.05 Contamination Control 2.06 Airborne Sampling Program/Methods 2.07 Respiratory Protection 2.08 Radioactive Source Control 2.09 Environmental Monitoring 2.10 Access Control and Work Area Setup 2.11 Radiological Work Coverage 2.12 Shipment and Receipt of Radioactive Material 2.13 Radiological Incidents and Emergencies 2.14 Personnel Decontamination 2.15 Radiological Considerations for First Aid 2.16 Radiation Survey Instrumentation 2.17 Contamination Monitoring Instrumentation 2.18 Air Sampling Equipment 2.19 Counting Room Equipment These lessons include objectives covering general information relative to the topic and common to all DOE sites, and objectives where the information may vary with the site (identified by "�"). Each DOE site should determine appropriate site academic lessons and objectives based on job responsibilities and requirements as identified through a site-specific job evaluation. Facilities that have not conducted a job evaluation should teach all Phase I lessons and objectives. c. The standardized academics training is designed to be delivered in a classroom setting. Examples of alternate delivery methods are: � Self-study of Phase I academic materials in support of comprehensive challenge exams for personnel who exceed entry-level pre-requisites � Self-study for initial and continuing training � Computer-based training In all cases, regardless of the setting or delivery methods, examination requirements should be consistent with those specified in Chapter 6 of the RCS. Continued Next Page Training and Qualification Program Description (continued) DOE-HDBK-1122-99 5 Initial Training (continued) Phase II - Practical Training: Phase I academic objectives provide the bases for skills and tasks to be performed in Phase II. Practical training utilizes a mixture of classroom and hands-on training. RCTs are taught to apply academic knowledge to the site-specific tasks. a. The practical phase consists of two parts: training and evaluation. Further guidance on the development and conduct of practical training and evaluation is given in the "Practical Training Guide" section of this training program. b. Tasks should be demonstrated to the trainee with emphasis on the critical elements of the task. Explanations should be given to the importance of the tasks and the adverse effects if not performed properly. Training of tasks may occur at any time or any order provided the prerequisite learning objectives (academics) of the task have been taught and the proficiency of those objectives documented. All training should be conducted without placing either personnel or facilities at unnecessary risk. RCTs who are not qualified should be under the direct supervision of a qualified RCT. Direct supervision requires a qualified RCT to accompany the trainee and be capable of intervening, if necessary. c. Trainees who have demonstrated the ability to perform a task should be evaluated using the Job Performance Measures (JPMs). The JPMs identify the knowledge and skills needed to accomplish the task. The trainees should be evaluated on a satisfactory/unsatisfactory basis for each task. Once a trainee demonstrates task qualification, and has obtained the appropriate signatures, he/she may perform that task without direct supervision.

Phase III - Oral Examination Boards: a. Oral examination boards should be used to validate the training of candidates for RCT positions. The board should assess the individual's response to normal and emergency situations. Questioning should not be of the type that could be covered in a written examination. b. The Radiological Control Manager should designate the board members and appoint a chairperson. The board should be comprised of at least three persons to include a Radiological Control Supervisor, Radiological Control staff, and line Continued Next Page Training and Qualification Program Description (continued) DOE-HDBK-1122-99 6 Initial Training (continued) management operations department supervisors and staff personnel, as applicable. c. Further guidance on the development and conduct of oral examinations is given in the "Oral Examination Boards" section of this training program. Phase IV - Facility Practical Training: a. At DOE sites with more than one facility and where RCT tasks at each facility may differ, site and facility tasks should be separated. The tasks that are common to all the facilities on the site should be included in Phase II training with the core tasks. Tasks unique to a facility should be added to the training program qualification standard as an attachment, as Phase IV training. b. Not all DOE sites will include Phase IV training in their programs. Phase IV training allows each site to qualify technicians to a select facility. The transfer of technicians between facilities requires that facility tasks be taught and RCT core qualification is current. c. Further guidance on the development and conduct of facility practical training is given in the "Facility Practical Training Guide" section of this training program. Additional Training Personnel entering the initial RCT training program should receive Radiological Worker II training if they are required to perform unescorted duties as a Radiological Worker or enter radiological areas prior to receiving the core academic instruction. Qualified RCTs satisfy the requirements of Radiological Worker (RW) training. Since RCTs perform radiological workers’ tasks (don/doff PCs, self-survey, comply with RWPs, etc.), radiological worker practical skills should be taught. Each site has the option of including RW training in their RCT program or to have RCTs go through the practical factors portion of RW training. The advantage of going through the RW practical factors is the resulting consistency of the program, where RCTs become refreshed on RW practices being taught and are thus less likely to direct different practices in the field. Continued Next Page Training and Qualification Program Description (continued) DOE-HDBK-1122-99 7 Additional Training (continued) RCTs may require additional training to enhance their overall performance and skills. These courses may include specialized training in CPR, first aid, hazardous material training, as well as human resources training in communication skills and assertiveness. Each contractor site should determine these requirements and ensure that they are met as applicable to RCTs at that site. Continuing Training Continuing training programs should be designed and implemented to maintain and enhance the proficiency of RCTs. This training provides the RCTs with review of required knowledge/skills, introductions to new knowledge/skills, lessons learned from others, and verifies by examination minimum levels of proficiency. Topics addressed in continuing training should include:

� selected topics from Phase I and Phase II training � new procedures and changes to existing procedures that have an impact on radiological controls � new equipment and changes or modifications to existing equipment or facilities � lessons learned from site operating experiences � lessons learned from industry operating experiences � identified deficiencies or poor task performance � items identified as requiring repetitive training because of high degree of difficulty and/or criticality or infrequent performance � annual retraining on emergency preparedness, including drills Experience has shown that effective continuing training programs incorporate the following features of technical training: � Lessons learned training that is embedded in lesson plans and emphasizes important points. � Exercises that utilize procedures in training materials, such as procedure use exercises that are interesting and hold the RCTs attention. � Concept application techniques that link theoretical concepts to practical applications. Continuing training should include written examinations as applicable, demonstrations of proficiency controlled by qualification standards and written and oral examinations to prepare for the comprehensive biennial requalification. Continued Next Page Training and Qualification Program Description (continued) DOE-HDBK-1122-99 8 Continuing Training (continued) Continuing training courses should be shared among DOE facilities to reduce redundant development effort and to maintain standardization. This shall be accomplished through the Training Oversight Group. For further guidance on the development of continuing training programs see DOE-STD-1060-93 "Guide to Good Practices for Continuing Training." All RCTs should complete a minimum of 40 hours of continuing training for each year of the biennial requalification cycle. Requalification A 2-year cycle of continuing training is recommended for requalification. Continuing training should serve as the basis for requalification. Biennial requalification should encompass a representative cross- section of Phase I learning objectives, site-specific and department- wide changes in requirements, lessons learned from operations and maintenance experience, and selected tasks. A comprehensive written examination and oral examination board are recommended within the last six months of the biennial requalification cycle. Requalification should consist of participation in continuing training, successful performance of tasks selected for requalification, successful completion of a comprehensive written examination and successful completion of an oral examination board. Instructor Training and Qualifications All classroom instruction should be provided by instructors qualified in accordance with the site's instructor qualification program. Training staff (contractor and subcontractor, if used) must possess both technical knowledge and experience, and the developmental and instructional skills required to fulfill their assigned duties. Training staff responsible for program management, supervision, and development must have and maintain the education, experience, and technical qualifications required for their jobs. Instructors should have the technical qualifications, which include adequate theory, practical knowledge, and experience, for the subject matter that they are assigned to teach. Continued Next Page Training and Qualification Program Description (continued)

DOE-HDBK-1122-99 9 Instructor Training and Qualifications (continued) Methods should be in place at each site to ensure that individual instructors meet and maintain position qualification requirements. Subject-matter experts without instructor qualification may provide training in their area of expertise. However, these subject-matter experts should be trained as instructors when this occurs routinely. RCT First-level Supervisors Direct or first-level supervisors of RCTs should have qualified as RCTs prior to entering the RCT Supervisor training & qualification program. RCT supervisors should have a greater depth of knowledge than that expected of an RCT. RCT supervisors should have supervisory and leadership capabilities to direct the work of RCTs; effectively interact with crafts, line supervisors, professional staff and other managers; and be able to respond and direct others in emergency and abnormal situations. RCT supervisors should participate in either RCT supervisor continuing training or RCT continuing training. RCT supervisors should be requalified every 2 years through comprehensive oral examination boards. Oral examination boards should focus on the ability to analyze radiological conditions and unusual situations and supervise subordinates. The Board constituted to evaluate RCT supervisor qualification should not include peers or subordinates as voting members. Site Training Program Development Next DOE-HDBK-1122-99 10 Site Training Program Development Job Evaluations The core course training materials provide the minimum basis for RCT training. However, in order to maintain the performance-based philosophy, a job evaluation should be performed to assess the needs of each individual DOE facility. A job evaluation is a documented review of RCT job scope and responsibilities. While a complete job/task analysis is desirable, a job evaluation provides the minimum amount of analysis to determine the tasks performed by an RCT. A job evaluation consists of: � development of a list of tasks performed by RCTs at the site � evaluation of the task list for training applicability and methods � comparison of identified tasks and associated topics to the site academic materials to determine which lessons and objectives apply � comparison of site task list to the core task list for applicability Site Academics Objectives The Site Academics section of Phase I training provides a broad scope of topics associated with RCT tasks. Inclusion of Phase I site training materials is dependent on the site-specific RCT duties. The job evaluation should be used to provide the basis for exclusion of site-specific learning objectives ("�") from the site program. The job evaluation should also be used to identify additional RCT tasks which require training. Some Site Academics objectives may be satisfied by other training courses attended by RCTs. Practical Training Each site should determine tasks for initial practical training of RCTs at that site. Below is a list of tasks that are performed by RCTs at most DOE sites. This list should be considered for inclusion in the site task list. In addition, each site may add other site-specific tasks as appropriate. The core task list is as follows: Continued Next Page Site Training Program Development (continued) DOE-HDBK-1122-99 11 Practical Training (continued)

• Perform a contamination survey • Perform a radiation survey • Obtain air samples • Perform a leak test on a radioactive source • Complete a performance test on portable hand held instruments • Complete a performance test on health physics counting equipment • Post a radiological area to reflect associated hazards • Perform a radioactive material shipment survey • Respond to a high airborne activity alarm • Respond to an uncontrolled release of radioactive material • Respond to a radiation alarm • Respond to an injured person located in a radiological area • Direct and monitor personnel decontamination Job Performance Measures (JPMs) should be used to assess the trainee's ability to perform identified tasks. Guidance on the development of JPMs is given in the "Practical Training Guide" section of this training program. Qualification Standards A qualification standard states the academic and practical requirements necessary for the successful completion of the RCT Program. Signatures on the standards should document satisfactory proficiency for the performance requirements. A list of personnel authorized to sign qualification standards should be maintained at each site. Guidance on the development of Qualification Standards is given in the Technician Qualification Standard section of this training program. Training Aids, Reference Materials Designated viewgraphs should be used. Site-specific training aids may be developed. Reference materials should be cited in each lesson plan. Evaluations Next DOE-HDBK-1122-99 12 Evaluations Academic Examinations Written examinations should be used to demonstrate satisfactory completion of academic training. Requirements for examinations include: � The minimum passing grade for examinations should be 80%. � Questions should be randomly selected from the supplied question bank. � Each RCT should be tested on all learning objectives through a combination of questions in the topic (Phase I) examinations and final comprehensive examination (after Phase II) for initial qualification. � One exam bank question may cover several learning objectives. The trainee should acknowledge, by signature, his/her participation in a post-examination review. Each site should designate a single point of contact as an examination bank custodian. DOE will provide a copy of a sample exam bank to exam bank custodians. An examination question bank should be maintained by the site examination bank custodian. Recommendations for exam banks and examinations include: � The examination question bank should include at least three (3) questions per learning objective � Each question in the exam bank should be numbered in accordance with the corresponding learning objective. � The standardized core course examination question bank should include Phase I academic material except for Site Academics learning objectives flagged by "�" as site-specific. � Questions may consist of multiple choice, fill in the blank, short answer, matching, and essay. In order to minimize test items requiring rote memorization, short answer and essay questions are the preferred method of testing. True/false questions should not be used. � Short answer and essay questions should have model answers to reduce the subjectivity in grading. All site-specific site exam-bank questions should be maintained by the site and should be developed for the flagged ("�") learning objectives. Continued Next Page Evaluations (continued)

DOE-HDBK-1122-99 13 Academic Evaluations (continued) Written examinations which cover objectives from one or more academic lessons should be administered at periodic intervals during Phase I to assess trainee classroom progress. Examinations generated by the site for academic lessons should use questions from the DOE sample exam bank and each site's site- specific exam bank. Remedial action for trainees who fail a topic examination should be the responsibility of each site. Final Comprehensive Written Examinations A final comprehensive written examination should be administered following the completion of Phase II and within the last six months of biennial re-qualification. It should cover a representative sample of the learning objectives from Fundamental Academic and appropriate Site Academic lesson plans. Students who fail the final comprehensive written examination should complete a formal remediation in the areas of weakness. The remediation requirements should be documented in a formal written plan and presented to the student. As a minimum, the remediation should include: � a review of the weaknesses noted by the final written examination � counseling to determine the depth and nature of the weaknesses and appropriate action based on the counseling � any retraining or self-study considered necessary to upgrade the areas of weakness Students failing three final comprehensive written examinations should be considered for dis-enrollment from the RCT program. Challenge Examinations Challenge examinations should be administered after a review of trainee records. The review should include: � previous work history � previous training and education Challenge examinations should be utilized for individuals with prior training, NRRPT registration, education, and experience. Study materials may be provided to individuals prior to examination. Continued Next Page Evaluations (continued) DOE-HDBK-1122-99 14 Challenge Examinations (continued) Challenge examinations may be administered for a group of academic lesson plans or for individual lesson plans. Challenge examinations should consist of questions from a representative sample of learning objectives for the lessons being challenged. Challenge examinations should be composed of questions from the academics examination question bank. Identified weaknesses in performance on challenge examinations should require formal remedial training and successful re-testing. Practical Training Qualification for each task should be achieved by actual task performance. When the actual task cannot be performed but is simulated, the conditions of the task performance, references, tools, and equipment should reflect the actual task to the fullest extent possible. Trainees are evaluated on a satisfactory/unsatisfactory basis for each on-the-job-training (OJT) task. Trainees should be given sufficient time to practice the task before an evaluation is conducted. Any practical training completed with less than 100% proficiency on critical steps constitutes a failure. Failure of any task will require remedial action, which may include a repeat demonstration of the task by an OJT trainer/evaluator or allowing the trainee to perform the task with direct supervision. Candidates failing three evaluations on the same task should be considered for dis-enrollment from the RCT program. Further guidance on the conduct of practical training is given in the "Practical Training Guide" section of this training program.

Oral Examination Boards The oral examination board provides an opportunity to identify areas of weakness related to performance of RCT duties. Candidates who fail an oral examination should complete a formal remediation in the areas of weakness. The remediation requirements should be documented in a formal written plan and presented to the candidate. Continued Next Page Evaluations (continued) DOE-HDBK-1122-99 15 Oral Examination Boards (continued) Candidates failing three final oral examinations should be considered for dis-enrollment from the RCT program. Guidance on the conduct of oral examination boards is given in the "Oral Examination Board Manual" section of this training program. Disqualification Procedures and Policies Any trainee who does not successfully complete a final comprehensive examination or oral examination board should be considered for disqualification. Disqualification means that an individual cannot perform RCT tasks unless accompanied by and under the direct supervision of a qualified RCT. Any trainee who does not successfully complete an evaluated on- the-job task should be disqualified from further performance of that task. Disqualification means that an individual cannot perform the specific task(s) unless accompanied by and under the direct supervision of a qualified RCT. Disqualified individuals should receive a management review of individual circumstances to determine appropriate actions. An RCT who fails to complete continuing training requirements during the requalification period should be disqualified. Site- generated exceptions may be made for continuing training based on long- or short-term medical leaves, use of subcontractors, etc. These exceptions may be necessary if an RCT is unable to attend continuing training, and make-up of that cycle is not practical. Administration Next DOE-HDBK-1122-99 16 Administration Training and Qualification Records Training records and course documentation shall meet the requirements of 10 CFR 835 Subpart H and should meet Chapter 7 of the DOE RCS. Examinations should clearly identify missed questions and scores. Recommended signatures for RCT qualification are: a. Completion of Phase I (Academic Training): 1) Instructor signature on qualification standard stating trainee has met the requirements of the learning objectives through examination. Following successful completion of challenge examinations, the trainee should receive credit for the academic material covered by the challenge examination. 2) Trainee signature on the examination acknowledging participation in a post-exam review b. Completion of Phase II (Practical Training): 1) Trainee signature on the qualification standard verifying performance of the task 2) OJT trainer/evaluator signature on the qualification standard verifying satisfactory trainee performance of the task c. Completion of final comprehensive written examination: 1) Instructor signature on qualification standard stating trainee has met the requirements of the learning objectives through examination. 2) Trainee signature on the examination acknowledging participation in a post-exam review d. Completion of Phase III (Oral Examination Board): 1) Signature by the chairperson of the oral examination board on the qualification standard verifying the trainee's satisfactory performance, based on the assessment of all voting members Continued Next Page Administration (continued) DOE-HDBK-1122-99 17

Training and Qualification Records (continued) 2) Trainee signature verifying successful completion of the oral examination e. Final Approval: 1) Radiological Control Manager (or designee) signature on the qualification standard acknowledging final approval of the standardized technician position qualification. f. Completion of Phase IV (Facility-specific Practical Training) - optional: 1) Trainee signature on the qualification standard verifying performance of the task 2) OJT trainer/evaluator signature on the qualification standard verifying successful trainee performance of the task 3) Radiological Control Manager (or designee) signature on the qualification standard acknowledging approval of facility qualification Training Program Development/Change Requests All requests for program changes and revisions should be submitted through the appropriate DOE technical standards change process. RCT Training Program Evaluation Verification of the effectiveness of Radiological Control training should be accomplished by surveying a limited subset of former students in the workplace. This evaluation should include observation of practical applications and discussion of the course material, and may include an associated written examination. DOE/EH has issued guidelines for evaluating the effectiveness of radiological training through the DOE Operations Offices and DOE Field Offices. These guidelines are included as an attachment to this Program Management Guide. For additional guidance, refer to DOE-STD-1070-94, “Guide for Evaluation of Nuclear Facility Training Programs.” The guidelines contained in these documents are relevant for the establishment and implementation of post-training evaluation and retention testing programs. Continued Next Page Administration (continued) DOE-HDBK-1122-99 18 RCT Training Program Evaluation (continued) In response to the Defense Nuclear Facilities Safety Board (DNFSB) Recommendation 91-6, DOE committed to develop an implementation plan to upgrade radiation protection programs at DOE defense nuclear facilities. The implementation plan detailed DOE’s plans to develop and implement radiation protection post- training evaluation and retention testing programs. Post-training evaluations will be used to identify opportunities for improving course materials, upgrading instruction methods and techniques, and determining the need for additional training. Retention testing will indicate when individual performance or testing fails to meet expectations. Corrective actions for deficiencies identified in retention testing will be incorporated in the individual’s development plan and the site’s training program on an appropriate schedule. In addition, Chapter 6 of the DOE RCS states that sites should implement a training effectiveness verification program. This program, which is in addition to performance evaluations routinely performed by the site’s training department, is to verify the effectiveness of radiological control training by surveying a limited subset of former students in the workplace. This recommendation applies to both DOE defense nuclear facilities and DOE facilities not classified as defense nuclear facilities. Per DOE’s commitment to DNFSB, it is expected that all defense nuclear facilities will implement these or equivalent programs. DOE facilities not classified as defense nuclear facilities should also strive to implement such programs. Line management should monitor progress of program implementation.

Training for Subcontracted RCTs Subcontracted RCTs should have the same knowledge and qualifications required of site technicians performing the same duties. Subcontracted RCTs should meet the requirements as outlined in Chapter 6 of the DOE RCS. References Next DOE-HDBK-1122-99 19 References REFERENCES U.S. Department of Energy, DOE-STD-1098-99, “Radiological Control Standard.” U.S. Department of Energy, “Occupational Radiation Protection,” 10 CFR 835, 1998. U.S. Department of Energy, “Implementation Guidance for Use with 10 CFR 835, Occupational Radiation Protection,” 1999. U.S. Department of Energy, DOE Order 5480.2A, “Personnel Selection, Qualification, and Training for DOE Nuclear Facilities,” 1994. DOE-HDBK-1122-99 20 This page intentionally left blank. DOE-HDBK-1122-99 ATTACHMENT EVALUATING THE EFFECTIVENESS OF RADIOLOGICAL TRAINING REV 1, 1998 Guidelines to Establish and Conduct a Post-Training Evaluation Program and a Retention Testing Program DOE-HDBK-1122-99 A-ii TABLE OF CONTENTS I. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1 A. Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1 B. Purpose of Post-Training Evaluation and Retention Testing . . . . . . . . . . . . . . . . . . . . . . A-2 C. Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-2 D. Guidance From Department of Energy Standards (DOE-STD) 1070-94 . . . . . . . . . . . . A-2 E. DOE Radiological Control Standard (RCS), Article 613.7 . . . . . . . . . . . . . . . . . . . . . . . A-3 F. Elements of Article 613.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-3 G. Qualifications of Evaluators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-3 H. Correlation Between Evaluation and Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-3 I. Training Requirements Based on Employee Duties . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-4 J. Stakeholder Consensus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-4 K. A Checklist for Establishing and Implementing a Post-Training Evaluation Program . A-4 II. PROGRAM ADMINISTRATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-4 A. Schedules For Conducting Post-Training Evaluation and Retention Testing . . . . . . . . . A-4 B. Sample Sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-5 C. Interpreting Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-7 III. GUIDELINES FOR COURSE POST-TRAINING EVALUATION AND RETENTION TESTING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-9 A. General Instructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-9 B. General Employee Radiological Training (GERT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-9 C. Radiological Worker I Training (RW I) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-10 D. Radiological Worker II Training (RW II) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-12 E. Radiological Control Technician Training (RCT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-14

IV. REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-21 DOE-HDBK-1122-99 A-1 I. INTRODUCTION This guideline supports implementation of the "U.S. Department of Energy Radiological Control Standard," article 613.7 (1998). A. Purpose The purpose of a training program and its associated evaluation phase is to assure that workers are qualified to perform their jobs competently and safely. This purpose should be kept foremost while establishing the procedural controls of the post-training evaluation process at each site. The purpose of this document is to aid DOE sites in developing procedures and practices to conduct post-training evaluation of the following four core radiological control training courses: GERT, RW I, RW II, and RCT training. Similar procedures and practices should be applied to post-training evaluation of the supplemental radiological control training courses as they become available. Several of the key elements of a post-training evaluation program are also applicable to the establishment and implementation of a retention testing program. A retention testing program differs from a post-training evaluation program in that the results of a retention testing program are used to assist in: (1) identifying when individual performance fails to meet expectations; (2) identifying adverse trends in radiological performance; and (3) correcting individual performance deficiencies. The results of a post-training evaluation program are directed towards improving the training program rather than correcting individual performance deficiencies. This document provides guidance on the issues and practices that should be considered in setting up such programs. It is not intended to prescribe specific practices that must be followed by all sites. The diversity of conditions among the various DOE sites will require that the application of good practices recommended in this guideline may differ from site to site. That diversity includes the nature of the training organization, the nature of the trainees, the number of trainees, and the frequency in which the trainees will apply the training. Conducting evaluations will require training evaluators at each site. In particular, the consistency in which any set of evaluators will rate the skill and knowledge levels of the former trainees will need to be emphasized. It is essential that the judgement of skill levels not vary appreciably from one evaluator to the next. Expertise to conduct such training may be available at the site or may be acquired from offsite sources. Several written references are provided. The results of post-training evaluations may be used to upgrade core course materials as described in the Program Management Guides. DOE-HDBK-1122-99 A-2 B. Purpose of Post-Training Evaluation and Retention Testing Post-training evaluation has four primary purposes: 1. To measure the retention of skills and knowledge provided during training for extended periods following the training; 2. To measure the degree to which the training is used on the job; 3. To provide feedback to improve the training; and 4. To measure the cost effectiveness of the training. A primary purpose of retention testing is to incorporate corrective actions into individual development plans based on deficiencies observed during the retention testing process. C. Assumptions

1. These guidelines will be specific to evaluation of the radiological control training contained in the core training materials. 2. The applications of these guidelines will be unique for each DOE site. 3. Each DOE site will formalize its Radiological Safety Training Post-Training Evaluation Program and Retention Testing Program in site procedures. 4. These guidelines summarize the requirements of other documents, which explain more fully the process of training evaluation. 5. These guidelines are sufficient for knowledgeable individuals to use to create site controlling procedures. 6. Personnel engaged in post-training evaluation or retention testing will require some training or previous evaluation experience. 7. Individual sites may choose to include this evaluation program as part of site-wide evaluation of training. 8. Other radiological control training may be considered for evaluation along with core training. 9. The phrase "in the workplace" in the RCS cited below is taken figuratively to mean at the work location. While observation of skills would be preferable while engaged in real work at the actual workplace, the discussion of content may or may not be best done at that location. Written testing would be best done in a classroom setting where quiet and privacy may be maintained. D. Guidance From DOE-STD-1070-94 DOE-STD-1070-94, "Guidelines For Evaluation of Nuclear Facility Training Programs," appendix A, objective 8.0, "Training Program Evaluation," contains seven criteria. Three of these criteria are applicable to the purpose of these guidelines: Criteria 8.1: A comprehensive evaluation of individual training programs is conducted by qualified individuals on a periodic basis to identify program strengths and weaknesses. Criteria 8.3: Feedback from trainee performance during training is used to evaluate and refine the training program. Feedback from former trainees and their supervisors is used to evaluate and refine the training program. DOE-HDBK-1122-99 A-3 Criteria 8.5: Improvements and changes to initial and continuing training are systematically initiated, evaluated, tracked, and incorporated to track training deficiencies and performance problems. E. DOE Radiological Control Standard, Article 613.7 Article 613.7 states: Verification of the effectiveness of radiological control training should be accomplished by surveying a limited subset of former students in the workplace. This verification is in addition to performance evaluations routinely performed by training departments. This evaluation should include observation of practical applications, discussions of the course material, and may include written examinations. The survey should be performed by radiological control managers and supervisors, quality assurance personnel, or senior instructors after the former student has had the opportunity to perform work for several months. The results should be documented. F. Elements of Article 613.7 The following elements are stated: 1. The purpose of the evaluation is to verify the effectiveness of radiological control training. 2. The survey should be made on a subset of students several months after the training. 3. The survey may be conducted by training or nontraining personnel. 4. The survey should be conducted at the workplace. 5. The survey should be conducted in addition to any evaluations made as part of the training process. 6. The survey should consist of:

o Observation of practical applications of the training; o Discussion of the course material with the trainee; and o An optional written examination, as appropriate, to measure retention. 7. The results should be documented. G. Qualifications of Evaluators Personnel who perform the evaluations should be technical experts on radiological control practices and should possess, or be trained to develop, interviewing skills and observational skills. H. Correlation Between Evaluation and Training The radiological control core training courses are nationally standardized. Any changes to the training materials, including the test banks, should be correlated by training oversight groups. How that correlation will be conducted is described in the program management guides for the courses. Site specific radiological control training may be evaluated and revised by that site. DOE-HDBK-1122-99 A-4 I. Training Requirements Based on Employee Duties Table 3-1 of the RCS displays the types of workers and the recommended level of radiological training. J. Stakeholder Consensus Effective evaluation of training can only be conducted in a climate of consensus by all stakeholders. An essential element in the evaluation process is the development of consensus in advance of the evaluation. This involves agreement on the objectives of the evaluation, methods of evaluation to be used, and the required level of support that can be provided. Agreement should also be reached on whether the evaluation process will undergo the same approval cycle as that of other training materials. Note that it may not be possible to obtain the same level of consensus in the development of a retention testing program, as the results of this program are used to provide remedial training for individuals exhibiting performance deficiencies. The retention testing program must be conducted in a manner such that there is no question as to the fairness and unbiased manner in which the program is implemented. K. A Checklist for Establishing and Implementing a Post-Training Evaluation Program The following steps are essential in formalizing the post-training evaluation process and should be considered in developing site procedures: 1. Formalization via procedures of radiological control post-training evaluation; 2. Establishment of evaluation goals, objectives, and schedule, including building consensus among stakeholders; 3. Qualification of evaluators; 4. Preparation of evaluation materials; 5. Conduct of evaluations; 6. Reporting evaluation results; and 7. Program revision as applicable. Note that this checklist is also applicable to a retention testing program. II. PROGRAM ADMINISTRATION A. Schedules For Conducting Post-Training Evaluation and Retention Testing It is suggested that based on predicted proficiencies, post-training evaluation should be scheduled 3 to 6 months following the training. Conducting the evaluation at least 3 months following the training would provide an opportunity for the trainee to have used the skills on the job, while not extending the evaluation beyond 6 months to assure that some level of retention could be expected. Site specific conditions may make it desirable to extend the period between training and evaluation, but data to support the extension decision should be collected to assure that the purposes of the evaluation are being met. Note that this schedule is also applicable to a retention testing program.

DOE-HDBK-1122-99 A-5 B. Sample Sizes The subset of former students to be sampled should be large enough to provide valid and reliable data for meeting the needs of the training and radiological control organizations. The number selected to be evaluated depends on the degree of precision and reliability desired. Individuals to be evaluated should be selected using random number tables or some other randomizing technique unless the population size is so small that the randomizing effort will be of no value. Table 1 is a matrix that describes a suggested level of sample size that can provide an appropriate level of comfort and can be used to establish the sample sizes to be used at each site. The percentages are based on the information provided in Dr. N.M. Dixon's "Evaluations: A Tool for HRD Quality." Note that these sample size guidelines also apply to a retention testing program. DOE-HDBK-1122-99 A-6 TABLE 1 Determining Sample Size for Post-Training Evaluations and Retention Testing Number of Trainees Per Year Per Course Percent of Trainees to Be Evaluated 1-40 41-100 101-160 161-250 251-380 381-600 601-900 901-1600 1600-3500 Over 3500 90 - 100% X 80 - 89% X 70 - 79% X 60 - 69% X 50 - 59% X 40 - 49% X 30 - 39 % X 20 - 29% X 10 - 19% X 3 - 9% X DOE-HDBK-1122-99 A-7 C. Interpreting Results Interpretation of results should be based on the original purpose(s) for conducting the evaluation. Interpretations can be centered on questions to be asked of the data. Table 2 shows samples of the type of questions that could be asked for each of the four purposes for evaluating training. DOE-HDBK-1122-99 A-8 TABLE 2 Interpreting Evaluation Results Purpose for Evaluation Questions to Ask of Data To measure the retention of skills and knowledge provided during training for extended periods following the training. a. What is the average percentage of retention? b. What is the range of retention (highest and lowest)? c. Which objectives (knowledge/skill) were best retained? Which were least retained? d. How frequently were the least retained knowledge/skills used on the job? e. How many trainees fell below minimum passing requirements for the class? Why? f. What other variables, such as additional training, might have had an impact on retention? To measure the degree to which training is used on the job. a. How frequently is the knowledge/skill from training used on the job? b. Does the way in which a task is taught in training match exactly the way in which it is performed on the job? c. How is management/supervision reinforcing the knowledge/skills taught in the training at the worksite? To provide feedback to improve the training. a. Which knowledge/skill is not being used on the job? Why? b. If knowledge/skill is not retained, but is essential to job function, how can training make the learning of that skill more job like? c. What is the opinion of trainees regarding the benefits of the training after the extended time period? d. Were the training materials retained by the trainee after the class, and were they helpful as references back at the worksite? e. Have the trainees made any contact with instructors since the class for assistance? For what purpose? Was the contact helpful? f. What knowledge/skills need expanded or additional training? To measure the cost effectiveness of the training. a. What is the actual cost of delivering this training? b. Does the data suggest that trainees were not positively affected by the training?

How? c. Is the knowledge/skill provided by the training directly related to the work the trainee performs? d. What percent improvement in job performance resulted from completion of the training? To incorporate corrective actions into individual development plans based on deficiencies observed during the retention testing process. a. What percentage of other individuals performed satisfactory in retention testing? b. How frequently were the unretained skills used on the job? c. How much remedial training has the individual already received? d. What changes have occurred that resulted in the individual completing original training, but failing retention testing? e. How critical to the individuals job function were the deficient items identified in retention training? f. How likely is enhanced training going to result in improving the individuals performance to an acceptable level? DOE-HDBK-1122-99 A-9 III. GUIDELINES FOR COURSE POST-TRAINING EVALUATION AND RETENTION TESTING A. General Instructions It may be beneficial and appropriate at some sites to combine in-field practical exams with training content discussions. Thus, the observer can pose the identified questions while observing actual work being performed. 1. In-Field Observation: Five skill areas could be included in in-field observations: prejob preparations, knowledge level, radiological practices, emergency response practices, and exit practices. A checklist could be developed using a weighted point value system and identifying critical practices. The checklist could be modified to fit individual site requirements. 2. Discussion Questions: Discussion with employees should determine whether the knowledge acquired in the training has been transferred to the job. The intent is not simply to measure retention of specific facts. The depth of the discussion would be directly correlated to the level of radiological hazard, which may be encountered on the job. 3. Written Exam: Written exams for evaluation of training effectiveness are optional, but may be used to evaluate the effectiveness of the training. These written exams may test retention of specific content from the courses. It may be desirable to use written tests that measure the ability to apply the knowledge from the training. In that instance, questions could be developed to measure student reaction to case studies. Such case study questions would allow each site to tailor the testing to its particular facilities, procedures, practices, and conditions. B. General Employee Radiological Training (GERT) 1. In-Field Observation: Some sites may desire to survey GERT trainees to determine their knowledge of hazardous areas to avoid. 2. Discussion Questions: GERT trained employees access controlled areas. They should be aware of the hazards they may face in an emergency and how they should respond to those hazards. The following questions may be part of the discussion: a. How many millirems of exposure do you normally receive on your job in a year? b. What is your annual dose limit; administrative dose guideline; when was the last time it was reported? DOE-HDBK-1122-99 A-10 c. What types of radiation hazards exist at your facility? d. How do you practice As Low As Reasonably Achievable (ALARA) in your work? e. What is the sound of the emergency alarms for your workplace? f. How are employees at your work area warned of radiation hazards? g. What obligations do you have to maintain exposure to radiation ALARA? h. What is dosimetry and what precautions should be taken when handling

dosimeters? i. What radiological training are you required to take? How does your training relate to your job? 3. Written Exam: GERT trained employees are not required to pass a written examination. C. Radiological Worker I (RW I) 1. In-Field Observation: Table 3 is a sample chart, which illustrates the in-field activities and weighing system that could be used to observe RW I trainees. The numbers shown are based on conditions and priorities at a particular DOE site. Other sites may determine a different weighing system. Sites could choose to develop scenarios or drills in which to conduct in-field observations, or simply observe workers in their normal working environments. DOE-HDBK-1122-99 A-11 TABLE 3 Sample RW I In-Field Observations Chart ITEM EVALUATED POINT VALUE SCORE Stated purpose of entry 2 Stated radiation levels (w/units) 6 Stated special instructions listed on radiological work permit (RWP) 6 Selected dosimetry in accordance with RWP 4 Dosimetry worn in accordance with procedures 4 Recorded appropriate information on the RWP prior to entry 4 Entered only areas designated on the RWP 3 Followed special instructions (i.e., material to be taken into the radiological areas, etc.) 9 Practiced ALARA (time, distance, shielding) 18 Completed task as per RWP 9 Stated appropriate actions to take when a radiation monitor alarmed 5 Verified survey instrument in operation 5 Surveyed for contamination properly (probe speed/distance, return probe to position) 12 Resurveyed area when count rate taken 4 Stated proper actions to take if survey instrument should alarm 5 Recorded and totaled exposure on RWP upon exit 4 Unidentified significant mistake: Action involves violating instructions that lead to spread of contamination outside designated boundaries, or of unnecessary exposure to worker, jeopardizing personnel safety or creating a radiological hazard (eat, drink, chew, etc., in RCA). AUTOMATIC FAILURE: DEDUCT 21 POINTS COMMENTS: 100 Possible Score % Actual Score DOE-HDBK-1122-99 A-12 2. Discussion Questions: RW I personnel enter radiological buffer areas and radiation areas, but do not engage in activities requiring protective clothing. The following questions could be part of the discussion: a. How many millirems of exposure do you normally receive on your job in a year? b. What is your annual dose limit; administrative dose guideline; when was the last time it was reported? c. What types of radiation hazard exist at your facility? d. What types of radiation are you exposed to in the course of your work? How do you protect yourself from these hazards? e. How do you practice ALARA in your work? f. What is the sound of continuous air monitor alarms for your workplace? g. How are employees at your work area warned of radiation hazards? h. What obligations do you have to maintain exposure to radiation ALARA? i. What is dosimetry and what precautions should be taken when handling dosimeters? j. What radiological training are you required to take? How does your training relate to your job? 3. Written Exam: RW I training has recommended objectives in for the core course plus any site specific objectives added for the site. The following approaches to retesting are being successfully applied throughout the DOE complex: a. Conducting written retests twice each year on a sample of the trainees taught during the previous 6 months.

b. Retesting a sample of trainees 4 to 6 months following their training. c. Retesting all objectives for a sample of trainees. d. Retesting only those objectives that were found to be difficult for trainees to learn, remember, or apply in the previous 6 months. Whichever pattern a site selects, the results should provide the data needed to meet the determined purposes of the evaluations. D. Radiological Worker II (RWII) 1. In-Field Observation: Table 4 shows a sample chart that illustrates the in-field activities and weighing system that could be used to observe RW II trainees. The numbers shown are based on conditions and priorities at a DOE site. Other sites may determine a different weighing system. Sites could choose to develop scenarios or drills in which to conduct in-field observations, or simply observe workers in their normal working environments. DOE-HDBK-1122-99 A-13 TABLE 4 Sample RW II In-Field Observations Chart ITEM EVALUATED POINT VALUE SCORE Stated purpose of entry 3 Stated radiation levels and contamination levels (with units) 5 Selected protective clothing (PC's) and dosimetry in accordance with RWP 5 Donned PCs properly 5 Dosimetry worn in accordance with procedures 3 Recorded appropriate information on the RWP prior to entry 5 Entered only areas designated on the RWP 6 Followed special instructions listed on the RWP 8 Practiced ALARA (time, distance, shielding) 10 Completed tasks as per RWP 6 Removed PC's to minimize spread of contamination 10 Placed items in proper receptacles upon doffing 5 Verified survey instrument in operation 6 Surveyed for contamination properly 10 Stated appropriate actions for response to continuous air monitor alarm 8 Recorded exposure on RWP upon exit 5 Unidentified significant mistake: Action involving violating instructions that lead to spread of contamination outside designated boundaries, or of unnecessary exposure to worker, jeopardizing personnel safety or creating a radiological hazard (eat, drink, chew, etc., in RCA). AUTOMATIC FAILURE: DEDUCT 21 POINTS COMMENTS: 100 Possible Score % Actual Score DOE-HDBK-1122-99 A-14 2. Discussion Questions: RW II personnel enter all radiological areas and engage in activities requiring protective clothing. The following questions could be part of the discussion: a. How many millirems of exposure do you normally receive on your job in a year? b. What is your annual dose limit; administrative dose guideline; when was the last time it was reported? c. What types of radiation hazard exist at your facility? d. What types of radiation are you exposed to in the course of your work? How do you protect yourself from these hazards? e. How do you practice ALARA in your work? f. What is the sound of continuous air monitor alarms for your workplace? g. How are employees at your work area warned of radiation hazards? h. What obligations do you have to maintain exposure to radiation ALARA? i. What is dosimetry? What precautions should be taken when handling dosimeters? j. What practices do you take to assure you do not spread contamination while donning or doffing PC's? k. What radiological training are you required to take? How does your training relate to your job? 3. Written Exam: RW II training has recommended objectives for the core course plus any site specific objectives added for the site. Several approaches to retesting are being successfully applied throughout the DOE complex, as described for the RW I course.

Whichever pattern a site selects, the results should provide the data needed to meet the determined purposes of the evaluations. E. Radiological Control Technician (RCT) Training: 1. In-Field Observation: RCT training consists of several phases. In Phase I, core lessons teach academic theory. Different sites have organized and emphasized the information in different ways to fit the needs and practices at the sites. In Phase II, practical skills are taught based on a site specific job assessment. One difficulty in performing on-the-job observations is correlating the schedule of the observer with the schedule of the trainee. One approach to resolving this problem is to associate specific skills or tasks learned in the training with work evolutions that normally occur at the site. Table 5 lists the tasks that will commonly appear in Phase II training in the left-hand column. The top row lists work evolutions where these tasks might be observed. A matrix could be prepared by marking an "X" in a box to show that a particular task can be observed in a particular evolution. The matrix could then be used to correlate with the procedures governing that evolution or task. An alternative is to prepare a checklist to use during the observation. Table 6 shows a sample of such a checklist. DOE-HDBK-1122-99 A-15 TABLE 5 RCT Performance Tasks Cross-Referenced to Work Evolutions WORK EVOLUTIONS PERFORMED IN THE FIELD PERFORMANCE TASKS Air Sampling Contamination and Radiation Survey Personnel and Material Release Survey Use of Counter-Scaler Equipment Personnel Decontami-nation Equipment Decontami- nation Access and Radioactive Source Control AIR SAMPLING TASKS GROUP Draw and field count a grab sample Calculate the activity of a grab air sample Calculate a continuous air monitor (CAM) unit alarm setpoint Perform a background check of a CAM Perform CAM source/ efficiency check Conduct an air sample media exchange CONTAMINATION MONITORING TASKS GROUP Perform preoperational checks of contamination monitoring instrumentation Perform a removable contamination survey Perform a gross large area smear contamination survey Perform a direct contamination survey Document contamination survey results DOE-HDBK-1122-99 WORK EVOLUTIONS PERFORMED IN THE FIELD PERFORMANCE TASKS Air Sampling Contamination and Radiation Survey Personnel and Material Release Survey Use of Counter-Scaler Equipment Personnel Decontami-nation Equipment Decontami- nation Access and Radioactive Source Control A-16 RADIATION MONITORING SKILLS GROUP Perform preoperational checks of radiation monitoring instrumentation Perform a general area dose rate survey Perform a contact dose rate survey Document radiation survey results COUNTER-SCALER EQUIPMENT TASKS GROUP Perform counter-scaler preoperational checks Field count a sample with the counter-scaler PROTECTIVE EQUIPMENT TASKS GROUP Don anticontamination clothing Remove anticontamination clothing HAZARDS POSTING TASKS GROUP Post an area to reflect hazards Repost an area Document posting changes/updates DOE-HDBK-1122-99 WORK EVOLUTIONS PERFORMED IN THE FIELD PERFORMANCE TASKS Air Sampling Contamination and Radiation Survey Personnel and Material Release Survey Use of Counter-Scaler Equipment Personnel Decontami-nation Equipment Decontami- nation Access and Radioactive Source Control A-17 PERSONNEL DECONTAMINATION TASKS GROUP Perform skin/hair decontamination

Perform personal effects decontamination Perform nasal smears RADIOACTIVE SOURCES TASKS GROUP Perform an inventory of radioactive sources Leak check radioactive sources Document radioactive source inventory/leak check DOCUMENTATION TASKS GROUP Complete field logbook entry Complete a request for procedure change Complete a Radiological Problem Report form DOE-HDBK-1122-99 A-18 TABLE 6 Sample RCT Field Observation Guide RCT FIELD OBSERVATION GUIDE #123 EVOLUTION: Personnel Release Survey Instructions to Field Observer Brief the subject RCT prior to commencing the field observation to relieve concerns about being observed. Participant must perform the task without help. A debriefing after the completion of the evolution may be appropriate in order to reinforce good habits and to correct weaknesses. PERFORM PERSONNEL RELEASE SURVEYS Perform a self survey or a survey of an individual for personnel release using portable contamination monitoring instrumentation. Make comments in the section at the end of this document. Use the reference number associated with each of the skills listed below. 1 Inadequate 3 Adequate 5 Outstanding 1. Was the type or level of survey performed as called out by the Radiation Work Permit or other procedure? 2. Was it verified by the RCT that daily preoperational checks on the instrument(s) had been completed? 3. Was a preuse function check of the instrument performed? 4. Did the RCT determine the background radiation level prior to commencing the personnel release survey? 5. Did the RCT hold the instrument probe at the proper distance? 6. Did the RCT scan for contamination at the proper speed? 7. While surveying, did the RCT monitor for an audible increase in count rate above background? *** IF Contamination was discovered *** - - - 8. Was Radiological Control supervision notified as required? 9. Was a thorough whole-body survey of the individual performed? 10. Was the event documented in accordance with prescribed procedures? DOE-HDBK-1122-99 PERFORM PERSONNEL RELEASE SURVEYS Perform a self survey or a survey of an individual for personnel release using portable contamination monitoring instrumentation. Make comments in the section at the end of this document. Use the reference number associated with each of the skills listed below. 1 Inadequate 3 Adequate 5 Outstanding A-19 11. If the individual was confirmed to be contaminated, were the recommended steps taken to preclude the spread of contamination and to proceed to a decontamination facility? Describe the actions taken in the comments section at the end of this document. NOTE: If personnel decontamination will be performed, see RCT Field Observation Guide #XYZ. COMMENTS (List by reference number) DOE-HDBK-1122-99 A-20 2. Discussion Questions: RCT's support radiological work and engage in all activities in radiological areas. The following questions could be part of the discussion: a. How many millirems of exposure do you normally receive on your job in a year? b. What is your annual dose limit? Administrative dose guideline; when was the last time it was reported? c. What types of radiation hazard exist at your facility? d. What types of radiation are you exposed to in the course of your work? How do you protect yourself from these hazards? e. How do you practice ALARA in your work? f. What is the sound of continuous air monitor alarms for your workplace? g. How are employees at your work area warned of radiation hazards? h. What are the biological symptoms you might display with increasing exposures to

radiation? i. What obligations do you have to maintain exposure to radiation ALARA? j. What is dosimetry? What precautions should be taken when handling dosimeters? k. What practices do you take to assure you do not spread contamination while donning or doffing PC's? l. What radiological training are you required to take? How does your training relate to your job? 3. Written Exam: Each site should determine critical objectives to be retested for each lesson or class. Consideration should be given to the likelihood of a trainee having the opportunity to apply the skills from the training in an actual work environment. If the trainee is in training for an extended period of time, 2 to 4 months may pass without having had the opportunity to use the skills in actual work. DOE-HDBK-1122-99 A-21 IV. REFERENCES The following materials provide a wide range of detailed data regarding principles and practices in evaluating training and will be useful for those desiring a more indepth understanding of the evaluation process. Dixon, Nancy M., "Evaluations: A Tool for HRD Improvement," University Associates, Inc. in Association with the American Society for Training and Development, San Diego, California, 1990. "Guide to Good Practices: Evaluation Instrument Examples," DOE-STD-1006-92, The Office of Scientific and Technical Information, U.S. Department of Energy, Washington, D.C., July 1992. "Guidelines For Evaluation of Nuclear Facility Training Programs," DOE-STD-1070-94 (Review Copy), U.S. Department of Energy, Washington, D.C., April 19, 1994. Kirkpatrick, Donald L., "Evaluating Training Programs," American Society for Training and Development, Alexandria, Virginia, 1975. Kirkpatrick, Donald L., "More Evaluating Training Programs," American Society for Training and Development, Alexandria, Virginia, 1987. Newby, A.C., "Evaluation Training Handbook," Pfeiffer & Company, San Diego, California, 1992. Phillips, Jack J., "Handbook of Training Evaluation and Measurement Methods," Gulf Publishing Company, Houston, Texas, 1983. Robinson, Dana Gaines and James C., "Training For Impact," Jossey-Bass, Inc., San Francisco, California, 1989. DOE-HDBK-1122-99 A-22 This page intentionally left blank. Cover