DOE-HDBK-1214-2014, Conduct of Operations Assessment Field Handbook

NOT MEASUREMENT SENSITIVE DOE-HDBK-1214-2014 July 2014 CONDUCT OF OPERATIONS ASSESSMENT FIELD HANDBOOK U.S. Department of Energy Washington, D.C. 20585                                                                                                                     DOE‐HDBK‐1214‐2014 TABLE OF CONTENTS PART I PERFORMANCE-BASED ASSESSMENT OVERVIEW 1 A. Introduction 1 B. Field Assessment and Integrated Safety Management 3 C. General Practices 5 1. Safety ................................................................................................................................... 5 2. Interactions During Observations .................................................................................... 6 PART II OBSERVING/ASSESSING OPERATIONS 10 Generic Items Common to All Areas 10 A. Observing/Assessing Organization and Administration 11 B. Observing/Assessing Shift Routines and Operating Practices 15 C. Observing/Assessing Control Area Activities 20 D. Observing/Assessing Communications 23 E. Observing/Assessing On-Shift Training 26 F. Observing/Assessing Investigation of Abnormal Events, Conditions, and Trends 29 G. Observing/Assessing Notifications 33 H. Observing/Assessing Control of Equipment and System Status 35 I. Observing/Assessing Lockouts and Tagouts (LOTO) 40 J. Observing/Assessing Independent Verification 45 K. Observing/Assessing Logkeeping 50 L. Observing/Assessing Turnover and Assumption of Responsibilities 53 M. Observing/Assessing Control of Interrelated Processes 56 N. Observing/Assessing Required Reading 61 O. Observing/Assessing Timely Instructions/Orders 64 P. Observing/Assessing Technical Procedures 67 Q. Observing/Assessing Operator Aids 74 R. Observing/Assessing Component Labeling 78 S. Observing/Assessing Radiological Control 81 T. Observing/Assessing Training 85 U. Observing/Assessing Drills 89 i                 DOE‐HDBK‐1214‐2014 FOREWORD This Department of Energy (DOE) Handbook is approved for use by all DOE elements and their contractors. This handbook was developed by the DOE Facility Representative (FR) community from the original Operations Assessment Field Handbook, which was developed by the Office of Environmental Management’s Conduct of Operations group in 1994. Conduct of Operations is one of the safety management programs recognized in the Nuclear Safety Rule (10 C.F.R. 830, Nuclear Safety Management, Subpart B, Safety Basis Requirements). DOE Order (O) 422.1, Conduct of Operations, (required for Hazard Category 1, 2, and 3 nuclear facilities, and at the direction of line management elsewhere) requires that FRs be assigned to oversee conduct of operations programs and invokes DOE Standard (STD) 1063, Facility Representatives. DOE O 232.2, Occurrence Reporting and Processing of Operations Information, assigns Facility Representatives responsibilities for the occurrence reporting system. This Handbook is designed to assist assessors in performing routine Conduct of Operations field observations (which include walkthroughs and tours), as well as to provide basic guidance in preparing for a more formal assessment. The handbook is organized into two major parts:  Part I, Performance-Based Operations Assessment Overview, covers general information, definitions, and items applicable to all areas of operations assessment.

 Part II, Observing/Assessing Conduct of Operations Areas, has guidance tailored to specific Conduct of Operations topical areas aligned with the organization of DOE O 422.1, Conduct of Operations, plus three topical areas related to observing/assessing radiological control, training, and drills. DOE O 252.1A, Technical Standards Program, dated February 23, 2011, states that DOE handbooks provide “a compilation of good practices, lessons learned, or reference information that serve as resources on specific topics.” This handbook does not contain requirements statements and cannot be made mandatory via DOE regulatory provisions or contracts. The application of the techniques described in this handbook is not mandatory, and the methods and techniques may be used at the discretion of contractor line management. Beneficial comments (recommendations, additions, deletions, etc.) and any pertinent data that may be of use in improving this document should be sent to: James O’Brien HS-30/GTN U.S. Department of Energy Washington, D. C. 20585 Phone (301) 903-1408 ii   DOE‐HDBK‐1214‐2014 PART I PERFORMANCE-BASED OPERATIONS ASSESSMENT OVERVIEW     DOE‐HDBK‐1214‐2014 PART I PERFORMANCE-BASED ASSESSMENT OVERVIEW A. Introduction This handbook is designed to assist in performing routine Conduct of Operations field observations (which include walkthroughs and tours), as well as provide basic guidance in preparing for a more formal assessment. DOE Order 422.1, Conduct of Operations, defines the key terms:  A Conduct of Operations Program is the formal documentation, practices, and actions implementing disciplined and structured operations that support mission success and ensure worker, public, and environmental protection. The program goal is to minimize the likelihood and consequences of human fallibility or technical and organizational system failures.  The term “operations” encompasses the work activities of any facility or organization, from building infrastructure, to print shops and computer centers, to scientific research, and to nuclear facilities. While many hazards can be dealt with through engineered solutions, people still have to perform operations, and they can and do make mistakes. Good Conduct of Operations helps to ensure that management systems are designed to anticipate and mitigate the consequences of human fallibility or potential latent conditions and provide a vital barrier to prevent injury, environmental insult or asset damage, and to promote mission success. Part I, Performance-Based Operations Assessment Overview, covers general information, definitions, and items applicable to all areas of operations assessment. Part II, Observing/Assessing Conduct of Operations Areas, has guidance tailored to specific Conduct of Operations topical areas aligned with the organization of DOE O 422.1, Conduct of Operations, plus three topical areas frequently observed as part of Conduct of Operations field assessments: radiological controls, training, and drills. Each of the topical areas in Part II is organized as follows: 1. A brief Discussion of the purpose of the particular topical area, and a subsection covering those portions of other topical areas that may be combined with the assessment. 2. A Document Review section listing the types of documents that should be reviewed for the topical area, followed by an example.

3. A Field Observations section listing situations or operations related to the topical area, followed by examples of what to expect and which aspects of the topical areas are most likely to be observed. 4. An Interview section providing suggested interview topics and questions, lists of appropriate personnel titles or positions related to the topical area, and suggested methods to validate leads discovered during the interviews. 1     DOE‐HDBK‐1214‐2014 Terminology To assess something means to evaluate an action, operation or item and make a determination whether it is acceptable. Field observations by FRs or other assessors are the most frequent assessment method for Conduct of Operations. Personnel interviews and document reviews are also common, and all three provide information for assessments. An assessment is used to determine whether something meets specified requirements. In DOE/NNSA, these requirements are primarily listed in documents such as DOE Directives, the Code of Federal Regulations (CFR), and other regulatory and standards documents. In this handbook, this type of assessment will be referred to as a formal assessment. Formal assessments, particularly readiness reviews, involve selecting/developing Criteria and Review Approach Documents (CRADS) that provide assessment criteria (based on requirements) and guidelines for conducting the assessment. A good source of CRADS is the Contractor Requirements Document section of DOE Orders and their implementation documents; for example, Attachment 2 to Appendix A of the Conduct of Operations Order provides program requirements that can be used as CRADS. Deficiencies (including safety issues) are items or actions that do not conform to a stated requirement or performance standard. Weaknesses are conditions or events that, if left unaddressed, may lead to non-compliance with requirements. Findings are significant deficiencies or safety issues that warrant a high level of attention on the part of DOE/NNSA management. If the requirements (acceptance criteria) are met but the assessor has information or suggestions for improvement, this is referred to as an observation (also opportunity for improvement), which may or may not require management attention. Facility Manager will be the position used throughout this handbook to denote the person with responsibility (administrative and operational) for an entire facility, building, or area. Other position titles that mean the same thing are Building Manager or Area Manager. Program (sometimes Process) supervisors and managers have similar responsibilities for all matters associated with their process, including coordinating with affected facility managers. Imminent Danger means conditions or practices in the workplace where a danger exists which could reasonably be expected to cause death or serious physical harm either immediately or before the abatement of such danger, through normal procedures, would otherwise be required. In an imminent danger situation, warn others of the danger and ensure that they stop work (following site-specific procedures). General Guidance This handbook covers the requirements from DOE O 422.1, and incorporates most of the detailed attributes in Appendix A of Attachment 2 (Conduct of Operations Matrix) of the Order. Before conducting assessments per this Handbook, assessors should consult their site/facility’s Matrix or other implementing documents for applicable attributes.

Field observations of Conduct of Operations topics usually also involve observation of radiological controls practices and an evaluation of the training of contractor personnel. This 2     DOE‐HDBK‐1214‐2014 Handbook includes guidance on these topics in Part II, as well as a section on emergency response drills. The general practices discussed in Section C of this overview should be followed whenever applicable. B. Field Assessment and Integrated Safety Management 1. Requirements Basis (DOE Directives and Standards) DOE Policy (P) 226.1B, Department of Energy Oversight Policy, establishes a Department- wide oversight process, and DOE O 226.1B, Implementation of Department of Energy Oversight Policy, establishes departmental oversight program requirements. DOE Guide (G) 226.1-2, Federal Line Management Oversight of Department of Energy Nuclear Facilities provides detailed guidance and examples for oversight programs and functions. Conduct of Operations is one of the safety management programs recognized in the Nuclear Safety Rule (10 CFR 830, Nuclear Safety Management). DOE O 422.1 (required for Hazard Category 1, 2, and 3 nuclear facilities, and at the direction of line management elsewhere) requires that FRs be assigned to oversee conduct of operations programs and invokes DOE-STD-1063, Facility Representatives. DOE O 232.2, Occurrence Reporting and Processing of Operations Information, assigns FRs responsibilities for the occurrence reporting system. 2. Integrated Safety Management (ISM) and Field Observation ISM is the integration of safety awareness and good practices into all aspects of work conducted at DOE. DOE work execution should protect workers and other people and cause no harm to the environment. Safety is an integral part of each job, not a stand-alone program. FRs and other field assessors play a role in DOE oversight of all of ISM’s 7 Guiding Principles and 5 Core Functions. The ISM Guiding Principles describe the environment or context for work activities, and most ISM principles apply to all the Core Functions. a. ISM Guiding Principles Each Guiding Principle has Attributes (denoted by ) that are expected, observable behaviors and organizational characteristics intended for use as a tool to describe expectations of organizations and employees. However, they are not requirements nor are they intended for use as assessment criteria. The attributes listed below are some of the attributes that may be observed daily; over time, observations of these attributes will indicate how well facilities and the organization are implementing and maintaining ISM principles (i.e., situational awareness). Note: In ISM the term “safety” is used synonymously with environment, safety, and health (ES&H) to encompass protection of the public, the workers, and the environment. 3     DOE‐HDBK‐1214‐2014 1. Line Management Responsibility for Safety –  Line managers periodically take steps to reinforce safety, including personal visits and walkthroughs to verify that their expectations are being met.  Line managers practice visible leadership in the field by placing “eyes on the problem,” coaching, mentoring, and reinforcing standards and positive behaviors. Deviations from expectations are corrected promptly and, when appropriate, analyzed to understand why the behaviors occurred. 2. Clear Roles and Responsibilities  All personnel understand the importance of adherence to standards.

 All personnel know their authority and responsibilities for safety.  Personnel at all levels of the organization are held accountable for shortfalls in meeting standards and expectations related to fulfilling safety responsibilities. 3. Competence Commensurate with Responsibilities  Individuals have in-depth understanding of safety and technical aspects of their jobs.  The organization values and practices continuous learning, and requires employees to participate in recurrent and relevant training and encourages educational experiences to improve knowledge, skills, and abilities. 4. Balanced Priorities  Protecting the workers, the public and the environment are a priority whenever activities are planned or performed.  Managers recognize that aggressive mission and production goals can appear to send mixed signals on the importance of safety. Managers are sensitive to detect and avoid these misunderstandings, or to deal with them effectively if they arise. 5. Identification of Safety Standards and Requirements  Implementing plans, procedures and protocols are in place to translate requirements into action by the implementing organization.  Safety requirements are clearly specified and communicated to individuals performing operational tasks.  Compliance with applicable safety and technical requirements is expected and verified. 6. Hazard Controls Tailored to Work Being Performed.  Administrative and engineering hazard controls are tailored to the work and associated hazards .  Workers understand hazards and controls before beginning work activities.  Equipment is consistently maintained so that it meets design requirements. 7. Operations Authorization.  Readiness at the facility level is verified before hazardous operations commence. Pre-operational reviews confirm that controls are in place for known hazards. 4     DOE‐HDBK‐1214‐2014  Facility operations personnel maintain awareness of all facility activities to ensure compliance with the established safety envelope.  The work authorization process verifies that adequate preparations have been completed so work can be performed safely. These preparations include verifying that work methods and requirements are understood; verifying that work conditions will be as expected and not introduce unexpected hazards; and verifying that necessary controls are implemented. b. ISM Core Functions The core safety management functions for ISM provide the structure for any work activity that could potentially affect the workers, the public and the environment: 1. Define the Work Scope – Assessors review the tasks, resources, and expectations to identify mismatches and gaps; 2. Analyze the Hazards – Assessors check that hazard analyses are thorough and consider all risks; 3. Develop and Implement Hazard Controls – Assessors help to verify that controls, particularly Technical Safety Requirements (TSRs), are implemented; 4. Perform Work within Controls – Assessors help to ensure that work is performed safely; and 5. Provide Feedback and Continuous Improvement – Field assessments and reviews provide valuable feedback on the adequacy of controls and assist in trending and tracking corrective actions. C. General Practices 1. Safety Assessors are responsible for their own safety, and also for oversight of the overall contractor safety process. a. Personal Safety of assessors

Be aware of the hazards associated with entry into various areas of the facilities and take appropriate precautions, including adhering to the operating contractor’s rules for entry and work in these areas. Activities such as climbing or walking on elevated surfaces, where one could encounter unforeseen hazards, should not be attempted alone. Personnel touring facilities are subject to occupational hazards, the effects of which could be exacerbated if an injury occurred in a remote, seldom-visited area, particularly on back shifts. For that reason, assessors need to be particularly safety conscious during the back shift activities, and should always notify the operations center of their itinerary or accompany an operator on the operator’s rounds. Entry into radiation areas and areas requiring respiratory protection may be necessary. Assessors should plan their activities to minimize exposure, while providing effective oversight of contractor work activities. 5     DOE‐HDBK‐1214‐2014 b. General Safety Oversight Individuals responsible for operations have direct responsibility for the safety of those operations. A viable worker safety system requires commitment from managers and meaningful involvement of workers. Meaningful involvement requires each and every employee in an organization to be held accountable for his or her safety performance. 1. Managers should be held accountable for safety performance and they should be visibly present, addressing safety issues. Their commitment to worker safety may be gauged by observing how well management:  Trains workers how to work safely;  Ensures strict compliance with all applicable requirements and regulations;  Ensures strict compliance with precautions, limitations, requirements and constraints of work control documents;  Implements radiological protection policy and practices based on the precept that radiological exposures for workers are to be kept ALARA;  Ensures accountability for safety performance;  Communicates risk to the worker;  Solicits worker input regarding workplace hazards and selection of appropriate controls;  Encourages worker responsibility to demonstrate a strong, questioning attitude regarding work and the hazards associated with the work; and  Encourages workers to exercise their Stop Work authority when they think it is necessary. 2. At the worker level, work plans, operating procedures, and maintenance procedures are often used to implement safety controls at the task level. In order to evaluate these safety controls, determine how well:  Hands-on training, safety awareness training, and the identification of PPE familiarizes a worker with job duties, hazards and controls;  Pre-job briefings and walk-downs ensure workers are aware of hazards, and are knowledgeable on the proper use of prescribed controls;  Worker input is solicited concerning hazard controls; and  Changes that occur or are discovered after briefings are immediately analyzed and procedures updated accordingly. 2. Interactions During Observations During questioning (including interviews), remember that facility personnel do not have to answer all questions from memory, and even those subjects that are required to be memorized do not usually have to be answered “word for word” in accordance with the reference material. If they indicate that they need to consult a reference in order to answer a question, either allow them to obtain the reference or defer the question. Ask additional questions in order to determine the depth of their knowledge, but do not hold them accountable for their answers under these conditions. If it appears they should know the

6     DOE‐HDBK‐1214‐2014 subject from memory, follow up by consulting their qualification standard or asking the appropriate supervisor or training department person. a. Asking Questions during Observations Asking questions should be accomplished in a manner that will not distract the individuals performing the work from their duties. Whenever possible, ask questions during periods of inactivity or during scheduled breaks (not during lunch). Some general guidelines are:  Do not interrupt someone who is performing a task or is otherwise busy; this includes someone who is thinking about a situation or problem or who is noticeably stressed. In particular, avoid asking questions during shift or operations turnover, when personnel are responding to an alarm or abnormality, and during emergencies. Take notes and ask questions later.  Watch for the opportunity to ask a question, and consider how long the conversation may last and if the interviewee will need access to references (e.g., drawing or procedure).  Ask if this is a good time to talk (assessors may first need to ask the supervisor for permission to talk to a worker).  Do not ask question after question without stopping. Ask a question and then give the person time to think and answer before moving on to another question.  Questions should be related to the task at hand, rather than on some unrelated topic. This keeps distractions to a minimum. NOTE: Once an observation is complete, the assessor should discuss any unclear or questionable actions promptly with the appropriate person. Clarifying facts immediately following an observation (while information is fresh) can often resolve perceived discrepancies or deviations from requirements. b. Scheduled Interviews Scheduled interviews conducted separately from operations are the preferred setting to ask questions about complex or theoretical topics such as:  Fundamental theory questions to verify understanding of safety systems and components;  System/component operation and interrelationships;  Alarm and trip set-points;  Bases for applicable TSR Safety Limits (SL), Limiting Safety System Settings (LSSS), Limiting Conditions for Operations (LCO), Specific Administrative Controls (SAC) and Design Features (DF);  Bases for applicable SL and LCO Actions;  Site/Facility organization and responsibilities;  How their work fits into Integrated Safety Management functions and principles;  Work planning and implementation processes;  Security requirements; 7     DOE‐HDBK‐1214‐2014  Emergency/immediate actions that do not require prior permission to perform; and  Activities or evolutions that could not be observed during field observations, such as: o Electrical safety practices; o Radiological safety practices; o Equipment lockout/tagout (LOTO); o Post-maintenance testing; o Work on elevated surfaces; o Confined space entry; o Fire system impairments; and o Personnel response procedures (e.g., fire, personal injury, severe weather, and spills). 8     DOE‐HDBK‐1214‐2014 PART II OBSERVING/ASSESSING OPERATIONS 9     DOE‐HDBK‐1214‐2014 PART II OBSERVING/ASSESSING OPERATIONS Generic Items Common to All Areas These items apply to the respective areas of all the topical areas that follow. They follow the same organization, and should be considered a part of, each of the detailed paragraphs A through U. 1. Discussion

Review pertinent portions of Part I, Performance-Based Assessment Overview, such as Integrated Safety Management (ISM) and General Practices. Review the approved Conduct of Operations Matrix or other approval documents to determine the applicable Conduct of Operations attributes. Review the applicable DOE Conduct of Operations Standard for the topic (DOE STD 1029 through 1041) 2. Document Reviews Assessors should identify and review important procedures and manuals that pertain to the topic under review prior to beginning field observations and/or interviews. This will bring them up-to- date on how current operations are controlled and performed, thus allowing a more effective field assessment. Of particular importance are the Technical Safety Requirements (TSRs) applicable to the facilities, operations and/or programs reviewed. Some of the document reviews can be conducted in conjunction with field observations. For example, while touring the facility with an operator, review documents such as operating logs, round/tour inspection sheets, safety and radiological control postings, and radiation and safe work permits. 3. Field Observations A field observation may be planned as a stand-alone activity, or conducted during walkthroughs and during tours with operators and shift supervisors. If possible, exercises or drills provide an invaluable opportunity for a field observation. Work schedules (e.g., Plan of the Day or Plan of the Week) should be consulted to determine the most opportune time and place for observing active operations or support activities. Whenever possible, contact the work supervisor to verify times (except for an unannounced observation). Once the observations are complete, compare what was observed to the requirements contained in facility policies and procedures to identify any apparent deviations from requirements. Follow up on these leads using additional observations, interviews and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. When observing an operation where procedures are used, take along a copy of the procedure whenever practical 10     DOE‐HDBK‐1214‐2014 4. Interviews Interviews are normally performed after field observations are complete or during interludes between field observations (e.g., during breaks or periods of inactivity). Portions of these interviews can occur while work is going on, but assessors should remember that asking questions while a person is working may distract them from their normal duties. (Note: The exceptions are cases involving real or potential safety concerns or immediate danger.) Some questions may arise from things seen during field observations, and this is the time to follow up on and validate leads. After completing an interview, evaluate the information to determine if there are any apparent deviations from requirements or best practices. These leads can normally be validated or disproved by using one or more of the following:  Additional walkthroughs and tours of the facility;  Review of pertinent documentation;  Additional observations and interviews with operators;  Interviews with facility managers and operations/program supervisors;  Reviews of operating and program procedures;  Interviews with training department personnel; and  Interviews with engineering and maintenance department personnel. A. Observing/Assessing Organization and Administration 1. Discussion

The goal of assessing operations Organization and Administration (O&A) is to verify that its policies, programs and procedures produce a highly effective operations organization. The assessor will verify that management clearly defines roles and responsibilities, provides sufficient trained and qualified staff and adequate resources, holds personnel accountable for their actions, monitors and assesses operations, and analyzes and controls work hazards. O&A provides the philosophical underpinning of Conduct of Operations. Because it is an all-encompassing topic, O&A should be continuously evaluated to measure how well management has defined and communicated policies, how well working-level personnel understand management’s expectations and execute work, and how management monitors and assesses operations. Assessors should continuously ask themselves three questions:  Do workers know what to do?  Has management provided guidance about what to do?  Are workers properly trained? There are two complementary approaches to assessing O&A. The first is a top-down approach that emphasizes interviews with each level of management to determine their expectations and how they monitor operations performance throughout the organization. The second is a bottom- up approach that emphasizes observing and interviewing operators and first-line supervisors and then moving up the organization to determine if management expectations are being correctly 11     DOE‐HDBK‐1214‐2014 interpreted and met. Assessors should decide whether to use one or a combination of these approaches based on the facility being assessed and their particular assessment experience and style. Even though it is important to thoroughly review documents and conduct interviews for this topical area, it is imperative to go out and verify that floor-level performance matches management’s guidance and expectations. 1.1 Related Conduct of Operations Topical Areas Document reviews, field observations and interviews conducted to assess organization and administration will normally provide leads in other topical areas. Therefore, the assessment of this area can be combined with portions of other areas such as Communications, Investigation of Abnormal Events, Notifications, and Technical Procedures. 1.2 Conduct of Operations for organization and administration includes:  Organizational roles, responsibility, authority and accountability;  Management and worker accountability for the safe performance of work;  Adequate resources to accomplish operations;  Monitoring and self-assessment of operations;  Training and qualification/certification;  Methods for the analysis of hazards and implementation of hazard controls in the work planning and execution process; and  Methods for approving, posting, maintaining, and controlling access to electronic operations documents. 2. Document Reviews Facility policies should be reviewed to become familiar with management’s policies and goals for operations, safety and security, the means to achieve them, and controls instituted for the Conduct of Operations program. Document reviews may include:  Strategic plans (site level);  Policies and procedures that implement DOE/NNSA requirements for operations, safety, environment, and security (facility and activity levels);  Clear assignment of responsibility and authority;  Operations and material resources and staffing plans;  Training and qualification/certification records;

 Policies and procedures related to monitoring and assessing operations performance, including documented management self-assessments;  Pre-job, post-job, hazard analysis, and safety checklists;  Issues are being tracked and trended;  Relationships with other groups and Stop Work authority; and  Performance indicators. 12     DOE‐HDBK‐1214‐2014 When reviewing the organization’s goals, there are three questions the assessor should ask:  Do goals exist?  Are goals realistic, challenging, and measureable?  Do personnel understand the goals and have a clear idea of how to achieve them? While goals may be posted on bulletin boards throughout the facility, this does not indicate that workers understand how management expects them to achieve the goals. Assessors should talk to managers, supervisors, and operators to determine their understanding of the goals and how they are trying to achieve them. 3. Field Observations Observation is an important tool in determining how effectively management has communicated its policies to workers and how well workers meet management expectations. Specific examples of O&A that an assessor might observe include:  Postings and briefings of management policies and goals;  Postings or briefings of how well goals are being met;  Management conducting tours, walkthroughs, or self-assessments;  Personnel are trained and qualified to perform their duties;  Problems (issues) are quickly identified and addressed without fear of retribution;  Safety is integrated into work planning;  Systems designed to minimize the effects of human performance failures;  Pre-job briefings (safety, hazards, and hazard controls);  Post-job (after action) meetings;  Tasks being accomplished without excessive overtime (i.e., sufficient staffing);  Support personnel are available when needed (particularly on back shifts);  Adequate material, tooling, equipment, and safety gear available to support tasks; and  Electronic documents are accessible and controlled. As an example of how to conduct an O&A observation, walk through the facility and identify apparent operations and material deficiencies. Once this has been completed, accompany a supervisor or manager while they conduct a scheduled management tour or walkthrough. During the observation, note what actions the manager takes when issues or deficiencies are identified, specifically:  Did they identify the issues and deficiencies seen on the assessor’s walkthrough?  Are issues and deficiencies documented?  Is work halted, if necessary for safety?  Are issues and deficiencies entered into a tracking system? These observations may lead to an unscheduled interview with the supervisor or manager. Some questions might include: 13     DOE‐HDBK‐1214‐2014  How do you monitor performance? What happens if progress toward goals falls short?  How often do you perform facility tours/walkthroughs?  How do you address issues and deficiencies?  How do you ensure effective corrective actions are identified and implemented? 4. Interviews An important objective of interviews is to determine how well goals are communicated from top management down to the front-line workers. Interviews with managers, supervisors, and workers will give an indication of how well goals are communicated and understood, and how well management monitors facility operations. Interview goals should include determining whether:

 Operations policies exist and are communicated to workers;  Goals are communicated from and to each level of the organization;  Personnel are recognized for notable safety improvement actions or ideas;  Goals are adjusted/modified as needed, based on assessment, observation, feedback and measurement;  Management effectively plans for safety;  Resources are sufficient to accomplish assigned tasks;  Staffing shortfalls (current or anticipated) are addressed under a long-term staffing plan;  Management takes an active role in operations by monitoring operating performance;  Other organizations (e.g., Quality Assurance) observe operations and provide feedback;  Observation/assessment issues are tracked and corrected;  Personnel are held accountable for repeated or willful violations of operating practices/procedures (e.g., counseling, retraining, or disciplinary action); and  Operator, technical, support staff, supervisor and management personnel are properly trained and qualified/certified. Some example interview questions for managers at the facility or activity level include:  What are your operating goals and how do you communicate them to workers? How are these goals achieved and tracked?  How do you monitor operations and safety performance and improvement?  How do you plan for safety?  How does management support job planning? How do you incorporate ISM into work planning? How are workers included in the job planning processes?  How do you analyze hazards and implement hazard controls?  How do you approve electronic documents and ensure they are accessible, current, and secure?  What is your involvement in developing training for your staff and workers? 14     DOE‐HDBK‐1214‐2014  How do your supervisors and operators know what their responsibilities are, and those of their immediate subordinates? How do they know the scope of their authority?  What is your perspective of safety versus production? What are your worker’s impressions of safety versus production?  Does management consider operational and safety performance when conducting supervisor performance reviews, appraisals, or promotions? Some example Interview questions for workers include:  What are the management’s goals?  How often do you see management in the facility?  How do you plan for safety?  What is your perception of safety versus production? B. Observing/Assessing Shift Routines and Operating Practices 1. Discussion The goal of assessing Shift Routines and Operating Practices to verify standards for the professional conduct and level of performance expected of operators and supervisors to ensure facilities and/or processes are operated safety and efficiently. Some of these standards may also be applied to certain maintenance and technical staff personnel. The assessor will verify sound operating practices ensure that shift operators are alert, informed of conditions, demonstrate responsibility for their work, and operate equipment properly. Review pertinent portions of Part I, Performance-Based Assessment Overview, such as training questions and general practices. Portions of these guidelines may be incorporated into the assessment. 1.1 Related Conduct of Operations Topical Areas

Document reviews, field observations and interviews conducted to assess shift routines and operating practices will normally provide leads in other topical areas. Therefore, the assessment of this area can be combined with portions of other areas such as Control Area Activities, Communications, Control of Equipment and System Status, Logkeeping, Turnover and Assumption of Responsibilities, Control of Interrelated Processes, and Technical Procedures. 1.2 Conduct of Operations for shift routines and operating practices includes:  Awareness of the status of systems and equipment;  Prompt response to instrument indications, warnings and alarms;  Prompt notifications of changes, difficulties, abnormalities and emergencies;  Procedure adherence and proper use of records/logs;  Proper authorization to operate equipment; and  Professional and disciplined performance of duties. 15     DOE‐HDBK‐1214‐2014 2. Document Reviews To illustrate how to perform these types of document reviews, consider a review of completed round/tour inspection sheets. Various items on the sheet can serve as a lead to develop interview questions and conduct other observations. For example, look for equipment parameters that are out of specification or are approaching an out of specification condition. If found, ask the operator what actions should be taken if the parameter goes out of specification, or make up a hypothetical situation and ask the same type of question. Look at the round sheets for vague check offs such as “check sump level.” A follow-on question for this may be “What do you look for when you perform this check?” then the operator’s response can be compared with the answers of other operators. This technique of using document review as a source for interview questions and observations can be an extremely effective tool for an assessor. 3. Field Observations Operators, maintenance and other support personnel, and shift supervisors from each shift should be observed, if possible. Preferably the observation of a particular shift position should begin at shift turnover, since vital information about the operating status of equipment/systems and operations and maintenance in progress is passed between shifts at this time. The following situations present good opportunities for a field observation:  Shift turnover;  Pre-shift or pre-job briefing;  Control Room operations;  Process or rate change operations;  Routine equipment operations;  Operator/Utility rounds;  Startup or shutdown of systems/equipment;  Post-maintenance testing;  Maintenance or surveillance activities that require planning and coordination with support personnel, auxiliary operators, control area operators and shift supervisors; and  Exercises or drills. While observing these situations, watch for good practices such as:  During shift turnover conducted between the on-coming and off-going operators and supervisors, pay attention to the information passed between them concerning operations (planned or in progress), status of facility systems and equipment, and any abnormal conditions that exist in the facility. Review any narrative logs, round/tour inspection sheets, and any other documentation used in the turnover process.  During a shift crew briefing, observe the process used and the information exchanged. Compare the information disseminated at the briefing with the information passed between the operators during their turnover. New or pertinent Timely Instructions/Orders and Required Reading should be covered. Note any important information disseminated and ask personnel questions later in order to get an idea of the effectiveness of the briefing.

16     DOE‐HDBK‐1214‐2014  Operators and supervisors keep each other informed of facility status changes, abnormalities, difficulties, potential hazards, and unexpected situations. Look for proper response to instrument readings, indications, and abnormal conditions, such as: o Operators believe their indications (i.e., treat them as accurate) unless proven otherwise; operators check other indicators to confirm unexpected readings. o Operators take prompt action to investigate, document and correct abnormal conditions; operators are alert to deviations from expected trends. o Operators identify inaccurate or malfunctioning instruments and inform their supervisor (also evaluate how operators treat “nuisance” alarms, and determine how long it normally takes to repair malfunctioning instruments). o If an automatic protective action occurs, operators determine the cause and take corrective actions as necessary before resetting the protective device.  Operators and supervisors display professional and disciplined behavior: o Potential distractions such as electronic devices (e.g., radio, TV, music players, or games), personal telephone calls, excessive non-work related talking, and horseplay are prohibited. o Written materials not related to work are prohibited (operators may read/view materials that relate to their duties). o Supervisors practice and maintain proper conduct and discipline.  Operators/utility personnel regularly tour their assigned areas, and the tours are thorough enough to determine equipment status (including support/standby equipment). Personnel exhibit good housekeeping practices, and are familiar with any work planned or in progress.  Round sheets are current and ensure thorough recording of parameters, and include: o Frequency and time of recording instrument readings (and allowable delays). o Operational and/or safety settings/limits, where applicable. o Normal (expected) maximum/minimum operating parameters or conditions. o Alarm/annunciator functionality checks, where applicable. o Comment (written narrative) section. o Logical arrangement of entries (e.g., grouped in the order they are encountered on a standard tour). o Readings are delayed beyond the allowable range; the actual time and an explanation for the delay are recorded.  Operators use round sheets properly: o Note (circle or highlight) data readings or conditions outside the normal or maximum/minimum range promptly, and report them to their supervisor. o Take proper corrective action(s) to restore proper functioning. o Make narrative records of important events, abnormal conditions and corrective actions. o Does not ignore or accept existing deficiencies as “normal” (i.e., normalization of deviance). 17     DOE‐HDBK‐1214‐2014  Supervisors review round sheets for trends, abnormalities, and proper data and narrative entries during each shift and take appropriate actions to resolve issues with equipment or the associated rounds procedures. Supervisors periodically monitor operator rounds for proper execution and make any necessary adjustments.  Personnel are aware of what equipment they are authorized to operate: o Personnel operating equipment have appropriate qualification and certification. o Trainees are properly supervised. o Management designates routine operations that do not require permission for performance, and supervisors approve non-routine operation of facility or process controls.

o During emergencies, personnel may take immediate actions for worker, public, and environmental protection without permission, and inform supervisors promptly.  Operators, maintenance and support personnel comply with safety programs (e.g. industrial, chemical, explosive, or radiological): o Trained and qualified for, and aware of, expected hazards and take action to minimize them. Utilize protection practices to maintain personnel exposure As Low as Reasonably Achievable (ALARA) and within facility controls for radiation, chemicals, electromagnetic fields, toxic materials, and other personnel hazards. o Comply with all posted personnel protection requirements and precautions, and use appropriate controls such as lockout/tagout, work permits, radiological work permits, and confined space permits. o Report and take corrective action for hazard protection deficiencies. o Notify protection personnel prior to activities that affect hazard protection status (e.g., Industrial Hygiene, Industrial Safety, Electrical Safety, Fire Protection, or Radiological Protection). o Material, tooling, equipment, safety gear (e.g., Personal Protection Equipment - PPE), and facilities needed for safe operations are available. o Management/supervisors periodically review exposure trends of operators to detect and correct adverse factors that contribute to personnel exposures.  Startup or shutdown of systems/components is performed correctly: o Operators or maintenance personnel brief or discuss the job, and get proper and timely authorization to commence work. o Use of and adherence to procedures for system startup or shutdown. o Operator adherence to and use of posted personnel protection requirements, proper use of monitoring instruments, and adherence to radiation and safe work permits. o Operator response to indications as valves are operated and system components are energized or de-energized. o Response to actuated/tripped protective devices (normally, devices are reset/replaced only after determining the cause of the actuation or trip and ensuring that the condition that caused the trip no longer exists). o Proper communications and notifications of changes in system status by all involved. o Knowledge, skill and level of qualification of operations personnel. 18     DOE‐HDBK‐1214‐2014  At the end of the shift, observe and evaluate the turnover process. If there is no following shift, observe how personnel secure the work and work areas. 4. Interviews The following personnel may be interviewed to assess shift routines and operating practices:  Operators, utility and maintenance personnel;  Shift/operations supervisors, operations/program managers, or foremen;  Facility managers;  Radiological Control Technicians, Instrument and Control Technicians, and Industrial Safety/Hygienists;  Nuclear, explosive and industrial safety specialists; and  Training department personnel. For example, while observing a roving operator after shift turnover, the following types of questions could be used to assess level of knowledge about shift routines:  What information should be passed on during shift turnover? Do you use a checklist to aid the process? What guidance have you received from the shift supervisor or facility manager concerning how to conduct shift turnover? Do you feel that the process used at turnover supports your needs? What changes, if any, should be made to improve the process?

 What are your duties and responsibilities during the shift?  What areas of the facility are you responsible for? What things do you normally look for during the facility tour? How often do you conduct tours? How do you know what parameters are acceptable (e.g., TSR controls and minimum/maximum operating limits)? Describe the process that you use to document, report, and correct deficiencies discovered during tours (including required immediate actions, if applicable).  Are there any areas in the facility that have special safety requirements? How do you know where they are and what safety equipment is required?  Are there any Controlled Areas in the facility? How do you know what personnel protective equipment (PPE) (including dosimetry) is required to enter the area? Have you been trained in the use of PPE? Do you receive continuing/refresher/requalification training in the use of PPE?  How do you normally communicate with the control area operators and shift supervisor? How do they communicate information to you? Do you ever use radios to communicate and, if so, are there any areas where the use of radios is prohibited?  Are there any areas in the facility that cannot be accessed or used? If so, where are they and why can they not be accessed or used (if the reason is poor maintenance or housekeeping, follow up to determine why this condition is considered acceptable)?  What are your responsibilities regarding the use of procedures? Are there different levels of procedures? What equipment are you authorized to operate? Are you required to have procedures with you when operating equipment and systems, or do you operate from memory? 19     DOE‐HDBK‐1214‐2014 C. Observing/Assessing Control Area Activities 1. Discussion The goal of assessing Control Area Activities is to verify that control areas properly function as the coordination point for important facility or program activities and operations. Operations practices in the control area should be formal and conducted in a professional, business-like manner. This area addresses important elements necessary to support safe and efficient facility operation. Control areas normally come in two basic configurations: a control room and an “at-the- controls” area. A Control Room is the physical area (e.g., room or booth) where the facility/process or portions of the facility/process operations are monitored and controlled; it may range in size from a desk or computer terminal to a room full of instrumentation and control panels. At-the-controls area is a designated area containing monitoring panels and/or operating mechanisms where special access and controls are necessary; examples include the space in front of and to the immediate sides of an operating control panel, or the area in the immediate vicinity of facility, process, workstation, or experiment controls. Review pertinent portions of Part I, Performance-Based Assessment Overview, such as training questions and general practices. Portions of these guidelines may be incorporated into the assessment. 1.1 Related Conduct of Operations Topical Areas Document reviews, field observations and interviews conducted to assess control area activities will normally provide leads in other topical areas. Therefore, the assessment of this area can be combined with portions of other areas such as Shift Routines and Operating Practices, Communications, On-shift Training, and Control of Interrelated Processes.

1.2 Conduct of Operations for a control or at-the controls areas includes:  Access control;  Formality and discipline;  Surveillance of control panels and timely response to indications and alarms;  Limiting operator concurrent or ancillary duties; and  Proper authorization to operate equipment. 2. Document Reviews Examples of documents that should be reviewed when assessing control area activities include:  Program and operating procedures;  Authorizations to operate control area equipment;  TSRs; and  Control area access lists. 20       DOE‐HDBK‐1214‐2014 3. Field Observations Assessors should ensure they are properly authorized to enter the control or at-the-controls area, and that it will not exceed the personnel limit for the area. Access requirements include permission to enter, when personnel are allowed to enter, and under what conditions personnel may enter. Entry into the area is defined as physically crossing the boundary or passing objects over the boundary, and communication into the area is not allowed except to request permission to enter. As a reminder to personnel, control areas that are open (i.e., no walls or partitions) are usually marked (e.g., white paint, white tape, rope, chains, or other similar methods) and a sign indicating “Control Room” or “At-the-Controls- Area” indicates boundaries. There are certain general attributes to watch for at all times while conducting control area observations:  Access control: Areas are clearly identified and all personnel understand the boundaries of the area; access is limited to personnel who have legitimate business in the area; entry into the area is formal (e.g., requesting and receiving permission to enter).  Formality of operations: All personnel display professional, attentive, and disciplined behavior. o Activities are limited to essential, authorized operations; o Communications are clear and concise; o Work-related distractions are minimized; o Non-work distractions are prohibited (e.g., radio, music, personal phone calls, games or horseplay). The following situations are good candidates for conducting observations:  Beginning of operations and shift turnovers;  Startup and shutdown of systems and equipment;  Process or rate change operations; and  Facility maintenance and special operations controlled from the control area. During shift turnovers, observe the operators and supervisors from each shift, since vital information about the operating status of equipment and systems and any planned or in-progress surveillances, testing or maintenance should be passed between shifts during turnover. Observe:  How personnel prepare for shift turnover (e.g., reviewing panels and logs).  How and what information is passed between the operators and supervisors. Particular attention should be paid to operations planned or in progress (e.g., maintenance, surveillances and testing), the status of facility systems and equipment (including explanations of current alarm or warning conditions), and any abnormal conditions or trends.  Compare the information disseminated at the shift crew briefing with information passed between the operators during their turnover. Once turnover is complete, observe to verify that:  Operators monitor their panels and are alert to control panel indicators and alarms; 21     DOE‐HDBK‐1214‐2014  Communications across boundaries and access to control areas are limited to official business only;

 Operators notice trends to detect problems early;  Operators respond quickly to warnings or alarms and keep their supervisor apprised of the situation;  Supervisors take action to troubleshoot and correct alarm conditions;  “Nuisance” alarms are not allowed to continue for extended periods;  Concurrent operations that will affect control panel indications or operations are briefed and coordinated;  Ancillary duties are limited so as not to interfere with normal duties;  Work-related tasks such as lockouts/tagouts, maintenance and testing, procedure review, required reading or administrative work do not constitute a significant portion of shift duties (may need to bring in another person to perform these duties if the workload is high); and  Trainees operating control area equipment are authorized, and work under the direct supervision of the normally-assigned operator. There are several aspects of activities in the control area that can be assessed during these situations. For example, while observing the startup of a system or component the following might be observed:  Operator response to indications as valves are operated and system components are energized or de-energized;  Communications between the control area and field personnel performing actions such as valve or system manipulations;  Operator response to alarms and abnormal conditions;  Operation of control area equipment by operators;  Use of status boards in the control area; and  Ancillary duties performed by control area operators. 4. Interviews The following personnel may be interviewed to assess control area activities:  Control area and at-the-controls operators;  Shift supervisors or foremen;  Facility managers and operations/program supervisors;  Maintenance and technical staff; and  Training department personnel. Most of the interviews should be conducted with control area operators and shift supervisors. For example, while observing a control area operator the following types of questions could be used:  What are your duties and responsibilities during the shift? 22     DOE‐HDBK‐1214‐2014  How is access to this area controlled? What personnel should be granted access? Is there a limit to the number of people allowed in this area?  Who normally operates the equipment in this area? How do you know who is authorized to operate this equipment? Under what circumstances do personnel other than yourself operate this equipment during your shift?  Do you normally perform ancillary duties while on shift? What types of ancillary duties do you perform? How do you know what ancillary duties are authorized during your shift?  What types of operations, maintenance and testing activities do you control from this area? Describe the process for coordinating and controlling these activities.  Are you notified before work is performed on any safety or safety system support system? D. Observing/Assessing Communications 1. Discussion Highly reliable communication systems are essential for the safe and efficient operation of facilities and the protection of facility personnel. Proper verbal (face-to-face or electronic) instruction and communications policies are necessary to ensure accurate, unambiguous communications among operations and support personnel. Management should provide guidance regarding the use of all forms of audible communications; this includes face-to-face and telephonic communications, and radio, public address, paging, text and alarm (horns, sirens, and bells) communications devices/equipment and their use.

Review pertinent portions of Part I, Performance-Based Assessment Overview, such as general practices. Portions of these guidelines may be incorporated into the assessment. 1.1 Related Conduct of Operations Topical Areas Document reviews, field observations and interviews conducted to assess communications will normally provide leads in other topical areas. Therefore, the assessment of communications systems can be combined with portions of other areas such as O&A, Shift Routines and Operating Practices, Control Area Activities, On-shift Training, Turnover and Assumption of Responsibilities, and Control of Interrelated Processes. NOTE: Good verbal communications should be assessed in all areas. 1.2 Conduct of Operations for communications includes:  Communication of abnormal and emergency conditions;  Communications during routine operations;  Proper placement, use and testing of communications equipment; and  Proper verbal communications practices, including use of abbreviations and acronyms. 23     DOE‐HDBK‐1214‐2014 2. Document Reviews Examples of documents and other written information that can be reviewed to assess communications include:  Communications policies and procedures;  TSRs (communications systems);  Records of communication system/equipment testing;  Portable radio restrictions and postings;  Radio frequency/channel assignments;  Laptop/tablet, cell phone, and pager restrictions and postings;  Turnover checklists; and  Approved lists of abbreviations and acronyms. 3. Field Observations Clear and concise verbal (face-to-face or electronic) communications and the proper use of the public address and portable communications equipment should be assessed whenever the opportunity presents itself. Poor communication can cause or compound problems, particularly during emergency situations, shift turnovers, pre-job briefings, and complex operations. The following situations are good candidates for observing communications:  Periodic testing of emergency communication systems and equipment;  Maintenance and other support activities that require communications between roving operators, technicians, control area operators, and supervisors;  Control area evolutions;  Startup and shutdown of systems and equipment;  Drills and exercises;  Response to abnormal events;  Occurrence notifications;  Shift turnover and shift crew briefings; and  On-shift training activities. There are several aspects of communication that can be assessed during these situations:  Clear, concise and correct verbal communications: o Are communications free from ambiguity? Do they use the phonetic alphabet? Do personnel avoid using words with multiple meanings or similar sounds (e.g., use ‘raise’ and ‘lower’ instead of ‘increase’ and ‘decrease’)? Do they use noun names and numbers (e.g., “open valve Golf forty-seven” instead of “open G47”)? o Are communications concise? Are messages short and to the point? o Are communications correct? o Are ‘repeat-backs’ used when appropriate? 24     DOE‐HDBK‐1214‐2014  Operation of emergency communication systems and equipment: o Are all personnel promptly alerted to the situation? o Is it clear to all that this is an abnormal/emergency situation? o Is it clear to the receiver who is giving the instructions or orders?  Use of the Public Address (PA) system: o Can everyone hear announcements clearly? If not, are alternate methods used to ensure everyone is alerted?

o Do only authorized personnel use the PA system? Is the use of acronyms avoided? o Do all personnel stop what they are doing (if safe to do so) and listen to the announcement?  Use of portable radios (controlled to prevent electronic interference with facility equipment, particularly Infrared or Ultraviolet fire detectors) o Is the same level of formality used? o Do personnel use plain language and avoid code phrases? Observing the activities above should be supplemented with observations of routine operations and maintenance communications conducted during walkthroughs and tours of facilities and control areas. Operators, maintenance and technical personnel, and shift supervisors from each shift should be observed to get a broad picture of communication accuracy, reliability, and effectiveness. If possible, the observations of a particular shift position should start at shift turnover since vital information about the operations, maintenance, and special operations in progress or planned at the facility is discussed at this time. For example, to observe communications while touring the facility with an operator the following methodology could be used:  Observe shift turnover between the on-coming and off-going operators. Pay attention to the information passed verbally between the operators concerning operations planned or in progress, status of facility systems and equipment, and any unusual or abnormal conditions. Review any documentation associated with the turnover process.  Attend the shift crew briefing and observe how the information is communicated/exchanged.  Observe the operator’s performance. Observe communications-related activities such as the use of portable radios, telephones, and face-to-face communications.  At the end of the shift, observe the turnover process between this operator and the on- coming operator. A similar process can be used to observe communications between shift supervisors, maintenance or technical personnel, and other support personnel when conducting operations at the facility. This is particularly useful during corrective maintenance or troubleshooting. 4. Interviews The following personnel may be interviewed to assess communications:  Operators and maintenance department personnel; 25     DOE‐HDBK‐1214‐2014  Shift supervisors or foremen;  Facility managers and operations supervisors;  Emergency management personnel; and  Training department personnel. Most of the interviews accomplished to assess Communications should be conducted with operators, maintenance and other support personnel, and shift supervisors. For example, while observing a shift supervisor beginning at the shift turnover, the following questions could be used:  How do you determine what information should be communicated at turnover to the on- coming operator? Are you required to use a turnover check sheet to document this information? What guidance have you received from the shift supervisor or facility manager regarding the type of information to be passed at turnover?  What equipment do you normally use to communicate with other shift positions? Do you ever use radios to communicate? If so, are there any areas in the facility that radios cannot be used? Where are they?  What type of information is passed on the facility public address system? Who normally uses this equipment? How frequently is it used?  Are emergency communications systems tested periodically?

 Is there a procedure that addresses how to communicate? If so, what type of information is contained in the procedure? Is there an approved list of abbreviation and acronyms? When is it to be used? Are there any circumstances that require the use of “repeat backs?” Can you give an example of one these circumstances? E. Observing/Assessing On-Shift Training 1. Discussion The goal of assessing On-Shift Training practices is to verify that trainees are supervised and controlled before, during and after training activities. This helps ensure quality training experiences and minimizes accidental, inadvertent or incorrect manipulation of equipment by trainees. Note: On-Shift Training is routinely referred to On-the-Job Training (OJT). The primary purpose of OJT is to allow personnel to acquire first-hand experience by performing or observing operations, special processes, tests, inspections, and other work activities. Qualified and effective instructors are also vital to the successful outcome and control of OJT; instructors assure quality and consistent training without compromising the safety and reliability of systems and equipment. Review pertinent portions of Part I, Performance-Based Assessment Overview, such as training questions and general practices. Portions of these guidelines may be incorporated into the assessment. 26     DOE‐HDBK‐1214‐2014 1.1 Related Conduct of Operations Topical Areas Document reviews, field observations and interviews conducted to assess on-shift training will normally provide leads in other topical areas. Therefore, the assessment of this area can be combined with portions of other areas such as Control Area Activities, Communications, Logkeeping, and Technical Procedures. 1.2 Conduct of Operations for on-shift training includes:  On-shift training program;  Authorization and documentation of training; and  Supervision and control of personnel under instruction (including suspension of training). 2. Document Reviews While observing an OJT session in the field, look for and review lesson plans and task completion documentation. After reviewing these documents, observe how and when they are used by OJT trainers. Examples of documents that should be reviewed when assessing OJT include:  Site or facility OJT procedures, plans of instruction, guides and lesson plans;  Supervisor, OJT Trainer, and trainee qualification cards and standards; and  Training Implementation Matrix. Just prior to observing OJT training, the assessor should review the OJT training syllabus (or equivalent, such as a plan of instruction) to determine the requirements that the OJT trainer will meet during the training session. For example, these requirements could include the following:  Material, tools and equipment needed to conduct the OJT;  Instructor preparation (e.g., reserve facility, verify conditions, and obtain records);  Terminal objective and enabling objectives;  Pre-job briefing, including hazards and hazard control, safety precautions, warnings, and prerequisites;  Communications between the trainee and the trainer, including the circumstances under which the trainee can actually operate equipment;  Control of the trainee by the OJT Trainer, including how close (physical proximity) the trainer will be to the trainee and actions required if an abnormality or emergency is encountered; and  Performance evaluation requirements. 3. Field Observations

Assessment of on-shift training should be conducted by observing on-shift training in progress. The facility manager, operations supervisor, training manager, or shift supervisor should know the time and location of on-shift training events that will take place during the assessment. Whenever possible, observations should be conducted during each shift. This will provide a good overview of the quality and consistency of the training conducted at the facility. 27     DOE‐HDBK‐1214‐2014 Examples of training observation opportunities include:  Performance of a lockout or tagout;  Lining up a system for startup or shutdown;  Starting or stopping equipment;  Conducting rounds and tours;  Making entries in logbooks and round sheets;  Conducting surveillances and in-service inspections; and  Connecting test equipment. When conducting observations of OJT it is important to remember that that the assessment covers conduct of training and control of trainees, not the trainee’s level of knowledge (i.e., the program not the trainee). Trainees need to have a clear understanding of what it is they are about to do, why they are doing it, and the expected effects their action(s) have on the equipment, system, or process. Unlike training conducted in a simulator or training facility, OJT allows the trainee to directly impact the status of facility operations. For this reason, OJT should be closely supervised and controlled in a way that prevents the trainee from accidentally or incorrectly operating a component, piece of equipment, or system that might cause danger to themselves, other personnel, or the environment, or adversely affect the operation of the facility. When observing OJT, try to begin when the trainer first meets with the trainee and continue until they finish the training session. The following are aspects of OJT supervision and control that the assessor should observe:  Was the number of trainees assigned to a trainer (student-to-instructor ratio) consistent with on-shift training policy?  Does the trainer coordinate with facility supervision prior to beginning training, and keep appropriate personnel apprised of the progress of training?  Does the trainer check the trainee’s qualifications prior to beginning?  Does the trainer explain the objectives to the trainee prior to starting an operation or task?  Does the trainer ensure the trainee understands how the equipment operates and the effects of operating the equipment before allowing the trainee to execute the operation or task?  Does the instructor provide clear instructions prior to allowing the trainee to perform the desired task?  Do the trainer and trainee use current, approved operating procedures?  Does the trainer remain in a position (e.g., arm’s length) to prevent the trainee from making a mistake (e.g., starting the wrong pump or opening the wrong valve) at all times during the training? Is there a clear, specific phrase used to indicate the suspension of training, such as “hands off” or “I have the controls”?  Does the trainer provide instruction on how to stop training if an actual problem was encountered? 28     DOE‐HDBK‐1214‐2014  Does the trainer allow the trainee to exercise their own knowledge without excessive coaching, asking leading questions, or prompting?  Was communication between the trainer and trainee clear and concise?  Does the trainer review (critique) the trainee’s actions after training is completed?

 Does the trainer document satisfactory completion of training on the trainee’s qualification card? 4. Interviews The following personnel may be interviewed to assess on-shift training:  OJT trainers in the operations and maintenance departments;  Unqualified operators and maintenance personnel;  Operations and maintenance supervisors and managers;  Facility managers and operations supervisors; and  Training department managers. An interview with an OJT trainer may include the following questions:  What training requirements will you fulfill prior to being qualified as an OJT Trainer?  How are you evaluated as an OJT Trainer? Who does the evaluations? How often are your evaluated?  How often do you have to re-qualify, and what does this entail?  Are there proficiency requirements, such as performing OJT training within the last six months in order to remain current?  What forms are used to plan, conduct, review and document OJT?  How do you ensure the trainee understands the requirements for the task at hand?  What actions are taken if an unusual situation arises?  How do you ensure all appropriate personnel (e.g., operations and maintenance) know when and where OJT is occurring? Do these personnel, in turn, keep you apprised of conditions or plans that may affect you?  Are trainees ever used to support operations? If so, what are the rules for using trainees to support operations? F. Observing/Assessing Investigation of Abnormal Events, Conditions, and Trends 1. Discussion The goal of assessing operations practices for Investigation of Abnormal Events, Conditions and Trends to ensure that events are identified, reported analyzed, and corrected to prevent recurrence. Specifically, the program should ensure the following:  Events are thoroughly investigated in a timely manner by designated investigators to assess the impact of the event;  The root cause of events is determined; 29     DOE‐HDBK‐1214‐2014  Corrective actions to prevent recurrence of events are implemented and tracked to completion;  Investigation reports are shared with appropriate personnel; and  Suspected sabotage is immediately investigated. Review pertinent portions of Part I, Performance-Based Assessment Overview, such as training questions and general practices. Portions of these guidelines may be incorporated into the assessment. 1.1 Related Conduct of Operations Topical Areas Document reviews, field observations and interviews conducted to assess investigations will normally provide leads in other topical areas. Therefore, the assessment of this area can be combined with portions of other areas such as O&A, Notifications, and technical procedures. 1.2 Conduct of Operations for investigation of abnormal events, conditions and trends includes:  Identifying events or conditions requiring investigation;  Designating qualified investigators; and  The investigation process, including reporting, causal analysis, corrective action determination, tracking and trending, and lessons learned. 2. Document Reviews While the best way to evaluate an investigation program is to observe it in action, a more practical method is to review investigative reports and associated records. If there is an ongoing investigation, concentrate on that; if not, then review a representative sampling of reports and conduct thorough interviews. Documents that can be reviewed to assess investigations include:

 Site or facility procedures and policies on investigation of events;  Training and qualification requirements and records for personnel involved in event investigations;  Investigation reports (including Environment, Safety and Health Reporting and Occurrence Reporting per DOE Orders 231.1 and 232.2);  Corrective action management procedures and databases;  Event tracking and/or trending databases; and  Lessons Learned files. Items that an assesso should look for when conducting reviews of investigation reports include:  Are specific events that require investigation listed? Are the criteria for identifying other events/conditions discussed? Examples of specific events should include: o Violation of safety/design limits (e.g., TSRs); o Abnormal or unexpected system performance that adversely affects operations or safety; 30     DOE‐HDBK‐1214‐2014 o Abnormal or unexpected safety conditions; o Events reportable to DOE/NNSA or other agencies; o Unplanned shutdown or significant loss of function; o Procedural violation or personnel error with actual or potential personnel injury or facility damage; o Radiological or toxic material release limits are exceeded; o Lost special nuclear material; o Recorded data is out-of-specification or shows unexpected trends, with actual or potential adverse impact on operations or safety; o Actual or suspected sabotage; o Measuring and test equipment is found to be out of calibration, with actual or potential impact on operations or safety; o Repetitive problems; and o Near miss situations (as directed by appropriate authority).  Is a senior manager responsible for the investigation?  Were the investigators qualified to investigate this particular type of event? Investigators should: o Be technically qualified and experienced, and o Have no vested interest in the results.  Data collected after the event: Were all personnel involved in the event contacted to gain a complete picture of the event? Was the data collected in a timely manner? o Recorded data should include initial conditions, operator statements, pertinent computer/instrument printouts or charts, pertinent documentation and records (procedures, permits, hazards analysis, photographs, etc.), and other appropriate information. o Records and data are annotated to prevent misinterpretation. o Event facts are presented in chronological order.  The root cause of the event: Does it make sense? o Event is analyzed to determine equipment and personnel response, procedure and equipment adequacy, human performance factors, and safety impact. o Causes are determined; look out for cases where investigators quickly determine the cause to be operator error or a training problem, as these easy determinations (“round up the usual suspects”) can mask underlying problems. o Is the extent of the condition determined to understand the breadth of the issue?  Corrective actions: Do they appear appropriate to prevent recurrence? Do they address the identified causal factors? Are they approved and tracked to completion?  The thoroughness of the investigation: Did the investigator exhaust all avenues of potential information in the course of the investigation and consider multiple causes?  The overall format and content of the report: Is the report logically written, indicating that the investigation was performed in a systematic fashion? Is there evidence of management review and approval?

31     DOE‐HDBK‐1214‐2014 o Report should contain a description of the event, its impact, root causes, corrective actions, and lessons learned (shared with appropriate site/facility personnel and other sites/organizations as appropriate). o Report should be approved by the responsible manager and reviewed by other managers and safety personnel.  Indications of investigator bias.  Are appropriate events entered into the site/facility trend analysis process/program? o Periodic summaries of event analysis and trends should be provided to managers.  Events are evaluated for lessons learned. 3. Field Observations While assessment of investigation of abnormal events, conditions and trends requires an emphasis on document reviews and interviews, Assessors should observe an ongoing investigation if possible. Observations could include the following:  Investigators should observe immediate and supplemental actions for later analysis and take care not to intrude until the system or program is in a safe and stable condition.  Investigators should take actions to preserve the event scene (to the extent possible) to allow data collection; however, this should not interfere with ongoing operations, such as placing systems into a safe and stable condition, unless vital to understanding the event. Photographs may be used in combination with drawings and schematics to capture information that may be lost when placing the system in a safe and stable condition.  Investigation data should be preserved for future reference and properly classified.  Investigators should demonstrate familiarity with the affected systems, instrumentation, and operations.  Investigators should demonstrate good investigation techniques.  Operators and other affected personnel should be trained on the lessons of the event upon their return to work.  Investigators should determine if the event or problem applies to other facilities or programs; if so, this information should be communicated to appropriate management as soon as possible. 4. Interviews The following personnel may be interviewed to assess investigation of events:  Selected members of the investigation team;  Shift supervisors and foremen;  Facility managers and operations/program managers;  Operators and other personnel involved or affected by the event;  Supervisors and managers responsible for event identification, analysis, corrective actions, tracking and trending; and  Personnel responsible for development of lessons learned and training associated with events. 32     DOE‐HDBK‐1214‐2014 A line of questioning for a member of the investigation team might proceed as follows:  As a member of the investigation team, what are your responsibilities? Are they delineated in writing?  What qualified you to investigate this particular event?  Briefly explain the process employed in performing an investigation.  Discuss how events are reconstructed.  How are lessons learned from investigations used to help prevent recurrence of similar events? G. Observing/Assessing Notifications 1. Discussion The goal of assessing operations practices for event Notifications is to ensure timely response. Notifications should be timely in order for upper level and/or supporting agencies or organizations to analyze the reported conditions and muster needed resources and/or direction. Timely notification also allows for the dissemination of information about an event to other similar facilities or programs; they, in turn, can take action to prevent occurrence of the event at their facilities. Notifications may also be required to fulfill statutory requirements regarding the health and safety of workers, the public and the environment.

Review pertinent portions of Part I, Performance-Based Assessment Overview, such as training questions and general practices. Portions of these guidelines may be incorporated into the assessment. 1.1 Related Conduct of Operations Topical Areas Document reviews, field observations and interviews conducted to assess notifications will normally provide leads in other topical areas. Therefore, the assessment of this area can be combined with portions of other areas such as O&A and Investigation of Abnormal Events, Conditions, and Trends. 1.2 Conduct of Operations for notifications includes:  Procedures for internal and external notifications, and  Communications equipment for notifications 2. Document Reviews While the best way to evaluate the notifications process is to observe it in action, a more practical method is to review past notifications and associated records. If there is an ongoing event that requires notifications, concentrate on that; if not, then review a representative sampling of reports and conduct thorough interviews. Documents should provide a structured methodology that allows the proper people to be notified in a timely fashion concerning any events that warrant invoking the procedure. Of particular importance are notifications required by procedure or directives/regulations (e.g., TSRs or OSHA). 33     DOE‐HDBK‐1214‐2014 Documents that can be reviewed to assess notifications include:  The facility programs and procedures that spell out the responsibilities for making notifications and associated documentation. Particular attention should be given to those procedures that flow down the requirements of DOE Order 151.1C, Comprehensive Emergency Management System, DOE Order 231.1B, Environment, Safety and Health Reporting, and DOE Order 232.2, Occurrence Reporting and Processing of Operations Information.  Filed reports to assure all reportable aspects of the event were identified.  Facility operating logs and records, to determine if any reportable events were not properly identified and reported.  Notification lists.2 * A check of facility logs and records will sometimes reveal occurrences of reportable conditions that were not identified or were not reported in a timely manner. A check may also be made of the list of personnel to be notified to ensure that it is complete and that it contains up-to-date telephone numbers or other instructions for reaching the designated personnel. Finally, a review of past notifications should indicate whether they have been accomplished in a consistent, thorough and timely manner. Key elements that an assessor should look for when conducting reviews include:  Procedures for internal, DOE/NNSA, and external notifications, including events, persons to be notified, persons responsible to make notifications, contact information, and recordkeeping: o Responsibilities for making notifications are specifically assigned to positions or persons; o Notification timeliness standards are established; o Primary and alternate personnel to be notified for each event are identified and documented; o Contact information for the personnel to be notified are kept current and available to notifying personnel; and o All notifications are documented in formal records that include date, time, reason, person notified, and person making notification. 3. Field Observations

One item to be assessed by field observation is that adequate equipment for making notifications is available at the main control area (such as an Emergency Operations Center or Incident Command Post) and/or other appropriate locations. Assessment of notifications may also be performed in conjunction with a pre-planned facility drill. Regardless, a walkthrough of a facility’s notification process could be an adequate substitute. 34     DOE‐HDBK‐1214‐2014 If there is no opportunity to observe the notification process, the following items may be reviewed:  The use of a procedure or checklist by the person responsible for making the notifications;  Notification timeliness as required by directive or regulation;  Mechanisms used for notification (email, text message, verbal) and protocols for preference and use of each;  Level of detail of information provided and inquisitiveness of operations organization to gather and disseminate information; and  The accuracy of the notification contact list and what contingencies are in place if someone on the list cannot be readily contacted. 4. Interviews The following personnel may be interviewed to assess notification of events:  Facility and emergency operations center operators and supervisors, and  Facility managers and operations supervisors. An example line of questioning for the facility manager or operations supervisor might proceed as follows:  Briefly describe how the notification process is executed.  As a supervisor, what role do you play in the notification process?  What training did you receive in order to properly execute your event notification responsibilities?  What documentation is required?  Are there aids (e.g., check lists) that assist you in notifying the appropriate personnel? If so, how detailed is it and how is it kept current (do you periodically verify the contact telephone numbers)? H. Observing/Assessing Control of Equipment and System Status 1. Discussion The goal of assessing Control of Equipment and System Status is to verify operation within design limits and that configuration controls are in place to ensure subsequent changes do not allow operations outside of these limits. Facility operations personnel should know the status of all equipment and systems so they can maintain proper control. Review pertinent portions of Part I, Performance-Based Assessment Overview, such as training questions and general practices. Portions of these guidelines may be incorporated into the assessment. 1.1 Related Conduct of Operations Topical Areas Document reviews, field observations and interviews conducted to assess control of equipment and system status will normally provide leads in other topical areas. Therefore, 35     DOE‐HDBK‐1214‐2014 the assessment of this area can be combined with portions of other areas such as Shift Routines and Operating Practices, Lockouts and Tagouts, Independent Verification, Logkeeping, Control of Interrelated Processes, Technical Procedures, Operator Aids and Component Labeling. 1.2 Conduct of Operations for control of equipment and system status includes:  Authorization to operate and control equipment and systems;  Control of system alignments;  Compliance with safety and operational limits;  Configuration control, including control of temporary modifications;  Proper use of lockouts/tagouts;  Identification and correction of deficiencies;  Maintenance, including post-maintenance testing and return to service; and

 Awareness and timely response to control panel and indicator/alarm issues. 2. Document Reviews Examples of documents that may be reviewed to assess equipment and system status include:  Program, operating and maintenance/testing procedures;  Work and configuration/change control procedures;  System status and equipment deficiency logs;  Lockout/Tagout log;  Shift supervisor and operator logs;  Equipment and system alignment check sheets;  System design descriptions;  Equipment databases;  Equipment drawings and diagrams (e.g., electrical schematics and piping and instrumentation diagrams);  Maintenance work requests and completed work packages (including post-maintenance testing); and  Temporary modification documentation. Some document reviews will be conducted to follow up on system/equipment control issues that were observed during a tour/walkthrough, or based on information gained through personnel interviews. For example, if equipment was out of service during a walkthrough, the following documents could be checked:  System/equipment status and deficiency logs;  Lockout/Tagout log;  Shift supervisor and operator logs;  Shift turnover checklists;  Maintenance work package; and 36     DOE‐HDBK‐1214‐2014  Configuration/change control package. If the assessor has the time and expertise, they could walk down a system and compare conditions to the current, approved configuration drawing (such as a piping and instrumentation diagram). 3. Field Observations There are certain general attributes to watch for at all times while conducting control of equipment and system status observations:  System alignment: It is vital that safety systems, safety support systems and important- to-safety systems be properly aligned for operation. Once aligned, any changes to a system should be properly authorized, controlled, tested, tracked, and documented. Examples of activities that may require alignment are system startup after shutdown, repair or maintenance, outage recovery, or mode changes. o Restoration of safety-related systems following maintenance includes functional testing of their capability (operability); o Deviations from the approved alignment (including lockouts and tagouts) should be tracked and controlled (e.g., a status board or equivalent); o Alignment checklists are approved by an appropriate supervisor or manager; and o Records of equipment and system alignments are retained for operator reference (also see configuration control, below).  Awareness of changes: Normally the facility/operations supervisor is responsible for maintaining system configuration and should authorize status changes for major equipment and systems. The supervisor should also ensure that affected operators are made aware of all changes.  Proper work authorization and coordination between departments/divisions, particularly between operations and maintenance personnel.  Compliance with operational limits: If a safety or safety-related system is to be taken out of service for activities such as maintenance, testing or modification, it will not be able to perform its design function for the duration of the activity (i.e., no longer “operable” for safety design purposes). The supervisor should take any actions required to comply with TSRs (e.g., a Limiting Condition for Operation or a Safety Administrative Control) prior to taking the system out of service, and ensure that appropriate personnel are aware of the limiting conditions and required actions.

 Configuration control: Normally this is the responsibility of system engineers, but supervisors and operators should be aware of the importance of controlling the configuration of equipment and systems, particularly those that are referenced in the safety basis. The objective of change control is to maintain consistency among design requirements, the physical configuration, and the related facility documentation, even as changes are made. o Authorized personnel approving the work should ensure that the change control process, including the Unreviewed Safety Question process, is used for changes that could impact the safety analysis or the hazard controls. 37     DOE‐HDBK‐1214‐2014 o Drawings and procedures should be updated as part of the work processes to implement a change. Other documents will need to be updated and issued as “as- built” documents following implementation. All should be updated in a timely manner. o Operations personnel and all other affected organizations have ready access to current, approved engineering documents. o Post-implementation testing should be completed and all acceptance criteria satisfied prior to turnover to operations (unless specific tests are to be done post-turnover). o More information can be found in DOE Orders 420.1, 433.1 and 440.1 and DOE- STD-1073-2003.  Management of deficiencies: Equipment deficiencies should be identified, analyzed for their effect on equipment and system operability, repaired, tested, tracked and documented. o Operators should document equipment deficiencies (e.g., logs, tags and/or status board) so that other operators can maintain awareness of the deficiency. They should also log any operational affects such as changes in indications and/or alarms. o Supervisors should determine the safety effects of the component (see compliance with operational limits, above), and submit the deficiency for repair using the established maintenance work control process. Note: Low priority equipment (not safety-related) may not be repaired for some time, depending on the maintenance work load. o The status of work in progress should be tracked and operators kept up-to-date on the progress. o The work package will specify any re-testing necessary for return to service. This ensures the component is functioning properly and that no new problems were introduced during repairs. The appropriate supervisor should authorize its return to service.  Lockout/Tagout: Supervisors approve lockouts and tagouts and track their status.  Awareness of control panel and local alarm issues: Operators and supervisors should be aware of inoperable alarms, alarms with temporary set points, and multiple-input alarms that do not provide indication of a subsequent condition. Some other limitations: o Deficient alarms are documented and entered into the maintenance work control system for correction. o Operators take appropriate actions to monitor conditions when alarms are unreliable. Nuisance alarms are fixed in a timely manner. o Operators and supervisors are aware of alarms expected during normal operations.  Control of temporary modifications: Administrative systems control temporary modifications such as electrical jumpers or lifted leads, pulled circuit cards, disabled alarms, piping jumpers or blocks, disabled relief valves, strainers or filters temporarily installed or removed, temporary shielding, and blocked drains. These controls include:

o Engineering review and approval of the design and safety of the modification before installation; o Written authorization for the installation; 38     DOE‐HDBK‐1214‐2014 o Independent verification of installation and removal of the temporary modification; o Documentation of the modifications; and o Periodic audits of all installed temporary modifications. The following situations are good candidates for conducting observations:  Maintenance operations that require interface or coordination with other shift positions and the use of lockouts and tagouts (including post-maintenance testing);  Startup and shutdown of systems and equipment that require component alignment;  Process or rate change operations;  Installation of temporary equipment or system modifications; and  Equipment and system surveillance, inspection and testing (including return to service). Observing the activities and operations discussed above should be supplemented with several facility walkthroughs and tours with operators, maintenance personnel, and shift supervisors. During walkthroughs, the following additional aspects of equipment control can normally be evaluated:  Documentation of and compliance with operational limits for equipment and systems;  Control of alarm status and actions taken to monitor equipment parameters when alarms are de-energized;  Administrative control systems used to review, approve, document, and audit the installation of temporary modifications;  Control and availability of equipment and system documents such as procedures, drawings, and specifications;  Equipment deficiency identification, documentation, and communication; and  Housekeeping and waste management following changes to equipment and system status. 4. Interviews The following personnel may be interviewed to assess control of equipment and system status:  Operators and maintenance department personnel;  Shift supervisors or foremen;  Facility managers and operations supervisors; and  System engineers. For example, the following questions may be used to interview an operator or shift supervisor:  Describe the process used to identify, document, and communicate equipment deficiencies. Who is responsible for identifying deficiencies? How are they corrected?  What are your system operational limits?  When and how are equipment and system alignments conducted? How are they tracked and documented? Is independent verification required?  What actions do you take when alarms are temporarily disabled or de-energized?  What actions do you take for nuisance alarms? 39     DOE‐HDBK‐1214‐2014  Describe the process used to review, approve, control, document, and audit the installation of temporary modifications.  How is a lock and tag placement accomplished? What are your responsibilities? Describe the process used to review, approve, place, track, and remove locks and tags.  How is independent verification of lockouts and tagouts, system alignments/re- alignments, and temporary modifications accomplished and documented? I. Observing/Assessing Lockouts and Tagouts (LOTO) 1. Discussion

A LOTO program may be referred to as a Hazardous Energy Control program (or similar name). Clear roles and responsibilities, training, the verification of absence of energy, and integration with related areas such as control of equipment/systems and the work control process are particularly important. The goal of assessing the Lockout and Tagout (LOTO) program is to verify that personnel are protected from injury. The LOTO program serves to implement the OSHA Control of Hazardous Energy Standard (29 CFR 1910.147 & .269). Hazardous energy control is implemented in the field by application of locks and tags to preclude the unexpected release of energy or unexpected start up of equipment, which can cause serious injury or death. Proper LOTO of equipment also serves to protect equipment from damage and protect the physical boundaries of plant systems. 1.1 Related Conduct of Operations Topical Areas Document reviews, field observations and interviews conducted to assess LOTO will normally provide leads in other topical areas. Therefore, the assessment of this area can be combined with portions of other areas such as Control of Equipment and System Status, Independent Verification, and Component Labeling. 1.2 Conduct of Operations for LOTO includes:  Procedures and personnel roles and responsibilities;  Operations practices;  Compliance with OSHA rules and federal regulations for the protection of workers;  Compliance with NFPA standard electrical requirements;  Control of components used in the LOTO program;  Training on LOTO; and  Use and control of Caution tags. 2. Document Reviews Some terms associated with LOTO (energy control) are:  An energy source is any source of electrical, mechanical, hydraulic, pneumatic, chemical, thermal, or other type of energy. 40     DOE‐HDBK‐1214‐2014  A lockout is the placement of a lock and tag on an energy isolating device, in accordance with an established procedure, assuring the energy isolating device and the equipment being controlled cannot be energized or operated until the lockout device is removed.  An authorized worker is one who has been trained and qualified to position components to isolate and verify absence of energy and secure such components with locks and tags.  An affected worker is one whose job requires operation or use of a machine or equipment for which servicing or maintenance is performed under lockout, or whose job requires work in an area where servicing or maintenance is performed. The following documents may be reviewed when assessing LOTO:  LOTO (energy control) procedures;  List/database of equipment/system LOTO points, and associated hazard evaluations;  Active and completed LOTO permits;  LOTO program reviews, inspections and/or audits;  Training records, including the list of personnel qualified to authorize or perform LOTO; and  Lessons learned, required reading or other methods used to inform employees of deficiencies discovered in the LOTO program or processes. Always review several facility LOTO indexes, logs, or status boards to identify what equipment or systems are currently locked and tagged, as the results will provide input to plan a field observation. Important items to check include:  Verify that there is a tagout (hazardous energy control) permit or equivalent for every active entry in the LOTO control index/log/status board. Additionally, check for evidence that status is updated regularly.

 If there is not a one-to-one correlation, then a permit is missing or was cleared and not removed from active status.  Each permit should be properly filled out (e.g., scope, component, alignment position, positioner, verifier, energy sources, control boundary, notes, and confirmation of readiness to work) and signed. Some document reviews will be conducted to follow up on LOTO issues that were observed during a tour/walkthrough, or based on information gained through personnel interviews. For example, the following documents could be checked for a locked and tagged component seen during a walkthrough:  System/equipment status and deficiency boards/logs;  LOTO sheets and log;  Shift supervisor and operator logs;  Shift turnover checklists;  Maintenance work package; and  Configuration/change control package. 41     DOE‐HDBK‐1214‐2014 3. Field Observations It is important to remember that poorly planned or executed LOTO can result in serious injury or death (most sites consider an improper LOTO as an imminent danger situation). Planning should be performed by LOTO specialists who possess the skills and experience to determine risks, perform hazard analyses, identify controls and good practices, and specify the means to isolate energy sources. Proper planning results in a definitive scope of work, which in turn provides the basis for an effective LOTO. It is also important to ensure that LOTO planners, supervisors and workers do not unthinkingly rely on “off the shelf” LOTO packages. Just because they have performed a specific LOTO ten times in the past year does not mean that conditions cannot change. Each time work is required, the contractor should assess the scope of work and look for any changes (e.g., configuration changes or a new energy source including alternate sources). Additionally, the LOTO package should be independently reviewed; for example, a system engineer, a safety specialist, the affected supervisor and the responsible department manager can perform reviews. Finally, the contractor should be cautious about relying on walkdowns to identify energy sources. In the case of electricity, wires are often invisible to the observer (e.g., routed through walls, floors and conduits or above false ceilings). Therefore, the planners should refer to prints whenever possible, including “as-built” drawings. Operations of LOTO activities available for assessment can be identified via discussions with LOTO managers or supervisors, review of the Plan of the Day or Plan of the Week, or review of the listing of active LOTOs. The following situations are good candidates for observation:  Periodic and corrective maintenance of equipment and systems;  Equipment or system alignment, repair and testing;  System impairments (e.g., fire protection system);  Returning a system to operation; and  Work involving LOTO that is performed by subcontractors or other outside service personnel. There are several aspects of the program that should be assessed during these operations:  Lockout devices and tags are singularly identifiable and never used for other purposes.  Lockout devices are constructed to allow attachment of locks and tags to energy isolation devices without difficulty, and are substantial enough to prevent removal without excessive force or unusual techniques.  Appropriate authorization is obtained prior to beginning the LOTO, and all TSR actions are completed as necessary.

 Appropriate personnel are notified prior to beginning the LOTO.  Determination of isolation boundaries involve a combination of drawings, walkdowns, and technical personnel to the degree necessary to understand the hazardous energy sources and associated isolation boundaries. 42     DOE‐HDBK‐1214‐2014  Verification of the absence of energy is critical, both during the initial isolation and during independent verification.  Physical placement of tags and locks conforms to the LOTO permit, and are performed in the prescribed sequence.  Tag entries are complete and legible.  Tags are placed so as not to obscure indications or controls, while remaining readily apparent to operators.  Independent verification of equipment/system alignment and absence of energy are truly performed independently.  LOTO performed by subcontractors or other outside service personnel meet site/plant LOTO standards.  Appropriate personnel are notified upon completion of the LOTO and the index/log/status board is updated accordingly. One portion of the LOTO assessment should cover equipment or systems that have already been locked and tagged:  Select an active LOTO permit.  Walkdown the system and locate each lock/tag indicated on the permit.  Verify the proper placement and integrity of the lockout devices and tags (ensure they have not been degraded by the environment to which they are exposed).  Verify that tags are securely attached, properly filled out, and legible.  Look for signs of a lack of total energy isolation (e.g., fluid or air leaking past a locked- closed valve, loose electrical jumpers, or no discernable separation where an air gap should be).  Verify that affected operators and supervisors are aware of the status of all locked/tagged equipment under their control. Another portion of the review should be observation of a planned or on-going operation that involves LOTO:  Observe/review the preparation, review, and approval process for LOTO conducted by the responsible manager or supervisor.  Review the LOTO permit and accompanying tags.  Verify that affected workers and supervisors are notified of the LOTO.  Observe an authorized worker position components, verify the absence of energy, and place the locks and tags.  Observe independent verification of the alignment, absence of energy, and proper LOTO.  Verify appropriate personnel are notified of the LOTO.  Verify the LOTO index/log/status board is updated with the new equipment/system status. 43     DOE‐HDBK‐1214‐2014 4. Interviews The following personnel may be interviewed to assess LOTO:  Operators and maintenance/utility department personnel;  LOTO authorized workers and affected workers;  Shift supervisors or foremen;  Facility managers and operations/program supervisors;  LOTO Manager or Supervisor;  Hazardous Energy Control program manager; and  Training department personnel. Some example Interview questions for LOTO supervisors and managers include:  What is the purpose of the LOTO (Hazardous Energy Control) Program? What are your responsibilities under the program?  What training have you had concerning how to plan or supervise LOTO? How was the training conducted? Have you received any retraining? Have you distributed any lessons learned?  How do you plan LOTOs? How do you identify hazards and evaluate hazard controls? Who reviews your work?  How do you prepare a LOTO permit?  Describe the review and approval process

 How do authorized workers verify the absence of energy? What types of energy do they check for?  How is lock and tag placement accomplished?  What are the duties and techniques of LOTO verifiers?  How do you review completed LOTO? How do you record the discrepancies that you find? Describe the process used to resolve these discrepancies. How is this documented?  When is a tagout (only) utilized, versus using a LOTO?  Are air gaps ever used instead of a lockout?  Describe the LOTO notification process.  Is temporary clearance of locks/tags allowed? If so, how is this accomplished?  How do you remove locks and tags for absent workers (or lost keys or damage)?  Are subcontractors allowed to perform LOTO? If so, how are they trained and monitored?  How are lock keys controlled?  Do you use Caution tags? If so, what are the circumstances, limitations and procedure? Do you periodically review caution tags? How long should a caution tag remain in place without taking actions necessary to remedy the conditions that required hanging the caution tag in the first place?  How are LOTO lessons learned disseminated? 44     DOE‐HDBK‐1214‐2014 Some example Interview questions for LOTO authorized workers include:  What is the purpose of the LOTO (Hazardous Energy Control) Program? What are your responsibilities under the program?  What training have you had concerning how to do LOTO? How was the conducted? Have you received any retraining? Have you received any lessons learned?  What information appears on a LOTO permit?  What do authorized locks and tags look like?  How do you verify the absence of energy? What types of energy do you check for?  How do you position components? Is the sequence in which you align components and place LOTO important?  How do you perform the duties of a LOTO verifier? What techniques do you use to verify the position of locked components?  How do you remove locks and tags?  How is equipment/system alignment checked after the locks and tags are removed?  Describe the LOTO notifications process.  Are you notified of LOTO deficiencies or mishaps? Do not limit interviews to those persons directly involved in authorizing or placing locks and tags. Select a few personnel not involved in the LOTO (i.e., affected workers) and ask them the meaning and significance of locks and tags. Sample questions may include:  What is the purpose of the LOTO (Hazardous Energy Control) Program? What are your responsibilities under the program?  What training have you had concerning locks and tags?  Are you notified before a piece of equipment for which you are responsible or operate is locked and tagged?  Are there any circumstances where you can remove a lock or tag in order to operate equipment?  What do you do if you have a question or comment regarding a lock or tag? J. Observing/Assessing Independent Verification 1. Discussion The goal of assessing Independent Verification (IV) is to verify that critical equipment configuration is maintained in accordance with controlling documents. If implemented effectively, IV reduces the chance that human error will cause an operational failure. Review pertinent portions of Part I, Performance-Based Assessment Overview, such as training questions and general practices. Portions of these guidelines may be incorporated into the assessment. 45     DOE‐HDBK‐1214‐2014 1.1 Related Conduct of Operations Topical Areas

Document reviews, field observations and interviews conducted to assess independent verifications will normally provide leads in other topical areas. Therefore, the assessment of this area can be combined with portions of other areas such as Control of Equipment and System Status and LOTO. 1.2 Conduct of Operations for Independent Verification includes:  Identification of structures, systems, and components, operations, and programs requiring independent verification;  Situations requiring independent verification; and  Methods for performing and documenting independent verification. 2. Document Reviews A list of documents to be reviewed during the assessment may include:  Lists of structures, systems, and components, operations, programs and situations that require IV (facility- or program-specific);  IV procedures;  Equipment/system alignment check sheets and LOTO permits; and  IV training records. Note: Some portions of the review of program documents that identify items and situations that require IV will have to be confirmed during interviews. Interviews can indicate if management considered all safety and safety-related components/systems, and how they determined those items whose mispositioning could challenge safety equipment or cause undesirable results. IV procedures usually include a procedure covering the IV program, along with facility- and program-specific procedures that call for IV in specific cases. The following should be checked during the review of the IV program and a sample of facility- and/or program-specific procedures that include IV requirements:  Structures, systems, and components, operations, and programs requiring IV should include components whose mispositioning could: o Challenge safety-related equipment; o Cause shutdowns or other undesirable results; and o Lead to unintended toxic or radioactive material release.  Situations requiring IV should include: o Equipment that is required to be available and it is reasonably possible that components were mispositioned. Management approval should be required for repositioning, followed by independent verification; o System alignments; and o Lineups to take equipment out of service or return it to service (e.g. isolation boundaries, equipment under maintenance or repair, instrumentation lineups for testing and their restoration, or work on backup components and their restoration). 46     DOE‐HDBK‐1214‐2014  Exemptions from independent verification requirements: o Components whose mispositioning does not affect system performance; o Components whose mispositioning is immediately known to operators; and o Situations where significant radiation or toxic exposure would be required for verification; in these and similar situations, alternate means of determining position are considered and any such exemptions are approved by senior operations management.  Methods for performing IV: o Procedures should provide examples or reference documentation explaining how to perform verification of particular components (e.g. manual valves, solenoid-, motor- and air-operated valves, circuit breakers, blank flanges, removable links and fuses, or control power availability); o Methods used to achieve independence in verifications, such as minimizing interactions (time and space) between the operator and the verifier. The procedure should also cover special situations where this kind of independence may not be practical, such as when verifying a throttle valve’s position by opening or shutting the valve (see concurrent verification);

o Components under LOTO will not be manipulated for IV; and o Verifiers do not change component position or status to correct a mispositioned component or other inconsistency. Procedures should detail the provisions to deal with mispositioning and other inconsistencies.  Procedures should generally favor direct local position checks over remote indicators, process indications (e.g., flow, pressure or voltage), or surveillance testing.  Requirements for performing periodic verification of critical components during operation, which would not normally need IV.  Situations allowing concurrent dual verification and how it is conducted: o Provisions should be made for maintaining independence to the maximum extent possible, and o Concurrent dual verifications should be specifically authorized. o Methods for conducting concurrent dual verification should be specified.  Documenting IV: o LOTO permit sheets; o Alignment sheets; o Work packages; o Change control packages; and o Test packages.  Training requirements for IV: o General, and o Specific (e.g., facility, program or component). 47     DOE‐HDBK‐1214‐2014 3. Field Observations Operations that require IVs and are available for assessment can be identified via discussions with managers and supervisors and review of the Plan of the Day or Plan of the Week. The following situations are good candidates for observation:  Periodic and corrective maintenance of equipment and systems that requires component manipulation and/or LOTO;  Equipment or system alignment, repair or testing;  System impairments (e.g., fire protection system);  Returning a system to operation; and  Other work involving LOTO. In all cases, IV instructions minimize the interaction between the performer and the verifier in order to preserve the independence of each. When observing IV, the assessor should see some form of time-distance spacing between the two verifiers. Remember that an independent check is required. Time spacing means that the two verifiers should not check a component at the same time. Distant spacing means that neither verifier should be able to see the other check the component (with the exception of dual verification). Operating experience alone may not prepare an individual to perform IV. Specific training on IV techniques is usually required. To observe two operators conducting a system alignment that requires IV, the following methodology may be used:  Accompany the first operator as he aligns the system. Observe the use of a system diagram or check sheet and the operator’s skills and abilities in locating and operating system components. Note the location of the second operator during the alignment.  Once the system has been aligned by the first operator, observe the second operator’s IV techniques: o Did she remain separated (time and distance spacing) from the first operator during his alignment? o Did she follow a specific process or series of sequential steps, if required? o Does she demonstrate the ability to locate and verify the position of all system components (e.g. manual valves, motor-, solenoid- & air-operated valves, blank flanges, circuit breakers, removable links, and fuses)? o Does she verify the position of system components without manipulating them? o Does she use direct local position checks rather than relying on remote indicators or process indications?

o Did she use concurrent dual verification? If so, did the situation require the dual verification or could it have been accomplished using an alternate method? o Does she document the results of her verification? NOTE: The indicated state of equipment or systems (i.e., the apparent position of valves and switches) is not always reliable. Valves can end up in the wrong position due to poor 48     DOE‐HDBK‐1214‐2014 procedures, labeling, lighting, and accessibility, or the wrong valve may have been operated and/or checked. Subsequent checks may fail if the state of the valve is inferred by observing how much of the stem is showing, as the design of a valve may be atypical or unusual. Similarly, checking that a valve is open (or closed) by attempting to operate it in the desired direction can be misleading, as the valve may be jammed or frozen. Visual indicators of the valve state may also fail if they have become disconnected from the functional part of the valve (e.g., the presence of a seal can lead to the assumption the valve is open). Switches controlling electrical equipment present similar challenges. The wrong switch may be operated, the correct switch may be operated in the wrong direction, or there can be an interruption between the switch and the device (e.g., a wire disconnected or breaker tripped). In addition, an indicator light could represent a signal to a relay (which could fail), rather than indicating power to the equipment. A switch being in the “on” position and/or a light being illuminated does not necessarily mean the equipment has started. Instead of relying solely on the apparent position of valves or switches, it is important to confirm the effect of the equipment to be sure that no failure has occurred. For example, when working with dangerous materials in hoods, scientists may attach small pieces of tissue paper to the sashes so that the movement of air into the hood becomes visible. Neither the ventilation fan switch being in the “on” position nor the sound of the fan motor operating are assumed to be the same as the hood drawing air.  Follow-on questions should cover: o Actions to be taken if the first operator encounters a component with a Danger Tag attached; o Actions to be taken if the independent verifier finds a component mispositioned or is not certain about the component’s position; and o Actions to be taken if the independent verifier believes the only way to verify a throttle valve position is to watch the first operator position the valve. * An underlying principle of independent verification is that anyone may make a mistake; this also means that any inconsistency identified by the verifier may be the verifier's mistake. As a result, the verifier should not change the position or status of a component to correct a perceived inconsistency. Whenever an inconsistency is discovered, the verifier immediately stops and notifies their immediate supervisor. 4. Interviews The following personnel may be interviewed during the assessment:  Operators who are qualified to conduct IV;  Authorized LOTO operators;  Shift supervisors or foremen;  Facility managers and operations/program supervisors;  Manager responsible for the IV program; and 49     DOE‐HDBK‐1214‐2014  Training department personnel. Some example Interview questions for managers responsible for the IV program and procedures include:  How did you select the structures, systems, components, operations, and programs requiring IV?

 How did you determine what situations require IV?  If direct local position checks are not possible, how do you select an alternate method for verification? Do you use surveillance checks to show component position?  How do you maintain the IV program?  How do you assess the IV program? Some example Interview questions for independent verifiers include:  How were you trained to conduct an IV?  When are you required to conduct IVs?  How do you perform an IV? What are the preferred methods for verifying component position?  How do you independently verify a throttle valve position?  How are you evaluated on conducting IVs? Some example Interview questions for managers and supervisors include:  Is there an approved listing of all components and situations that require IV?  Do the facility- and program-specific procedures adequately identify when IV is required? Do the procedures give adequate guidance to operators on how to verify the position of all components requiring IV?  Do you use concurrent dual verifications? If so, what are the requirements?  How do you evaluate operator-conducted IVs? How often do you evaluate?  Do you periodically verify critical components during operation which would not normally need a second check?  How do you train operators in IV? How are operators trained in techniques appropriate to each facility’s equipment? K. Observing/Assessing Logkeeping 1. Discussion The goal of assessing Logkeeping is to verify the process for thorough, accurate and timely recording of equipment information for performance analysis and trend detection. Logs provide a consistent method for transferring information between personnel and shifts (turnover), and serve as an accurate history of facility, program and process operations and activities. Review pertinent portions of Part I, Performance-Based Assessment Overview, such as training questions and general practices. Portions of these guidelines may be incorporated into the assessment. 50     DOE‐HDBK‐1214‐2014 1.1 Related Conduct of Operations Topical Areas Document reviews, field observations and interviews conducted to assess logkeeping will normally provide leads in other topical areas. Therefore, the assessment of this area can be combined with portions of other areas such as Shift Routines and Operating Practices, On- shift Training, Control of Equipment and System Status, and Turnover and Assumption of Responsibilities. 1.2 Conduct of Operations for Logkeeping includes:  Use of narrative logs for key positions;  Prompt and accurate recording of information;  Defined type, scope and format for log entries;  Methods for correcting log entries;  Periodic supervisory reviews; and  Document retention. 2. Document Reviews A list of documents to be reviewed during the assessment may include:  Logkeeping procedures;  Shift turnover checklists;  Pre-operational checklists;  Shift supervisor and operator logs;  Round/tour inspection sheets;  Radiological surveys;  LOTO permits; and  Equipment deficiency logs. Logkeeping procedures should provide written direction on the format for log entries, including legible, permanent, smear-proof entries capable of machine copying, and written direction on electronic log entries (if used). Procedures should also ensure that narrative logs are provided for key positions, including:  Operations supervisor and control area operators;

 Stations staffed part-time (provides continuity and information pass-down); and  Narrative sections on tour/round sheets when a separate narrative log is not included. Logs may be either hard-copy or electronic. Management should provide written direction on information to be recorded in each log, including the following elements to be recorded in at least one log (not necessarily all in the same log):  Facility mode changes;  Criticalities and criticality information (for reactors or critical experiments); 51     DOE‐HDBK‐1214‐2014  Abnormal facility configurations;  Status changes of safety-related or other major equipment;  Occurrence of reportable events;  Starting and completing surveillance tests;  Entering and exiting TSRs, such as Limiting Conditions for Operations;  Security incidents;  Out-of-specification readings or results;  Shift reliefs; and  Significant information concerning emergencies, abnormal, or unexpected events. Logkeeping procedures should have specific requirements for:  How to make late entries, including noting the actual time late entries are made and prohibiting rewriting logs to make entries appear timely.  How to correct log entries. A widely-accepted industry standard is to make a single lineout through the incorrect entry (without obscuring it) and writing the correct entry in a nearby space, with the date and initials of the person making the correction.  Periodic supervisory review of logs for accuracy, completeness, timeliness, trends, and conformance with management direction. As a minimum, these reviews should include control area logs and operating stations outside the control area.  Keeping logs available for operator review after return from periods of absence.  Storing and preserving logs for the expected life of the facility (or as directed by DOE and National Archives and Records Administration). 3. Field Observations The following situations are good candidates for conducting observations:  Shift turnover;  Mode changes;  Control area activities;  Pre-operational checks;  Inspection and maintenance activities, particularly those that involve LOTO, alignments and/or post-testing; and  Operator/utility tours. For example, while observing a shift turnover the following items might be observed:  Operator and supervisor review of logs and panels;  Initial shift log entries;  Consistency in the use of logs and information recorded between shift positions; and  Pre-operational check lists. 52     DOE‐HDBK‐1214‐2014 Assessors should also conduct facility walkthroughs or tours with roving operators. These provide excellent opportunities to assess logs maintained by operators, maintenance/utility personnel, and shift supervisors. While touring, the following methodology may be used:  Observe the thoroughness, accuracy, timeliness, and legibility of log entries;  Look for the consistency of entry formats in all logs;  Look for clear, concise and understandable narratives;  Note how corrections are made to log entries; and  Look for evidence of periodic reviews of logs by supervisors. Once the observations have been completed, compare what was observed to the requirements contained in facility policies and procedures to identify any apparent deviations from requirements. Follow up on these leads using additional observations, interviews and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist.

4. Interviews The following personnel may be interviewed to assess logkeeping:  Supervisors and operators;  Maintenance/utility department personnel;  Facility managers and operations/program supervisors; and  Radiological control technicians. Some example Interview questions for shift supervisors include:  What types of information do you record in your log? What guidance have you been given on the type, scope and format of log entries?  How do you correct log errors? Is this method used by all shift positions?  Do you review any logs maintained by other shift positions? If so, how do you conduct and document these reviews? Are logs reviewed by facility management? How are these reviews conducted?  Where are completed logs kept? Describe the process used to retrieve completed logs. L. Observing/Assessing Turnover and Assumption of Responsibilities 1. Discussion The goal of assessing Turnover and Assumption of Responsibilities is to verify accurate transfer of information and responsibilities during shift/operator relief which, in turn, helps ensure continued safe operation. Proper turnover presents on-coming operators and supervisors with the current, overall status of the facility and knowledge of upcoming/planned activities. The turnover process should include a comprehensive review of appropriate written and visual information, be guided by a checklist, and complemented by a discussion between the off-going and on-coming shift personnel. Although the guidelines in this topical area specifically address 53     DOE‐HDBK‐1214‐2014 operations turnover at facilities that operate 24 hours a day, facility management at facilities that operate should follow the spirit and intent of the guidance on a single-shift basis. Review pertinent portions of Part I, Performance-Based Assessment Overview, such as general practices. Portions of these guidelines may be incorporated into the assessment. 1.1 Related Conduct of Operations Topical Areas Document reviews, field observations and interviews conducted to assess turnovers will normally provide leads in other topical areas. Therefore, the assessment of this area can be combined with portions of other areas such as Shift Routines and Operating Practices, Communications, Logkeeping, Required Reading and Timely Instructions/Orders. 1.2 Conduct of Operations for Turnover includes:  A formal turnover process is established for all key positions;  Turnovers result in the safe and effective transfer of equipment status and in-progress or planned activities from one shift/workgroup to the next; and  A process is established for reliefs that occur during a shift. 2. Document Reviews Turnover procedures should contain provisions for documenting a review of checklists or other documents that record key information appropriate for the position (either operational or supervisory), such as:  Facility operating mode and status;  Key process parameters;  Key tank or vessel levels;  Status of safety equipment;  Operational limits in effect;  TSRs in effect, such as Limiting Conditions for Operations, and the status of all required actions;  Any procedures or processes, either standard or temporary, in progress;  Changes in radiological or hazardous material conditions;  Waste management status;  Required samples or analyses; and  Upcoming or in-progress maintenance, testing, or evolutions. A list of documents to be reviewed during the assessment may include:

 Turnover process/procedures;  Shift/operator turnover checklists; and  Operating logs and round/tour inspection sheets. Turnover procedures should also contain provisions for conducting supervisor or operator reliefs during shifts. These turnovers may include a less exhaustive process than the regular shift 54     DOE‐HDBK‐1214‐2014 change as long as the oncoming person is at least as knowledgeable as they would be from a regular turnover. 3. Field Observations If the personnel under observation will be using a turnover checklist, take a copy of the checklist to follow along. To assess the shift turnover process at a facility, the following methodology may be used while observing the turnover between on-coming and off-going shift supervisors:  The on-coming supervisor should conduct document/log reviews and control panel/display walkdowns prior to the turnover in order to determine facility status, alarms, lineups, and equipment configuration: o Document/log reviews normally extend back the shorter of 24 hours or their last shift. o For control areas, the on-coming and off-going supervisors may jointly walkdown the control panels and displays.  The supervisors should then discuss turnover documentation (e.g., the turnover checklist) and any up-coming or planned activities and clarify any questions. Once the on-coming supervisor is ready, she should formally state that she is assuming responsibility and make a narrative log entry to that effect.  Pay close attention to the information passed between the supervisors during the turnover for later comparison with observations during the shift.  Supervisors should conduct briefings, as needed, for their on-coming operators and appropriate support personnel (e.g., utility and maintenance personnel): o The briefing should discuss items such as facility and equipment status, abnormalities and TSR conditions, up-coming operations (e.g., work packages and TSR surveillances), lessons learned (may be required reading), and currently effective timely orders. o Observe the process used and the information exchanged, and compare the information disseminated at the briefing with the information that was passed between the supervisors during their turnover.  Observe the supervisor as they performs their duties. Look for any activities or operations in progress that were not discussed or briefed at turnover. Observe the status of equipment and systems to identify any differences between what was briefed and what was observed.  If a relief during the shift is observed, verify that the relieving supervisor/operator is at least as knowledgeable as he would be from a regular turnover  At the end of the shift, observe the turnover process between supervisors. 4. Interviews The following personnel may be interviewed to assess turnover and assumption of responsibilities:  Shift supervisors and operators;  Facility and operations/program managers; and 55     DOE‐HDBK‐1214‐2014  Maintenance/utility department personnel. Some example Interview questions for operators include:  What guidance have you received from the shift supervisor concerning how to conduct shift turnover? Do you feel that the process used to at turnover supports your needs?  How do you determine that your relief is fit for duty?  How do you determine what information should be passed at turnover to the on-coming operator? Are you required to use a turnover checklist to aid the process? If so, what type of information is recorded and how is this information used during the shift?

 What documents do you review prior to or during shift turnover? How does the turnover checklist aid in the document review process?  Are there any control panels that you walkdown during shift turnover? If so, how do you do this and what information does it provide you? Do you normally do the walkdown with the off-going operator?  Do you conduct a verbal turnover to an on-coming operator? What type of information do you relay?  Is a shift crew briefing conducted as part of the turnover process? When is it conducted? What type of information is passed on during the brief? Who attends the briefing?  How do you conduct turnover for reliefs that occur during the shift? What type of information do you relay to your relief? What guidance have you received from facility management concerning reliefs during the shift? M. Observing/Assessing Control of Interrelated Processes 1. Discussion The goal of assessing Control of Interrelated Processes is to verify that operating practices are in place to ensure that interrelated processes do not adversely affect safety or operations. Interrelated processes are processes or activities that can affect operations, but are under the control of persons other than the affected operators. Interrelated processes are most often shared support systems, and management should define responsibilities for their control. Establishing lines of communication between operating personnel and process support personnel are an effective means to promote coordination of activities. Examples of the most common shared support systems are:  Electrical distribution (including backup systems such as uninterruptible power supplies and batteries);  Steam and condensate return;  Natural gas;  Heating, ventilation and cooling;  Compressed air;  Cooling water;  Potable water; 56     DOE‐HDBK‐1214‐2014  Hydraulics;  Vacuum and contaminated vacuum;  Wastewater collection (including drains and sumps);  Fire suppression and alarm;  Lightning and severe weather detection;  Seismic; and  Public address. Interrelated processes are not limited to physical connections, such as a shared cooling water system directly connected to pumps in several other systems. A process or activity may require that several independent (not directly connected) systems be operating before that process/activity can begin (or continue). An example is an activity that requires that the building wet-pipe fire suppression, the building air handling units, the site steam system, and the site compressed air system all be operable in order to begin the activity. The activity operators depend on these systems but do not control their operation. Examples of other processes/activities that can affect operations and are not under operations control are special tests (e.g., air or radiation surveys, pressure or vacuum tests, and chemistry tests), exercises and drills, tooling and property audits, instrument test, alignment and calibration, material movements, and security activities. Training Operators and support personnel (e.g., maintenance, utilities, and technical) are trained on the systems/components they operate or work on. This training should include knowing the relationship between the systems/components they are responsible for and all the other systems that depend on the operability of their systems/components. In addition, they should know all the safety implications associated with working on their systems/components.

This integrated knowledge enhances their ability to understand trends, problems, or potential problems. Such knowledge increases their ability to initiate corrective action, or to inform the process support personnel of the situation, and enables them to understand how their actions may affect the unique process. Integrated knowledge is developed through training, experience, and communication. Operator and support personnel knowledge of these system interrelationships is vital. Understanding the relationship between their system and all systems that rely on their system allows them to anticipate problems; they know that if they are planning to perform maintenance on their system then they should coordinate with those responsible for the systems that will be affected. Operators will know how the loss of one of these support systems affects their systems and can take actions accordingly. Operations personnel should be trained to monitor facility conditions (including trends in operations), analyze available information, and diagnose problems. This will allow them to:  Anticipate the response of interacting systems or unique processes when changes are made to a process under their control; 57     DOE‐HDBK‐1214‐2014  Anticipate changes in processes under their control in response to changes in processes controlled by others;  Evaluate degrading conditions and take appropriate action to prevent potential consequences; and  Recognize the symptoms of abnormal and emergency conditions and prevent or minimize the consequences. Open Communications This leads to the second vital requirement: effective communications between operating and support personnel. In most cases, direct communication between operators and support personnel is all that is necessary to assure the operator is aware of and considers the potential effects on their systems and equipment. However, if operations personnel do not know that a process or activity that will affect systems and/or equipment under their control is scheduled or has developed a problem, they may not notice the effect on their systems until it is too late. Open lines of communication enable work groups to coordinate interrelated activities in pre-job planning sessions and briefings. Effective communication between operators and support personnel is an essential aspect of teamwork, and is vitally important to safe and reliable operations; operators and support personnel should be able to function effectively as a team. Team deficiencies such as poor communication are considered significant during normal operations, but may become major obstacles to making proper decisions and initiating appropriate corrective actions during abnormal and emergency conditions. 1.1 Related Conduct of Operations Topical Areas Document reviews, field observations and interviews conducted to assess control of interrelated processes will normally provide leads in other topical areas. Therefore, the assessment of this area can be combined with portions of other areas such as Shift Routines and Operating Practices, Control Area Activities, Communications, Control of Equipment and System Status and Operator Aids. 1.2 Conduct of Operations for Control of Interrelated Processes includes:  Defining responsibilities and lines of communication with respect to the control of interrelated processes, and  Training personnel to understand and control interrelated processes

Review pertinent portions of Part I, Performance-Based Assessment Overview, such as training and general practices. Portions of these guidelines may be incorporated into the assessment. 2. Document Reviews The list of documents could include:  Administrative policies or procedures that detail operator and support personnel responsibilities (e.g., trend analysis, interpretation of out-of-specification parameters, and timely corrective actions); 58     DOE‐HDBK‐1214‐2014  Administrative policies that detail lines of communication between operators and support personnel;  Administrative policies that emphasize teamwork between work groups;  Administrative policies and procedures discussing work planning and scheduling, system impairments, shift and pre-job briefings, and proper work authorization;  Master maintenance schedule;  Surveillance/In-service Inspection schedule;  Key shift position logs and round sheets; and  Facility- and process-specific training and qualification records. 3. Field Observations The most likely opportunity to observe how operators and support personnel control interrelated processes will come from scheduled maintenance or surveillances. The Plan of the Day or Plan of the Week meeting should be attended, or the documents reviewed, to determine interrelated process events to observe. If the meeting is attended, assessors should determine if all interrelated process owners/managers are represented, and assess the quality of the information exchange. Assessors should attend the daily or shift briefing in the selected facility to verify that what has been planned has been communicated to the appropriate operations personnel. The briefing should discuss how the planned activity will affect operations, what precautions/actions should be taken in anticipation of the activity, and how communications will be established and maintained, and what post-activity testing is required. Actions taken to comply with TSRs are particularly important. Loss or unavailability of a safety support system/equipment will cause the affected safety system/equipment to be declared inoperable and require operators to enter a Limiting Conditions for Operation (LCO). The briefing should also cover when and how they expect to exit from the LCO. The following are other possible sources of field observations:  Planned electrical outages;  Planned system outages;  System startup or shutdown;  Planned exercises or drills; and  Special tests. The following provide the opportunity for observations if they occur during observations in the facilities:  A work package arrives;  System/equipment troubleshooting is required;  System/equipment isolation (LOTO) is required;  Operators notice an adverse trend on their instruments; 59     DOE‐HDBK‐1214‐2014  Roving operator reports an adverse trend or out-of-specification reading;  Alarms are received or reported;  Unannounced exercise or drill;  Loss of site or facility electrical power; and  Loss of site or facility system. Assessors should also observe process support personnel and assess how well actions and results are communicated back to the operators in time to allow meaningful process control. If it can be accomplished without unduly distracting operators or support personnel, assessors can posit situations or indications and ask them what they would do. These questions may be followed up during interviews.

Once field observations are complete, compare what was observed to the requirements contained in facility policies and procedures to identify any apparent deviations from requirements. Any deviations identified by field observations can be followed up on using additional observations, interviews, and document reviews to confirm or disprove them and determine if there programmatic breakdowns or if other problems exist. 4. Interviews The following personnel may be subject to interviews when assessing this topical area:  Facility/program supervisors and operators;  Maintenance, utility and other support personnel;  Support personnel supervisors/foremen; and  Facility managers and operations/program managers. The primary focus of interviews should be to determine operators’ level of technical knowledge and experience working with interrelated processes, and how aware they are of the need for open communications (e.g., teamwork). For example, the following questions may be asked:  What are the important support systems for your facility/process?  How do you know if personnel are planning to work on a support system during your shift?  What support systems, if lost, would cause one or more of your safety systems to be declared inoperable?  Which support systems could be lost or their performance degraded and you would have no way of knowing (e.g., no indicators or alarms available to warn you)?  What are the requirements for a safety support system’s operability in the TSRs?  What indicators and/or alarms do you monitor while personnel are working on a safety support system?  How do you communicate and coordinate activities with support personnel?  What training have you received on interrelated processes? Teamwork? 60     DOE‐HDBK‐1214‐2014 Some example Interview questions for process support personnel include:  What are the safety implications associated with working on your systems?  What other systems or equipment are dependent upon your systems’ operability?  Who do you inform when you are performing your work? What authorization do you need to begin work?  How do you establish and maintain communications with affected operators when you are working on a support system?  What training have you received on interrelated processes? Teamwork? Some example Interview questions for shift supervisors, facility managers, and operations/program supervisors include:  What are the important support systems for your facility/process?  What support systems, if lost, would cause one or more of your safety systems to be declared inoperable?  Which support systems could be lost or their performance degraded and you would have no way of knowing (e.g., no indicators or alarms available to warn you)?  What are the requirements for a safety support system’s operability in the TSRs?  What indicators and/or alarms do your operators monitor while personnel are working on a safety support system?  How do you know if personnel are planning to work on a support system during your shift?  How do you brief your operators about upcoming work on an important support system?  How do you and/or your operators communicate and coordinate activities with support personnel?  Have you had any problems with support systems? Have you ever lost a safety support system that caused one of you safety systems to be declared inoperable? What actions did you take? How could the problem have been prevented?

 Have you had any problems with communications during work on a support system? How could the problem have been prevented?  How do you promote teamwork between work groups?  What training have you received on interrelated processes? Teamwork? N. Observing/Assessing Required Reading 1. Discussion The goal of assessing the Required Reading (RR) program to verify its function of keeping workers updated on current and recent events, changes, lessons learned and other important information that is related to their job assignments. Because RR is focused on information related to specific job assignments, it is normally administered at the department or division level (e.g., Operations, Manufacturing, Maintenance, 61     DOE‐HDBK‐1214‐2014 Utilities, and Radiation Safety). Therefore it is common for each department/division to have its own RR policy and procedure. Some examples of what should be included in the RR program are:  Lessons learned and operating experiences;  Policy changes;  Procedure or equipment change summaries (not the complete, revised procedure);  Company memos that are of concern to employees; and  Training announcements. The Conduct of Operations Order is not very specific about what exactly should be included in the program, thus giving management broad latitude in deciding what to include as RR. As important as it is to assess what the program accomplishes, it is almost equally important to verify that the program is not being misused. Assessors should be sensitive to troubling signals such as:  Large and unfocused RR files: Large volumes of reading not related to job assignments will dilute the significance of important information and most likely lead to cursory reading of all assignments.  RR being used as a substitute for training: Because formal training takes time to develop and deliver, it is tempting to just have workers read and sign a required reading document instead.  RR that tries to address complex and/or complicated subjects: Presenting long and involved treatises or technical papers is not a substitute for training and discussion.  RR presented in a busy atmosphere: Reading important information requires concentration, so it should not be performed in noisy or stressful situations. Review pertinent portions of Part I, Performance-Based Assessment Overview, such as general practices. Portions of these guidelines may be incorporated into the assessment. 1.1 Related Conduct of Operations Topical Areas Document reviews, field observations and interviews conducted to assess RR can be combined with portions of Turnover and Assumption of Responsibilities and Timely Instructions/Orders. 1.2 Conduct of Operations for Required Reading includes:  Identification of material to be distributed via RR;  Tailoring RR to specific workers or groups of workers; and  Documenting the timely completion of RR. 2. Document Reviews A list of documents to be reviewed during the assessment may include:  Department RR administrative procedure or policy (may be included in a lessons learned program and/or as part of Shift/Crew Briefing/Meeting procedures and checklists); 62     DOE‐HDBK‐1214‐2014  RR assignment sheets (there could be a cover sheet for each document or a centralized assignment sheet in the front of a binder or equivalent);  RR acknowledgement/certification/completion sheets;  RR file indexes;  Crew/shift briefing/meeting checklists; and

 Retained documentation and dead file (if required). From the administrative procedures or policies, assessors should determine:  Who has been given responsibility for administering the RR program;  How information is screened to ensure appropriate content and the target audience;  How due dates are determined, and how RR with due dates that require completion before beginning work or resuming a shift is being completed on time;  How management reviews the RR program; and  How RR is documented, tracked to completion, and retained or disposed of. Assessors will verify that these actions are actually performed during subsequent document reviews and interviews. Assessors should select a period of time (e.g., one month) and review RR assignments for each department during that time. Assessors should look for:  Entries are organized (e.g., chronological order) and have due dates assigned;  All appropriate personnel are included;  RR is acknowledged/completed by the due date; and  Completed assignments are removed from the RR binder (or equivalent). For this topical area, deviations from expectations can be normally derived from document reviews and followed up with interviews. 3. Field Observations Observations are primarily used to confirm requirements noted in document reviews and interviews. Crew briefings/meetings should be observed to see how RR is targeted and presented. Although the emphasis is on operating personnel, assessors should also assess the RR program in other departments such as Maintenance and Utilities. For example, while observing a crew briefing the following items might be observed:  Time allowed for workers to complete RR;  Any distractions that occurred that could impair the effectiveness of the workers’ ability to absorb the material; and  Support for RR, such as discussions with a supervisor, SME or training personnel. 63     DOE‐HDBK‐1214‐2014 4. Interviews The following personnel may be interviewed to assess required reading:  Personnel participating in several different department’s RR program;  Shift supervisors and foremen;  Facility and operations/program managers; and  Departmental RR administrator. Follow-on interviews with workers may be used to test relevance, understanding and retention:  In general, do RR assignments assist you in your job?  How often do you review the RR file for assignments? Does the file document removals?  Briefly discuss the two most recent reading assignments.  Did the RR accomplish its goal, or would the material be better presented in a formal training setting?  How do you complete RR assignments if you have been absent from work? Are the files easily accessible?  Do you think the program is valuable? Why or why not? Some example Interview questions for managers and supervisors include:  What criteria are used to determine the information included in your department’s RR binder?  Who is responsible for ensuring that RR is completed by the assigned due date?  How do you ensure 100% completion of reading assignments (particularly for affected personnel who have been absent from work)?  How do you get items you want included in the RR program?  How often do you review your RR program?  How do you determine the effectiveness of RR assignments?  Who is responsible for removing completed reading?  Who is responsible for the final disposition of RR documentation? O. Observing/Assessing Timely Instructions/Orders

1. Discussion The goal of assessing Timely Instructions/Orders (sometimes referred to as Standing or Shift orders) is to verify that it functions to provide workers timely and essential written direction and guidance related to their jobs. Timely orders may not conflict with approved procedures, and the directives for timely orders should include provisions to prevent their use as a substitute for administrative or operational procedure revisions. Like required reading, timely orders are normally administered at the department or division level (e.g., Operations, Manufacturing, Maintenance, Utilities, and Radiation Safety). 64     DOE‐HDBK‐1214‐2014 Timely orders remain in effect until canceled or superseded by the originator. The duration of timely orders may vary, and should be segregated into daily and long-term categories:  Long-term orders (e.g., > 90 days) should be kept current. The originator should periodically review them (e.g., semi-annually or annually) to assure the information is still applicable, and they should be tracked for replacement or incorporation into permanent, approved procedures. Long-term orders may contain: o Specific clarifications, amplifications and directions to be followed in the performance of approved procedures, and o Policy or instructions for matters not prescribed by procedures, such as special tests or evolutions. In these cases the orders should specify how to perform the tests or evolutions and any limitations.  Daily and other short-term orders should have an expiration date (e.g., 24 hours or 30 days), and have provisions for documenting personnel review every shift for daily items (including those that are delayed or remain in force longer than a day). Short-term orders may contain: o Announcements of administrative items, policies, procedure changes or equipment design changes not considered necessary or appropriate for entry into the RR program; o Operating experience information; o Instructions or notifications regarding maintenance and test activities not directly related to facility equipment or other specific evolutions; and o Other information necessary to keep operating personnel aware of current facility activities. Review pertinent portions of Part I, Performance-Based Assessment Overview, such as general practices. Portions of these guidelines may be incorporated into the assessment. 1.1 Related Conduct of Operations Topical Areas Document reviews, field observations and interviews conducted to assess timely instructions/orders can be combined with Turnover and Assumption of Responsibilities and Required Reading. 1.2 Conduct of Operations for Timely Instructions/Orders includes:  Approved and appropriate use of timely orders, and  Configuration control of timely orders. 2. Document Reviews A list of documents to be reviewed during the assessment may include:  Department timely orders procedure or policy (may be associated with the required reading program and/or as part of Shift/Crew Briefing/Meeting procedures and checklists);  Summary lists or tables of timely orders;  Timely orders binders (or equivalent); and 65     DOE‐HDBK‐1214‐2014  Crew/shift briefing/meeting checklists. Review of the documents listed above should help determine how the program is administered. From the administrative procedure for timely orders, an assessor can usually identify:  Who is designated review and approval authority;

 The process for distribution to appropriate personnel, including documenting receipt of timely orders;  The process for removing or canceling superseded, expired or outdated orders; and  Requirements for periodic management reviews to ensure only appropriate and current items are distributed and that appropriate personnel review them within time limits. Assessors will verify that these actions are actually performed during subsequent document reviews and interviews. Assessors should select a period of time (e.g., six months) and review timely orders issued by each department during that time. Assessors should ensure that:  Expiration and review dates are not exceeded;  Superseded, expired or outdated orders are removed; and  Timely orders are not being used to modify or replace approved procedures. For long- term orders, a procedure change should be initiated to incorporate the information in the timely order 3. Field Observations Timely orders observations are primarily concerned with assessing how well the orders are communicated, which usually occurs during morning meetings, crew or shift briefings, and/or during turnovers. Some items to look for include:  How supervisors ensure operators receive and understand the information contained in the timely orders;  How operators implement timely orders; and  Supervisors ensure timely orders have not exceeded their expiration date. 4. Interviews The following personnel may be interviewed to assess timely instructions/orders:  Operators;  Supervisors and foremen;  Facility and operations/program managers/supervisors; and  Designated Timely Instructions/Orders administrator. Some example Interview questions for operators include:  How often are you required to review timely orders?  Are the orders pertinent and understandable?  Do you think the program is useful? Why? 66     DOE‐HDBK‐1214‐2014 Some example Interview questions for managers and supervisors include:  How do you ensure that all operators read timely orders?  How often do you review timely orders?  Are you permitted to issue a timely order to modify a procedure? If not, who is?  How are you notified of a new instruction or order? P. Observing/Assessing Technical Procedures 1. Discussion The goal of assessing Technical Procedures is to verify that accurate, understandable written technical procedures are developed and maintained to ensure safe and effective facility, program and equipment operation. Operator use of procedures and management attention to the procedures program are also part of assessing this area. Adherence to technical procedures is a cornerstone of both formality of operations and integrated safety management. Because of this, most sites require all site personnel (including on-site subcontractors) to strictly comply with all approved procedures. Procedure adherence means performing actions or activities as specified in the procedure to produce the expected result. It does not imply that every action to be taken by the user is stated in the procedure (for instance, the procedure may require a gage reading without also including steps to mount an access platform and get down.), or condone malicious compliance, which is following a procedure knowing that unwanted, unlawful or unexpected consequences may result. Technical Procedures provide instructions that facilitate performance of operational activities without direct supervision in a safe, environmentally sound, and efficient manner and within the design basis of the facility or operation.

Although technical procedures are developed and maintained at the department/division level, the development, review, authorization, change control, and use of technical procedures are usually specified at the site level. This approach facilitates consistent format, use of terms, required level of detail, design conformance, and control. Personnel are expected to know their site/facility policies for using technical procedures. Terms vary, but the policies normally include several level-of-use definitions, ranging from ‘reference before performance’ for simple or repetitive tasks, to ‘reader-worker with signoffs’ for very complex, hazardous, or rare tasks. Site/Facility policies also may include techniques and situations requiring place-keeping, including signoffs, check boxes, or circle/slash (marking the next step to be done with a circle in the margin, and drawing a slash through the circle when it is completed) or other methods. Assessors should be familiar with the level of use and place- keeping requirements for any procedures they observe in use. Some general rules of procedure adherence are that personnel are to stop/suspend work and notify supervision when:  A procedure cannot be followed;  A procedure step produces an unexpected or undesirable result; and 67     DOE‐HDBK‐1214‐2014  A procedure has the potential for an unexpected or undesirable result. Typographical errors in procedures, such as misspellings, word omissions, and obvious misnumbering that do not cause ambiguity are not cause for suspension of work activities; the continued use of a procedure containing a typo is not considered an adherence violation. Technical procedures for emergency and abnormal situations should be clearly distinguishable from normal operating procedures, and are usually divided into two sections: (1) Immediate actions, and (2) Supplementary actions. Management normally expects operators to have the immediate actions memorized. After the immediate actions have been taken, operators should refer to the emergency procedure to verify that all immediate actions were completed correctly, and then perform the supplementary actions listed in the procedure. (Note: Alarm/Annunciator response procedures are similar but designed primarily for use by control panel operators.) When assessing Technical Procedures, review pertinent portions of Part I, Performance-Based Assessment Overview, such as training questions and general practices. Portions of these guidelines may be incorporated into the assessment. 1.1 Related Conduct of Operations Topical Areas Document reviews, field observations and interviews conducted to assess technical procedures will normally provide leads in other topical areas. Therefore, the assessment of this area can be combined with portions of other areas such as O&A, Shift Routines and Operating Procedures, On-shift Training, Investigation of Abnormal Events, Conditions and Trends, Control of Equipment and System Status and Control of Interrelated Processes. 1.2 Conduct of Operations for Technical Procedures includes:  Personnel adhere to procedures;  Procedure development and control, including procedure use requirements;  Processes for procedure change or revision;  Training on new, changed, or revised procedures; and  Availability and use of only the most current revisions of procedures. 2. Document Reviews A list of documents that can be reviewed when assessing technical procedures includes:

 The technical procedure writer’s guide;  Index/database of site/facility/program technical procedures;  Document management procedures (should include policy/procedures for creating, reviewing, approving, changing and revising technical procedures); and  The procedure that addresses the site’s policy on the use of technical procedures (e.g., procedure use and adherence). The process for technical procedure development should dictate a consistent format and terminology. The procedure should be technically accurate and the language should be clear, definitions explained, and the level of detail appropriate for the operators’ skill, experience, and 68     DOE‐HDBK‐1214‐2014 training. No more than one action should be stated in any one step. Some other attributes to look for during the review are that:  Scope and Applicability are clear and apparent;  Prerequisites and initial conditions are clearly specified;  Headings for Cautions, Warnings and Notes should appear on the same page as the step(s) to which they are applicable; they should be clear, not contain actions, and precede the step(s) to which they are applicable;  Tools, equipment, and materials are specified and procedures provide measures to document their calibration or condition before use;  Critical steps include signature/initial/checkoff/stamp blocks, and there should be only one action per block;  Hold points requiring independent verification or approval are clearly indicated;  Procedures for multiple equipment trains are clearly distinguishable from each other;  Instructions for transferring between procedures are clear; and  Referenced documents are clearly identified. Where applicable, procedures should ensure that:  Instrument readings and tolerances are specified and conform to instrument scales or readability;  Procedures contain explicit parameters and do not require mental arithmetic to determine acceptability; calculations are clearly explained and there is space to record them;  Step sequence conforms to the normal operational sequence for which the procedure is developed;  Procedures reflect human factors considerations (e.g., procedure callouts exactly matching equipment labels, units in procedures match instrument markings, charts and graphs easily read, and important steps or information highlighted);  Emergency procedures provide guidance for both single and multiple casualties;  When procedures use or refer to other procedures or steps, the referenced documents are clearly identified to prevent confusion when transferring between them; and  Procedures specify the restoration or shutdown steps for equipment following tests or other operations. Changes and revisions to technical procedures should be controlled and approved. Some attributes to look for are:  Directives include a documented process for review and approval of revisions and changes. Directives may also use only a revision process or may use an electronic publishing process. In all cases, configuration control should be maintained.  Procedure changes intended for more than one-time use are documented in a location readily available for operator reference and noted in Timely Orders/Instructions and/or turnover documents. 69     DOE‐HDBK‐1214‐2014  Directives contain provisions for initiation of changes or revisions if procedure problems are found, including provisions for emergent changes or revisions necessary to proceed with operations when a procedure is faulty.

 Directives contain provisions for initiating a procedure revision when changes remain in effect for extended periods (e.g., more than six months) or when several changes have accumulated (e.g., more than five).  Directives contain provisions for including all outstanding changes in any procedure’s revision.  Directives include provisions for implementing revisions for permanent equipment modifications or replacements, and implementing changes for temporary equipment modifications.  Directives include provisions to review procedure development records of the reason for key steps to prevent inadvertent deletion or change.  Directives include provisions to use walkthroughs (procedure execution with actual or simulated operation of components by subject matter experts) to validate procedure changes and revisions. Directives define when actual operation of equipment during a walkthrough is authorized.  Provisions include important support personnel in the review of procedures affecting safety-related equipment or emergency response, such as safety and emergency managers.  Minor changes are defined and the process for making and approving minor changes specified. This process should be easy to use and be accomplished in a short period of time. Personnel are trained when technical procedures are issued, changed or revised. The procedure process should include provisions for communicating important procedure changes and revisions to personnel through approved methods such:  Formal training: If the changes are significant, personnel may have to be qualified and/or certified to the new revision or receive partial training to clarify and emphasize the changes.  Informal training: Changes may be explained using various informal methods such as Timely Orders/Instructions, RR, turnover or shift briefing, e-mail or other appropriate method.  Once the training is accomplished the affected personnel training files should be updated. Approved procedures should be maintained. Attributes to look for are that:  Procedures are periodically reviewed for technical accuracy and human factors;  The frequency of periodic procedure reviews is specified, and considers the complexity and maturity of operations and facility life cycle;  Procedures are reviewed after a significant occurrence, either human error or equipment upset;  Procedure reviews include comparison to source documents to verify accuracy; and  Procedure reviews include validation walkthroughs. 70     DOE‐HDBK‐1214‐2014 Some common procedure problem areas that can lead to operator errors are listed below:  Cautions, Warnings or Notes appear after the steps to which they apply; these statements should immediately precede the steps or text to which they apply.  Cautions/Warnings/Notes and Headings appear on a different page than the one containing the steps or text to which they apply.  Cautions/Warnings/Notes contain actions.  A procedural step contains more than one action.  Referencing information that appears in other procedures or documents (e.g., calculations, operator aids, tables, performance curves, or logic diagrams) when that information can be incorporated into the procedure itself.  Readability level (i.e., words, sentence and paragraph structure) is not consistent with the educational level of the workers who routinely use the procedure.  Use of technical terms, acronyms and jargon that are not in common usage by the workers using the procedure.

 Using complex or rarely used words when simple, common words will do.  Level of detail is above that (i.e., not easily understood) of the workers who routinely use the procedure.  Steps are not presented in a logical sequence or do not reflect the sequence used in the field.  Required actions and acceptance criteria are not clearly stated.  Acceptance criteria do not correspond to instrument scales being utilized (e.g., the acceptance criteria is listed in liters per second but the instrument used to provide the reading is scaled in gallons per minute).  Mental arithmetic (e.g., calculations) is required to solve for parameters that are then used to determine acceptability.  Steps are arranged without considering “big picture” concepts such as improving personal safety, minimizing radiation exposure or minimizing process down time.  Using one procedure for a process where two or more procedures should have been used. While observing procedure use, determine the level-of-use and revision number of the procedures in use to determine whether the latest approved revisions of procedures are being used and that they are being used properly. 3. Field Observations The most effective method of assessing technical procedures is to observe as many activities as possible that require the use of procedures. The facility Plan of the Day or Plan of the Week should be reviewed to determine the time and place for observing operations (or exercises/drills, if possible). When it is practical, assessors should bring a copy of the current approved procedure with them so they can assess the quality of the procedure and how well workers follow it. 71     DOE‐HDBK‐1214‐2014 Procedures should be current, properly approved, readily available, and should work as written. Workers should follow the procedures and, when they cannot be executed as written or when they have questions about the procedure, should stop work and notify supervision. The following can normally be assessed during the observation of procedures:  Availability of current approved procedures: o Controlled copies of all necessary procedures (e.g., normal operating, emergency, abnormal situations, and alarm/annunciator response) are available at control areas and other appropriate locations; o Personnel know how to obtain copies of procedures (electronic or hard-copy) and know how to ensure they have the most current approved copy; and o Working copies of procedures are verified against controlled copies during evolutions.  Workers/supervisors follow procedure use requirements: o Operators are trained in procedure use requirements and management oversight reinforces the expectations; o Operators know the level-of-use requirements for critical, specific, reference and general use procedures, and understand terminology such as ‘per’, ‘reference’ and ‘as outlined in’; o Personnel know how to report deficient procedures and initiate changes or revisions to correct them, including minor changes requiring pen-and-ink changes; o Operators and supervisors have committed immediate actions to memory, and know to obtain the procedure to check proper completion of immediate actions and perform supplemental/follow-up actions; and o Workers and supervisors/managers know that, during emergency conditions, they may take necessary action to place the facility or program in a safe and stable condition without first initiating a procedure change.

 Workers adhere to procedures, and if problems are encountered or questions arise the workers stop and contact their supervisor: o Typographical errors are corrected (e.g., pen-and-ink changes), and o Changes/revisions are properly initiated.  Workers demonstrate an understanding of and adherence to prerequisites, initial conditions, hold points, cautions, warnings and notes.  Workers demonstrate proper place keeping: o Start and stop of operations, including process interruptions or crew change, and o Occasions where operators temporarily leave the base procedure to perform actions elsewhere in the same procedure or in another procedure (ensure that the user returns to the next step in the base procedure after performing the referenced actions).  Workers demonstrate proper performance of Reader-Worker Checkoff and other methods used for critical steps (e.g., signatures, initials, or stamps). While observing operations, an assessor can make a subjective determination that the workers understand the procedure(s) they are using. Proper training and experience allow workers to spot 72     DOE‐HDBK‐1214‐2014 problems with a procedure step quickly or, in many cases, before they take the next step or series of steps. They are, in effect, looking ahead in the procedure and determining if the up-coming steps can be accomplished as written. This ability reflects favorably on the procedure development, review and training processes. An effective method for conducting observations is to compare two or more different operators conducting the same procedure. This will allow an assessor to see any variations in the use of procedures by several operators. This methodology will also uncover deviations that are site- or facility-wide problems. 4. Interviews The following personnel may be interviewed to assess technical procedures:  Operators and maintenance/technical personnel;  Shift and first line supervisors;  Facility and operations/program managers; and  Qualified technical procedure writers and their supervisors. Some example Interview questions for procedure users include:  How do you obtain procedures (hard copy and electronic)? How do you ensure the procedure is the most current and approved copy?  What do you do when you find a typographical error in a procedure?  What do you do when you have a question about a procedure?  What do you do when you cannot perform a step as it is written?  What are the different level-of-use categories? What are the basic requirements for each?  How clear and understandable are the procedures you use? How often do you find mistakes in procedures (not counting typos)?  How do you ensure you are at the proper place in a procedure when you come back from lunch or a break?  Describe how the Reader-Worker method is performed.  How easy is it to make a suggestion or request a change to a procedure? Are your requests addressed quickly? If your request is denied, does management explain why? NOTE: During interviews pay close attention to how operators respond to questions about stopping work whenever they have a problem with or a question about a procedure. Operators should feel comfortable about stopping work under those conditions, and show no fear of retribution for stopping work for procedure issues or safety concerns, especially when scheduling pressure is high. Some example Interview questions for supervisors and managers include:

 Do you have input into the development or modification of technical procedures? Is the process for making comments, suggestions or change requests easy to use?  What are the rules for ‘pen-and-ink’ changes?  What actions are required when an inaccuracy is found in a procedure? 73     DOE‐HDBK‐1214‐2014  Are changes and revisions to procedures addressed as soon as possible?  Is there any circumstance where you can bypass the normal revision and review process?  How do you expect operators to use technical procedures?  How do you ensure operators are using the most current approved copy of a technical procedure? Some example Interview questions for technical procedure writers and their supervisor/manager include:  How were you selected to write technical procedures? Did you receive any training? Do you use a technical writer’s guide?  How do you determine that a new procedure is needed?  Do you keep records as you develop a procedure? Do you document the rationale for key steps in a procedure?  Do workers have input into the procedure writing and modification process?  When you begin developing a procedure, how do you ensure the level of detail is appropriate for the users?  How do you ensure headers for cautions, warnings and notes remain on the same page as the applicable step(s)?  What human factors considerations do you incorporate into procedures?  If a procedure referenced in one of your procedures is revised, how do you know it has been changed?  How do you prioritize procedure change requests?  What is the average process time for procedure changes and revisions?  How many changes can be outstanding before you are required to revise the procedure?  Do you make procedure changes for temporary equipment modifications?  How do you validate procedure changes and revisions?  How often are procedures reviewed?  Other than operator input, what events or occurrences might trigger a procedure change or revision? Q. Observing/Assessing Operator Aids 1. Discussion The goal of assessing Operator Aids is to verify they provide accurate, current and approved information to operators. Their primary purpose is to serve as a convenient reminder or a quick source of information, thus enhancing the user’s ability to perform a task. Operator aids may not substitute for or conflict with procedures (although they may supplement approved procedures). Operator aids are most useful when the way a task is performed makes it impractical to refer to the procedure, such as when the task requires both hands. 74     DOE‐HDBK‐1214‐2014 Operator aids are viewed as a convenience to the individual using them, not a requirement. Even though copies of procedures or portions of procedures may be used as operator aids, the requirement is to follow the procedures. Operator aids may be classified as permanent or temporary. Temporary aids (such as for a special test) should include an effective date or period (e.g., from 03/28/12 to 04/25/12). Operator aids are not to be used for personnel and equipment safety situations where danger tags are required. Additionally, operator aids are not to be used in lieu of posted safety signs required by other oversight agencies such as OSHA. Examples of operator aids are system/equipment operating curves, system drawings, copies of a procedure or portions of a procedure, photographs, information tags or sheets, charts, graphs, diagrams of infrequently-used equipment, or valve lineups.

Do not confuse operator aids with posted information. Posted information, such as a list of telephone numbers or codes, simply assists personnel in day-to-day affairs but does not provide guidance. Other examples of posted information include radiation, security, hazardous material and safety signs and posters, manufacturer’s labels and signs on or near equipment, commercially procured signs and posters such as First Aid or CPR posters, evacuation maps, or component labels. Some non-nuclear facilities do not have an official Operator Aid program. Where this is the case, the assessment of this topical area becomes a walkthrough of the facility to determine whether uncontrolled operator aids are being used in the facility. 1.1 Related Conduct of Operations Topical Areas Document reviews, field observations and interviews conducted to assess operator aids will normally provide leads in other topical areas. Therefore, the assessment of this area can be combined with portions of other areas such as Control of Equipment and System Status and Component Labeling. 1.2 Conduct of Operations for Operator Aids includes:  Technical evaluation and management approval;  Administrative control of installed aids, including periodic review for adequacy and correctness; and  Operator Aids do not obscure equipment or interfere with controls. Review pertinent portions of Part I, Performance-Based Assessment Overview, such as general practices. Portions of these guidelines may be incorporated into the assessment. 2. Document Reviews The administrative policy or procedure should include, as a minimum, a process for developing, approving and controlling operator aids. 75       DOE‐HDBK‐1214‐2014 Maintaining a centrally located file of all operator aids helps personnel quickly review and take appropriate action to correct, update, and remove obsolete operator aids and posted information documents, and enhances the periodic review process. Directives controlling operator aids should include provisions for maintaining a master list and copies of all approved operator aids in an appropriate centralized area. The listing and associated documentation should include:  Unique identification numbers (control numbers) for each aid;  An explanation of the need for and location(s) of each operator aid;  Management approval, with date of approval, revision and location for each aid;  A listing of the source documents (references) used to obtain the operator aid information, including revision status. This assists personnel in finding and updating the aids when source material changes. If an operator aid has no source document, that should be noted in the records; and  May include completion dates of required audits. Operator aids should be reviewe