DOE-HDBK-1208-2012 Volume II, Accident and Operational Safety Analysis, Volume II, Operational Safety Analysis Techniques

DOE-HDBK-1208-2012 July 2012 DOE HANDBOOK Accident and Operational Safety Analysis Volume II: Operational Safety Analysis Techniques U.S. Department of Energy Washington, D.C. 20585 NOT MEASUREMENT SENSITIVE DOE-HDBK-1208-2012 i ACKNOWLEDGEMENTS This Department of Energy (DOE) Accident and Operational Safety Analysis Handbook was prepared under the sponsorship of the DOE Office of Health Safety and Security (HSS), Office of Corporate Safety Programs, and the Energy Facility Contractors Operating Group (EFCOG), Industrial Hygiene and Safety Sub-group of the Environmental Health and Safety Working Group. The preparers would like to gratefully acknowledge the authors whose works are referenced in this document, and the individuals who provided valuable technical insights and/or specific reviews of this document in its various stages of development: Writing Team Co-Chairs: • David Pegram, DOE Office of Health Safety and Security (HSS) • Richard DeBusk, Lawrence Berkley National Laboratory (LBNL) Writing Team Members: • Marcus Hayes, National Nuclear Security Administration (NNSA) • Jenny Mullins, DOE Oak Ridge Site Office (ORO) • Bill Wells, Lawrence Berkley National Laboratory (LBNL) • Roger Kruse, Los Alamos National Laboratory (LANL) • Rick Hartley, Babcock and Wilcox Technical Services Pantex (BW-PTX) • Jeff Aggas, Savannah River Site (SRS) • Gary Hagan, Oak Ridge Y-12 National Security Complex (Y12) Advisor: • Earl Carnes, DOE Office of Health Safety and Security (HSS) Technical Editors: • Susan Keffer, Project Enhancement Corporation • Erick Reynolds, Project Enhancement Corporation DOE-HDBK-1208-2012 ii DOE-HDBK-1208-2012 iii INTRODUCTION – HANDBOOK APPLICATION AND SCOPE Volume II of the Accident and Operational Safety Analysis1 • Initiating an OSR: Without a serious accident to drive management attention, OSRs must be motivated by preliminary research data and justifications. Handbook builds upon the concepts, philosophy, processes, and techniques presented in Volume I. Volume II, Chapter 1 has been structured using the same logical sequence for organizing a task team and executing the Department of Energy (DOE) Federal Accident Investigations that is presented in Volume I, Chapter 2. Volume II is not intended to be a standalone volume, but supplements Volume I, Chapter 2 with the key differences in the analysis techniques for an Operational Safety Review (OSR) in order to prevent accidents. Therefore, where the process, methods, tools, roles and responsibilities are nearly identical, Volume II refers to the equivalent sections and useful attachments in Volume I. The enhanced analysis techniques presented in this volume of the Handbook have been extracted from academic research and validated through industry application and practice. The techniques are for performance improvement and learning and, thusly, can be applicable to both Accident Investigation (AI) and OSR. Tables 1-2 and 1-3, in Volume II present a side by side comparison of the “What Happened?” and the “Why did it Happen?” analysis processes in Volume I and Volume II. The concepts, techniques, and attachments in this Volume include: • Managing OSRs like other tasks; with scope, cost, and schedule controls. • Understanding Contemporary Accident Causation. • Mapping “Work-as-Done” versus “Work-as-Planned” Gap (∆Wg • Capturing Information in the systematic Barrier Analysis Matrix (BAM). ) Analysis Using the Comparative Timeline (CTL).

• Consolidating various organizational factors into analysis tools: Culture Attribute Matrix (CAM); Missed Opportunity Matrix (MOM); Human Error Precursor Matrix (TWIN); Latent Organizational Weakness Table (LOW) • Structuring organizational lines of inquiry: Safety culture LOIs are cross-walked to the seven guiding principles of DOE P 450.4A, Integrated Safety Management Policy. • Applying the Causal Factors Analysis (CFA) charting method to the OSR. Accident Investigations (AI) and Operational Safety Reviews (OSR) are valuable for evaluating technical issues, safety management systems and human performance and environmental 1 The term operational safety analysis for the purposes of this Handbook should not be confused with application of other DOE techniques contained within nuclear safety analysis directives or standards such as 10 CFR 830 Subpart B, or DOE-STD-3009. DOE-HDBK-1208-2012 iv conditions to prevent accidents, through a process of continuous organizational learning. This Handbook brings together the strengths of the experiences gained in conducting Department of Energy (DOE) accident investigations over the past many years. That experience encourages us to undertake analyses of lower level events, such as near misses, and adds insights from High Reliability Organization (HRO)/Learning organizations and Human Performance Improvement (HPI). Analysts will benefit from reviewing DOE Standard Human Performance Improvement Handbook, Volume 1 and 2; and attending formal HPI training. Volume I focuses on investigating one major serious event and analyzing how to prevent its recurrence. A fundamental difference in Volume II arises from the emphasis on an OSR that systematically selects and analyzes a set of minor events to extract recommended organization level changes that reduces the potential for major serious events from occurring in the first place. The recommended techniques apply equally well to DOE Federal-led accident investigations conducted under DOE Order (O) 225.1B, Accident Investigations, dated March 4, 2011, (Volume I) contractor-Led accident investigations or under DOE O 231.1A, Chg 1, Environment, Safety and Health Reporting, dated June 3, 2004, or Operational Safety Reviews (Volume II) as a element of a “Contractor Assurance Program.” The application of the techniques described, as applied to contractor-led accident investigations or OSRs, are completely non-mandatory and are applied at the discretion of contractor line managers. Only a selected few accidents, events, or management concerns may require the level and depth of analysis by the contractor’s line management. Volume I - Chapter 1 provides the fundamental concepts of accident dynamics, accident prevention, and accident analysis. Volume I - Chapter 2 provides the process for organizing an accident investigation including, selecting the team, assigning roles, collecting and recording information and evidence; organizing and analyzing the information, forming Conclusions (CON) and Judgments of Need (JON), and writing the final report. Volume II - Chapter 1 provides the adaptation of the above concepts and processes to an Operational Safety Review, as an approach to preventing accidents by revealing organizational weaknesses before they result in an accident. DOE-HDBK-1208-2012 v Table of Contents

ACKNOWLEDGEMENTS ........................................................................................................... i INTRODUCTION, APPLICATION AND SCOPE ........................................................................iii ACRONYMS .............................................................................................................................vii CHAPTER 1. THE OPERATIONAL SAFETY REVIEW PROCESS ........................................ 1-1 1. THE OPERATIONAL SAFETY REVIEW PROCESS ..................................................... 1-1 1.1 Establishing the Need for an Operational Safety Review .......................................... 1-1 1.1.1 OSR Appointing Official ....................................................................................................... 1-3 1.1.2 Appointing the Operational Safety Review Team ............................................................... 1-3 1.1.3 Briefing the OSR team ......................................................................................................... 1-4 1.2 Organizing the OSR ..................................................................................................... 1-5 1.2.1 Establishing OSR Information Access and Release Protocols ............................................. 1-5 1.3 Managing the Review Process .................................................................................... 1-6 1.4 Controlling the Review Process .................................................................................. 1-6 1.4.1 Monitoring Performance and Providing Feedback ............................................................. 1-6 1.4.2 Controlling Cost and Schedule ............................................................................................ 1-7 1.4.3 Assuring Quality .................................................................................................................. 1-9 1.5 Investigate the Event(s) to Determine “What” Happened .......................................... 1-9 1.5.1 Understand Work-as-Done (WAD) .................................................................................... 1-14 1.5.2 Understand Work-as-Planned (WAP) ............................................................................... 1-24 1.5.3 Compare Work-as-Done to Work-as-Planned to Determine “What” Went Wrong ......... 1-26 1.6 Investigate the Event(s) to Determine “Why” It Happened ...................................... 1-29 1.6.1 Expand Comparative Timeline .......................................................................................... 1-29 1.6.2 Consolidate Indicators of Deeper Organizational Factors ................................................ 1-35 1.6.3 Determine Causal Factors ................................................................................................. 1-36 1.7 Developing Conclusions and Judgments of Need to Improve Organizational Safety Systems ...................................................................................................................... 1-39 1.7.1 Conclusions ....................................................................................................................... 1-39 1.7.2 Judgments of Need ........................................................................................................... 1-39

1.8 Reporting the Results ................................................................................................ 1-39 1.9 Performing Verification Analysis, Quality Review and Validation of Conclusions 1-40 1.9.1 Review by the Health, Safety, Quality, and Security Management .................................. 1-40 1.9.2 Document the Reviews in the Records ............................................................................. 1-40 1.10 Submitting the Report ................................................................................................ 1-41 Appendix A. Glossary ........................................................................................................... A-1 Appendix B. References ....................................................................................................... B-1 Attachment 1. Reference Examples and Supplemental Material ....................................... 1-1 Attachment 2. Bibliography .................................................................................................. 2-1 DOE-HDBK-1208-2012 vi Table of Tables Table 1-1: OSR Team Members Should Meet These Criteria ............................................................... 1-4 Table 1-2: Comparison of Investigative Tools to Determine “What” Happened ............................... 1-10 Table 1-3: Comparison of Analysis Tools to Determine “Why” Event Happened ............................. 1-12 Table 1-4: Sample of Safety Culture Lines of Inquiry ......................................................................... 1-31 Table of Figures Figure 1-1: Example Report Outline ...................................................................................................... 1-8 Figure 1-2: Timeline ............................................................................................................................. 1-15 Figure 1-3: Steps to Map Work-as-Done ............................................................................................. 1-16 Figure 1-4: Work Process Maps Being Compared to Determine Gaps in Work .................................. 1-17 Figure 1-5: Barrier Analysis Process to Capture “Barriers-As-Done” .................................................. 1-18 Figure 1-6: Identify Where the Threat and Hazard Come Together on Process Map ......................... 1-19 Figure 1-7: Systematic Barrier Analysis Do-Loop ................................................................................ 1-19 Figure 1-8: Capturing Information in the Systematic Barrier Analysis Matrix (SBAM) ....................... 1-21 Figure 1-9: Comparative Timeline ....................................................................................................... 1-27 Figure 1-10: Expanded Comparative Timeline ...................................................................................... 1-30 Figure 1-11: Causal Factors Analysis (CFA) Chart .................................................................................. 1-38 DOE-HDBK-1208-2012 vii ACRONYMS AI Accident Investigation AIB Accident Investigation Board BAM Barrier Analysis Matrix BTC Break-the-Chain CA Corrective Actions CAM Culture Attribute Matrix CFA Causal Factors Analysis CFR Code of Federal Regulations CON Conclusions CTL Comparative Timeline DOE Department of Energy DOE G DOE Guide DOE M DOE Manual DOE O DOE Order DOE P DOE Policy

ECAQ Extraneous Conditions Adverse Quality ECF Events and Causal Factors EFCOG Energy Facility Contractors Operating Group EOC Extent of Conditions and Causes FOIA Freedom of Information Act HPI Human Performance Improvement HRO High Reliability Organization HSS Office of Health, Safety and Security ISM Integrated Safety Management ISMS Integrated Safety Management System JON Judgment of Need LOW Latent Organizational Weakness Table MOM Missed Opportunity Matrix NSRC SBAM Systematic Barrier Analysis Matrix Nuclear Safety Review Concepts OSR Operational Safety Review DOE-HDBK-1208-2012 viii TWIN Task, Work Environment, Individual Capabilities, Human Nature (TWIN) Analysis Matrix (Human Performance Error Precursors) WAD Work-as-Done WAP Work-as-Planned DOE-HDBK-1208-2012 1-1 CHAPTER 1. THE OPERATIONAL SAFETY REVIEW PROCESS (Application of the Expanded Event and Causal Factor Analysis to Prevent Accidents Before They Happen) 1. THE OPERATIONAL SAFETY REVIEW PROCESS 1.1 Establishing the Need for an Operational Safety Review The Operational Safety Review (OSR) is based on the premise that major accidents or incidents are not caused by individual errors, but are set up by the organizational environment the employee works in. Therefore, it is not necessary to wait until a catastrophic accident to use the investigation and analysis process defined in this handbook because the underlying organizational issues are at work all the time and can be preemptively identified through review of precursor incidents. Lesser consequential or information-rich events or negative trend indicators investigated using the OSR process can identify key organizational factors that, if not identified and corrected, could lead to a more catastrophic event. The OSR process is presented as an enhanced version of the investigation and analysis process used for accident investigations as described in Volume I, Chapter 2. It offers a more in depth review of the latent conditions that represent weakened safety systems that could lead to future accidents. Methods presented in Volume II, Chapter 1 could be used by an AI team to enhance an investigation or an AI JON could recommend an OSR for a particular area needing expanded extent of conditions review. In the OSR, an objective is to systematically breakdown the events into basic components relative to how the organization had originally planned to do work2 and why they thought their plan would succeed3 Also, the OSR points out the organizational or safety culture weaknesses that landed the organization where they are and, if not corrected, will prevent corrective actions from being effective or long lasting. By using a framework of how organizations plan work in reverse (like the Break-the-Chain framework introduced in Volume I, Chapter 1), weaknesses can be . The OSR process addresses the organizational drift from the established and proven safety system and provides usable/actionable feedback to the organization focused at those points where the safety system drifted and subsequently failed. 2 Described by the organization’s application of the Integrated Safety Management (ISM) Core Functions to their work processes or the logical, physics-based application of the ISM Core Functions described in the Break-the-Chain framework introduced in Chapter 1.

3 As described in Volume I, Chapter 1, latent organizational weaknesses and the organization’s culture degrade an organization’s ability to succeed in the work that they had planned. If these are not recognized and corrected, corrective action or organizational improvement are limited and short-lived. 1. 1 DOE-HDBK-1208-2012 1-2 identified and shared with the organization in such a way that it provides directed and actionable corrective actions. When fully implemented and verified effective, these actions should stop, or least reduce, the recurrence of unwanted events. The three key distinctions between the Accident Investigation Board (AIB) in Volume I, Chapter 2 and the OSR team in Volume II, Chapter 1 are: 1) there is no mandatory scope to focus on preventing a particular accident; 2) there is no requirement that the OSR team be appointed by or contain Federal employees; and 3) there is no required review process or approach. Using AI process and methods for an OSR can be completely contained in a contractor’s organization review or investigation process, including established improvement programs such as Six Sigma. The contractor has the flexibility in investigative approaches selected. However, the fundamental concepts of accident dynamics, accident prevention, and accident analysis laid out in Volume I, Chapter 1 and many of the practical approaches to accident/incident analyses presented in Volume I, Chapter 2 still apply. Therefore, where applicable, Volume II, Chapter 1 references the content of Volume I, Chapter 2 for practical steps to investigate incidents and offers expanded approaches for a contractor-led OSR. Planning and implementing an Operational Safety Review, as presented in Volume II, Chapter 1, utilizes the same basic steps and processes used for the Accident Investigation in Volume I, Chapter 2: • Define the Scope of the Investigation and Select the Review Team • Collect the Evidence • Investigate “what happened” • Analyze “why it happened” • Define and Report the Judgments of Need and Corrective Actions The lack of an existing major accident as a motivation results in the need for a justification that must be researched and presented to a potential management sponsor. The preliminary “white paper” proposal explains the information pointing to a need for an OSR and the potential gains in safety enhancements that justify the investment in time and resources. Various management concerns, an unreviewed safety question, assessment findings or JONs, trending reports, external lesson learned reports could be the source justification. The level of management sponsorship will influence the resources available to form an OSR team. This, in turn, will influence the scope of the review and determine the extent to which latent conditions can be explored. The breadth of a review into many occurrences or incidents is influenced not so much by what did happen but the possibilities of what could happen. These consequential possibilities can branch out in multiple directions with the probing analysis into core latent conditions. This expansion of potential consequences can be the basis for additional OSR JONs but can also cause the scope of the investigation to escalate beyond reasonable practicality. Controlling the scope as well as the thoroughness of the review will be essential to delivering useful and lasting results to management. To avoid an OSR that could try to change the whole world in one review, limit the scope to “attainable” segments with practical, deliverable objectives.

DOE-HDBK-1208-2012 1-3 1.1.1 OSR Appointing Official Upon notification of an occurrence or other indicator justifying an OSR, the Sponsoring Manager will function as the Appointing Official and selects the OSR Chairperson. The Appointing Official, with the assistance of the Team Chairperson, selects three to six other team members, one of whom should be an experienced, trained accident investigator or accident analyst. To minimize conflicts of interest influences, the Chairperson and the accident investigator should be from different reporting elements than the organization being reviewed. The appointing official an OSR is a senior manager with budget authority, unless this responsibility is delegated to the contractors Office for Health, Safety, Quality, and/or Security. The contractor management roles and responsibilities should parallel similarly with those of the Appointing Official for Accident Investigations, the Heads of Program Elements for Accident Investigations, and the Heads of Field Elements for establishing and supporting OSRs as defined in the Table 1-1. 1.1.2 Appointing the Operational Safety Review Team A list of prospective Chairpersons who meet minimum qualifications should be available from the OSR Program Manager. The OSR Program Manager should also maintain a list of qualified Team members, consultants, advisors, and support staff, including particular areas of expertise. The Appointing Official, with the help of the OSR Program Manager, and the selected OSR Chairperson, assess the potential scope of the review and identify other team members needed to conduct the review. In selecting these individuals, the chairperson and appointing official follow the criteria defined in Table 1-1. Additional suggested restrictions concerning the selection of OSR team members and Chairpersons include: members should not have a supervisor-subordinate relationship with another Board member and should not have any conflict of interest or line management responsibility for day-to-day operation or direct oversight of the facility, area, or activity involved in the incident. Advisory staff or consultants may be necessary to provide knowledge of management systems or organizational concerns or technical expertise. A dedicated and experienced administrative coordinator (see Volume I, Appendix C) is recommended to facilitate recordkeeping, evidence tracking, meeting coordination, and report writing. The appointing official should appoint the OSR team as soon as possible after the justification is established by issuing an appointment memorandum. The appointment memorandum establishes the OSR team’s authority and releases all members of the OSR team from their normal responsibilities/duties for the period of time the Team is convened. The appointment memorandum also includes the scope of the review, the names of the individuals being appointed to the Team, a specified completion date for the final report (nominally 30 calendar days), and any special provisions deemed appropriate. A Sample Appointment Memorandum may be found in Volume I, Appendix D. DOE-HDBK-1208-2012 1-4 Table 1-1: OSR Team Members Should Meet These Criteria Role Qualifications Chairperson • Senior manager with demonstrated project management competence and briefing skills. • Knowledgeable of accident investigation techniques. • Experienced in conducting accident investigations or operational safety reviews through participation in at least one investigation or OSR.

• Preferably trained in the method of this handbook. Team Members • Organization employee with investigative and analytical skills. • Subject matter expertise in technical areas and management systems related to the incident, including knowledge of the safety management system policy and integrated safety management system. • Familiarity with as much of the organizations’ operations, organization, and culture as possible. Team Advisor/Consultant • Knowledgeable in evaluating technical systems, management systems, the adequacy of policy and its implementation, and the execution of line management oversight. • Industry working knowledge in the analytical techniques used to determine accident causal factors 1.1.3 Briefing the OSR team The appointing official is responsible for briefing all OSR team members as soon as possible after their appointment to ensure that they clearly understand their roles and responsibilities. The briefing emphasizes: • The scope of the review; • The Team’s authority to examine organizations and management systems, including line management oversight, as potential causes of an accident, up to and beyond the level of the appointing official; • The necessity for avoiding conflicts of interest; • Evaluation of the effectiveness of management systems, as defined by DOE P 450.4A, Integrated Safety Management Policy, or an equivalent contractor/industry policy; • Pertinent incident/occurrence information and special concerns of the appointing official based on site occurrence patterns or other considerations. DOE-HDBK-1208-2012 1-5 1.2 Organizing the OSR The Operational Safety Review is a complex project that involves a significant workload, time constraints, sensitive issues, cooperation between team members, and dependence on others. The Chairperson for a contractor-led OSR team, can also be known by other titles, depending on the type of investigation or operational safety review, including Investigation Leader or Team Leader, but for simplicity of this document, this person is referred to here as OSR Chairperson. To finish the investigation within the time frame required, the OSR Chairperson, must exercise good project management skills and promote teamwork. The Chairperson’s initial decisions and actions will influence the tone, tempo, and degree of difficulty associated with the entire review. Using the same methods and skills as an AIB Chairperson, Volume I, Section 2.2 provides the OSR Chairperson with techniques and tools for planning and organizing the investigation. 1.2.1 Establishing OSR Information Access and Release Protocols The Chairperson is responsible for establishing protocols relating to information access and release. These protocol concerns are listed in Table 2-4 of Volume I, Chapter 2. Information access and other control protocols maintain the integrity of the review and preserve the privacy and confidentiality of interviewees and other parties. The Freedom of Information Act (FOIA) and Privacy Act may apply to information generated or obtained during a contractor-led operational safety review. These two laws dictate access to and release of government records. The Chairperson should obtain guidance from a legal advisor or the FOIA/Privacy Act contact person at the site, field office, or Headquarters regarding questions of disclosure, or the applicability of the FOIA or Privacy Act. The FOIA provides access to Federal agency records except those protected from release by exemptions. Anyone can use the FOIA to request access to government records.

The OSR team must ensure that the information it generates is accurate, relevant, complete, and up-to-date. Court reporters may be used in more serious investigations to record interviews, and interviewees should be allowed to review and correct transcripts. FOIA would not normally apply to contractor-led operational safety reviews. The Privacy Act protects government records on citizens and lawfully admitted permanent residents from release without the prior written consent of the individual to whom the records pertain. Specifically, when the Privacy Act is applicable, the OSR team is responsible for: • Informing interviewees why information about them is being collected and how it will be used. • Ensuring that information subject to the Privacy Act is not disclosed without the consent of the individual, except under the conditions prescribed by law. Information that can normally be disclosed includes name, present and past positions or “grade” (e.g., GS-13), 1. 2 DOE-HDBK-1208-2012 1-6 annual salaries, duty station, and position description. Therefore, the OSR team should not request this information unless it is relevant to their investigation. 1.3 Managing the Review Process As an investigation proceeds, the OSR Chairperson uses a variety of management techniques, including guiding and directing, monitoring performance, providing feedback on performance, and making decisions and changes required to meet the OSR objectives and schedule. Because these activities are crucial, the Chairperson may designate an individual to oversee management activities in case the Chairperson is not always immediately available. Using similar methods and skills as an AI Board Chairman in Volume I, Chapter 2, Section 2.3 provides the OSR Chairperson with techniques and tools for managing, controlling, and closing out the investigation. 1.4 Controlling the Review Process Throughout the review and analysis and report writing, the OSR Chairperson is responsible for controlling Team performance, cost, schedule, and quality of work. Techniques for implementing these controls are described below. 1.4.1 Monitoring Performance and Providing Feedback The Chairperson uses daily meetings to monitor progress and to measure performance against the schedule of activity milestones. Team members are given specific functions or activities to perform and milestones for completion. The Chairperson assesses the progress and status of the OSR investigation periodically by asking such questions as: • Is the OSR on schedule? • Is the OSR within scope? • Are Team members, advisors, consultants, and support staff focused and effective? • Are additional resources needed? • Are daily team meetings still necessary and productive, or should the interval between them be increased? The OSR Chairperson should be informed on the status of the OSR and be prepared to make decisions and provide timely feedback to the team members, site personnel, and other parties affected by the review. Frequently, decisions must be made when there is not time to reach consensus among the team members. When this occurs, the Chairperson informs the team members of the decision and the reason for the urgency. Intermediate milestone revisions can then be made, if events or practical considerations so dictate. 1.3 1.4 DOE-HDBK-1208-2012 1-7 1.4.2 Controlling Cost and Schedule Cost and schedule should be controlled to ensure that planning and execution activities are within the established budget and milestones.

• Cost Control: The OSR Chairperson is responsible operating within any budget prescribed for the review. The Chairperson should prepare a cost estimate for the activities to be conducted during the review, if needed. The Chairperson may issue a memo authorizing costs incurred by team members, including additional travel expenses, hotel rates over per diem, and incidental expenses. Control can be exercised over costs by using advisors and consultants only when required and by limiting travel (such as trips home for the weekend) during the onsite investigation. The Sponsoring Manager is the point of contact for these concerns. A method for estimating costs should be agreed upon early in the OSR, and the estimate should be reviewed each week to ensure that the cost of the work is not exceeding the estimate, or that any cost growth is justified and can be funded. • Schedule Control: Progress against the scheduled milestones can be assessed during daily progress meetings with the team. As problems arise, the schedule may be adjusted or resources applied to offset variances. Because of the relatively short time frame involved, the Chairperson must identify and resolve problems immediately to maintain the schedule, or re-evaluate it with the appointing official as circumstances require. 1.4.2.1 How to Motivate and Focus an Investigation Team Recognize that any mention made of systematic tools equates to tedious work. As such, the first and most important point of order for the OSR Chairperson is to motivate the team to give their all throughout the investigation. A way of doing this is let the team know what needs to be done and why. The best tool for this is the OSR report outline. The example report outline shown below: • Parallels how an OSR will be conducted – logical layout that makes sense to the OSR team and the receiving manager. • Tells what is needed to complete OSR and the tools needed to complete it (on full version in Attachment 1). • Motivates team to focus and to drive to a successful completion because they have the clear objective to complete the report. This example format, shown in Figure 1-1, is structured along the fundamental organizational learning process to compare work-as-done to work-as-planned. The review process mirrors this in Volume I, Chapter 2 by comparing how management expected work to be done, work-as- planned (documented in the technical basis section), against how work was done, work-as-done (documented in the event summary), to determine the size and importance of the gaps (∆Wg). After laying out what” did not work in Chapter 3, the team next explores “why” things did not go according to plan in Chapter 4, explains how significant the issues were in Chapter 5, and identifies what organizational factors degraded to a weakened condition in Chapter 6. Finally, DOE-HDBK-1208-2012 1-8 additional observations, the Judgments of Need, and Lessons to be Learned can be listed in Chapters 7, 8, & 9: Chapter 1 Executive Summary 1.1 Event Summary 1.2 Impact of Event 1.3 Org Weaknesses & Recommend Chapter 2 Introduction (compare work-as-done to work-as-imagined/planned) 2.1 Event Summary (“work-as-done”) 2.2 Technical Basis (“work-as-planned”) 2.3 Important Gaps Between Work-as-Done and Work-as-Planned Chapter 3 Event Facts (“what happened”  ∆Wg) 3.1 Initial Conditions 3.2 Consequences 3.3 Event Facts (∆Wg #1, #2, #3 etc.) Chapter 4 Organizational Causal Factors (“why” it happened) 4.1 Chart A (whys for ∆Wg #1) 4.2 Chart B (whys for ∆Wg #2) 4.3 Chart C (whys for ∆Wg

#3) Chapter 5 Impact of Event and What Influenced It 5.1 Consequences 5.2 Significance Chapter 6 Organizational Weaknesses (deeper organizational issues) 6.1 Feedback 6.2 Learning Organization 6.3 Organizational Weaknesses 6.4 Culture 6.5 HRO Practices 6.6 Precursors Chapter 7 Extraneous Conditions Adverse to Quality Chapter 8 Judgments of Need (JON) Chapter 9 Lessons To Be Learned (organizational learning) Figure 1-1: Example Report Outline DOE-HDBK-1208-2012 1-9 1.4.3 Assuring Quality Formal quality control measures are necessary because of the need for accuracy, thoroughness, and perspective, in order to support justifications for change and overcome potential push-back when management is presented with the impacts of the JONs. The Chairperson has the prerogative to implement any quality assurance measures deemed necessary. At a minimum, the Chairperson should ensure that the OSR report is technically accurate, complete, and internally consistent. When analytical results are developed into conclusions, all verified facts, the results of analyses of those facts, and the resulting conclusions should be both consistent and logical. Volume I, Chapter 2, Section 2.8 provides further detail on assuring report quality. 1.5 Investigate the Event(s) to Determine “What” Happened The fundamental construct of the tools introduced in this chapter is the Break-the-Chain framework as a logical way to break events down into clear investigative and hopefully corrective steps. This focus, at the physics level, points to where corrective actions are required to fix the safety physics. Many of the tools to detect organizational drift, such as detecting the gap between work-as-done by the employee and work-as-planned by the organization, are the similar to those introduced in Volume 1, Chapter 2. The key here is that one cannot fix problems they cannot see. Seeing the gaps between work-as-done and work-as-planned provide the required eye-opening moment to recognize a need for change. A fundamental enhancement is to detect subtle discrepancies in the detail instructions employed by DOE organizations to perform work, to prove, rather than assume, safety. The two tools introduced here are the Comparative Time Line (CTL4) and the Systematic Barrier Analysis Matrix (SBAM)5 Most of the tools to accomplish a lasting organizational improvement are based on systematically collecting evidence of latent organizational weaknesses, unhealthy safety culture, missed opportunities, human performance error precursors, and human performance error modes. This diverse evidence is structured in the CTL and organized in tables (e.g., Latent Organizational Weakness Table (LOW), Culture Attribute Matrix (CAM), Missed Opportunity Matrix (MOM), Human Performance Error Precursor Table (TWIN), etc.). These tables and the CTL can then be used to extrapolate locally detected problems to a more global set of indicators . The one key challenge, after recognizing gaps at the employee level and putting in corrective actions in place at the physics level, is to recognize and begin to fix the setup factors at the organizational level to ensure the corrective actions are effective and long lasting. There is no magic recipe to accomplish this. This requires organizations to struggle with the concepts, to attempt to assess their current state, and to experiment to improve the work environment such that it is conducive to productive, safe work.

4 The concept and primary structure of the Comparative Time Line, CAM, and MOM was adopted from the work of Bill Corcoran, Nuclear Safety Review Concepts (NSRC). 5 The discrimination of Systematic Barrier Analysis is used to differentiate from the earlier introduced barrier analysis matrix. 1. 5 DOE-HDBK-1208-2012 1-10 of organizational weaknesses (e.g., Latent Organizational Weaknesses, Table 1-1 in Volume I, Chapter 1). Only then will the managers be able to get a sense of the challenges they have to improve the work environment for their employees. To help frame the noted similarities and differentiate the contractor opportunities when conducting OSRs, Table 1-2 and Table 1-3 are provided to frame discussions in this chapter. Common acronyms found in Table 1-2 and Table 1-3 are defined here. Work-as-Done (WAD) Work-as-Planned (WAP) Comparative Time Line (CTL) Latent Organizational Weakness Table (LOW) Culture Attribute Matrix (CAM) Missed Opportunity Matrix (MOM) Human Performance Error Precursor Table (TWIN) Extent of Conditions and Causes (EOC) Table 1-2: Comparison of Investigative Tools to Determine “What” Happened Activity Basic Events & Causal Factors (ECF) Analysis (AI) Expanded ECF Analysis(OSR) Work-As- Done (WAD) Collect and Document Human Evidence Witness statements Interviews (witness list, interview schedule, interview forms) Timeline (convert stories to sequence of events) Develop work-as-done process maps (understand actual work sequence) Same as AI but add CTL to collect what happened (Column #2 CTL) Conduct Barriers Analysis Barrier analysis matrix to understand barriers-as-done by employees Systematic barrier analysis matrix to understand barriers-as-done by employees. Collect barriers in CTL (Column #5 CTL) Collect and Catalog Physical Evidence Collect (items from scene, photos/videos, sketches, maps, etc.) Electronically catalog (evidence log, photo log) Preserve (chain of custody) Capture evidence and logs in electronic files Same as AI but add CTL to substantiate what happened (Column #2 CTL) DOE-HDBK-1208-2012 1-11 Activity Basic Events & Causal Factors (ECF) Analysis (AI) Expanded ECF Analysis(OSR) Collect and Catalog Documentary Evidence Collect (records of work activities e.g., log books, access logs, training records, etc.) Catalog Preserve (chain of custody) Capture evidence and logs in electronic files Same as AI but add CTL to substantiate what happened (Column #2 CTL) Develop a Picture of What Employees Were Trying to Accomplish Assimilate all the above to understand why the decisions made by the workers made sense to them at the time. Same as AI Work-As- Planned (WAP) Collect and Catalog Documentary Evidence Collect, read and understand applicable federal regulatory requirements Code of Federal Regulations (CFRs), DOE directives, and DOE requirements. Collect, read and understand organizational policies, procedures, hazards analyses, drawings, work instructions, training. Interview management and procedure writers to understand their intent of planned work. Develop work-as-planned process maps (understand planned work sequence) Catalog Preserve Capture evidence and logs in electronic files Same as AI but add CTL to collect what should have happened (Column #3 CTL) and initial conditions on CFA Chart Conduct Barriers Analysis Barrier analysis matrix to understand barriers-as-planned by management and process designers.

Systematic barrier analysis matrix to understand barriers-as-planned by management and process designers. Collect barriers in CTL (Column #5 CTL) Develop a Technical Basis for the Event Review management and work planning and control systems and line management oversight to determine if adequate to deliver required level of safety. Determine, based on the physics, if procedures would have worked and provided the requisite level of safety if they were executed properly. Same as AI but add the step to look for ineffective or non-value added process steps and recommend removal (Column #4 CTL) DOE-HDBK-1208-2012 1-12 Activity Basic Events & Causal Factors (ECF) Analysis (AI) Expanded ECF Analysis(OSR) Compare WAP – WAD to determine gaps (∆Wg) Systematically Evaluate Difference Between WAD and WAP Compare work-as-done process to work-as-planned process to determine if procedures ineffective or improperly implemented, Compare barriers-as-done to barriers- as-planned to determine if barriers are missing or flawed because of design or because not implemented and maintained. List gaps (∆Wg) that matter, i.e., that lead to event/accident. Systematically evaluate changes or differences between WAD and WAP (∆Wg to include process differences and barriers) in CTL(Compare Column #2 with Column #3 and document in Column #4 CTL) Significant (∆Wg), that is those that caused the accident, are placed on the side of CFA Chart. Table 1-3: Comparison of Analysis Tools to Determine “Why” Event Happened Activity Basic Events & Causal Factors (ECF) Analysis (AI) Expanded ECF Analysis (OSR) Causal Factors Evaluate why barriers failed Compare and analyze barrier analysis matrices. Evaluate flawed or missing barriers in CTL (Column #5 CTL) Summarize by comparing BAM for barriers as done and barriers as planned. Evaluate Things That have Changed Conduct Change Analysis The changes or differences in WAD and WAP performed in the “what” went wrong phase of investigation Surmise the Causal Factors of the Event Develop Events & Causal Factor Chart Use CFA Chart as a different tool. Understand why each significant ∆Wg occurred. Identify Root Causes Perform Causal Factors Analysis Identify root and contributing causes in the CFA Chart (reds and yellow delineation) to focus JONs and subsequent corrective actions. DOE-HDBK-1208-2012 1-13 Activity Basic Events & Causal Factors (ECF) Analysis (AI) Expanded ECF Analysis (OSR) Deeper Organizational Factors Determine How Long Problem Persisted and How Wide It Is Across The Organization Address the questions on the ISM Core Functions at the employee level. Based on response, surmise the health of the management systems in regard to their ability to establish a work environment in which work can be performed safely. List if this event had an organization-wide impact or only local influence, whether previous similar event have occurred from which lessons were learned and if assessments have identified problems in this area and if so the corrective actions taken in Column #5 CTL. Extend CTL to document culture elements, human performance error precursors, human performance error mode, missed opportunities and indicators of latent organizational weaknesses in Column #6 CTL. Summarize human performance error precursors in TWIN matrix. Summarize extent of conditions and causes (EOC) in EOC matrix. Surmise Latent Organizational Weaknesses and Organizational Culture Issues That Could Have Led to Event

Address the questions on the ISM Principles at the management level. Based on response, surmise the concerns at the organizational level, with management systems or line management oversight. Summarize Latent Organizational Weaknesses in LOW table Summarize Culture attributes in Culture Attribute Matrix (CAM) based on response: and surmise the health of the culture of the organization relative to its leadership, employee/worker engagement, and organizational learning ability. Summarize missed opportunities in Missed Opportunity Matrix (MOM) The investigative tools to determine the “what” happened for both AI and OSRs are shown in Table 1-2. Only tools not previously introduced in Volume I, Chapter 2 will be introduced in concept in this chapter. If the reader has an interest in using these processes, they are referred to the using organizations that have much practical experience (see Acknowledgements for user organizations). These investigative tools systematically allow investigation teams to explore what parts of their system failed to work as desired. It should be noted that any tool introduced in Table 1-2 is just “a” tool, not “the” tool that can be used. If contractors have other tried and tested tools that DOE-HDBK-1208-2012 1-14 fulfill the activities noted in Table 1-2, they are welcome to substitute as appropriate to their needs. 1.5.1 Understand Work-as-Done (WAD) The most important thing any review team can do to fight hindsight bias is to see how work was actually done before reading any procedure. The effect of reading the procedure is huge. It has an enormous impact on the team’s ability to objectively see how real work is done. It is more useful to see the event through the eyes of the worker in order to understand why his/her decisions made sense to him/her at the time. To fix the problem, the team must understand the context of the decisions the persons made at the time of the events, because it is this context or work environment which is the culprit to the events being reviewed. As noted in Table 1-2, tools to understand work-as-done start with the human evidence because it represents the most volatile information available. The classical tool to collect human evidence is the interview. Eventually the “stories” from interviews need to be converted to sequential events or a time line, to gain a clearer picture of the path to what went wrong. 1.5.1.1 Develop Timeline and/or Work Process Map The most powerful tool in the investigative arsenal is to systematically separate events in time and space. This will allow the investigation team to see things that were hidden from view on a superficial look (i.e., determine the gap between work-as-planned and work-as-done, ΔWg). There are two primary ways this can be done. The first is the systematic use of the timeline, and later the comparative timeline. This tool is best used when the investigation team is uncertain as to what was trying to be accomplished when the event took place. The second is the work process map. This is best used if the work being done at the time of the event was directed by documented procedures. Either a timeline of events or a work process map of how work was actually conducted by the employee at the time of the incident will be the ultimate output of the human evidence phase of the investigation. This will give the investigation team a visual structure of how work was being done at the time of the event.

1.5.1.1.1 Steps to Develop a Timeline A timeline is used to collect, catalog and organize events by time. A simple example of a timeline is provided in Figure 1-2. In its simplest form, the timeline consist of three columns, Column 1 captures the date and time of the event sequence, Column 2 captures what happened, and Column 3 is for team questions about the entry that can be researched at a later time. The timeline converts interviews into linear language and linear thoughts. It converts a complex event into comprehendible steps and causes. DOE-HDBK-1208-2012 1-15 Figure 1-2: Timeline 1.5.1.1.2 Steps to Process Map How “Work-as-Done” Since most work done in DOE organizations is typically directed by procedures, another way to understand how work was being done at the time of the event is simply go and watch a similar process being worked. A work process map (also known as a work flow diagram) is simply a set of sequential boxes connected by arrows that shows what happens in each step of the process. The process of mapping the work has the advantages of being conducted at the work location such that the employees are much more comfortable and it affords the team the opportunity to see as well as hear. Another advantage is that by simply asking questions about someone conducting work precludes the appearance of any blame going on because all questions are directed at better understanding what each step of the work process. It is important to understand how actual work is actually done before looking at procedures (“work-as-planned”) to minimize the effects of hindsight bias. [Dekker, 2006]1 The steps to conduct a process map are provided here in summary. After selecting a process to investigate, the next step is to map out how work is done (Figure 1-3). A process map at the QuestionsDate/Time What Happened 2 31 Interview Information used to Develop Event Timeline Steps to populate • To start, populate from initial timeline from fact finding (fact-finding meeting or critique) • Compare “testimony” of what happened with documentary evidence required in Column 2 • Generate list of follow-on questions and evidence required in Column 3 • Collect more evidence • Reconstruct or re-enact to improve understanding • Re-interview until the facts are known or they stop changing DOE-HDBK-1208-2012 1-16 physical level (i.e., each step must represent something physical that somebody does to produce a product such that it can be verified by direct observation). Figure 1-3: Steps to Map Work-as-Done When developing the “work-as-done” process map, make sure the process flows are complete (i.e., no missing process steps where action is being taken that is not accounted for, or where people are interjecting things into the process that are not known about). Having a complete process flow is the absolutely first The completed work-as-done process map can be compared to the work-as-planned process map. The work-as-planned may already be diagrammed in the established procedure or it too may need to be diagrammed by the team from the written sequence, see Section 1.5.2, Understand “Work-as-Planned.” Typically, processes are modified over time based on things that have gone thing one must have and must verify accurate. To simplify the problem, document one part of the process at a time until all portions have been covered.

This is the classical Six Sigma approach of documenting the “as-is” process (“work-as-done”) to allow comparison to the “to-be” process (“work-as-planned”). By framing the investigation in terms of standard Six Sigma processes allows later process leaning and optimization using standard Six Sigma and Lean Techniques. A key step in the process mapping is that the process map must match how work is actually getting accomplished. To get this accurate, this may take several iterations involving more than one work crew so that one can get a sense of the variability in work between workers, in different shifts, etc. It is important to recognize when there is acceptable variability that the process can tolerate and those instances in which the observed variability either reduces the effectiveness of the process or increases the chances of something negative happening. This may be particularly important in those critical steps where the threat and hazard come together. Map Work-as-Done (WAD) Validate WAD Walk Process Down ID other functions who contribute to the process Select System or Process Refine Scope WAD+ Work as Performed? Iterate until WAD is the same As workers perform work No Yes DOE-HDBK-1208-2012 1-17 wrong or feedback from the customer. Over time, the processes can become contorted by many additional non-value-added steps. These “questionable” process steps are fairly obvious once work-as-done is compared to work-as-planned (Figure 1-4). Figure 1-4: Work Process Maps Being Compared to Determine Gaps in Work 1.5.1.2 Systematic Barrier Analysis The Barrier Analysis Process for System Accidents is divided into two phases. • Phase I focuses on understanding how work is actually being performed on the shop floor and understanding the actual barriers (barriers-as-done) used by the workers. • Phase II of the Barrier Analysis Process consist of comparing “work-as-done” and “barriers- as-done” with “work-as-planned” to look for gaps that require closing. 1.5.1.2.1 Steps to Conduct Barrier Analysis on “Work-As-Done” (Barriers-As-Done) Using the “work-as-done” process map, the steps to conduct the barrier analysis for system events are shown Figure 1-5. If required, the scope of the barrier analysis for system may be reduced to ensure the process can be taken to conclusion and yet not take an inordinate amount of resources (yellow hexagon in Figure 1-5). [Bush and Harkins, 2010]2 DOE-HDBK-1208-2012 1-18 Figure 1-5: Barrier Analysis Process to Capture “Barriers-As-Done” Based on the consequence selected and the “work-as-done” process map, the team then identifies hazards, threats, and existing barriers in the work process. With an understanding of how work gets done, the team focuses on how to mistake-proof the work process against things that could lead to an undesired consequence (based on the Break-the-Chain Framework). Sources of information on hazards, threats and barriers could be from • existing hazard analysis tables, • threats by conducting a thorough analyses using tools recommended in Appendix B, and • barriers from those observed in use by the workers. The most significant hazard is selected first. Then, a threat is selected that could impact the hazard. Based on this hazard-threat combination, each step in the work process where a hazard and threat can come together is identified and flagged (yellow highlighted steps in Figure 1-5).

For each step where the threat and hazard can come together, the team determines if barriers exist to block the threat from the hazard (Figure 1-6). Note these can only be barriers actually observed while walking down the work process. For each barrier identified, its effectiveness and ID Hazards, Threats, Barriers Do Barriers Block Threats? Identify Consequence to avoid Select Biggest Hazard Yes No ID a Critical Threat to Hazard Determine Where Threat & Hazard Come Together Pick Next Most Critical Threat and Continue Process Determine Where Barriers Exists List Barriers of No-Value List Barriers of Missing Barriers ID other functions who contribute to process Next Threats? Next Hazard Publish BAM Publish W-A-D Threats Barriers Hazards A Enter into BAM Finish Phase 1 Pick Next Largest Hazard and Continue Process Yes Yes No DOE-HDBK-1208-2012 1-19 significance in blocking the threat from the hazard are documented in the columns 5 and 6 of the Systematic Barrier Analysis Matrix (SBAM) (Figure 1-8). Figure 1-6: Identify Where the Threat and Hazard Come Together on Process Map The process is continued for each threat against the selected hazard and, again, for each new hazard selected. The do-loop process is shown in Figure 1-7. The output of Phase I is the publication of the “work-as-done” process map and the publication of the SBAM with barriers- as-done. Figure 1-7: Systematic Barrier Analysis Do-Loop Consequence Hazard Threat Continue Step #1 Step #2 Step #3 Step #4Step #2 Step #4 Threat Hazard Threat Hazard DOE-HDBK-1208-2012 1-20 Steps to Document Barrier-As-Done To document the results of the system event barrier analysis, the team fills out the SBAM detailed below and illustrated in Figure 1-8. The SBAM is based on the Break-the-Chain Framework (BTC) introduced in Volume I, Chapter 1. The BTC will systematically prevent consequential events, if it used before beginning work. If an event occurs, barrier analysis can be used systematically to determine where in the six step process the system failed to provide focused corrective actions. This barrier analysis process is intended to be self-explanatory but a knowledgeable analyst should help maintain the team’s focus to get the biggest return on time investment. The barriers, as management expects to be in place from documented hazard analyses and resultant procedures – barriers-as-planned, are compared to barriers that are actually used by the workers (barriers-as-done) to determine if there are any gaps to maintain safety, quality, or security. The Systematic Barrier Analysis Matrix (SBAM) – The System Event to Avoid • To maintain focus on the goal and consequence to avoid, the system events to avoid are placed on the top of each SBAM. For example, if the Barrier Analysis was being conducted for a event, the Goal and Consequence to Avoid would be listed as:  Organization Goal: Ensure the Viability of the organization  Consequence to Avoid: Shutting down facility • The first data entry is to identify the system event to avoid.  Tie the system to the consequence to avoid.  If more than one event exists, ensure they are in priority order of impact to the consequence to avoid. Work only one event at a time. • Repeat the cycle for each hazard and enter results on the SBAM.  Ensure all  Ensure hazards are put in priority order of causing the undesired event hazards are identified.

• Ensure each hazard is minimized and protected (Break-the-Chain Framework)  Ensure hazard is minimized first, if possible.  Ensure hazard is protected second, required. Goal: Ensure Viability of Organization Consequence to Avoid: Shutting Down Business for Extended Period of Time System Event to Avoid: List Event (or Contributor Event) DOE-HDBK-1208-2012 1-21 Primary Hazards to Minimize and Protect: List Hazards that if Released Would Cause Event Col #1 Col #2 Col #3 Col #4 Col #5 Col #6 Col #7 Col #8 Col #9 Process Step Threat to Hazard Barrier Between Threat & Hazard Hazard Protecte d Effectiveness of Barrier Significance of Barrier Additional Barrier Needed Actions to Implement Effectiveness of Future Barrier Result Result Result Result Result Result Result Result Result Figure 1-8: Capturing Information in the Systematic Barrier Analysis Matrix (SBAM) Column 1 – Process Step List only the process step where the threat and hazard come together. All other steps in the work-as-done process map are not critical steps with regards to Barrier Analysis. Column 2 – Threat to Hazard • Based on the hazards identified at each step in the process flow, what are the threats to it?  Is there a threat from human error?  Is there a threat from equipment, tooling or facility malfunction?  Is there a threat from natural disasters or sabotage?  Is there a threat from something else?  Use a simple logic diagram like TapRoot or ORPs Reporting/Categorization Structure (DOE G 231.1-2, Occurrence Reporting and Performance Analysis Guide) or the Fishbone to ensure all the threats are systematically considered. • Identify each step in the process map where threats and hazards can come together by color coding or numbering the process map step. These become the process steps to focus efforts. Column 3 – Barriers between Threat and Hazard • For each critical process step in the physical process map (i.e., where identified hazards and threat come together), identify all the existing  Use a simple logic diagram like TapRoot or ORPs Cause Analysis Tree (DOE G 231.1-2) or Fishbone to ensure the appropriate barrier selected for each defined threat. barriers that prevent the threats from unleashing the hazards identified. DOE-HDBK-1208-2012 1-22  Identify the type of barrier observed (engineered or administrative barrier), as this indicates how robust the barrier is. Administrative barriers are not very robust and, if relied upon, one wants to make sure those who are implementing these barriers know their purpose and are verifying they are in place and effective before each operation.  Identify how the need for the barrier was recognized. Was it designed into the system or inserted to fix a weakness? • For each critical process step in the physical process map (i.e., where identified hazards and threats come together), identify whether the existing barriers prevent an event from occurring or if they mitigate the consequence of an event that could occur. Column 4 – Hazard Protected • Based on the system event selected, identify the hazards that, as a result of a threat (human error, equipment, tooling, or facility malfunction, natural disasters or sabotage, etc.) can be released causing the event. A critical process step is defined for this analysis as a step in the process where potential threats could interact with the hazard that could be released.

 Go through the physical process map and identify the threats and hazards in each step. If a threat to the hazard identified cannot happen in a particular process step, it becomes a non-critical process step.  After threats and hazards have been identified for each critical process step, copy them over to Column 3 of the BAM and annotate which step in the process map they come from. Column 5 – Effectiveness of Barrier Evaluate the effectiveness of the existing barriers by addressing the following three questions: • Are the Barriers Rigorously Designed?  Are the barriers in the right place in the process?  Are the barriers the right barriers for the threat? • Are the Barriers Fully Implemented and Effective?  Are the barriers in place?  Does the worker know about the barriers and is able and willing to use them?  Are the barriers effective, i.e., do they work as designed?  Do the barriers prevent the threat or mitigate the consequences of the hazard release? • Were the Barriers Maintained, Implemented and Verified Effective before Work was Begun?  Is there some type of configuration management system that controls the process and barriers as documented? DOE-HDBK-1208-2012 1-23  Does the worker know about the barriers and able and willing to use them?  Are the barriers verified operational before work is begun? Column 6 – Significance of Barrier The later in the sequence of work a barrier fails, the more significant it becomes because fewer barriers follow it to protect the undesired event. • To help determine this, identify where the barrier falls in the sequence of work (i.e., Plant Results, Worker Behavior, Job Site Conditions, and Organizational Processes & Values – Volume I, Chapter 1, Figure 1-12). • Document how many barriers were observed between the threat and hazard. • If there is more than one barrier, document if they are independent of each other such that if one fails, the others will not.  Document how independence was observed.  Identify what happens if the barrier fails by addressing the questions:  Does the worker know this or believe this?  Does management know or believe this?  How does one know? (provide documentation) Column 7 – Remove Threat or Hazard, Fix/Remove Barrier, Additional Barriers Needed • Using barriers incurs cost. Can the need for a barrier be removed by removing either the threat or hazard from the process step? • If a barrier exists but it is weak, can it be strengthen before thinking about adding an additional barrier? • If during the review, the existing barriers were inappropriate or if additional barriers are required to decrease the probability for the process leading to a consequential event, identify them here. Be sure to consider the life-time cost of maintaining the barrier vs. the risk of the event happening. Column 8 – Actions to Implement Additional Barriers • Describe what is required to implement the additional barriers identified in column 7, and • To whom the responsibility for designing and implementing will be assigned to. Column 9 – Effectiveness of Future Barrier • To help management make informed decisions, evaluate the effectiveness of the recommendation in Column 8. Is the recommended barrier an administrative barrier or an engineered barrier and as such, what is the probability of sustained success? DOE-HDBK-1208-2012 1-24 1.5.1.3 Develop a Picture of What Workers Were Trying to Accomplish

Using all the information collected above, the investigation team develops a picture of what the workers were trying to accomplish at the time of the events by reconstructing or re-enacting the events. This allows the team to see holes in their logic sequence to allow further iterations of data collection to continue. Getting into the head of the employee to understand why the decisions made sense to them at the time enables the team to gain an appreciation of how the system set the worker up to err. 1.5.2 Understand Work-as-Planned (WAP) The comparison of work-as-done to work-as-planned can reveal not only the drift in the work-as- performed, but also, the possible inadequacy in the process of developing the work-as-planned. Therefore, in a manner similar to acquiring data to understand work-as-done, documentary evidence needs to be collected to determine how work was planned by management or the procedure writers. 1.5.2.1 Collect and Catalog Documentary Evidence Documents often provide important evidence for identifying causal factors of an accident. Documentary evidence for work-as-planned consist of any documents that communicate management expectations of how, when, where, and by whom work activities are to be performed. These may consist of policy documents, roles and responsibilities, work instructions, step-by-step technical procedures, etc. This evidence gives important clues to possible underlying causes of errors, malfunctions, and failures that led to the incident. 1.5.2.2 Read and Understand Documentary Evidence Examples of documentary evidence the investigation team may require to establish how management expected work to be done could include: • Contract documents • Code of Federal Regulations • DOE Directives (orders, standards, guides) • Policy documents • Roles and responsibilities documents • Fitness for duty requirements • Procedures (program documents, manuals, work instructions, technical procedures, drawings, as-built drawings, etc.) • Hazard Analyses (Job Hazard Analyses, Process Hazard Analyses, Documented Safety Analyses, etc.) This information needs to be read and understood by the team members to understand the work that was planned and how it was communicated to the worker. DOE-HDBK-1208-2012 1-25 In addition to the written information, explore the training and qualification of the employees conducting the work by reviewing the training plans for those who were to use the procedures. 1.5.2.3 Interview Process Writers/Developers Because of the possible technical content and volume of documents to review, the team may want to locate and interview the process writers and developers to ensure the team has a complete and accurate understanding of how management had intended work to be done. To avoid the chance of introducing any biases in this effort, the team may need to call in subject matter experts not directly related to the event on the various processes in lieu of the procedure writer of a procedure in question. 1.5.2.4 Develop Technical Basis for the Event

Based on both the written documentation and the interviews with the procedure developers, the team should develop a technical basis for the incident. That is they should understand what work the procedure writer was trying to get accomplished and get a sense of whether the procedure writer provided procedures that were understandable, actionable, and that provided the requisite level of protection to the employee. That is, were the hazards, threats and barriers adequately identified in the written procedures? The team should challenge and not assume the procedures provide the required level of safety. Past experience has shown that many procedures were written by people who did not talk to the employees, did not physically walk down the process, and who did not incorporate modern human performance into their procedures such that it minimizes the errors of the employees. 1.5.2.5 Develop a Process Map of How Was Intended to be Done In a fashion similar to work-as-done, the team should use the written procedures and develop a work-as-planned process map to allow comparison of work-as-done. 1.5.2.6 List the Hazards Documented in the Procedures Review each procedure (or hazard analysis) that pertained to the incident to identify where hazards were called out. Document these in relationship to the process step in which they were identified (note, the hazards may not be called out in each process step and this may be an issue if they change based on which sequence in the process the employee is in). 1.5.2.7 List the Threats Documented in the Procedures Review each procedure that pertained to the incident to identify where threats to the hazard were called out. Document these in relationship to the process step in which they were identified (note, the threats may not be called out in each process step and this may be an issue if they change based on which sequence in the process the employee is in). 1.5.2.8 List the Barriers Documented in the Procedures Review each procedure that pertained to the incident to identify where barriers between the threats and hazard were called out. Document these in relationship to the process step in which they were identified (note, the barriers may not be called out in each process step and this may be an issue if they change based on which sequence in the process the employee is in). DOE-HDBK-1208-2012 1-26 1.5.2.9 Develop a Picture of How Work Was Planned Using all of the above, ensure the team gets together and develops a picture of how the procedure writers intended work to be performed. Again, be critical because if the procedure is fraught with problems, how can we expect the worker to be error free? 1.5.3 Compare Work-as-Done to Work-as-Planned to Determine “What” Went Wrong Now, with a firm understanding of work-as-planned, a comparison of work-as-done and work- as-planned can be accomplished. This can be done in a very systematic way by using a comparative timeline in which the work-as-done and work-as-planned can be compared on a time sequence or process step-by-step basis. 1.5.3.1 Systematically Compare WAD to WAP Using Comparative Timeline To systematically separate the event in time and space, the OSR team uses the Comparative Timeline (CTL) to compare work-as-planned to work-as-done for each stage of work or time step of the event. The CTL begins the data-to-information conversion process by systematically compiling and comparing the results of the concurrent reviews of interviews, exhibits, and attachments. The CTL introduced in this document has been adapted from William Corcoran’s Phoenix Handbook. [Corcoran, 2003]3

• Start with the event timeline. The CTL is constructed by modifying the original event timeline. To create the CTL (Figure 1-9): • Transfer the data from Column 1, “Date/Time,” from the timeline to the CTL. • Transfer the data from Column 2, “What Happened,” from the timeline to the CTL. • Delete the question column and add:  Column 3, “What Should Have Happened”  Column 4, “Immediate Consequence of Difference”  Column 5, “Significance of Difference” Populate Column 3 using what should have been done (work-as-planned), which was obtained through the team’s review of directives and procedures attachments (technical basis for the event). In Column 4, list the immediate consequence of the difference between Column 2 and Column 3. Specify the significance of this difference on the consequence of the event in Column 5. Human error often occurs when things change. Over time, the CTL systematically organizes the data with continually finer resolution. Finer time resolution expands the problem and allows the DOE-HDBK-1208-2012 1-27 team to see changes and determine where human error affected the process. Human errors reveal themselves as the gaps between work-as-planned and work-as-done. Figure 1-9: Comparative Timeline To investigate the gaps between work-as-planned and work-as-done, the OSR team should review each time sequence entry and compare what happened (Column 2, Figure 1-9) against what should have happened if the established process had been followed (Column 3, Figure 1-9). What should have happened is derived from an understanding of the technical basis of the event. The team documents any observed gaps in Column 4 of Figure 1-8. In Column 5 of the CTL the team enters how significant the gap was to the consequence of the event. Significance is characterized by the significance indicator and by the type of factor (i.e., vulnerability, triggering, exacerbating, or mitigating). In the “Significance of Difference” column of the CTL (Column 5), list the significance indicators using questions such as: • Was this event sequence a precursor to a more significant event? (Precursor) Date/ Time What Happened What Should Have Happened Immediate Consequences of Difference Significance of Difference (impact on consequence) 1 2 3 4 5 Original Timeline Work-as-done Gap between “work-as- planned” and “work-as- done” ∆Wg Work-as- imagined Steps to populate • Build on the event timeline (question column goes away, add Columns 3-5) • Column 2 represents the “work-as-done” • Use the “work-as-imagined” to populate Column 3 • List the immediate consequence of the difference between Columns 2 & 3 in Column 4 • Specify the significance of this difference on the event consequence in Column 5 (Used with permission from Phoenix Handbook, William Corcoran, NSRC) DOE-HDBK-1208-2012 1-28 • Did the causes behind this event sequence involve more than the group being investigated? How long has this event been going on without being detected or corrected? (Extent) • Did this event sequence involve a failed or missing barrier? How many barriers were in place? Effective barriers must be known and in use. Questions about barriers should be investigated. How many barriers were breached during this event? How many of the remaining barriers were either robust or fragile? Would the employee and the supervisor agree to your barrier analysis? (Barrier)

• How did this event sequence affect the credibility of the organization regarding areas of high interest to customers, regulators, and the corporation? (Campaign Item) In the “Significance of Difference” column of the CTL (Column 5), classify each causal factor as one of four types: • Vulnerability (a stagnation factor that sets the stage for the event to happen, i.e., a latent factor). Vulnerability factors affect the probability of an event occurring. Examples include susceptibility to weather, inadequate knowledge of hazards, incomplete pre-job brief, people absent from pre-job brief, unknown equipment deficiencies, doing too many things at once, informality of changes, inadequate or untimely CAs from a previous event, cutting corners after previous “successes,” management condoning inadequate employee performance by taking no previous action, and preventive maintenance backlog. • Triggering (an active factor that sets off the event). Triggering factors affect the probability of an event occurring. Examples include changes to previously agreed upon start-up procedures, personnel changes, design changes, test initiations, procedure changes, machinery starts, mode transitions, reorganizations, weather changes, and service condition changes. • Exacerbating (a factor that makes the event worse). Exacerbating factors affect the severity of the consequences. Examples include time of occurrence, reluctance to take appropriate radical actions, inadequate problem analysis before taking action, unknown equipment deficiencies, inadequate understanding of previously successful interventions, ill-conceived employee intervention, attempts to recover from an unrecoverable situation, and not recognizing the coupling between activities, processes, and equipment. • Mitigating (a factor that keeps the event from getting any worse). Mitigating factors affect the severity of the consequences. Examples include Initial Conditions (weather effects), time of occurrence, correct performance of safety devices, correct performance of inspections and tests to reveal latent vulnerabilities, inspections and tests that were conducted to detect one type of problem but uncovered another problem. The CTL will provide the team with new insights to scrutinize documents versus testimony and develop a continuing list of inconsistencies to be resolved during the investigation. DOE-HDBK-1208-2012 1-29 1.5.3.2 Determine Event Facts (∆Wg) Those events in the CTL that raise concern with the team because of glaring gaps between “work-as-done” and “work-as-imagined” that appeared to contribute to the event will be those event facts (∆Wg) that will be used to explore causal factors in the analysis phase of the OSR. 1.6 Investigate the Event(s) to Determine “Why” It Happened The purpose of any analytic technique is to answer the question — “WHY” the event happened. That is, why did the organization allow itself to degrade to such a state that the event in question happened? To begin this understanding (i.e., to develop causal factors), the CTL initially developed is expanded to collect important information related to human performance challenges, missed opportunities, organizational culture attributes, and potential latent organizational weaknesses. By collecting this important information for each time sequence, biases that the team members may have as they enter the investigation process are removed or at least minimized resulting in a much more objective investigation.

1.6.1 Expand Comparative Timeline During the Investigation Phase, the CTL was populated with event facts and associated gaps between work-as-imagined and work-as-done. This information included failed or missing barriers, whether the event step was a set-up, triggering, exacerbating or mitigating factor, and whether the event was localized or wide-spread throughout the organization. The CTL should now be expanded to include a review of the factors that may have contributed to the event. A sixth column is added to the CTL to allow the comparison of each event step to newly identified factors such as: • Human Performance Error Precursors (obtained from Figure 1-3 from Volume I, Chapter 1), • Human Error Mode (obtained from Volume I, Section 1.5.2 Human Performance Modes – Cognitive Demands), • Latent Organizational Weaknesses (obtained from Table 1-1 from Volume I, Chapter 1), • Missed Opportunities, • Culture Elements (obtained from Volume I, Section 1.10.5 Organizational Culture in Volume I, Chapter 1 and Volume II, Table 1-4). To populate the expanded the CTL, the OSR team places the previously mentioned information associated with each time sequence into column 6, as shown in Figure 1-10. The location the classification of human performance error modes, human error modes, and latent organizational weaknesses can be found in the references noted in Volume I, Chapter 1. Missed opportunities come from a reflection on the various management systems in place to look for missed opportunities to catch the event in its infancy to preclude the event now being investigated. The three safety culture factors that give indication of safety culture issues first introduced in Volume I, Chapter 1 are provided in the next section. See sample of a completed CTL in Attachment 1. 1. 6 DOE-HDBK-1208-2012 1-30 Figure 1-10: Expanded Comparative Timeline 1.6.1.1 Safety Culture Every modern report on major accidents that have occurred the last few years indicate unhealthy safety culture as a contributing factor (e.g., NASA Columbia, BP Texas City, BP Deepwater Horizon, Fukushima, etc.). This is a topic of safety culture is clearly important to understanding the deeper organizational issues that allowed the organization to drift to the condition that set up the event being investigated. To ensure the problem that gets fixed, stays fixed, an understanding of the organization’s safety culture must be addressed. The cultural-based assessment is accomplished by examining the three principal culture shaping factors (leadership, employee engagement, organizational learning introduced in Volume I, Chapter 1). Samples of lines of inquiry for each of the three factors can be found in Table 1-4. These questions are based on the seven guiding principles of DOE P 450.4A. A more detailed list of questions can be found in Attachment 1of this document. These lines of inquiry can be used by the team to determine whether culture deficiencies affected the event. The answers to Date/ Time What Happened What Should Have Happened Immediate Consequences of Difference Significance of Difference (impact on consequence) Other Factors 1 2 3 4 5 6 Original Investigation Event Timeline Significance of Difference • How did the difference change the final outcome? • Did it have a set-up, triggering, exacerbating or mitigating effect? • Was it a failed or missing barrier? • Is it a precursor to a worse event? • Is this a problem extensive or localized?

Qualify when Possible. Other Factors • DOE safety culture focus area (leadership, employee engagement, organizational learning), • Human error mode, • Human performance error precursors, • Missed opportunities, and • Latent organizational weaknesses. DOE-HDBK-1208-2012 1-31 the questions may be used to determine the facts of the event, which, along with the analytical tools described earlier will enable the team to determine whether deficiencies found in management systems and line management oversight, are causal factors for the incident. Table 1-4: Sample of Safety Culture Lines of Inquiry (Full table can be found in Attachment 1) LEADERSHIP SAFETY CULTURE ATTRIBUTE Line Management Responsibility for Safety Leadership and culture are two sides of the same coin; neither can be realized without the other. Leaders create and manage the safety culture in their organizations by maintaining safety as a priority, communicating their safety expectations to the workers, setting the standard for safety through actions not talk (walk-the talk), leading needed change by defining the current state, establishing a vision, developing a plan, and implementing the plan effectively. Leaders cultivate trust to engender active participation in safety and to establish feedback on the effectiveness of their organization’s safety efforts. SOURCE OF ATTRIBUTE FROM ISM GUIDE (DOE G 450.4-1C) • Leaders assure plans integrate safety into all aspects of an organization’s activities considering the consequences of operational decisions for the entire life-cycle of operations and the safety impact on business processes, the organization, the public, and the environment. 1.1 What type of evidence do you see that demonstrates line managers understand and accept their safety responsibilities inherent in mission accomplishment by not depending on supporting organizations to build safety into line management work activities? 1.2 What type of evidence do you see that demonstrates line managers regularly and promptly communicate important operational decisions, their basis, expected outcomes, potential problems, and planned contingencies? • Leaders understand their business and ensure the systems employed provide the requisite safety by identifying and minimizing hazards, proving the activity is safe, and not assuming it is safe before operations commence. 2.1 What type of evidence do you see that demonstrates line managers have a clear understanding of their work activities and their performance objectives, and how they will conduct their work activities safely while accomplishing their performance objectives? 2.2 What type of evidence do you see that demonstrates key technical managers are assigned for long terms of service to provide institutional continuity and constancy regarding safety requirements and expectations? 2.3 What type of evidence do you see that demonstrates defense in depth is designed into highly-hazardous operations and activities, and includes independent, redundant, and diverse safety systems, which are not overly complex? • Leaders consider safety implications in the change management processes. 3.1 What type of evidence do you see that demonstrates that line managers maintain a strong focus on the safe conduct of work activities? DOE-HDBK-1208-2012 1-32 LEADERSHIP • Leaders model, coach, mentor, and reinforce their expectations and behaviors to improve safe business performance.

4.1 What type of evidence do you see that demonstrates line managers are committed to safety? 4.2 What type of evidence do you see that demonstrates line managers spend time on the floor? • Leaders value employee involvement, encourage individual questioning attitude, and instill trust to encourage raising issues without fear of retribution. 5.1 What type of evidence do you see that demonstrates line managers are skilled in responding to employee questions in an open, honest manner? • Leaders assure employees are trained, experienced and have the resources, the time, and the tools to complete their job safely. 6.1 What type of evidence do you see that demonstrates staffing levels and capabilities are consistent with the expectation of maintaining safe and reliable operations? 6.2 What type of evidence do you see that demonstrates the organizational staffing provides sufficient depth and redundancy to ensure that all important safety functions are adequately performed? • Leaders hold personnel accountable for meeting standards and expectations to fulfill safety responsibilities. 7.1 Are responsibility and authority for safety well defined and clearly understood as an integral part of performing work. 7.2 What type of evidence do you see that demonstrates organizational safety responsibilities are sufficiently comprehensive to address the work activities and hazards involved? • Leaders insist on conservative decision making with respect to proven safety system and recognize that production goals, if not properly considered and clearly communicated, can send mixed signals on importance of safety. 8.1 Do managers recognize that aggressive mission and production goals can appear to send mixed signals on the importance of safety? • Leadership recognizes that humans make mistakes and take actions to mitigate this. 9.1 What type of evidence do you see that demonstrates hazard controls are designed with an understanding of the potential for human error? DOE-HDBK-1208-2012 1-33 EMPLOYEE INVOLVEMENT SAFETY CULTURE ATTRIBUTE Individual Attitude and Responsibility for Safety Safety is everyone’s responsibility. As such, employees understand and embrace the organization’s safety behaviors, beliefs, and underlying assumptions. Employees understand and embrace their responsibilities, maintain their proficiency so that they speak from experience, challenge what is not right and help fix what is wrong and police the system to ensure they, their co-workers, the environment, and the public remain safe. SOURCE OF ATTRIBUTE FROM ISM GUIDE (DOE G 450.4-1C) 1. Individuals team with leaders to commit to safety, to understand safety expectations, and to meet expectations. 1.1 What type of evidence do you see that demonstrates personnel at all levels of the organization are held accountable for shortfalls in meeting standards and expectations related to fulfilling safety responsibilities? 1.2 What type of evidence do you see that demonstrates that individuals understand and demonstrate responsibility for safety? 2. Individuals work with leaders to increase the level of trust and cooperation by holding each other accountable for their actions with success evident by the openness to raise and resolve issues in a timely fashion. 2.1 What type of evidence do you see that demonstrates individuals promptly report errors and incidents? 2.2 What type of evidence do you see that demonstrates that individuals are systematic and rigorous in making informed decisions that support safe, reliable operations?

3. Everyone is personally responsible and accountable for safety, they learn their jobs, they know the safety systems and they actively engage in protecting themselves, their co-workers, the public and the environment. 3.1 What type of evidence do you see that demonstrates personnel at all levels of the organization are held accountable for shortfalls in meeting standards and expectations related to fulfilling safety responsibilities? 3.2 What type of evidence do you see that demonstrates people and their professional capabilities, experiences, and values are regarded as the organization’s most valuable assets? 3.3 Does the organization maintain a highly knowledgeable workforce to support a broad spectrum of operational and technical decisions? 4. Individuals develop healthy skepticism and constructively question deviations to the established safety system and actively work to avoid complacency or arrogance based on past successes. 4.1 What type of evidence do you see that demonstrates individuals cultivate a constructive, questioning attitude and healthy skepticism when it comes to safety? 4.2 What type of evidence do you see that demonstrates individuals are aware of and counteract human tendencies to simplify assumptions, expectations, and analysis? 5. Individuals make conservative decisions with regards to the proven safety system and consider the consequences of their decisions for the entire life-cycle of operations. 5.1 What type of evidence do you see that demonstrates individuals are mindful of the potential impact of equipment and process failures; they are sensitive to the potential of faulty assumptions and errors, and demonstrate constructive skepticism? 5.2 What type of evidence do you see that demonstrates individuals recognize that errors and imperfections are likely to happen? 5.3 What type of evidence do you see that demonstrates individuals are systematic and rigorous in making informed decisions that support safe, reliable operations? DOE-HDBK-1208-2012 1-34 EMPLOYEE INVOLVEMENT 6. Individuals openly and promptly report errors and incidents and don’t rest until problems are fully resolved and solutions proven sustainable. 6.1 What type of evidence do you see that demonstrates individuals promptly report errors and incidents? 7. Individuals instill a high level of trust by treating each other with dignity and respect and avoiding harassment, intimidation, retaliation, and discrimination. Individuals welcome and consider a diversity of thought and opposing views. 7.1 What type of evidence do you see that demonstrates individuals cultivate a constructive, questioning attitude and healthy skepticism when it comes to safety? 7.2 What type of evidence do you see that demonstrates individuals are aware of and counteract human tendencies to simplify assumptions, expectations, and analysis? 8. Individuals help develop healthy collaborative relationships within their organization and between their organization and regulators, suppliers, customers and contractors. ORGANIZATIONAL LEARNING SAFETY CULTURE ATTRIBUTE Organizational Learning for Performance Improvement The organization learns how to positively influence the desired behaviors, beliefs and assumptions of their healthy safety culture. The organization acknowledges that errors are a way to learn by rewarding those that report, sharing what is wrong, fixing what is broken and addressing the organizational setup factors that led to employee error. This requires focusing on reducing recurrences by correcting deeper, more systemic causal factors and systematically monitoring performance and interpreting results to generate decision-making information on the health of the system.

SOURCE OF ATTRIBUTE FROM ISM GUIDE (DOE G 450.4-1C) 1. The organization establishes and cultivates a high level of trust; individuals are comfortable raising, discussing and resolving questions or concerns. 1.1 What type of evidence do you see that demonstrates credibility and trust are present and continuously nurtured? 1.2 What type of evidence do you see that demonstrates that open communications and teamwork are the norm? 1.3 What type of evidence do you see that demonstrates a high level of trust is established in the organization? 2. The organization provides various methods to raise safety issues without fear of retribution, harassment, intimidation, retaliation, or discrimination. 2.1 What type of evidence do you see that demonstrates systems of checks and balances are in place and effective at all levels of the organization to make sure that safety considerations are adequately weighed and prioritized? 2.2 What type of evidence do you see that demonstrates processes are established to identify and resolve latent organizational weaknesses that can aggravate relatively minor events if not corrected? DOE-HDBK-1208-2012 1-35 ORGANIZATIONAL LEARNING 3. Leaders reward learning from minor problems to avoid more significant events. 3.1 What type of evidence do you see that demonstrates the organization actively and systematically monitors performance through multiple means, including leader walk- arounds, issue reporting, performance indicators, trend analysis, benchmarking, industry experience reviews, self- assessments, and performance assessments? 3.2 Are frequent incident reviews conducted promptly after an incident to ensure data quality to identify improvement opportunities? 4. Leaders promptly review, prioritize, and resolve problems, track long-term sustainability of solutions, and communicate results back to employees. 4.1 What type of evidence do you see that demonstrates that line managers a strong focus on the safe conduct of work activities? 4.2 What type of evidence do you see that demonstrates vigorous corrective and improvement action programs are in place and effective? 5. The organization avoids complacency by cultivating a continuous learning/improvement environment with the attitude that “it can happen here.” 5.1 What type of evidence do you see that demonstrates operational anomalies, even small ones, get prompt attention and evaluation – this allows early detection of problems so necessary action is taken before problems grow? 5.2 Are candid dialogue and debate and a healthy skepticism encouraged when safety issues are being evaluated? 6. Leaders systematically evaluate organizational performance using: workplace observations, employee discussions, issue reporting, performance indicators, trend analysis, incident investigations, benchmarking, assessments, and independent reviews. 6.1 What type of evidence do you see that demonstrates line managers are in close contact with the front-line; they pay attention to real-time operational information? 6.2 What type of evidence do you see that demonstrates organizations know the expertise of their personnel? 7. The organization values learning from operational experience from both inside and outside the organization. 7.1 What type of evidence do you see that demonstrates operating experience is highly valued, and the capacity to learn from experience is well developed? 8. The organization willingly and openly engages in organizational learning activities.

8.1 What type of evidence do you see that demonstrates line managers throughout the organization set an example for safety through their direct involvement in continuous learning by themselves and their followers on topics related to technical understanding and safety improvement? 8.2 What type of evidence do you see that demonstrates training to broaden individual capabilities and to support organizational learning is available and encouraged? 1.6.2 Consolidate Indicators of Deeper Organizational Factors Having listed indicators of culture, human performance, latent organizational weaknesses at each of the time steps in the CTL, the team consolidates in one location the various factors to allow DOE-HDBK-1208-2012 1-36 comparison against similar factors observed by other organizations to allow extrapolation to high-level organizational issues. The tools to accomplish this include: • Culture Attribute Matrix (CAM) • Missed Opportunity Matrix (MOM) • Human Error Precursor Matrix (TWIN) • Latent Organizational Weakness Table (LOW) There is nothing magical or prescriptive about these matrices. To provide an example of how an organization could go about presenting the information, examples are provided in Attachment 1. 1.6.3 Determine Causal Factors Accidents rarely result from a single cause. This is particularly true if we are investigating a system accident because many, hopefully independent systems and barriers were put in place to ensure the catastrophic event did not occur. And hence, if an incident occurred, it had to be a result of the breakdown in multiples systems. Events and causal factors charting is useful in identifying the multiple causes and graphically depicting the triggering conditions and events necessary and sufficient for an incident to occur. This is useful in an OSR where multiple events are charted for comparison in looking for common system or organizational weaknesses. Causal factors charting is a graphical display of the event and is used primarily for compiling and organizing evidence to portray the sequence of the events and their causal factors that led to the incident. The earlier analytical techniques (e.g., CTL, process mapping, barrier analysis, and change analysis) are used to inform the team and to support the development of the events and causal factors chart. After the major event facts are fully identified, analysis is performed to identify the causal factors. Causal factors charting is widely used in major event investigations, because it is relatively easy to develop and provides a clear depiction of the information generated by the team. By carefully tracing the events and conditions that allowed the incident to occur, team members can pinpoint specific events and conditions that, if addressed through corrective actions, would prevent a recurrence. 1.6.3.1 Causal Factors Analysis (CFA) Charting An alternative method to determining and explaining event causal factors that was introduced in Volume I, Chapter 2 is the Causal Factors Analysis (CFA) chart. This chart has been developed to fully support the methodology introduced in Volume I, Chapters 1 and 2 to determine first “what” went wrong then focus on “why” it went wrong. 1.6.3.1.1 Populate the “What” Factors on CFA Chart The first three sections of the CFA Chart (Figure 1-11) that should be filled out during the investigation stage include: Step 1 - Initial conditions Step 2 - Consequence and final event DOE-HDBK-1208-2012

1-37 Step 3 - Irrefutable chain of events At the top of the CFA Chart (Figure 1-11) are the initial existing conditions, at the time of the event, that were intended to succeed. Examples of initial conditions include procedures, equipment, personnel, level of training, etc.; that are the normal starting point before beginning the type of hazardous work. This does not ensure success of the intended operation, but rather suggests something was in place that indicated success was achievable, based on prior experience. Next, the final consequences are listed in the hexagon, the final event in the preceding rectangle, and the direct cause of the final event are listed in the next rectangle (Figure 1-11). Using the significant physical events identified earlier (ΔWg), the team populates the CFA event facts (“Whats #1, #2, #3 in Figure 1-11). The goal is to develop a systematic description of initiating events that led to the final event on the staircase. The description should contain unimpeachable evidence for each physical step (a full engineering analysis and proof for each). When filling in the event facts (ΔWg • Each event (primary event) should be active and stated as a noun and one verb and quantified whenever possible (include date and time of event when known). ), work backward from the final event/direct causes (this will be an iterative process). • Examine the significance of the first event that immediately precedes the final event in the final event sequence by asking, “Is this our initiating event? If this event had not occurred, would the final event have occurred?” If the answer is yes, the event is not significant, and should not be in the event fact chain. Proceed to the next event on the timeline (continually working backward from the final event). • If the final event would not have occurred without the event in question, then ask follow-on questions to determine if the event in question represented normal activity with expected consequences. If the event in question was intended and produced the expected outcomes, then it is not significant and should not be in the Event Fact Chain. However, if the event deviated from what was intended or had unwanted consequences, then it is a significant event and should remain in the Event Fact Chain. • Each subsequent event identified should be examined using the same significance test. Each subsequent event should be derived from the event and conditions immediately preceding it. Secondary events and those conditions that contribute to the primary event should be denoted on the right of the primary event in the CFA Chart and will be developed as part of the Analysis section of the investigation. As a helpful tool, list in pencil the key issues, themes, and factors from the detailed CTL and the sticky notes next to each event fact in the CFA Chart. Use Team Interview notes and develop lines of questioning such as, “Did inadequate procedures fail us? Did we follow the procedures? Did training fail us? Did expert knowledge let us down? Was it an equipment failure? Was it something else?” • To be significant, the factors must be key contributors to the event. DOE-HDBK-1208-2012 1-38 • Separate those factors that contributed to the event from those that are important findings, but did not directly contribute to the event [those important findings that did not contribute to the event but need to be fixed are called Extraneous Conditions Adverse to Quality (ECAQs)]. [Corcoran, 2003]

• Summarize or collect similar events into the proper level of categorization, which is determined by the ability of CFAs to effectively address all issues within the category. Re- interview as more questions arise, and repeat the process for each final event. Figure 1-11: Causal Factors Analysis (CFA) Chart 1.6.3.1.2 Populate the “Why” Factors on CFA Chart For each physical event identified in the Investigation Phase (what happened), the associated organizational factors (why and how they happened) are listed on the CFA Chart (Figure 1-11). For each sequential event fact, ask the question, “Does the causal factor affect the nature, magnitude, or timing of the consequence, either adversely or beneficially?” Each factor obtained in this line of questioning is placed to the side of the corresponding event fact. Continue the line of questioning until the factors that answer the question appear less important than the preceding factor. [Corcoran, 2003] The further one gets from an event in time and space, the more obvious the organizational causal factors become (Figure 1-11). As the investigation continues, latent organizational weaknesses become apparent. Some latent organizational weaknesses may have been present for several years before the event occurred. IC #1 Hazards IC #2 Threats IC #3 Barriers IC #4 What #1 What #2 What #3 Initiating Event Direct Cause Final Event Condition and Cause Condition and Cause Condition and Cause Condition and Cause Condition and Cause Condition and Cause Condition and Cause Consequence Why? Why? Why? Why? Why? Why? Why? Initial Conditions generated from work-as-planned Significant ∆Wg factors generated from comparison of work-as- planned to work-as-done ∆Wg Direct cause and final event obtained from work-as-done 1.7 DOE-HDBK-1208-2012 1-39 1.7 Developing Conclusions and Judgments of Need to Improve Organizational Safety Systems CONs and JONs are key elements of the OSR that must be developed by the OSR team. Additional useful instructions on developing the CONs and JONs can be found in Volume I, Section 2.7. 1.7.1 Conclusions Conclusions are significant deductions derived from the OSR team’s analytical results. They are derived from and must be supported by the facts plus the results of testing and the various analyses conducted. Conclusions may: • Include concise statements of the common causal factors of the events determined by analysis of facts and the extent of conditions. • Be statements that alleviate potential confusion on issues that were originally suspected causes. • Address significant concerns arising out of the events that are unsubstantiated or inconclusive. • Be used to highlight positive aspects of performance revealed during the review, where appropriate. 1.7.2 Judgments of Need JONs are the managerial controls and safety measures determined by the OSR team to be necessary to prevent or minimize the probability or severity of future occurrences. JONs should be linked to causal factors and logically flow from the conclusions. They should be: • Stated in a clear, concise, and direct manner, • Based on the facts/evidence, • Stated so that they can be the basis for corrective action plans. 1.8 Reporting the Results

The purpose of the OSR report is to clearly and concisely convey the results of the review. The report contents will help the reader understand what kinds of events have been happening (the descriptions and chronologies), why they have been happening (the causal factors and conditions). Most importantly, the report concisely presents what can be done effectively to reduce these types of events and reduce the potential for more serious consequences (the Judgments of Need). OSR results are reported without attributing individual fault or proposing punitive measures. 1. 8 DOE-HDBK-1208-2012 1-40 • The OSR report constitutes an accurate and objective record of the events and provides complete and accurate details and explicit statements of: • The OSR team’s review process, • Facts pertaining to the events, including relevant management systems involved, • Analytical methods used and their results, • Conclusions of the OSR team, including the causal factors of the events, • Judgments of Need for corrective actions to reduce these types of events and reduce the potential for more serious consequences. When completed, this report is submitted to the appointing official for acceptance and dissemination. Additional useful instructions on writing the report can be found in Volume I, Section 2.8. 1.9 Performing Verification Analysis, Quality Review and Validation of Conclusions Before releasing the report outside the OSR team, the team reviews it to ensure its technical accuracy, thoroughness, and consistency, and to ensure that organizational concerns, safety management systems, and line management oversight processes are properly analyzed as possible organizational safety system weaknesses. The OSR Chairperson should plan and schedule sufficient time for this review to maintain the appropriate investigation cycle. The following are review considerations for quality of the report as covered in Volume I, Section 2.8. Most important, maintain open dialog about the content and direction of the review. Ongoing impacted managers’ awareness and emphasizing system or process improvement without blame will help generate support for the JONs. • Structure and Format • Technical and Policy Issues • Verification Analysis • Classification and Privacy Review • Factual Accuracy Review 1.9.1 Review by the Health, Safety, Quality, and Security Management The Chairperson or Appointing Official should coordinate a review of the OSR report by the organization’s Health, Safety, Quality, and Security Management. OSR reports are reviewed prior to acceptance by the appointing official. Comments are provided to the appointing official for incorporation prior to report publication and distribution. 1.9.2 Document the Reviews in the Records Documentation that these reviews have been completed should be retained in the permanent OSR records file. For addition guidance on the closeout of the OSR see Volume I, Section 2.3.8, 1.9 1.10 DOE-HDBK-1208-2012 1-41 “Managing Onsite Closeout Activities.” However, for contractor-led OSRs, closeout activity and record archiving should be according to the contractor’s policies. 1.10 Submitting the Report Once the report has been finalized, the OSR Chairperson provides the draft final report to the appointing official for acceptance. If the appointing official determines that the OSR team has met its obligation to conduct a thorough review and analysis of the organizational conditions, that the report fully describes the weaknesses and their causal factors, and that it provides Judgments of Need sufficient to improve safety performance, the report is formally accepted. The statement of report acceptance from the appointing official is included in the final report (see Volume I, Section 2.10).

DOE-HDBK-1208-2012 1-42 DOE-HDBK-1208-2012 Appendix A. Glos s ary DOE-HDBK-1208-2012 DOE-HDBK-1208-2012 A-1 Glossary Accident: An unwanted transfer of energy or an environmental condition that, due to the absence or failure of barriers or controls, produces injury to persons, damage to property, or reduction in process output. Accident Investigation: The systematic appraisal of unwanted events for the purpose of determining causal factors, subsequent corrective actions, and preventive measures. Accident or Emergency Response Team: A team or teams of emergency and accident response personnel for a particular site. This team may be composed of a number of teams from the site, such as local police and firefighter units, emergency medical personnel, and hazardous material teams. Analysis: The use of methods and techniques for arranging data to: (a) assist in determining what additional data are required; (b) establish consistency, validity, and logic; (c) establish necessary and sufficient events for causes; and (d) guide and support inferences and judgments. Analytical Tree: Graphical representation of an accident in a deductive approach (general to specific). The structure resembles a tree—that is, narrow at the top with a single event (accident) and then branching out as the tree is developed, and identifying root causes at the bottom branches. Appointing Official: A designated authority responsible for assigning Accident Investigation Boards for investigations, with responsibilities as prescribed in DOE O 225.1B. Barrier: Anything used to control, prevent, or impede energy flows. Common types of barriers include equipment, administrative procedures and processes, supervision/management, warning devices, knowledge and skills, and physical objects. Barrier Analysis: An analytical technique used to identify energy sources and the failed or deficient barriers and controls that contributed to an accident. Board Chairperson: The leader who manages the accident investigation process, represents DOE in all matters regarding the accident investigation, and reports to the appointing official for purposes of the accident investigation. Board Members: A group of three to six DOE staff assigned to investigate an accident. This group reports to the Board Chairperson during the accident investigation. Causal Factor: An event or condition in the accident sequence necessary and sufficient to produce or contribute to the unwanted result. Causal factors fall into three categories: • Direct cause • Contributing cause • Root cause. DOE-HDBK-1208-2012 A-2 Cause: Anything that contributes to an accident or incident. In an investigation, the use of the word “cause” as a singular term should be avoided. It is preferable to use it in the plural sense, such as “causal factors,” rather than identifying “the cause.” Chain of Custody: The process of documenting, controlling, securing, and accounting for physical possession of evidence, from initial collection through final disposition. Change: Stress on a system that was previously in a state of equilibrium, or anything that disturbs the planned or normal functioning of a system. Change Analysis: An analytical technique used for accident investigations, wherein accident- free reference bases are established, and changes relevant to accident causes and situations are systematically identified. In change analysis, all changes are considered, including those initially considered trivial or obscure.

Conclusions: Significant deductions derived from analytical results. Conclusions are derived from and must be supported by the facts, plus results from testing and analyses conducted. Conclusions are statements that answer two questions the accident investigation addresses: what happened and why did it happen? Conclusions include concise recapitulations of the causal factors (direct, contributing, and root causes) of the accident determined by analysis of facts. Contributing Cause: An event or condition that collectively with other causes increases the likelihood of an accident but that individually did not cause the accident. Controls: Those barriers used to control wanted energy flows, such as the insulation on an electrical cord, a stop sign, a procedure, or a safe work permit. Critical Process Step: A step in the process where potential threats could interact with the hazard that could be released. For accident analysis, the absence of hazards or threads in a process step makes it a non-critical step. Direct Cause: The immediate events or conditions that caused the accident. DOE Accident Investigator: An individual who understands DOE accident investigation techniques and has experience in conducting investigations through participation in at least one Federal investigation. Effective October 1, 1998, DOE accident investigators must have attended an accident investigation course of instruction that is based on current materials developed by the Office of Corporate Safety Programs. DOE Operations: Activities funded by DOE for which DOE has authority to enforce environmental protection, safety, and health protection requirements. DOE Site: A tract either owned by DOE, leased, or otherwise made available to the Federal government under terms that afford DOE rights of access and control substantially equal to those it would possess if it held the fee (or pertinent interest therein) as agent of and on behalf of the government. One or more DOE operations/program activities carried out within the boundaries of the described tract. DOE-HDBK-1208-2012 A-3 Energy: The capacity to do work and overcome resistance. Energy exists in many forms, including acoustic, potential, electrical, kinetic, thermal, biological, chemical, and radiation (both ionizing and non-ionizing). Energy Flow: The transfer of energy from its source to some other point. There are two types of energy flows: wanted (controlled—able to do work) and unwanted (uncontrolled—able to do harm). Event: An occurrence; something significant and real-time that happens. An accident involves a sequence of events occurring in the course of work activity and culminating in unintentional injury or damage. Events and Causal Factors Chart: Graphical depiction of a logical series of events and related conditions that precede the accident. Eyewitness: A person who directly observed the accident or the conditions immediately preceding or following the accident. Fatal Injury: Any injury that results in death within 30 calendar days of the accident. Field Element: A general term for all DOE sites (excluding individual duty stations) located outside the Washington, D.C., metropolitan area. General Witness: A person with knowledge about the activities prior to or immediately after the accident (the previous shift supervisor or work controller, for example). Hazard: The potential for energy flow(s) to result in an accident or otherwise adverse consequence.

Heads of Field Elements: First-tier field managers of the operations offices, the field offices, and the power marketing administrations (administrators). Human Factors: The study of human interactions with products, equipment, facilities, procedures, and environments used in work and everyday living. The emphasis is on human beings and how the design of equipment influences people. Investigation: A detailed, systematic search to uncover the “who, what, when, where, why, and how” of an occurrence and to determine what corrective actions are needed to prevent a recurrence. Investigation Report: A clear and concise written account of the investigation results. Judgments of Need: Managerial controls and safety measures necessary to prevent or minimize the probability or severity of a recurrence of an accident. Lessons Learned: A “good work practice” or innovative approach that is captured and shared to promote its repeated application. A lesson learned may also be an adverse work practice or experience that is captured and shared to avoid recurrence. DOE-HDBK-1208-2012 A-4 Occurrence: An event or condition that adversely affects or may adversely affect DOE or contractor personnel, the public, property, the environment, or DOE mission. Occurrence Reporting and Processing System (ORPS): The reporting system established and maintained for reporting occurrences related to the operation of DOE facilities. Operational Safety Analysis (OSA): is defined as the application of analytical methods to understand the potential consequences to life, health, property, or environment, caused by failure, due to human performance, or an element of a safety management system, within an operational environment. Point of Contact: A DOE staff member who is assigned the role of liaison with the Accident Investigation Program Manager in the Office of Corporate Safety Programs (HS-23), who administers the accident investigation program. In this role, the point of contact ensures that DOE site teams are trained in collecting and maintaining initial accident investigation evidence and that their activities are coordinated with accident and emergency response teams. Principal Witness: A person who was actually involved in the accident. Socio-technical: refers to the interrelatedness of social and technical aspects of an organization using the principle that the interaction of social and technical factors creates the conditions for successful (or unsuccessful) organizational performance. Verification Analysis: A validation technique that determines whether the logical flow of data from analysis to conclusions and Judgments of Need is based on facts. This technique is conducted after all the analyses are completed. Root Cause: The causal factor(s) that, if corrected, would prevent recurrence of the accident. Root Cause Analysis: Any methodology that identifies the causal factors that, if corrected, would prevent recurrence of the accident. Target: A person, object, or animal upon which an unwanted energy flow may act to cause damage, injury, or death. Threat: An action or force from human error, equipment malfunctions, operational process malfunctions, facility malfunctions or from natural disasters that could cause or trigger a hazardous energy release. DOE-HDBK-1208-2012 Appendix B. References DOE-HDBK-1208-2012 DOE-HDBK-1208-2012 B-1 References DOE Order 210.2A, DOE Corporate Operating Experience Program, August 8, 2011

DOE Order 225.1B, Accident Investigations, March 4, 2011. DOE Order 232.2, Occurrence Reporting and Processing of Operations Information, August 30, 2011 DOE Order 231.1B, Environment, Safety and Health Reporting, June 27, 2011 DOE Order 360.1C, Federal Employee Training, July 8, 2011. DOE Order 422.1, Conduct of Operations, June 29, 2010 DOE Order 450.2, Integrated Safety Management, April 25, 2011 DOE Guide 450.4-1C, Integrated Safety Management System Guide; September 29, 2011. DOE Policy 450.4A, Integrated Safety Management Policy, April 25, 2011 DOE Standard (STD)-HDBK-1028-2009 Human Performance Improvement Handbook, Volumes 1 and 2, June 2009 DOE Accident Investigation Electronic Reference Tool, January 2011 Accident Investigation Day Planner, a Guide for Accident Investigation Board Chairpersons DOE-STD-1146-2007, General Technical Qualification Standard, Section 5.1., December 2007 DOE-STD-1160-2003, Occupational Safety Functional Area Qualification Standard Competencies, Section 1.4, March 2003. Center for Chemical Process Safety, Guidelines for Investigating Chemical Process Incidents, American Institute of Chemical Engineers, New York, New York, March 2003. Defense Nuclear Federal Safety Board, Tech 35 – Safety Management of Complex, High-Hazard Organizations, December 2004 https://www.directives.doe.gov/directives/0210.2-BOrder-a� https://www.directives.doe.gov/directives/0232.2-BOrder� https://www.directives.doe.gov/directives/current-directives/231.1-BOrder-b/view� DOE-HDBK-1208-2012 B-2 DOE-HDBK-1208-2012 Attachment 1. Reference Examples and Supplemental Material DOE-HDBK-1208-2012 DOE-HDBK-1208-2012 1-1 Crosswalk between ISM Core Functions and the Break-the-Chain Framework ISM Core Function Break-the-Chain Framework CF #1: Define Scope Work is clearly defined, including the boundaries, priority, resources required and expectations for completion. The level of detail required in the work scope is commensurate with the importance and complexity of the work and the potential risk, the associated hazards, and the controls needed to mitigate hazards. BTC Step #1: Identify the Consequence to Avoid Catastrophic consequences are listed in priority order to: • Remind everyone of the potential catastrophic consequences to avoid each day • Pinpoint where barriers are most needed; the severity of the consequences will drive the number and type of barriers selected • Ensure barriers protecting highest priority consequences receive top protection against degradation • Encourage constant review of resources against consequences, to ensure the most severe consequences are avoided at all times and at any cost. Efforts to protect against catastrophic events should never be diluted by an organization’s efforts to prevent less-consequential events. Focus must be maintained on system accidents to assure that the needed attention and resources are available to prevent them. CF #2: Identify Hazards Task-level, or work planning control, identifies hazards tailored to the work performed. It identifies hazards with the potential to harm workers, the facilities or the environment. Pantex provides each worker with an awareness of their work place hazards. BTC Step #2: Identify the Hazard to Protect and Minimize • Identify the hazard • Minimize the hazard • Reduce interactive complexity and tight coupling DOE-HDBK-1208-2012 1-2 ISM Core Function Break-the-Chain Framework

CF #3: Develop and Implement Hazard Controls Controls identified and tailored as appropriate to adequately address the hazards identified with the work. Provide each worker an awareness of the controls hat protect their safety from identified hazards. Implement controls in a manner that is sufficient to ensure they sufficiently accomplish their intent. BTC Step #3: Reduce Threats • Identify and reduce threats from human error, faulty equipment, tooling, facilities, and from natural BTC Step #4: Manage Defenses • Manage Defenses to Reduce the Probability of the Systems Accident • Manage Defenses to Mitigate the Consequences of the System Accident CF #4: Perform Work Within Controls Supervisors evaluate work packages, before starting work, to ensure controls are in place to mitigate hazards. Work is performed in accordance with identified controls and evaluated to indicate how safely work is performed. BTC Step #5: Reduce Vulnerability to the Hazard Through Strong Culture of Reliability • Understand concept of culture of reliability, how its measured, enhanced and sustained • Demonstrate conservative operational decisions with regards to the selected safety system CF #5: Feedback and Improvement Mechanisms (including independent means) collect data and to generate information to make improvements to all phases of planning and conducting the work safely. BTC Step #6: Minimize Gap Between Work- as-Imagined & Work-as-Done • Ensure BTC framework effective at process start-up • Encourage worker-supervisor interactions • Track and trend performance indicators • Perform Causal Factors Analyses on “information-rich” events • Learn from other people’s mistakes DOE-HDBK-1208-2012 1-3 Safety Culture Lines of Inquiry Addressing the Seven ISM Guiding Principles (Developed by EFCOG Safety Culture Task Group) LEADERSHIP SAFETY CULTURE ATTRIBUTE Line Management Responsibility for Safety Leadership and culture are two sides of the same coin; neither can be realized without the other. Leaders create and manage the safety culture in their organizations by maintaining safety as a priority, communicating their safety expectations to the workers, setting the standard for safety through actions not talk (walk-the talk), leading needed change by defining the current state, establishing a vision, developing a plan, and implementing the plan effectively. Leaders cultivate trust to engender active participation in safety and to establish feedback on the effectiveness of their organization’s safety efforts. SOURCE OF ATTRIBUTE FROM ISM GUIDE (DOE G 450.4-1C) 1. Leaders assure plans integrate safety into all aspects of an organization’s activities considering the consequences of operational decisions for the entire life-cycle of operations and the safety impact on business processes, the organization, the public, and the environment. 1.1 What type of evidence do you see that demonstrates line managers understand and accept their safety responsibilities inherent in mission accomplishment by not depending on supporting organizations to build safety into line management work activities? 1.2 What type of evidence do you see that demonstrates line managers regularly and promptly communicate important operational decisions, their basis, expected outcomes, potential problems, and planned contingencies? 2. Leaders understand their business and ensure the systems employed provide the requisite safety by identifying and minimizing hazards, proving the activity is safe, and not assuming it is safe before operations commence.

2.1 What type of evidence do you see that demonstrates line managers have a clear understanding of their work activities and their performance objectives, and how they will conduct their work activities safely and accomplish their performance objectives? 2.2 What type of evidence do you see that demonstrates key technical managers are assigned for long terms of service to provide institutional continuity and constancy regarding safety requirements and expectations? Is organizational knowledge valued and efforts made to preserve it when key players move on? 2.3 What type of evidence do you see that demonstrates facilities are designed, constructed, operated, maintained, and decommissioned using consensus industry codes and standards, where available and applicable, to protect workers, the public, and the environment? 2.4 What type of evidence do you see that demonstrates applicable requirements from laws, statutes, rules and regulations are identified and captured so that compliance can be planned, expected, demonstrated, and verified? 2.5 What type of evidence do you see that demonstrates clear, concise technical safety directives are centrally developed, where necessary, and are based on sound engineering judgment and data? Are DOE directives and technical standards actively maintained up to date and accurate? 2.6 What type of evidence do you see that demonstrates a clearly-defined set of safety requirements and standards is invoked in management contracts, or similar agreements? DOE-HDBK-1208-2012 1-4 LEADERSHIP Are accepted process used for identification of the appropriate set of requirements and standards? And is this set of requirements is comprehensive and do they include robust quality assurance, safety, and radiological and environmental protection requirements? 2.7 What type of evidence do you see that demonstrates implementing plans, procedures and protocols are in place to translate requirements into action by the implementing organization? 2.8 What type of evidence do you see that demonstrates technical and operational safety requirements clearly control the safe operating envelope? Is the safety envelope clearly specified and communicated to individuals performing operational tasks? 2.9 What type of evidence do you see that demonstrates exemptions from applicable technical safety requirements are both rare and specific, provide an equivalent level of safety, have a compelling technical basis, and are approved at an appropriate organizational level? 2.10 What type of evidence do you see that demonstrates compliance with applicable safety and technical requirements is expected and verified? 2.11 What type of evidence do you see that demonstrates willful violations of requirements are rare, and personnel and organizations are held strictly accountable in the context of a just culture? Are unintended failures to follow requirements are promptly reported, and personnel and organizations are given credit for self-identification and reporting of errors? How do you really know? 2.12 What type of evidence do you see that demonstrates the organization actively seeks continuous improvement to safety standards and requirements through identification and sharing of effective practices, lessons learned, and applicable safety research? What type of evidence do you see that demonstrates the organization committed to continuously rising standards of excellence? 2.13 What type of evidence do you see that demonstrates work hazards are identified and controlled to prevent or mitigate accidents, with particular attention to high consequence events with unacceptable consequences? Through your interviews and direct interactions, do the workers understand hazards and controls before beginning work activities? 2.14 What type of evidence do you see that demonstrates the selection of hazard controls considers the type of hazard, the magnitude of the hazard, the type of work being performed, and the life-cycle of the facility? Are these controls designed, implemented, and maintained commensurate with the inherent level and type of hazard? 2.15 What type of evidence do you see that demonstrates

DOE-HDBK-1208-2012 1-5 LEADERSHIP safety analyses identifying work hazards are comprehensive and based on sound engineering judgment and data? 2.16 What type of evidence do you see that demonstrates defense in depth is designed into highly-hazardous operations and activities, and includes independent, redundant, and diverse safety systems, which are not overly complex? Do defense in depth controls include engineering controls, administrative processes, and personnel staffing and capabilities? 2.17 What type of evidence do you see that demonstrates emphasis is placed on designing the work and/or controls to reduce or eliminate the hazards and to prevent accidents and unplanned releases and exposures? 2.18 What type of evidence do you see that demonstrates the following hierarchy of defense in depth is recognized and applied: (1) elimination or substitution of the hazards, (2) engineering controls, (3) work practices and administrative controls, and (4) personal protective equipment? Are inherently safe designs preferred over ones requiring engineering controls? Is prevention emphasized in design and operations to minimize the use of, and thereby possible exposure to, toxic or hazardous substances? 2.19 What type of evidence do you see that demonstrates equipment is consistently maintained so that it meets design requirements? 2.20 What type of evidence do you see that demonstrates safety margins are rigorously maintained? What type of evidence do you see that demonstrates design and operating margins are carefully guarded and changed only with great thought and care? Is special attention placed on maintaining defense-in-depth? 2.21 What type of evidence do you see that demonstrates organizations implement hazard controls in a consistent and reliable manner? Is safety embedded in processes and procedures through a functioning formal integrated safety management system? Are facility activities governed by comprehensive, efficient, high-quality processes and procedures? 2.22 What type of evidence do you see that demonstrates formal facility authorization agreements are in place and maintained between owner and operator? 2.23 What type of evidence do you see that demonstrates readiness at the facility level is verified before hazardous operations commence? Are pre-operational reviews used to confirm that controls are in place for known hazards? 2.24 What type of evidence do you see that demonstrates facility operations personnel maintain awareness of all facility activities to ensure compliance with the established safety envelope? 2.25 What type of evidence do you see that demonstrates DOE-HDBK-1208-2012 1-6 LEADERSHIP work authorization is defined at the activity level? Does the work authorization process verify that adequate preparations have been completed so that work can be performed safely? Do these preparations include verifying that work methods and requirements are understood; verifying that work conditions will be as expected and not introduce unexpected hazards; and verifying that necessary controls are implemented? 2.26 What type of evidence do you see that demonstrates the extent of documentation and level of authority for work authorization is based on the complexity and hazards associated with the work? 3. Leaders consider safety implications in the change management processes.

3.1 What type of evidence do you see that demonstrates line managers maintain a strong focus on the safe conduct of work activities? What type of evidence do you see that demonstrates line managers maintain awareness of key performance indicators related to safe work accomplishment, watch carefully for adverse trends or indications, and take prompt action to understand adverse trends and anomalies? 4. Leaders model, coach, mentor, and reinforce their expectations and behaviors to improve safe business performance. 4.1 What type of evidence do you see that demonstrates line managers are committed to safety? Are the top-level line managers the leading advocates of safety and demonstrate their commitment in both word and action? Do line managers periodically take steps to reinforce safety, including personal visits and walkthroughs to verify that their expectations are being met? 4.2 What type of evidence do you see that demonstrates line managers spend time on the floor? Do line managers practice visible leadership in the field by placing “eyes on the problem,” coaching, mentoring, and reinforcing standards and positive behaviors? Are deviations from expectations corrected promptly and, when appropriate, analyzed to understand why the behaviors occurred? 5. Leaders value employee involvement, encourage individual questioning attitude, and instill trust to encourage raising issues without fear of retribution. 5.1 What type of evidence do you see that demonstrates line managers are skilled in responding to employee questions in an open, honest manner? Do line managers encourage and appreciate the reporting of safety issues and errors and not disciplining employees for the reporting of errors? Do line managers encourage a vigorous questioning attitude toward safety, and constructive dialogues and discussions on safety matters? 6. Leaders assure employees are trained, experienced and have the resources, the time, and the tools to complete their job safely. 6.1 What type of evidence do you see that demonstrates staffing levels and capabilities are consistent with the expectation of maintaining safe and reliable operations? 6.2 What type of evidence do you see that demonstrates the organizational staffing provides sufficient depth and redundancy to ensure that all important safety functions are adequately performed? DOE-HDBK-1208-2012 1-7 LEADERSHIP 6.3 What evidence do you have that demonstrates to line managers the organization is able to build and sustain a flexible, robust technical staff and staffing capacity? Are pockets of resilience established through redundant resources so that adequate resources exist to address emergent issues? Does the organization develop sufficient resources to rapidly cope and respond to unexpected changes? 6.4 What type of evidence do you see that demonstrates adequate resources are allocated for safety upgrades and repairs to aging infrastructure? Are modern infrastructure and new facility construction pursued to improve safety and performance over the long term? 7. Leaders hold personnel accountable for meeting standards and expectations to fulfill safety responsibilities.

7.1 Are responsibility and authority for safety well defined and clearly understood as an integral part of performing work. 7.2 What type of evidence do you see that demonstrates organizational safety responsibilities are sufficiently comprehensive to address the work activities and hazards involved? 7.3 What type of evidence do you see that demonstrates the line of authority and responsibility for safety is defined from the Secretary to the individual contributor? Does each of these positions have clearly defined roles, responsibilities, and authorities, designated in writing and understood by the incumbent? 7.4 What type of evidence do you see that demonstrates ownership boundaries and authorities are clearly defined at the institutional, facility, and activity levels, and interface issues are actively managed? 7.5 Are organizational functions, responsibilities, and authorities documents maintained current and accurate? 7.6 Are reporting relationships, positional authority, staffing levels and capability, organizational processes and infrastructure, and financial resources commensurate with and support fulfillment of assigned or delegated safety responsibilities? 7.7 What type of evidence do you see that demonstrates all personnel understand the importance of adherence to standards? 7.8 What type of evidence do you see that demonstrates line managers review the performance of assigned roles and responsibilities to reinforce expectations and ensure that key safety responsibilities and expectations are being met? 7.9 What type of evidence do you see that demonstrates personnel at all levels of the organization are held accountable for shortfalls in meeting standards and expectations related to fulfilling safety responsibilities? Is accountability demonstrated both by recognition of excellent safety performers as well as identification of less-than- adequate performers in holding people accountable, in the context of a just culture, managers consider individual DOE-HDBK-1208-2012 1-8 LEADERSHIP intentions and the organizational factors that may have contributed? 8. Leaders insist on conservative decision making with respect to the proven safety system and recognize that production goals, if not properly considered and clearly communicated, can send mixed signals on the importance of safety. 8.1 Do organization managers frequently and consistently communicate the safety message, both as an integral part of the mission and as a stand-alone theme? 8.2 Do managers recognize that aggressive mission and production goals can appear to send mixed signals on the importance of safety? Are managers sensitive to detect and avoid these misunderstandings, or to deal with them effectively if they arise? What type of evidence supports your claim? 8.3 What type of evidence do you see that demonstrates the organization demonstrates a strong sense of mission and operational goals, including a commitment to highly reliable operations, both in production and safety? Are safety and productivity both highly valued? 8.4 What type of evidence do you see that demonstrates safety and productivity concerns both receive balanced consideration in funding allocations and schedule decisions? Are resource allocations adequate to address safety? If funding is not adequate to ensure safety, operations are discontinued? 9. Leadership recognizes that humans make mistakes and take actions to mitigate this.

9.1 What type of evidence do you see that demonstrates hazard controls are designed with an understanding of the potential for human error? Are error-likely situations identified, eliminated, or mitigated? Is the existence of known error-likely situations communicated to workers prior to commencing work along with planned mechanisms to assure their safety? What is your proof? 9. Leaders develop healthy, collaborative relationships within their own organization and between their organization and regulators, suppliers, customers and contractors. DOE-HDBK-1208-2012 1-9 EMPLOYEE INVOLVEMENT SAFETY CULTURE ATTRIBUTE Individual Attitude and Responsibility for Safety Safety is everyone’s responsibility. As such, employees understand and embrace the organization’s safety behaviors, beliefs, and underlying assumptions. Employees understand and embrace their responsibilities, maintain their proficiency so that they speak from experience, challenge what is not right and help fix what is wrong and police the system to ensure they, their co-workers, the environment, and the public remain safe. SOURCE OF ATTRIBUTE FROM ISM GUIDE (DOE G 450.4-1C) 1. Individuals team with leaders to commit to safety, to understand safety expectations, and to meet expectations. 1.1 What type of evidence do you see that demonstrates personnel at all levels of the organization are held accountable for shortfalls in meeting standards and expectations related to fulfilling safety responsibilities? Is accountability demonstrated both by recognition of excellent safety performers as well as identification of less-than-adequate performers? In holding people accountable, in the context of a just culture, do managers consider individual intentions and the organizational factors that may have contributed? 2.1 What type of evidence do you see that demonstrates individuals understand and demonstrate responsibility for safety? Are safety and its ownership apparent in everyone's actions and deeds? Are workers actively involved in identification, planning, and improvement of work and work practices? Do workers follow approved procedures? Can workers at any level stop unsafe work or work during unexpected conditions? Is there any evidence that they have stopped work? 2. Individuals work with leaders to increase the level of trust and cooperation by holding each other accountable for their actions with success evident by the openness to raise and resolve issues in a timely fashion. 2.1 What type of evidence do you see that demonstrates individuals promptly report errors and incidents? Do individuals feel safe from reprisal in reporting errors and incidents; they offer suggestions for improvements? 2.2 What type of evidence do you see that demonstrates individuals are systematic and rigorous in making informed decisions that support safe, reliable operations? Are workers expected and authorized to take conservative actions when faced with unexpected or uncertain conditions? Do you have any evidence that they have ever exercised this right? Do line managers support and reinforce conservative decisions based on available information and risks? 3. Everyone is personally responsible and accountable for safety, they learn their jobs, they know the safety systems and they actively engage in protecting themselves, their co-workers, the public and the environment.

3.1 What type of evidence do you see that demonstrates personnel at all levels of the organization are held accountable for shortfalls in meeting standards and expectations related to fulfilling safety responsibilities? Is accountability demonstrated both by recognition of excellent safety performers as well as identification of less-than-adequate performers? In holding people accountable, in the context of a just culture, do managers consider individual intentions and the organizational factors that may have contributed? 3.2 What type of evidence do you see that demonstrates people and DOE-HDBK-1208-2012 1-10 EMPLOYEE INVOLVEMENT their professional capabilities, experiences, and values are regarded as the organization’s most valuable assets? Do organizational leaders place a high personal priority and time commitment on recruiting, selecting, and retaining an excellent technical staff? 3.3 Does the organization maintain a highly knowledgeable workforce to support a broad spectrum of operational and technical decisions? Is the right technical and safety expertise embedded in the organization and when necessary is outside expertise is employed? 3.4 What type of evidence do you see that demonstrates individuals have in-depth understanding of safety and technical aspects of their jobs? 3.5 What type of evidence do you see that demonstrates the technical qualification standards are defined and personnel are trained accordingly? Do technical support personnel have expert-level technical understanding? Do managers have strong technical backgrounds in their area of expertise? 3.6 What type of evidence do you see that demonstrates assignments of safety responsibilities and delegations of associated authorities are made to individuals with the necessary technical experience and expertise? In rare cases, if this is not possible, are corrective and compensatory actions taken? 4. Individuals develop healthy skepticism and constructively question deviations to the established safety system and actively work to avoid complacency or arrogance based on past successes. 4.1 What type of evidence do you see that demonstrates individuals cultivate a constructive, questioning attitude and healthy skepticism when it comes to safety? Do individuals question deviations, and avoid complacency or arrogance based on past successes? Do team members support one another through both awareness of each other’s actions and constructive feedback when necessary? 4.2 What type of evidence do you see that demonstrates individuals are aware of and counteract human tendencies to simplify assumptions, expectations, and analysis? Are diversity of thought and opposing views welcomed and considered? Is intellectual curiosity encouraged? 4.3 What type of evidence do you see that demonstrates individuals are intolerant of conditions or behaviors that have the potential to reduce operating or design margins? Are anomalies thoroughly investigated, promptly mitigated, and periodically analyzed in the aggregate? Is the bias set on proving work activities are safe before proceeding, rather than proving them unsafe before halting? Do personnel not proceed and do not allow others to proceed when safety is uncertain? Do you have any evidence that they ever have exercised this right? 4.4 What type of evidence do you see that demonstrates individuals outside of the organization (including subcontractors, temporary employees, visiting researchers, vendor representatives, etc.) understand their safety responsibilities?

DOE-HDBK-1208-2012 1-11 EMPLOYEE INVOLVEMENT 5. Individuals make conservative decisions with regards to the proven safety system and consider the consequences of their decisions for the entire life-cycle of operations. 5.1 What type of evidence do you see that demonstrates individuals are mindful of the potential impact of equipment and process failures; they are sensitive to the potential of faulty assumptions and errors, and demonstrate constructive skepticism? Do they appreciate that mindfulness requires effort? 5.2 What type of evidence do you see that demonstrates individuals recognize that errors and imperfections are likely to happen? Do they recognize the limits of foresight and anticipation, and watch for things that have not been seen before? Do they appreciate that error-likely situations are predictable, manageable, and preventable, and seek to identify and eliminate latent conditions that give rise to human performance errors? 5.3 What type of evidence do you see that demonstrates individuals are systematic and rigorous in making informed decisions that support safe, reliable operations? Are workers expected and authorized to take conservative actions when faced with unexpected or uncertain conditions? How do you know? Do line managers support and reinforce conservative decisions based on available information and risk? 6. Individuals openly and promptly report errors and incidents and don’t rest until problems are fully resolved and solutions proven sustainable. 6.1 What type of evidence do you see that demonstrates individuals promptly report errors and incidents? Is there a sense that they feel safe from reprisal in reporting errors and incidents? Do they offer suggestions for improvements? 7. Individuals instill a high level of trust by treating each other with dignity and respect and avoiding harassment, intimidation, retaliation, and discrimination. Individuals welcome and consider a diversity of thought and opposing views. 7.1 What type of evidence do you see that demonstrates individuals cultivate a constructive, questioning attitude and healthy skepticism when it comes to safety? Do individuals question deviations, and avoid complacency or arrogance based on past successes? Do team members support one another through both awareness of each other’s actions and constructive feedback when necessary? 7.2 What type of evidence do you see that demonstrates individuals are aware of and counteract human tendencies to simplify assumptions, expectations, and analysis? Is diversity of thought and opposing views welcomed and considered? Is intellectual curiosity encouraged? 9. Individuals help develop healthy collaborative relationships within their organization and between their organization and regulators, suppliers, customers and contractors. DOE-HDBK-1208-2012 1-12 ORGANIZATIONAL LEARNING SAFETY CULTURE ATTRIBUTE Organizational Learning for Performance Improvement The organization learns how to positively influence the desired behaviors, beliefs and assumptions of their healthy safety culture. The organization acknowledges that errors are a way to learn by rewarding those that report, sharing what is wrong, fixing what is broken and addressing the organizational setup factors that led to employee error. This requires focusing on reducing recurrences by correcting deeper, more systemic causal factors and systematically monitoring performance and interpreting results to generate decision-making information on the health of the system.

SOURCE OF ATTRIBUTE FROM ISM GUIDE (DOE G 450.4-1C) 1. The organization establishes and cultivates a high level of trust; individuals are comfortable raising, discussing and resolving questions or concerns. 1.1 What type of evidence do you see that demonstrates credibility and trust are present and continuously nurtured? Do line managers reinforce perishable values of trust, credibility, and attentiveness? Is the organization just – that is, does the line managers demonstrate an understanding that humans are fallible and when mistakes are made, the organization seeks first to learn as opposed to blame? Is the system of rewards and sanctions aligned with strong safety policies and reinforces the desired behaviors and outcomes? 1.2 What type of evidence do you see that demonstrates open communications and teamwork are the norm? Are people comfortable raising and discussing questions or concerns? Are good news and bad news both valued and shared? 1.3 What type of evidence do you see that demonstrates a high level of trust is established in the organization? Is reporting of individual errors is encouraged and valued? What methods are available for personnel to raise safety issues, without fear of retribution? 2. The organization provides various methods to raise safety issues without fear of retribution, harassment, intimidation, retaliation, or discrimination. 2.1 What type of evidence do you see that demonstrates systems of checks and balances are in place and effective at all levels of the organization to make sure that safety considerations are adequately weighed and prioritized? 2.2 Do safety and quality assurance positions have adequate organizational influence? 2.3 What type of evidence do you see that demonstrates processes are established to identify and resolve latent organizational weaknesses that can aggravate relatively minor events if not corrected? Are linkages among problems and organizational issues examined and communicated? 3. Leaders reward learning from minor problems to avoid more significant events. 3.1 What type of evidence do you see that demonstrates the organization actively and systematically monitors performance through multiple means, including leader walk-arounds, issue reporting, performance indicators, trend analysis, benchmarking, industry experience reviews, self-assessments, and performance assessments? Is feedback from various sources integrated to create a full understanding? DOE-HDBK-1208-2012 1-13 ORGANIZATIONAL LEARNING 3.2 What type of evidence do you see that demonstrates organization members convene to swiftly uncover lessons and learn from mistakes? 3.3 Are frequent incident reviews conducted promptly after an incident to ensure data quality to identify improvement opportunities? 3.4 What type of evidence do you see that demonstrates expertise in causal analysis is applied effectively to examine events and improve safe work performance? Is high-quality causal analysis is the norm? Is causal analysis performed on a graded approach for major and minor incidents, and near-misses, to identify causes and follow-up actions? Are even small failures viewed as windows into the system that can spur learning? 3.5 Do performance improvement processes encourage workers to offer innovative ideas to improve performance and to solve problems? 3.6 What type of evidence do you see that demonstrates line managers are actively involved in all phases of performance monitoring, problem analysis, solution planning, and solution implementation to resolve safety issues?

4. Leaders promptly review, prioritize, and resolve problems, track long-term sustainability of solutions, and communicate results back to employees. 4.1 What type of evidence do you see that demonstrates line managers have a strong focus on the safe conduct of work activities? Are line managers maintain awareness of key performance indicators related to safe work accomplishment, watch carefully for adverse trends or indications, and take prompt action to understand adverse trends and anomalies? 4.2 What type of evidence do you see that demonstrates vigorous corrective and improvement action programs are in place and effective? Is there a rapid response to problems and closeout of issues ensures that small issues do not become large ones? Are managers actively involved to balance priorities to achieve timely resolutions? 5. The organization avoids complacency by cultivating a continuous learning/improvement environment with the attitude that “it can happen here.” 5.1 What type of evidence do you see that demonstrates operational anomalies, even small ones, get prompt attention and evaluation – this allows early detection of problems so necessary action is taken before problems grow? 5.2 Are candid dialogue and debate, and a healthy skepticism encouraged when safety issues are being evaluated? Are differing professional opinions welcomed and respected? Is it ever used? Are robust discussion and constructive conflict recognized as a natural result of diversity of expertise and experience? 5.3 What type of evidence do you see that demonstrates individuals are systematic and rigorous in making informed decisions that support safe, reliable operations? Are workers expected and authorized to take conservative actions DOE-HDBK-1208-2012 1-14 ORGANIZATIONAL LEARNING when faced with unexpected or uncertain conditions? Do line managers support and reinforce conservative decisions based on available information and risks? 5.4 What type of evidence do you see that demonstrates operations personnel are held to high standards of both technical understanding and detailed task-oriented performance? Do operations personnel provide reliable and consistent responses to expected occurrences? Are flexible responses to unexpected occurrences based on continuous preparation and training? Are formality and discipline in operations is valued? 5.5 What type of evidence do you see that demonstrates organizational systems and processes are designed to provide layers of defenses, recognizing that people are fallible? Are prevention and mitigation measures used to preclude errors from occurring or propagating? Are error-likely situations sought out and corrected, and recurrent errors carefully examined as indicators of latent organizational weaknesses? Do managers aggressively correct latent organizational weaknesses and measure the effectiveness of actions taken to close the gaps? 6. Leaders systematically evaluate organizational performance using: workplace observations, employee discussions, issue reporting, performance indicators, trend analysis, incident investigations, benchmarking, assessments, and independent reviews.

6.1 What type of evidence do you see that demonstrates line managers are in close contact with the front-line; they pay attention to real-time operational information? Is maintaining operational awareness a priority? How do you know? Do line managers identify critical performance elements and monitor them closely? 6.2 What type of evidence do you see that demonstrates organizations know the expertise of their personnel? What evidence do you have that line managers defer to qualified individuals with relevant expertise during operational upset conditions? Are qualified and capable people closest to the operational upset empowered to make important decisions, and are held accountable justly? 6.3 What type of evidence do you see that demonstrates performance assurance consists of robust, frequent, and independent oversight, conducted at all levels of the organization? Does performance assurance include independent evaluation of performance indicators and trend analysis? 6.4 Are performance assurance programs guided by plans that ensure a base level of relevant areas are reviewed? Are assessments performed against established barriers and requirements? 6.5 What type of evidence do you see that demonstrates efficient redundancy in monitoring is valued; higher levels of redundancy are recognized as necessary for higher risk activities? 6.6 What type of evidence do you see that demonstrates performance assurance includes a diversity of independent “fresh looks” to ensure completeness and to avoid complacency? DOE-HDBK-1208-2012 1-15 ORGANIZATIONAL LEARNING Is there a mix of internal and external oversight reviews reflects an integrated and balanced approach? Is this balance is periodically reviewed and adjusted as needed? 6.7 What type of evidence do you see that demonstrates the insights and fresh perspectives provided by performance assurance personnel are valued? 6.8 Is organizational feedback actively sought to make performance assurance activities more value-added? 6.9 Is complete, accurate, and forthright information is provided to performance assurance organizations? 6.10 What type of evidence do you see that demonstrates results from performance assurance activities are effectively integrated into the performance improvement processes, such that they receive adequate and timely attention? Are linkages with other performance monitoring inputs examined, high-quality causal analyses are conducted, as needed, and corrective actions are tracked to closure with effectiveness verified to prevent future occurrences? 6.11 What type of evidence do you see that demonstrates line managers throughout the organization set an example for safety through their direct involvement in oversight activities and associated performance improvement? 6.12 Are senior line managers periodically briefed on results of oversight group activities to gain insight into organizational performance and to direct needed corrective actions? 6.13 What type of evidence do you see that demonstrates periodic ISM reviews, assessments, and verifications are conducted and used as a basis for ISM program adjustments and implementation improvements? 7. The organization values learning from operational experience from both inside and outside the organization. 7.1 What type of evidence do you see that demonstrates operating experience is highly valued, and the capacity to learn from experience is well developed? Does the organization regularly examine and learn from operating experiences, both internal and in related industries?

8. The organization willingly and openly engages in organizational learning activities. 8.1 What type of evidence do you see that demonstrates line managers throughout the organization set an example for safety through their direct involvement in continuous learning by themselves and their followers on topics related to technical understanding and safety improvement? 8.2 What type of evidence do you see that demonstrates the organization values and practices continuous learning, and requires employees to participate in recurrent and relevant training and encourages educational experiences to improve knowledge, skills, and abilities? Are professional and technical growth formally supported and tracked to build organizational capability? 8.3 What type of evidence do you see that demonstrates training to broaden individual capabilities and to support organizational learning is available and encouraged – to appreciate the potential for unexpected conditions; to recognize and respond to a variety of problems and anomalies; to understand complex technologies and capabilities to respond to complex events; to develop flexibility at DOE-HDBK-1208-2012 1-16 ORGANIZATIONAL LEARNING applying existing knowledge and skills in new situations; to improve communications; to learn from significant industry and DOE events? 8.4 Are mental models, practices, and procedures updated and refreshed based on new information and new understanding? 8.5 What type of evidence do you see that demonstrates training effectively upholds management’s standards and expectations? Beyond teaching knowledge and skills, are trainers adept at reinforcing requisite safety values and beliefs? 8.6 Do managers set an example for safety through their personal commitment to continuous learning and by their direct involvement in high-quality training that consistently reinforces expected worker behaviors? 8.7 Do managers encourage informal opinion leaders in the organization to model safe behavior and influence peers to meet high standards? DOE-HDBK-1208-2012 1-17 The intent is to look for gaps (∆Wg • Threats from human performance ) to determine first “what” did not work (Chapter 3) so that later in the report we can explore “why” things did not go according to plan (Chapter 4). We then explore how significant the gap was (Chapter 5) and what organizational factors allowed us to degrade to this condition (Chapter 6) so that Judgments of Need and Lessons to be Learned need to be implemented to correct the problem (Chapters 7, 8, & 9). Chapter 1 Executive Summary 1.1 Event Summary 1.2 Impact of Event 1.3 Org Weaknesses & Recommend Chapter 2 Introduction (compare work-as-done to work-as-imagined/planned 2.1 Event Summary (“work-as-done”) Sources (exhibits): go down and watch work being done, complete the barrier analysis “work-as-done” process flow chart, conduct interviews of those directly involved in the event, review logbooks, process records, etc. and collect information in timeline & comparative timeline 2.1.1 Work flow (what were we trying to physically get done and what was the process the workers were using to do it?) 2.1.2 Hazard Analyses (those pertinent to this event that the worker identified  hazards-as- recognized) 2.1.3 Threats (those pertinent to this event that the worker identified  threats-as-recognized) • Threats from equipment, tooling, or facility problems • Threats from natural disasters • Threats from other sources

2.1.4 Barriers (those pertinent to this event (for each threat) that the worker identified  barriers-as-done) 2.1.5 Culture (were the workers buy into the process and controls they had to work with?) 2.2 Tech basis (“work-as-planned”) Sources (Attachments): CFR requirements, OSHA requirements, DOE orders, roles and responsibilities, plant policies, procedures, work instructions, fitness for duty requirements, interviews with support organizations (e.g., engineers, trainers, etc.). 2.2.1 Background on the event 2.2.1.1 What were we trying to physically accomplish? 2.2.1.1.1 Technical analysis of what went wrong 2.2.1.2 Work process flow intended to accomplish the above 2.2.2 Regulatory requirements  (10 CFR 851, 10 CFR 830, DOE Orders, TBPs, etc.) 2.2.3 Policies or Procedures (that pertain to this event) DOE-HDBK-1208-2012 1-18 2.2.3.1 Policies 2.2.3.2 Procedures (process flow maps etc.) 2.2.4 Training and Qualifications (of participants in event – related to this event) 2.2.4.1 Training & Qualifications (supervisors) 2.2.4.2 Training & Qualifications (workers) 2.2.5 Hazard Analyses (DSA/HAR, JSHA, etc. -those parts focused on the event that we documented) 2.2.6 Threats (those pertinent to this event that were identified in written procedures) • Threats from human performance • Threats from equipment, tooling, or facility problems • Threats from natural disasters • Threats from other sources 2.2.7 Barriers (those pertinent to this event that were documented) 2.3 Important Gaps Between WAD and WAP The idea here is not everything is important. The team will have a sense at this point of what really bothers them with regard to the incident and the organizational factors that set it up. It is based on this insight that the important gaps are selected for further analysis. Chapter 3 Event Facts (“what happened”  ∆Wg) Based on this W-A-D to W-A-I comparison, the event facts or ∆Wg facts i.e., “what happened” are determined in Chapter 3 and the causal factors associated with those event facts or ∆Wg why or “why it happened” are determined in Chapter 4. 3.1 Initial Conditions (what we started with and assumed we would be successful) Sources: comparative timeline, causal factor chart, look at pre-job brief, readiness review, work authorization 3.2 Consequences (what went wrong) Sources: Consequence WBS, comparative timeline, causal factor chart 3.3 Event Facts (∆Wg – brief description) 3.3.1 Chart A (∆Wg #1) 3.3.2 Chart B (∆Wg #2) 3.2.2 Chart C (∆Wg #3) Chapter 4 Organizational Causal Factors (“why” it happened) 4.1 Chart A (word story to explain the boxes in causal factor chart) 4.2 Chart B (word story to explain the boxes in causal factor chart) 4.3 Chart C (word story to explain the boxes in causal factor chart) Chapter 5 Impact of Event and What Influenced It DOE-HDBK-1208-2012 1-19 The safety, quality, and security impact of this event in terms of consequences and significance is presented in Chapter 5. The consequences are framed in terms of actual, expected or potential consequences with a discussion of why you picked the one you did in Section 3.2 Here we try to work with the responsible organization to determine what parts of the gaps truly represent a threat to the system as it was designed (i.e., how significant the ”what” issues were). 5.1 Consequences Review all consequences in terms of the “Ds” and discuss why you picked the one selected in Section 3.2 (flag with red box in consequence WBS.)

Source: Consequence Work Break Down Structure 5.2 Significance Discuss significance of the “what” gap between work-asplanned and work-as-done (remember, not everything is important) by addressing: Source: Chapters 2, 3 & 4 1.2.1 Threats (human error, equipment, facility, tooling malfunction, nat. disaster, other) Source: CTL, TWIN, Modified TapRoot Cause Tree 5.2.2 Flawed Defenses (what we did not do to block threats) Source: Barrier Analysis Matrix 5.2.3 Campaign Items (items of importance to our customer or regulator because these type of events have cropped up in other places) Chapter 6 Organizational Weaknesses (“why”) Discuss significance of “why” the gap between work-as-planned and work-as-done is important by addressing how the organization: failed to obtain feedback on its systems (i.e., the gap between work-as- planned and work-as-done); failed to learn as an organization; infused organizational weaknesses into business processes because of past business decisions; let the culture degrade to an unhealthy state, and failed to execute its HRO practices. We want to state whether because of the above, this event is a precursor to a more significant event in the future if not fixed now, to motivate organizational change. 6.1 Feedback Source: Evidence collected on contractor assurance system assessments, mgt assessments, oversight, etc. and Extent of Conditions (EOC) matrix  look at those things that would be in green oval between “worker behavior” and “plant results,” 6.2 Learning Organization Source: MOM, Campaign items 6.3 Organizational Weaknesses Source: CTL, LOW Table (derived from reviewing all the “conditions” noted on CFA Chart 6.4 Culture Source: CAM or culture survey or safety culture climate survey 6.5 HRO Practices Source: HRO assessment 6.6 Precursors Source: CTL, EOC, MOM, LOW, CAM, HRO assessment Chapter 7 ECAQ Extraneous Conditions Adverse to Quality (ECAQs) are causal factors that were discovered in the investigation but did not contribute to this event are presented in Section 7. Source: Collection through investigation Chapter 8 JON (Evaluate impact of JON and develop good measures of effectiveness) DOE-HDBK-1208-2012 1-20 Recommendations in the way of Judgments of Need (JONs) and Lessons to be Learned (LTBL) are presented in Sections 8 & 9. Proper selection requires knowing the error mode the worker was in at the time of the event. We do not want to just check the box and add more non-value added corrective actions. JON and LTBL should be written not only for “what” was found wrong (Chapter 4 & 5) but also to address those organizational factors that allowed us to degrade to this condition (Chapter 6) Source: JON/CA Matrix Chapter 9 LTBL (organizational learning) Source: JON/CA Matrix DOE-HDBK-1208-2012 1-21 Table 1 – Systematic Barrier Analysis Matrix (SBAM) Goal: Ensure Viability of Organization Consequence to Avoid: Shutting Down Organization Identify event trying to avoid: Primary hazards to minimize and protect: Col #1 Column #2 Column #3 Column #4 Column #5 Column #6 Column #7 Column #8 Column #9 Process Step Identifier Threat to Hazard Barrier between Threat and Hazard Hazard Protected Effectiveness of Barrier Significance of Barrier in Protecting Hazard Remove Threat or Hazard or Fix/Remove Barrier or Add Additional Barriers Actions to implement additional barrier Effectiveness of future barrier Where in the “work- as-done” process flow is the barrier located?

Human error threat? Equipment, tooling, facility threat? Natural disaster threat? Other threat? Engineered barrier? Administrative barrier? Preventive or mitagative barrier? Identified in planning or reactive to problem? Barrier Design (right barrier in right place in process for threat)? Barrier Implementation (i.e., are barriers implemented in right place in process, does worker know about the barrier and is willing to use it, do barriers work as designed)? Barrier Maintained (configuration controlled, Barrier verified operational before work)? How important to protecting hazard? Where does the barrier fall in the sequence of work (Plant Results, Worker Behavior, Job Site Conditions, and Organizational Processes & Values)? How many barriers between threat and hazard? Are multiple barriers independent? If barrier missing, weak, or want more confidence can the hazard or threat be removed? If not, what modifications to existing barriers would decrease the risk or what barriers you would add? Discuss cost vs. risk List who assigned to and tracking # Based on engineered or admin barrier evaluate effectiveness DOE-HDBK-1208-2012 1-22 Table 2 – Example of Expanded Comparative Timeline (CTL) Date/ Time What Happened (“work-as- done”) What Should Have Happened (“work-as- imagined”) Immediate Consequences of GAP between “work-as imagined” vs. “work-as-done” Significance of GAP on Consequence Precursor to this event – other more serious events (type of factor) List Barriers that Worked Flawed Defenses (missing/remaining) Extent (spatial/temporal) Campaign Item Other Factors Culture Factor HP Error Precursors Human Performance Error Mode Missed opportunity Latent Organizational Weakness (LOW) 08/05 There was a need for replacement shoring for previous job. Planner found paneling on Internet search, intending to purchase for job through the VMI system (not intended for life- safety equipment). Planner discussed use of Mighty Lite panels with personnel in the Yard Group and may have discussed with Industrial Safety. Note: The initiative taken by the planner to find alternate and improved equipment to The responsibility for making the actual technical decisions on life safety equipment such as excavation shoring should be assigned to personnel who are qualified and expected to do so. (Business 101, Quality 101, DOE QA 101) - Best Business Practice Responsible personnel should have developed a full technical basis for the specification and proper use of flexible shoring equipment*. Responsible personnel should have discussed and documented approval of using group supervision. * This life safety equipment had two components – fabrication & assembly. Technical basis for selection and proper assembly of excavation equipment was not formally documented nor peer reviewed. Precursor to this event – to other more serious event (vulnerability, triggering, exacerbating, or mitigating factor): - Vulnerability factor for this and future events involving purchase and use of life safety equipment (would have better understood the proper assembly and use of shoring equipment if tech basis developed) Barriers that Worked: - List Barriers that worked Flawed Defenses (barriers missing/penetrated, number of barriers remaining – fragility of remaining barrier): - Missing barrier - Lack of tech basis - Partial barrier - planner did consult engineer and yard group about purchase - Fragile remaining barrier - supervisor was

the remaining barrier and he did not recognize a hazard - Number of remaining barriers? Extent (localized or plant-wide, what were previous events or LL, what assessments performed?): - Plant-wide impact - Previous events/LL? Culture Elements: Leadership: Employee Engagement: Organizational Learning: - Decision Making- Conservative assumptions to demonstrate actions safe rather than prove unsafe not used Human Performance: - Resources - Complete, accurate, and up- to-date as-purchased and procedures not completed HP Error Precursors: - Inadequate standards - Unclear roles/responsibilities HP Error Mode: - Skill-Based; Rule-Based, Knowledge- Based? Missed opportunities & expected result: - Had the chance consider this was life safety equipment and request formal review or at least formal training on proper assembly from vendor LOW: - Procedure development – human factors not considered in procedure development and implementation DOE-HDBK-1208-2012 1-23 Date/ Time What Happened (“work-as- done”) What Should Have Happened (“work-as- imagined”) Immediate Consequences of GAP between “work-as imagined” vs. “work-as-done” Significance of GAP on Consequence Precursor to this event – other more serious events (type of factor) List Barriers that Worked Flawed Defenses (missing/remaining) Extent (spatial/temporal) Campaign Item Other Factors Culture Factor HP Error Precursors Human Performance Error Mode Missed opportunity Latent Organizational Weakness (LOW) facilitate excavations was a good thing to do. The fabrication met OSHA requirements. Assembly would meet OSHA requirements, if “correctly” done. The inherent flexibility of the equipment placed additional requirements on the “proper” assembly of the shoring to meet OSHA requirements. - Assessments performed? Campaign Item: - Is this issue high on customer’s concern? DOE-HDBK-1208-2012 1-24 Table 3 - Example of Missed Opportunities Matrix (MOM) Who Situation Opportunity (action) Expected Result Impact on Consequence/Remarks Planner (purchaser of Mighty Lite shoring equipment) Replacing existing shoring equipment (hydraulic and iron system) Had chance to consider purchase of replacement shoring equipment as life safety equipment (just like would be done for scaffolding) and request formal review of purchase against an established technical basis or at least formal training on proper assembly from vendor The responsibility for making the actual technical decisions on life safety equipment such as excavation shoring would be assigned to personnel who are qualified and expected to do so. The responsible personnel would have developed a full technical basis for the specification and proper use of flexible shoring equipment*. The responsible personnel would have discussed and documented approval using group supervision. Hazards of assembly and moving would have been recognized, instructions would have been developed and training needs would have been determined and acted upon. Would have addressed the inaccurate risk perception. Yard Group supervisor Authorization of Mighty Lite shoring equipment for use by Yard Group. Had the chance recover from initial informal acceptance by recognizing that flexibility also introduced variability in assembly and requiring manufacturer training.

Per WI 02.01.01.05.03, “How to Conduct a JSHA,” either a procedure would have been written or a JSHA performed. Would have recognized that because of the Manufacturer’s instructions which stated training required that the Manufacturer should have been requested to provide training. Would recognize that flexible Mighty LIte shoring equipment had two components – fabrication & assembly. The fabrication met OSHA requirements regardless of how used. The assembly would meet OSHA requirements, only if “correctly” done such that the inherent flexibility of the equipment placed additional requirements on the “proper” assembly of the shoring to meet OSHA requirements. Hazards associated with assembly and moving of shoring equipment would have been documented and available for pre- job briefs and for reference in work packages. Yard Group supervisor During preliminary walk down work package Had chance to review package and balance hazards from excavation and assembly of shoring equipment to ensure resources placed to minimize risk. Recognize the need for the work package to identify shoring equipment as a task. Identify hazards associated with shoring equipment assembly. Would have increased the odds of a successful assembly and use of shoring equipment. Uncleared personnel Uncleared personnel assembling shoring equipment Uncleared workers could have questioned work relative to their knowledge and experience. Pre-job would have been repeated with all employees involved. STOP work authority would have been executed when the apparent hazard of the shoring equipment rotation was beginning. Used by permission Bill Corcoran, NSRC DOE-HDBK-1208-2012 1-25 Table 4 - Example of TWIN Analysis Matrix TASK DEMANDS Number of Times Occurred INDIVIDUAL CAPABILITIES Number of Times Occurred Interpretation Requirements 2 Schedule Pressure to Get Work Done High Workload (memory requirements) First Time Task 3 Simultaneous – Multiple Tasks 3 Knowingly Broke a Rule Unexpected Conditions Encountered Problem Solved Wrong (inaccurate mental model) Time Pressure 5 Fatigue or Illness Repetitive Actions/ Monotony Lack of Experience With Task 14 Unclear Goals, Roles, or Responsibilities 3 Inaccurate Mental Model of Tasks 4 Lack of, or Unclear Standards 4 Misunderstood Communication Confusing Procedure/ Vague Guidance 1 No Communications Delays; Idle Time; Worker Got Lost Personal Issue Lost Focus (medical, financial, emotional) WORK ENVIRONMENT HUMAN NATURE Confusing Controls/Display Made Bad Assumption 5 Overconfidence 11 Mind Set (I could have sworn it was right) Distractions/ Interruptions 3 Complacency (done task many times before) Unexpected Equipment Condition Disoriented or Confused During Task Production Emphasis by Supervisor Mental Shortcut (assumptions easily confirmed) Unavailable Parts or Tools – Made Do Boring Task Changes/ Departures from Routine 2 Work with People I Do Not Know or Like Personality Conflicts On the Job Stress (perceived threat to well-being) Work-Arounds 3 Habit Patterns Caused by Wrong Actions Left Unchecked for Long Time 8 Hidden Systems Response 1 Inaccurate Risk or Hazard Perception 14 Adverse Environmental Condition (Heat/Cold) Focused on Task – Missed Big Picture 2 Poor Access to Equipment /Human Factors Limited Short-Term Memory Swing Shift Work Uncertain About Job Requirements Summary of Human Performance Precursors DOE-HDBK-1208-2012 1-26 Table 5: Example of Safety Culture Attribute Matrix (CAM)

Conflicts between Observed/ Inferred Cultural Attributes and Functional Safety Culture Observed/ Inferred Behavior (Not so good) ARTIFACTS Functional Behavior (Better) ESPOUSED BELIEFS Direct Result of Observed/ Inferred Downstream Result of Observed/ Inferred Significance of Difference Between Observed/ Inferred Behavior and Espoused Values & Beliefs Institutions A more formal conduct of operations focused on bringing (purchasing, accepting, authorizing) those critical tools/equipment on Plant-site that are expected to protect our workers from hazards. : The front- end of bringing life-safety on site to protect the workers is weak and the supervisor in the field is expected to catch and fix everything to ensure the safety of his/her workers. Supervisor had Mighty Lite shoring equipment that was never reviewed as meeting requirements by a SME (structural engineers), was never formally assessed for adequate and workable instructions, and was never reviewed to ascertain the proper training. There could be more serious injury involving the use of shoring or other life safety equipment* because the acquisition, acceptance, and/or authorization process is informally accomplished. * Note there may be other items the supervisor has to accomplish because his/her support system does not and he/she is the last barrier to protect the workers. Supervisors sometimes too busy to supervise. Supervisors frequently are more involved in coordinating activities, accommodating work management shortfalls, or dealing with meetings and paperwork than providing critical job and safety oversight of their workforce. When this is combined with the expectation that the supervisor is the catch all for all safety functions not adequately addressed by the system, the organization is set up for eventual failure. Norms Workers should challenge every aspect of their business from the early procurement all the way down to the work execution and not let anything be passed to the lower levels without sufficient challenge and resolution. : There is a lack of questioning attitude among the workforce. People forget that getting work accomplished is a team effort that requires workers as well as supervisors to question work that does not make sense and to actively seek out and identify hazards so that all workers are sensitized to the hazards and barriers in the workplace. The attempt to lift the completed heavy shoring assembly (> 1500 lbs) manually was not challenged by any worker at the site (40 lb lifting restriction & crushing hazard ignored). Warning flags should have been raised because of the concern for potential back injury and because of the potential crushing injuries. No concerns were raised and as a result a worker was injured. Hazards exist and change in every aspect of maintenance because of the expansive nature of work, because of all the unknown variables with people, equipment and training and because of the dynamic environment in which the work is done. No single supervisor or worker will ever be able to identify and protect against every potential hazard. A team approach where everyone is looking, questioning, and challenging every aspect of the work is required to increase the chances of identifying the job site hazards to ensure protection of the workers. The workforce is unengaged. Maintenance "ownership" is either not or only marginally practiced or promoted. Workers have not internalized management expectations for safe and error-free work, are not involved sufficiently in problem-solving and troubleshooting, and are unaware of performance shortfalls in their functional area.

Used by permission Bill Corcoran, NSRC DOE-HDBK-1208-2012 1-27 Example: Causal Factors Analysis (CFA) Chart – An Electrical Penetration Event Craft Workers were exposed to an uncontrolled energized circuit (no barriers left) Circuit damaged, breaker tripped. Crafts drilled into 12-68 concrete floor 4/17/07 Penetration Permit Requested, Processed, and Issued without Utilities being Identified 3/26/07 Requested 4/2/07 Issued Job Planned & Approved without Electrical Hazards Identified 3/6/07 Work Authorized without Implementing Controls associated with Electrical Hazards 3/12/07 Consequence Final Event Direct Cause Initiating Event Work Request submitted to Work Management Center to design, fabricate and install a work platform in Bldg. 12-68. Process in Place to Request Penetration Permits (WI 02.01.01.05.23) Initial Conditions Penetration Process Required LO/TO of Identified/Known Energized Circuits within 6 inches of Penetration. (WI 02.01.01.05.23) Hit energized circuit Need for Work Platform around CNC machine in Bldg. 12-68 to improve safety of worker In-process Feedback Concerning Electrical Hazard not Incorporated 4/3/07 A complete hazards analysis was not included in the work package as per WI 02.06.04.02.03. E D C A B NDE Issued the Penetration Permit to the Crafts vs. the Utility Locators without Utilities being located. Contrary to WI 02.01.01.05.23. Causal Analysis, or “Whys”, continued on next pages. Work-as-Planned versus Work-as-Done Gap Analysis, ΔWg, or the “Whats”. DOE-HDBK-1208-2012 1-28 Example: CFA Chart (Cont’d) Upon initial walk down, a thorough scope of the project was not recognized. Did not perceive the electrical hazard. Pre-HA was only partially used as evidenced by lack of controls flowed down to the work package, (e.g., electric drill stop identified but not implemented) Lost sight of installation since focus was on the fabrication of the platform. Fabrication of platform “overshadowed installation.” Facility/utility electrical drawings were not used in the planning process. JON 3 Electrical distribution in the floor was not considered or included in the engineering’s design as part of the installation and potential interference. JON 2 Structural was the primary focus and no electrical disciplines were involved in the walk down. Focus on structural, therefore, did not consider electrical. Work Management Center verbally (PX-4776 not submitted) requested engineering to provide structural platform design only. Use of hazards analysis checklist is informal. Process for initial and updates prior to finalizing work plans is informal. JON 7 Fabrication and installation combined into the same task (5 task steps for fabrication, 1 task step to install). Planner Hazards Awareness Training does not include visual recognition of the presence of imbedded electrical. JON 6 For this project, a draft Pre-HA was used in lieu of the PX-5110, as required by Planner’s Handbook. It should be noted that the Planner’s Manual still requires form PX-4772, which is now completed by Craft Supervisor. A DOE-HDBK-1208-2012 1-29 Example: CFA Chart (Cont’d) Requirements of Safety Gram dated 04/ 25/06, specifically for electric drill stop, not included in work package. Safety Gram was not Implemented in a timely manner into governing procedures or JSHA. NOTE: No formal read and sign for this Safety Gram.

No formal policy governing Safety Grams and the requirement to incorporate into procedures/ JSHAs within a set period of time. JON 9 Safety’s approval of Work Order did not contribute to hazards identification. (Required per procedures) Safety did not walk down project and did not complete a Pre-HA. (Not formally required per procedures) Based upon electronic review of the work package, complexity of job did not require a walk down and Pre-HA. (Not formally required per procedures) The focus of the work package scope emphasized fabrication and did not identify electrical hazards. There is not a documented decision tree that would tell Safety which job to walk down. (Not a formally documented process) JON 5 B DOE-HDBK-1208-2012 1-30 Example: CFA Chart (Cont’d) Craft Supervisor did not identify and/or implement controls for electrical hazard. PX-4771 completed by Craft Supervisor did not identify the electrical hazard or penetration work. Mandatory control to use electric drill stop not included as a control per 4/ 25/06 Safety Gram. JON #9 Though aware of Safety Gram, understood battery powered drill to be double insulated and afforded protection. (Facilitative assumption) Incomplete Hazard Analysis during planning (see planning) JON 3, 6 & 7 The PX-4771 was a pre- canned form, replicated from previous project. Convenient and common shop practice. Work Order Package did not identify the electrical hazard Didn’t recognize electrical hazards (see above) Walk-down prior to initial authorization not performed. Use of electric drill stop not specified in Work Order Package. JON #5 Boiler Shop didn’t have a electric drill stop readily available. Craft Supervisor did not implement controls for electrical hazard Presumed work package adequate - to identify hazards and required controls. Based on review of approved work package, and prior to penetration permit issued, work authorized. Presumed existing the penetration permit process would adequately identify utilities. Assumed based on past successes would assure future success. C DOE-HDBK-1208-2012 1-31 Example: CFA Chart (Cont’d) NDE was unaware of WI 02.01.01.05.23 and their role/ responsibility with regards to location of utilities was not proceduralized, consequently NDE checked for ferrous metals only, not utilities. JON 1 & 10 Shop Coordinator sent permit to NDE, contrary to WI 02.01.01.05.23. Practice instituted for Craft work in Zone 12 South in MAA in 2001 was propagated for crafts Plantwide in 08/2006. NOTE: The 2001 was an evolution from the 1996 lightning bo