DOE-HDBK-3012-96, Guide to Good Practices for Operational Readiness Reviews (ORR) Team Leader's Guide

TS INCH-POUND DOE-HDBK-3012-96 June 1996 SUPERSEDING DOE-HDBK-3012-94 September 1994 DOE HANDBOOK GUIDE TO GOOD PRACTICES FOR OPERATIONAL READINESS REVIEWS (ORR) TEAM LEADER'S GUIDE U.S. Department of Energy AREA MISC Washington, D.C. 20585 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. WELCOME This Portable Document Format (PDF) file contains bookmarks, thumbnails, and hyperlinks to help you navigate through the document. All items listed in the Contents are linked to the corresponding sections. In addition, if you click on a section heading while you are reading the text, you will return to the Contents. Click on the DOE seal below to move to the Contents page. This document has been reproduced directly from the best available copy. Available to DOE and DOE contractors from the Office of Scientific and Technical Information, P.O. Box 62, Oak Ridge, TN 37831; (423) 576-8401. Available to the public from the U.S. Department of Commerce, Technology Administration, National Technical Information Service, Springfield, VA 22161; (703) 487-4650. Order No. DE96009471 DOE-HDBK-3012-96 iii FOREWORD This Department of Energy handbook, Guide to Good Practices for Conducting Operational Readiness Reviews (ORR) Team Leader's Guide, is approved for use by all DOE Components and their contractors. Beneficial comments (recommendations, additions, deletions) and any pertinent data that may improve this document should be sent to: Defense Programs, Office of Engineering, Operations, Security and Transition Support by letter or by using the self-addressed Document Improvement Proposal (DOE F 1300.3) appearing at the end of this document. DOE-HDBK-3012-96 iv INTENTIONALLY BLANK DOE-HDBK-3012-96 v TABLE OF CONTENTS PURPOSE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 PRIOR TO PREVISIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Initial Duties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Team Selection Duties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Scheduling/Coordination Responsibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Data Packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Team Member Preparation for both Previsit and ORR . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 PREVISIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Initial Team Meeting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 PREPARING FOR THE ORR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Team Leader Responsibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Finalize the ORR Implementation Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Establish ORR Length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Review Coordinator Responsibilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

ORR DUTIES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Opening Meeting at Site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Team Meetings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Final Team Meeting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Close-out Meeting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 POST ORR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Final Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Finding Closure and Team Follow-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 LESSONS LEARNED FROM PREVIOUS ORR/RAs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Achieving Readiness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Items to Note Before Proceeding With a DOE ORR . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Administration of the Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Conduct of Drills and Other Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Logistics and Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Team Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Developing the Implementation Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Defining the Scope and Graded Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Post ORR Corrective Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Miscellaneous Lessons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Generic Lessons Learn Which Apply to The Design, Construction, Operation, and Decommissioning of DOE Facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 DOE-HDBK-3012-96 vi INTENTIONALLY BLANK DOE-HDBK-3012-96 vii APPENDICES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 KEY POINTS TO DISCUSS DURING INITIAL CONTACT WITH SITE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . APPENDIX 1-1 SITE CONTACT MEMORANDUM . . . . . . . . . . . . . . . . . . . . . . . . . . . . APPENDIX 2-1 LOGISTICS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . APPENDIX 3-1 PRELIMINARY SCHEDULE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . APPENDIX 4-1 PREVISIT SCHEDULE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . APPENDIX 5-1 CHECKLIST FOR ORR TEAM INFORMATION PACKAGES . . . . . . . . APPENDIX 6-1 DETAILED SCHEDULE FOR ORR TEAM . . . . . . . . . . . . . . . . . . . . . . APPENDIX 7-1 TRANSMITTAL LETTER FOR DATA PACKAGES . . . . . . . . . . . . . . . APPENDIX 8-2 TEAM MEMBER DELIVERABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . APPENDIX 9-1 INSTRUCTIONS FOR COMPLETING ORR FORMS . . . . . . . . . . . . . . APPENDIX 10-1 PRESTART AND POST-START DETERMINATION CRITERIA FOR OPERATIONAL READINESS REVIEWS . . . . . . . . . . . . . . . . . . APPENDIX 11-1

DOE-HDBK-3012-96 viii INTENTIONALLY BLANK DOE-HDBK-3012-96 1 TEAM LEADER, TEAM MEMBER, & REVIEW COORDINATOR ORR DUTIES AND RESPONSIBILITIES PURPOSE: This guidance section provides instructions, explanations and examples for the performance of all phases of an Operational Readiness Review (ORR). Details pertinent to the Team Leader (TL), Team Members (TM) and Review Coordinator (RC) are outlined. An appendix contains sample forms and correspondence which are typically used to initiate and perform the ORR. Although this document was written specifically for use by DOE ORR Team Leaders, its use may also be beneficial to contractor ORR Team Leaders. The handbook is also useful for Team Leaders of Readiness Assessments conducted in accordance with requirements of DOE O 425.1. Lessons learned, which are promulgated with this handbook, will benefit any line manager, particularly those preparing a facility or process for startup or restart. PRIOR TO PREVISIT ORR DUTIES TL Initial Duties: After the projected start date is established, the TL should perform the following: 1. Comply with commitments made in the approved Plan of Action. 2. Contact the identified points of contact to discuss details about the upcoming ORR and negotiate prospective dates for the previsit and ORR. 3. Send a letter, as soon as possible after the above negotiations, formalizing the notification of the ORR to site points of contact (see Appendices 2, 3, 4, and 5). 4. Advise the Review Coordinator of dates and agenda for the previsit. 5. Initiate the process of selecting team members to support the ORR as soon as possible. TL Team Selection Duties: The importance of this task cannot be overemphasized. No other task has such a direct impact on the overall quality of the ORR. The TL selects and organizes the previsit and ORR teams. Additional guidance on team selection can be found in sections 5.4.2 and 5.4.3 of the ORR Standard, DOE-STD-3006-95. Guidelines for completing the process follow: 1. Candidates may be obtained from any independent sources within DOE or from contractors not directly responsible for the oversight or operation of the facility under review. Previous TLs are valuable resources in reviewing the past performance of team members. The areas to be reviewed at a minimum are: Operations, Maintenance, Training, Safety Envelope Verification, Engineering Support, Environmental Safety and Health, Emergency Preparedness, Radiation Protection, Occupational Health and Industrial Safety, Management, Fire Protection, and Quality Assurance. Other areas reviewed can include, if appropriate, configuration management, criticality safety, and DOE management. Team members must have technical experience in their area of review. DOE-HDBK-3012-96 2 TL 2. Remember that team members must not only have technical experience relevant to their assignment, but also must have assessment experience and be familiar with the facility/process in question. Typically, familiarization with the facility/process is gained during the previsit. In determining the number of team members, experience indicates more members for less time is the best option. 3. Determine whether any conflict of interest, actual or perceived, exists for any candidate or source of candidates. If so, those candidates must be rejected from further consideration. A real conflict of interest (COI) would exist for a contractor who has, previous to the ORR, provided direct support to a facility in an area which will be reviewed during the ORR. For example, DOE has used mentors at some facilities to assist in improving the conduct of operations at the facility. It would be inappropriate for this contractor to be a part of the ORR team.

A perceived COI could exist when a contractor has provided indirect support to the facility or was an employee of the facility at some other point in their career. An example of indirect support could be participation on a review team evaluating facility safety basis documentation. The second case is self explanatory. These cases are more subjective, but the TL should be sensitive to them and avoid them if possible. 4. No individual directly involved in the management or operation of a facility can participate as a team member in his/her area of responsibility at the facility. 5. With regard to selecting administrative staff for the ORR, it must be noted that the administration of the forms and summaries is a prodigious task. It has been found that one person (the ORR Review Coordinator) should be dedicated to administering the forms, e.g., logging in and tracking. This task is very time consuming once forms begin being turned in and increases as the ORR progresses. In addition, at least one other person with a word processor should be dedicated to editing the forms. Consideration should be given to bringing in supplemental editing staff toward the close of the ORR. It should also be recognized that a significant amount of copying will need to be accomplished at the close of the ORR. 6. Concurrent with the above tasks, work with the site point-of-contact to obtain counterparts. A counterpart is an individual designated by the site to assist in coordinating the ORR team's inspection activities. Each ORR team member should have a counterpart for each functional area under evaluation. DOE-HDBK-3012-96 3 7. As soon as team members and counterparts have been selected, disseminate a list showing counterparts' telephone, facsimile numbers, and e:mail address, if available, to team members. The list is important for informing team members of their selection and their counterpart, which allows for early contact and preparation. Scheduling/Coordination Responsibilities RC • After team selection is made, obtain social security number, date of birth, address, phone number, fax number, type of clearance, and funding information for all team members. • Assist the TL with developing team member/counterpart chart. • Prepare ORR notification letter and enclosures for the TL (Appendix 2). • Make a contact list for the previsit ORR team. The list should include site contact names, phone numbers, fax numbers, hotel information, site security office number, per diem rates. (Some of this information will be provided to the by the TL). • Send draft ORR Implementation Plan and CRADs to ORR previsit team members for their review. • Reserve a block of rooms for ORR previsit team members who will travel form outside the area. Send the hotel a team list, if required. ADDITIONAL GUIDANCE FOR PREPARING FOR BOTH PREVISIT AND ORR These duties involve the preparation of the data packets, travel, and other logistics. The TL should manage the performance of the following items. TL • Data Packets: Data packets are used to provide team members with information and documents that enable them to better prepare for the previsit and ORR. • Working with the RC, assemble and distribute data packets to team members. The normal makeup of the data packet is provided in Appendix 6. Mail data packets early to ensure that team members have their packets at least two weeks prior to the previsit.

• Other Logistics: Verify compliance with the request for logistic support that was sent (Appendix 3) with the letter of confirmation. RC • Inform team members, usually these instructions are included in the data packet, to call the hotel and confirm their reservation and to guarantee it with a credit card. DOE-HDBK-3012-96 4 RC • Make security clearance arrangements for the ORR previsit. Complete DOE F 5631.20 (277's), Request for Visit or Access Approval forms for previsit team. Discuss access requirements with the site point-of-contact before the previsit to determine which Sigmas are needed. Discuss security procedures for any uncleared team members. • Make sure the team has maps of the site and how to get to the site and hotel from the airport. Maps should be requested from the site. • Make contact with site coordinator to introduce and discuss logistical questions. Discuss sigma/security requirements for cleared and uncleared team members and if there are any special procedures to bring in equipment, such as laptop computers, or supplies such as computer disks. • Verify that all team members' clearance information and all necessary forms for access to the site have been forwarded to the site. Verify that this has occurred at least one week prior to the previsit. • Complete the necessary forms for access to the site for all team members and forward the data to the site. • Verify that all reference material needed by the team from HQ has been provided to team members four to five days before the team's arrival for the previsit and ORR. Team Member Preparation for both Previsit and ORR TM • Each team member should receive and review their data packet. The data packet provides team members with information on the site, their assignments, including deliverables, and responsibilities of the team member. • Each team member is to contact the Hotel to confirm their reservation and guarantee it with a credit card. • Each contractor is responsible for providing clearance information to the RC. Provide the following clearance information: 1. team members' name 2. social security number 3. date of birth 4. organization/company name 5. the type of clearance held. DOE-HDBK-3012-96 The Health unit in the A-wing basement in GTN can provide medical exams for fitting respirators to1 federal employees only. Contractors will need to arrange for this on their own. 5 TM • If special training or fittings are required, such as respirator fitting, for access to certain facilities, arrange to have any pre-conditions completed before arrival for the previsit. For example, respirator fitting requires a medical exam before the fit test can be administered . Other special training could include Radworker II and1 Hazardous Waste Operations Training. The TL will inform all team members of any special requirements. • Team members are to contact their counterpart during the preparation phase to obtain information they feel is necessary or to arrange for additional documentation to be readily available onsite. However, keep these requests to a minimum. • The site appointed counterpart is a crucial element to the ORR inspection. They should be in constant communication with the team member during the review and will provide the best source of site specific information and access available to the subteams.

PREVISIT: The previsit is a fact finding and familiarization visit conducted by as many team members as possible. A comprehensive previsit lays the foundation for a thorough and efficient ORR and normally satisfies the requirement for team members to be familiar with the facility/process. The following are items to consider for completion during the previsit. TL • Brief site management on the mission, the methodology, and the scope of the ORR, as well as the logistical support required from the site. • Arrange for the Operations Office/Area Office and/or contractor management to brief the team on safety, security, the organization, processes and operations of the facility. Specifics of the startup (e.g., initial shut-down, restart following upgrade, restart following a safety shut-down) should be included in the briefing. • Complete final review and editing of the CRADs. Have TMs provide their proposed CRAD changes prior to the end of the previsit. • Attend a walkthrough at the facility/process and observe any current operations to acquaint team members with the facility and processes in question. TL • Attend any special training or fitting, such as respirator training and fitting, which might be necessary to allow for access to the facility during the ORR. Verify that all DOE-HDBK-3012-96 For example, respirator fitting requires a medical exam before the fit test can be administered. The2 Health Unit, in the A-wing basement of Germantown, provides these exams to federal employees only. 6 team members have met any pre-conditions prior to their arrival at the site .2 • Develop a preliminary listing of Shift Evolutions and Interviews and provide it to the site so they can plan and schedule events and personnel availability. Each team member should provide input prior to completion of the previsit. Efforts should be made to coordinate the interviews among team members to minimize the impact on site personnel. • Schedule a counterpart meeting. Each team member should have acquaintedhim/herself with their counterpart and developed their inspection requirements during the preparation phase. • Arrange for any documents designated as required reading for team members to be available to team members. Scheduling periods of free time during the previsit to allow this to be accomplished is valuable, particularly if any of the required reading is classified. • Investigate other or additional hotel accommodations for the ORR if necessary. TM • Review the ORR Implementation Plan including the Criteria and Review Approach Documents (CRADs). Ideally, each team member should have received an advance copy of the plan to enable a thorough preview of their assigned objectives and criteria. • Team members should arrive at the site as scheduled, having met any prerequisites for site access which were provided by the TL. • During the previsit, Team Members will: a. Receive briefings by the Operations Office/Area Office and/or contractor management on safety, security, the organization, processes, and operations of the facility as well as, specific information concerning the nature of the restart or startup. b. Walkthrough the facility or process area to observe any current operations to acquaint team members with the facility and processes in question. TM c. Complete any special training or fitting, such as respirator training and fitting, DOE-HDBK-3012-96 7

which might be necessary to allow for access to the facility during the ORR. d. Develop their Evolution and Interview List(s) to be provided to the site so they can plan and schedule events and personnel availability. e. Meet with their counterpart and develop a methodology for completing their assigned criteria. f. Expect and plan to be in regular communication with the site-appointed counterparts as they will provide the best source of site specific information and access available to the team member. g. Complete any required reading of documents that cannot be accomplished readily elsewhere, e.g. classified documents. h. Provide proposed CRAD changes to the TL before the end of the previsit for inclusion in the Implementation Plan. Initial Team Meeting: The following specific items are suggested for discussion. TL • Define the ORR team's relationship with the site. The ORR is conducted to verify the site's readiness to safely conduct operations. • Emphasize that the ORR inspection is an arduous and time consuming process for which team members are expected to provide their complete and undivided attention. • Establish that the TL is responsible for all aspects of the inspection. • Define the expected schedule for the ORR, including the time frame for submission of all deliverables (see Appendix 9). • Explain the need for and use of the various forms: - Assessment Forms (Form 1s) are used to document the methods and actions taken in the criteria evaluation process. Additional guidance is provided in Appendix 4 of the ORR Standard and Appendix 10 of this document. - Deficiency Forms (Form 2s) are used to document the findings uncovered during the criteria evaluation process. Findings are to be documented (i.e., a Form 2 drafted) as soon as there is reasonable evidence to substantiate a finding. This allows the TL to present site management a daily briefing of emerging issues and allows the site to better facilitate the finding validation process. Additional guidance is provided in Appendix 4 of the ORR Standard and Appendix 10 of this document. TL • Provide administrative information such as area phone numbers, dining locations, DOE-HDBK-3012-96 8 etc., at this meeting. • Advise team members to keep the TL abreast of all concerns and new developments as they arise to minimize confusion and mis-communication as information is collected. • Advise team members that participation in any site activity by the team members (e.g., special tours) not directly related to the review must be cleared by the TL. All discussions with site management outside the assigned CRADs should also be cleared through the TL. • Insist that all complaints, concerns, suggestions, and information be passed through the TL as soon as possible. Identify problems with counterparts to the TL. • Advise team members not to meet with external regulatory or oversight agencies, such as state environmental protection agencies, without the permission of the TL. Exceptions must be approved in advance with the TL who must notify the Area and/or the Site Office Manager. • Advise team members to strictly adhere to the need-to-know criteria in dealing with classified shapes, components, data or processes. Team members should immediately identify any difference of opinion which arises with the site on access to information.

• Explain that no contractor business development will take place onsite. Any such activity will result in removal from the ORR. This does not mean that a contractor must refrain from giving out a business card, if asked. It does mean they are not permitted to solicit business from site personnel. • Explain the format to be used for completing the final report and how the team members' input will be integrated into the report. PREPARING FOR THE ORR Team Leader Responsibilities TL • Finalize the ORR Implementation Plan: The Implementation Plan should be finalized immediately following or even during the previsit, if possible. Further information concerning development of the Implementation Plan is contained in section 5.9.2 and Appendices 1 through 4 of the ORR Standard. DOE-HDBK-3012-96 9 TL • Prior to release by the TL, the plan should be forwarded to the Office of Environment, Safety and Health (EH) and the Defense Nuclear Facility Safety Board (DNFSB), if applicable, for review and comment, as discussed in section 5.3.2 of the ORR Standard. It is important to remember that although neither EH nor DNFSB approval of the Implementation Plan is required, it is appropriate to allow adequate time for their review and comments. The TL should make a good faith effort to resolve all comments. • Establish ORR Length: In developing the ORR schedule, several factors need to be considered, among them the complexity of the facility/process being reviewed, the number of team members available, and the number of interviews and evolutions to be conducted. Experience has shown that even limited scope ORRs need a minimum of a week for field work. Several days are then needed to analyze the data, conduct follow-up, and develop coherent and informative conclusions, as well as to complete the extensive administrative requirements prior to leaving the site. Two weeks as a general guideline has been a challenging but achievable goal. Review Coordinator Responsibilities RC • After ORR dates are confirmed, reserve a block of rooms and send rooming list to the hotel with names and check-in/check-out dates, if required. Make the reservation under the DOE ORR team. Make arrangements for a meeting room at the hotel for the initial team meeting, if necessary. • Prepare security clearance forms for all team members. Complete form DOE F 5631.20 (formerly 277) Request for Visit or Access Approval for team. 277's are not completed for uncleared team members. • Provide the site point-of-contact with a list of names and social security numbers in order for them to track clearance status. Ensure the site security office has received a copy of this list either from site point-of-contact or by directly faxing them a copy. • Assist in preparing data packets for team members (see Appendix 6). The following information is provided by the RC: a. the hotel meeting room number, if applicable; b. where and when to get badges; c. badge office hours; d. special arrangements for any uncleared team members or personal computers; DOE-HDBK-3012-96 10 RC e. what to do in case a team member gets delayed and can't be at the right place at the right time; f. if any special training is required; g. where the onsite team room is located; • Provide the following: - blank forms with the reviewer's objectives and criteria filled in, and provide the

same information on a diskette; - Maps; - Detailed schedule, (Appendix 7); - Team member assignment and counterpart matrix; - Team member deliverables, (Appendix 9); - DOE O 425.1 and Standard DOE-STD-3006-95; and - Sample format for Forms (Appendix 10), if the TL chooses to use this format. • Make arrangements to mail data packets to team members. Packages should be delivered to team members at least two weeks before the beginning of the ORR. • Keep the site point-of-contact informed of any team member/schedule changes by updating and faxing the team member assignment chart. Make sure latest date is on the chart. • Keep all team members informed of the status of progress and potential or actual changes in ORR start dates. • Prepare package to mail to hotel. The package should include any information/documents the TL and DOE-HQ team members need for the ORR and the standard formats/examples such as latest team member assignments, latest agendas and schedules; computer disk with all ORR information; blank Form 1s and Form 2s; previous ORR closeout summary examples; and the template for the word processing software to be used. Mail the package to the hotel. The package should arrive at the hotel before the team arrives. ORR DUTIES The ORR begins with the arrival of the team at the hotel or site. TL • Ensure all team members have completed all access requirements of the site. • The TL normally meets every morning with site managers to brief them on the issues raised by the previous day's review activities. The TL presents the site with draft Form 2s detailing the findings uncovered during the previous day's inspections. This greatly enhances site/team communication and allows the site to better facilitate the DOE-HDBK-3012-96 11 validation process. TL • Hold daily team meetings, usually held in the afternoon, to discuss emerging issues. - Functional Area Leaders, if assigned, or team members brief the TL on the issues uncovered during the day (this is not intended to be a travelogue of the day's events). Issues deemed sufficiently substantive by the TL will be drafted onto Form 2s. - Some duplication of effort is to be expected and it can often lead to beneficial cross-talk and fuller understanding of the issues. - Issues raised may be assigned to a different functional area or subteam if it is felt that better coverage will result. - Any disputes between functional areas or subteams that occur are resolved by the TL. • Additional guidance for the conduct of the ORR can be found in section 5.4.6 of the ORR Standard. Opening Meeting at Site: This meeting is the "kickoff" meeting for the commencement of the ORR. Items to consider for completion during this meeting include: TL • Make introductions. • Allow for a series of briefs by the site explaining the site's management and organization, its functions, operations and interfaces. • Brief the audience on the ORR process and the ORR schedule. • Brief site audience and team on the rules for interaction between team members and site personnel. The team will not act as safety monitors nor direct operations, etc. Team Meetings: Items Which Should be Reiterated Throughout the Inspection: TL • Encourage the identification of any unclear issues. They still need to be identified so that others may consider them at a later date. • Re-emphasize that team members should focus on that portion of the organization directly responsible for their area. Contact outside their area, specifically including interviews with senior site management, must be discussed in advance with the TL.

DOE-HDBK-3012-96 12 Team Meetings (continued) TL • Insist on validation of facts. ORR teams validate facts they collect on a continuous basis. The Area or Site Office and the counterparts are the primary points of contact for validating facts. • It is also important to try and gauge the response of site management to the conclusions that are drawn from the validated facts. • Maintain a running record of the important issues brought out at each nightly meeting. • The technical editor or review coordinator will review all Forms and summaries. Team members are encouraged to use the technical editor's assistance. If appropriate, all forms will be reviewed by a derivation classifier. TM • Team members are responsible for providing the TL a draft Form 2 for each finding by 0800 the next morning. This allows the TL time to review all Form 2s prior to the daily site briefing. • Validation of facts is a must. The Area or Site Office and the counterparts are the primary points of contact for validating facts. If facts have not been validated, don't use them. If you have validated facts, know with whom or by what reference, you validated and be sure that they know and that they have the authority to validate. • Assessment Forms (Form 1s) are used to document the methods and actions taken in the criteria evaluation process. Team members are encouraged to update their Form 1s daily to ensure complete record keeping of all inspection activities. The TL may choose to require each team member provide daily draft Form 1s for his/her review. Additional guidance is provided in Appendix 4 of the ORR Standard, and Appendix 10 of this document. • Deficiency Forms (Form 2s) are used to document the findings identified during the criteria evaluation process. When issues are unclear, they still need to be identified so that others may consider them at a later date. If you begin to have suspicions about something, report the concern and let the site follow it up. Avoid delaying the drafting of a Form 2 until there is overwhelming evidence as this may excessively delay the validation and correction processes. Additional guidance is provided in Appendix 4 of the ORR Standard and Appendix 10 of this document. During the ORR, the RC works closely with the TL providing administrative and logistical support. RC • Hand out latest team member assignments and agendas/schedules and any guidance DOE-HDBK-3012-96 13 on writing Forms and summaries (see Appendix 10). • Provide administrative and logistical support to team members. • Get to know the site, i.e., locations of library, copy center, fax machine, supplies. • Find out about site-specific security procedures, i.e., do team member notes need to be reviewed for classification? If so, where? • Check all computers and printers to make sure they are working properly. Arrange for repairs or replacement if necessary. • Load Form 1 and 2 files onto all computers. • Provide sample Functional Area Summaries to each functional area leader. This helps ensure the inputs for the final report are consistent in content and format. • Attend team meetings and manage the Form 1 and Form 2 tracking system(s). A suggested procedure to manage the forms is as follows: (1) get a hard copy with diskette from each inspector or subgroup team leader; (2) the form(s) are logged in; (3) a copy should go to the TL as soon as possible and a copy is placed in the logbook; (4) after the TL reviews and comments on the form and it has been edited by the technical editor, the form is returned to the inspector for revision or signature; and (5) revisions and editing are then accomplished between the inspector and the technical editor with the TL reviewing only the final edited and signed version submitted by the inspector.

Final Team Meeting: This is the last opportunity to work directly with team members and clarify any questions that have persisted throughout the ORR. Final Form 1s and Form 2s are due prior to this meeting as is a draft summary evaluation of the assigned objectives for inclusion in the final report. This is the last opportunity to discover and address dissenting opinions and conflicts prior to presenting the ORR results to the site. Any such issues must be addressed at this time to avoid any confusion at the Close-out meeting. TL • In preparation for the Close-out Meeting announce and discuss the following items: a. The closeout meeting will be kept short. b. Attendance will not be limited. Anyone interested may attend. c. All findings are subject to the approval of the TL and the Senior Advisers, if applicable. • Discuss the final report schedule and deliverables. TL • All findings should be discussed at this meeting. A consensus must be reached by DOE-HDBK-3012-96 14 the team on each findings' categorization as prestart, post-start, observation, or, if appropriate, deleted. Discussing all findings at this meeting will help bring the team to a unified position on the general status of the facility or process being reviewed. TM • All Form 1s and Form 2s should have been submitted to the TL. They must be signed-off by the TL. • Summary evaluations should also have been completed and submitted to and signed-off by TL prior to this meeting. • Be prepared to justify all findings. Close-out Meeting: The close-out meeting is the forum by which the TL informs the site management of the major issues identified during the ORR. There should be absolutely no surprises at this meeting as counterparts and site staff should have been kept well apprised of all issues during the conduct of the ORR. TL • Prepare a discussion based on the Form 1s, Form 2s, and conclusions drawn by the ORR team. RC • Assist the TL in preparing the close-out briefing. • Make arrangements for the site to mail information that was brought with the ORR team or collected during the review to the team members. This package should be sent on the day of the close-out meeting. • Make copies of Form 1s and 2s for dissemination to site management. TM • All team members will remain near the close-out meeting and be prepared to answer questions regarding their findings should the TL need their support. POST ORR: Final Report TL • The Senior Advisers, if applicable, must review and concur with the final report before it is issued. • Consistent with the number of findings, the final report should be issued within two weeks of the end of the inspection. The TL is the issuer of the ORR report. TL • The Final Report should be sent to the restart authority with copies to the applicable program office, operations and/or area office, EH, and the DNFSB if applicable. DOE-HDBK-3012-96 15 • Each TM should sign the final report indication his/her concurrence with the conclusions of the ORR in his/her area of responsibility or review. • If a TM has a dissenting opinion, refer to DOE-STD-3006-95, Planning and Conduct of Operational Readiness Reviews, for guidance. • Detailed guidance for the preparation of the final report is contained in sections 5.5 and 5.9.3 of the ORR Standard. • Ensure that all ORR files that may be needed in the future are maintained in one location as determined by the DOE line management for the site.

• The completion of the ORR, and the finalizing of the report do not signify the end of the ORR process. Several actions may require the participation of the TL including briefings, interpretations and possible defense of findings, review of action plans, and review of finding closure plans. Additional guidance can be found in section 5.7 of the ORR Standard. Finding Closure and Team Follow-up: TM • Team members may be called upon to assist the TL in several ways including briefings, finding interpretation, review of action plans, or review of finding closure. The TL will notify team members when such assistance will be needed. Section 5.7 of the ORR Standard provides additional discussion concerning post-ORR activities. RC • Draft thank-you letters to site for TLs signature. • Collect, condense, index, and file pertinent ORR information to include: a. all correspondence related to the ORR; b. all final Form 1s and Form 2s signed by the inspector and TL; c. the Evolution and Interview List and schedule; d. the approved Plan of Action; e. the approved Implementation Plan; and f. the approved Final Report with all signatures from the team members and TL; g. facility basis documentation. DOE-HDBK-3012-96 16 LESSONS LEARNED FROM PREVIOUS ORR/RAs The following items were learned from conducting previous ORRs and Readiness Assessments (RAs) from which both line managers and reviewers could benefit. ACHIEVING READINESS: The successful completion of any ORR will be strongly influenced by the degree to which readiness to start program work has been achieved prior to the start of the ORR. While it is the responsibility of line management to achieve a condition of readiness, experience indicates that the ORR Team Leader (TL) can influence the success of the process to gain readiness during his/her preparations for the ORR. In many cases, the potential for the particular problem to occur will become evident during previsits or other ORR preparations. The following lessons learned reflect areas in which the ORR TL and Team Members (TM) may improve the success of the ORR. Experience shows that prerequisites should provide significant detail and be fully measurable in order to permit line management to track each prerequisite to completion. The ORR/RA standard stresses the fact that the prerequisites should be tied to the core requirements, which are what will be evaluated in the readiness determination. So, ensure that the ORR Prerequisites specified in the Plan of Action support achieving readiness. A thorough Management Self-Assessment to assist line management in verifying that readiness has been achieved is an important final step in preparing for the ORR. The ORR Implementation Plan should reflect an intention to review the results of the management self-assessment. Inadequate, incomplete, or undefined incorporation of the safety basis documentation into procedures and policies has frequently resulted in delays in starting an ORR or in significant findings during the ORR. This is frequently caused by late development and approval of the safety basis documentation. Line management may not fully grasp the time and effort necessary to develop and implement all of the "flow down" procedures and surveillances necessary to put the requirements of the approved safety documentation into practice. Ensure full implementation of the safety basis.

Inadequate or incomplete evidence files or other verifiable documentation which demonstrates that the prerequisite conditions have been met may result in an inability to verify readiness has been achieved. The implementation plan should clearly indicate the intentions to review these evidence files. A responsible representative of the line management team must be prepared to demonstrate to the ORR TM that the conditions required to be met prerequisite to resumption, have actually been met. In most cases, this individual will be the counterpart who is assigned to work with the individual ORR TM. The counterparts should be identified and utilized during the previsit. DOE-HDBK-3012-96 17 Lack of understanding by assigned counterparts of their responsibility to support the ORR TM. The counterpart must present the information and documentation requested. The counterpart must accept responsibility for gaining information and resolving questions. The counterparts should be fully dedicated to a single ORR TM for the duration of the ORR field work. This expectation should be clearly defined during the previsit. Lack of experience on the part of line management as to the expectations of the ORR team in the areas of drills and evolutions expected to occur during the ORR will lead to difficulty in completing established objectives. The TL and TM must devote considerable attention to communicating expectations to responsible line management at the facility level. Inadequate validation and verification of operational or maintenance/surveillance procedures which are newly prepared or recently modified will result in readiness not being achieved when the ORR starts. The adequacy and maturity of these procedures should be evaluated during the previsit. If problems are apparent, they should be identified to senior management at the end of the previsit. Lack of formal, structured preparation by DOE line management of the personnel, programs, and documentation to be evaluated during the ORR will delay completion of required activities. DOE line management should be encouraged to conduct a Management Self-Assessment to ensure that prerequisite conditions for which they are responsible have been met. Potential problems in this area should be apparent during the pre-visit and if noted, pointed out to senior DOE management at that time. Note: It will usually be apparent to the observant team leader during the previsits and other ORR preparations when the problems with achieving readiness will become a problem during the ORR. Although the ORR TL will not be able to deal with the problems directly, his identification of the problems to the appropriate DOE or contractor line manager should be considered. ITEMS TO NOTE BEFORE PROCEEDING WITH A DOE ORR: Thoroughly review the ORR Plan of Action (POA) which, among other things, defines the scope of the ORR and lists the prerequisites for the ORR. The success of the ORR will be enhanced if the POA provides specific detail as to the scope of each core requirement, and defines the prerequisites in sufficient detail. Prerequisites should be specific and define measurable conditions. If the ORR TL is identified prior to final approval of the POA, he should be included in the POA review and approval process and should ensure that the POA contains adequate detail to support a successful ORR. The POA should be approved as early as possible in the startup/restart process.

Gain familiarity with the facility and the project programs prior to the start of the ORR. Site access training, facility walkthroughs, and document reviews are essential for team DOE-HDBK-3012-96 18 members to gain the necessary familiarity with the project prior to the kickoff of the ORR. Unescorted and unrestricted access to the facility is essential to effectively complete the review. This will require some investment of time and money in the preparation process. Get agreement, during development of the ORR plans, between the facility contractor and the Department of Energy on the details of the operations that are available for demonstration. Coldruns, partial simulations, and full walkthroughs are options, but an effort must be made to achieve a demonstration as close as possible to actual operations as allowed by the current restrictions of the shutdown. Obtain and review the documented results of the contractor ORR/RA, including corrective action plans and evidence files documenting corrective action taken. Copies of corrective action documentation need to be readily available to the DOE ORR/RA team. Corrective action closure packages should be prepared in accordance with DOE-STD-3006-95, Section 5.7.3. Give consideration to requiring the contractor to deliver a completed set of surveillance procedures and authorization basis documents to the team leader as a prerequisite to the ORR/RA. Ensure that records, plans, and other documentation requested by the ORR/RA team are readily available, preferably in a central location. A review, several days in advance of the ORR/RA should be used to verify that what was requested is really there. In addition, this will reinforce the importance of the information requests by the individual team members. ADMINISTRATION OF THE RESULTS: Conduct a training session prior to the ORR/RA or no later than the beginning of the review process to communicate the expectations and deliverables required of the team. Format, content, and style (how to use and fill out) of Forms 1 (Assessment Forms), of Forms 2 (Deficiency Forms), and functional area summaries should be discussed to eliminate a compounding of the administrative load as the review progresses. Acceptable samples of these deliverables should be provided to all the team members prior to the start of the review. This guide provides explicit instructions on the use of the forms, and the team leader can use this to provide training to the team. The technical editor or the RC should be equally aware of the requirements and could conduct the training. Provide draft Form 2s to site personnel as soon as an issue starts to be identified. Be sure that it is understood that the forms are draft documents (and should be marked as such) to avoid legal or public record exposure issues. Providing these forms to the site provides them an opportunity to produce further information that may clarify the issue and creates an air of openness that will contribute favorably to the review. DOE-HDBK-3012-96 19 CONDUCT OF DRILLS AND OTHER OPERATIONS Specify requirements for Emergency Preparedness or facility drills prior to the start of the ORR/RA. Detailed guidance on drill expectations, e.g. type of drill, desired preconditions, scenario requirements, etc., should be provided to the facility prior to the ORR/RA. Ensure that site personnel understand fully the conditions which are desired to be simulated during each drill and performance demonstration. All documentation such as Radiological Work Permits, Work Procedures, etc., must reflect the simulated condition.

Define, during the preparation phase, the role, authority and responsibilities of the ORR Team when it is monitoring the conduct of an emergency exercise or drill. The ORR Team must not disrupt the flow of the exercise. The ORR team is evaluating both the facility and the emergency preparedness training organization and drill control. These roles, authority and responsibilities require definition and must be fully understood by the facility managers. Evaluate whether the contractor ORR/RA ensures the integration of equipment operability, procedure viability, and training of the operators in a performance-based methodology. Additionally, a detailed plan for the progression from completion of the ORR/RA to startup should be verified if a significant difference exists between the operations that can be demonstrated and normal operations. LOGISTICS AND SUPPORT Communicate logistical needs and requirements to the site well in advance of the review. For example, - Specify what documents need to be available to the team and that these documents need to be in a central location for easy access by the team during the review. - Specify that computers need to be up-to-date models. Printers and copying machines, with adequate supplies of paper, toner, etc. need to be dedicated to the team for the duration of the review. - Request administrative support personnel (typist with WordPerfect, or selected software, experience if at all possible), as they are especially helpful when preparing the final pieces of documentation for the final report. - In cases where the review involves classified facilities/operations, schedule derivative classifiers to come to the work area and review the draft documentation at the end of the day during the last week of the review. - Request sufficient office space and computers to support, at a minimum, each functional area sub-group. Working in one large room is difficult since many discussions take place among site, review team, and field office groups. DOE-HDBK-3012-96 20 -. Investigate whether laptops can be brought on-site. This is usually not an option when the review involves classified material. Some form of property accountability pass may be required by the site. Establish clear formal and informal communications between the facility and the review team as early as possible. Define the "rules of the road." This will improve the effectiveness and efficiency of the ORR/RA. Explain the open, "filters off" nature of the daily team meetings, and that other briefings can be held with management. While important, the number and the length of these meetings should be minimized to allow the members of the review team to do their job. TEAM ISSUES Emphasize that team members are dedicated for the duration of the review. If a team member completes their review before the end of the ORR/RA, and is authorized to leave, their complete Forms 1 and 2 must be signed-off (approved) by the TL before they leave the site. If a summary is due from the team member, the summary must be accepted by the TL prior to the team member leaving the site. Emphasize to team members the need to familiarize themselves with the Implementation Plan and the facility/process under review before the ORR/RA. This should be accomplished through the team members' qualification process, which should establish the minimum requirements for facility familiarity, prior to the review. Additionally, the TL is responsible in the team selection to ensure that the team members have the required technical expertise (education/experience) to perform a valid review of the functional area assigned. This should also be documented through the selection and acceptance process.

Ensure that team members fully understand the scope of the ORR/RA and the time frame in which it must occur. In some cases, time management decisions and sampling plans will be important to successful completion of the ORR/RA within the required schedule. Ensure counterparts have been assigned to each team member or functional area group. Early identification (usually during the pre-visit) of documents needed, interviews expected, evolutions and drills requested is helpful to both the team member and the counterpart. Counterparts need to understand they should be prepared to present the information required. Team members need to be respectful of their counterpart time constraints and schedule as few after-hours meetings or interviews as possible. Capture issues and observations in writing as soon as possible. Forms 2 should be prepared shortly after an issue is identified. Forms 1 should be prepared shortly thereafter to capture the full extent of the finding. This will enhance the flow of communication throughout the review and contribute to openness on the part of both the team and the facility. DOE-HDBK-3012-96 21 DEVELOPING THE IMPLEMENTATION PLAN Ensure the objectives and criteria clearly encompass all aspects of the Core Objectives required in the approved Plan of Action (POA). Generally, this will be all core objectives in an ORR and some subset of the total for an RA. The ORR standard breaks the twenty Core Requirements into 36 objectives which translate easily into functional areas. Experience has shown this methodology to be successful. Additionally, the "geographic" scope of the ORR/RA must be considered. The POA should specify the bounds of the facility, systems, and personnel involved in the activity under review and the approaches for the objectives and criteria should reflect these boundaries. Confirm that the criteria review and approach documents (CRADs) are written with a clear understanding of the facility systems and processes under review. A generic CRAD should be tailored during the pre-visit to successfully complete a through and critical review. Without this understanding and modification of the CRADs, the review effort will suffer. Fully involve the team early in the process for training and implementation plan development. Early dialogue with the facility to gain understanding of the activity in progress and the contractor activities in progress is particularly useful. The implementation plan used by the contractor and DOE should parallel in numerous respects. Give the POA and the Implementation Plan to oversight groups (EH, DNFSB, State Agencies as required) as soon as possible. Early review and input from all stakeholders will reduce last minute perturbations. Determine the "mode" of the facility in reference to Standards Requirements Information Documents (S/RIDs), Necessary and Sufficient, Rule-making, the DOE Order system and other initiatives related to order compliance. Conduct the review appropriately with respect to the mechanism selected by the facility. DOE O 425.1 requires a statement of order compliance with respect to DOE Orders or S/RIDs. Statements to this effect should be included in the final report in addition to the overall evaluation of the compliance posture required. DEFINING THE SCOPE AND GRADED APPROACH Define the scope of the DOE ORR consistent with that of the contractor ORR. Differences between the two result in difficulties in performing the DOE ORR. DOE O

425.1 provides a structure for such consistency. Differences will occur in the evaluation of the core requirements specifically related to DOE management, which are necessarily absent from the contractor review. DOE-HDBK-3012-96 22 Go beyond the defined scope in a case where an ongoing program that supports the process or facility under review but which is outside the scope of the review, presents potential problems. If the team is to make a judgement as to the ability for operations to be conducted safely and it requires investigation at the interface of the process/facility with the supporting program, then going beyond the defined scope is proper. Define the graded approach as it applies to each CRAD as specifically as possible. POST ORR CORRECTIVE ACTIONS Although the ORR/RA follow-up corrective actions are not the responsibility of the TL, line management may ask for input and advice as to the necessary action(s) to correct the issues identified during the ORR/RA. Experience has shown that the value and effectiveness of the ORR/RA can be significantly decreased by ineffective corrective actions to resolve the issues identified during the ORR/RAs. The TL, through interest in the effectiveness of the overall ORR/RA process, retains an interest in the post ORR/RA corrective actions. In general, the following items need attention: Facility management sometimes starts taking immediate action to correct the symptoms identified by the ORR/RA team without adequate attention to understanding the root cause and programmatic basis of the issue. As a result, the corrective action may be incomplete, shallow, and short lived. DOE line management review of the status of the contractor ORR/RA corrective actions should require corrective action closure packages for the contractor ORR/RA which meet the requirements of DOE-STD-3006-95, section 5.7.3. In that way, the contractor may more effectively evaluate their own corrective actions before reporting readiness to proceed to DOE. When DOE line management forwards the DOE ORR/RA to the responsible contractor, the actions expected as to the contractor's corrective action should be clearly stated. Unclear or unstated expectations result in false starts and failure of the responsible contractor to meet DOE's expectations. This in turn has resulted in delays and incomplete actions following several ORR/RAs. In those cases where issues have been identified as the responsibility of a DOE field organization, DOE-HQ should formally communicate corrective action expectations to the responsible field organization. The expectations should include corrective action plans and closure packages. DOE-HDBK-3012-96 23 MISCELLANEOUS LESSONS Engage a technical editor, if at all possible, during the second week of the review. The final report left at the close-out briefing, although a draft, is the site or facility's first encounter with the full results of the review and as such warrants a technical editing. This person should stay until all Forms 1, Forms 2, and summaries have been approved, the executive summary edited, and the final report formatted, collated, and printed. The introductory material (cover sheet, background, scope, etc.) can be written before the review is started, thus reducing the workload during the review and allowing the team to concentrate on issues at hand. Do not assign any CRADs to the TL, however technically qualified.

Identify an assistant TL. Having an assistant TL can greatly improve efficiency in the conduct of the ORR/RA. An assistant TL is particularly beneficial during the labor intensive ORR preparation phase. Select an RC with previous experience in the ORR/RA process, as this function is essential in the assimilation of the data provided by the team members. Without accuracy and timeliness in this function, conclusions are subject to question. Select the TL carefully. The TL should be senior to the team members. He or she should not be a peer or a co-worker of the team members. Selection of the appropriate TL is important in that the TL must exercise some degree of control over the team. Evaluate OJT, drills, and other training evaluation against the established guidance in DOE Orders, standards, guides, and handbooks. This helps eliminate the subjective aspects of the review. Additionally, this intention should be communicated to the contractor for the conduct of the contractor ORR/RA. GENERIC LESSONS LEARN WHICH APPLY TO THE DESIGN, CONSTRUCTION, OPERATION, AND DECOMMISSIONING OF DOE FACILITIES During startup of new facilities, numerous cases have been observed in which the surveillance requirement supporting the safety basis could not be completed, or if completed did not properly verify the safety basis requirement. Surveillance procedures should be completed, validated, and executed as part of the startup program, prior to the implementation of the supported Technical Safety Requirements (TSRs). Actually executing the surveillance will reveal such problems as: test points which are not installed; test points which are inaccessible; and other interfering interlocks or functions. These problems have occurred repeatedly in previous ORR/RAs. Additionally, determination must be made that the surveillance actually tests the function or DOE-HDBK-3012-96 24 protective action upon which the safety basis depends. Previous ORR/RAs have revealed numerous occasions in which surveillances have been conducted and through either invalid acceptance criteria or a misunderstanding of the as-built configuration, the surveillance did not test the required functions. DOE-HDBK-3012-96 25 APPENDICES STANDARD FORMS AND LETTERS Appendix 1: Key Points to Discuss during Initial Contact with Site Appendix 2: Site Contact Memorandum Appendix 3: Logistics Appendix 4: ORR Preliminary Schedule Appendix 5: Previsit Schedule Appendix 6: Checklist for ORR Team Information Packages Appendix 7: Detailed Schedule for ORR Team Appendix 8: Transmittal letter for data packages to team members Appendix 9: Team Member Deliverables Appendix 10: Instructions and Samples for Completing ORR Forms Appendix 11: Prestart and Post-start Determination Criteria DOE-HDBK-3012-96 26 INTENTIONALLY BLANK DOE-HDBK-3012-96 APPENDIX 1-1 APPENDIX 1 KEY POINTS TO DISCUSS DURING INITIAL CONTACT WITH SITE Refer to the Plan of Action, and: 1. Contact DOE Headquarters' Line Management Director of Facility. Facility Director's Name Telephone Number 2. Identify yourself as the Team Leader for the ORR. 3. Discuss the tentative review schedule. Previsit dates: to ORR dates: to 4. Explain overview of review process (if requested). Previsit and Preparation - Site Visit: 2-3 day for site and facility familiarization, walkthrough, and presentation to site of inspection process. - Information needed from site: (list to be provided)

- Counterparts: Site expected to provide list of counterpart(s) for each area. Onsite Review Period - Purpose: Identify findings and observations for the safe startup of facility process(es). - General: Up to 30 team members, approximately a two week period, though with a graded approach, as few as 10-15 team members may be needed for one week. - Logistic Support: Conference room, work space, computers, phones, fax, etc. (list to be provided). 5. Request point-of-contact at site to support review and to receive formal correspondence. Name: _______________________________________________ Telephone number:____________________________________ 6. Leave your telephone number and advise site contact that written confirmation will follow. DOE-HDBK-3012-96 APPENDIX 1-2 INTENTIONALLY BLANK DOE-HDBK-3012-96 APPENDIX 2-1 APPENDIX 2 SITE CONTACT MEMORANDUM Refer to the Plan of Action. Operations Office or Area Office Manager Operational Readiness Review of [Facility] . An Operational Readiness Review (ORR) of [Facility] will be performed. [Site point-of- contact] has been identified as the lead Department of Energy contact at the site. [Team Leader] will lead the review team. Enclosed are a preliminary schedule for the overall review and a detailed schedule for the onsite portion, an agenda for the pre-review visit, a list of support required for the onsite portion of the review, a list of site documents required to prepare the team for the review, and an ORR summary document. Please forward one copy of each requested document to [Team Leader] by [Date] . Please contact [Team Leader] (phone #) if you wish to discuss these items or any other aspect of the review. Your assistance is greatly appreciated. Team Leader Attachments cc w/Attachments: Manager, Area Office Program Office Director Facility Manager _______________________________________ *Attachments ORR Logistics (Appendix 3) ORR Review Schedule (Appendix 4) ORR Previsit Schedule (Appendix 5) DOE-HDBK-3012-96 APPENDIX 2-2 INTENTIONALLY BLANK DOE-HDBK-3012-96 APPENDIX 3-1 APPENDIX 3 OPERATIONAL READINESS REVIEW LOGISTICS The following list suggests items the TL should consider. It should be personalized by the TL for each ORR. These items should be discussed and agreed upon during the previsit. 1. The team will require work space for approximately [#] people to review documents, conduct interviews, prepare reports, and conduct meetings. 2. Work spaces should be equipped with several IBM compatible PCs and laser printers. A current version of WordPerfect (or other preferred software) should be installed on each system. One copy of a familiar graphics program should be available on one of the team support staff's computers. 3. Work spaces should have phones with contractor and DOE phone books. 4. The team will require ready access to photocopy machines. 5. The team will require ready access to a facsimile machine. 6. A list of support personnel or a single contact to repair equipment, receive phone messages, and aid in logistical problems should be available at the start of the review. 7. Adequate visitor parking should be available for team members. 8. The team may require ready access to a classifier and a classified document repository.

9. The team will require ready access to the following documents in a convenient, centralized location at the site: (Note: the following are examples of documents that the TL can request to have available during the ORR). The TL may require the team members to review these documents to accomplish qualification for site familiarity, thus they may be required during the previsit. The TL should communicate these desires to the site as soon as possible. - DOE Orders and mandatory standards (DOE 5480.4). - General facility schematics. - List of Facility Representatives and Facility Managers (per DOE 5000.3). - Hierarchy of documents: Top-level policy and site wide procedures down to operating instructions. - Current listing of titles, numbers, and revisions of manuals/procedures. - Implementation plans submitted to Headquarters for DOE Orders. These would include all of the Orders specified in DOE 5480, DOE 5500-series Orders, DOE 4330.4, and DOE 5700.6. - Any exemption, exception, deviations, variances, or waivers requested for ES&H Orders. - DOE and Contractor ES&H oversight plans/programs, and self-assessment program manuals. - All DOE Field/Area Office and Contractor self-assessment reports for the past 12 months. * Contractor and DOE organization charts with management/supervisor names. DOE-HDBK-3012-96 APPENDIX 3-2 * Functional description for environmental, safety, and health (ES&H) organizations (NQA-1). * General site map. - Contractor ORR and/or Management Self Assessment reports. - The Tiger Team/TSA Action Plan(s) and the most recent Status Report for the Tiger Team Assessment. - A list of functional review reports issued within the last 12 months by DOE- [ Field Office] regarding [Site] (DOE 5482.1). - All DNFSB, EH, NS, IG, EPA, OSHA reports issued within the last 12 months. - The most recent contractor triennial management review (required per DOE 5480.5) of the internal safety review system. - Site Development Plan (ref. 4300.1). - Annual Site Environmental Reports; the most recent, even if in draft form, and the preceding year (ref. 5400.1 Ch. II). - Environmental Monitoring Plan and it's associated Meteorological Information/Monitoring Program (ref. 5400.1 Ch. IV). - Spill Prevention, Control and Countermeasures Contingency Plans (if applicable). - Inventory of hazardous chemicals (those above Threshold Planning Quantities per SARA Title III) by room, area, or building, as appropriate to planning walkthroughs. (Remove portions that are classified). - SARA Title III Tier One and Two reports. - Most recent site-wide EIS and/or EA. - National Emission Standards for Hazardous Air Pollutants (NESHAP) report (if applicable). - All Type A accident investigations reports for the last two years. - OSHA 200 log from past two years. - Site Radiation Control Manual and associated implementation plan. - DOE and Contractor Quality Assurance Program Manual. - List of Quality Assurance plans and implementation procedures. - DOE and Contractor Conduct of Operations Manual. - List of Conduct of Operations plans and implementation procedures. - Operator training program documentation. - List of ES&H policies and procedures of the Field Office and Contractor. - Contractor's ES&H Manual. * List of applicable Technical Safety Requirements. * Applicable Safety Analysis Report(s). - Site-specific Explosives Safety Manual (if applicable). - Explosives Safety Policy Statements (if applicable). - Explosives Handling, Storage, and Disposal facility Design Bases/SARs (if applicable). - Site Fire Protection Manual. - Site Pre-fire Plans. - All Fire Department Procedures. - Fire Protection staff organization charts and job descriptions. - Staffing plans and any analyses relating to staffing for the past 12 months. - Applicable DOE Order Compliance Documents and Compliance Schedule Agreements. - Site Emergency Plans

DOE-HDBK-3012-96 APPENDIX 4-1 APPENDIX 4 OPERATIONAL READINESS REVIEW PRELIMINARY SCHEDULE Activity Date(s) Previsit Onsite Review Close-out Meeting Report Preparation Issue Report HQ Briefing Note: The above dates are tentative. The team intends to conclude identified activities within the allotted windows on the schedule, but adjustments may be made to accommodate particular issues that may require more or less attention. DOE-HDBK-3012-96 APPENDIX 4-2 INTENTIONALLY BLANK DOE-HDBK-3012-96 APPENDIX 5-1 APPENDIX 5 PREVISIT SCHEDULE NAME OF FACILITY Activity Date Participants Site Access Month, Day, Year Full ORR Team Previsit Start Month, Day, Year Full ORR Team Orientation Site Management Meeting Full ORR Team(1) Month, Day, Year* (a.m.) ORR TL Presentation Site Management(2) Month, Day, Year* (a.m.) Full ORR Team Counterpart Meeting Site Counterparts(3) Month, Day, Year* (p.m.) Full ORR Team Site/Facility Training and Full ORR Team Processing(4) Month, Day, Year* (p.m.) Facility/Process Walkthrough Full ORR TeamMonth, Day, Year* Previsit End Month, Day, Year Full ORR Team *Site to specify times and locations. Orientation Meeting - The goal is to receive from the site an overview of the site activities in progress and(1) any special operating guidance required for the ORR team members. Include in the presentation, all safety and security briefs to ensure ready access, overviews of the organizational structure, document hierarchy, assessment activities, occurrence reporting and tracking, training organization, lessons learned programs, and any other desired elements (approximate Time: 2 hours). TL ORR Presentation - The TL will make a short presentation on the ORR scope and process to the site(2) (approximate Time: 30 minutes). Counterpart Meeting - Each ORR team member or functional area is assigned a site counterpart. This(3) meeting provides an introduction of the site counterparts to the Team (approximate Time: 1-2 hours). Site/Facility Training and Processing - The goal is to receive from the site the training and processing(4) (e.g., respirator training and fitting) so as to permit all team members ready access to the site/facility during the ORR. DOE-HDBK-3012-96 APPENDIX 5-2 INTENTIONALLY BLANK DOE-HDBK-3012-96 APPENDIX 6-1 APPENDIX 6 CHECKLIST FOR ORR TEAM INFORMATION PACKAGES DESCRIPTION APPENDIX No. Transmittal letter. Appendix 8 Maps from airport to hotel/site. Provided by site. Schedules for ORR team. Appendices 4, 5, and 7 DOE ORR Plan of Action. Provided by TL DOE ORR Implementation Plan. Provided by TL Site organization charts. Provided by site. Team member assignment and counterpart matrix. Provided by RC ORR Order (425.1) and Standard Provided by RC (DOE-STD-3006-95) Contractor ORR Report. Provided by TL Site DOE Validation Report Provided by TL Other Required Reading Documents Provided by TL Team Member Deliverables List Appendix 9 Blank Form 1s and 2s on diskette (note: the objective and criteria Provided by RC should be filled in on the forms) DOE-HDBK-3012-96 APPENDIX 6-2 INTENTIONALLY BLANK DOE-HDBK-3012-96 APPENDIX 7-1 APPENDIX 7 DETAILED SCHEDULE FOR ORR TEAM [Month, Day, Year] Travel to . Optional: You may pick up your badge at . [Month, Day, Year] Team meeting at the site (Room , a.m./p.m.) [Month, Day, Year] Start ORR. [Month, Day, Year] Tentative date for Close-out meeting.

[Month, Day, Year] Travel from . DOE-HDBK-3012-96 APPENDIX 7-2 INTENTIONALLY BLANK DOE-HDBK-3012-96 APPENDIX 8-1 APPENDIX 8 TRANSMITTAL LETTER FOR DATA PACKAGES TO TEAM MEMBERS DATE: NOTE TO: [Name of Site/Facility] Operational Readiness Review Team Members FROM: NAME , DOE ORR Review Coordinator SUBJECT: Operational Readiness Review at [site] . Attached is the detailed schedule for the Operational Readiness Review (ORR) at [site] . Your badge will be ready for pick up at the Badge Office which is located . You will need access authorization for Sigmas . The badge office is open from a.m. to p.m. The site is providing a team room located in for our use throughout the entire review period. Attached are the functional area assignments; the review methodology for ORR team members; a [site] organization chart; team member duties and responsibilities; required reading assignments, list of deliverables, and the assigned Criteria and Review Approach Documents (CRADs). If you have any questions or if I can be of further assistance, please call me (301-903- ). DOE-HDBK-3012-96 APPENDIX 8-2 INTENTIONALLY BLANK DOE-HDBK-3012-96 APPENDIX 9-1 APPENDIX 9 TEAM MEMBER DELIVERABLES - Qualification Summary: due before or during the previsit. - Security clearance data: due as required by the Review Coordinator. - Comments on Criteria and Review Approach Documents: due before or during the previsit. - Evolution and Interview list: input due to TL prior to the end of the previsit. - Form 1 for each CRAD Objective assigned: due for signature before the final team meeting of the ORR. May consider daily draft. TL may require a current draft for review at any time. - Form 2 for each deficiency identified: due in draft form by 0800 the morning after the deficiency is identified and due in final form before the final team meeting of the ORR. - Final Report Summary: due in draft form before the final team meeting of the ORR and signed-off by the TL before leaving the site. - Final Report, including Form 1s, 2s, and functional area Summaries, reviewed, signed by the team member and TL by the end of the ORR. Any dissenting opinions must be prepared and submitted by the end of the ORR. - Lessons Learned for inclusion in the report and in later versions of this handbook. DOE-HDBK-3012-96 APPENDIX 9-2 INTENTIONALLY BLANK DOE-HDBK-3012-96 APPENDIX 10-1 APPENDIX 10 INSTRUCTIONS FOR COMPLETING ORR FORMS ORR ASSESSMENT FORM 1 OBJECTIVE: CRITERIA: CRITERION MET DATE: 6/ 4/96 YES NO OBJECTIVE: (2 spaces) Copy the Objective from the Implementation Plan. Criterion: (2 spaces) Copy the criterion as written in the Implementation Plan. Records Reviewed: o(F4) Cite a specific document (number, title, rev.) o(F4) Cite next document, etc. Interviews Conducted: o(F4) Use title, do not use name (use site identifer, as site will respond) Shift Performance Evolution: o(F4) Provide description of activity observed. Use N/A if not applicable Discussion of Results: Record Review: (2 spaces) Discuss the record review or write None, if not applicable. Interviews: (2 spaces) Discuss the interviews or None if not conducted. Shift Performance: (2 spaces) Discuss the shift performance observed or write None, if not applicable.

Conclusion:(2 spaces) Tie together discussion of key points resulting in your conclusion. State whether the criterion was met. Issue(s): o(F4) Write a brief, full sentence description of each issue. This should be an exact match of the title on the Form 2. Cite all Form 2s associated with this criteria. If no Form 2s were generated under this criteria, write NONE. DOE-HDBK-3012-96 APPENDIX 10-2 APPENDIX 10 SAMPLE ORR ASSESSMENT FORM 1 FUNCTIONAL OBJECTIVE EP.2, Rev. 2 CRITERIA MET AREA: EP DATE: June 4, 1996 PAGE NO. 2 YES NO X OBJECTIVE: A baseline compliance status review of DOE Orders 5500.1B, 5500.2B, 5500.3A has been performed. Noncompliance items have been addressed. Criteria All noncompliance issues are adequately addressed by DOE approved compliance schedule approvals (CSA) or exemptions. The CSAs include an adequate technical basis and schedule for attaining compliance. (DP-AP-202) Compensatory measures that are specified in the CSAs are adequately implemented. (DP-AP-202) Approach Record Review: Review order compliance packages for the listed orders, including all applicable CSAs and exemptions. Interviews: For orders that are not fully implemented, interview management personnel to ensure they are aware of this noncompliance and action necessary to fully implement the order requirements. Shift Performance: Where appropriate, observe the implementation of any specified compensatory measures within the facility to determine their effectiveness. Records Reviewed: • 5500.1B-STCS-93-016 • 5500.1B-CSA-94-123, Rev.1 • 5500.2B-CSA-93-072 • 5500.3A-CSA-93-005 • 5500.3A-CSA-93-006 • 5500.3A-CSA-93-024 • 5500.3A-CSA-93-001, Rev. 2 • 5500.7B-STCS-93-010 • 5500.7B-STCS-93-015 • 5500.3A-CSA-93-007A • 5500.3A-CSA-94-127 DOE-HDBK-3012-96 APPENDIX 10-3 Interviews Conducted: • Emergency Services Manager NMPD/Reactors • Manager of Emergency Services Technical • Administrative Assistant to Operations Management • Lead Emergency Preparedness Coordinator • DOE Area Representative EP • Emergency Management Preparedness Coordinator • Emergency Management Preparedness Coordinator • Shift Advisor on loan to Emergency Planning • Administrative Assistant to Operations Management Shift Performance Evolution: Facility DOE-HQ-ORE Drill Discussion of Results: Records Reviewed: The Emergency Preparedness Program for the Facility (the original used the name of the facility) was reviewed to ensure that the emergency plan and implementing procedures and instructions for the Facility, and applicable portions of the site plan, complied with or have Compliance Schedules Approvals (CSAs) for DOE 5500 Series Orders. The following DOE 5500 Series Order Compliance Packet CSAs were reviewed: XXX-DOE-5500.3A-CSA-93-007A; Personnel accountability was demonstrated for the facility personnel during the Emergency Preparedness Drill X-F Facility DOE-HQ-ORE Evaluated Drill [ . . . "must be preformed in 30 minutes not to exceed 45 minutes" . . . 5500.3A.11.c.(6)(c)]. The Facility has implemented personnel accountability in procedure "Personnel Accountability" [EPIP-XXXX-203 Rev. 2] using a newly established manual badge system.

Another issue of this CSA, is personnel accountability for Outside Facilities, XXX-F, XXX-F, and the area which uses a supervisor accountability system . The supervisory accountability system is not consistently proceduralized for the facility. In addition, neither the manual badge system nor the supervisor accountability system have been found reliable for personnel working on the roof. A facility compensatory measure is visitors and personnel are required to respond to direction from the AEC. Also, personnel working on the roof are required to notify the Control Room when entering and exiting. A facility procedure addressing the above points will ensure a consistent approach to Area accountability. Offsite Response Interfaces: Offsite Response Interface is the responsibility of the Site Emergency Preparedness Organization, it is not an issue for Emergency Preparedness Organization. XXX-DOE- 5500.2B-CSA-93-072 for Consolidated Emergency Response Facility is a site not and a facility issue. DOE-HDBK-3012-96 APPENDIX 10-4 Notification: XXX Area and the Site ERO has adequate communication networks. Medical Support: The area has adequate medical support. Recovery and Reentry: Adequate procedures exist for Reentry [Emergency Exposure Guidelines, EPIP- XXXX-202 Rev. 2; Personnel Accountability, EPIP-XXXX-203 Rev. 2; Emergency Response Team Dispatch, EPIP-XXXX-201, Rev. 2] and Recovery [Recovery, EPIP-XXXX-206 Rev.2] Operations at the facility. Public Information: Public Information is the responsibility of the Site Emergency Preparedness Organization, it is not an issue for this facility's ERO. Training: XXX is not in full compliance with systematic and performance based training requirement [5500.3A.11.c(11)(b)(1)]. CSA, XXX-DOE-5500.3A-CSA-94-127, schedules compliance with this requirement 9/30/95. Interviews: All interview comments are included in the Records Review section. Shift Performance: XXX Facility DOE-HQ-ORE Drills (see TR.3 for comments). Conclusion: All XXXX non-compliance issues have Compliance Schedule Approval agreements. Facility CSAs for hazards assessment, Emergency Action Levels (EALs), and EPZ have expired and are addressed in a finding. The compensatory measures for the CSAs provide an adequate level of protection for the health and safety of workers and the public due to their conservative nature. The criteria for this objective have not been met. Issue(s): • Compliance Schedules Approvals (XXX-DOE-5500.3A-CAS-93-005, 006,024) committed to complete a facility Interim Hazards Assessment leading to a compliant (EAL) procedure has not been completed on schedule (EP.2-1). (Pre-start) Inspector: __________________ Approved:_____________________ Team Member Team Leader DOE-HDBK-3012-96 APPENDIX 10-5 INSTRUCTIONS FOR COMPLETING ORR DEFICIENCY FORM 2 Objective: Criterion: Finding______ Prestart____ Issue No.: Observ.______ Post-Start____ Date: 6/ 4/96 ISSUE: (2 spaces) Write a brief, full sentence, description of the issue. Reference the corresponding Form 1. Use the present tense. REQUIREMENT: (2 spaces) Write the requirement, from a rule, DOE order, mandatory standard, or site requirements document. Reference locations are provided in the CRAD. REFERENCE(S): (2 spaces) Be specific. Include section and paragraph, if possible, of all references that support the conclusion that existing conditions are deficient. This must match the requirement(s) stated above.

DISCUSSION: (2 spaces) Describe the issue. The discussion should be self-explanatory and support the conclusion. Use the present tense. Cite specific example(s) for a generic issue. CONCLUSION: (2 spaces) State whether it is a pre-start or a post-start finding. The ORR Implementation Plan should provide evaluation guidelines. Inspector: Approved: Team Leader DOE-HDBK-3012-96 APPENDIX 10-6 ORR DEFICIENCY FORM 2 SAMPLE Functional Objective Finding X Pre-Start X Issue No.: EP.2-1 Area: EP No.: 2 Observe. Post-Start Date: 8/24/94 ISSUE: XXX Area Compliance Schedules Approvals (XXX-DOE-5500.3A-CAS-93-005, 006,024) committed to complete a facility Interim Hazards Assessment leading to a compliant Emergency Action Level (EAL) procedure has not been completed on schedule (August 22, 1994). REQUIREMENT: EALs are require by DOE Order 5500.3A, 11.be)[2] and 5500.3A,11.c(3)[2] ". . ., the specific criteria used to recognize and categorize events, must be developed for the spectrum of potential Operational Emergencies identified by the hazards assessment." REFERENCE(S): XXX-DOE-5500.3A-CAS-93-005; XXX-DOE-5500.3A-CAS-93-006; XXX-DOE-5500.3A-CAS-93-024; XXX-940093 memorandum, X-Area USQ Resolution Matrix. DISCUSSION: Compliance Schedule Approval (CSA) agreements were approved for required upgrades to the Emergency Preparedness Program. Due to a variety of circumstances, the EAL procedure was not developed in conformance with the agreed to schedule. The CSA compensatory actions remain in place while a new CSA is evaluated. Reportedly, XXXX (CONTRACTOR) has proposed a new schedule for compliance with an end date of 10/31/94. CONCLUSION: This order compliance issue should be evaluated and a new schedule accepted by DOE. Pending resolution of this matter, the agreed upon compensatory measures should continue. Resolution of this matter prior to restart is considered appropriate due to the significance of the topic and the need for Headquarters agreement on the delay in achieving compliance. Inspector:____________________ Approved:_____________________ Team Member Team Leader DOE-HDBK-3012-96 APPENDIX 11-1 APPENDIX 11 PRESTART AND POST-START DETERMINATION CRITERIA FOR OPERATIONAL READINESS REVIEWS The following checklist may be used by the TL and the ORR TMs to evaluate if an issue must be corrected prior to startup. A. Initial Screening 1. Does this issue involve a safety system? 2. Does this issue involve processes, functions or components identified in the Technical Safety Requirements, Operational Safety Requirements or nuclear safety control procedures? 3. Does this issue involve potential adverse environmental impact exceeding regulatory or site specific release limits? 4. Does this issue impact non-safety processes, functions or components which could adversely impact safety related processes, functions or components? 5. Is this issue non-compliant with WSRC or DOE-SR approved startup documents? 6. Does this issue indicate a lack of adequate procedures or administrative systems? 7. Does this issue indicate operational or administrative non-compliance with procedures or policy? 8. Has this issue occurred with a frequency that indicates past corrective actions have been lacking or ineffective? 9. Does this issue require operator training not specified in existing facility training requirements? 10. Does the issue involve a previously unknown risk to worker or public safety and health or a previously unknown threat of environmental insult or release?

If the response to any of the above is yes, further evaluation in accordance with the issue impact criteria below is required. B. Issue Impact 1. Does the loss of operability of the item prevent safe shutdown, or cause the loss of essential monitoring? DOE-HDBK-3012-96 APPENDIX 11-2 2. Does the loss of operability of the item require operator action in less than ten (10) minutes to prevent or mitigate the consequences of events described in the Safety Analysis? 3. Does the loss of operability of the item cause operation outside the TSR/OSRs or Safety Analysis? 4. Does the loss of operability of the item result in a reduction of the margin of safety as described in the Safety Analysis? 5. Does the issue indicate a lack of control which can have a near term impact on the operability or functionality of safety related systems? 6. Does the issue involve a violation or potential violation of worker safety or environmental protection regulatory requirements which poses a significant danger to workers, the public, or of environmental insult or release? If the response to any of the above questions is yes, the item should be considered a startup item. DOE-HDBK-3012-96 CONCLUDING MATERIAL Review Activity: Preparing Activity: DOE-DP-45 DOE Field Offices DP-21/23/25/31/33 AL EH-11/30/63 CH Project Number: EM-23 ID MISC-OO32 NE-44 NV ER-8.1 OR SR RF National Laboratories INEL Area Offices Amarillo Area Office Kirtland Area Office FOREWORD