DOE-HDBK-3012-2003, Guide to Good Practices for Operational Readiness Reviews (ORR)

DO DOE HANDBOOK GUIDE TO GOOD PRACTICES F OPERATIONAL READINESS RE (ORR) TEAM LEADER'S GUIDE U.S. Department of Energy ARE Washington, D.C. 20585 DISTRIBUTION STATEMENT A. Approved for public release; distribution NOT MEASUREMENT SENSITIVE E-HDBK-3012-2003 SUPERSEDING DOE-HDBK-3012-96 OR VIEWS A MISC is unlimited. DOE-HDBK-3012-2003 FOREWORD This Department of Energy handbook, Guide to Good Practices for Conducting Operational Readiness Reviews (ORR) Team Leader's Handbook, is approved for use by all DOE Components and their contractors. The guidance in this document is the distillation of the experience and lessons learned during numerous operational readiness reviews contributed by team leaders with a wide range of backgrounds. There are many ways to perform a readiness review correctly. Each review will be conducted at a unique facility and will involve a unique team. Every team leader will learn something new during each review. In some cases, these new lessons will conflict with the advice provided in this document. Such lessons learned and all other beneficial comments (recommendations, additions, deletions) and any pertinent data that may improve this document should be submitted to the EH ORR web site: http://tis.eh.doe.gov/ORR. i http://tis.eh.doe.gov/ORR DOE-HDBK-3012-2003 CONTENTS ACRONYMNS ............................................................................................................................................ v 1.0 PURPOSE AND SCOPE..................................................................................................................... 1 2.0 APPLICABILITY................................................................................................................................ 1 3.0 PLANNING AN ORR.......................................................................................................................... 1 3.1 ESTABLISH A TIME LINE ........................................................................................................... 1 3.2 PRIOR TO THE PRE-VISIT .......................................................................................................... 2 3.2.1 INITIAL RESPONSIBILITIES................................................................................................ 2 3.2.1.1 Team Leader Responsibilities....................................................................................... 2 3.2.1.2 Review Coordinator Scheduling/Coordination Responsibilities ............................... 4 3.2.2 DATA PACKETS, TRAVEL, AND OTHER LOGISTICAL RESPONSIBILITIES.......... 4 3.2.2.1 Team Leader Responsibilities....................................................................................... 4 3.2.2.2 Review Coordinator Responsibilities ........................................................................... 4 3.2.2.3 Team Member Responsibilities .................................................................................... 5 3.3 ORR PRE-VISIT .............................................................................................................................. 6 3.3.1 RESPONSIBILITIES ................................................................................................................ 6 3.3.1.1 Team Leader Pre-visit Responsibilities ....................................................................... 6 3.3.1.2 Team Members' Responsibilities.................................................................................. 6

3.3.2 INITIAL TEAM MEETING DURING THE PRE-VISIT ..................................................... 7 3.4 FINAL PREPARATIONS FOR THE ORR................................................................................... 8 3.4.1 RESPONSIBILITIES ................................................................................................................ 8 3.4.1.1 Team Leader Responsibilities....................................................................................... 8 3.4.1.2 Review Coordinator Responsibilities ........................................................................... 9 4.0 THE ORR ........................................................................................................................................... 10 4.1 ON-SITE ORR RESPONSIBILITIES.......................................................................................... 10 4.1.1. TEAM LEADER RESPONSIBILITIES ................................................................................. 10 4.1.2 TEAM MEMBER RESPONSIBILITIES ................................................................................ 11 4.1.3 REVIEW COORDINATOR RESPONSIBILITIES ............................................................... 12 4.2 MANAGING THE OUTCOME OF THE ORR OR READINESS ASSESSMENT................ 12 ii DOE-HDBK-3012-2003 4.3 FINAL TEAM MEETING............................................................................................................. 13 4.3.1 RESPONSIBILITIES .............................................................................................................. 14 4.3.1.1 Team Leader Responsibilities..................................................................................... 14 4.3.1.2 Team Member Responsibilities .................................................................................. 14 4.4 CLOSEOUT MEETING................................................................................................................ 14 4.4.1 RESPONSIBILITIES .............................................................................................................. 14 4.4.1.1 Team Leader Responsibilities..................................................................................... 14 4.4.1.2 Review Coordinator Responsibilities ......................................................................... 15 4.4.1.3 Team Member Responsibilities .................................................................................. 15 5.0 POST ORR ......................................................................................................................................... 15 5.1 THE FINAL REPORT................................................................................................................... 15 5.1.1 RESPONSIBILITIES .............................................................................................................. 15 5.1.1.1 Team Leader Responsibilities..................................................................................... 15 5.1.1.2 Team Member Responsibilities .................................................................................. 15 5.1.1.3 Review Coordinator Responsibilities ......................................................................... 16 6.0 LESSONS LEARNED FROM PREVIOUS ORR/RAS ................................................................. 16

6.1 ACHIEVING READINESS........................................................................................................... 16 6.2 ITEMS TO NOTE BEFORE PROCEEDING WITH AN ORR ................................................ 17 6.3 ADMINISTRATION OF THE RESULTS................................................................................... 18 6.4 CONDUCT OF DRILLS AND OTHER OPERATIONS ........................................................... 18 6.5 LOGISTICS AND SUPPORT ....................................................................................................... 19 6.6 TEAM ISSUES ............................................................................................................................... 19 6.7 DEVELOPING THE IMPLEMENTATION PLAN ................................................................... 20 6.8 DEFINING THE SCOPE AND GRADED APPROACH ........................................................... 20 6.9 POST-ORR CORRECTIVE ACTIONS ...................................................................................... 21 6.10 MISCELLANEOUS LESSONS LEARNED.............................................................................. 21 6.11 GENERIC LESSONS LEARNED THATAPPLY TO THE DESIGN, CONSTRUCTION, OPERATION, AND DECOMMISSIONING OF DOE FACILITIES .................................... 22 iii DOE-HDBK-3012-2003 APPENDIX 1: KEY POINTS TO DISCUSS AT INITIAL CONTACT WITH SITE...................... 25 APPENDIX 2: SITE CONTACT MEMORANDUM ........................................................................... 26 APPENDIX 3: ORR LOGISTICS.......................................................................................................... 27 APPENDIX 4: ORR PRELIMINARY SCHEDULE............................................................................ 30 APPENDIX 5: PRE-VISIT SCHEDULE .............................................................................................. 30 APPENDIX 6: CHECKLIST FOR ORR TEAM INFORMATION PACKAGES............................ 31 APPENDIX 7: DETAILED SCHEDULE FOR ORR TEAM.............................................................. 32 APPENDIX 8: TRANSMITTAL LETTER FOR DATA PACKAGES TO TEAM MEMBERS..... 33 APPENDIX 9: TEAM MEMBER DELIVERABLES.......................................................................... 34 APPENDIX 10: INSTRUCTIONS FOR COMPLETING ORR FORMS.......................................... 35 APPENDIX 11: PRE-START AND POST START DETERMINATION CRITERIA FOR ORRs 44 iv DOE-HDBK-3012-2003 ACRONYMNS CD Computer disk CRAD Criteria and Review Approach Documents DNFSB Defense Nuclear Facilities Safety Board DOE Department of Energy DOE GTN Department of Energy Germantown EH Office of Environment, Safety and Health ORR Operational Readiness Review POA Plan-of-Action PDs Process Descriptions PSDs Process System Diagrams RA Readiness Assessment S/RID Standards Requirements Information Documents SSC Safety structures, systems and components TSRs Technical Safety Requirements USQ Unreviewed Safety Question USQD Unreviewed Safety Question Determination v DOE-HDBK-3012-2003 1.0 PURPOSE AND SCOPE This guidance provides instructions, explanations and examples for the performance of all phases of an Operational Readiness Review (ORR). Details pertinent to the Team Leader, Team Members, and Review Coordinator are outlined. Appendices contain sample forms and correspondence, which are typically used to initiate and perform the ORR.

2.0 APPLICABILITY The document is written to be useful to both Contractor and DOE ORR Teams and Team Leaders. The handbook is written to include considerations that result when the Team Leader or members travel to a remote location. The specific actions that are the result of traveling to the site will not be required for those individuals who are from the site. The handbook is also useful for Team Leaders of Readiness Assessments (RAs) conducted in accordance with requirements of DOE O 425.1C. Lessons learned, which are promulgated with this handbook, will benefit any line manager, particularly those preparing a facility or process for startup or restart. There are many challenges to leading an ORR or RA. The planning, selection of the review team, and development of the Implementation Plan are the most difficult. 3.0 PLANNING AN ORR 3.1 ESTABLISH A TIME LINE The following guidance is idealized, but neglecting any component of the preparation will complicate the conduct of the remaining phases, and frustrate the review team in its efforts to be efficient and effective. An Operational Readiness Review begins long before the conduct of the review at the identified site, facility, or process. Once selected, the Team Leader has the responsibility for all aspects of the review. The following list gives the Team Leader an idea of the time line he/she will need to be aware of and ultimately manage to ensure a successful Operational Readiness Review. 1. The Startup Notification Report identifies the startup facility or activity and is initiated one year prior to the ORR. It is the first step in the review planning process and establishes the DOE/Contractor agreement regarding the startup review approach, including the Approval Authority. 2. The Plan-of-Action (POA) is developed by the Contractor and DOE (a separate POA is prepared for the DOE and the Contractor ORR) approximately six months before the conduct of the ORR and provides the Authorization Authority=s (line management) expectations regarding the breadth and scope of the ORR. The Team Leader is identified and prerequisites for startup readiness are defined. 3. A Contractor Line Management Self-Assessment should be conducted prior to the Contractor ORR giving the facility or activity=s line manager the confidence of readiness to operate safely. It should be conducted by the facility/activity’s management, should review Core Requirements and prerequisites, should validate completed processes (reducing rework), and conclude with an integrated wrap-up. The Management Self-Assessment should be completed and issues resolved prior to starting the Contractor ORR. 1 DOE-HDBK-3012-2003 4. DOE validation provides DOE endorsement of the Readiness to Proceed Memorandum. DOE validation confirms the results of the Management Self-Assessment while not being another readiness review itself. It is performed prior to the conduct of the DOE ORR. 5. An ORR Implementation Plan is developed approximately three months before the ORR. It establishes the team membership, defines the Review strategy (functional areas per the POA), the Criteria and Review Approach Document (CRAD), and outlines the mechanics of the review. Each ORR requires a separate Implementation Plan. 6. A Contractor ORR is conducted eight weeks before the ORR to confirm the facility/activity is indeed ready for operation. It should occur only when readiness is achieved. It should parallel the DOE ORR Review but in greater depth and culminating with a final report.

7. Resolution of findings (i.e., issues) identified during the Contractor ORR minimizes the probability of their recurrence. Finding resolutions should address root causes, confirm DOE oversight processes, and eliminate the possibility for similar failures. It is not a time to rush. All findings should be resolved and fixed to the maximum extent achievable. Finding closure should be completed three weeks prior to the ORR to permit orderly verification of the adequacy of the resolution actions. 8. The Readiness to Proceed Memorandum is issued just prior to the DOE ORR and establishes management=s commitment that readiness to safely commence program work has been achieved, confirms completion of prerequisites, and identifies any open items. 9. The DOE ORR is conducted to confirm the Contractor has achieved readiness to safely start program work and the DOE site office is prepared to oversee the operations. Generally the ORR requires two weeks of dedicated time on site: one week of field observations that includes sharing potential findings in real time; a second week for writing the Assessment Forms, Final Report, and Exit Briefing. 10. Finding closure of the DOE ORR is the responsibility of the Contractor. Pre-start findings must be accepted and verified closed by the DOE line management before operations can start. Post start findings must have a corrective action plan (CAP) approved before startup or restart is authorized. DOE Line Management must accept the corrective action plans for post start findings 11. Finally, the Startup Authority grants permission to commence program operation. The facility/activity executes an approved startup plan to transition to normal operations. Briefings are provided to internal and external oversight organization as requested. The final report is archived, lessons learned forwarded, and information posted to the ORR web site. 3.2 PRIOR TO THE PRE-VISIT 3.2.1 INITIAL RESPONSIBILITIES 3.2.1.1 Team Leader Responsibilities Initial Duties: As soon as the Team Leader is identified in the POA, the Team Leader should perform the following: • Comply with commitments made in the approved Plan of Action. • Identify points of contact and support at the facility with whom the Team Leader and Review Coordinator will interface for continuing support and information throughout the process. • Identify the Review Coordinator. • Contact the identified points of contact to discuss details about the upcoming ORR and negotiate prospective dates for the pre-visit and ORR. • Prepare a draft Implementation Plan including designation of functional areas and development of draft CRADs. • Send a letter or e-mail, as soon as possible after the above negotiations, formalizing the notification of the ORR to the site points of contact (see Appendices 2, 3, 4, and 5). 2 DOE-HDBK-3012-2003 • Advise the Review Coordinator of dates and agenda for the pre-visit. • Initiate the process of selecting team members to support the ORR as soon as possible. Team Selection: The importance of this task cannot be overemphasized. No other task has such a direct impact on the overall quality of the ORR. The Team Leader selects and organizes the pre-visit and ORR teams. Additional guidance on team selection can be found in sections 5.4.2 and 5.4.3 of the ORR Standard, DOE-STD-3006-2000. Guidelines for completing the process follow: • Candidates may be obtained from any independent sources within DOE or from contractors not

directly responsible for the oversight or operation of the facility under review. Previous Team Leaders are valuable resources in reviewing the past performance of team members. The specific scope of the ORR will be specified in the POA. The areas to be reviewed normally include: Operations, Maintenance, Training, Documented Safety Analysis or other safety basis implementation, Engineering Support, Environmental Safety and Health, Emergency Preparedness, Radiation Protection, Occupational Health and Industrial Safety, Management, Fire Protection, and Quality Assurance. Additional areas for consideration could include Safeguards and Security, Procedures, Testing, Procurement, Packaging and Transportation, Design, Construction, Management Systems, and Project Management. Other areas reviewed can include, if appropriate, configuration management, criticality safety, and DOE management (in the DOE ORR). • Team members must have technical experience in their area of review. They must also have assessment experience and be familiar with the facility/process in question. Typically, familiarization with the facility/process is gained during the pre-visit. Experience has shown that under normal circumstances, no team member should be assigned more than two CRADs. This may require additional team members, which is warranted since it ensures a thorough review is conducted. More team members ensure that team members with defensible technical expertise evaluate all technical areas of the review. • Determine whether any conflict of interest, actual or perceived, exists for any candidate or source of candidates. If so, those candidates must be rejected from further consideration. A real conflict of interest would exist for a contractor who has, previous to the ORR, provided direct support to a facility in an area that will be reviewed during the ORR. For example, DOE has used mentors at some facilities to assist in improving the conduct of operations at the facility. It would be inappropriate for this contractor to be a part of the ORR team. A perceived conflict of interest could exist when a contractor has provided indirect support to the facility or was an employee of the facility at some other point in their career. An example of indirect support could be participation on a review team evaluating facility safety basis documentation. The second case is self-explanatory. These cases are more subjective, but the Team Leader should be sensitive to them and avoid them if possible. • No individual directly or recently involved in the management or operation of a facility can participate as a team member in his/her area of responsibility at the facility. • The development and management of the Assessment Forms and summaries is a prodigious task. One person (the ORR Review Coordinator) should be dedicated to administering the Forms, e.g., logging in and tracking. This task is very time consuming once Forms begin being turned in and increases as the ORR progresses. In addition, at least one other person, and a word processor, should be dedicated to editing the Forms. Consideration should be given to bringing in supplemental editing staff toward the close of the ORR. A significant amount of copying is required at the close of the ORR for the Exit Briefing and any copies of the draft Final Report to be left at the site.

• Concurrent with the above tasks, work with the site point-of-contact to identify counterparts. A counterpart is an individual designated by the site to assist in coordinating the ORR team's activities. Each ORR team member should have a counterpart for each functional area under evaluation. 3 DOE-HDBK-3012-2003 • As soon as team members and counterparts have been selected, disseminate a list showing counterparts' telephone, facsimile numbers, and e-mail address, if available, to team members. The list is important for informing team members of their selection and their counterpart, which allows for early contact and preparation. • An adequate number of properly trained and experienced counterparts is critical to a successful ORR. The counterpart must have the technical knowledge necessary to support the team member. The counterpart must also understand that he/she must fully support the team member to ensure the team member gets the information he/she requires and the answers he/she requests. In addition, the counterpart ensures that management is aware of emerging issues and supports providing information to clarify or resolve the issue before it becomes a problem. Counterparts must be available during the ORR and prior to it as necessary to support the team during preparations for the ORR. 3.2.1.2 Review Coordinator Scheduling/Coordination Responsibilities After team selection is made, obtain social security number, date of birth, address, phone number, fax number, type of clearance, and funding information for all team members. • Assist the Team Leader with developing a team member/counterpart chart. • Assist the Team Leader in the preparation of the draft implementation plan. • Prepare an ORR notification letter and enclosures for the Team Leader (Appendix 2). • Make a contact list for the pre-visit ORR team. The list should include site contact names, phone numbers, fax numbers, hotel information, site security office number, per diem rates (some of this information will be provided to the team by the Team Leader). • Send draft ORR Implementation Plan and CRADs to ORR pre-visit team members for their review. • Reserve a block of rooms for ORR pre-visit team members who will travel from outside the area. Send the hotel a team list, if required. 3.2.2 DATA PACKETS, TRAVEL, AND OTHER LOGISTICAL RESPONSIBILITIES Note: Although travel and hotel arrangement details may not apply to team members from the site location, the other items do apply and must be completed for all team members. 3.2.2.1 Team Leader Responsibilities Data packets are used to provide team members with information and documents that enable them to better prepare for the pre-visit and ORR. Working with the Review Coordinator, assemble and distribute data packets to team members. The normal makeup of the data packet is provided in Appendix 6. Mail data packets early to ensure team members have their packets at least two weeks prior to the pre-visit. Some of the items in Appendix 6 may not be available prior to the pre-visit. The key is to provide maximum information to each team member to gain maximum benefit from the pre-visit. Other Logistics: Verify compliance with the request for logistic support that was sent (Appendix 3) with the letter of confirmation. 3.2.2.2 Review Coordinator Responsibilities • Inform team members to call the hotel and confirm their reservation and to guarantee it with a credit

card. Usually these instructions are included in the data packet. • Make security clearance and restricted area access arrangements for the ORR pre-visit. Complete DOE F 5631.20 (formerly 277's), Request for Visit or Access Approval forms for the pre-visit team. Discuss access requirements with the site point-of-contact before the pre-visit to determine which sigmas are needed. Discuss security procedures for any uncleared team members. 4 DOE-HDBK-3012-2003 • Make sure the team has maps of the site and directions how to get to the site and hotel from the airport. Request maps from the site. • Make contact with site or facility coordinator to introduce and discuss logistical questions. Discuss sigma/security requirements for cleared and uncleared team members and determine if there are any special procedures to bring in equipment, such as laptop computers, or supplies (e.g., such as computer disks). • Verify that all team members' clearance information and all necessary forms for access to the site and facility have been forwarded to the site. Verify this has occurred at least one week prior to the pre- visit. Some team members may use their laptop computers. Determine site access policies and requirements associated with laptop computers. Inform team members of what forms and information are required prior to the site pre-visit. Gather completed forms in advance of the ORR to ensure access during the review. • Complete the necessary forms for access to the site for all team members and forward the data to the site. • Verify that all required reference material has been provided to team members four to five days before the team's arrival for the pre-visit and ORR. 3.2.2.3 Team Member Responsibilities Each team member should receive and review their data packet. The data packet provides team members with information on the site, their assignments, including deliverables, and responsibilities of the team member. Team members are to contact the hotel to confirm their reservation and guarantee it with a credit card. Each team member is responsible for providing clearance information to the Review Coordinator. Provide the following clearance information: 1. Name 2. Social security number 3. Date of birth 4. Organization/company name 5. Type of clearance held. If special training or fittings are required, such as respirator fitting, for access to certain facilities, arrange to have any pre-conditions completed before arrival for the pre-visit1. For example, respirator fitting requires a medical exam before the fit test can be administered. Other special training could include Radworker II and Hazardous Waste Operations training. The Team Leader will inform all team members of any special requirements. Team members are to contact their counterpart during the preparation phase to obtain information they feel is necessary or to arrange for additional documentation to be readily available onsite. However, keep these requests to a minimum. The site appointed counterpart is a crucial element to a successful ORR. They should be in constant communication with the team member during the review and will provide the best source of site-specific information and access available to the sub-teams. 1 The Health unit in the A-wing basement of DOE GTN can provide medical exams for fitting respirators to Federal employees only. Contractors will need to arrange for this on their own. 5 DOE-HDBK-3012-2003 3.3 ORR PRE-VISIT

3.3.1 RESPONSIBILITIES The pre-visit is a fact finding and familiarization visit attended by as many team members as possible. If some team members are not able to attend the pre-visit, arrangements for compensatory measures must be made to ensure each assigned individual is prepared to be an effective ORR team member. A comprehensive pre-visit lays the foundation for a thorough and efficient ORR and normally satisfies the requirement for team members to be familiar with the facility/process. 3.3.1.1 Team Leader Pre-visit Responsibilities • Brief site management on the mission, the methodology, and the scope of the ORR, as well as the logistical support required from the site. • Arrange for the Operations Office/Area Office and/or contractor management to brief the team on safety, security, the organization, processes and operations of the facility. Specifics of a startup (e.g., initial shut-down, restart following upgrade, restart following a safety shut-down) should be included in the briefing. • Complete final review and editing of the implementation plan with special emphasis on the CRADs. Have team members provide their proposed CRAD changes prior to the end of the pre-visit. • Attend a walkthrough at the facility/process and observe any current operations to acquaint team members with the facility and processes in question. • Attend any special training or fitting, such as respirator or radiation worker training and fitting, which might be necessary to allow for access to the facility during the ORR. Verify that all team members have met any pre-conditions prior to their arrival at the site. • Develop a preliminary listing of Shift Evolutions (to include operations, drills/upsets, maintenance, surveillance and other activities to be observed during the ORR) and Interview list, and provide it to the site so they can plan and schedule events and personnel availability. Each team member should provide input prior to completion of the pre-visit. Effort should be made to coordinate the interviews among team members to minimize the impact on site personnel. • Schedule a counterpart meeting. Each team member should provide input prior to completion of the pre-visit. Effort should be made to coordinate the interviews among team members to minimize the impact on site personnel. • Schedule a counterpart meeting. Each team member should have acquainted him/herself with their counterpart and developed their inspection requirements during the preparation phase. • Arrange for any documents designated as required reading for team members to be available to team members. Scheduling periods of free time during the pre-visit to allow this to be accomplished is valuable, particularly if any of the required reading is classified. • Investigate other or additional hotel accommodations for the ORR if necessary. 3.3.1.2 Team Members' Responsibilities Review the ORR Implementation Plan including the CRADs. Ideally, team members should have received an advance copy of the Implementation Plan to enable a thorough preview of their assigned objectives and criteria. Team members should arrive at the site as scheduled, having met any prerequisites for site access provided by the Team Leader. 6 DOE-HDBK-3012-2003 During the pre-visit, team members will: • Receive briefings by the operations office/site office and/or contractor management on safety, security, the organization, processes, and operations of the facility as well as, specific information concerning the nature of the restart or startup.

• Walkthrough the facility or process area to observe any current operations to acquaint team members with the facility and processes in question. • Complete any special training or fitting, such as respirator training and fitting, which might be necessary to allow for access to the facility during the ORR. • Develop their Shift Evolution and Interview list(s) to be provided to the site so they can plan and schedule events and personnel availability. • Meet with their counterpart and develop a methodology for completing their assigned criteria. • Expect and plan to be in regular communication with the site-appointed counterparts as they will provide the best source of site-specific information and access available to the team member. • Complete any required reading of documents that cannot be accomplished readily elsewhere, e.g., classified documents. • Provide proposed CRAD changes to the Team Leader before the end of the pre-visit for inclusion in the Implementation Plan. During the pre-visit, draft CRADs should be shared and discussed with counterparts to ensure accuracy and completeness. 3.3.2 INITIAL TEAM MEETING DURING THE PRE-VISIT At the first team meeting of the Pre-visit, the Team Leader should: • Emphasize that the ORR is an arduous and intense process for which team members are expected to provide their complete and undivided attention. Discuss time commitment for preparation and during the ORR. The ORR will be a full time, dedicated assignment. Work hours will be long to accommodate the schedule, including possible evolutions and interviews on the off shifts. Saturday work may be required to complete the forms on schedule. Local team members will have no time for their regularly assigned work. • Define the ORR team's relationship with the site. The ORR is conducted to verify the site's readiness to safely conduct operations. Clarify/restate that the ORR will not start until the contractor facility management has declared readiness and the appropriate authority has directed the Team Leader to start the ORR. If the scheduled dates for the ORR must change, the Team Leader will keep the team members informed of schedule changes. Team members will also need to inform the Team Leader of their availability when the schedule changes.  • Establish that the Team Leader is responsible for all aspects of the inspection. • Define the expected schedule for the ORR, including the time frame for submission of all deliverables (see Appendix 9). • Explain the need for and use of the various forms: - Assessment Forms (Form 1s) are used to document the methods and actions taken in the criteria evaluation process. Additional guidance is provided in Appendix 4 of the ORR Standard and Appendix 10 of this document. - Deficiency Forms (Form 2s) are used to document the findings identified during the criteria evaluation process. Findings are to be documented (i.e., a Form 2 drafted) as soon as there is reasonable evidence to substantiate a finding. This allows the Team Leader to present site management a daily briefing of emerging issues and allows the site to better facilitate the finding validation process. Additional guidance is provided in Appendix 4 of the ORR Standard and Appendix 10 of this document. 7 DOE-HDBK-3012-2003 • Provide administrative information such as area phone numbers, dining locations, etc., at this meeting.

• Advise team members to keep the Team Leader abreast of all concerns and new developments as they arise to minimize confusion and miscommunication as information is collected. • Advise team members that the Team Leader must clear participation in any site activity by the team members (e.g., special tours) not directly related to the review. All discussions with site management outside the assigned CRADs should also be cleared through the Team Leader. • Insist that all complaints, concerns, suggestions, and information be passed through the Team Leader as soon as possible. Identify problems with counterparts to the Team Leader. • Advise team members not to meet with external regulatory or oversight agencies, such as state environmental protection agencies, without the permission of the Team Leader. Exceptions must be approved in advance with the Team Leader who must notify the appropriate management personnel. • Advise team members to strictly adhere to the need-to-know criteria in dealing with classified shapes, components, data, or processes. Team members should immediately identify any difference of opinion that arises with the site on access to information. • Explain that no contractor business development will take place onsite. Any such activity will result in removal from the ORR. This does not mean that a contractor must refrain from giving out a business card, if asked. It does mean they are not permitted to solicit business from site personnel. • Explain the format to be used for completing the final report and how the team members' input will be integrated into the report. 3.4 FINAL PREPARATIONS FOR THE ORR 3.4.1 RESPONSIBILITIES 3.4.1.1 Team Leader Responsibilities Finalize the ORR Implementation Plan. The Implementation Plan should be finalized immediately following or even during the pre-visit, if possible. Further information concerning development of the Implementation Plan is contained in section 5.9.2 and Appendices 1 through 4 of the ORR Standard. • Prior to approval by the Team Leader, the Implementation Plan should be forwarded via the local DOE Site Office to the Office of Environment, Safety and Health (EH) for review and comment, as discussed in section 5.3.2 of the ORR Standard. It is important to remember that although EH approval of the Implementation Plan is not required, it is appropriate to allow adequate time for their review and comments. The Team Leader should make a good faith effort to resolve all comments. It may also be appropriate to provide the plan to DOE and contractor line management for review to confirm the technical accuracy of the information in the plan. • Establish the length of the ORR. In developing the ORR schedule, several factors need to be considered, among them the complexity of the facility/process being reviewed, the number of team members available, the number of interviews and evolutions to be conducted, and the number of shift evolutions to be observed. Experience has shown that even limited scope ORRs need a minimum of a week for fieldwork. Several days are then needed to analyze the data, conduct follow-up, and develop coherent and informative conclusions, as well as to complete the extensive administrative requirements prior to leaving the site. Two weeks as a general guideline has been a challenging but achievable goal.

• Establish the ORR schedule. If possible, the first day(s) should address administrative reviews and management interviews. This will help assure review team familiarity with the processes and the people. Next, observation of evolutions should be scheduled. Save level of knowledge interviews for last to aid in evaluating whether observed miss steps were a matter of an individual failure or a 8 DOE-HDBK-3012-2003 general problem. Particular care is necessary to establish a satisfactory evolution schedule. Maximum realism with actual or surrogate material is important. The schedule of events must meet the needs of all team members, but must fit within the available time, which is nominally three days. Smart sampling of realistic evolutions carried to their logical conclusion is the goal. Inputs from all team members will be required to establish a satisfactory schedule. The draft schedule should be presented to the facility and discussed during or shortly after the pre-visit. The Defense Nuclear Facilities Safety Board (DNFSB) staff, EH, and line management representatives may observe all, or some, of the ORR. Instruct team members regarding protocol when being observed. The team member should neither exclude nor defer to the observers, who should not be active participants in the conduct of the ORR processes. Any concerns resulting from the observations should be discussed with the team leader. 3.4.1.2 Review Coordinator Responsibilities After ORR dates are confirmed, reserve a block of rooms and send a rooming list to the hotel with names and check-in/check-out dates, if required. Make the reservation under the name of the DOE or contractor ORR team. Make arrangements for a meeting room at the hotel for the initial team meeting, if necessary. Prepare security clearance forms for all team members. Complete form DOE F 5631.20 (formerly 277s) Request for Visit or Access Approval. These are not completed for uncleared team members. Provide the site point-of-contact with a list of names and social security numbers in order for them to track clearance status. Ensure the site security office has received a copy of this list either from site point- of-contact or by directly faxing them a copy. Assist in preparing data packets for team members (see Appendix 6). The Review Coordinator provides the following information: 1. The hotel meeting room number, if applicable; 2. Where and when to get badges; 3. Badge office hours; 4. Special arrangements for any uncleared team members or personal computers; 5. What to do in case a team member gets delayed and can't be at the right place at the right time; 6. If any special training is required; and 7. Where the onsite team room work space is located. Coordinate with the team leader and provide the following items as listed in Appendix 6: 1. Blank Forms with the reviewer's objectives and criteria filled in, and provide the same information on a diskette; 2. Maps; 3. Detailed schedule (Appendix 7); 4. Team member assignment and counterpart matrix; 5. Team member deliverables (Appendix 9), including completed example forms and required summaries; 6. Site Organization charts; 7. Management Self Assessment reports and Contractor ORR reports as applicable; 8. Plan of Action and Implementation Plan 9. DOE O 425.1C and DOE-STD-3006-2000. 9 DOE-HDBK-3012-2003

Make arrangements to mail data packets to team members. Packages should be delivered to team members at least two weeks before the beginning of the ORR. If possible, arrange for electronic or computer disk distribution of current site and facility documents. Keep the site point-of-contact informed of any team member/schedule changes by updating and faxing the team member assignment chart. Make sure latest date is on the chart. Keep all team members informed of the status of progress and potential or actual changes in ORR start dates. Prepare a package to mail to hotel and to have available at the initial team meeting. The package should include any information and/or documents the Team Leader and DOE-HQ team members need for the ORR, such as the standard format used and examples from previous reviews; latest agendas and schedules; computer disks with all ORR information; blank Forms 1 and 2; previous ORR closeout summary examples; and the template for the word processing software to be used. The package must be provided to each team member no later than the first team meeting at the beginning of the ORR. Encourage the site to provide electronic access to information required by the team members. As an alternative, many sites can provide documentation on a computer disk. Getting the information on computer disk has worked well in several recent reviews. 4.0 THE ORR The ORR will not start until the Team Leader has received the formal direction to start the ORR from the Authorization Authority or senior contractor manager. This formal direction means that all prerequisites for starting the ORR have been met and that readiness to commence operations has been achieved. 4.1 ON-SITE ORR RESPONSIBILITIES 4.1.1. TEAM LEADER RESPONSIBILITIES • Ensure all team members have completed all access requirements of the site. • Hold daily team meetings (usually held in the afternoon) to discuss emerging issues. Functional Area Leaders, if assigned, or team members brief the Team Leader on the issues uncovered during the day (this is not intended to be a travelogue of the day's events). Issues deemed sufficiently substantive by the Team Leader will be identified as potential findings and may be drafted onto Form 2s. Some duplication of effort is to be expected and it can often lead to beneficial cross-talk and fuller understanding of the issues. Issues raised may be assigned to a different functional area or sub-team if it is felt that better coverage will result. The Team Leader resolves any disputes between functional areas or sub-teams that occur. Additional guidance for the conduct of the ORR can be found in section 5.4.6 of the ORR Standard. • Invite the site to have observers (not a participant) at the team meeting to hear first hand the team issues and concerns. Remind team members that their counterparts should be kept aware of the emerging issues. • Meet frequently (daily is recommended) with site managers to brief them on the issues raised by the previous day's review activities. The Team Leader discusses potential findings and other issues uncovered during the previous day's activities. This greatly enhances site/team communication and allows the site to better facilitate the validation process. • Conduct the opening meeting at site. This meeting is the "kickoff" meeting for the commencement of the ORR. Items to consider for completion during this meeting include: 10 DOE-HDBK-3012-2003

1. Make introductions. 2. Allow for a series of briefs by the site explaining the site's management and organization, its functions, operations and interfaces. 3. Brief the audience on the ORR process and the ORR schedule. 4. Brief site audience and team on the rules for interaction between team members and site personnel. The team will not act as safety monitors or direct operations. The team will refrain from distracting the operators. The team will not violate site rules, cross boundaries without authority, violate control room/area boundaries or in any other way become an initiating event during the ORR. The team will not act as evolution or drill observers. The team will strive to be invisible so as to not influence the performance of the facility. Request that all operators be briefed on these rules and in particular that no action should be taken in response to questions or discussion by the team. The team will initiate action, preferable through a supervisor or escort, if an immediate unsafe condition is encountered. The following items need to be reiterated at afternoon team meetings throughout the inspection: • Encourage the identification of any unclear issues. They still need to be identified so that others may consider them at a later date. • Re-emphasize that team members should focus on that portion of the organization directly responsible for their area. Contact outside their area, specifically including interviews with senior site management, must be discussed in advance with the Team Leader. • Insist on validation of facts. ORR teams validate facts they collect on a continuous basis. Facility and DOE line management and the counterparts are the primary points of contact for validating facts. • It is also important to try and gauge the response of site management to the conclusions that are drawn from the validated facts. • Maintain a running record of the important issues brought out at each nightly meeting. The technical editor or Review Coordinator will review all Forms and summaries. Team members are encouraged to use the technical editor and Senior Advisor’s assistance. The Senior Advisor should also review and provide assistance with completion of the required forms and summaries. If appropriate, all Assessment Forms will be reviewed by a derivation classifier. 4.1.2 TEAM MEMBER RESPONSIBILITIES Team members should provide the Team Leader a draft Form 2 for each finding as soon as sufficient information has been gathered to define the finding. It is important to ensure factual accuracy. The Facility and DOE line management and the counterparts are the primary points of contact for validating facts. If facts have not been validated, do not use them. If you have validated facts, know with whom or by what reference you validated. Be sure that person has the authority to validate. It is important that the team exercise the responsibility for drawing conclusions and reaching judgments from facts. Each team member must keep accurate and complete notes of the activities. Notes must include titles of individuals interviewed as well as the notes of the interview results; title, document number, date, and revision number for all documents reviewed as well as the conclusions or information resulting from the document reviews; and details of all evolutions that are observed. This information will be required to complete the assessment and deficiency forms.

Assessment Forms (Form 1s) are used to document the methods and actions taken in the criteria evaluation process. Team members are encouraged to update their Form 1s daily to ensure complete record keeping of all inspection activities. The Team Leader may choose to require each team member provide daily draft Form 1s for his/her review. Additional guidance is provided in Appendix 4 of the ORR Standard, and Appendix 10 of this document. 11 DOE-HDBK-3012-2003 Deficiency Forms (Form 2s) are used to document the findings identified during the criteria evaluation process. When issues are unclear, they still need to be identified so that others may consider them at a later date. If you begin to have suspicions about something, report the concern and let the site follow it up. Avoid delaying the drafting of a Form 2 until there is overwhelming evidence as this may excessively delay the validation and correction processes. Additional guidance is provided in Appendix 4 of the ORR Standard and Appendix 10 of this document. The final ORR Report is required to contain sections concerning ISM System status, Lessons Learned, and adequacy of contract requirements in addition to the summary of results. The Team Leader may task individual team members with responsibility for collection of material from other team members and development of these sections. Team members may also be tasked with development of the draft presentation materials for the closeout meeting. 4.1.3 REVIEW COORDINATOR RESPONSIBILITIES During the ORR, the Review Coordinator works closely with the Team Leader providing administrative and logistical support. Specifically: • Hand out latest team member assignments and agendas/schedules and any guidance on writing Forms and summaries (see Appendix 10). • Provide administrative and logistical support to team members. • Get to know the site, i.e., locations of library, copy center, fax machine, supplies. • Establish access and gain familiarity with site electronic document processes including intranet access when appropriate. • Find out about site-specific security procedures, i.e., do team member notes need to be reviewed for classification? If so, where? • Check all computers and printers to make sure they are working properly. Arrange for repairs or replacement if necessary. • Load Form 1 and 2 files onto all computers or provide on disk if team members will be using lap top computers. • Provide sample Functional Area Summaries to each functional area leader. This helps ensure the inputs for the final report are consistent in content and format. • Attend team meetings and manage the Form 1 and Form 2 tracking system(s). A suggested procedure to manage the Forms is as follows: - Get a hard copy with diskette from each inspector or subgroup team leader. To maintain configuration control of the report, the Review Coordinator and technical editor make all electronic changes after the diskette has been provided to the Review Coordinator. - Log in the Form(s). - Give a copy to the Team Leader as soon as possible and place a copy in the logbook. - After the Team Leader reviews and comments on the Form and it has been edited by the technical editor, return the Form to the inspector for revision or signature. Revisions and editing are then accomplished between the inspector and the technical editor with the Team Leader reviewing only the final edited and signed version submitted by the inspector.

4.2 MANAGING THE OUTCOME OF THE ORR OR READINESS ASSESSMENT The task of the ORR team is to confirm that readiness to commence operations has been achieved. A critical task of the Team Leader is to assist and facilitate the team in making the correct determination. Every ORR team will identify findings, some of which will be judged as requiring resolution prior to start of operations. A critical question for the team is whether the number or the significance of the findings leads to a conclusion that the ORR Team should not recommend that operations be commenced when the 12 DOE-HDBK-3012-2003 findings are resolved. Experience indicates that this question may arise at any time during the ORR including during the final team meeting. The decision to recommend that operations NOT be commenced represents a significant failure on the part of the organization tasked to achieve readiness and must not be made lightly or without clear basis in fact. The ORR Team Leader has a significant challenge to ensure the facts clearly support a decision that readiness has not been achieved to safely commence operations. A decision of that magnitude should be thoughtful and not made quickly. No single team member should be permitted to inappropriately influence the remainder of the team. The facts that support the decision should be written and factually verified and persuasive to the team as a whole before a decision is final. The team should be required to consider the proposal over night to ensure that the facts still lead to the same conclusion that readiness has not been achieved and that resolution of the findings will not provide adequate confidence for the team to recommend start of operations. In the event that the ORR Team determines that readiness has not been achieved and that a recommendation to start operations when the findings are resolved cannot be given, the appropriate path forward must be defined. The ORR Team Leader must keep the Authorization Authority or contractor senior manager who directed the ORR to start informed of the conditions and appraised of the possible options. The final decision as to the path forward will reside with the Authorization Authority or Contractor Senior Manager. Options that should be considered if a recommendation to commence operations cannot be made are to either suspend the review or to complete the review with a determination that the review is not satisfactory. In both cases, it is understood that line management responsible for achieving readiness was not successful. In all cases, it is important that the team provide a written report of the activities completed, the issues identified, and the recommended path forward. In most cases, it should be possible for the report to reflect areas of the review that were completed and satisfactory with the resolution of identified findings. Those areas that are completed should not require additional review when the review is restarted. These might typically include support functions such as training or maintenance as example that might be adequate to support safe operations even though the operations themselves may not be ready. The team that reassembles to complete the review should be of appropriate size to review the areas that remain. In a few cases in the past, the state of readiness was so deficient that the entire review was repeated, including the formation of a new team and the development of a new Plan of Action and a new Implementation Plan. In cases such as that, the written report should identify the significantly deficient status of achieving readiness and recommend that the review be repeated when readiness has been achieved. More is required, under these circumstances, than simply resolving the findings. The entire process for achieving readiness must be repeated with emphasis on a more successful outcome.

4.3 FINAL TEAM MEETING This is the last opportunity to work directly with team members and clarify any questions that have persisted throughout the ORR. Final Forms 1 and 2 are due prior to this meeting, as is a draft summary evaluation of the assigned objectives for inclusion in the final report. This is the last opportunity to discover and address dissenting opinions and conflicts prior to presenting the ORR results to the site. Any such issues must be addressed at this time to avoid any confusion at the closeout meeting. Since no new information will be discussed at the final team meeting, it is appropriate that this meeting be restricted to team members only. 13 DOE-HDBK-3012-2003 4.3.1 RESPONSIBILITIES 4.3.1.1 Team Leader Responsibilities In preparation for the closeout Meeting announce and discuss the following items: • The closeout meeting will be kept short. There will be no surprises at the closeout. Facility management and specific counterparts should clearly understand all items discussed at the closeout. • The site management will specify attendance at the final closeout. • All findings are subject to the approval of the Team Leader and the Senior Advisers, if applicable. Approval of all findings must be complete prior to the final closeout meeting with the site. • Discuss the final report schedule and deliverables. All findings should be discussed at this meeting. A consensus must be reached by the team on the categorization of each finding as pre-start, post start, observation, or, if appropriate, deleted. Discussing all findings at this meeting will help bring the team to a unified position on the general status of the facility or process being reviewed. The final conclusion, major issues, and overall themes resulting from the ORR will be discussed and agreed upon at this final meeting. 4.3.1.2 Team Member Responsibilities Be prepared to discuss and justify all findings. Be prepared to participate in the discussion of the final conclusion, major issues and overall themes resulting from the ORR. In addition, prior to the closeout meeting with the site, all Forms 1 and 2 should have been completed, signed, and submitted to the Team Leader. The completion of the forms means that factual accuracy of the individual forms has been confirmed with the counterparts. All inputs to the final report must be submitted by each team member and accepted by the Team Leader before the closeout meeting and before the team member leaves the site. Summary evaluations should also have been completed and submitted to and signed-off by Team Leader prior to the closeout meeting. 4.4 CLOSEOUT MEETING The closeout meeting, or exit briefing, is the forum by which the Team Leader informs the site management of the major issues identified during the ORR. There should be no surprises at this meeting as counterparts and site staff should have been kept well apprized of all issues during the conduct of the ORR. 4.4.1 RESPONSIBILITIES 4.4.1.1 Team Leader Responsibilities Prepare the exit briefing based on the Forms 1, Forms 2, and conclusions drawn by the ORR team. As a goal, provide a draft copy of the final report to the senior managers. At a minimum, prepare the executive summary of the draft report as a handout to everyone attending the briefing. Factual accuracy reviews of the final, completed report may delay delivery of the final, signed report for a week.

In some cases, it is appropriate to request selected team members to participate in the closeout briefings. If the team leader chooses this option, it is important to ensure the briefing retains a consistent theme and conclusion. 14 DOE-HDBK-3012-2003 4.4.1.2 Review Coordinator Responsibilities • Assist the Team Leader in preparing the closeout briefing. • Make arrangements for the site to mail information that was brought with the ORR team or collected during the review to the team members. This package should be sent on the day of the closeout meeting. • Make copies of Forms 1 and 2 for dissemination to site management. 4.4.1.3 Team Member Responsibilities All team members who remain on site should attend the closeout meeting and be prepared to participate in the exit briefing or answer questions regarding their findings should the Team Leader request their support or participation. If team members will present portions of the exit briefing, they should prepare presentation materials for review and acceptance by the team leader prior to the closeout meeting. 5.0 POST ORR 5.1 THE FINAL REPORT 5.1.1 RESPONSIBILITIES 5.1.1.1 Team Leader Responsibilities The final report should be completed with all signatures prior to leaving the site. As noted above, it may be appropriate to delay the final Team Leader approval signature for up to a week to permit a final factual accuracy review of the entire report by the site. The Senior Adviser, if applicable, should review and concur with the final report before it is issued. The Team Leader approves the ORR report and submits it to the Authorization Authority, or the senior contractor manager who directed the start of the ORR. The final report should be sent to the Authorization Authority or senior contractor manager with copies to the applicable line managers, program office, operations and/or site office, and EH. The Authorization Authority or senior contractor line management should accomplish additional distribution. Ensure each team member has signed the final report indicating his/her concurrence with the conclusions of the ORR in his/her area of responsibility or review. If a team member has a dissenting opinion, refer to DOE-STD-3006-2000, Planning and Conduct of Operational Readiness Reviews, for guidance. Detailed guidance for the preparation of the final report is contained in sections 5.5, 5.6, 5.8, and 5.9.3 of the ORR Standard. Ensure that all ORR files that may be needed in the future are maintained in one location as determined by the line management and site procedures. The completion of the ORR and the finalizing of the report do not signify the end of the ORR process. Several actions may require the participation of the Team Leader including briefings, interpretations and possible defense of findings, and if requested by line management, review of action plans, and review of finding closure plans. Additional guidance can be found in section 5.7 of the ORR Standard. 5.1.1.2 Team Member Responsibilities Team members may be called upon to assist the Team Leader in several ways including briefings, finding interpretation, review of action plans, or review of finding closure. The Team Leader will notify team 15 DOE-HDBK-3012-2003 members when such assistance will be needed. Section 5.7 of the ORR Standard provides additional discussion concerning post-ORR activities.

Resolution of findings and approval of Corrective Action Plans are the prerogative of the line managers responsible for facility operations and must not be usurped by the ORR Team. While the ORR team members should provide information as requested concerning the meaning or basis for a finding and render judgment, if requested, as to whether a series of corrective actions will be adequate, they must avoid being placed in a position to approve or disapprove any actions that are the responsibility of line management. 5.1.1.3 Review Coordinator Responsibilities • Draft thank-you letters for Team Leader's signature. • Collect, condense, index, and file pertinent ORR information to include: - All correspondence related to the ORR; - All final Forms 1 and 2 signed by the inspector and Team Leader; - The Shift Evolution and Interview list and schedule; - The approved Plan of Action; - The approved Implementation Plan; - The approved Final Report with all signatures from the team members and Team Leader; and Facility basis documentation. 6.0 LESSONS LEARNED FROM PREVIOUS ORR/RAs The following items were learned from conducting previous ORRs and RAs from which both line managers and reviewers could benefit. An updated lessons learned file can be accessed on the ORR link of the EH web page. 6.1 ACHIEVING READINESS The successful completion of any ORR will be strongly influenced by the degree to which readiness to start program work has been achieved prior to the start of the ORR. While it is the responsibility of line management to achieve a condition of readiness, experience indicates that the ORR Team Leader can influence the success of the process to gain readiness during his/her preparations for the ORR. In many cases, the potential for the particular problem to occur will become evident during pre-visits or other ORR preparations. The following lessons learned reflect areas in which the ORR Team Leader and team members may improve the success of the ORR. • Experience shows that prerequisites should provide significant detail and be fully measurable in order to permit line management to track each prerequisite to completion. The ORR/RA standard stresses the fact that the prerequisites should be tied to the Core Requirements, which are what will be evaluated in the readiness determination. So, ensure that the ORR Prerequisites specified in the Plan of Action support achieving readiness. • A thorough Management Self-Assessment (although not required by DOE O 425.1C) to assist line management in verifying that readiness has been achieved is an important final step in preparing for the ORR. The Contractor ORR Implementation Plan should reflect an intention to review the results of the Management Self-Assessment. The DOE ORR will review the results of the Contractor ORR. • Inadequate, incomplete, or undefined incorporation of the safety basis documentation into procedures and policies has frequently resulted in delays in starting an ORR or in significant findings during the ORR. This is frequently caused by late development and approval of the safety basis documentation. Line management may not fully grasp the time and effort necessary to develop 16 DOE-HDBK-3012-2003 and implement all of the "flow down" procedures and surveillances necessary to put the requirements of the approved safety documentation into practice. Ensure full implementation of the safety basis. • Inadequate or incomplete evidence files or other verifiable documentation that demonstrate the

prerequisite conditions have been met may result in an inability to verify readiness has been achieved. The Implementation Plan should clearly indicate the intentions to review these evidence files. • A responsible representative of the line management must be prepared to demonstrate to the ORR team that the conditions required to be met as a prerequisite to resumption, have actually been met. In most cases, this individual will be the counterpart who is assigned to work with the individual ORR Team member. The counterparts should be identified and utilized during the pre- visit. • Assigned counterparts must understand their responsibility to support the ORR team member. The counterpart must present the information and documentation requested. The counterparts must accept responsibility for gaining information and resolving questions. The counterparts should be fully dedicated to a single ORR team member for the duration of the ORR field work. This expectation should be clearly defined during the pre-visit. • Lack of experience on the part of line management as to the expectations of the ORR team in the areas of drills and evolutions expected to occur during the ORR will lead to difficulty in completing established objectives. The Team Leader and team member must devote considerable attention to communicating expectations to responsible line management at the facility level. • Inadequate validation and verification of operational or maintenance/surveillance procedures that are newly prepared or recently modified will result in readiness not being achieved when the ORR starts. An experienced ORR team leader or team member may identify concerns with the adequacy and maturity of these procedures during the pre-visit. If problems are apparent, they should be identified to senior management at the end of the pre-visit. • Lack of formal, structured preparation by DOE line management of the personnel, programs, and documentation to be evaluated during the ORR will delay completion of required activities. DOE line management should be encouraged to conduct a Management Self-Assessment to ensure that prerequisite conditions for which they are responsible have been met. Potential problems in this area may be apparent during the pre-visit and, if noted, pointed out to senior DOE management at that time. Note: It should be apparent to the observant Team Leader during the pre-visits and other ORR preparations if difficulties with achieving readiness will be problematic. Although the ORR Team Leader will not be able to deal with the problems directly, his identification of the problems to the appropriate DOE or contractor line manager should be considered. 6.2 ITEMS TO NOTE BEFORE PROCEEDING WITH AN ORR • Thoroughly review the ORR Plan of Action, which, among other things, defines the scope of the ORR and lists the prerequisites for the ORR. The success of the ORR will be enhanced if the POA provides specific detail as to the scope of each core requirement, and defines the prerequisites in sufficient detail. Prerequisites should be specific and define measurable conditions. If the ORR Team Leader is identified prior to final approval of the POA, he should be included in the POA review and approval process and should ensure that the POA contains adequate detail to support a successful ORR. The POA should be approved as early as possible in the startup/restart process.

• Gain familiarity with the facility and the project programs prior to the start of the ORR. Site access training, facility walkthroughs, and document reviews are essential for team members to gain the necessary familiarity with the project prior to the kickoff of the ORR. 17 DOE-HDBK-3012-2003 • Unescorted and unrestricted access to the facility is essential to effectively complete the review. This will require some investment of time and money in the preparation process. Get agreement, during development of the ORR plans, between the facility contractor and the Department of Energy on the details of the operations that are available for demonstration. Cold runs, partial simulations, and full walkthroughs are options, but an effort must be made to achieve a demonstration as close as possible to actual operations as allowed by the current restrictions of the shutdown. • Obtain and review the documented results of the contractor ORR/RA, including corrective action plans and evidence files documenting corrective action taken. Copies of corrective action documentation need to be readily available to the DOE ORR/RA team. Corrective action closure packages should be prepared in accordance with DOE-STD-3006-2000, Section 5.7.3. • Give consideration to requiring the contractor to deliver a completed set of surveillance procedures and authorization basis documents to the team leader as a prerequisite to the ORR/RA. • Ensure that records, plans, and other documentation requested by the ORR/RA team are readily available, preferably in a central location. A review several days in advance of the ORR/RA should be used to verify that what was requested is really there. In addition, this will reinforce the importance of the information requests by the individual team members. 6.3 ADMINISTRATION OF THE RESULTS • Conduct a training session prior to the ORR/RA or no later than the beginning of the review process to communicate the expectations and deliverables required of the team. Format, content, and style (how to use and fill out) of Forms 1 (Assessment Forms), of Forms 2 (Deficiency Forms), and functional area summaries should be discussed to eliminate a compounding of the administrative load as the review progresses. Acceptable samples of these deliverables should be provided to all the team members prior to the start of the review. This handbook provides explicit instructions on the use of the forms, and the team leader can use this to provide training to the team. The technical editor or the Review Coordinator should be equally aware of the requirements and could conduct the training. • Advise site personnel as soon as an issue starts to be identified. The use of a draft Form 2 may be a method to clearly state the issue. Be sure that it is understood that the forms are draft documents (and should be marked as such) to avoid confusion or drawing of premature conclusions. Providing these forms to the site provides them an opportunity to produce further information that may clarify the issue and creates an air of openness that will contribute favorably to the review. 6.4 CONDUCT OF DRILLS AND OTHER OPERATIONS • Specify requirements for Emergency Preparedness or facility drills prior to the start of the ORR/RA. Detailed guidance on drill expectations, e.g., type of drill, desired preconditions, scenario requirements, etc., should be provided to the facility prior to the ORR/RA.

• Ensure that site personnel understand fully the conditions that needed to be simulated during each drill and performance demonstration. All documentation such as Radiological Work Permits, Work Procedures, etc., must reflect the simulated condition. • Define, during the preparation phase, the role, authority and responsibilities of the ORR team when it is monitoring the conduct of an emergency exercise or drill. The ORR team must not disrupt the flow of the exercise. The ORR team is evaluating both the facility and the emergency preparedness training organization and drill control. These roles, authority and responsibilities require definition and must be fully understood by the facility managers. • Evaluate whether the contractor ORR/RA ensures the integration of equipment operability, procedure viability, and training of the operators in a performance-based methodology. Additionally, 18 DOE-HDBK-3012-2003 a detailed plan for the progression from completion of the ORR/RA to startup should be verified if a significant difference exists between the operations that can be demonstrated and normal operations. 6.5 LOGISTICS AND SUPPORT Communicate logistical needs and requirements to the site well in advance of the review. For example: • Specify what documents need to be available to the team and that these documents need to be in a central location for easy access by the team during the review. • Specify that computers need to be up-to-date models. Printers and copying machines, with adequate supplies of paper, toner, etc., need to be dedicated to the team for the duration of the review. • Specify the word processing software needed (e.g., Word, PowerPoint). • Request administrative support personnel (typists with selected software experience if at all possible), as they are especially helpful when preparing the final pieces of documentation for the final report. • In cases where the review involves classified facilities/operations, schedule derivative classifiers to come to the work area and review the draft documentation at the end of the day during the last week of the review. • Request sufficient office space and computers to support, at a minimum, each functional area sub- group. Working in one large room is difficult since many discussions take place among site, review team, and field office groups. • Investigate whether laptops can be brought on-site. This is usually not an option when the review involves classified material. The site may require some form of property accountability pass. Establish clear formal and informal communications between the facility and the review team as early as possible. Define the "rules of the road." This will improve the effectiveness and efficiency of the ORR/RA. Explain the open, "filters off" nature of the daily team meetings, and state that other briefings can be held with management. While important, the number and the length of these meetings should be minimized to allow the members of the review team to do their job. 6.6 TEAM ISSUES • Emphasize that team members are dedicated for the duration of the review. If a team member completes their review before the end of the ORR/RA, and is authorized to leave, their completed Forms 1 and 2 must be signed-off (approved) by the Team Leader before they leave the site. If a summary is due from the team member, the Team Leader must accept the summary prior to the team member leaving the site.

• Use site personnel as members of the review team but know that demands for his/her attendance to normally assigned duties can frequently interfere. This can be mitigated somewhat by insisting that site personnel who are team members work in the space allocated to the team for the duration of the review, instead of in their separate office/facilities. • Use a dedicated group of team members to evaluate drills, evolutions, and upsets if possible as it can be valuable to the team leader. A group leader can assign specific observational roles to each group member. Having the group hold a post-event critique will allow development of a form 1 that provides an integrated discussion of the events. The other group members can include discussions of pertinent aspects of the event in their form 1 as required by their assigned CRAD. • Emphasize to team members the need to familiarize themselves with the Implementation Plan and the facility/process under review before the ORR/RA. This should be accomplished through the team members' qualification process, which should establish the minimum requirements for facility familiarity, prior to the review. Additionally, the Team Leader is responsible in the team selection to ensure that the team members have the required technical expertise (education/experience) to 19 DOE-HDBK-3012-2003 perform a valid review of the functional area assigned. This should also be documented through the selection and acceptance process. • Ensure that team members fully understand the scope of the ORR/RA and the time frame in which it must occur. In some cases, time management decisions and sampling plans will be important to successful completion of the ORR/RA within the required schedule. • Ensure counterparts have been assigned to each team member or functional area group. Early identification (usually during the pre-visit) of documents needed, interviews expected, evolutions and drills requested is helpful to both the team member and the counterpart. • Inform counterparts they should be prepared to present information requested. Inform team members they need to be respectful of their counterpart’s time constraints and therefore schedule as few after-hours meetings or interviews as possible. • Capture issues and observations in writing as soon as possible. Deficiency Forms (Form 2) should be prepared shortly after an issue is identified. Assessment Forms (Form 1) should be prepared shortly thereafter to capture the full extent of the finding. This will enhance the flow of communication throughout the review and contribute to openness on the part of both the team and the facility. 6.7 DEVELOPING THE IMPLEMENTATION PLAN • Ensure the objectives and criteria clearly encompass all aspects of the Core Requirements defined in the approved Plan of Action. Generally, this will be all core objectives in an ORR and some subset of the total for an RA. The ORR standard breaks the eighteen Core Requirements into 36 objectives, which translate easily into functional areas. Experience has shown this methodology to be successful. Additionally, the "geographic" scope of the ORR/RA must be considered. The POA should specify the bounds of the facility, systems, and personnel involved in the activity under review and the approaches for the objectives and criteria should reflect these boundaries.

• Confirm that the CRADs are written with a clear understanding of the facility systems and processes under review. A generic CRAD should be tailored during the pre-visit to successfully complete a thorough and critical review. Without this understanding and modification of the CRADs, the review effort will suffer. • Fully involve the team early in the process for training and Implementation Plan development. Early dialogue with facility personnel to gain understanding of the activity in progress and the contractor activities in progress is particularly useful. The Implementation Plan used by the contractor and DOE should be parallel in numerous respects. • Give the POA and the Implementation Plan to oversight groups (EH, DNFSB, State Agencies as required) as soon as possible. Early review and input from all stakeholders will reduce last minute perturbations. • Determine the contract requirements for the facility in reference to Standards Requirements Information Documents (S/RIDs), Work Smart Standards, DOE Orders, Policies, Manuals, Notices, etc., and Integrated Safety Management. Conduct the review in a manner consistent with the requirements contractually invoked on the facility. Statements regarding the status of the facility in these areas should be included in the final report in addition to the overall evaluation of the compliance posture required. 6.8 DEFINING THE SCOPE AND GRADED APPROACH • Define the scope of the DOE ORR consistent with that of the contractor ORR. Differences between the two result in difficulties in performing the DOE ORR. DOE O 425.1C provides a structure for such consistency. Differences will occur in the evaluation of the Core Requirements specifically related to DOE management, which are necessarily absent from the contractor review. 20 DOE-HDBK-3012-2003 • Going beyond the defined scope in a case where an ongoing program that supports the process or facility under review but which is outside the scope of the review, presents potential problems. If the team is to make a judgment that operations can be conducted safely and it requires investigation at the interface of the process/facility with the supporting program, then going beyond the defined scope is proper. • Define the graded approach as it applies to each CRAD as specifically as possible. • POST-ORR CORRECTIVE ACTIONS Although the ORR/RA follow-up corrective actions are not the responsibility of the Team Leader, line management may ask for input and advice as to the necessary action(s) to correct the issues identified during the ORR/RA. The value and effectiveness of the ORR/RA can be significantly decreased by ineffective corrective actions to resolve the issues identified during the ORR/RA. The Team Leader, through interest in the effectiveness of the overall ORR/RA process, retains an interest in the post- ORR/RA corrective actions. In general, the following items need attention: • Facility management sometimes starts taking immediate action to correct the symptoms identified by the ORR/RA team without adequate attention to understanding the root cause and programmatic basis of the issue. As a result, the corrective action taken may be incomplete, shallow, and short-lived. • DOE line management review of the status of the contractor ORR/RA corrective actions should require corrective action closure packages for the contractor ORR/RA that meet the requirements of DOE-STD-3006-2000, section 5.7.3. In that way, the contractor may more effectively evaluate their own corrective actions before reporting readiness to proceed to DOE.

• When DOE line management forwards the DOE ORR/RA to the responsible contractor, the actions expected as to the contractor's corrective actions should be clearly stated. Unclear or unstated expectations result in false starts and failure of the responsible contractor to meet DOE's expectations. This in turn has resulted in delays and incomplete actions following several ORR/RAs. • In those cases where issues have been identified as the responsibility of a DOE field organization, DOE-HQ should formally communicate corrective action expectations to the responsible field organization. The expectations should include corrective action plans and closure packages. 6.10 MISCELLANEOUS LESSONS LEARNED • Engage a technical editor, if at all possible, during the second week of the review. The final report left at the closeout briefing, although a draft, is the site or facility's first encounter with the full results of the review and as such warrants a technical editing. This person should stay until all Forms 1, Forms 2, and summaries have been approved, the executive summary edited, and the final report formatted, collated, and printed. The introductory material (cover sheet, background, scope, etc.) can be written before the review is started, thus reducing the workload during the review and allowing the team to concentrate on issues at hand. • Do not assign any CRADs to the Team Leader, however technically qualified. • Identify an Assistant Team Leader or Senior Advisor. Having an Assistant Team Leader/Senior Advisor can greatly improve efficiency in the conduct of the ORR/RA. An Assistant Team Leader/Senior Advisor is particularly beneficial during the labor intensive ORR preparation phase. • If possible, include at least one trainee on the ORR team so that the pool of experienced readiness review team members continues to increase. • Select a Review Coordinator with previous experience in the ORR/RA process, as this function is essential in the assimilation of the data provided by the team members. Without accuracy and timeliness in this function, conclusions are subject to question. 21 DOE-HDBK-3012-2003 • Select the Team Leader carefully. The Team Leader should be senior to the team members. He or she should not be a peer or a co-worker of the team members. Selection of the appropriate Team Leader is important in that he/she must exercise some degree of control over the team. • Evaluate on-the-job-training, drills, and other training evaluation against the established guidance in DOE Orders, standards, guides, and handbooks. This helps eliminate the subjective aspects of the review. Additionally, this intention should be communicated to the contractor for the conduct of the contractor ORR/RA. 6.11 GENERIC LESSONS LEARNED THATAPPLY TO THE DESIGN, CONSTRUCTION, OPERATION, AND DECOMMISSIONING OF DOE FACILITIES • During startup of new facilities, numerous cases have been observed in which the surveillance requirement supporting the safety basis could not be completed, or if completed, did not properly verify the safety basis requirement. Surveillance procedures should be completed, validated, and executed as part of the startup program, prior to the implementation of the supported Technical Safety Requirements (TSRs). Actually executing the surveillance will reveal such problems as: test points that are not installed; test points that are inaccessible; and other interfering interlocks or functions. These problems have occurred repeatedly in previous ORR/RAs.

• Additionally, determination must be made that the surveillance actually tests the function or protective action upon which the safety basis depends. Previous ORR/RAs have revealed numerous occasions in which surveillances have been conducted and through either invalid acceptance criteria or a misunderstanding of the as-built configuration, the surveillance did not test the required functions. 22 DOE-HDBK-3012-2003 APPENDICES STANDARD FORMS AND LETTERS Appendix 1: Key Points to Discuss during Initial Contact with Site Appendix 2: Site Contact Memorandum Appendix 3: Logistics Appendix 4: ORR Preliminary Schedule Appendix 5: Pre-visit Schedule Appendix 6: Checklist for ORR Team Information Packages Appendix 7: Detailed Schedule for ORR Team Appendix 8: Transmittal letter for data packages to team members Appendix 9: Team Member Deliverables Appendix 10: Instructions and Samples for Completing ORR Forms Appendix 11: Pre-start and Post Start Determination Criteria 23 DOE-HDBK-3012-2003 INTENTIONALLY BLANK 24 DOE-HDBK-3012-2003 APPENDIX 1: KEY POINTS TO DISCUSS AT INITIAL CONTACT WITH SITE Refer to the Plan of Action, and: 1. Contact DOE or contractor Line Manager for the Facility. Facility _____________ Director/Manager’s Name ____________ Telephone Number __________ Email Address_______________ 2. Identify yourself as the Team Leader for the ORR. 3. Discuss the tentative review schedule. Pre-visit dates:_______ to ________ ORR dates: ________to _________ 4. Explain overview of review process (if requested). Pre-visit and Preparation • Site Visit: 2-3 day for site and facility familiarization, walkthrough, and presentation to site of inspection process. • Information needed from site: (list to be provided) • Counterparts: Site expected to provide list of counterpart(s) for each area. Onsite Review Period • Purpose: Identify findings and observations for the safe startup of facility process(es). • General: Up to 30 team members, approximately a two-week period, though with a graded approach, as few as 10-15 team members may be needed for one week. • Logistic Support: Conference room, workspace, computers, phones, fax, etc. (list to be provided). 5. Request point-of-contact at site to support review and to receive formal correspondence. Name: _______________________________________________ Telephone number:____________________________________ Email address________________ 6. Leave your telephone number and Email address. Advise site contact that written confirmation will follow. 25 DOE-HDBK-3012-2003 APPENDIX 2: SITE CONTACT MEMORANDUM Refer to the Plan of Action. Operations Office, Site Office Manager, or Facility Manager. Operational Readiness Review of [Facility] An Operational Readiness Review (ORR) of [Facility] will be performed. [Site points-of-contact] have been identified as the lead Department of Energy and contractor facility contacts at the site. [Team Leader] will lead the review team. Enclosed are a preliminary schedule for the overall review and a detailed schedule for the onsite portion, an agenda for the pre-review visit, a list of support required for the onsite portion of the review, a list of site documents required to prepare the team for the review, and an ORR summary document. Please forward one copy of each requested document to [Team Leader] by [Date]. Please contact [Team Leader] (phone #) if you wish to discuss these items or any other aspect of the review. Your assistance is greatly appreciated.

Team Leader Attachments cc w/Attachments: Manager, Area Office Program Office Director Facility Manager _______________________________________ *Attachments ORR Logistics (Appendix 3) ORR Preliminary Review Schedule (Appendix 4) ORR Pre-visit Schedule (Appendix 5) 26 DOE-HDBK-3012-2003 APPENDIX 3: ORR LOGISTICS The following list suggests items the Team Leader should consider. The Team Leader should personalize it for each ORR. These items should be discussed and agreed upon during the pre-visit. 1. The team will require workspace for approximately [#] people to review documents, conduct interviews, prepare reports, and conduct meetings. 2. Workspaces should be equipped with several IBM compatible PCs and laser printers. A current version of Word (or other preferred software) should be installed on each system. One copy of a familiar graphics program (PowerPoint) should be available on one of the team support staff's computers. Access to site electronic document system should be arranged. 3. Workspaces should have phones with contractor and DOE phone books. 4. The team will require ready access to photocopy machines. 5. The team will require ready access to a facsimile machine. 6. A list of support personnel or a single contact to repair equipment, receive phone messages, and aid in logistical problems should be available at the start of the review. 7. Adequate visitor parking should be available for team members. 8. The team may require ready access to an authorized derivative classifier and a classified document repository. 9. The team will require ready access to the following documents in a convenient, centralized location at the site. If possible, direct access to electronic document systems is preferable. The Team Leader may require the team members to review these documents to accomplish qualification for site familiarity, thus they may be required during the pre-visit. The Team Leader should communicate these desires to the site as soon as possible. Note: the following are examples of documents that the Team Leader can request to have available during the ORR. • DOE Orders and mandatory standards (DOE O 425.1) • General facility schematics. • List of Facility Representatives and Facility Managers. • Hierarchy of documents: Top-level policy and site wide procedures down to operating instructions. • Current listing of titles, numbers, and revisions of manuals/procedures. • Implementation plans submitted to Headquarters for DOE Orders. • Any exemption, exception, deviations, variances, or waivers requested for Environment, Safety and Health (ES&H). • Orders. • DOE and Contractor ES&H oversight plans/programs, and self-assessment program manuals. • All DOE Field/Site Office and Contractor self-assessment reports for the past 12 months. - Contractor and DOE organization charts with management/supervisor names. - Functional description for ES&H organizations (NQA-1). 27 DOE-HDBK-3012-2003 - General site map. - Contractor ORR and/or Management Self Assessment reports. • The most recent Integrated Safety Management Verification reviews, Design Safety Analysis, Management Self-Assessments, and Headquarters On-Site Review of Field Element Performance. • A list of functional review reports issued within the last 12 months by DOE- [Field Office] regarding [Site] (DOE N 251.21). • All DNFSB, Environmental Health, Nuclear Security, Inspector General, Environmental

Protection Agency, and Occupational Safety and Health Administration (OSHA) reports issued within the last 12 months. • Annual Site Environmental Reports; the most recent, even if in draft form, and the preceding year (ref. DOE O 231.1). • Environmental Monitoring Plan and it's associated Meteorological Information/Monitoring Program (ref. DOE O 231.1). • Spill Prevention, Control and Countermeasures Contingency Plans (if applicable). • Inventory of hazardous chemicals (those above Threshold Planning Quantities per SARA Title III) by room, area, or building, as appropriate to planning walkthroughs. (Remove portions that are classified). • SARA Title III Tier One and Two reports. • Most recent site-wide environmental impact statement and/or environmental assessment. • National Emission Standards for Hazardous Air Pollutants report (if applicable). • All Type A and Type B accident investigations reports for the last two years. • OSHA 200 log from past two years. • Site Radiation Control Manual and associated implementation plan. • DOE and Contractor Quality Assurance Program Manual. • Facility and activity operating/technical procedures. • Site and facility maintenance procedures and work packages. • List of Quality Assurance plans and implementation procedures. • DOE and Contractor Conduct of Operations Manual. • List of Conduct of Operations plans and implementation procedures. • Operator training program documentation. • List of ES&H policies and procedures of the Field Office and Contractor. • Contractor's ES&H Manual. • List of applicable Technical Safety Requirements. • Applicable Safety Analysis Report(s). • Site-specific Explosives Safety Manual (if applicable). • Explosives Safety Policy Statements (if applicable). • Explosives Handling, Storage, and Disposal facility Design Bases/Safety Analysis Reports (if applicable). • Site Fire Protection Manual. • Site Pre-fire Plans. • All Fire Department Procedures. • Fire Protection staff organization charts and job descriptions. • Staffing plans and any analyses relating to staffing for the past 12 months. • Applicable DOE Order Compliance Documents and Compliance Schedule Agreements. • Site Emergency Plans 28 DOE-HDBK-3012-2003 APPENDIX 4: ORR PRELIMINARY SCHEDULE Activity Date(s) Pre-visit Onsite Review Closeout Meeting Report Preparation Issue Report HQ Briefing Note: The above dates are tentative. The team intends to conclude identified activities within the allotted windows on the schedule, but adjustments may be made to accommodate particular issues that may require more or less attention. 29 DOE-HDBK-3012-2003 APPENDIX 5: PRE-VISIT SCHEDULE NAME OF FACILITY Activity Date Participants Site Access Month, Day, Year Full ORR Team Pre-visit Start Month, Day, Year Full ORR Team Site Management Orientation Month, Day, Year* (a.m.) Full ORR Team Meeting (1) ORR TL Presentation (2) Month, Day, Year* (a.m.) Full ORR Team Site Management Counterpart Meeting (3) Month, Day, Year* (p.m.) Site Counterparts Full ORR Team Site/Facility Training and Month, Day, Year* (p.m.) Full ORR Team Processing (4) Facility/Process Month, Day, Year* Full ORR Team Walkthrough Pre-visit End Month, Day, Year Full ORR Team

*Site to specify times and locations. (1) Orientation Meeting - The goal is to receive from the site an overview of the site activities in progress and any special operating guidance required for the ORR team members. Include in the presentation: all safety and security briefs to ensure ready access, overviews of the organizational structure, document hierarchy, assessment activities, occurrence reporting and tracking, training organization, lessons learned programs, and any other desired elements (approximate time: 2 hours). (2) Team Leader ORR Presentation - The Team Leader will make a short presentation on the ORR scope and process to the site (approximate time: 30 minutes). (3) Counterpart Meeting - Each ORR team member or functional area is assigned a site counterpart. This meeting provides an introduction of the site counterparts to the team (approximate time: 1-2 hours). (4) Site/Facility Training and Processing - The goal is to receive from the site the training and processing (e.g., respirator training and fitting) so as to permit all team members ready access to the site/facility during the ORR. 30 DOE-HDBK-3012-2003 APPENDIX 6: CHECKLIST FOR ORR TEAM INFORMATION PACKAGES DESCRIPTION APPENDIX No. Transmittal letter. Appendix 8 Maps from airport to hotel/site. Provided by site. Schedules for ORR team. Appendices 4, 5, and 7 Applicable ORR Plan of Action. Provided by Team Leader Applicable ORR Implementation Plan. Provided by Team Leader Site organization charts. Provided by site. Team member assignment and counterpart matrix. Provided by RC1 ORR Order (425.1C) and Standard Provided by RC (DOE-STD-3006-2000) Line Management Self-Assessment (Contractor ORR) Provide by Team Leader Contractor ORR Report (DOE ORR). Provided by Team Leader Site DOE Validation Report (DOE ORR) Provided by Team Leader Other Required Reading Documents Provided by Team Leader Team Member Deliverables List Appendix 9 Blank Forms 1 and 2 on diskette Provided by RC (note: the Objective and Criteria should be filled-in on the Forms) 1 RC = Review Coordinator 31 DOE-HDBK-3012-2003 APPENDIX 7: DETAILED SCHEDULE FOR ORR TEAM [Month, Day, Year] Travel to ______________________. Optional: You may pick up your badge at _____________. [Month, Day, Year] Team meeting at the site (Room _________, a.m./p.m.) [Month, Day, Year] Start ORR. [Month, Day, Year] Tentative date for closeout meeting. [Month, Day, Year] Travel from _______________. 32 DOE-HDBK-3012-2003 APPENDIX 8: TRANSMITTAL LETTER FOR DATA PACKAGES TO TEAM MEMBERS DATE: NOTE TO: [Name of Site/Facility] Operational Readiness Review (ORR) Team Members FROM: Name--DOE ORR Review Coordinator SUBJECT: Operational Readiness Review at [site]. Attached is the detailed schedule for the Operational Readiness Review (ORR) at [site] . Your badge will be ready for pick up at the Badge Office that is located ________. You will need access authorization for Sigmas _________. The badge office is open from ______a.m. to ______ p.m. The site is providing a team workspace located in ________ for our use throughout the entire review period. Attached are the functional area assignments; the review methodology for ORR team members; a [site] organization chart; team member duties and responsibilities; required reading assignments, list of deliverables, and the assigned Criteria and Review Approach Documents (CRADs).

If you have any questions or if I can be of further assistance, please call me at _____, or e-mail me at __________. 33 DOE-HDBK-3012-2003 APPENDIX 9: TEAM MEMBER DELIVERABLES 1. Qualification Summary: due before or during the pre-visit. 2. Security clearance data: due as required by the Review Coordinator. 3. Comments on Criteria and Review Approach Documents: due before or during the pre-visit. 4. Evolution and Interview list: input due to Team Leader prior to the end of the pre-visit. 5. Form 1 for each CRAD Objective assigned: due for signature before the final team meeting of the ORR. May consider daily draft. The Team Leader may require a current draft for review at any time. 6. Form 2 for each deficiency identified: due in draft form as soon as possible after the deficiency is identified and due in final form before the final team meeting of the ORR. 7. Final Report Summary: due in draft form before the final team meeting of the ORR and signed-off by the Team Leader before leaving the site. 8. Final Report, including Forms 1, 2, functional area summaries; reviewed, signed by the team member and Team Leader by the end of the ORR. Any dissenting opinions must be prepared and submitted by the end of the ORR. 9. Lessons Learned for inclusion in the report and in later versions of this Handbook before leaving the site. 34 DOE-HDBK-3012-2003 APPENDIX 10: INSTRUCTIONS FOR COMPLETING ORR FORMS Also see the DOE STD-3006, Planning and Conduct of Operational Readiness Reviews (ORR), Appendix 4, Writing Guide, for additional guidance. ORR ASSESSMENT FORM 1 FUNCTIONAL AREA CRITERIA : DATE: CRITERIA MET YES______ NO______ OBJECTIVE: As written in the Criteria and Review Approach Document (CRAD) for the functional area. Criteria As written in the CRAD. Approach Record Review: as written in the CRAD Interviews: as written in the CRAD Shift Performance: as written in the CRAD Records Reviewed: • Cite a specific document (number, title, rev.) • Cite next document, etc. Interviews Conducted: • Use title, do not use name (use site identifier, as site will respond) Shift Performance Evolution: • Provide description of activity observed. Use N/A if not applicable Discussion of Results: Discuss the record review or if not applicable, write “None”. Discuss the interviews or if not conducted, write “None”. Discuss the shift performance observed or if not applicable, write “None”. Conclusion: Tie together discussion of key points resulting in your conclusion. State whether the criterion was met. Issue(s): Identify with a signifier (letters of Functional Area, e.g., TR) and an arabic numeral (e.g., -1). Write a brief, full sentence description of each issue. This should be an exact match of the title on the Form 2. Cite all Form 2s associated with this criterion. If no Form 2s were generated under this criterion, write “None.” 35 DOE-HDBK-3012-2003 For example: TR-1: write the brief sentence describing the training issue. Inspector: __________________ Approved:_____________________ Team Leader 36 DOE-HDBK-3012-2003 SAMPLE ORR ASSESSMENT FORM 1 FUNCTIONAL AREA CM CRITERIA: 1 DATE: 2/4/03 CRITERIA MET YES______ NO X OBJECTIVE: CM-1 Safety structures, systems and components (SSC) are defined and a system to maintain control over their design and modification is implemented. The Wet Chemistry Operations systems and procedures, as affected by facility modifications, are consistent with the facility description and accident analysis included in the safety basis Modifications to the Wet Chemistry Operations have been reviewed for potential impact on training and qualification. (CORE REQUIREMENT 5, 7 and 9)

Criteria 1. An adequate process has been implemented to ensure that documentation for SSC critical to the safety of Wet Chemistry Operations exist and is kept current, as appropriate for safety functions and that necessary documentation is available to management, operators, and support personnel (DOE-STD- 1073-93, Ch. I.3; DOE Order 5480.19 Ch. II, Section VIII; DOE O 430.1, Sect. 6.f. (4); RUID 9663) 2. Drawings and documentation relied upon for operations and maintenance activities are consistent with the existing configuration of Wet Chemistry Operations. Technical baselines for the XXX Wet Chemistry processes have been defined in accordance with Y17-009INS and accurately reflect the as configured status, including isolated out-of service equipment. (DOE-STD-1073-93, Ch. 1.3; DOE O 4330.4B, Ch. I.3.6.4f; RUID 9859) 3. Wet Chemistry Operations’ safety systems are consistent with the assumptions and descriptions in the safety basis. (10 CFR 830.200) 4. Each SSC should be assigned a grade based on the most important type of design requirements applicable to it. The SSC grade should be used as the basis for the degree of control of all activities associated with the SSC. (DOE-STD-1073-93, Ch. I.3.1.1; RUID 9661) 5. The Unreviewed Safety Question Determinations (USQD) or USQD screenings are implemented per Y74-809 for the identification and disposition of modifications in support of Wet Chemistry Operations. (10 CFR 830.200) Approach Record Review: Review the documentation to ensure modifications conducted on Wet Chemistry Operations’ equipment since the 1994 stand-down have been accomplished consistent with Y15-187 or its equivalent predecessors (a listing of modifications to Wet Chemistry Operations) during the stand-down is contained in the BWXT Y-12, L.L.C. Plan of Action, Y/MA-7827. Review the Building 9212 BIO and OSR for the Wet Chemistry Operations to ensure they are consistent with the physical facility configuration. Determine if the equipment and systems, as described, are part of the technical baselines for Wet Chemistry processes in accordance with Y17-009INS. Determine whether appropriate process descriptions exist, whether these process descriptions describe the isolation of Wet Chemistry Operations from out of service equipment, and that these conditions are under configuration control. Determine whether any in-progress modifications essential to permit operation of the Wet Chemistry processes exist 37 DOE-HDBK-3012-2003 that are not already captured by the XXX Wet Chemistry restart plan. Review non-conformance reports (NCRs) related to Wet Chemistry for possible pre-start actions. Review the Master Equipment Lists (MEL) of XXX’s Wet Chemistry Operations’ systems to ensure the equipment is properly graded per Y15-001INS, labeled per Y10-176, and graded consistent with the safety basis. Ensure that utility, waste, and recycle system drawings have been updated and the defined boundaries between Wet Chemistry systems and utility systems are correctly identified. Determine whether Plant Air System SSCs supplying XXX Wet Chemistry processes have been graded, added to the respective utility MEL, and maintained. Review dispositioned USQ/USQDs for design changes, special procedures and tests, and other proposed changes against the requirements within Y74-809. Determine the status of all ongoing USQs and evaluate their implication on operations. Review records for temporary modifications and verify required actions are implemented.

Coordinate with the Training SME team member to review records of modifications and determines if training and qualification of operators and support personnel is appropriately identified to support the modification. Interviews: Interview management, operations and support personnel responsible for configuration management for Wet Chemistry Operations to determine if they are familiar with their roles, responsibilities, and interfaces with the operations organization. Shift Performance: Perform a system walk-down to determine whether there are uncontrolled modifications to safety systems supporting Wet Chemistry Operations. This walk down should evaluate the accuracy of drawings and other documentation for plant operation and maintenance. At least one recently completed modification or temporary modification should be observed and changes verified, including changes to operating procedures if applicable. Records Reviewed: • PD-XXX-9212-OXD, Revision 1, C-1 Wing Oxide Dissolver Process Description, 6/24/02 • PD-XXX-9212-PX, Revision 1, Primary Extraction and Feed Adjust Process Description • PD-XXX-9212-IE, Revision 1, C-1 Wing Intermediate Evaporators Process Description • PD-XXX-9212-SX, Revision 1, Secondary Extraction Centrifugal Contactors Process Description, 8/12/02 • PD-XXX-9212-IE, Revision 2, B-1 Wing Wiped-Film Evaporators Process Description, 9/27/02 • PD-XXX-9212-DNT, Revision 2, B-1 Wing Denitrator (DNT) System Process Description, 8/12/02 • IE-P1, Revision A, Primary Intermediate Evaporator Feed Receiving Tank F-5203, 6/5/02 • IE-P2, Revision A, Intermediate Evaporator Accumulation Tank F-5202E, 6/5/02 • IE-P3, Revision 0, Primary Intermediate Evaporator Feed Wall Tanks F-5202 o A-D, 5/1/02 • IE-P4, Revision 0, Primary Intermediate Evaporator Feed Head Tank F-5201, 5/1/02 • IE-P5, Revision C, Primary Intermediate Evaporator C-5201C, 8/19/02 • IE-P6, Revision B, Primary Intermediate Evaporator Condensate Tanks F-5000 A-C, 10/2/02 • IE-P7, Revision B, Primary Intermediate Evaporator Product Receiving., Tank o F-5202 F, 9/23/02 • IE-P8, Revision 0, Secondary Intermediate Evaporator Feed Head Tanks F-5302 A-B, 5/1/02 • IE-P9, Revision A, Secondary Intermediate Evaporator C-5301B, 6/5/02 • IE-P10, Revision A, Intermediate Evaporator Product Sampling Adjustment, Cooling, 6/5/02 38 DOE-HDBK-3012-2003 • IE-P11, Revision B, Secondary Intermediate Evaporator Product Stor Tanks F-5401 A-C, 9/23/02 • IE-P12, Revision B, Secondary Intermediate Evaporator Product Stor Tanks • F-5401 D-F, 6/25/02 • IE-P13, Revision 0, Intermediate Evaporator Cabinets Instrument Air Headers and Power Supply, 5/1/02 • IE-P14, Revision 0, Intermediate Evaporator Bubbler and Blowdown Air Supply, 5/1/02 • WFE-P1, Revision D, Wiped-film Evaporator - Evaporator C-330A Process System Diagram, 12/12/02 • WFE-P2, Revision B, Dated 1/27/03, Wiped-film Evaporator - Stillwell and Control Cabinet Process System Diag. • WFE-P3, Revision C, Wiped-film Evaporator - F-330A Product Tank Process System Diag., 1/27/03 Interviews Conducted: • Technical Support Manager • System Engineer for the Oxide Dissolver Process • System Engineer In-training for the Oxide Dissolver Process • System Engineer for the Primary Extraction Process • System Engineer for the Intermediate Evaporators Process • System Engineer for the Secondary Extraction Process • System Engineer for the Wipe-Film Evaporator Process • System Engineer for the Denitrator Process • XXX Configuration Management Manager • Facility Technical Support Representative • Restart System Engineering Supervisor • ET&I Electrician • Unreviewed Safety Question Determination Manager • Operations Supervisor • Planner • Facility Safety Manager • Instrument Engineers (2)

Shift Performance Evolutions: • Walkdown of Intermediate Evaporators System • Walkdown of Primary Extraction System • Denitrator High Temperature Interlock Surveillance • Secondary Extraction System Contactor Speed Adjustment • Management Review of Uncontrolled Change to Denitrator Discussion of Results: Record Review: The Intermediate Evaporator system was walked down against system drawings, including hardware, piping, labeling, and design features. Parts of the Primary and Secondary Extraction Systems and the Denitrator system were also walked down. There was excellent consistency between the physical configuration and the facility documentation. Safety design features were generally noted on the Process System Diagrams (PSDs). Component labeling was excellent. A small number of safety design features that could use improved notation on the PSDs for configuration management purposes were noted. Minor editorial differences between control panel labels and the PSD for the Denitrator were noted. 39 DOE-HDBK-3012-2003 These control panel labels did match the Denitrator High Temperature Interlock Procedure. Air supplies to XXX Wet Chemistry processes were appropriately graded and maintained. Grading boundaries were mostly, but not always, noted on PSDs. Controlled copies of PSDs and Process Descriptions (PDs) are currently not kept in the process areas. All system drawings that were sampled were found to be updated and to identify system boundaries. The PDs for the OXD, PE, IE, SE, WFE, and DNT systems were reviewed. Good consistency was noted between the PSDs, the PDs, and the process descriptions in the BIO. Process boundaries are identified, but many still do not comply with the guidance in Appendix E of XX-0011INS. While compliance with this guidance should be improved, it is not a significant issue as there is some justification for the exceptions. Description of isolation points from out-of-service equipment should be improved in some cases (PX system particularly noted) (CM-1-1). The PDs have developed enough to be excellent training aids, but could be more useful as configuration control documents if more design basis information (including safety basis information) were included. DOE-STD-3024, System Design Descriptions, though not directly applicable to PDs, could provide useful guidance for further development of the PDs. Six Change Request Packages were reviewed. All records indicated that changes have been executed in accordance with the site change control procedure. USQD packages associated with these and other changes were reviewed. Differences were noted between the execution of the USQD process and the intent of 10 CFR 830.203 and its guidance (CM-1-2). USQD Screen B1W-2003-002 was reviewed and found to identify the wrong system as the subject of the change, to screen out a change that might have needed to go to NNSA for approval, and to identify the wrong USQD worksheet number on its continuation pages. No essential in-progress modifications were noted. USQD B1W-99-003 was reviewed. This was a positive USQD resulting from a “discovery condition.” Notifications related to this USQD were made to NNSA via Occurrence Report ORO-XXX-XNUCLEAR- 1998-0094. Based upon records contained in the Occurrence Report, the appropriate steps were followed notifying NNSA of the change. Records for temporary modifications were reviewed and were found to be up-to-date with required actions implemented.

Interviews: System Engineers and the Restart System Engineering Supervisor were interviewed and MEL and Grading Work Sheet Packages were reviewed in order to determine if XXX Wet Chemistry SSCs have been graded and maintained on MELs, and properly labeled. Each SSC is graded based on function and functional requirements (safety features), and related source documents (e.g. Criticality Safety Evaluations). An SSC grade of 1-4 is assigned. The grading is documented on grading work sheets. With the exception noted below, XXX Wet Chemistry SSCs have been graded per Y15-001 INS, documented and maintained on MELs, and labeled in accordance with Y10-176. An exception was noted that software components of systems were not graded per Y15-001INS (CM-1-3). As a result, changes to software may not be appropriately controlled, and may fail to enter the USQ process. Based on the definition of SSC in Y15-001INS, software is considered a “component” and thus requires grading and inclusion in the Integrated Safety and Change Control Process. System Engineers, the Restart System Engineering Supervisor, the Technical Support Manager, the XXX Configuration Management Manager, an ET&I Electrician, and a Facility Technical Support Representative were interviewed about their configuration management roles, responsibilities, and interfaces. All were knowledgeable of and concerned with their assigned systems and responsibilities. System engineers understood the safety-related design features and functions of their systems. As the central point for complete knowledge and understanding of their systems, the system engineers could 40 DOE-HDBK-3012-2003 benefit from a better understanding and ownership of the instrument and control components of their systems, particularly the software components. Shift Performance: During observation of the Denitrator High Temperature Interlock Surveillance, it was noted that uncontrolled changes were made to the wiring of the Denitrator Control Cabinet (CM-1-4). This change was introduced through the Maintenance Job Request process. Since it did not go through the change process, it did not receive sufficient engineering and safety review and it did not get entered into the USQD process. The management review of this event was observed. It was a thorough process that identified causal factors and effective resolutions. Discussion of the unapproved change to the Denitrator Control Cabinet revealed that some personnel could benefit from training on identification of changes. Conclusion: The objective has not been met. Issue(s): CM-1-1 Software components of facility systems are not graded as required by the Configuration Management Program. As a result, changes to software may not be appropriately controlled, and may fail to enter the USQ process. (Post Start) CM-1-2 Some system design features that are important to safety are not adequately controlled by system documentation. (Post Start) CM-1-3 Implementation of the USQ process does not fulfill the intent of 10 CFR 830.203 or its associated guidance document. (Pre-Start) CM-1-4 Uncontrolled changes were made to the wiring of the Denitrator Control Cabinet. (Pre- Start) Inspector:_____________________________ Team Leader: ________________________ 41 DOE-HDBK-3012-2003 INSTRUCTIONS FOR COMPLETING ORR DEFICIENCY FORM 2 Objective: Criterion: Finding______ Observ.______ Pre-start____ Post start____ Issue No.: Date:

ISSUE: Write a brief, full sentence, description of the issue. Reference the corresponding Form 1. Use the present tense. REQUIREMENT: (Reference locations are provided in the CRAD). Write the requirement from a rule, DOE order, mandatory standard, or site requirements document REFERENCE(S): • Be specific. • Include section and paragraph. • If possible, of all references which support the conclusion that existing conditions are deficient. • This must match the requirement(s) stated above. DISCUSSION: Describe the issue. The discussion should be self-explanatory and support the conclusion. Use the present tense. Cite specific example(s) for a generic issue. CONCLUSION: State whether it is a pre-start or a post start finding. The ORR Implementation Plan should provide evaluation guidelines. Inspector: __________________ Approved:_____________________ Team Leader 42 DOE-HDBK-3012-2003 SAMPLE ORR DEFICIENCY FORM 2 Functional Area: CM Objective No.: 1 Finding: X Observ.: Pre-Start : X Post Start Issue No.: CM-1-4 Rev.: 1 Date: 1/31/03 ISSUE: Uncontrolled changes were made to the wiring of the Denitrator Control Cabinet. REQUIREMENT: The contractor responsible for a hazard category 1, 2, or 3 DOE nuclear facility must implement the DOE-approved USQ procedure in situations where there is a temporary or permanent change in the facility as described in the existing document safety analysis. Unreviewed Safety Question Determinations (USQD) or USQD screenings are implemented per Y74-809 for the identification and disposition of modifications in support of Wet Chemistry Operations. The change control process will ensure that changes are properly identified, developed, reviewed, approved, implemented, and documented. REFERENCE(S): • Y15-187, 10 CFR 830.203(d), Y-74-809 DISCUSSION: During observance of the Denitrator Over-temperature Interlock Surveillance, modifications to the rear of the control cabinet were noted. It was determined that there was no change requested to install these modifications. They were installed under a maintenance request. Changes cannot be installed under a maintenance request as it does not provide for engineering and safety review of the change. Since there was no change request package prepared, there was also no USQD for this change. CONCLUSION: As this finding relates to the potential for allowing uncontrolled changes to XXX SSCs and the safety basis without proper analysis and review, this is a pre-start finding. Inspector: __________________________ Team Leader: ________________________ 43 DOE-HDBK-3012-2003 APPENDIX 11: PRE-START AND POST START DETERMINATION CRITERIA FOR ORRs The Team Leader and the ORR team members may use the following checklist to evaluate if an issue must be corrected prior to startup. A. Initial Screening 1. Does this issue involve a safety system? 2. Does this issue involve processes, functions or components identified in the Technical Safety Requirements (TSR), Operational Safety Requirements (OSR) or nuclear safety control procedures? 3. 4. Does this issue involve potential adverse environmental impact exceeding regulatory or site specific release limits? 5. Does this issue impact non-safety processes, functions or components that could adversely impact safety related processes, functions or components? 6. Is this issue non-compliant with [site] or DOE [site office] approved startup documents? 7. Does this issue indicate a lack of adequate procedures or administrative systems?

8. Does this issue indicate operational or administrative non-compliance with procedures or policy? 9. Has this issue occurred with a frequency that indicates past corrective actions have been lacking or ineffective? 10. Does this issue require operator training not specified in existing facility training requirements? 11. Does the issue involve a previously unknown risk to worker or public safety and health or a previously unknown threat of environmental insult or release? If the response to any of the above is yes, further evaluation in accordance with the issue impact criteria below is required. 44 DOE-HDBK-3012-2003 B. Issue Impact 1. Does the loss of operability of the item prevent safe shutdown, or cause the loss of essential monitoring? 2. Does the loss of operability of the item require operator action in less than ten (10) minutes to prevent or mitigate the consequences of events described in the Safety Analysis? 3. Does the loss of operability of the item cause operation outside the TSR/OSRs or Safety Analysis? 4. Does the loss of operability of the item result in a reduction of the margin of safety as described in the Safety Analysis? 5. Does the issue indicate a lack of control which can have a near term impact on the operability or functionality of safety related systems? 6. Does the issue involve a violation or potential violation of worker safety or environmental protection regulatory requirements that pose a significant danger to workers, the public, or of environmental insult or release? If the response to any of the above questions is yes, the item should be considered a pre-start or startup item. 45 DOE-HDBK-3012-2003 46 CONCLUDING MATERIAL Review Activity: Preparing Activity: NNSA Service Center DOE Field Offices NA-117/124 CH NNSA Service Center ID EH-3/53 OR Project Number: EM-3 SRS OPER 0004 NE-44 ORP ER-8.1 RL SO National Laboratories LLNL INL ORNL SNL Site Offices YSO LSO LASO SSO PZSO NSO SRSO DOE-HDBK-3012-2003