DOE-HDBK-3012-2015, Team Leader's Good Practices for Readiness Reviews

DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. DOE-HDBK-3012-2015 SUPERSEDING DOE-HDBK-3012-2003 DOE HANDBOOK TEAM LEADER’S GOOD PRACTICES FOR READINESS REVIEWS U.S. Department of Energy Washington, D.C. 20585 NOT MEASUREMENT SENSITIVE DOE-HDBK-3012-2015 i FOREWORD This U.S. Department of Energy (DOE) Handbook, Team Leader’s Good Practices for Readiness Reviews, is approved for use by all DOE Components and their contractors. DOE Order (O) 425.1D, Verification of Readiness to Start up or Restart Nuclear Facilities, requires the conduct of a Readiness Review (RR) when starting or restarting nuclear operations under the conditions specified in the order. The readiness review may be either an Operational Readiness Review (ORR) or a Readiness Assessment (RA). The order specifies when each is required. This Handbook provides good practices for the specified Team Leader for either an ORR or an RA. Throughout this document, the term Readiness Review is to apply to whichever type of RR has been specified by the Startup Authorization Authority (SAA) in the approved Plan of Action (POA) which also designates the RR Team Leader for whom this document is intended to support. DOE Standard (STD) 3006-2010, Planning and Conducting Readiness Reviews, provides approaches and methods approved as acceptable for implementing the requirements of the Order. The Standard will be referenced throughout this Handbook whenever it should be consulted for the particular activity being discussed. The good practices in this document represent the distillation of the experience and lessons learned during numerous RRs contributed by Team Leaders with a wide range of backgrounds. The requirements for performing readiness reviews are contained in DOE O 425.1D and locally approved contractor and DOE procedures. Other than meeting requirements, Team Leaders have much flexibility in accomplishing a successful RR. Many of these flexibilities and good practices are described in this Handbook. Experienced Team Leaders are also a good source of good practices. There are many ways to perform a readiness review correctly. Each review will be conducted at a unique facility and will involve a unique team. Every Team Leader will learn something new during each review. In some cases, these new lessons will conflict with the advice provided in this document. Such lessons learned and all other beneficial comments (recommendations, additions, deletions) and any pertinent data that may improve this document should be submitted to the Department’s Office of Nuclear Facility Safety Programs (AU-32). DOE-HDBK-3012-2015 ii  TABLE OF CONTENTS 1.0  PURPOSE AND SCOPE ................................................................................................... 1  2.0  APPLICABILITY .............................................................................................................. 1  3.0  READINESS REVIEW PROCESS.................................................................................. 1  4.0  PRIOR TO THE READINESS REVIEW ....................................................................... 2  4.1.  Prior to the Pre-Visit ............................................................................................... 2  4.1.1.  Initial Actions.............................................................................................. 2  4.1.2.  Team Selection............................................................................................ 3

4.2.  RR Pre-Visit ............................................................................................................ 5  4.2.1.  Team Leader Actions .................................................................................. 5  4.2.2.  Initial Team Meeting during the Pre-Visit .................................................. 6  4.3.  Final Preparations for the RR ................................................................................. 7  5.0  READINESS REVIEW PROCESS.................................................................................. 8  5.1.  On-Site Team Leader Responsibilities ................................................................... 8  5.2.  Managing the Outcome of the RR .......................................................................... 9  5.3.  Final Team Meeting .............................................................................................. 11  5.3.1.  Team Leader Actions ................................................................................ 11  5.4.  Closeout Meeting .................................................................................................. 12  5.4.1.  Team Leader Actions ................................................................................ 12  6.0  POST READINESS REVIEW ....................................................................................... 12  6.1.  The Final Report ................................................................................................... 12  6.2.  Follow-up Activities ............................................................................................. 13  7.0  REFERENCES ................................................................................................................. 13  Appendix 1. Team Member Responsibilities ........................................................................... 1-1  Appendix 2. Review Coordinator Responsibilities ................................................................. 2-1  Appendix 3. Key Points to Discuss at Initial Contact with Site ............................................. 3-1  Appendix 4. Readiness Review Logistics ................................................................................. 4-1  Appendix 5. Checklist for Readiness Review Team Data Packets ........................................ 5-1  Appendix 6. Transmittal Letter for Data Packets to Team Members .................................. 6-1  Appendix 7. Team Member Deliverables ................................................................................ 7-1  Appendix 8. Pre-Start and Post Start Determination Criteria for Readiness Reviews....... 8-1  Appendix 9. Counterpart Responsibilities ............................................................................... 9-1  Appendix 10. Daily Team Meeting .............................................................................................. 1  FIGURES Figure 1: Readiness Review Process ............................................................................................. 2  DOE-HDBK-3012-2015 iii  ACRONYMNS AU DOE Office of Associate Under Secretary for Environment, Health, Safety and Security CRAD Criteria and Review Approach Document DNFSB Defense Nuclear Facilities Safety Board DOE U.S. Department of Energy IP Implementation Plan POA Plan of Action O Order ORR Operational Readiness Review RA Readiness Assessment RC Review Coordinator

RR Readiness Review STD Standard SA Senior Advisor TL Team Leader SAA Startup Authorization Authorit DOE-HDBK-3012-2015 1  1.0 PURPOSE AND SCOPE This Handbook provides instructions, explanations and examples for the performance of all phases of a Readiness Review (RR) to verify readiness to start or restart nuclear facilities, operations, and activities. Details pertinent to the Team Leader are the focus of this Handbook. Limited information for Team Members, Review Coordinators, and Counterparts is also included. Appendices contain focused subject discussions, checklists, as well as sample forms and correspondence, which are typically used to initiate and perform the RR. 2.0 APPLICABILITY The document is written to be useful to both contractor and U.S. Department of Energy (DOE) RR Teams and Team Leaders. The Handbook is written to include considerations that result when the Team Leader or members travel to a remote location. The specific actions that are the result of traveling to the site will not be required for those individuals who are from the site. This Handbook is written covering all aspects of a full contractor and DOE ORR. Since most RRs will be of a scope significantly less than a full ORR, the information in this handbook should be graded as appropriate for the unique scope of the RR. There are many challenges to leading an RR. The planning, selection of the review team, and development of the Implementation Plan (IP) are the most difficult, as well as the most important, activities for achieving a successful RR. 3.0 READINESS REVIEW PROCESS The RR process begins long before the conduct of the review at the identified site facility, operation, or activity. Once selected, the Team Leader has the responsibility for all aspects of the review. Figure 1 provides the Team Leader an idea of the time line he/she will need to be aware of and goals that may be achieved to ensure a successful RR. DOE-HDBK-3012-2015 2 *The MSA is not required by DOE Order.  However, experience  shows that starting a Readiness Review without an MSA greatly  increases the risk of delays and difficulties with the Readiness  Review and achieving authorization to start nuclear operations. 12 MONTHS  PRIOR Identify  Readiness Review  in Startup  Notification  Report Readiness Review Timeline 1‐Year Period Goals Identify  Team Leader in  Plan‐of‐Action 6 MONTHS  PRIOR Conduct Contractor  Line Management  Self‐Assessment (MSA)* Develop  Readiness Review  Implementation  Plan 3 MONTHS  PRIOR 4‐8 WEEKS  PRIOR Conduct  Contractor  Readiness  Review Resolve Contractor  Readiness Review  Findings 1‐3 WEEKS  PRIOR Issue “Readiness  to Proceed”  Memorandum Resolve  DOE Readiness  Review Findings Receive Permission  from Startup  Authorization  Authority to Commence  Program Work  Conduct DOE Readiness Review Figure 1: Readiness Review Process1 4.0 PRIOR TO THE READINESS REVIEW 4.1. Prior to the Pre-Visit 4.1.1. Initial Actions As soon as the Team Leader is identified in the Plan of Action (POA), the Team Leader should perform the following:  Comply with commitments made in the approved POA.  Identify the Review Coordinator (RC). The RC is the primary administrative assistant to the Team Leader. The RC should have experience in the administrative and planning tasks in preparation and conduct of an RR. Further information on RC responsibilities is provided in Appendix 2.

1 This timeline is what might be expected for a full ORR. RRs of smaller bredth and depth would be accomplished in less time. DOE-HDBK-3012-2015 3  Identify points of contact and support at the facility with whom the Team Leader and RC will interface for continuing support and information throughout the process.  Contact the identified points of contact to discuss details about the upcoming RR and negotiate prospective dates for the pre-visit and RR.  Prepare a draft IP including designation of functional areas and identification of team members. Assign development of draft criteria and review approach documents (CRADs) to the team members. The Team Leader may need to resolve overlap between CRADs. Depending on the size of the RR, the Team Leader may or may not assign him or herself a CRAD.  Send a letter or e-mail, as soon as possible after the above negotiations, formalizing the notification of the RR to the site points of contact (see Appendices 2, 3, 4, and 5).  Advise the RC of dates and agenda for the pre-visit.  Initiate the process of selecting team members to support the RR as soon as possible. 4.1.2. Team Selection This is the most important task for the Team Leader. The Team Leader selects and organizes the RR team. Additional input on team selection can be found in Sections 4.d.(2) and 4.e.(3) of DOE Order (O) 425.1D, Verification of Readiness to Start up or Restart Nuclear Facilities, and Sections 7.5 and 7.6 of, DOE Standard (STD) 3006-2010, Planning and Conducting Readiness Reviews. Information on Team Member responsibilities is provided in Appendices 1 and 7. Guidelines for completing the process follow:  Candidates may be obtained from any independent sources within DOE or from contractors not directly responsible for the oversight or operation of the facility under review. Previous Team Leaders are valuable resources in reviewing the past performance of team members.  Strength of the RR team is critical. Under no circumstances should the Team Leader accept a mediocre team. Team member qualification requirements include: o Technical knowledge of the area assigned for evaluation, including experience working in the technical area; o Knowledge of performance-based assessment processes and methods; and o Knowledge of facility, activity, or operation-specific information.  Travel funding for team members not resident at the location of the RR needs to be agreed upon between the local DOE office hosting the RR and the team member’s home office. Potential team members not resident at the site need to be queried as to their office’s ability/willingness to pay for the travel by the Team Leader. If the potential team DOE-HDBK-3012-2015 4 member’s home office will not pay, the Team Leader needs to work through the RC to ensure the availability of travel funds.  Prospective Team Members should have their management's endorsement that the work is full time for the period of the assessment.  Typically, familiarization with the facility/process is gained during the pre-visit.  Experience has shown that under normal circumstances, no team member should be assigned more than two CRADs and no CRAD should be assigned to more than one team member.  Determine whether any conflict of interest, actual or perceived, exists for any candidate or source of candidates.  No individual directly or recently involved in the management or operation of a facility can participate as a team member in his/her area of responsibility at the facility.

 Concurrent with the above tasks, work with the site point-of-contact to identify counterparts. A counterpart is an individual designated by the site to assist in coordinating the RR team's activities. Each RR team member should have a counterpart for each functional area under evaluation. See Appendix 9 for counterpart responsibilities.  As soon as team members and counterparts have been selected, disseminate a list showing counterparts' telephone, facsimile numbers, and e-mail address, if available, to team members. The list is important for informing team members of their selection and their counterpart, which allows for early contact and preparation.  For most RRs, a Senior Advisor should be included as part of the team. Inclusion of a Senior Advisor on the team should be considered even if not specified in the POA. The Senior Advisor will be an experienced individual with RR experience and objectivity to assist the Team Leader and the team in development of the IP and conduct of the RR. A Senior Advisor plays a particularly important role as an advisor and sounding board for the Team Leader when difficult or uncertain situations arise during the RR. Even the most experienced Team Leader will benefit from the presence of a trusted and experienced Senior Advisor. The inclusion of a Senior Advisor is the decision of the Team Leader if not specified in the POA. It should also be recognized that in some cases the dates for performing RRs, particularly those related to new construction, may become highly erratic and subject to continual change. This creates a dilemma for each member of the team as they attempt to plan other job-related activities and personal leave around the RR dates. Whenever possible, identification of primary and alternate team members for each of the key review areas should be included in the IP. The Team Leader is responsible for any alternate team member receiving the necessary access training for the facility, documentation, and planning information. DOE-HDBK-3012-2015 5 4.2. RR Pre-Visit The pre-visit is a fact-finding and familiarization visit attended by as many team members as possible. If some team members are not able to attend the pre-visit, arrangements for compensatory measures should be made to ensure each assigned individual is prepared to be an effective RR team member. 4.2.1. Team Leader Actions  Brief site management on the mission, the methodology, scope, breadth and depth, categorization of findings, prerequisites, and schedule for the RR, as well as the logistical support required from the site.  Arrange for the facility, operation, or activity management to brief the RR team on safety, security, the organization, the facility, operations, and activity; and the start or restart plan. The briefing should include: o Introductions; o Facility management and organization, its functions, operations and interfaces; o The rules for interaction between team members and site personnel; and o The expected time and process for factual accuracy of the RR report.  Continue the review and editing of the implementation plan with special emphasis on the CRADs. Have team members provide their proposed CRAD changes prior to the end of the pre-visit.  Attend a walkthrough at the facility/process and observe any current operations to acquaint team members with the facility and processes in question.

 Attend any special training or fitting, such as respirator or radiation worker training and fitting, which might be necessary to allow for access to the facility during the RR. Verify that all team members have met any pre-conditions prior to their arrival at the site.  Develop a preliminary listing of Evolutions (to include operations, drills/upsets, maintenance, surveillance and other activities to be observed during the RR) and Interview list, and provide it to the site so they can plan and schedule events and personnel availability. Each team member should provide input prior to completion of the pre-visit. Effort should be made to coordinate the interviews among team members to minimize the impact on site personnel.  Schedule a counterpart meeting. Each team member should have acquainted him/herself with their counterpart and developed inspection requirements as part of their preparation.  Arrange for any relevant program documents to be available to team members. Scheduling periods of free time during the pre-visit to allow this to be accomplished is valuable, particularly if any of the required reading is classified. If the site can establish DOE-HDBK-3012-2015 6 an RR support web server on which documentation will be loaded, ensure each team member has established access to that server.  Determine adequacy of space and support to be provided during the actual RR. Resolve any deficiencies that are identified. 4.2.2. Initial Team Meeting during the Pre-Visit At the first team meeting of the Pre-visit, the Team Leader should:  Emphasize that the RR will be a full time, dedicated assignment, work hours will be long to accommodate the schedule, and weekend work may be required to complete the forms on schedule. Team members from certain DOE offices may insist on having pre- approved Compensatory Time/Overtime in accordance with their office policies. The Team Leader may be asked to identify the number of such hours anticipated to be worked during the RR.  Emphasize that the RR is conducted to verify the facility's readiness to safely conduct operations after contractor facility management has declared readiness and the Startup Authorization Authority (SAA) has directed the Team Leader to start the RR. If the scheduled dates for the RR change, the Team Leader will keep the team members informed of schedule changes and team members will also need to inform the Team Leader of their availability when the schedule changes.  Establish that the Team Leader is responsible for all aspects of the RR.  Define the expected schedule for the RR, including the time frame for submission of all deliverables.  Explain the need for and use of the various forms. Be clear as to the expected content and format: o Assessment forms (Form 1) are used to document the methods and actions taken in the criteria evaluation process. Further information is provided in DOE-STD-3006- 2010. o Deficiency forms (Form 2), if used, document the findings identified during the criteria evaluation process. Further information is provided in DOE-STD-3006-2010. o Be clear that ALL forms for which the team member is responsible are to be completed and accepted by the Team Leader before the team member leaves the site. o Expectations for completing forms should be set. For example, completing forms at the end of the RR rather than daily may degrade the quality of recall and extend the completion of the team report.

 Provide administrative information such as area phone numbers, dining locations, etc., at this meeting. DOE-HDBK-3012-2015 7  Advise team members to keep the Team Leader abreast of all concerns and new developments as they arise to minimize confusion and miscommunication as information is collected.  Advise team that all complaints, concerns, suggestions, and information be passed through the Team Leader as soon as possible, including any problems with counterparts.  Advise team members not to meet with external regulatory or oversight agencies, such as state environmental protection agencies, without the permission of the Team Leader.  Advise team members to strictly adhere to the need-to-know criteria in dealing with classified shapes, components, data, or processes. Team members should immediately identify any difference of opinion that arises with the site on access to information.  Discuss the processes that will be required for classification reviews and any limitations regarding working notes and draft documents.  Explain that no contractor business development will take place onsite. Any such activity will result in removal from the RR.  Discuss that the Defense Nuclear Facilities Safety Board (DNFSB) staff, the Office of Associate Under Secretary for Environment, Health, Safety and Security (AU), and line management representatives may observe all, or some, of the RR. Instruct team members regarding protocol when being observed. The team member should neither exclude nor defer to the observers, who should not be active participants in the conduct of the RR processes. Any concerns resulting from the observations should be discussed with the Team Leader.  Explain the format to be used for completing the final report and how the team members' input will be integrated into the report. 4.3. Final Preparations for the RR  Finalize the RR IP. Further information concerning development of the IP is contained in DOE-STD-3006-2010.  The IP should be forwarded to the local DOE Field Element for distribution as required by DOE O 425.1D.  Establish the length of the RR. Factors to consider include: o The complexity of the facility/process being reviewed; o The number of team members available; o The number of interviews and evolutions to be conducted; and o The number of shift evolutions to be observed. It is important to recognize that the RR should be a smart sample of interviews and activities. It is not necessary that all operations conducted at the facility be observed. DOE-HDBK-3012-2015 8  Establish the RR schedule. Factors to consider include: o If possible, the first day(s) should address administrative reviews and management interviews; o Observation of evolutions should be scheduled. Smart sampling of realistic evolutions carried to their logical conclusion is the goal. Maximum realism with surrogate material is important, where practical; and o Save level of knowledge interviews for last if possible, to aid in evaluating whether observed missteps were a matter of an individual failure or a general problem. The draft schedule should be presented to the facility management and discussed during or shortly after the pre-visit. 5.0 READINESS REVIEW PROCESS The RR may not start until the Team Leader has received the formal direction to start the RR from the Startup Authorization Authority or senior contractor manager. Such formal direction is given after line management has declared that all prerequisites for starting the RR have been met and that readiness to commence nuclear operations has been achieved.

5.1. On-Site Team Leader Responsibilities  Ensure all team members have completed all access requirements of the site.  Conduct the opening meeting at site. This meeting is the "kickoff" meeting for the commencement of the RR. Items to consider for completion during this meeting if not covered during the pre-visit include: o Make introductions; o Allow for a series of briefs by the site explaining the site's management and organization, its functions, operations and interfaces; and o Brief the audience on the RR process and the RR schedule.  Brief site audience and team on the rules for interaction between team members and site personnel. The team: o Will not act as safety monitors or direct operations; o Will refrain from distracting the operators; o Will not violate site rules, cross boundaries without authority, violate control room/area boundaries or in any other way become an initiating event during the RR; and o Except for the case of imminent danger, will strive to be invisible so as to not influence the performance of the facility, operation, or activity. DOE-HDBK-3012-2015 9  Request that all operators be briefed on these rules and in particular that no action should be taken in response to questions or discussion by the team. The team will initiate action, preferable through a supervisor or escort, if an immediate unsafe condition is encountered. o Site personnel should be apprised of the expected time and process for their input for factual accuracy. Team member counterparts should be informed that factual accuracy is a continuing and ongoing process for which they hold major responsibility. Factual accuracy should however, be confined to the factual elements of the write-up and should not be a debate over the conclusions drawn by the team member from those facts. Contractor and DOE Field Element Management should be informed that prompt review of the draft final report is needed.  Hold daily team meetings (usually held in the afternoon) to discuss emerging issues. See Appendix 10 for daily team meeting information.  Meet frequently (daily is recommended) with site managers to brief them on the issues raised by the previous day's review activities. Frequently, senior site management will attend the daily team meeting for first hand input on review progress. The Team Leader discusses potential findings and other issues uncovered during the previous day's activities. This greatly enhances site/team communication and allows the site to better facilitate the validation process.  The following items need to be reiterated at daily team meetings throughout the review: o Encourage the identification of any unclear issues. They still need to be identified so that others may consider them at a later date. o Re-emphasize that team members should focus on that portion of the organization directly responsible for their area.  Insist on validation of facts. RR teams validate with facility and DOE line management and their counterparts facts they collect on a continuous basis. See Appendix 9. o Maintain a running record of the important issues brought out at each daily meeting. Individual team member schedule considerations may raise the possibility of a team member leaving the team prior to the completion of all team activities. If that situation were to occur, it is critical that the Team Leader not permit the departure until all Forms 1 and 2 have been submitted and accepted. If a team member is permitted to leave the site before all deliverables have been provided and accepted, the risk is great that difficulty will be experienced by the Team Leader in completing the final report.

5.2. Managing the Outcome of the RR The task of the RR team is to confirm that readiness to start or restart a facility, operation, or activity has been achieved. A critical task of the Team Leader is to assist and facilitate the team DOE-HDBK-3012-2015 10 in making the correct determination. Every RR team will identify findings, some of which will be judged as requiring resolution prior to start of operations. A critical question for the team is whether the number or the significance of the findings leads to a conclusion that the RR Team should not recommend that operations be commenced when the findings are resolved. Experience indicates that this question may arise at any time during the RR including during the final team meeting. The decision to recommend that operations NOT be commenced represents a significant failure on the part of the organization tasked to achieve readiness and should not be made lightly or without clear basis in fact. The RR Team Leader has a significant challenge to ensure the facts clearly support a decision that readiness has not been achieved to safely start or restart a facility, operation, or activity. A decision of that magnitude should be thoughtful and not made quickly. No single team member should be permitted to inappropriately influence the remainder of the team. The facts that support the decision should be written and factually verified and persuasive to the team as a whole before a decision is final. The team should be required to consider the proposal over night to ensure that the facts still lead to the same conclusion that readiness has not been achieved and that resolution of the findings will not provide adequate confidence for the team to recommend start of operations. In the event that the RR Team determines that readiness has not been achieved and that a recommendation to start operations when the findings are resolved cannot be given, the appropriate path forward should be defined. The RR Team Leader should keep the Startup Authorization Authority or contractor senior manager who directed the RR to start informed of the conditions and apprised of the possible options. The final decision as to the path forward will reside with the Startup Authorization Authority or Contractor Senior Manager. Options that should be considered if a recommendation to start or restart a facility, operation, or activity cannot be made are to either:  Suspend the review and document what areas have been completed satisfactorily; or  Complete the review with a determination that readiness has not been achieved. In both cases, it is understood that line management responsible for achieving readiness was not successful. In all cases, it is important that the team provide a written report of the activities completed, the issues identified, and the recommended path forward. In most cases, it should be possible for the report to reflect areas of the review that were completed and satisfactory with the resolution of identified findings. Those areas that are completed should not require additional review when the review is restarted. The team assembled to complete the review should be of appropriate size to review the areas that remain. In a few cases in the past, the state of readiness was so deficient that the entire review was repeated, including the formation of a new team and the development of a new POA and a new IP. In cases such as that, the written report should identify the significantly deficient status of achieving readiness and recommend that the review be repeated when readiness has been achieved. More is required, under these circumstances,

DOE-HDBK-3012-2015 11 than simply resolving the findings. In these circumstances, the entire process for achieving readiness should be repeated with emphasis on a more successful outcome. 5.3. Final Team Meeting This is the last opportunity to work directly with team members and clarify any questions that have persisted throughout the RR. Final Form 1s and Form 2s, if used, are due prior to this meeting, as is a draft summary evaluation of the assigned objectives for inclusion in the final report. This is the last opportunity to discover and address dissenting opinions and conflicts prior to presenting the RR results to the site. While the Form 1s will be complete, discussions at the final meeting frequently are required to reach agreement on the final categorizations of the findings as to pre-start or post-start or in some cases, observations. Any issues or disagreements should be addressed at this time to avoid any confusion at the closeout meeting. The final decision regarding finding categorization resides with the Team Leader who should strive to reach consensus. The option of a dissenting opinion is always available, but team consensus building efforts should be taken to avoid a disagreement so profound that a dissenting opinion results. 5.3.1. Team Leader Actions  Discuss all findings at the final team meeting. A consensus should be reached by the team on the categorization of each finding as pre-start, post-start, observation, or, if appropriate, deleted. Discussing all findings at this meeting will help bring the team to a unified position on the general status of the facility or process being reviewed. Further information regarding categorization of pre-start and post-start findings is provided in Appendix 8.  Discuss the final conclusion, major issues, and overall themes resulting from the RR and agreed upon at this final meeting. In preparation for the closeout meeting discuss the following items:  The closeout meeting will be kept short. There should be no surprises at the closeout. Facility management and specific counterparts should clearly understand all items discussed at the closeout.  The site management will specify attendance at the final closeout.  All findings are subject to the approval of the Team Leader. Approval of all findings should be complete prior to the final closeout meeting with the site.  The final report schedule and deliverables. DOE-HDBK-3012-2015 12 5.4. Closeout Meeting The closeout meeting, or exit briefing, is the forum by which the Team Leader informs the site management of the major issues identified during the RR. There should be no surprises at this meeting as counterparts and site staff should have been kept well apprised of all issues during the conduct of the RR. 5.4.1. Team Leader Actions  Prepare the exit briefing based on the Form 1s and Form 2s, if used, and conclusions drawn by the RR team.  As a goal, provide a draft copy of the final report to the senior managers.  Prepare the executive summary of the draft report as a handout to everyone attending the briefing.  State that factual accuracy reviews of the final, completed report may delay delivery of the final, signed report.  In some cases, it is appropriate to request selected team members to participate in the closeout briefings. If the Team Leader chooses this option, it is important to ensure the briefing retains a consistent theme and conclusion. 6.0 POST READINESS REVIEW

6.1. The Final Report The final report should be completed with all signatures prior to leaving the site. It may be appropriate to delay the final Team Leader approval signature to permit a final factual accuracy review of the entire report by the site. The Senior Advisor, if applicable, should review and concur with the final report before it is issued. The Team Leader approves the RR report and submits it to the Startup Authorization Authority, or the Senior Contractor Manager who directed the start of the RR with copies to the applicable line managers, program office, operations and/or Field Element, and AU. Ensure each team member has signed the final report indicating his/her concurrence with the conclusions of the RR in his/her area of responsibility or review. If a team member has a dissenting opinion, refer to DOE-STD-3006-2010 for further information. Detailed information for the preparation of the final report is contained in Sections 8.9 and 9.9 of DOE-STD-3006-2010. Ensure that all RR files that may be needed in the future are maintained in one location as determined by the line management and site procedures. DOE-HDBK-3012-2015 13 6.2. Follow-up Activities The completion of the RR and the finalizing of the report do not signify the end of the RR process. Several actions may require the participation of the Team Leader including briefings, interpretations and possible defense of findings, and if requested by line management, review of action plans, and review of finding closure plans. Additional information can be found in Sections 8.12 and 9.12 of DOE-STD-3006-2010. 7.0 REFERENCES DOE O 425.1D Chg 1, Verification of Readiness to Start Up or Restart Nuclear Facilities, April 16, 2010 DOE-STD-3006-2010, Planning and Conducting Readiness Reviews, May 6, 2010 DOE-HDBK-3012-2015 1-1  Appendix 1. Team Member Responsibilities PRIOR TO THE READINESS REVIEW (RR) PRE-VISIT  Each team member should receive and review their data packet. The data packet provides team members with information on the site, their assignments, including deliverables, and responsibilities of the team member.  Team members are to contact the hotel to confirm their reservation and guarantee it with a credit card.  Each team member is responsible for providing clearance information to the Review Coordinator (RC) as needed.  Plan for special training or fittings are required, such as respirator fitting, Radworker II, and Hazardous Waste Operations training. The Team Leader will inform all team members of any special requirements.  Team members are to contact their counterpart during their preparation to obtain information they feel is necessary or to arrange for additional documentation to be readily available onsite.  Team Members should insure that their management understands the RR is a full time job for the duration of the effort. RR PRE-VISIT  Review the RR Implementation Plan (IP) including the CRADs.  Arrive at the site as scheduled, having met any prerequisites for site access provided by the Team Leader.  Receive briefings by the Operations Office/Field Element and/or contractor management on safety, security, the organization, processes, and operations of the facility as well as, specific information concerning the nature of the restart or startup.  Walkthrough the facility or process area to observe any current operations to acquaint team members with the facility and processes in question.

 Complete any special training or fitting, such as respirator training and fitting, which might be necessary to allow for access to the facility during the RR.  Develop shift evolution and interview list(s) to be provided to the site so they can plan and schedule events and personnel availability.  Meet with the counterpart and develop a methodology for completing their assigned criteria. DOE-HDBK-3012-2015 1-2  Expect and plan to be in regular communication with the site-appointed counterparts as they will provide the best source of site-specific information and access available to the team member. If available, gain access to the RR support server upon which the documentation in support of the RR is loaded.  Complete any required reading of documents that cannot be accomplished readily elsewhere, e.g., classified documents.  Provide proposed CRAD changes to the Team Leader before the end of the pre-visit for inclusion in the IP. During the pre-visit, draft CRADs should be shared and discussed with counterparts to ensure accuracy and completeness. READINESS REVIEW PROCESS  Provide the Team Leader with any findings as soon as sufficient information has been gathered to define the finding.  Ensure factual accuracy with facility and DOE line management and the counterparts. It is important that the team exercise the responsibility for drawing conclusions and reaching judgments from facts.  Keep accurate and complete notes of the activities including titles of individuals interviewed as well as the notes of the interview results; title, document number, date, and revision number for all documents reviewed as well as the conclusions or information resulting from the document reviews; and details of all evolutions that are observed. Update their Assessment Forms (Form 1s) and Deficiency Forms (Form 2s), if used, daily to ensure complete record keeping of all inspection activities. Additional information is provided in DOE-STD-3006-2010.  Team Members should be prepared to assist other Team Members as peer reviewers of Form 1s, Form 2s, and other documents. This support greatly enhances the timeliness and quality of the final report.  Identify potential issues even if they are unclear so that others may consider them at a later date. If you begin to have suspicions about something, report the concern and let the site follow up. Additional information is provided in DOE-STD-3006-2010.  Assist the Team Leader in the preparation of these inputs to the final report: o Integrated Safety Management System status; o Lessons Learned; o Adequacy of contract requirements in addition to the summary of results; and o Develop the draft presentation materials for the closeout meeting; and DOE-HDBK-3012-2015 1-3 o Obtain assistance from the RC and Senior Advisor in completion of required forms and summaries. FINAL TEAM MEETING  Be prepared to discuss and justify all findings.  Be prepared to participate in the discussion of the final conclusion, major issues and overall themes resulting from the RR.  In addition, prior to the closeout meeting with the site, all Form 1s and Form 2s, if used, should have been completed, signed, and submitted to the Team Leader. The completion of the forms means that factual accuracy of the individual forms has been confirmed with the counterparts.  All inputs to the final report should be submitted by each team member and accepted by the Team Leader before the closeout meeting and before the team member leaves the site.

 Summary evaluations should also have been completed and submitted to and signed-off by Team Leader prior to the closeout meeting. CLOSEOUT MEETING All team members who remain on site should attend the closeout meeting and be prepared to participate in the exit briefing or answer questions regarding their findings should the Team Leader request their support or participation. If team members will present portions of the exit briefing, they should prepare presentation materials for review and acceptance by the Team Leader prior to the closeout meeting. FINAL REPORT Team members may be called upon to assist the Team Leader in several ways including briefings, finding interpretation, review of action plans, or review of finding closure. The Team Leader will notify team members when such assistance will be needed. DOE-HDBK-3012-2015 2-1  Appendix 2. Review Coordinator Responsibilities PRIOR TO THE PRE-VISIT  After team selection is made, obtain security information as appropriate for the site, address, phone number, fax number, type of clearance, and funding information for all team members.  Verify that all team members' clearance information and all necessary forms for access to the site and facility have been forwarded to the site. Verify this has occurred at least one week prior to the pre-visit.  Make a contact list for the pre-visit RR team. The list should include site contact names, phone numbers, fax numbers, hotel information, site security office number, per diem rates (some of this information will be provided to the team by the Team Leader).  Provide relevant internet and email addresses for the hotels, site, and points of contact.  Reserve a block of rooms for RR pre-visit team members who will travel from outside the area. Send the hotel information to the team. Inform team members to call the hotel and confirm their reservation and to guarantee it with a credit card. Usually these instructions are included in the data packet.  Assist the Team Leader with developing a team member/counterpart chart.  Assist the Team Leader in the preparation of the draft implementation plan.  Prepare a Readiness Review (RR) notification letter and enclosures for the Team Leader.  Send draft RR IP and CRADs to RR pre-visit team members for their review.  Assemble and distribute data packets to team members. The normal makeup of the data packet is provided in Appendix 5. Mail data packets early to ensure team members have their packets at least two weeks prior to the pre-visit. An example transmittal letter is provided in Appendix 6. Some of the items in Appendix 5 may not be available prior to the pre-visit. The key is to provide maximum information to each team member to gain maximum benefit from the pre-visit.  Encourage the site to provide electronic access to information required by the team members. As an alternative, many sites can provide documentation on electronic storage media. Verify compliance with the request for logistic support that was sent (Appendix 4) with the letter of confirmation.  Make sure the team has maps of the site and directions how to get to the site and hotel from the airport. Request maps from the site.  Make contact with site or facility coordinator to introduce and discuss logistical questions. Discuss sigma/security requirements for cleared and uncleared team members DOE-HDBK-3012-2015 2-2

and determine if there are any special procedures to bring in equipment, such as laptop computers, or supplies (e.g., electronic storage media).  Make security clearance and restricted area access arrangements for the RR pre-visit. Complete DOE F 5631.20 (formerly 277's), Request for Visit or Access Approval forms for the pre-visit team. Discuss access requirements with the site point-of-contact before the pre-visit to determine which sigmas are needed. Discuss security procedures for any uncleared team members.  Some team members may use their laptop computers. Determine site access policies and requirements associated with laptop computers, cell phones, smart phones, and other personal electronic equipment. Inform team members of what forms and information are required prior to the site pre-visit. Gather completed forms in advance of the RR to ensure access during the review.  Determine personal training, medical, and qualification requirements that must be met by each team member (i.e. Site access, Rad Worker, HazWoper, mask fit test, etc), identify the requirements to the team members and track completion.  Complete the necessary forms for access to the site for all team members and forward the data to the site.  Verify that all required reference material has been provided to team members four to five days before the team's arrival for the pre-visit and RR. FINAL PREPARATIONS FOR THE RR  After RR dates are confirmed, reserve a block of rooms and send a rooming list to the hotel with names and check-in/check-out dates, if required. Make arrangements for a meeting room at the hotel for the initial team meeting, if necessary.  Prepare security clearance forms for all team members. Complete form DOE F 5631.20, Request for Visit or Access Approval.  Assist in preparing data packets for team members (see Appendix 5) to include: o The hotel meeting room number, if applicable; o Where and when to get badges; o Badge office hours; o Special arrangements for any uncleared team members or personal computers; o What to do in case a team member gets delayed and can't be at the right place at the right time; o If any special training or qualifications are required; and o Where the onsite team room work space is located. DOE-HDBK-3012-2015 2-3  Coordinate with the Team Leader and provide the items listed in Appendix 5.  Make arrangements to mail data packets to team members. If possible, arrange for electronic distribution of current site and facility documents.  Keep the site point-of-contact informed of any team member/schedule changes by updating and faxing the team member assignment chart.  Keep all team members informed of the status of progress and potential or actual changes in RR start dates.  Prepare a package to mail to hotel and to have available at the initial team meeting. The package should include any information and/or documents the Team Leader and team members need for the RR, such as: o The standard format used and examples from previous reviews; o Latest agendas and schedules; electronic storage media with all RR information; o Blank Forms 1 and 2; o Previous RR closeout summary examples; and o The template for the word processing software to be used.  Encourage the site to provide electronic access to information required by the team members.  The development and management of the Assessment Forms and summaries is a prodigious task. RC should be dedicated to administering the forms, e.g., logging in and tracking.

 Consideration should be given to bringing in supplemental editing staff toward the close of the RR. A significant amount of copying is required at the close of the RR for the Exit Briefing and any copies of the draft Final Report to be left at the site. DURING THE READINESS REVIEW During the RR, the RC works closely with the Team Leader providing administrative and logistical support. Specifically:  Hand out latest team member assignments and agendas/schedules and any instructions on writing;  Provide forms and summaries;  Provide administrative and logistical support to team members;  Get to know the site, i.e., locations of library, copy center, fax machine, supplies;  Establish access and gain familiarity with site electronic document processes including intranet access when appropriate; DOE-HDBK-3012-2015 2-4  Find out about site-specific security procedures, i.e., do team member notes need to be reviewed for classification? If so, where?  Check all computers and printers to make sure they are working properly. Arrange for repairs or replacement if necessary.  Load Form 1 and Form 2, if used, files onto all computers or provide on electronic copies if team members will be using lap top computers.  Provide sample Functional Area Summaries to each functional area leader. This helps ensure the inputs for the final report are consistent in content and format.  Attend team meetings and manage the Form 1 and Form 2, if used, tracking system. A suggested procedure to manage the Forms is as follows: o Get a hard copy and electronic copy from each team member or subgroup Team Leader. To maintain configuration control of the report, the RC and technical editor make all electronic changes after the electronic copies have been provided to the RC. o Log in the form(s). o Give a copy to the Team Leader as soon as possible and place a copy in the logbook.  After the Team Leader reviews and comments on the form and it has been edited by the technical editor, return the form to the team member for revision or signature. Revisions and editing are then accomplished between the team member and the technical editor with the Team Leader reviewing only the final edited and signed version submitted by the team member. CLOSEOUT MEETING ACTIONS  Assist the Team Leader in preparing the closeout briefing.  Make arrangements for the site to mail information that was brought with the RR team or collected during the review to the team members. This package should be sent on the day of the closeout meeting.  Ensure that all classification reviews have been completed and appropriately documented on the report and on any team member notes that will leave the site.  Make copies of Form 1s and Form 2s, if used, for dissemination to site management. THE FINAL REPORT  Draft thank-you letters for Team Leader's signature.  Collect, condense, index, and file pertinent RR information to include: o All correspondence related to the RR; DOE-HDBK-3012-2015 2-5 o All final Form 1s and Form 2s, if used, signed by the team member and Team Leader; o The Shift Evolution and Interview list and schedule; o The approved Plan of Action (POA); o The approved IP; o The approved Final Report with all signatures from the team members and Team Leader. DOE-HDBK-3012-2015 3-1  Appendix 3. Key Points to Discuss at Initial Contact with Site  Tentative review schedule;  Pre-visit and Preparation;

 Two to three day site visit for facility familiarization, walkthrough, and presentation to site of inspection process;  Counterparts: Site expected to provide list of counterpart(s) for each area;  Training, qualification and clearance requirements for team members;  Onsite Review Period;  Purpose: Identify findings and observations for the safe startup of facility process(es);  General: Up to 30 team members, approximately a two-week period, though with a graded approach, as few as 10-15 team members may be needed for one week;  Logistic Support: Conference room, workspace, computers, phones, fax, etc. (list to be provided); and  Request point-of-contact at site to support review and to receive formal correspondence. DOE-HDBK-3012-2015 4-1  Appendix 4. Readiness Review Logistics The following list suggests items the Team Leader should consider. The Team Leader should personalize it for each Readiness Review (RR). These items should be discussed and agreed upon during the pre-visit. 1. The team will require workspace for approximately [#] people to review documents, conduct interviews, prepare reports, and conduct meetings. 2. Workspaces should be equipped with networked personal computers for all team members. 3. Access to site electronic document system should be arranged. 4. Workspaces should have phones with contractor and DOE phone books. 5. The team will have ready access to photocopy machines. 6. Point of contact to assist with equipment repair and aid in logistical problems should be available at the start of the review. 7. Access to an authorized derivative classifier and a classified document repository. 8. Access to the following documents in a convenient, centralized location at the site. If possible, direct access to electronic document systems is preferable. The Team Leader may require the team members to review these documents to accomplish qualification for site familiarity, thus they may be required during the pre-visit. The Team Leader should communicate these desires to the site as soon as possible. Note: the following are examples of documents that the Team Leader can request to have available during the RR.  DOE Orders and mandatory standards (Contract list A & B).  General facility schematics.  List of Facility Representatives and Facility Managers.  Hierarchy of documents: Top-level policy and site wide procedures down to operating instructions.  Current listing of titles, numbers, and revisions of manuals/procedures.  Implementation plans submitted to Headquarters for DOE Orders.  Any exemption, exception, deviations, variances, or waivers requested.  DOE and Contractor oversight plans/programs, and self-assessment program manuals. DOE-HDBK-3012-2015 4-2  All self-assessment reports (within the scope of the RR).  Contractor and DOE organization charts with management/supervisor names.  General site map. DOE-HDBK-3012-2015 5-1  Appendix 5. Checklist for Readiness Review Team Data Packets DESCRIPTION Transmittal letter Maps from airport to hotel/site Schedules for RR team Applicable RR Plan of Action Site organization charts Team Member assignment and counterpart matrix Draft IP RR Order 425.1D and DOE-STD-3006-2010 Line Management Self-Assessment (Contractor RR) Contractor RR Report (DOE RR) Site DOE Validation Report (DOE RR) Other Required Reading Documents (e.g. TSRs, DSA, engineering documents) Team Member Deliverables List

Access Information for Site RR data server Blank Forms 1 and 2 (Note: The Objective and Criteria should be filled in on the Forms) DOE-HDBK-3012-2015 6-1  Appendix 6. Transmittal Letter for Data Packets to Team Members DATE: NOTE TO: [Name of Site/Facility] Readiness Review (RR) Team Members FROM: Name -- RR Team Leader SUBJECT: Readiness Review at [site]. Attached is the detailed schedule for the Readiness Review (RR) at [site]. Your badge will be ready for pick up at the Badge Office that is located ____________. You will need access authorization for Sigmas _________. The badge office is open from ______a.m. to ______p.m. The site is providing a team workspace located in ______________ for our use throughout the entire review period. Attached are the functional area assignments; the review methodology for RR team members; a [site] organization chart; team member duties and responsibilities; required reading assignments; Personal training and qualification requirements (Radworker; Hazwoper; respirator fit; etc); list of deliverables, and the assigned Criteria and Review Approach Documents (CRADs). If you have any questions or if I can be of further assistance, please call me at _________, or e-mail me at __________. DOE-HDBK-3012-2015 7-1  Appendix 7. Team Member Deliverables 1. Qualification Summary: due before or during the pre-visit. 2. Security clearance data: due as required by the Review Coordinator. 3. Evolution and Interview list: input due to Team Leader prior to the end of the pre-visit. 4. Completed CRADs. 5. Forms 1 and 2. 6. Any dissenting opinions should be prepared and submitted by the end of the RR. 7. Lessons Learned for inclusion in the report. DOE-HDBK-3012-2015 8-1  Appendix 8. Pre-Start and Post Start Determination Criteria for Readiness Reviews The Team Leader and the Readiness Review (RR) team members may use the following checklist to evaluate if an issue should be corrected prior to startup. A. Determination that a Finding Exists Is the identified condition a nonconformance with a stated requirement that represents either: (1) a systematic failure to establish or implement an adequate program or control; or (2) a significant failure that could result in an unacceptable impact on safety of personnel, the facility, the general public, or the environment during nuclear operations? If the answer is yes, it is a finding. B. Pre-Start and Post-Start Determination 1. Does the loss of operability of the item prevent safe shutdown, or cause the loss of essential monitoring? 2. Does the loss of operability of the item require operator action in less than ten (10) minutes to prevent or mitigate the consequences of events described in the Safety Analysis? 3. Does the loss of operability of the item cause operation outside the Safety Analysis? 4. Does the loss of operability of the item result in a reduction of the margin of safety as described in the Safety Analysis? 5. Does the issue indicate a lack of control which can have a near term impact on the operability or functionality of safety related systems? 6. Does the issue involve a violation or potential violation of worker safety or environmental protection regulatory requirements that pose a significant danger to workers, the public, or of environmental insult or release? If the response to any of the above questions is yes, the item should be considered a pre-start finding. Otherwise, the issue should be considered a post-start finding.

DOE-HDBK-3012-2015 9-1  Appendix 9. Counterpart Responsibilities The role of the counterpart is one of a knowledgeable guide for the reviewer. They are enablers for the review team members. This allows the reviewer to put their time into reviewing instead of sorting out which documents and which people would best answer the question that the reviewer would like to have answered. The role of the counterpart is NOT to convince the reviewer that everything is wonderful. They should ensure that the reviewer is exposed to the appropriate documents, observe the appropriate operations, and interview the appropriate personnel who have the responsibility and the expert knowledge to speak with authority on the area in question. In that light the counterpart should:  Assist the team member in the development or review of the inspection plan to ensure the expectations for the plan will be met.  During the review gather documents, arrange interviews, and if necessary, suggest operations that will allow the team member to arrive at a satisfactory conclusion for the issue or area under discussion.  Keep the chain of command informed of any issues that are being developed. The review team is expecting communications to continue between counterparts and their supervisors.  Suggest to the reviewer, documents or interviews that would correct potential misconceptions that may be developing due to misinformation that has been presented to the reviewer.  Upon completion of the writing of the assessment forms and issue forms, the counterparts will be asked to review those forms for factual accuracy. The conclusions drawn are not subject to review, just the facts that are used to base the conclusions. Active counterparts that quickly provide documents and arrange interviews with appropriate personnel will greatly facilitate satisfactory completion of the review. Inactive counterparts that do not respond to requests in a timely manner or are not available to the reviewer result in the team member finding issues hard to resolve. Also an aggressive counterpart that tries to force his opinion on to the reviewer is not helpful to the team member. Effective counterparts can be the difference between a timely, efficient review and a drawn out, difficult review. DOE-HDBK-3012-2015   Appendix 10. Daily Team Meeting Here are some items for the Team Leader to consider when conducting the daily team meeting:  The team meeting should be an open communication forum between team members and the Team Leader.  Team members brief the Team Leader on the issues uncovered during the day (this is not intended to be a travelogue of the day's events). These briefings should be short and succinct. This only needs to include potential findings and if the team member has any unfulfilled needs for information. It is not necessary for a team member to discuss everything done that day as the Team Leader expects the team member is doing their job.  Issues deemed sufficiently substantive by the Team Leader will be identified as potential findings. Some duplication of effort is to be expected and it can often lead to beneficial cross-talk and fuller understanding of the issues. Issues raised may be assigned to a different team member if it is felt that better coverage will result. The Team Leader resolves any disputes between functional areas or sub-teams that occur.  Invite the site to have observers (not a participant) at the team meeting to hear firsthand the team issues and concerns. Remind team members that their counterparts should be kept aware of the emerging issues. Site observers are very beneficial in keeping the communications open and professional.  Site observers should not be allowed to participate in the team meetings as that is not the forum for discussion of issues with the site.  The Team Leader should ensure discussions are professional and team members are encouraged to participate but not dominate the discussion. Most questioning will come from the Team Leader or Senior Advisor.