DOE-EM-STD-5505-96, Operations Assessments

TS NOT MEASUREMENT SENSITIVE DOE-EM-STD-5505-96 May 1996 DOE LIMITED STANDARD OPERATIONS ASSESSMENTS U.S. Department of Energy AREA OPER Washington, D.C. 20585 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. WELCOME This Portable Document Format (PDF) file contains bookmarks, thumbnails, and hyperlinks to help you navigate through the document. All items listed in the Contents are linked to the corresponding sections. In addition, if you click on a section heading while you are reading the text, you will return to the Contents. Click on the DOE seal below to move to the Contents page. This document has been reproduced directly from the best available copy. Available to DOE and DOE contractors from the Office of Scientific and Technical Information, P.O. Box 62, Oak Ridge, TN 37831; (423) 576-8401. Available to the public from the U.S. Department of Commerce, Technology Administration, National Technical Information Service, Springfield, VA 22161; (703) 487-4650. Order No. DE96009462 DOE-EM-STD-5505-96 iii FOREWORD 1. This Department of Energy limited standard is approved for use by all DOE-EM field components. 2. Beneficial comments (recommendations, additions, deletions) and any pertinent data that may improve this document should be sent to the office of Nuclear Safety Policy (EH-31), U.S. Department of Energy, Washington, D.C. 20585, by letter or by sending the self- addressed Standardization Document Improvement Proposal (DOE F 1300.3) appearing at the end of this document. 3. This Department of Energy limited standard was prepared by the Environmental Management Office of Safety and Health (EM-4) with the assistance of Digital Systems Research, Inc. DOE-EM-STD-5505-96 iv Intentionally Blank CONTENTS Section Page DOE-EM-STD-5505-96 v 1. INTRODUCTION. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2. DEFINITIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.1 Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.2 Concern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.3 Finding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. USING THIS LIMITED TECHNICAL STANDARD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 4. OPERATIONS ASSESSMENT PROGRAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 4.1 Operations Assessment Program Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 4.2 Operations Assessment Program Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 4.2.1 Operations assessments conducted in accordance with a documented schedule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 4.2.2 Documentation of programmatic breakdowns or widespread problems (concerns) supported by individual items that do not meet requirements (findings) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 4.2.3 Communication of concerns and findings to contractor management . . . . . 6

4.2.4 Follow-up to verify concerns are corrected . . . . . . . . . . . . . . . . . . . . . . . . 6 4.2.5 Utilization of assessment results as a factor in determining contractor monetary incentives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 5. PERFORMING AN OPERATIONS ASSESSMENT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 5.1 Operations Assessment Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 5.2 Assessment Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 5.2.1 Resource Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 CONTENTS Section Page DOE-EM-STD-5505-96 vi 5.3 Assessment Team Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 5.3.1 Team Leader . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 5.3.2 Team Members . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 5.4. Assessment Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 5.4.1 Pre-assessment Information Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 5.4.1.1 Graded-Approach Document . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 5.4.1.2 Activity-Specific Information . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 5.4.1.3 Prior Assessments, Reports, etc. . . . . . . . . . . . . . . . . . . . . . . . . 11 5.4.2 Assessment Plan Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 5.4.2.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 5.4.2.2 Content and Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 5.5 Activity Briefing and Tour . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 5.6 Assessment Schedule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 5.7 Assessment In-brief . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 5.8 Assessment Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 5.8.1 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 5.8.1.1 What to Observe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 5.8.1.2 How to Observe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 5.8.1.3 Where and When to Observe . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5.8.1.4 Stop Work Authority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5.8.2 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5.8.2.1 How to Prepare for Interviews . . . . . . . . . . . . . . . . . . . . . . . . . 18 5.8.2.2 Conducting Interviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 5.8.2.3 Where and When to Conduct Interviews . . . . . . . . . . . . . . . . . . 20 5.8.3 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 5.8.3.1 What Documents to Review . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

CONTENTS Section Page DOE-EM-STD-5505-96 vii 5.8.3.2 Reviewing Documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 5.8.3.3 When and Where to Review Documents . . . . . . . . . . . . . . . . . . 22 5.8.4. Pursuing leads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 5.9 Daily Team Meetings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 5.9.1 Daily Meetings with Activity Management . . . . . . . . . . . . . . . . . . . . . . . . 27 5.10 Findings and Concerns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 5.11 Developing Concerns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5.12 Report Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5.12.1 Report Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 5.12.2 Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 5.13 Assessment Out-brief . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 6. GUIDELINES FOR ASSESSING OPERATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 6.1 Assessing Operations Organization and Administration (OPS O&A) . . . . . . . . . . . 31 6.1.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 6.1.2 Preparing to Assess OPS O&A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 6.1.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 6.1.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 6.1.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 6.2 Assessing Shift Routines and Operating Practices . . . . . . . . . . . . . . . . . . . . . . . . . . 36 6.2.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 6.2.1.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 6.2.2 Preparing to Assess Shift Routines and Operating Practices . . . . . . . . . . . 37 6.2.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 6.2.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 6.2.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 CONTENTS Section Page DOE-EM-STD-5505-96 viii 6.3 Assessing Control Area Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 6.3.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 6.3.1.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 6.3.2 Preparing to Assess Control Area Activities . . . . . . . . . . . . . . . . . . . . . . . 44 6.3.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 6.3.4. Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

6.3.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 6.4 Assessing Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 6.4.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 6.4.1.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 6.4.2 Preparing to Assess Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 6.4.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 6.4.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 6.4.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 6.5 Assessing Control of On-Shift Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 6.5.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 6.5.2 Preparing to Assess On-Shift Training . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 6.5.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 6.5.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 6.5.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 6.6 Assessing Investigation of Abnormal Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 6.6.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 6.6.2 Preparing to Assess Investigation of Abnormal Events . . . . . . . . . . . . . . . 58 6.6.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 6.6.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 6.6.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 CONTENTS Section Page DOE-EM-STD-5505-96 ix 6.7 Assessing Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 6.7.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 6.7.2 Preparing to Assess Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 6.7.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 6.7.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 6.7.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 6.8 Assessing Control of Equipment and System Status . . . . . . . . . . . . . . . . . . . . . . . . 65 6.8.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 6.8.1.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 6.8.2 Preparing to Assess Control of Equipment and System Status . . . . . . . . . 65 6.8.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

6.8.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 6.8.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 6.9 Assessing Lockouts and Tagouts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 6.9.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 6.9.1.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 6.9.2 Preparing to Assess Lockouts and Tagouts . . . . . . . . . . . . . . . . . . . . . . . 71 6.9.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 6.9.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 6.9.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 6.10 Assessing Independent Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 6.10.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 6.10.1.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 6.10.2 Preparing to Assess Independent Verification . . . . . . . . . . . . . . . . . . . . . 77 6.10.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 6.10.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 CONTENTS Section Page DOE-EM-STD-5505-96 x 6.10.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 6.11 Assessing Logkeeping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 6.11.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 6.11.1.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 6.11.2 Preparing to Assess Logkeeping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 6.11.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 6.11.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 6.11.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 6.12 Assessing Operations Turnover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 6.12.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 6.12.1.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 6.12.2 Preparing to Assess Operations Turnover . . . . . . . . . . . . . . . . . . . . . . . . 86 6.12.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 6.12.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 6.12.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 6.13 Assessing Operations Aspects of Facility Chemistry and Unique Processes. . . . . . 90

6.13.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 6.13.1.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 6.13.2 Preparing to Assess Operations Aspects of Facility Chemistry and Unique Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 6.13.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 6.13.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 6.13.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 6.14 Assessing Required Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 6.14.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 6.14.2 Preparing to Assess Required Reading . . . . . . . . . . . . . . . . . . . . . . . . . . 96 CONTENTS Section Page DOE-EM-STD-5505-96 xi 6.14.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 6.14.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 6.14.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 6.15 Assessing Timely Orders to Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 6.15.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 6.15.2 Preparing to Assess Timely Orders to Operators . . . . . . . . . . . . . . . . . 100 6.15.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 6.15.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 6.15.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 6.16 Assessing Operations Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 6.16.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 6.16.2 Preparing to Assess Operations Procedures. . . . . . . . . . . . . . . . . . . . . . 104 6.16.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.16.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 6.16.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 6.17 Assessing Operator Aid Postings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6.17.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6.17.2 Preparing to Assess Operator Aid Postings . . . . . . . . . . . . . . . . . . . . . 111 6.17.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 6.17.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 6.17.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

6.18 Assessing Equipment and Piping Labeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 6.18.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 6.18.2 Preparing to Assess Equipment and Piping Labeling . . . . . . . . . . . . . . . 114 6.18.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 6.18.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 CONTENTS Section Page DOE-EM-STD-5505-96 xii 6.18.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 6.19 Assessing Operations-Related Radiological Control . . . . . . . . . . . . . . . . . . . . . . 117 6.19.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 6.19.2 Preparing to Assess Operations-Related Radiological Control . . . . . . . 118 6.19.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 6.19.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 6.19.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 6.20 Assessing Operations-Related Training and Qualification . . . . . . . . . . . . . . . . . . 125 6.20.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 6.20.2 Preparing to Assess Operations-Related Training and Qualification . . . 125 6.20.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 6.20.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 6.20.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 6.21 Assessing Drills. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 6.21.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 6.21.2 Preparing to Assess Drills. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 6.21.3 Observation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 6.21.4 Interview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 6.21.5 Document Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 7. APPENDIXES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Appendix A. Sample Operations Assessment Program Schedule . . . . . . . . . . . . . . . . 135 Appendix B. Operations Assessment Flow Diagram . . . . . . . . . . . . . . . . . . . . . . . . . 136 Appendix C. Sample Operations Assessment Preparation Checklist . . . . . . . . . . . . . . 137 Appendix D. Sample Graded-Approach Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Appendix E. Sample Operations Assessment Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 Appendix F. Sample Schedule for an Operations Assessment . . . . . . . . . . . . . . . . . . . 144 CONTENTS Section Page DOE-EM-STD-5505-96 xiii

Appendix G. Sample Operations Assessment Report . . . . . . . . . . . . . . . . . . . . . . . . . 145 8. INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 DOE-EM-STD-5505-96 xiv Intentionally Blank DOE-EM-STD-5505-96 1 1. INTRODUCTION Conduct of operations includes those attitudes, processes, and precautions taken in the interest of safety. The exact features of a system of conduct of operations may be different at different activities. The common feature is a formality of operations which will vary in form and degree depending on the conditions and hazards of the activity. The most intensive application of conduct of operations would be found at the most hazardous activities subject to repetitive types of evolutions. Operational formality is a structured and systematic way of performing work. It is not simply a listing of functional areas, but rather a mind set, or way of doing business. A comprehensive program of operational formality should provide detailed guidance for performing essential elements of operations, such as: shift routines, communications, control of equipment, lockouts and tagouts, logkeeping, operations procedures, and equipment labeling. DOE Directives which require implementation of conduct of operations throughout the DOE complex are based on well-developed industrial operations practices. The guidelines in those DOE Directives were written to be flexible so that conduct of operations could be implemented at any Environmental Management (EM) activity. To ensure that DOE contractors have appropriately implemented conduct of operations, contractor operations must be assessed periodically. The purpose of this Department of Energy (DOE) limited technical standard is to (1) provide DOE Field Element assessors with a guide for conducting operations assessments, and (2) provide DOE Field Element managers with the criteria of the EM Operations Assessment Program. DOE-EM-STD-5505-96 2 This limited technical standard does not establish any new requirements, but rather incorporates and supersedes the existing criteria from: � Assistant Secretary of Energy for Environmental Restoration and Waste Management “Operations Assessment of Environmental Restoration and Waste Management (EM) Activities” memorandum of May 19, 1992; � Assistant Secretary of Energy for Environmental Restoration and Waste Management “Implementation of the Guidelines for the Conduct of Operations at DOE Facilities” memorandum of July 20, 1992; � Assistant Secretary of Energy for Environmental Restoration and Waste Management “Conduct of Operations at EM Support Services Facilities” memorandum of July 20, 1993; and � Assistant Secretary of Energy for Environmental Management “Operations Assessment Program Reviews” memorandum of July 7, 1994. This limited standard also updates information contained in the Operations Assessment Field Handbook dated June 24, 1994. Therefore, this limited standard supersedes the above noted memoranda and the Operations Assessment Field Handbook which can be discarded at this time. 2. DEFINITIONS 2.1 Activity. Any operational process, system, structure, equipment, or group that fulfills a programmatic purpose. Examples include, but are not limited to, storage areas, radioactive waste disposal and processing systems, burial grounds, environmental restoration projects, tank farms,

characterization and decontamination projects, and analytical laboratories. 2.2 Concern. A determination of a programmatic breakdown or widespread problem supported by one or more findings. DOE-EM-STD-5505-96 3 2.3 Finding. An individual item which does not meet a requirement. 3. USING THIS LIMITED TECHNICAL STANDARD The limited technical standard is divided into sections that describe: � The EM Operations Assessment Program (Section 4), � The overall operations assessment process including the application of assessment techniques, and how to conduct and report the results of an assessment (Section 5), � The process and techniques to be used to assess each element of conduct of operations, operations-related radiological control, activity drills, and operations-related training and qualification (Section 6). Section 4 which provides the criteria of the EM Operations Assessment Program is intended to be used by DOE Field Element managers who are responsible for EM activities. These managers should also be familiar with section 5 to ensure that personnel are applying effective techniques when planning and conducting operations assessments. Sections 5 and 6 are intended to be used as guidance when planning, conducting, and reporting operations assessments at EM funded activities. This limited technical standard provides a set of tools for field assessors conducting operations assessments. Although Sections 6.1 to 6.21 provide examples of how to assess specific areas, the general techniques of operations assessments (Section 5) may be applied to other areas of health and safety (e.g. fire protection, criticality safety , quality assurance, occupational safety, etc.). 4. OPERATIONS ASSESSMENT PROGRAM 4.1 Operations Assessment Program Objective. The objective of the EM Operations Assessment Program is to provide for consistent and standard assessments of activities, in order DOE-EM-STD-5505-96 4 to: � Establish responsibility and accountability for assessment of contractor operations and for corrective action as required; � Ensure that hazardous facilities and activities are competently operated in a manner providing proper assurance of worker and equipment safety, and protection of the environment; � Enable management to determine the operational condition of all activities on a comparable basis; � Provide the basis for management actions to improve activity operations; � Provide an objective methodology to track performance of an activity in conduct of operations; and � Provide an objective basis for contractor monetary incentive determination and for assignment of penalties, when appropriate. 4.2 Operations Assessment Program Criteria. DOE Field Element managers are responsible for carrying out the EM Operations Assessment Program which shall include: � Operations assessments conducted in accordance with a documented schedule � Documentation of programmatic breakdowns or widespread problems (concerns) supported by individual items that do not meet requirements (findings) � Communication of concerns and findings to contractor management � Follow-up to verify concerns are corrected � Utilization of operations assessment results as a factor in determining contractor monetary incentives 4.2.1 Operations assessments conducted in accordance with a documented schedule. Field Element managers shall ensure full operations assessments are conducted at least once every two

years and partial operations assessments are conducted at least once every six months for each EM activity. A full operations assessment shall cover all applicable elements of conduct of to: DOE-EM-STD-5505-96 Establish responsibility anaccountality for assessment of contractor operations and for corrective action as required; Ensure that hazardous faites and activities are competently operated in a manner providing proper assurance of worker and equipment safety, and protection of the environment; Enable management to determine the operational condition of all activities on a comparable basis; Provide the basis for management actions to improve activity operations; Provide an objective methodology to track performance of an activity in conduct of Operations; and Provide an objective basis for contractor monetary incentive determination and for assignment of penalties, when appropriate. 4.2 Operations Assessment Program Criteo@E Field Element managers are responsible for carrying out the EM Operations Assessment Program which shall include: Operations assessments conducted in accordance with a documented schedule Documentation of programmatic breakdowns or widespread problems (concerns) supported by individual items that do not meet requirements (findings) Communication of concerns and findings to contractor management Follow-up to verify concerns are corrected Utilization of operations assessment results as a factor in determining contractor monetary incentives 4.2.1 Operations assessments conducted in accordance with a documented séleddule Element managers shall ensure full operations assessments are conducted at least once every two years and partial operations assessments are conducted at least once every six months for each EM activity. A full operations assessment shall cover all applicable elements of conduct of DOE-EM-STD-5505-96 5 operations, operations-related radiological control, activity drills, and operations-related training and qualification. A partial operations assessment shall cover one or more applicable elements of conduct of operations, operations-related radiological control, activity drills, or operations-related training and qualification. Since other DOE Directives require assessments of areas besides conduct of operations, operations assessments may be performed in conjunction with these other assessments. Field Element managers shall maintain a documented operations assessment schedule which covers a two year period. Field Element managers should update the schedules semiannually and provide copies of the schedules to DOE Headquarters Program Managers and the Office of Safety and Health (EM-4). The sample operations assessment schedule in Appendix A illustrates an acceptable approach to scheduling operations assessments at the required periodicity. Scheduling operations assessments at the wastewater treatment plant and hazardous waste storage area is rather straightforward because operations always take place at the same physical location. However, scheduling operations assessments of the decontamination & decommissioning activity and pilot well monitoring activity are more challenging because these operations may occur at various locations across the site over the course of a year. By treating the decontamination & decommissioning activity or the pilot well monitoring activity as a discrete operational process that fulfills a

programmatic purpose, operations assessments may be scheduled on the respective activities regardless of the location of their operations. In this manner, the intent of the operations assessment program will be fulfilled. Field Element managers shall also ensure that the assessment teams are conducting operations assessments through the use of observations, interviews, and document reviews. Sections 5 and 6 describe how to perform an operations assessment and provide guidelines for assessing the elements of conduct of operations, operations-related radiological control, drills, and operations- DOE-EM-STD-5505-96 6 related training and qualification, respectively. 4.2.2 Documentation of programmatic breakdowns or widespread problems (concerns) supported by individual items that do not meet requirements (findings). Field Element managers shall ensure that the concerns resulting from operations assessments are documented by the assessment team and supported by one or more findings. Identification of concerns will require that assessment teams go beyond merely determining deviations from requirements to understanding the status of the programs. Therefore, assessment teams should aggressively evaluate the findings for underlying concerns. The process of developing concerns is described in section 5.11. 4.2.3 Communication of concerns and findings to contractor management. Field Element managers shall communicate the concerns and findings to contractor management. The Field Element managers should ensure that the contractor receives the results of the assessments in a timely manner so that prompt corrective action may be taken. Field Element managers may deliver the results of an assessment to contractor management either verbally, in writing, or both. 4.2.4 Follow-up to verify concerns are corrected. After completion of an operations assessment, Field Element managers shall follow up to verify concerns are corrected. Field Element managers may verify that concerns are effectively corrected during subsequent assessments. 4.2.5 Utilization of assessment results as a factor in determining contractor monetary incentives. Field Element managers shall utilize the results of operations assessments as a factor in determining contractor monetary incentives. When determining contractor monetary incentives, Field Element managers should consider the severity of concerns, completion of corrective actions, and overall contractor progress. DOE-EM-STD-5505-96 7 5. PERFORMING AN OPERATIONS ASSESSMENT 5.1 Operations Assessment Overview. An operations assessment analyzes and evaluates operations through the use of observations, interviews, and document reviews. The methodology used for this type of assessment is characterized by the following general process: � Assessment planning � Observations of routine operations, maintenance, special activities, and activity drills � Interviews with operations, maintenance, training, and radiological control organization personnel � Review of activity policies and procedures, logs, round/tour inspection sheets, radiological control records, etc. � Comparing the information gathered by the techniques above with the requirements contained in activity procedures, policies, and applicable DOE Directives to determine where deviations from requirements appear to exist � Further investigation using additional observations, interviews, and document reviews to

clarify these deviations from requirements and determine if programmatic breakdowns or widespread problems exist The operations assessment process does not rely on formulating checklists to ensure that every aspect of activity operations is evaluated against applicable requirements; rather, it provides a methodology to sample a wide variety of operations to develop a picture of how safely and efficiently the activity is being operated. A flow diagram that depicts the process of developing findings and concerns is contained in Appendix B. 5.2 Assessment Preparation. Early and consistent communications between the Operations Office and the activity to be assessed are vital to a successful assessment. The activity to be assessed should designate a point of contact to coordinate with the Operations Office in preparing for the assessment. A checklist that will aid the Operations Office when making DOE-EM-STD-5505-96 8 arrangements for an assessment is contained in Appendix C. 5.2.1 Resource Requirements. There are specific resource requirements necessary to support an assessment which should be coordinated through the site point of contact. These resources include a conference room and reference material. The conference room should provide enough space for scheduled interviews, document reviews, and daily team meetings. If possible, the conference room should be within walking distance from the activity being assessed to facilitate the daily assessment team meetings and debriefs. Upon arrival at the activity, the assessment team should verify that all support documentation requested for the assessment is available and staged for the assessment team's use. If not readily available, the location of the required documents should be determined. 5.3 Assessment Team Composition. Early identification of the assessment team leader and assessment team coordinator is important. The coordinator should arrange site-specific training (if necessary), take care of security requirements, and handle other assessment preparations. Assessment team members should be selected based on their technical ability and expertise in specific functional areas of the activity to be assessed, as well as assessment experience. Assessment teams should be comprised of personnel such as facility representatives, project managers, and technical experts. 5.3.1 Team Leader. The operations assessment team leader has responsibility for the overall performance of the assessment team. The team leader also serves as the spokesperson for any communication between the assessment team and management. During the preparation for and performance of an assessment, the team leader is the focal point for communicating observations, findings, and concerns to management. Assessment team leaders should be federal employees. The team leader must be capable of representing DOE to senior contractor management in a creditable way. Each field office should determine the training and DOE-EM-STD-5505-96 9 qualification required for team leaders. Prior to an assessment, the team leader should identify the members of the assessment team and prepare an assessment plan. Assessment plan development is discussed in section 5.4. 5.3.2 Team Members. The entire assessment team should be identified in advance of an assessment to allow adequate preparation and to enable the team to make the most efficient use of

time spent on the site. Conduct of operations assessable areas may be logically grouped and assigned to team members. For example, one team member may be responsible for assessing Shift Routines and Operating Practices, Control Area Activities, Operations Turnover, and Logkeeping, while another team member may focus on Lockouts and Tagouts, Independent Verification, and Control of Equipment and System Status. Team members should inform the team leader of the day-to-day assessment activities and of any difficulties in performing the assessment. Each field office should determine the training and qualification required for Team Members. 5.4. Assessment Planning. Research and planning prior to each assessment improves efficiency and produces the most effective use of limited assessment time and resources. Preparation prior to the visit will identify potential performance or technical problems, and it will also result in an assessment plan focused on the most important areas of activity operations. Assessment planning involves two steps: a pre-assessment information review and the development of an assessment plan. 5.4.1 Pre-assessment Information Review. The documents to be reviewed prior to the start of the assessment can be categorized generally as either documents that describe how the activity is designed and operated or documents that form the basis for DOE acceptance of operation. The documents listed in sections 5.4.1.2 and 5.4.1.3 may not be all inclusive. Assessors must decide which documents should be reviewed during assessment planning based on the scope of the assessment and the operations performed by the activity. DOE-EM-STD-5505-96 10 5.4.1.1 Graded-Approach Document. A good way to start the information review process is to review the activity-specific graded-approach document (plan, matrix, etc.) for conduct of operations. The graded-approach document is approved by DOE and is used by activity management to document conformance with requirements. The graded-approach document indicates whether a specific guideline applies to an activity, tells where and how each of the elements of conduct of operations is applied within the contractor's existing policies and procedures, and identifies any deviations or exemptions from the guidelines. In other words, an approved graded-approach document is an agreement between DOE and the contractor that defines the basis for activity operations and provides the requirements to which the activity will be assessed. Appendix D contains a sample graded-approach matrix. During the pre-assessment information review, assessors should review the graded-approach document for conduct of operations and determine which guidelines apply to the activity and which activity policies and procedures implement these requirements. Assessors should then review these policies and procedures to familiarize themselves with how the guidelines are applied at the activity. Assessors will normally evaluate operations against activity-specific procedures. If there are no activity procedures, assessors should use site procedures. If site procedures are not available, the assessment should be based directly on the guidelines of applicable Directives and Regulations related to conduct of operations. Also, if there is no approved graded-approach document, then assessors should evaluate operations against all applicable guidelines for conduct of operations.

5.4.1.2 Activity-Specific Information. In planning for an assessment, it is important that the assessment team be familiar with the activity management structure, the functions performed at the activity, the processes and hardware, and the importance and function of critical systems. Typically, the following documents provide this information: � Safety Analysis Reports DOE-EM-STD-5505-96 11 � Key administrative procedures � Organizational charts � Operational Safety Requirements/Technical Safety Requirements � Activity layout diagrams � System drawings - Wiring Diagrams - Schematics -Piping & Instrument Drawings � Lists of activity maintenance requests � Requests for Action � Order Compliance Schedule Agreements � Activity mission statement � Basis for Interim Operations � Health and Safety Plans � Operations Procedures � Quality Assurance Plan � Emergency Plan 5.4.1.3 Prior Assessments, Reports, etc. Of equal importance to understanding activity operations is the historical performance of the activity. Emphasis should be placed on the identification of repetitive or chronic performance problems, events, equipment failures, or potential or previous programmatic deficiencies. Documents typically containing historical performance information include the following: � Occurrence reports � Performance indicators and trends � Self-assessment reports � External assessment reports � Safety issues previously identified DOE-EM-STD-5505-96 @ Key administrative procedures @ Organizational charts ® Operational Safety Requirements/Technical Safety Requirements e Activity layout diagrams @ System drawings - Wiring Diagrams - Schematics -Piping & Instrument Drawings e Lists of activity maintenance requests e Requests for Action @ Order Compliance Schedule Agreements e Activity mission statement e Basis for Interim Operations @ Health and Safety Plans @ Operations Procedures ® Quality Assurance Plan @ Emergency Plan 5.4.1.3 Prior Assessments, Reports, 4¢ equal importance to understanding activity operations is the historical performance of the activity. Emphasis should be placed on the identification of repetitive or chronic performance problems, events, equipment failures, or potential or previous programmatic deficiencies. Documents typically containing historical performance information include the following: ® Occurrence reports e@ Performance indicators and trends @ Self-assessment reports e External assessment reports e Safety issues previously identified 11 DOE-EM-STD-5505-96 12 � Technical Safety Appraisals � Tiger Team assessment reports � Award fee appraisals By reviewing the above documents the assessment team can focus the assessment plan on the areas of interest at the activity and identify the specific documents to review, personnel to interview, and evolutions to observe. 5.4.2 Assessment Plan Development. Since an assessment is a "sampling" exercise, assessment team members must plan how to start the assessment and target key areas to look at, bearing in mind that following leads or observations will take up a large portion of assessment time. In preparing for an assessment, team members develop an assessment plan. The development of this plan is an individualized process in that the formality of the plan will vary from team to team, and some assessors will need to develop more detailed plans than others. Depending on the level of knowledge, skill, and style of the assessor, varying degrees of planning

are necessary. It is important that leads developed during the assessment, not the plan, drive the conduct of the assessment. This plan functions primarily as a list of starting points, not as a checklist, and must be flexible enough to respond to unique or non-routine operations at the site. 5.4.2.1 Approach. The assessment team should develop a plan that lists each area that has been chosen by the field element manager and the approach to be used for each area. The general process for developing the plan involves three steps: (1) The team leader should assign the chosen assessment areas to the assessment team members; (2) Assessors should identify the initial observations, interviews, and document reviews necessary to assess the guidelines and concepts for their assigned element of conduct of operations, operations-related radiological control, activity drills and operations-related training and qualification; and (3) Specific DOE-EM-STD-5505-96 13 interview questions may be developed by assessors for each area based on the level of knowledge, skill, and style of the assessor. 5.4.2.2 Content and Format. Appendix E contains a sample assessment plan. The assessment plan is detailed and establishes the objectives, criteria, observations, interviews, and document reviews that will be conducted to begin the assessment. Furthermore, a great deal of time and effort should not be expended in developing the plan. Normally, an adequate assessment plan can be developed in 30 minutes to two hours. 5.5 Activity Briefing and Tour. Introductory and safety briefs from the DOE Facility Representative and activity management should be required by the assessment team. As a minimum, these briefs should cover the graded-approach document for conduct of operations, a description of restricted or hazardous areas, and the status of any ongoing operations. Arranging a brief tour of the activity for those team members unfamiliar with the activity can also be beneficial. In addition to familiarizing the team members with the general layout of the activity, the tour provides orientation for security access, radiological protection requirements, and communications procedures. When possible, the activity tour should occur prior to the beginning of the assessment to allow for optimum use of time while actually conducting the assessment. 5.6 Assessment Schedule. The final step in the assessment preparation phase is the development of an assessment schedule. This schedule is developed by the assessment team, after the assessment plan has been completed, to inform activity management of the key interviews, observations, and drills that the team requires to conduct the assessment. The activity plan of the day and plan of the week are useful tools in developing a schedule for observable operational and maintenance-related events and activities. This schedule of key events should indicate the time of the interview or observation, the person being interviewed or event being observed, and the team member requesting the interview or observation. The DOE-EM-STD-5505-96 14 schedule should be provided to activity management at the in-brief for approval and is subject to change and revision as the assessment progresses. A sample assessment schedule is provided in Appendix F. 5.7 Assessment In-brief. A formal activity in-brief by the DOE Facility Representative, Activity Manager, and team leader should start each assessment. All assessment team members,

contractor management, and available activity employees should attend this meeting. The meeting should be lead by the team leader and generally performs the following functions: � Provides an opportunity for contractor management to describe the activity, operating status, and programs related to the assessment. The status of any formal investigations and corrective actions for occurrences should also be described. � Allows the team leader to describe the overall objective of the assessment and the concept that the assessment is a snapshot of the activity operations and procedures in place. As a result, draft procedures and other planned activities not currently in place will not be assessed or commented on in the assessment report. � Allows the team leader to outline the assessment plan and schedule of observations, interviews, and document reviews. � Allows the team leader to emphasize the working relationship between team members and activity personnel. Specifically, team members are not part of the activity management, and as such, team members should not direct any actions of the activity. � Allows the team leader to address stop work authority in the context of the Operations Office policy. � Allows team members to be introduced to activity personnel and functional area counterparts. The ten to fifteen minutes spent in an entrance meeting can save many hours of work and prevent misunderstandings between assessment team members and activity personnel. DOE-EM-STD-5505-96 15 5.8 Assessment Techniques In order to identify and validate deviations from requirements and determine if programmatic breakdowns or widespread problems exist, an assessor must develop leads through the use of observations, interviews, and document reviews and pursue leads to obtain more detailed information. The use of these techniques is described in the following sections. 5.8.1 Observation. Observation is usually the most effective technique used during an operations assessment. Observing activities as they are performed provides information directly related to the effectiveness of operations. Observations should be combined with interviews and document reviews to paint an accurate picture of activity operations. 5.8.1.1 What to Observe. In planning for the assessment, determine what initial observations should be conducted for each area of responsibility. Some activities that can be observed include: � Management control, communication, and command; � Personnel using procedures; � Personnel following rules, obeying signs, using safety equipment; � Operators performing inspection tours; � Control area activities including response to indications, access control, monitoring of control panels, control of routine operations and special activities; � Testing of emergency communication systems and the use of communications equipment such as radios; � On-shift training during routine operations, special activities, and drills; � Authorization, placement, verification, and removal of lockouts and tagouts during maintenance operations; � Independent verification of component and system alignment before start up or testing activities; DOE-EM-STD-5505-96 16 � Shift turnover between operators, maintenance personnel, shift supervisors or foremen, and personnel in the control area; � The recording of data in logbooks or on round sheets by operators, maintenance personnel, control area operators, and shift supervisors or foremen;

� Process water sampling and analysis; � The control and use of activity procedures during routine operations, activities, emergency situations, and drills; � Placement, control, and removal of jumpers and temporary equipment in use at the activity; � Activity drills and exercises conducted during the assessment; � Operator aids posted throughout the activity; � Number and reason alarms actuated; � The labeling of equipment and systems in the activity. 5.8.1.2 How to Observe. In preparing for and performing observations, there are several things to keep in mind: � Be familiar with the operations � Be familiar with the guidelines and use them as an expectation baseline � Pay close attention to ensure every detail of the observation is taken in and recorded � Follow your intuition. If something does not look or seem right, check further. Assume your intuition is correct until your research proves otherwise � Take copious notes. This will make it easier to follow up on your observations � Data gathered from observations should be verified and cross-checked by the following methods: observe the same operation being performed by different personnel, observe different operations on the same shift, or observe different shifts performing similar operations These principles can be applied to all observations conducted during an assessment. For example, while observing maintenance on a contaminated valve at a waste-water processing DOE-EM-STD-5505-96 e@ Shift turnover between operators, maintenance personnel, shift supervisors or foremen, and personnel in the control area; e@ The recording of data in logbooks or on round sheets by operators, maintenance personnel, control area operators, and shift supervisors or foremen; e@ Process water sampling and analysis; e The control and use of activity procedures during routine operations, activities, emergency situations, and drills; e Placement, control, and removal of jumpers and temporary equipment in use at the activity; e Activity drills and exercises conducted during the assessment; @ Operator aids posted throughout the activity; @ Number and reason alarms actuated; e@ The labeling of equipment and systems in the activity. 5.8.1.2 How to Observeln preparing for and performing observations, there are several things to keep in mind: e Be familiar with the operations e Be familiar with the guidelines and use them as an expectation baseline e Pay close attention to ensure every detail of the observation is taken in and recorded e@ Follow your intuition. If something does not look or seem right, check further. Assume your intuition is correct until your research proves otherwise @ Take copious notes. This will make it easier to follow up on your observations @ Data gathered from observations should be verified and cross-checked by the following methods: observe the same operation being performed by different personnel, observe different operations on the same shift, or observe different shifts performing similar operations These principles can be applied to all observations conducted during an assessment. For example, while observing maintenance on a contaminated valve at a waste-water processing 16 DOE-EM-STD-5505-96 17 facility, the assessor may review the work documentation and applicable facility operating and radiological control procedures before the observation. This will allow the assessor to become

familiar with the guidelines to be used as an expectation baseline for the observation. While observing the operation, the assessor should watch the performance of as many operations personnel as possible and take notes on what is observed. Finally, the assessor should review these notes to identify any apparent deviations from requirements and determine the follow up actions necessary to validate or disprove them. 5.8.1.3 Where and When to Observe. Observations can be conducted the entire time that the assessor is at an activity. Observations can take place during scheduled activities (drills, maintenance operations, system alignments, etc.) or can be conducted during activity walkthroughs and tours. The activity plan of the day or plan of the week should aid in identifying the time and place for observing scheduled activities. 5.8.1.4 Stop Work Authority. In general, assessors should be familiar with and abide by the Operations Office's policy concerning stop work authority. While observing an operation, if an unsafe situation arises in which operators fail to recognize a potential safety concern, the assessor has a responsibility to point this out to the operators. Having the safety concern pointed out, the operators should correct the problem. If they do not, the next step might be to contact the work supervisor so that he or she can put a stop to the unsafe work. If the situation requires immediate action to prevent personal injury, the assessor must be the authority in stopping the unsafe work. An assessor who stops unsafe work should notify the team leader immediately. The team leader will take the necessary steps to inform activity management. 5.8.2 Interview. During any assessment, the interview process will be an important information-gathering tool. Interviews are an effective method of determining level of knowledge and familiarity with activity policies and procedures. An interview can be the key step in following up on leads. To ensure that all relevant information is obtained efficiently and DOE-EM-STD-5505-96 18 accurately, each interview should be planned and organized. 5.8.2.1 How to Prepare for Interviews. Planning is critical to the success of any interview. It enables the assessor to maximize the use of interview time. The planning process does not have to be formal; it may be simply a matter of determining what information is being sought. The assessor should determine which interviews can be conducted during walkthroughs, special activities, routine operations at the activity, and which interviews require coordination and scheduling with activity management. In planning the interview, the assessor should identify the goals for the interview and the items to be discussed so that a logical sequence of questions can be developed beforehand. It may also be appropriate to bring relevant reference documentation for review during interviews. For an interview with the Radiological Control Manager, it may be useful to have survey records and the pertinent radiological control manual readily available. 5.8.2.2 Conducting Interviews. Two types of questions are routinely used during the interview process: open-ended and closed-ended. A good mix of these two types of questions should provide the assessor with enough information. An open-ended question places the burden of conversation on the interviewee and gives the assessor time to analyze what the interviewee is

saying. It reduces the total number of questions asked. An open-ended question elicits more than a yes or no response and is very useful when starting a line of questioning in a new subject area. For example, asking an operator to "explain" or "describe" an event is an open-ended question. The advantage of open-ended questions is that they usually provide a large amount of information about the topic of interest to the assessor. The interviewee does most of the talking in responding to them, and the information provided is generally volunteered. A closed-ended question is a specific question that is often answered with only one or two words. Examples are "what," "when," "who," and "where" questions. Closed-ended questions DOE-EM-STD-5505-96 19 place the burden of conversation on the assessor in that the assessor spends much more time thinking of and stating the question than it takes the interviewee to answer it. The advantages of closed-ended questions are that they allow the interviewer to obtain specific information about a topic of interest, are less time-consuming to answer, and are easier to record. Assessors should avoid using only closed-ended questions as they can elicit insufficient information, and require a lot of follow-up. The proper combination of open-ended and closed-ended questions will provide the most efficient use of time for both the assessor and the activity. To prevent the interviewee from digressing from the topic of interest, the assessor must direct the interview. The assessor’s use of leading questions should be minimized. In a leading question, the assessor gives the interviewee a partial answer and expects the interviewee to complete it, or the assessor gives the complete answer and expects the interviewee to agree or disagree with it. During the interview, the interviewer should continuously evaluate the information being gathered, and ask follow up questions when necessary. This may lead to additional paths of discussion that are necessary to clarify the topic. Before concluding the interview, the assessor should attempt to summarize the information received from the interviewee. This will ensure that the assessor has correctly interpreted and recorded the information provided during the interview. After completion of an interview, the assessor should decipher, analyze, and evaluate the notes taken and identify areas that require follow up. The assessor should decide whether follow-up observations, document reviews, or interviews are required to clarify the information gathered during the interview. An important point to remember is that the information gathered during an interview, observation, or document review should be corroborated with other information. In order to validate apparent deviations from requirements and identify programmatic breakdowns DOE-EM-STD-5505-96 20 or widespread problems, more than one assessment technique should normally be used. 5.8.2.3 Where and When to Conduct Interviews. During an operations assessment, most interviews should be conducted while performing observations and document reviews. If necessary, interviews can also be scheduled and conducted separately. For example, operators and shift supervisors can be interviewed during rounds and while conducting operations, as time permits. However, the assessor must ensure that the operators are not distracted from performing

their duties while being interviewed during operational activities. If it is necessary, these personnel can also be interviewed in a more formal setting such as an office or conference room. In general, operators and shift supervisors can be interviewed at their work location. Conversely, it is usually best to interview activity managers, operations supervisors, and staff personnel in their offices or in another convenient location. Assessors should coordinate with the interviewee to establish the time and location of scheduled interviews. To minimize the impact on activity operations, assessors should always plan to arrive at the interview location on time, or reschedule the interview. 5.8.3 Document Review. The assessment team should review applicable documents both prior to and during the assessment. In general, document reviews are conducted for the following reasons: � To gain familiarity with the requirements contained in activity policies, procedures, etc. � To validate or disprove apparent deviations from requirements and identify programmatic breakdowns or widespread problems � To investigate apparent inconsistencies between activity policies and procedures, and DOE Directives DOE-EM-STD-5505-96 21 � To refresh the assessors knowledge of the elements of conduct of operations. Applicable DOE standards and guidelines should be reviewed prior to the assessment. There are DOE Standards for each element of conduct of operations. � To identify equipment or shift-duty performance problems Since operations assessments should be focused on operations, assessors should not spend the majority of their time conducting document reviews. Document reviews should be used by assessors to establish their expectations concerning activity operations and when pursuing leads developed from observations and interviews. 5.8.3.1 What Documents to Review. As discussed in section 5.4.1, assessors should review selected documents when planning an assessment. Additional documents that should be reviewed during the assessment include: � Activity program and operations procedures and policy manuals � Lockout and Tagout procedure, logs, and "Danger" and "Caution" tags � Round/tour inspection sheets � Training records and databases � Notification lists and documentation � Equipment and system alignment records � Equipment deficiency logs and identification tags � Maintenance requests and work packages � Temporary modification control records � Operating logs � Shift turnover checklists � Chemistry sampling and analysis records � Required reading files and indexes � Equipment and system operating procedures � Timely orders to operators DOE-EM-STD-5505-96 To refresh the assessors knowledge of the elements of conduct of operations. Applicable DOE standards and guidelines should be reviewed prior to the assessment. There are DOE Standards for each element of conduct of operations. To identify equipment or shift-duty performance problems Since operations assessments should be focused on operations, assessors should not spend the majority of their time conducting document reviews. Document reviews should be used by assessors to establish their expectations concerning activity operations and when pursuing leads developed from observations and interviews. 5.8.3.1 What Documents to Reviews discussed in section 5.4.1, assessors should review selected documents when planning an assessment. Additional documents that should be reviewed

during the assessment include: Activity program and operations procedures and policy manuals Lockout and Tagout procedure, logs, and "Danger" and "Caution" tags Round/tour inspection sheets Training records and databases Notification lists and documentation Equipment and system alignment records Equipment deficiency logs and identification tags Maintenance requests and work packages Temporary modification control records Operating logs Shift turnover checklists Chemistry sampling and analysis records Required reading files and indexes Equipment and system operating procedures Timely orders to operators 21 DOE-EM-STD-5505-96 22 � Operator aids posted in the activity � Radiological control survey logs � Radiological postings � Equipment calibration records � Approved lists of abbreviations and acronyms � Emergency communication system test records 5.8.3.2 Reviewing Documents. In general, an assessor should keep the following guidance in mind when conducting document reviews: � Identify questions for interviews and key steps or sections of operating procedures to observe. � Identify administrative and technical source documents and higher-level procedures to check. � Take specific and accurate notes to allow efficient follow up, if necessary. Documents can be reviewed before, during, or after observations and interviews. To start an assessment of chemistry and unique processes, for example, the assessor could decide to review operating documents such as activity logs and cross-check them for consistency with each other and the governing procedure. For instance, if a sample was taken, analyzed, and reported, do the sample logs and the discharge logs agree? Was the sample taken, analyzed, and the results reported prior to the discharge occurring? Is this required? If discrepancies are found, additional observations, interviews, and document reviews should be conducted. The assessor could also start the assessment of this area by reviewing activity and site procedures. Once these procedures are reviewed, an observation of the sampling process could be conducted to determine if deviations from requirements exist. 5.8.3.3 When and Where to Review Documents. Document reviews can be conducted in conjunction with observations and interviews, as well as separately. For example, while DOE-EM-STD-5505-96 23 following an operator conducting rounds throughout the activity, an assessor can normally review the following documents: � Operator round/tour inspection sheets and other narrative logs � Activity operating procedures at various work stations � Radiological control postings and survey logs � Control area operating logs and other documents that may be kept in the control room such as the Lockout and Tagout log, Material Deficiency log, Occurrence Notification lists, etc. � Operator aids log As stated above, documents can also be reviewed as a separate activity. Complicated documents such as program procedures and policy manuals may require dedicated, in-depth review by assessors. These reviews can be done at the activity (in a meeting room, for example), or at the assessment team's operating base. 5.8.4. Pursuing leads. "Pursuing leads" is the process of following up on an apparent deviation from requirements using observations, interviews, and document reviews. A lead is considered a deviation from expectations or an apparent deviation from requirements that is

developed as a result of observations, interviews, or document reviews. The process of pursuing leads should continue until all leads have been validated, and any programmatic breakdowns or widespread problems are identified, or until sufficient evidence exists to disprove the apparent discrepancy. The ability to identify leads and pursue leads depends on the intuition, experience, and tenacity of the assessor. To effectively identify leads, an assessor must: � Trust intuition. If something does not seem right, keep looking until verified right or wrong. � Do not assume more than evidence supports. � Maintain a questioning attitude about all that is observed during the assessment. � Be familiar with the requirements of each area of conduct of operations and activity policies and procedures. DOE-EM-STD-5505-96 24 Furthermore, to pursue leads effectively, an assessor must: � Seek supporting information and evidence through follow up observations, interviews, and document reviews � Investigate the programmatic aspects (Is there a programmatic breakdown or widespread problem, or is this an isolated case?) � Be tenacious and persistent until satisfied with the answers obtained � Use good judgement when determining how much time is to be spent on one issue (If pursuit of a lead will result in time constraints on the assessment, discuss the issue with the team leader.) Pursuing leads is a vital part of the operations assessment process. It is not sufficient to just identify apparent deviations from requirements. In order to develop well-supported findings and concerns, the assessment team must confirm all findings and develop concerns that direct activity management's attention to the underlying causes. To illustrate the process of pursuing leads, consider an operator pumping a wastewater retention basin to the environment through a permitted outfall via a natural creek flowing next to the site boundary. During the assessment, an assessor notes that one operator bypasses the pH meter when starting the discharge to the creek. This action is not stated in the procedure. This is a deviation from the expectation that operators should follow a procedure and should not take important action which is not covered in a procedure. The assessor needs to determine why the operator took an action which was not covered in the procedure. Bypassing alarms, warning devices and instrumentation is a serious operational matter. For instance, the assessor could decide to interview the operator during the basin pumping operation. During the interview, the operator states that she always bypasses the pH meter on starting the pump because the shift supervisor has instructed her to do so. Now the situation has changed in that it is not necessarily one operator who is not following procedures; it DOE-EM-STD-5505-96 25 is the shift supervisor disregarding written procedures and instructing the shift operators accordingly. So far, the assessor has been limited to observing only one shift. Therefore, an observation of an operator on a different shift might be appropriate. During this observation, the assessor notes that the second operator also bypasses the pH meter in the same way. When questioned, the operator states that the shift supervisor provided the same instruction since it reduces the alarms that are received in the control room. The assessor now can conclude that this practice which is not

allowed by procedure is not limited to one operator or shift. There are several directions in which the assessor could proceed. For example, the assessor could interview personnel from the engineering or maintenance or training organization. The questioning may indicate that supervisors and operators are living with system deficiencies because of a design or maintenance problem with the pH circuit or defective cell system. The logical questions that the assessor should ask are: Have the problem alarms been identified? Has engineering evaluated the problems? Is operations compensating for poor design or performance? How was the determination made that the alarms received are false? What was the basis and who authorized bypassing an NPDES instrument? Is bypassing the cell approved? Why are there no compensatory measures in the procedure to ensure that the cell is not bypassed during the discharge? There are some areas of concern that may arise from this scenario. The fundamental issue of concern related to conduct of operations is the bypassing of a safety or monitoring system. Another area of potential concern is that there may be a reportable issue to the state or federal environmental protection agency because the monitoring system is part of the site environmental permit. An additional area of concern might be why is there a bypass at all. More generally, why do trained personnel, who should be knowledgeable about alarms, permits, bypassing systems, instrumentation, procedural protocol, not challenge the verbal direction to bypass DOE-EM-STD-5505-96 26 systems without extraordinary compensatory measures. Other areas that the assessor might investigate to pursue leads further include: � Determining if the procedure change system is too cumbersome for operators to use for known problems � Investigating to see if the technical engineering group submits procedure changes in response to design problems � Analyzing if the procedure writer's group has been tasked with higher priority projects than making changes to the retention basin procedure � Exploring activity management involvement in assuring that procedures are followed according to management expectation and that they remain current 5.9 Daily Team Meetings. The time and location for a daily meeting between all team members should be established. The meeting should be chaired by the team leader and should last approximately one hour. To ensure that every team member participates, the meeting should be structured in a relatively informal manner with enough opportunity for open discussion. Typically, each team member provides a short summary of any leads, potential findings or concerns that are developing, and any areas of inquiry and issues to be pursued the next day. All team members should have the opportunity to question observations and comment on potential similarities or overlaps with their activities. However, team members should not go into lengthy discussions describing an accounting of their days activities. During the day, team members may discover information that is the responsibility of other team members. For large teams, it is helpful to designate a written method for passing information from member to member. It may be advantageous to utilize a 3x5 index card listing the following information: the lead discovered, the name of the individual who discovered it, and the

group to whom it is being passed. This information should be passed as soon as possible but not DOE-EM-STD-5505-96 27 later than the daily team meeting. During the discussion by each team member, the team leader should take notes summarizing leads, findings, and concerns. The information could be recorded using a flip chart or overhead transparency, to allow the team leader to track the resolution of each item as the assessment progresses. This record also provides the team leader with a list of topics to be discussed with activity management, normally accomplished by a separate daily meeting between the team leader and the managers of the activity. 5.9.1 Daily Meetings with Activity Management. The management of the activity should be provided with periodic information regarding the status of the assessment, findings, and areas in which management attention could facilitate the conduct of the assessment. The team leader can either allow contractor management to attend the daily assessment team meeting or can meet separately with contractor management immediately following the daily team meeting. Any safety issues that require corrective action should be communicated immediately to activity management. Open communication is imperative. Feedback from the activity can prevent inaccuracies from being included in the findings. The most effective way to resolve misunderstandings is to promptly contact the management of the activity and request their assistance. This request should be made by the team leader. By coordinating all communication with activity management through the team leader, the team leader is in a position to monitor the assessment and ensure it progresses smoothly. 5.10 Findings and Concerns. The end product of the assessment process is the identification and reporting of findings and concerns. The assessment report provides activity management with areas for improvement, as reported through these concerns. Findings and concerns are defined as follows: DOE-EM-STD-5505-96 28 � A finding is an individual item that does not meet requirements. � A concern is a determination of a programmatic breakdown or widespread problem supported by one or more findings. While findings are the “facts” and are observable evidence of deviations from policy, procedure, or requirements in a DOE Order or activity policy or procedure, concerns are the issues that management must address in order to eliminate the recurrence of findings. For example, an operator is observed performing an incomplete self survey before leaving a posted Contamination Area. This is a finding that could be written as follows: “An operator exited a Contamination Area without conducting self-monitoring as required by the posted procedure.” An individual finding that indicates a programmatic breakdown or widespread problem can also rise to the level of a concern. For instance, an assessor reviews the graded-approach document and realizes that it has not been approved by DOE. This is a concern that can be written as follows: “There is no approved graded-approach document for implementing conduct of operations at the activity .” Concerns can result from an individual finding, as discussed above, or also from several findings in one or more areas of the assessment. For instance, an assessor discovers that 7 of 10 Control Area operators are not aware of maintenance activities in progress, do not control routine

operations, and do not participate in the shift turnover process as required by activity procedures. These findings indicate that there are widespread programmatic breakdowns regarding the control of activity equipment and systems. This is a concern that can be written as follows: “Waste- treatment control room operators are not aware of all operations in progress and do not adequately control the waste treatment process. They are normally not briefed at shift turnover.” This example illustrates how findings from two areas of an assessment, Control Area Activities and Shift Turnover, can result in a concern. The flow diagram in Appendix B shows the process used to develop findings and concerns. DOE-EM-STD-5505-96 29 5.11 Developing Concerns. The final team meeting should focus on development of concerns based on all findings uncovered during the assessment. The daily team meetings should have familiarized each team member with the findings developed by other assessors. This familiarity enables the team to consider all findings while developing activity-wide concerns. Prior to the concerns generation meeting, it is helpful to post the findings identified by each assessor on the walls of the meeting room so they are readily available for reference. Each assessor should develop any preliminary concerns based on the entire team's findings. The concerns may focus on the areas examined by the individual assessor or they could be from another area. Preparation will ensure that the meeting progresses smoothly. The team leader should facilitate the session and start by choosing one assessor to enumerate any concerns that they have developed. The first assessor should present concerns one at a time, explaining the evidence supporting them. The team should discuss each concern as it is presented, either changing the wording and scope as necessary to make the concern accurately reflect conditions in the activity or discarding it altogether if there is not enough evidence to support it. This process should continue until all assessors have presented their concerns and the team has reached consensus on the final list. The team leader should decide in those cases where there is a split opinion. 5.12 Report Generation. When the assessment is completed, all team members must have completed their investigations and have sufficient documentation to support all findings and concerns. The out-brief should provide activity management with the findings and concerns from the current assessment. 5.12.1 Report Format. The final report should be in the format illustrated in Appendix G and contain the following elements: DOE-EM-STD-5505-96 30 � Introduction . The introduction should provide a brief overview of the purpose and scope of the assessment and should address the status of the activity's graded-approach document for conduct of operations. �� Concerns. Concerns should be organized into the following five areas, if applicable: Activity Management, Control of Operations, Maintenance, Training, and Radiological Control. � Findings. Findings should be grouped by the applicable Element of Conduct of Operations, followed by findings in the areas of Radiological Controls, Drills, and Training, as appropriate. � Assessment Team Members. A complete listing of assessment team members and their organizational affiliation should be included as an attachment to the report.

5.12.2 Distribution. Copies of the draft report should be provided for information to activity management and assessment team members at the exit meeting. Additionally, a copy of each operations assessment report shall be submitted by the assessment team to the senior EM executive at the Operations Office and the responsible Headquarters offices. 5.13 Assessment Out-brief. The assessment out-brief, conducted by the team leader, should provide immediate, organized feedback to the activity regarding the assessment team's conclusions. As with the activity in-brief, the DOE Facility Representative, team members, and contractor management should attend this meeting. The team leader should prepare opening remarks followed by a clear presentation of all concerns developed by the team. Questions by activity management on specifics of the concerns should be encouraged by the team leader. Team members most knowledgeable in the findings that support individual concerns should be prepared to assist the team leader in answering questions when needed. DOE-EM-STD-5505-96 31 6. GUIDELINES FOR ASSESSING OPERATIONS 6.1 Assessing Operations Organization and Administration (OPS O&A). 6.1.1 Discussion. The organization and administration of operations should ensure that a high level of performance in DOE activity operations is achieved through effective implementation and control of operational activities. A high level of performance in DOE operations is accomplished by establishing high operating standards, by communicating operating standards to the working level, by providing sufficient resources to the operations department, by ensuring personnel are well trained, by closely monitoring performance in operations, and by holding workers and their supervisors accountable for their performance in conducting evolutions. 6.1.2 Preparing to Assess OPS O&A. To develop an assessment plan for OPS O&A, assessors should first review the activity-specific graded-approach document and determine which guidelines are applicable at the activity. This will allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. OPS O&A provides the philosophical underpinning of Conduct of Operations. As such, OPS O&A provides the framework for the management controls contained in sections 6.2 through 6.18. Because it is an all-encompassing section, OPS O&A should be continuously evaluated by all assessors. All assessors should measure how well management has defined and communicated their policies, how well working-level personnel understand and execute management's expectations, and how management monitors operations. Assessors should continuously ask themselves three questions: � Do workers know what is expected of them? DOE-EM-STD-5505-96 32 � Has management provided guidance about their expectations? � Are workers properly trained? There are two complementary approaches to assessing OPS O&A. The first is a top-down approach that emphasizes interviews with each level of management to determine their expectations and how they monitor operations performance throughout the organization, down to the operating level. The second approach emphasizes observing and interviewing operators to

determine their level of knowledge and then moving up the organization to determine management expectations. Assessors should decide whether to use one or a combination of these approaches based on the activity being assessed and their particular assessment experience and style. The nature of this area of conduct of operations necessitates the assessor's use of interviews. However, assessors must actually go out and verify that activity performance matches management guidance. 6.1.3 Observation. Observation can be an important tool in determining how effectively management has communicated its policies to workers and how well workers execute management expectations. There are numerous observations an assessor can make to determine if policies are being communicated and implemented. Specific examples of activities that an assessor might observe include: � Activity walkthroughs � Activity management conducting scheduled walkthroughs � Pre-job briefings � Postings of management safety policies and goals � Graphs and charts of operations output and performance � Postings of safety goals and actual worker performance DOE-EM-STD-5505-96 33 As an example of how to conduct an observation of OPS O&A, consider the first two examples from the above list. First, walk through the activity to identify apparent operations and material deficiencies. Once this has been completed, accompany a manager or supervisor while they conduct a scheduled management walkthrough. During the observation, note what actions the manager takes when deficiencies are identified, specifically: � Are deficiencies documented? � Did the manager identify the deficiencies that you previously identified? The observation may lead to an unscheduled interview with the manager. Some questions related to this example include: � How do you monitor activity performance? � How often do you perform activity walkthroughs? � How do you address deficiencies that are identified during walkthroughs? Once the observation has been completed, compare what was observed to the requirements contained in activity polices and procedures to identify any apparent deviations from requirements. Pursue leads using additional observations, interviews and document reviews to confirm or disprove those leads and determine if programmatic breakdowns or widespread problems exist. The above is just one example of an observation that could be performed while assessing OPS O&A. 6.1.4 Interview. Depending on the leads developed during observations, interviews related to the organization and administration of the activity are an effective method of determining where potential deficiencies exist. One of the primary objectives of interviews is to determine how well goals are communicated from top management down through the organization's front-line workers. As stated earlier, OPS O&A affects all levels of the organization. As a result, it is imperative that assessors talk to personnel at all levels of the organization. Interviews with managers, supervisors, and a sampling of front-line workers will give an indication of how well DOE-EM-STD-5505-96 34 goals are communicated and understood, in addition to how management monitors activity operations. Interviews of operations and maintenance department personnel should occur during observations and document reviews. Scheduled interviews should also be conducted to develop and clarify

leads. Some interview goals include determining whether: � Operations policies exist and are communicated to workers � Goals are communicated to and from each level of the organization � Resources are sufficient to accomplish assigned tasks � Activity management takes an active role in activity operations by monitoring operating performance � Training is formalized for supervisory and management positions � Activity management effectively plans for safety Some example interview questions for managers at the activity include: � What are your operating goals and how do you communicate them to workers? How are these goals achieved and tracked? � How do you monitor operations and safety performance and improvement? � How do you plan for safety? � What is your involvement in developing training for your workers? � How do your supervisors and operators know what their responsibilities are, and those of their immediate subordinates? How do they know the scope of their authority? � What is your perspective of safety versus production? What are your worker's impressions of safety versus production? � What type of supervisory training do you and your managers/ supervisors receive? � How does management support job planning? How are workers included in the job planning process? DOE-EM-STD-5505-96 goals are communicated and understood, in addition to how management monitors activity operations. Interviews of operations and maintenance department personnel should occur during observations and document reviews. Scheduled interviews should also be conducted to develop and clarify leads. Some interview goals include determining whether: @ Operations policies exist and are communicated to workers @ Goals are communicated to and from each level of the organization e Resources are sufficient to accomplish assigned tasks e Activity management takes an active role in activity operations by monitoring operating performance @ Training is formalized for supervisory and management positions e Activity management effectively plans for safety Some example interview questions for managers at the activity include: @ What are your operating goals and how do you communicate them to workers? How are these goals achieved and tracked? @ How do you monitor operations and safety performance and improvement? @ How do you plan for safety? @ What is your involvement in developing training for your workers? e@ How do your supervisors and operators know what their responsibilities are, and those of their immediate subordinates? How do they know the scope of their authority? e@ What is your perspective of safety versus production? What are your worker's impressions of safety versus production? @ What type of supervisory training do you and your managers/ supervisors receive? @ How does management support job planning? How are workers included in the job planning process? 34 DOE-EM-STD-5505-96 35 Interview questions for operators include: � What are management's goals? � How often do you see management in the activity? � How do you plan for safety? � What is your impression of safety versus production? Finally, a question that could be asked of training, maintenance, and radiological control organization personnel is: � What is the working relationship between operations and support organizations (Training, Maintenance, Radiological Control)? After completing an interview, the information gathered should be evaluated to determine if there

are any apparent deviations from activity requirements. These leads can normally be validated or disproved by conducting one or more of the following: � Observations of and interviews with operators � Tours of the activity and interviews with shift supervisors or foremen � Interviews with shift supervisors, activity managers, and operations supervisors � Reviews of operating and program procedures � Interviews with training, maintenance and radiological control organization personnel 6.1.5 Document Review. Activity policies may be reviewed during the assessment to become familiar with activity operations and management expectations concerning goals and authority. However, since document review only describes administrative performance, actual performance should be verified through interviews and observations. Examples of documents that could be reviewed include: � Management safety policies and goals � Performance indicators � Operations resources and staffing plans DOE-EM-STD-5505-96 Interview questions for operators include: @ What are management's goals? @ How often do you see management in the activity? @ How do you plan for safety? @ What is your impression of safety versus production? Finally, a question that could be asked of training, maintenance, and radiological control organization personnel is: @ What is the working relationship between operations and support organizations (Training, Maintenance, Radiological Control)? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by conducting one or more of the following: @ Observations of and interviews with operators e Tours of the activity and interviews with shift supervisors or foremen e@ Interviews with shift supervisors, activity managers, and operations supervisors e Reviews of operating and program procedures e Interviews with training, maintenance and radiological control organization personnel 6.1.5 Document ReviewActivity policies may be reviewed during the assessment to become familiar with activity operations and management expectations concerning goals and authority. However, since document review only describes administrative performance, actual performance should be verified through interviews and observations. Examples of documents that could be reviewed include: @ Management safety policies and goals e@ Performance indicators @ Operations resources and staffing plans 35 DOE-EM-STD-5505-96 36 � Policies and procedures related to monitoring operations performance, safety goal performance, and management training � Pre-job or safety checklists � Documented management self-assessments � Management training records When reviewing documents, assessors may decide to review the activity's goals. There are three questions the assessor should ask concerning goals: � Do goals exist? � Are goals measurable, challenging, and auditable? � Do all personnel understand the goals and have familiarity with the actions needed to achieve them? The existence of goals is only part of the answer, however. While goals may be posted on bulletin boards throughout the activity, this in itself does not indicate that workers understand how management expects those posted goals to be executed. After becoming familiar with activity goals, assessors may want to conduct informal interviews with managers, supervisors, and

operators to determine their understanding of goals. 6.2 Assessing Shift Routines and Operating Practices. 6.2.1 Discussion. The purpose of Shift Routines and Operating Practices is to set standards for the professional conduct and level of performance expected of operations personnel to ensure facilities are operated safely and efficiently. The guidelines establish the overall framework for single and multi-shift operations and apply to routine operations as well as to the detection of and response to abnormal conditions. 6.2.1.1 Approach. Since the observations, interviews, and document reviews conducted to DOE-EM-STD-5505-96 37 assess Shift Routines and Operating Practices will normally provide leads in other elements of conduct of operations (Logkeeping, for example), the assessment of shift routines can be combined with other elements of conduct of operations such as Shift Turnover, Control Area Activities, Communications, Operations Aspects of Facility Chemistry and Unique Processes, and Logkeeping. This combined approach is presented in the following sections. 6.2.2 Preparing to Assess Shift Routines and Operating Practices. To develop an assessment plan for Shift Routines and Operating Practices, assessors should first review the activity-specific graded-approach document and determine which guidelines are applicable at the activity. This will allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. 6.2.3 Observation. The activity plan of the day and plan of the week should be reviewed to determine the time and place for observing work in progress and drills. Discussion and coordination with the shift supervisor or person in charge is very useful to establish opportunity for observations. The assessor should be sure the person in charge knows the reason for the observation. Observations of shift routines should be conducted during walkthroughs and tours with operators and shift supervisors, and during the conduct of special operations and drills. The following situations are good candidates for conducting observations: � Maintenance activities that require interface or coordination with roving operators, control area operators, and shift supervisors � Startup and shutdown of systems and equipment � Process or rate change operations � Activity drills � Routine operations of equipment � Control Room operations DOE-EM-STD-5505-96 38 � Operator and Supervisor inspection tours There are several aspects of Shift Routines and other related elements of conduct of operations that can be assessed during these observations. For example, while observing the startup or shutdown of a system or a component, the following items might be observed: � Operator response to indications and alarms as valves are operated and system components are energized or deenergized � Operator adherence to posted personnel protection requirements, use of monitoring instruments, and adherence to radiation and safe work permits � Resetting of protective devices, if trips occur � Knowledge, skill and level of qualification of operations personnel � Notification of changes in system status by operators and shift supervisors � Communication between shift positions

Observations during routine and abnormal events may present the opportunity to observe the following: � Workers conducting assigned tasks expeditiously should not interfere with good watchstanding practices � Operators notifying the shift supervisor (or cognizant manager for research and test facilities) of any unexpected situations � Operations personnel adhering to the requirements of the facility industrial safety program. Proper hearing, eye, head, foot, and respiratory protection should be worn in designated areas to reduce the potential for injury. � Ladders or other approved means are being used to access equipment located in the overhead when permanent steps or catwalks are not available. � Personnel exercising appropriate precautions when entering or working in or around energized panels or equipment. For example, ensure that electrical panel closures are securely fastened prior to making the breakers operable to energize equipment. DOE-EM-STD-5505-96 Operator and Supervisor inspection tours There are several aspects of Shift Routines and other related elements of conduct of operations that can be assessed during these observations. For example, while observing the startup or shutdown of a system or a component, the following items might be observed: Operator response to indications and alarms as valves are operated and system components are energized or deenergized Operator adherence to posted personnel protection requirements, use of monitoring instruments, and adherence to radiation and safe work permits Resetting of protective devices, if trips occur Knowledge, skill and level of qualification of operations personnel Notification of changes in system status by operators and shift supervisors Communication between shift positions Observations during routine and abnormal events may present the opportunity to observe the following: Workers conducting assigned tasks expeditiously should not interfere with good watchstanding practices Operators notifying the shift supervisor (or cognizant manager for research and test facilities) of any unexpected situations Operations personnel adhering to the requirements of the facility industrial safety program. Proper hearing, eye, head, foot, and respiratory protection should be worn in designated areas to reduce the potential for injury. Ladders or other approved means are being used to access equipment located in the overhead when permanent steps or catwalks are not available. Personnel exercising appropriate precautions when entering or working in or around energized panels or equipment. For example, ensure that electrical panel closures are securely fastened prior to making the breakers operable to energize equipment. 38 DOE-EM-STD-5505-96 39 � Workers conducting a thorough tour of all areas within his/her responsibility, making appropriate equipment inspections at designated times at least once per shift (inspected less frequently because of adverse radiological or equivalent personnel safety conditions, or more frequently if problems have been encountered.) During walkthroughs and inspections by operators and supervisors, the following aspects of Shift Routines and other related elements of conduct of operations can normally be evaluated: � The scope, detail, and periodicity of operator inspection tours � Adherence to personnel protection practices, postings, radiation and safe work permits, etc. � Response to instrument readings, indications, and abnormal conditions

� Use and adequacy of the notification process regarding changes in status, abnormalities, and difficulties encountered during operations � Communication between shift positions � The use of narrative operating logs and round/tour inspection sheets � The adequacy of shift turnover documentation and the process used � Workers checking components, such as electrical panels, alarm panels, auto-start standby equipment, and breakers for unusual conditions. � Workers inspecting all areas for which they are responsible � Supervisory personnel periodically monitoring operator rounds. Operators, maintenance personnel, and shift supervisors from each shift should be observed during the assessment. If possible, the observation of a particular shift position should start at shift turnover since vital information about the operating status of equipment and systems, and operations and maintenance in progress, is passed between shifts at this time. To observe an operator performing rounds the following methodology could be used: � Observe shift turnover conducted between the on-coming and off-going operators. Pay attention to the information passed verbally between the operators concerning operations DOE-EM-STD-5505-96 Workers conducting a thorough tour of all areas within his/her responsibility, making appropriate equipment inspections at designated times at least once per shift (inspected less frequently because of adverse radiological or equivalent personnel safety conditions, or more frequently if problems have been encountered.) During walkthroughs and inspections by operators and supervisors, the following aspects of Shift Routines and other related elements of conduct of operations can normally be evaluated: The scope, detail, and periodicity of operator inspection tours Adherence to personnel protection practices, postings, radiation and safe work permits, etc. Response to instrument readings, indications, and abnormal conditions Use and adequacy of the notification process regarding changes in status, abnormalities, and difficulties encountered during operations Communication between shift positions The use of narrative operating logs and round/tour inspection sheets The adequacy of shift turnover documentation and the process used Workers checking components, such as electrical panels, alarm panels, auto-start standby equipment, and breakers for unusual conditions. Workers inspecting all areas for which they are responsible Supervisory personnel periodically monitoring operator rounds. Operators, maintenance personnel, and shift supervisors from each shift should be observed during the assessment. If possible, the observation of a particular shift position should start at shift turnover since vital information about the operating status of equipment and systems, and operations and maintenance in progress, is passed between shifts at this time. To observe an operator performing rounds the following methodology could be used: Observe shift turnover conducted between the on-coming and off-going operators. Pay attention to the information passed verbally between the operators concerning operations 39 DOE-EM-STD-5505-96 40 planned or in progress, status of activity systems and equipment, and any abnormal conditions that exist in the activity. Review any narrative logs, round/tour inspection sheets, and any other documentation used in the turnover process. � Attend the shift crew briefing. Observe the process used and the information exchanged.

Compare the information disseminated at the briefing with the information that was passed between the operators during their turnover. � Accompany the operator as he performs his duties. Observe the items listed in section 6.2.3. If the activity has a control area, observe the performance of the control area operator and the interface between shift positions. � At the end of the shift, observe the turnover process between this operator and the oncoming operator. Once the observation has been completed, compare what was observed to the requirements contained in activity policies and procedures to identify any apparent deviations from requirements. Pursue leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. A similar process can be used to observe control area operators and shift supervisors during operations and drills conducted by the activity. 6.2.4 Interview. Interviews of operations and maintenance department personnel should occur during observations and document reviews. Additional scheduled interviews should also be conducted to follow up and validate leads. The following personnel might be interviewed during the assessment: � Operators and maintenance department personnel � Shift supervisors or foremen � Activity managers and operations supervisors � Radiological control technicians � Activity operations staff DOE-EM-STD-5505-96 41 � Training department personnel Most of the interviews accomplished to assess Shift Routines and Operating Practices should be conducted with operators, maintenance personnel, and shift supervisors. Whenever possible, these interviews should occur during observations of operations, and activity walkthroughs and tours. To pursue leads, scheduled interviews with the activity manager, and training, maintenance, and radiological control organization personnel may also be necessary. For example, while observing a roving operator beginning at shift turnover, the following types of questions could be used to assess level of knowledge about shift routines: � What are your duties and responsibilities during the shift? � What things do you normally look for during an activity tour? How often do you conduct tours? What areas of the activity are you responsible for? Describe the process that you use to document, report, and correct deficiencies discovered during tours. � Do you use a round/tour inspection sheet to document your tours? What types of information do you record? How do you document abnormal readings? What do you do to report and correct the condition? � What are your responsibilities regarding the activity industrial safety program? Are there any areas in the activity that have special safety requirements? How do you know where they are and what safety equipment is required? � Are there any Controlled Areas in the activity? How do you know what personnel protective equipment (PPE) must be used to enter the area? Have you been trained in the use of PPE? � How do you normally communicate with the control area operator and shift supervisor? How do they communicate urgent information to you? Do you ever use radios to communicate? Are there any areas in the activity that radios can't be used? If so, where are they? � How do you determine what information should be passed at turnover to the on-coming

operator? Are you required to use a turnover checklist to aid the process? What guidance have you received from the shift supervisor or activity manager concerning how to conduct DOE-EM-STD-5505-96 42 shift turnover? Do you feel that the process used at turnover supports your needs? What changes, if any, should be made to improve the process? Many of the questions asked during a walkthrough with an operator may be prompted by observing the work area for which the operator is responsible. After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: � Tours with several more operators � Observations of and interviews with control area operators � Tours of the activity and interviews with shift supervisors � Interviews with radiological control technicians � Interviews with activity managers and operations supervisors � Reviews of operating and program procedures � Interviews with training and maintenance department personnel 6.2.5 Document Review. Examples of documents that should be reviewed include: � Operating logs and round/tour inspection sheets � Alarm status log � Radiation and safe work permits � Safety and radiological postings � Shift turnover checklists � Health and safety plans Most document reviews necessary to assess Shift Routines and Operating Practices can be conducted in conjunction with observations. For example, while touring the activity with an operator, a review of documents such as operating logs, round/tour inspection sheets, operations DOE-EM-STD-5505-96 shift turnover? Do you feel that the process used at turnover supports your needs? What changes, if any, should be made to improve the process? Many of the questions asked during a walkthrough with an operator may be prompted by observing the work area for which the operator is responsible. After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: Tours with several more operators Observations of and interviews with control area operators Tours of the activity and interviews with shift supervisors Interviews with radiological control technicians Interviews with activity managers and operations supervisors Reviews of operating and program procedures Interviews with training and maintenance department personnel 6.2.5 Document ReviewExamples of documents that should be reviewed include: Operating logs and round/tour inspection sheets Alarm status log Radiation and safe work permits Safety and radiological postings Shift turnover checklists Health and safety plans Most document reviews necessary to assess Shift Routines and Operating Practices can be conducted in conjunction with observations. For example, while touring the activity with an operator, a review of documents such as operating logs, round/tour inspection sheets, operations 42 DOE-EM-STD-5505-96 43 procedures, safety and radiological control postings, and radiation and safe work permits, can be accomplished. To illustrate how to perform a document review, consider a round/tour inspection sheet. Various items on the sheet can serve as a lead to develop interview questions and conduct

other observations. For example, look for equipment parameters that are out of specification or are approaching an out of specification condition. If found, ask the operator what actions should be taken if that parameter goes out of specification, or make up a hypothetical situation and ask the same type of question. Look at the round sheets for vague checkoffs such as "check sump level." A follow on question for this may be "What do you look for when you perform this check?" and then the operator's response can be compared with the answers of other operators. Another example might be, the recording of key equipment parameters during tours to provide a record of equipment performance and can be used to reconstruct events leading up to unusual occurrences or system malfunctions. Is there evidence that these records are used to improve equipment performance or maintenance? Is there evidence that the round sheets are used for short-term trending by operators and other technical support organizations? Review round sheets (those which are in use and completed) for technical content such as: � Round/Tour Inspection Sheets Round inspection sheets should where appropriate include maximum/minimum equipment parameters or expected operating ranges to enable operators to recognize abnormal readings quickly. � Safety limits derived from Technical Specifications or Operational Safety Requirements should be highlighted. � Equipment should be listed on round sheets in the same order that it would be encountered during a normal tour of the operating station, and the round sheets should include a narrative section. This technique of using document review as a source for interview questions and observations can be an extremely effective tool for an assessor. DOE-EM-STD-5505-96 44 6.3 Assessing Control Area Activities. 6.3.1 Discussion. Because most control areas or control rooms function as the coordination point for important activity work and operations, activities in the control area or room must be formalized and conducted in a professional, businesslike manner. Control Area Activities addresses important elements that are necessary to support safe and efficient activity operation. 6.3.1.1 Approach. As discussed in Assessing Shift Routines and Operating Practices, control area activities are an integral part of shift routines. This "combined" approach of assessing activities in the control area with shift routines is presented in that section. Since it may be necessary to assess activities in the control room separately, the following sections discuss how to assess Control Area Activities only. 6.3.2 Preparing to Assess Control Area Activities. To develop an assessment plan for Control Area Activities, assessors should first review the activity-specific graded-approach document and determine which guidelines are applicable at the activity. This will allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. 6.3.3 Observation. Control Area Activities is an observation-intensive area of conduct of operations to assess. Observations of activities in the control area should be conducted during activity walkthroughs and tours, and during the conduct of operations and drills. The activity plan

of the day and plan of the week should be reviewed to determine the time and place for observing operations and drills. The following situations are good candidates for conducting observations: � Startup and shutdown of systems and equipment � Process or rate change operations DOE-EM-STD-5505-96 45 � Activity drills � Maintenance and special operations controlled from the control area � Excessive and extraneous communication to and from the control room, for example persons calling the control room to have persons paged, phone calls not related to official business. There are several aspects of activities in the control area that can be assessed during these situations. For example, while observing the startup of a system or component the following might be observed: � Control of personnel access into the control area and the "at-the-controls" area � Behavior and atmosphere in the control area by operators on shift and other personnel in the area � Operator response to indications as valves are operated and system components are energized or deenergized � Operator response to alarms and abnormal conditions � Operation of control area equipment by operators � Use of status boards in the control area � Ancillary duties performed by control area operators � Control area operators should not be overburdened with administrative responsibilities, and control area access should be limited so that operators will not be distracted from properly monitoring facility parameters. During the assessment, control area operators from each shift should be observed to get a broad picture of the level of knowledge, skill, and qualification of operators. If possible, these observations should start at shift turnover since vital information about the operating status of equipment and systems, maintenance and operations in progress is passed between shifts during turnover. DOE-EM-STD-5505-96 Activity drills Maintenance and special operations controlled from the control area Excessive and extraneous communication to and from the control room, for example persons calling the control room to have persons paged, phone calls not related to official business. There are several aspects of activities in the control area that can be assessed during these situations. For example, while observing the startup of a system or component the following might be observed: Control of personnel access into the control area and the "at-the-controls” area Behavior and atmosphere in the control area by operators on shift and other personnel in the area Operator response to indications as valves are operated and system components are energized or deenergized Operator response to alarms and abnormal conditions Operation of control area equipment by operators Use of status boards in the control area Ancillary duties performed by control area operators Control area operators should not be overburdened with administrative responsibilities, and control area access shouldlionaited so that operators will not be distracted from properly monitoring facility parameters. During the assessment, control area operators from each shift should be observed to get a broad picture of the level of knowledge, skill, and qualification of operators. If possible, these observations should start at shift turnover since vital information about the operating status of equipment and systems, maintenance and operations in progress is passed between shifts during

turnover. 45 DOE-EM-STD-5505-96 46 To observe a control area operator, the following methodology could be used: � Observe the shift turnover conducted between the on-coming and off-going control area operators. Pay attention to the information passed verbally between the operators concerning operations planned or in progress, status of activity systems and equipment, and any abnormal conditions that exist in the activity. � Attend the shift crew briefing. Compare the information disseminated at the briefing with the information that was passed between the operators during their turnover. � Observe the control area operator as duties are performed. � At the end of the shift, observe the turnover process between this operator and the on-coming operator. Once the observation has been completed, compare what was observed to the requirements contained in activity policies and procedures to identify any apparent deviations from requirements. Pursuing these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. 6.3.4. Interview. Interviews of operations and maintenance department personnel should occur during observations and document reviews. Additional scheduled interviews should also be conducted to follow up and validate leads. The following personnel might be interviewed during the assessment: � Control area operators � Shift supervisors or foremen � Activity managers and operations supervisors � Activity operations staff � Training department personnel Most of the interviews accomplished to assess Control Area Activities should be conducted with DOE-EM-STD-5505-96 47 operators, and shift supervisors. Whenever possible, these interviews should occur during observations of operations, and activity walkthroughs and tours. To pursue leads, scheduled interviews with the activity manager and training department personnel may also be necessary. For example, while observing a control area operator beginning at shift turnover, the following types of questions could be used to assess his level of knowledge: � What are your duties and responsibilities during the shift? � How do you control access to this area? How do you decide what personnel should be granted access? Is there an official access list for the control area? Is there a limit to the number of people allowed in this area? � Who normally operates the equipment in this area? How do you know who is authorized to operate this equipment? Under what circumstances do personnel other than yourself operate this equipment during your shift? � Do you normally perform ancillary duties while on shift? What types of ancillary duties do you perform? How do you know what ancillary duties are authorized during your shift? � What types of operations, and maintenance activities do you control from this area? Describe the process for controlling these operations. � What is the policy concerning reading newspapers or magazines in the control room? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by conducting one or more of the following: � Observations of and interviews with other control area operators � Interviews with shift supervisors � Interviews with activity managers and operations supervisors

� Reviews of program procedures A similar methodology to the one described above can be used to observe specific activity operations and drills, considering that the list of interview candidates and sample questions DOE-EM-STD-5505-96 48 provided may not be all inclusive. 6.3.5 Document Review. Examples of documents that should be reviewed when assessing activities in the control area include: � Program and operating procedures � Control area access lists � Alarm status log Most document reviews necessary to assess control area activities can be conducted in conjunction with observations. For example, while observing an operator in the control area, review control area access lists and other operational postings. If necessary, reviews of program procedures and other documents that are not routinely used during activity operations can be fit in between scheduled interviews and observations. 6.4 Assessing Communications. 6.4.1 Discussion. Highly reliable communication provides accurate transmission of information within an activity. Since accurate communication is essential for the safe and efficient operation of facilities, guidance should be provided by activity management regarding the use of all forms of audible communication. This includes oral (face-to-face), telephone, radio, public address, and special (horns, sirens, and bells) communication equipment and methods used during operations. 6.4.1.1 Approach. As discussed in Assessing Shift Routines and Operating Practices, communication is an integral part of shift routines. This "combined" approach of assessing communications with shift routines is presented in that section. Since it may be necessary to assess communications separately, the following sections discuss how to assess this area of conduct of operations only. DOE-EM-STD-5505-96 49 6.4.2 Preparing to Assess Communications. To develop an assessment plan for Communications, assessors should first review the activity-specific graded-approach document for conduct of operations and determine which guidelines are applicable at the activity. This will allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. 6.4.3 Observation. Observations of activity communications should be conducted during walkthroughs and tours with operations department personnel, and during the conduct of special operations and drills. The activity plan of the day and plan of the week should be reviewed to determine the time and place for observing operations and drills. The following situations are good candidates for observing communication: � Periodic testing of emergency communication systems and equipment. Emergency communications systems should be periodically tested to ensure that they are functional. Control areas should have the capability of overriding other users of the public address system for emergency announcements. � Maintenance and other activities that require communications between roving operators, control area operators, and shift supervisors � Startup and shutdown of systems and equipment � Activity drills � Occurrence notifications � Shift turnover and shift crew briefings � On-shift training activities � Monitoring radio channels used by operations

There are several aspects of communication that can be assessed during these situations. For example, while observing a fire drill, the following items might be observed: � Operation of emergency communication systems and equipment DOE-EM-STD-5505-96 50 � Use of the activity public address system. � Use of portable radios by various shift positions � Use of abbreviations, acronyms, and "repeat backs" during communications � Use of standardized terminology and phonetic alphabet � Communications during the occurrence notification process � Communications between drill team controllers Observing the activities above should be supplemented with observations of routine shift communications conducted during walkthroughs and tours of the activity and control area. Operators, maintenance personnel, and shift supervisors from each shift should be observed to get a broad picture of communication accuracy, reliability, and effectiveness. If possible, the observations of a particular shift position should start at shift turnover since vital information about the operations, maintenance, and special operations in progress at the activity, is passed at this time. To observe communications while touring the activity with an operator, the following methodology could be used: � Observe the shift turnover conducted between the on-coming and off-going operators. Pay attention to the information passed verbally between the operators concerning operations planned or in progress, status of activity systems and equipment, and any abnormal conditions that exist at the activity. Review any documentation used to communicate written information in the turnover process. � Attend the shift crew briefing. Observe the information exchanged. � Accompany the operator as he performs his duties. Observe the use of portable radios, the public address system, telephones and face-to-face communications between the operator and other shift positions. If the activity has a control area, observe communications between the control area operator and other shift positions. DOE-EM-STD-5505-96 51 � At the end of the shift, observe the turnover process between this operator and the on-coming operator. Once the observation has been completed, compare what was observed to the requirements contained in activity policies and procedures to identify any apparent deviations from requirements. Pursue these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. A similar process can be used to observe communications between shift supervisors, maintenance personnel, and other operations department personnel when conduction operations at the activity. 6.4.4 Interview. Interviews of on shift personnel should occur during observations and document reviews. Additional scheduled interviews might also be necessary to follow up and validate leads. The following personnel might be interviewed during the assessment: � Operators and maintenance department personnel � Shift supervisors or foremen � Activity managers and operations supervisors � Radiological control technicians � Activity operations staff � Training department personnel Most of the interviews accomplished to assess Communications should be conducted with operators, maintenance personnel, and shift supervisors. Whenever possible, these interviews

should occur during observations of operations and activity walkthroughs and tours. To pursue leads, scheduled interviews with the activity manager and training department personnel may also be necessary. For example, while observing a roving operator beginning at shift turnover, the following types of questions could be used to assess his level of knowledge about the use of communication equipment and methods: DOE-EM-STD-5505-96 52 � How do you determine what information should be communicated at turnover to the on- coming operator? Are you required to use a turnover check sheet to document this information? What guidance have you received from the shift supervisor or activity manager regarding the type of information to be passed at turnover? � What equipment do you normally use to communicate with other shift positions? Do you ever use radios to communicate? If so, are there any areas in the activity that radios can't be used? Where are they? � What type of information is passed on the activity public address system? Who normally uses this equipment? How frequently is it used? � Is there a procedure that addresses how to communicate? If so, what type of information is contained in the procedure? Is there an approved list of abbreviations and acronyms? When is it to be used? Are there any circumstances that require the use of "repeat backs" Can you give an example of one of these circumstances? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: � Additional walkthroughs and tours of the activity � Observations of and interviews with roving operators, control area operators, and maintenance operators � Tours of the activity and interviews with shift supervisors � Interviews with activity managers and operations supervisors � Reviews of operating and program procedures � Interviews with training department personnel A similar methodology to the one described above can be used to observe other shift positions during operations and drills, considering that the list of interview candidates and sample questions provided may not be all inclusive. DOE-EM-STD-5505-96 53 6.4.5 Document Review. Examples of documents and other written information that can be reviewed to assess Communications include: � Approved lists of abbreviations and acronyms � Records of periodic emergency communication equipment testing � Portable radio postings � Turnover checklists � Program and operating procedures � Emergency plan Most document reviews necessary to assess activity communications can be conducted in conjunction with observations. For example, while touring the activity with an operator, an assessor should look for and review the approved list of abbreviations and acronyms, portable radio postings, and turnover checklists. After reviewing these documents, observe how and when they are used by operators. If necessary, reviews of program procedures and other documents that are not routinely used during activity operations can be fit in between scheduled interviews and observations. 6.5 Assessing Control of On-Shift Training. 6.5.1 Discussion. An effective On-Shift Training Program ensures that unqualified personnel are properly supervised and controlled to avoid operational errors and to maximize use of the

trainee's time. On -shift training should be conducted so that the trainee satisfactorily completes all the required training objectives and receives maximum learning benefit from this experience. Additionally, the program should be structured to ensure that On-the-Job Trainers (OJT) have received the proper training necessary to qualify them as OJT Instructors. On-shift training is not intended to cover all aspects of an effective training and qualification program. Rather, it addresses only the component that includes actual hands-on the equipment DOE-EM-STD-5505-96 54 by the trainee. Additional information regarding training is included in section 6.20. 6.5.2 Preparing to Assess On-Shift Training. To develop an assessment plan for On-Shift Training, assessors should first review the activity-specific graded-approach document for conduct of operations and determine which guidelines are applicable at the activity. This will allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. 6.5.3 Observation. Assessment of on-shift training should be conducted by observing on-shift training in progress. The activity manager, operations supervisor, training manager, or shift supervisor should know the time and location of on-shift training events that will take place during the assessment. Whenever possible, observations should be conducted during each shift. This will provide a good overview of the quality and consistency of the training conducted at the activity. Examples of observations of training evolutions include: � Performing a lockout or tagout � Lining up a system for startup or shutdown � Activity drills � Conducting rounds and tours � Making entries in logbooks and roundsheets � Performing an analysis, taking samples When conducting observations, it is important to note that what is being evaluated is not the student's level of knowledge, but how the training is controlled and supervised. When observing an activity involving on-shift training, start when the trainer first meets with the student and continue until they finish the training session. The following are aspects of controlling and supervising on-shift training that an assessor should observe: DOE-EM-STD-5505-96 55 � Was communication between the trainer and trainee clear and concise? � Does the trainer explain the objectives to the student prior to starting an operation or task? � Does the trainer fully explain or ensure that the student understands all equipment and processes before allowing the student to execute the desired task? � Does the instructor use a method for verifying the student's understanding of instructions prior to allowing the student to perform the desired task? � Was the number of trainees assigned to a trainer consistent with the on-shift training policy? � Does the trainer provide direction to the trainees on how to stop training if an actual problem condition was encountered? � Does the trainer allow the trainee to exercise their own knowledge without excess coaching, leading, or prompting? � Is the instructor qualified? Does the line organization believe that training and the quality of the trainee performance as a result of the training are the responsibility of the training organization?

Once the observation has been completed the assessor should discuss any unclear aspects with the OJT instructor. Clarifications immediately following a training session can often resolve many apparent deviations. For example, an instructor may not have appeared to review procedural prerequisite requirements prior to allowing the student to perform a given task. Through an open discussion of this matter, the OJT trainer may point out that the prerequisites were reviewed the previous day and that the student understood all requirements. The assessor should then pursue leads that have not been resolved using additional observations, interviews, or document reviews and determine if programmatic breakdowns or widespread problems exist at the activity. 6.5.4 Interview. Most of the interviews accomplished to assess on-shift training should be conducted during observations. Depending on the leads developed during observations, additional scheduled interviews with operations and training department personnel may be DOE-EM-STD-5505-96 56 necessary to pursue leads. A sample listing might include the following: � OJT trainers in the operations and maintenance departments � Unqualified operators and maintenance personnel � Operations and maintenance supervisors and managers � Training manager and his instructors � Activity managers and operations supervisors An example line of questioning for the OJT trainer to determine his level of knowledge concerning activity OJT procedures, might proceed as follows: � What training requirements must you fulfill prior to being qualified as an OJT Trainer? � Do you use any forms that are later used to debrief the trainee on their performance? � How do you know whether or not the trainee understands the requirements for the task at hand? � What actions are taken if an unusual situation arises? � How are you evaluated as an OJT trainer? Who does the evaluations? How often are you evaluated? � Do you have any requalification requirements/periodicity? If so, what are they? As evidenced, the line of questioning could go on at great length. While the answers to such questions will lead the assessor to his next question, it is a good idea to jot down several questions prior to starting the interview. After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved using one or more of the following: � Additional observations of on-shift training activities � Interviews with qualified trainers and unqualified operators � Interviews with training department personnel DOE-EM-STD-5505-96 necessary to pursue leads. A sample listing might include the following: OJT trainers in the operations and maintenance departments Unqualified operators and maintenance personnel Operations and maintenance supervisors and managers Training manager and his instructors Activity managers and operations supervisors An example line of questioning for the OJT trainer to determine his level of knowledge concerning activity OJT procedures, might proceed as follows: What training requirements must you fulfill prior to being qualified as an OJT Trainer? Do you use any forms that are later used to debrief the trainee on their performance? How do you know whether or not the trainee understands the requirements for the task at hand? What actions are taken if an unusual situation arises?

How are you evaluated as an OJT trainer? Who does the evaluations? How often are you evaluated? Do you have any requalification requirements/periodicity? If so, what are they? As evidenced, the line of questioning could go on at great length. While the answers to such questions will lead the assessor to his next question, it is a good idea to jot down several questions prior to starting the interview. After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved using one or more of the following: Additional observations of on-shift training activities Interviews with qualified trainers and unqualified operators Interviews with training department personnel 56 DOE-EM-STD-5505-96 57 � Interviews with activity managers, operations supervisors, and shift supervisors � Reviews of operating and program procedures 6.5.5 Document Review. Examples of documents that could be reviewed when assessing On- Shift Training include: � Site or activity training procedures and policy � Supervisor, OJT trainer, and trainee qualification cards � Training Implementation Matrix (TIM) Prior to observing OJT training, the assessor may spend some time ensuring that he is familiar with the procedures and policies governing OJT training. For example, by reviewing an activity's OJT training syllabus, an assessor would be able to understand the requirements that the OJT trainer must fulfill during the training session. For example, these requirements could include the following: � Pre-job brief which might include warnings, cautions, prerequisites � Communication requirements between the trainee and the trainer. � Control of the trainee by the OJT trainer The assessor should also become familiar with the procedure(s) being used by the instructor and trainee to perform their tasks. By observing the OJT trainer in action, the assessor will be better able to evaluate what should occur before, during, and after the training session. 6.6 Assessing Investigation of Abnormal Events. 6.6.1 Discussion. In general terms, the abnormal event investigation program is necessary to investigate abnormal events thoroughly, identify the root cause of events, ensure all necessary notifications are completed, comply with DOE requirements, and ensure appropriate corrective action steps are established to minimize the chance of event recurrence. DOE-EM-STD-5505-96 58 6.6.2 Preparing to Assess Investigation of Abnormal Events. To develop an assessment plan for Investigation of Abnormal Events, assessors should first review the activity-specific graded- approach document for conduct of operations and determine which guidelines are applicable at the activity. This will allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. Assessment of this area presents several challenges in that it might appear difficult to observe the process in action, but with proper planning and coordination between the team leader and activity management, assessors can collect enough data points to determine how effectively the activity has implemented the guidelines of this area of conduct of operations.

6.6.3 Observation. While assessment of investigation of abnormal events requires an emphasis on document reviews and interviews, it is possible to perform an observation of an on-going investigation. However, before allowing an assessor to observe the investigation process, the team leader should receive concurrence from the activity manager in order to minimize any distractions to the investigation team. Observations could include the following: � Management/supervisory response with respect to classification and notification of a pre- planned drill event � Observation of an on-going investigation of a previously identified and reported event Observation of this area is extremely difficult, but in the absence of an on-going investigation, an assessor can walk through the necessary steps required for an actual investigation. Whether observations are made during an actual investigation or during a walkthrough, the following methodology can be used: � Determine the person responsible for leading the investigation. Review this person's qualifications as the investigation team leader. DOE-EM-STD-5505-96 59 � Observe the person responsible for leading the investigation during the investigative process. � Observe the investigation that is in progress. Determine if it is being conducted in accordance with activity procedures and DOE Directives. Specifically, observe the process used to conduct the investigation (i.e., reconstruction of the event, event analysis and evaluation, root cause analysis, corrective action determination). Once the observation has been completed, compare what was observed to the requirements contained in activity policies and procedures to identify any apparent deviations from requirements. Pursue these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. 6.6.4 Interview. The following personnel may be interviewed to assess Investigation of Abnormal Events: � Selected members of the investigation team � Shift supervisors � Activity managers and operations supervisors � Training department personnel A line of questioning for a member of the investigation team might proceed as follows: � As a member of the investigation team, what are your responsibilities? � Are these responsibilities delineated in writing? � What training have you had in the area of root cause analysis? � Briefly explain the process employed in performing an investigation. � How are lessons learned from investigations used to help prevent recurrence of similar events? � How does management determine the level of investigation required for a particular event? � Discuss how events are reconstructed. DOE-EM-STD-5505-96 @ Observe the person responsible for leading the investigation during the investigative process. @ Observe the investigation that is in progress. Determine if it is being conducted in accordance with activity procedures and DOE Directives. Specifically, observe the process used to conduct the investigation (i.e., reconstruction of the event, event analysis and evaluation, root cause analysis, corrective action determination). Once the observation has been completed, compare what was observed to the requirements contained in activity policies and procedures to identify any apparent deviations from

requirements. Pursue these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. 6.6.4 Interview The following personnel may be interviewed to assess Investigation of Abnormal Events: @ Selected members of the investigation team e@ Shift supervisors e Activity managers and operations supervisors @ Training department personnel A line of questioning for a member of the investigation team might proceed as follows: @e Asamember of the investigation team, what are your responsibilities? e Are these responsibilities delineated in writing? @ What training have you had in the area of root cause analysis? e Briefly explain the process employed in performing an investigation. @ How are lessons learned from investigations used to help prevent recurrence of similar events? @ How does management determine the level of investigation required for a particular event? e Discuss how events are reconstructed. 59 DOE-EM-STD-5505-96 60 After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved using one or more of the following: � Interviews with additional operators and shift supervisors � Interviews with additional investigation team members � Additional interviews with the activity manager or operations supervisor � Reviews of additional investigation reports � Reviews of program and policy procedures 6.6.5 Document Review. Documents that can be reviewed to assess this area include: � Site or activity procedures and policies on investigation of abnormal events � Training and qualification requirements for personnel involved in investigating reportable events � Historical event report files � Trends and analysis of previous events � Lessons learned file � Abnormal event corrective action plans The best way to evaluate an investigation program is to observe it in action. Another effective method is to take an investigative report and associated records and review them. Items that an assessor could look for when conducting reviews include: � The list of personnel contacted during the event. Were all personnel involved in the event contacted to gain a complete picture of the event? � The root cause of the event. Does it make sense? � The overall format and content of the report. Is the report logically written, indicating that the investigation was performed in a systematic fashion? Is there evidence of management review and approval? � The thoroughness of the investigation. Did the investigator exhaust all avenues of potential DOE-EM-STD-5505-96 After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved using one or more of the following: Interviews with additional operators and shift supervisors Interviews with additional investigation team members Additional interviews with the activity manager or operations supervisor Reviews of additional investigation reports Reviews of program and policy procedures 6.6.5 Document ReviewDocuments that can be reviewed to assess this area include: Site or activity procedures and policies on investigation of abnormal events

Training and qualification requirements for personnel involved in investigating reportable events Historical event report files Trends and analysis of previous events Lessons learned file Abnormal event corrective action plans The best way to evaluate an investigation program is to observe it in action. Another effective method is to take an investigative report and associated records and review them. Items that an assessor could look for when conducting reviews include: The list of personnel contacted during the event. Were all personnel involved in the event contacted to gain a complete picture of the event? The root cause of the event. Does it make sense? The overall format and content of the report. Is the report logically written, indicating that the investigation was performed in a systematic fashion? Is there evidence of management review and approval? The thoroughness of the investigation. Did the investigator exhaust all avenues of potential 60 DOE-EM-STD-5505-96 61 information in the course of the investigation and consider multiple causes? � Indications of investigator bias as a result of preconceived ideas about what the investigation results should be. � The event history. Was a complete and accurate event history constructed as part of the investigation? By reviewing the items listed above, an accurate picture of the capability of one investigator can be drawn. This, however, may or may not be representative of the site. To get a better picture of the program as a whole, a second report done by a different investigator should be reviewed to determine if there is consistency in the overall site or activity program. This type of review can be time consuming, therefore, assessors may want to pursue only a few aspects of this review in detail. 6.7 Assessing Notifications. 6.7.1 Discussion. Notifying DOE personnel following certain events or conditions helps ensure the safety and efficiency of DOE facilities. Timely notification of appropriate DOE personnel and other agencies, when required, is employed to ensure that the facility is responsive to public health and safety concerns. For events that require notification of DOE personnel (and when appropriate, state and local officials), it is essential that information be gathered and transferred in a systematic, controlled method. Procedures that define responsibilities and provide for adequate documentation are used to control the process and ensure that the notification procedure is effective. Following notification, the DOE can trend and analyze the reported conditions in an effort to focus needed resources or direction. Additionally, central analysis allows for the dissemination of lessons learned to other facilities where similar potential exists for occurrence of such events. DOE-EM-STD-5505-96 62 The final purpose of a prompt notification system is to demonstrate responsiveness to the health and safety of the public. Since notifications is one of the first steps following a reportable incident or event, assessment of this chapter can be accomplished in conjunction with Investigation of Abnormal Events. 6.7.2 Preparing to Assess Notifications. To develop an assessment plan to assess Notifications, assessors should first review the activity-specific graded-approach document for conduct of operations and determine which guidelines are applicable at the activity. This will

allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. 6.7.3 Observation. Assessment of notifications could be performed in conjunction with a pre- planned activity drill. If, however, a drill is not performed during the course of the assessment, a walkthrough of an activity's notifications procedure could be an adequate substitute. To observe the notification process, an assessor should observe an activity drill or actual event. During these situations, the following items might be observed: � The use of a procedure or checklist by the person responsible for making the notification � Notification timeliness as required by DOE O 232.1 � The accuracy of the notification contact list Once the observation has been completed, compare what was observed to the requirements contained in activity policies and procedures to identify any apparent deviations from requirements. Pursue these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. DOE-EM-STD-5505-96 63 6.7.4 Interview. In general, activity personnel who are directly or indirectly associated with the notification process should be interviewed to verify their familiarity with the requirements of activity procedures. A list of personnel who might be interviewed includes: � Activity operators � Shift supervisors � Activity managers and operations supervisors � Occurrence Notification Center (ONC) Supervisor An example line of questioning for the activity manager or operations supervisor might proceed as follows: � Briefly describe how the notification process is executed. � As the Operations Supervisor, what role do you play in the notification process? � What training is required to properly execute your responsibilities for notification of abnormal events? � What documentation is required while performing the notification procedure? � Are there procedures that aid you in notifying the appropriate personnel? If so, how detailed is the notifying chain? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: � Interviews with several more operators � Interviews with shift supervisors � Interviews with activity managers and operations supervisors � Reviews of operating and program procedures 6.7.5 Document Review. Examples of documents that could be reviewed when assessing notifications include: DOE-EM-STD-5505-96 6.7.4 Interview In general, activity personnel who are directly or indirectly associated with the notification process should be interviewed to verify their familiarity with the requirements of activity procedures. A list of personnel who might be interviewed includes: Activity operators Shift supervisors Activity managers and operations supervisors Occurrence Notification Center (ONC) Supervisor An example line of questioning for the activity manager or operations supervisor might proceed as follows: e Briefly describe how the notification process is executed. e As the Operations Supervisor, what role do you play in the notification process?

@ What training is required to properly execute your responsibilities for notification of abnormal events? @ What documentation is required while performing the notification procedure? Are there procedures that aid you in notifying the appropriate personnel? If so, how detailed is the notifying chain? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: Interviews with several more operators Interviews with shift supervisors Interviews with activity managers and operations supervisors Reviews of operating and program procedures 6.7.5 Document ReviewExamples of documents that could be reviewed when assessing notifications include: 63 DOE-EM-STD-5505-96 64 � DOE O 232.1 � The activity programs and procedures which spell out the responsibility and authority for making notifications � Activity notification records, to verify the completeness of information as required by activity procedures and DOE O 232.1 � Filed reports, to ensure that all reportable aspects of the event were identified � Activity operating logs and records, to determine if any reportable events were not properly notified � Notification lists � Emergency plan A good starting point for reviewing documents is the site or activity procedure for making notifications related to reportable events. These documents should be checked to ensure that they provide a structured methodology that allows the proper people to be notified in a timely fashion concerning any events that warrant invoking the procedure. Some key elements to look for are: � Specific responsibilities for conducting notifications � A list of events and conditions that require notifications � Identification of who should be notified, priorities, and time requirements to be met � Definition of records that must be kept to document notifications such as reason, time accomplished, and personnel contacted Using the list of events and conditions requiring notifications to be made, a check of activity logs and records will sometimes reveal occurrences of reportable conditions that were not handled within the notification system. A check may also be made of the list of personnel to be notified to ensure that it is complete and that it contains up-to-date telephone numbers or other instructions for reaching the designated personnel. Finally, there should be a log of past notifications. A review of these documents should indicate whether notifications are DOE-EM-STD-5505-96 DOE O 232.1 The activity programs and procedures which spell out the responsibility and authority for making notifications Activity notification records, to verify the completeness of information as required by activity procedures and DOE O 232.1 Filed reports, to ensure that all reportable aspects of the event were identified Activity operating logs and records, to determine if any reportable events were not properly notified Notification lists Emergency plan A good starting point for reviewing documents is the site or activity procedure for making notifications related to reportable events. These documents should be checked to ensure that they provide a structured methodology that allows the proper people to be notified in a timely fashion concerning any events that warrant invoking the procedure. Some key elements to look for are:

Specific responsibilities for conducting notifications A list of events and conditions that require notifications Identification of who should be notified, priorities, and time requirements to be met Definition of records that must be kept to document notifications such as reason, time accomplished, and personnel contacted Using the list of events and conditions requiring notifications to be made, a check of activity logs and records will sometimes reveal occurrences of reportable conditions that were not handled within the notification system. A check may also be made of the list of personnel to be notified to ensure that it is complete and that it contains up-to-date telephone numbers or other instructions for reaching the designated personnel. Finally, there should be a log of past notifications. A review of these documents should indicate whether notifications are 64 DOE-EM-STD-5505-96 65 accomplished consistently and completely. 6.8 Assessing Control of Equipment and System Status. 6.8.1 Discussion. The purpose of control of equipment and system status is to ensure that activity operations personnel know the status of all equipment and systems and maintain appropriate control at all times. This element of conduct of operations provides an overall perspective on control of equipment and system status and discusses the use of a broad range of administrative control systems and programs such as lockouts and tagouts, equipment and system alignments, equipment deficiency identification, and work authorization. 6.8.1.1 Approach. Since the observations, interviews, and document reviews conducted to assess Control of Equipment and System Status will normally provide leads in other elements of conduct of operations, such as Lockouts and Tagouts and Independent Verification, the assessment of equipment control can be combined with these elements of conduct of operations. This "combined" approach is presented in the following sections. 6.8.2 Preparing to Assess Control of Equipment and System Status. To develop an assessment plan for Control of Equipment and System Status, assessors should first review the activity-specific graded-approach document for conduct of operations and determine which guidelines are applicable at the activity. This will allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. 6.8.3 Observation. Observations should be conducted during walkthroughs and tours with operators and shift supervisors, and during the conduct of special activities. The activity plan of the day and plan of the week should be reviewed to determine the time and place for observing DOE-EM-STD-5505-96 66 operations. The following situations are good candidates for conducting observations: � Maintenance operations that require interface or coordination with other shift positions and the use of lockouts and tagouts � Startup and shutdown of systems and equipment that require component alignment � Process or rate change operations � Installation of temporary equipment or system modifications � Equipment and system testing and return to service � Equipment reported and placed out of commission There are several aspects of the program that can be assessed during these operations. For

example, while observing the shutdown of a system or a component to conduct preventive maintenance, the following items might be observed: � Performance and documentation of a system alignment upon shutdown, prior to testing, and before return to service � Authorization, communication, and documentation of status changes from system shutdown to return to service � Preparation, review, and approval of maintenance and testing documentation � The preparation, review, approval, documentation, and placement of locks and tags � Independent verification of locks and tags, and system alignments Observing the activities and operations discussed above should be supplemented with several activity walkthroughs and tours with operators, maintenance personnel, and shift supervisors. During walkthroughs, the following additional aspects of equipment control can normally be evaluated: � Documentation of and compliance with operational limits for equipment and systems � Control of alarm status and actions taken to monitor equipment parameters when alarms are deenergized � Administrative control systems used to review, approve, document, and audit the installation DOE-EM-STD-5505-96 operations. The following situations are good candidates for conducting observations: Maintenance operations that require interface or coordination with other shift positions and the use of lockouts and tagouts Startup and shutdown of systems and equipment that require component alignment Process or rate change operations Installation of temporary equipment or system modifications Equipment and system testing and return to service Equipment reported and placed out ofntaission There are several aspects of the program that can be assessed during these operations. For example, while observing the shutdown of a system or a component to conduct preventive maintenance, the following items might be observed: Performance and documentation of a system alignment upon shutdown, prior to testing, and before return to service Authorization, communication, and documentation of status changes from system shutdown to return to service Preparation, review, and approval of maintenance and testing documentation The preparation, review, approval, documentation, and placement of locks and tags Independent verification of locks and tags, and system alignments Observing the activities and operations discussed above should be supplemented with several activity walkthroughs and tours with operators, maintenance personnel, and shift supervisors. During walkthroughs, the following additional aspects of equipment control can normally be evaluated: Documentation of and compliance with operational limits for equipment and systems Control of alarm status and actions taken to monitor equipment parameters when alarms are deenergized Administrative control systems used to review, approve, document, and audit the installation 66 DOE-EM-STD-5505-96 67 of temporary modifications � Distribution and control of equipment and system documents such as procedures, drawings, and specifications � Equipment deficiency identification, documentation, and communication � Authorization and isolation of maintenance Operators, maintenance personnel, and shift supervisors from each shift should be observed during the assessment. If possible, the observation of a particular shift position should start at shift turnover since vital information about the operating status of equipment and systems, and

activities and maintenance in progress, is passed between shifts at this time. To observe a control area operator, the following methodology could be used: � Observe the shift turnover conducted between the on-coming and off-going operators. Pay attention to the information passed verbally between the operators concerning activities planned or in progress, status of activity systems and equipment, and any abnormal conditions that exist in the activity. � Attend the shift crew briefing. Observe the information exchanged. � Observe the operator as duties are performed. Observe the items listed in section 6.8.3 as well as the interface between this operator and other shift positions, and how the operational status of activity equipment and systems is controlled. � At the end of the shift, observe the turnover process between this operator and the oncoming operator. Once the observation has been completed, compare what was observed to the requirements contained in activity policies and procedures to identify any apparent deviations from requirements. Pursue these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. A similar process can be used to observe roving operators and shift supervisors, DOE-EM-STD-5505-96 68 as well as activities and drills conducted by the activity. 6.8.4 Interview. Interviews of operations and maintenance department personnel should occur during observations and document reviews. Additional scheduled interviews should also be conducted to follow up and validate leads. The following personnel might be interviewed during the assessment: � Operators and maintenance department personnel � Shift supervisors or foremen � Activity managers and operations supervisors � Activity operations staff � Training department personnel Most of the interviews accomplished to assess equipment control should be conducted with operators, maintenance personnel, and shift supervisors. Whenever possible, these interviews should occur during observations of activities, and activity walkthroughs and tours. To pursue leads, scheduled interviews with the activity manager and training department personnel may also be necessary. For example, while observing an operator beginning at shift turnover, the following types of questions could be used to assess level of knowledge: � Describe the process used to identify, document, and communicate equipment deficiencies. Who is responsible for identifying deficiencies? How are they corrected? � What are your operational limits? How do you document compliance? � When and how are equipment and system alignments conducted? How are they documented? � What actions do you take when alarms are temporarily disabled or deenergized? � Describe the process used to review, approve, document, and audit the installation of temporary modifications. � How is a lock and tag placement accomplished? What are your responsibilities? Describe the process used to review, approve, place, and remove locks and tags. When are locks, "Danger," and "Caution" tags used? Why are they different? What information is contained DOE-EM-STD-5505-96 as well as activities and drills conducted by the activity.

6.8.4 Interview Interviews of operations and maintenance department personnel should occur during observations and document reviews. Additional scheduled interviews should also be conducted to follow up and validate leads. The following personnel might be interviewed during the assessment: @ Operators and maintenance department personnel e@ Shift supervisors or foremen e Activity managers and operations supervisors e Activity operations staff @ Training department personnel Most of the interviews accomplished to assess equipment control should be conducted with Operators, maintenance personnel, and shift supervisors. Whenever possible, these interviews should occur during observations of activities, and activity walkthroughs and tours. To pursue leads, scheduled interviews with the activity manager and training department personnel may also be necessary. For example, while observing an operator beginning at shift turnover, the following types of questions could be used to assess level of knowledge: e Describe the process used to identify, document, and communicate equipment deficiencies. Who is responsible for identifying deficiencies? How are they corrected? @ What are your operational limits? How do you document compliance? @ When and how are equipment and system alignments conducted? How are they documented? @ What actions do you take when alarms are temporarily disabled or deenergized? e Describe the process used to review, approve, document, and audit the installation of temporary modifications. @ How is a lock and tag placement accomplished? What are your resijtiasgitDescribe the process used to review, approve, place, and remove locks and tags. When are locks, “Danger,” and "Caution" tags used? Why are they different? What information is contained 68 DOE-EM-STD-5505-96 69 in the lockout and tagout log? How are periodic reviews conducted? How do you record the discrepancies that you find? Describe the process used to resolve these discrepancies. How is this documented? What training have you had concerning how to do lockouts and tagouts? How was the training conducted? Have you received any retraining? Describe the lockout and tagout notification process. � How is independent verification of lockouts and tagouts and system alignments accomplished and documented? What training have you received regarding how to do independent verification? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: � Tours and interviews with operators � Observations of and interviews with control area operators � Tours of the activity and interviews with shift supervisors � Interviews with activity managers and operations supervisors � Reviews of operating and program procedures � Interviews with training and maintenance department personnel A similar methodology to the one described above can be used to observe other activity operations and drills, considering that the list of interview candidates and sample questions provided may not be all inclusive. 6.8.5 Document Review. A list of documents that should be reviewed during the assessment includes: � Program and operating procedures � Equipment deficiency log � Equipment and system alignment checksheets DOE-EM-STD-5505-96 70 � Maintenance work requests and completed work packages

� Temporary modification control documentation � Lockout and tagout log � Shift supervisor and operator logs Most document reviews will be conducted to follow up on equipment deficiency or equipment control issues that were observed during a walkthrough or based on information gained through personnel interviews. For example, if a piece of equipment was noted to be shut down during a walkthrough, the following documents could be checked to ensure the status is being tracked in the proper fashion: � Equipment deficiency log � Lockout and tagout log � Shift turnover checklists � Maintenance work requests, and maintenance work packages 6.9 Assessing Lockouts and Tagouts. 6.9.1 Discussion. The purpose of Lockouts and Tagouts is to provide a method for equipment status control through component tagging or locking which should protect personnel from injury, protect equipment from damage, maintain operability of plant systems, and maintain the integrity of the physical boundaries of plant systems. If there is a potential for equipment damage or to inadvertent activation of equipment, a facility Lockout/Tagout should be established and used. The Lockout/Tagout program should provide for independent verification of the removal from service and the restoration to service of safety-related and other facility equipment. The Lockout/Tagout Program is intended to meet the requirements of 29 CFR 1910, as a minimum. 6.9.1.1 Approach. As discussed in Assessing Control of Equipment and System Status, DOE-EM-STD-5505-96 71 lockouts and tagouts are an integral part of equipment control. This combined approach of assessing Lockouts and Tagouts with Control of Equipment and System Status is presented in that section. Since it may be necessary to assess Lockouts and Tagouts separately, the following sections discuss how to assess this area of conduct of operations only. 6.9.2 Preparing to Assess Lockouts and Tagouts. To develop an assessment plan for Lockouts and Tagouts, assessors should first review the activity-specific graded-approach document for conduct of operations and determine which guidelines are applicable at the activity. This will allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. 6.9.3 Observation. Operations of activities available for assessment can be identified via review of the activity's plan of the day or plan of the week. These documents should list the routine operations, maintenance, and special activities that will take place during the assessment. The following situations are good candidates for observation: � Periodic and corrective maintenance of equipment and systems � System testing � Returning a system to operation � Periodic lockout and tagout reviews and inspections � Use of group lockouts and tagouts � Work performed by outside contractors or other outside service personnel There are several aspects of the program that can be assessed during these operations. For instance, while observing periodic and corrective maintenance, the following items can normally be observed: � Identification of energy sources DOE-EM-STD-5505-96 72 � Use of system prints and other documentation � Physical placement of tags and locks � Lockout and tagout log documentation

� Tag entries � Notification process Several lockouts and tagouts should be observed during the assessment. Observing the activities and operations discussed above should be supplemented with activity tours and, if necessary, walkthroughs of the approval, placement, verification, and removal of lockouts and tagouts with operators, maintenance personnel, and shift supervisors. To observe an operation that involves placing and verifying tags, the following methodology could be used: � Observe the preparation, review, and approval process for a lockout or tagout conducted by the responsible supervisor. � Observe the independent verification of the tagout. � Review the lockout or tagout record sheet, log index, and accompanying tags. � Observe an operator as the required locks or tags are hung. � Verify the personnel placing and verifying tags are properly trained and qualified. � Observe the notification process. Once the observation has been completed, compare what was observed to the requirements of the program procedure to identify any deviations from requirements. Pursue these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. A similar process can be used to observe other aspects of the program. DOE-EM-STD-5505-96 e Use of system prints and other documentation e@ Physical placement of tags and locks e@ Lockout and tagout log documentation @ Tag entries e Notification process Several lockouts and tagouts should be observed during the assessment. Observing the activities and operations discussed above should be supplemented with activity tours and, if necessary, walkthroughs of the approval, placement, verification, and removal of lockouts and tagouts with Operators, maintenance personnel, and shift supervisors. To observe an operation that involves placing and verifying tags, the following methodology could be used: @ Observe the preparation, review, and approval process for a lockout or tagout conducted by the responsible supervisor. @ Observe the independent verification of the tagout. e@ Review the lockout or tagout record sheet, log index, and accompanying tags. @ Observe an operator as the required locks or tags are hung. e@ Verify the personnel placing and verifying tags are properly trained and qualified. @ Observe the notification process. Once the observation has been completed, compare what was observed to the requirements of the program procedure to identify any deviations from requirements. Pursue these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. A similar process can be used to observe other aspects of the program. 72 DOE-EM-STD-5505-96 73 6.9.4 Interview. Interviews of operations and maintenance department personnel should occur during observations and document reviews. Additional scheduled interviews should also be conducted to follow up and validate leads. The following personnel could be interviewed during the assessment: � Operators and maintenance department personnel � Shift supervisors or foremen � Activity managers and operations supervisors � Activity operations department staff � Training department personnel Interviews conducted to assess the Lockout and Tagout Program should be aimed at determining the level of knowledge and skills of:

� Personnel who conduct maintenance and those that place, verify, and remove locks and tags, (i.e., operators and maintenance personnel) � Personnel who review, approve, and manage lockouts and tagouts, (i.e., shift supervisors, activity managers, and operations supervisors) � Personnel who work at the activity but do not review, approve, place or remove locks and tags (operations department staff) Based on the information gathered from these interviews, the assessor should determine whether adequate knowledge of the program is demonstrated by personnel at all levels of the activity. The assessor can normally conduct a cursory review of lockout and tagout training. If deficiencies appear to exist at one or more levels, a thorough review of training and other aspects of the program must be conducted to get to the root of the problem. Sample questions for interviews with operators, maintenance personnel, shift supervisors, activity managers, and operations supervisors include: � What is the purpose of the Lockout and Tagout Program? What are your responsibilities DOE-EM-STD-5505-96 74 under the program? � How do you prepare a lockout or tagout? � Describe the review and approval process. � What information is contained in the lockout and tagout log? � How is lock and tag placement accomplished? � How do you review lockouts and tagouts? How do you record the discrepancies that you find? Describe the process used to resolve these discrepancies. How is this documented? � When are locks, "Danger," and "Caution" tags used? Why are they different? � What training have you had concerning how to do lockouts and tagouts? How was the training conducted? Have you received any retraining? � Describe the lockout and tagout notification process. The following questions could be used to assess the level of knowledge of personnel who work at the activity but do not review, approve, place, or remove locks and tags: � What is the purpose of the Lockout and Tagout Program? What are your responsibilities under the program? � When are locks, "Danger," and "Caution" tags used? Why are they different? What are your responsibilities with regard to locks and tags? � What training have you had concerning lockouts and tagouts? How was the training conducted? Have you received any retraining? Finally, if apparent deficiencies exist in the level of knowledge of activity operations personnel, training department personnel should be interviewed. The following questions could be asked to help evaluate lockout and tagout training: � How is lockout and tagout training accomplished? Is retraining accomplished? What is the periodicity for conducting training? � Describe the process used to determine the level of training required for operations personnel. DOE-EM-STD-5505-96 under the program? How do you prepare a lockout or tagout? Describe the review and approval process. What information is contained in the lockout and tagout log? How is lock and tag placement accomplished? How do you review lockouts and tagouts? How do you record the discrepancies that you find? Describe the process used to resolve these discrepancies. How is this documented? When are locks, "Danger," and "Caution" tags used? Why are they different? What training have you had concerning how to do lockouts and tagouts? How was the training conducted? Have you received any retraining? Describe the lockout and tagout notification process.

The following questions could be used to assess the level of knowledge of personnel who work at the activity but do not review, approve, place, or remove locks and tags: What is the purpose of the Lockout and Tagout Program? What are your responsibilities under the program? When are locks, "Danger," and "Caution" tags used? Why are they different? What are your responsibilities with regard to locks and tags? What training have you had concerning lockouts and tagouts? How was the training conducted? Have you received any retraining? Finally, if apparent deficiencies exist in the level of knowledge of activity operations personnel, training department personnel should be interviewed. The following questions could be asked to help evaluate lockout and tagout training: How is lockout and tagout training accomplished? Is retraining accomplished? What is the periodicity for conducting training? Describe the process used to determine the level of training required for operations personnel. 14 DOE-EM-STD-5505-96 75 � How do you ensure that the required level of knowledge is obtained by operations personnel as a result of this training? � Are deficiencies discovered during lockout and tagout reviews factored into training? Describe this process. � How is the training documented? � Describe the process used to ensure that the appropriate personnel are retrained. The above list of interview candidates and questions is not all inclusive. Assessors must use additional interviews, observations, and document reviews as necessary to thoroughly investigate any apparent deviations, and determine if programmatic breakdowns or widespread problems exist. Note that an assessor may be assigned to training. If it is necessary to assess lock and tag training, these interviews would be accomplished by the training assessor. 6.9.5 Document Review. Examples of documents that may be reviewed when assessing lockouts and tagouts include: � Lockout and tagout program procedures � Active and completed lockout and tagout record sheets � "Danger" and "Caution" tags � Lockout and tagout authorization log index � Key control documentation, and � Training records Periodic reviews of the lockout and tagout program should be conducted by the activity, and evidence of the process should be apparent in the lockout and tagout program documentation. Checking for evidence of a review program, observing a review in process, or personally performing part or all of a review is a good technique for an assessor to use to obtain an accurate picture of the activity's program. DOE-EM-STD-5505-96 76 The following is a general description of how an activity might conduct a review of the lockout and tagout program. The review should start with the index or log. A check is usually made to verify that there is a tagout sheet for every active entry in the index. If there is not a one to one correlation, then a tagout sheet is missing or was cleared and not removed from the index. Secondly, each tagout sheet should be assigned to an operator qualified to participate in the lockout and tagout program. The operator should walk through every item on the tagout record sheet and verify that all locks or tags are still hanging on the proper device. When the operator completes the review it should be documented on the record sheet. Any discrepancies should be noted along with the corrective action that was taken. When all tagout sheets have been

reviewed, an entry should be made in the tagout index to record the review and summarize any discrepancies that were found and then corrected. The following is an example of how a portion of the review process can be conducted by an assessor. A review of a specific lockout or tagout could be done by starting with either the lockout and tagout log or with the locks and tags that have been placed. The first method is described below: � Select a lockout or tagout record sheet from the log. � Review the record sheet and log index. Check for periodic audits and reviews to determine whether they are conducted, and if the discrepancies noted are documented and corrected. � Walk through the activity and locate each lock or tag indicated on the record sheet. � Check that each lock and tag is properly placed, and that all required information on the tags is legible and complete. Compare the information on the record sheet and the tag. Once the review has been completed, compare what was observed to the requirements of the program procedure to identify any leads. DOE-EM-STD-5505-96 77 6.10 Assessing Independent Verification. 6.10.1 Discussion. Independent Verification is the act of checking that a given operation conforms to established operational criteria, as well as checking a component position independent of activities related to establishing the component's position. A comprehensive Independent Verification Program will identify components to be included in the program, define when Independent Verification is required, and prescribe the methods to be used when performing Independent Verification. If implemented effectively, Independent Verification reduces the chance that human error will cause an operational failure. Independent auditing of operations should confirm that established operational requirements are met. 6.10.1.1 Approach. As discussed in Assessing Control of Equipment and System Status, Independent Verification is an integral part of equipment control. This combined approach of assessing Independent Verification with Control of Equipment and System Status is presented in that section. Since it may be necessary to assess Independent Verification separately, the following sections discuss how this could be accomplished. Independent verifications are conducted in a manner such that each check constitutes an actual identification of the component and a determination of both its required and actual positions. Checks are performed minimizing interaction between the personnel operating components and those performing the independent verifications. 6.10.2 Preparing to Assess Independent Verification. To develop an assessment plan for Independent Verification, assessors should first review the activity-specific graded-approach document for conduct of operations and determine which guidelines are applicable at the activity. This will allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment DOE-EM-STD-5505-96 78 techniques. 6.10.3 Observation. The following special activities and operations could demonstrate Independent Verification: � The alignment of valves and switches in preparation for system startup, maintenance, testing, and prior to return to service

� Verifying the status of a system or component which is in doubt Some of these events will be listed in the activity plan of the day or plan of the week. Other situations that require Independent Verification will be event driven. If an Independent Verification is not scheduled to occur during the assessment, the assessor should ask the activity manager to arrange a demonstration or a walkthrough of the procedure. When observing Independent Verification, the assessor should see some form of time-distance spacing between the two verifiers. Remember that an independent check is required. Time spacing means that the two verifiers should not check a component at the same time. Distance spacing means that neither verifier should be able to see the other check the component. To observe two operators conducting a system alignment that requires Independent Verification, the following methodology could be used: � Accompany the first operator as he aligns the system as listed on the checksheet. Observe the use of the checksheet and the operator's skills and abilities concerning locating and operating system components. � Once the system has been aligned by the first operator, observe any communication between the first and second operators, and follow the second operator as he verifies the position of system components. Pay particular attention to the use of the checksheet, the operator's skills and abilities concerning locating system components and verifying their component position, and the independence of the second check from the original alignment. DOE-EM-STD-5505-96 79 Once the observation has been completed, compare what was observed to the requirements contained in activity policies and procedures to identify any apparent deviations from requirements. Pursue these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. A similar process can be used to observe other activities that require Independent Verification such as the placement of a lockout or tagout. 6.10.4 Interview. Interviews of operations and maintenance department personnel should occur during observations and document reviews. Additional scheduled interviews should also be conducted to follow up and clarify leads. The following personnel might be interviewed during the assessment: � Operators who are qualified to conduct Independent Verification � Shift supervisors � Activity managers and operations supervisors Most of the interviews accomplished to assess Independent Verification should be conducted with operators, maintenance personnel, and shift supervisors. Whenever possible, these interviews should occur during observations of operations, and activity walkthroughs and tours. To pursue leads, scheduled interviews with the activity manager and training and maintenance department personnel may also be necessary. For example, the following questions could be used to evaluate an operator's level of knowledge regarding the program: � When were you trained on how to conduct Independent Verification? � When are you required to conduct Independent Verification? � How do you perform Independent Verification? � How do you independently verify a throttle valve position? � How are you evaluated on conducting Independent Verification? DOE-EM-STD-5505-96 80 The following question could be used to interview managers:

� How do you train operators in Independent Verification? � When do you require Independent Verification be conducted? � How would you independently verify a throttle valve? � How do you ensure that an operator properly performs Independent Verification? � What systems require Independent Verification? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: � Tours with several more operators � Tours of the activity and interviews with shift supervisors � Interviews with activity managers and operations supervisors � Reviews of operating and program procedures � Interviews with training and maintenance department personnel 6.10.5 Document Review. A list of documents to be reviewed during the assessment would include: � Independent Verification program procedures � Operating logs and system lineup checksheets � Quality assurance plan Assessors should first determine when and on what systems Independent Verification is required by reviewing the appropriate administrative procedure, and then review activity logs to find events that require Independent Verification. If documentation of Independent Verification is not required, some other technique for checking that verification has been completed will have to be used (i.e., interviews). DOE-EM-STD-5505-96 The following question could be used to interview managers: @ How do you train operators in Independent Verification? @ When do you require Independent Verification be conducted? @ How would you independently verify a throttle valve? e@ How do you ensure that an operator properly performs Independent Verification? @ What systems require Independent Verification? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: @ Tours with several more operators @ Tours of the activity and interviews with shift supervisors e Interviews with activity managers and operations supervisors e Reviews of operating and program procedures e Interviews with training and maintenance department personnel 6.10.5 Document ReviewA list of documents to be reviewed during the assessment would include: @ Independent Verification program procedures @ Operating logs and system lineup checksheets ® Quality assurance plan Assessors should first determine when and on what systems Independent Verification is required by reviewing the appropriate administrative procedure, and then review activity logs to find events that require Independent Verification. If documentation of Independent Verification is not required, some other technique for checking that verification has been completed will have to be used (i.e., interviews). 80 DOE-EM-STD-5505-96 81 Another area to investigate is Independent Verification training. The activity should have a reference that instructs personnel on how to verify the position of various activity components. In addition, personnel should receive training on how to perform Independent Verification. Training records are usually held by a supervisor or by the training department. Note that an assessor may be assigned to training. If it is necessary to assess Independent Verification training, these

interviews may be accomplished by the training assessor. 6.11 Assessing Logkeeping. 6.11.1 Discussion. The purpose of logkeeping is to record activity status and events as required to provide an accurate history of activity operations. A log is defined as "a narrative sequence of events or functions performed at a specific shift position." This limited standard, on the other hand, does not attempt to differentiate between narrative records maintained by various shift positions, and records such as Resource Conservation Recovery Act (RCRA) or operator round/tour inspection sheets, radiological survey records, equipment deficiency logs, and lockout and tagout logs. There are two reasons for this: (1) the guidelines in each area of conduct of operations address fundamental principles such as timeliness of recordings and legibility that should be applied to the maintenance of inspection sheets and other records as well as narrative logs, and, (2) round/tour inspection sheets and other non-narrative records frequently contain a narrative section, making differentiation between logs and inspection sheets unnecessary. 6.11.1.1 Approach. As discussed in Assessing Shift Routines and Operating Practices, Logkeeping is an integral part of shift routines. This combined approach of assessing Logkeeping with shift routines is presented in that section. Since it may be necessary to assess Logkeeping separately, the following sections discuss how to assess this area of conduct of operations only. DOE-EM-STD-5505-96 82 6.11.2 Preparing to Assess Logkeeping. To develop an assessment plan for Logkeeping, assessors should first review the activity-specific graded-approach document for conduct of operations and determine which guidelines are applicable at the activity. This will allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. 6.11.3 Observation. Observations of Logkeeping should be conducted during walkthroughs and tours of the activity with operators and shift supervisors, and during the conduct of special activities and drills. The following situations are good candidates for conducting observations: � Maintenance operations that require interface or coordination with surveillance operators, control area operators, and shift supervisors � Startup and shutdown of systems and equipment � Process or rate change operations � Activity drills There are several aspects of logkeeping that can be assessed during these operations. For example, while observing an operator as he performs a system or component startup, the following items might be observed: � Timeliness of entries � The type, scope, and format of entries made to document the startup by all shift positions involved in the procedure � Legibility of entries and method used to make corrections � Consistency in the use of logs and information recorded between shift positions � Periodic supervisory review Observing the activities and operations discussed above should be supplemented with several activity walkthroughs and tours with operators and shift supervisors. During walkthroughs, all DOE-EM-STD-5505-96 83 logs maintained by operators, maintenance personnel, and shift supervisors should be assessed.

While touring through the activity with an operator, the following methodology could be used: � During the tour, observe the timeliness, legibility, etc. of the entries made in the round/tour inspection sheet. � Review any operational records that are kept by the activity. Examples include the control area and shift supervisor's logs, equipment deficiency logs, the lockout and tagout log, and radiological survey logs and records. Once the observation has been completed, compare what was observed to the requirements contained in activity policies and procedures to identify any apparent deviations from requirements. Pursue these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. 6.11.4 Interview. Most interviews of operations and maintenance department personnel should occur during observations and document reviews. Additional scheduled interviews should also be conducted to follow up and clarify leads. The assessment plan should identify the initial interviews necessary to start the assessment. The following personnel might be interviewed during the assessment: � Operators and maintenance department personnel � Shift supervisors or foremen � Activity managers and operations supervisors � Radiological control technicians, and � Activity operations staff While observing a shift supervisor or worker, the following types of questions could be used to assess his level of knowledge about logkeeping: DOE-EM-STD-5505-96 84 � What types of information do you record in your log? What guidance have you been given on the type, scope, and format of log entries? � How do you correct log errors? Is this method used by all shift positions? � Do you review any logs that are maintained by other shift positions? If so, how do you conduct and document these reviews? Are logs reviewed by activity management? How are these reviews conducted? � Where are completed logs kept? Describe the process used to retrieve completed logs. The information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: � Walkthroughs and tours with operators � Observations of and interviews with control area operators � Additional tours of the activity and interviews with shift supervisors � Interviews with radiological control technicians � Interviews with activity managers and operations supervisors � Reviews of operating and program procedures 6.11.5 Document Review. A list of documents to be reviewed during the assessment should be included in the assessment plan. Examples of documents that should be reviewed include: � Program and operating procedures � Shift supervisor and operator logs and round/tour inspection sheets � Radiological survey records � Shift turnover checklists � Lockout and tagout logs � Equipment deficiency logs � Maintenance work request logs � Occurrence reports DOE-EM-STD-5505-96 What types of information do you record in your log? What guidance have you been given on the type, scope, and format of log entries? How do you correct log errors? Is this method used by all shift positions? Do you review any logs that are maintained by other shift positions? If so, how do you conduct and document these reviews? Are logs reviewed by activity management? How are these reviews conducted?

Where are completed logs kept? Describe the process used to retrieve completed logs. The information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: Walkthroughs and tours with operators Observations of and interviews with control area operators Additional tours of the activity and interviews with shift supervisors Interviews with radiological control technicians Interviews with activity managers and operations supervisors Reviews of operating and program procedures 6.11.5 Document ReviewA list of documents to be reviewed during the assessment should be included in the assessment plan. Examples of documents that should be reviewed include: Program and operating procedures Shift supervisor and operator logs and round/tour inspection sheets Radiological survey records Shift turnover checklists Lockout and tagout logs Equipment deficiency logs Maintenance work request logs Occurrence reports 84 DOE-EM-STD-5505-96 85 A correlation of information recorded in different places during the same period may prove useful to identify communication or teamwork problems. During assessment planning, the administrative procedure for logkeeping should be reviewed to gain familiarity with site requirements. This background knowledge is a necessary prerequisite to making an intelligent evaluation of the logkeeping program. The remainder of the document reviews necessary to assess logkeeping can be conducted in conjunction with observations. For example, while touring the activity with an operator, a review of documents such as shift supervisor and operator logs, round/tour inspection sheets, operations procedures, radiological surveys, lockout and tagout logs, and equipment deficiency logs can normally be accomplished. The individual logs and round sheets should be examined to ensure that the elements required by administrative procedures are contained in the various documents and that they provide enough information to accurately document the history of activity operations. In another example, a return to an area may be made to make a correlation of several related logs and records to ensure they are properly prepared and meet requirements when taken as a whole. This often yields a larger view of the organization and may point to teamwork or communication problems. For example, groups often log conflicts with or failures to properly work with other groups as a way of covering their own situation. These sorts of entries in the records should prompt follow up. In another example of an assessment technique, a review of a log or record for an extended period of time often gives insight to other types of problems. If it is difficult to get something repaired or corrected, or if there is sufficient support or follow up for deficiencies or inoperable equipment, it is often recorded in the logs where the appeal to make corrections was documented. Such appeals, if repeated, often signal other problems that warrant attention. DOE-EM-STD-5505-96 86 6.12 Assessing Operations Turnover. 6.12.1 Discussion. Operations turnover provides on-coming operators and supervisors with an accurate picture of the overall status of the activity. The turnover process should include a comprehensive review of appropriate written and visual information, be guided by a checklist, and complemented by a discussion between the off-going and on-coming shift personnel. Although

the guidelines in this section specifically address operations turnover at facilities that operate 24 hours a day, the spirit and intent of the guidance should be followed by activity management at facilities that operate on a single-shift basis. In many cases, the turnover in a day-to-day operation is more difficult than that encountered in 24-hour shift operations. An example is often observed in activities where the person who normally initiates the daily activities is unavailable and an alternate person feels the need to initiate the activity. This is common when starting up laboratories or laboratory equipment and for sampling or inspection rounds of permitted storage. Breakdowns of turnover information may be of considerable concern in such cases. (In these cases, there is often no continuity of turnover) 6.12.1.1 Approach. As discussed in Assessing Shift Routines and Operating Practices, Operations Turnover is an integral part of shift routines. This combined approach of assessing shift turnover with shift routines is presented in that section. Since it may be necessary to assess Operations Turnover separately, the following sections discuss how to assess this area of conduct of operations only. 6.12.2 Preparing to Assess Operations Turnover. To develop an assessment plan for Operations Turnover, assessors should first review the activity-specific graded-approach document for conduct of operations and determine which guidelines are applicable at the activity. This will allow them to determine the scope of the assessment and which applicable DOE-EM-STD-5505-96 87 activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. 6.12.3 Observation. Observations of Operations Turnover should be conducted during walkthroughs and tours of the activity with operators, maintenance personnel, radiological control technicians, and shift supervisors. If possible, personnel from each group or shift should be observed during the assessment. To observe the shift turnover process at an activity, the following methodology could be used: � Observe the shift turnover conducted between an on-coming and off-going operator. Pay attention to the information passed verbally between the operators concerning activities planned or in progress, status of activity systems and equipment, and any abnormal conditions that exist in the activity. Review any narrative logs, round/ tour inspection sheets, and any other documentation used in the turnover process. � Attend the shift crew briefing. Observe the process used and the information exchanged. Compare the information disseminated at the briefing with the information that was passed between the operators during their turnover. � Accompany the operator as he performs his duties. Look for any activities or operations in progress that were not discussed or briefed at turnover. Also pay attention to the status of equipment and systems to identify any differences between what was briefed and what you observe. � At the end of the shift, observe the turnover process between this operator and the on-coming operator. Once the observation has been completed, compare what was observed to the requirements contained in activity policies and procedures to identify any apparent deviations from requirements. Pursue these leads using additional observations, interviews, and document

DOE-EM-STD-5505-96 88 reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. A similar process can be used to observe control area operators, shift supervisors, maintenance personnel, and radiological control technicians conducting operations turnover. 6.12.4 Interview. Interviews of operations and maintenance department personnel should occur during observations and document reviews. Additional scheduled interviews should also be conducted to follow up and clarify leads. The following personnel might be interviewed during the assessment: � Operators and maintenance department personnel � Shift supervisors or foremen � Activity managers and operations supervisors � Radiological control technicians � Activity operations staff � Training department personnel Most of the interviews accomplished to assess operations turnover should be conducted with operators, maintenance personnel, and shift supervisors. Whenever possible, these interviews should occur during observations of shift turnover, activity operations and activities, and during activity walkthroughs and tours. To pursue leads, scheduled interviews with the activity manager and training department personnel may also be necessary. For example, while observing a roving operator beginning at turnover, the following types of questions could be used to assess the effectiveness of the shift turnover process: � What guidance have you received from the shift supervisor or activity manager concerning how to conduct shift turnover? Do you feel that the process used at turnover supports your needs? � How do you determine what information should be passed at turnover to the on-coming operator? Are you required to use a turnover check sheet to aid the process? If so, what type DOE-EM-STD-5505-96 89 of information is recorded on the check sheet? How is this information used during the shift? � What documents do you review during shift turnover? How does the turnover check sheet aid in the document review process? � Are there any control panels that you walk through during shift turnover? If so, how do you do this and what information does it provide you? � Do you conduct a verbal turnover to the on-coming operator? What type of information do you pass? � Is a shift crew briefing conducted as part of the turnover process? When is it conducted? What type of information is passed during the brief? Who attends the brief? � How do you conduct turnover for reliefs that occur during the shift? What type of information do you pass to your relief? What guidance have you received from activity management concerning reliefs during the shift? The information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: � Observations of and interviews with several more operators � Tours of the activity and interviews with shift supervisors � Interviews with radiological control technicians � Interviews with activity managers and operations supervisors � Reviews of operating and program procedures � Interviews with training department personnel A similar methodology to the one described above can be used to observe and interview control area operators, maintenance personnel. radiological control technicians, and shift supervisors.

6.12.5 Document Review. Examples of documents that should be reviewed include: � Program and operating procedures DOE-EM-STD-5505-96 of information is recorded on the check sheet? How is this information used during the shift? What documents do you review during shift turnover? How does the turnover check sheet aid in the document review process? Are there any control panels that you walk through during shift turnover? If so, how do you do this and what information does it provide you? Do you conduct a verbal turnover to the on-coming operator? What type of information do you pass? Is a shift crew briefing conducted as part of the turnover process? When is it conducted? What type of information is passed during the brief? Who attends the brief? How do you conduct turnover for reliefs that occur during the shift? What type of information do you pass to your relief? What guidance have you received from activity management concerning reliefs during the shift? The information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: Observations of and interviews with several more operators Tours of the activity and interviews with shift supervisors Interviews with radiological control technicians Interviews with activity managers and operations supervisors Reviews of operating and program procedures Interviews with training department personnel A similar methodology to the one described above can be used to observe and interview control area operators, maintenance personnel. radiological control technicians, and shift supervisors. 6.12.5 Document ReviewExamples of documents that should be reviewed include: Program and operating procedures 89 DOE-EM-STD-5505-96 90 � Operating logs and round/tour inspection sheets � Shift turnover checksheets The site or activity administrative procedures should be reviewed as part of pre-assessment preparations to identify the actions which must be completed to conduct a proper operations turnover. The remainder of the document reviews necessary to assess operations turnover can be conducted in conjunction with observations and interviews. For example, while observing a turnover between shift supervisors, a review of documents such as operating logs and turnover checklists can be conducted. 6.13 Assessing Operations Aspects of Facility Chemistry and Unique Processes. 6.13.1 Discussion. This section describes the operational monitoring of activity chemistry or unique process data to ensure that parameters are maintained. The term "unique processes" includes all parameters that require monitoring for process control including chemistry parameters. The guidelines of this section use chemistry as an example. The requirements for activity-specific process control policies may be detailed in one or more administrative procedures. It is expected that operators understand the parameters that are recorded, how process changes affect those parameters, and the actions to be taken to correct process parameters when they approach an out of specification condition. When matrixed organizations, such as chemistry labs, are responsible for measuring process control parameters, operations personnel should ensure that sample results are received in a timely manner so that they can be applied for process control.

6.13.1.1 Approach. As discussed in Assessing Shift Routines and Operating Practices, Operations Aspects of Facility Chemistry and Unique Processes are an integral part of shift routines. This "combined" approach of assessing Operations Aspects of Facility Chemistry and DOE-EM-STD-5505-96 91 Unique Processes with shift routines is presented in that section. Since it may be necessary to assess Operations Aspects of Facility Chemistry and Unique Processes separately, the following sections discuss how to assess Operations Aspects of Facility Chemistry and Unique Processes only. 6.13.2 Preparing to Assess Operations Aspects of Facility Chemistry and Unique Processes. To develop an assessment plan for this area of conduct of operations, assessors should first review the activity-specific graded-approach document for conduct of operations and determine which guidelines are applicable at the activity. This will allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. Drills are particularly effective tools in the assessment of this area. At least, they provide the observations and indications from which the assessors can pursue more fundamental information. Activity and site organization as well as the activity's purpose will play a large role in assessment planning. Assessors should determine if and where matrixed organizations interface with operations personnel and how those organizations affect process control. The activity description and tour should provide an assessor with an understanding of the activity process. When the assessment begins, noticing which indications, such as gauges, meters, or sight glasses, are recorded on round sheets will provide a good starting point. 6.13.3 Observation. The activity's plan of the week or plan of the day should be reviewed to determine if there are major processes occurring that will require the monitoring of parameters. The following are sources of possible observations: � An operator recording and acting on process parameters DOE-EM-STD-5505-96 92 � A technician monitoring parameters or taking samples � An operator performing maintenance that will cause an alarm condition Operators and technicians from each shift should be observed during the assessment. If possible, the observation of a particular shift position should start at shift turnover since vital information about the status of activity processes is passed at this time. An effective way to observe operators is to follow them during their inspection tours. If they are taking round sheets, observe what parameters are monitored and the parameter normal, high, and low values recorded on the round sheet. If the actual parameter value is outside the high/low band, observe what actions the operator takes, if any. If a maintenance action or operator will cause an expected alarm, then are the appropriate operations personnel informed prior to receipt of the alarm? Because matrixed technicians are outside the operations organization, observing the communication of sample results or parameter monitoring will be required. Once again, an effective means of conducting the observation is to contact the matrixed organization, and arrange to follow a technician while sampling or monitoring operations. Observe the technician take the

sample or monitor the parameter as before. If the sample requires analysis to obtain a parameter, observe if the results are returned to the operations organization in time to have meaningful process control. Once an observation has been completed, compare what was observed to the requirements contained in activity policies and procedures to identify any apparent deviations from requirements. Pursuing these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. DOE-EM-STD-5505-96 93 6.13.4 Interview. Interviews of operations personnel should occur during observations and document reviews. Additional scheduled interviews should also be conducted to follow up and clarify leads. The following personnel may be subject to interviews when assessing this area of conduct of operations: � Activity operators � Matrixed technicians � Shift supervisors � Matrixed organization supervisors � Activity managers and operations supervisors The primary focus of interviews should be to determine the level of technical knowledge that is required by each area of conduct of operations, and a working knowledge of the administrative requirements. For example, the following general questions could be asked of operators: � What are the general requirements when a parameter leaves its normal control band? � Have you ever received data from a matrixed organization (such as a chemist) on which you could not have reacted in a timely manner? � Do you conduct trend analysis? (i.e., look at recorded parameter data to understand why it changed and determine if the trend predicts the parameter leaving the normal band). � Have you ever received an alarm that was caused by maintenance? Were you informed prior to receiving that alarm? General questions to ask matrixed technicians are: � Who do you inform when you are taking samples or monitoring parameters at the activity? � How do you ensure that samples taken at multiple locations are not switched? � Are you required to inform anyone when complete sampling? Who? � Are you required to inform the activity of the results? How? � What are you required to do prior to performing work that will cause an alarm condition? Interviews with operators and technicians are most effectively conducted in conjunction with DOE-EM-STD-5505-96 6.13.4 Interview Interviews of operations personnel should occur during observations and document reviews. Additional scheduled interviews should also be conducted to follow up and clarify leads. The following personnel may be subject to interviews when assessing this area of conduct of operations: Activity operators Matrixed technicians Shift supervisors Matrixed organization supervisors Activity managers and operations supervisors The primary focus of interviews should be to determine the level of technical knowledge that is required by each area of conduct of operations, and a working knowledge of the administrative requirements. For example, the following general questions could be asked of operators: What are the general requirements when a parameter leaves its normal control band? Have you ever received data from a matrixed organization (Such as a chemist) on which you could not have reacted in a timely manner? Do you conduct trend analysis? (i.e., look at recorded parameter data to understand why it changed and determine if the trend predicts the parameter leaving the normal band).

Have you ever received an alarm that was caused by maintenance? Were you informed prior to receiving that alarm? General questions to ask matrixed technicians are: Who do you inform when you are taking samples or monitoring parameters at the activity? How do you ensure that samples taken at multiple locations are not switched? Are you required to inform anyone when complete sampling? Who? Are you required to inform the activity of the results? How? What are you required to do prior to performing work that will cause an alarm condition? Interviews with operators and technicians are most effectively conducted in conjunction with 93 DOE-EM-STD-5505-96 94 observations. Interviews with shift supervisors, activity managers, and operations supervisors could include the following questions: � Can you remember encountering a process problem that could have been prevented if sample results had been received earlier? � Do you expect your operators to conduct trend analysis? � How do you conduct trend analysis? � How do you ensure that operators understand the activity process? � What actions do you expect operators to take when an out of specification parameter is discovered? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: � Tours with several more operators � Tours of the activity and interviews with shift supervisors � Interviews with technicians � Interviews with activity managers and operations supervisors � Reviews of operating and program procedures � Interviews with training department personnel 6.13.5 Document Review. A list of documents to be reviewed during the assessment should be included in the assessment plan. Some will be reviewed during assessment planning; others will be reviewed only if pursuing leads requires that they be examined. The list of documents includes: � Emergency preparedness documents and safety analyses � Various administrative policies that detail operator responsibilities for trend analysis, out-of- specification-parameter actions, activity-specific process training, etc. � Various key shift position roundsheets and logs DOE-EM-STD-5505-96 observations. Interviews with shift supervisors, activity managers, and operations supervisors could include the following questions: Can you remember encountering a process problem that could have been prevented if sample results had been received earlier? Do you expect your operators to conduct trend analysis? How do you conduct trend analysis? How do you ensure that operators understand the activity process? What actions do you expect operators to take when an out of specification parameter is discovered? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: Tours with several more operators Tours of the activity and interviews with shift supervisors Interviews with technicians Interviews with activity managers and operations supervisors Reviews of operating and program procedures Interviews with training department personnel 6.13.5 Document ReviewA list of documents to be reviewed during the assessment should

be included in the assessment plan. Some will be reviewed during assessment planning; others will be reviewed only if pursuing leads requires that they be examined. The list of documents includes: Emergency preparedness documents and safety analyses Various administrative policies that detail operator responsibilities for trend analysis, out-of- specification-parameter actions, activity-specific process training, etc. Various key shift position roundsheets and logs 94 DOE-EM-STD-5505-96 95 � Chemistry Laboratory Quality Assurance/Quality Control Plan � Any procedure with hold points for which a matrixed organization is responsible As noted in the previous list, the required documents for this area of conduct of operations may be distributed among many activity and site administrative policies. The documents to be reviewed during assessment planning are the activity or site administrative policies and procedures. Most of the matrixed organization policies, including QA/QC plans, will not be examined unless pursuing leads draws an assessor into those documents. This area is one where the review of safety analyses and the emergency preparedness documentation prepared for the management of unusual situations is applicable. The quality of that documentation is a reflection of the preparation to deal with off-normal situations, including those imposed by adjacent activities. An examination of roundsheets where parameters are recorded should be informative. Assessors should identify any out of specification parameters which have been noted, and if any action was taken as a result. Also, assessors should look for trends where the parameter is close to being out of specification and appears to require action to avoid an out of specification condition. The chemistry laboratory QA/QC plan or similar document may provide information on how samples are tracked including chain of custody procedures. If an activity has a history of receiving lab results late, especially if the process continued to operate with chemistry out of specification, it may be necessary to examine the lab's receipt logs and chain of custody forms in an attempt to identify points of delay. The method used to forward lab results to the activity should also be examined. A problem may exist concerning the switching or confusion of samples. If the assessor understands the process, then it is possible to review sample results and determine if they make sense. DOE-EM-STD-5505-96 96 Finally, administrative policies should be examined to ensure that personnel responsibilities are clearly defined regarding monitoring process activities and taking corrective action for abnormal conditions. In addition, personnel training material could be reviewed to ensure these responsibilities and required actions are covered in the curriculum. 6.14 Assessing Required Reading. 6.14.1 Discussion. The Required Reading Program provides a means to communicate information to operations personnel that is related to their job assignment. The program is intended for management's use to ensure that operators receive current and appropriate job assignment information. One would expect management to use this system to provide information that operators might receive in recurring training, but requiring operators to read all DOE Directives is not what is intended by this area of conduct of operations. Large volumes of

reading will dilute the significance of important information and most likely will preclude the reading of all assignments by some personnel. Some examples of what should be included in the required reading program are : � Lessons Learned from the DOE complex � Industry standards � Company memos that are of concern to operations personnel but have not been forwarded to them � Operations procedure change summaries (not the complete, revised operating procedure) The items that are included in the required reading program should be listed in an administrative policy or procedure. 6.14.2 Preparing to Assess Required Reading. To develop an assessment plan for Required Reading, assessors should first review the activity-specific graded-approach document for conduct of operations and determine which guidelines are applicable at the activity. This will DOE-EM-STD-5505-96 97 allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques The use of observation in the assessment of required reading programs is of limited use. Normally, assessors will conduct document reviews followed by interviews to evaluate program effectiveness. Interviews will establish where programmatic breakdowns are occurring while document reviews determine that problems exist. 6.14.3 Observation. The use of observation as a technique for assessing required reading has limited value. Observations can be used to confirm requirements noted in document reviews and interviews. Shift turnover, for instance, could be observed to determine what action was taken by management to ensure completion of required reading. 6.14.4 Interview. Assessment planning should consider completing enough interviews to determine what operators think of the required reading program, if managers understand the purpose of required reading, and how the program is administered. There is no specification on who is responsible for administering the program; therefore, each activity could have a different person responsible. The following personnel could be interviewed: � Operations personnel participating in the program � Shift supervisors and foremen � Activity mangers � Required reading administrator (varies from activity to activity but should be specified in the administrative procedure or policy) The purpose of interviews is to establish if the program is understood and used by managers, administered according to procedure, and respected by operators. The following questions could be asked of activity managers, operations supervisors, shift supervisors, or the program DOE-EM-STD-5505-96 98 administrator: � What criteria is used to determine what information is included in the required reading binder? � Who is responsible for ensuring that required reading is completed by the assigned due date? � Who is responsible for removing completed reading and the final disposition of documentation? � How do you ensure that items you want included in the readings are assigned? (directed to activity managers who have delegated the program responsibility to a program administrator) � How do you ensure 100-percent completion of reading assignments by operators? The following questions could be addressed to operators to determine their level of knowledge

regarding the required reading program: � How often do you review the required reading file for assignments? � Do you think the program is valuable? Why or why not? � Tell me about the content of the material from your last reading? � Tell me what you remember about--(a specific document which the person has recently signed off as complete) A particularly effective method for assessing the quality of required reading programs is for the assessor to gain familiarity with several documents included (or that should be included) in the program and then question the personnel administering and using the required reading program concerning their knowledge of those specific documents. This will give information on the process from beginning to end and can be used to assess not only the mechanics of the program, but also the quality and effectiveness of the overall program objectives, or how effectively the program gets pertinent information to the workers and supervisors in a timely manner. After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be DOE-EM-STD-5505-96 99 validated or disproved by using one or more of the following: � Interviews with several more operators � Interviews with shift supervisors � Interviews with activity managers and operations supervisors � Reviews of program procedures � Interviews with training and maintenance department personnel 6.14.5 Document Review. Document review provides a good starting point for required reading assessment. Typically, assessors should review: � The administrative procedure or policy � Reading assignment sheets (there could be a cover sheet for each reading or a centralized assignment sheet in the front of the binder) � Reading assignment file � Retained documentation and dead file (if required) Identification of program components can be accomplished during assessment planning and can be found in the administrative procedure or policy for the activity. Review of the previous list of documents will allow an assessor to become familiar with how the program is administered. From the administrative procedure assessors should determine: � Who is responsible for administering the required reading program and who actually assigns readings � How does management review the program � How are documents screened to ensure appropriate information is directed to the right operations personnel Assessors should verify that these actions are actually performed during subsequent document reviews and interviews. When reviewing the reading assignment sheet, or its equivalent, DOE-EM-STD-5505-96 validated or disproved by using one or more of the following: Interviews with several more operators Interviews with shift supervisors Interviews with activity managers and operations supervisors Reviews of program procedures Interviews with training and maintenance department personnel 6.14.5 Document ReviewDocument review provides a good starting point for required reading assessment. Typically, assessors should review: The administrative procedure or policy Reading assignment sheets (there could be a cover sheet for each reading or a centralized assignment sheet in the front of the binder) Reading assignment file Retained documentation and dead file (if required) Identification of program components can be accomplished during assessment planning and can be

found in the administrative procedure or policy for the activity. Review of the previous list of documents will allow an assessor to become familiar with how the program is administered. From the administrative procedure assessors should determine: Who is responsible for administering the required reading program and who actually assigns readings How does management review the program How are documents screened to ensure appropriate information is directed to the right Operations personnel Assessors should verify that these actions are actually performed during subsequent document reviews and interviews. When reviewing the reading assignment sheet, or its equivalent, 99 DOE-EM-STD-5505-96 100 assessors should look for the following information: � Over a selected period, (one month is good but could vary depending on the size of the program) determine how many assignments have been completed by the due date � Entries are in chronological order and have due dates assigned � All operation personnel are included � Completed readings are documented � Completed readings are removed Note that there is no requirement that entries be in chronological order, however, administrative problems can lead to interview questions addressing how the program is run. Assessors should review selected readings to determine if they appear to be appropriate and are screened to the correct operations personnel. This analysis may also lead to some interview questions. For this area of conduct of operations, deviations from expectations can be normally derived from document reviews and followed up with interviews. 6.15 Assessing Timely Orders to Operators. 6.15.1 Discussion. Timely Orders to Operators provide a means for management to communicate short term information, administrative instructions, and direction to operations personnel. Timely orders can take many forms including contractor memos, temporary administrative procedures, or even notes written on loose leaf paper. The primary consideration for this program is that management has knowledge and approval of what is issued as a timely order. Timely orders are not intended to modify or replace operating procedures. 6.15.2 Preparing to Assess Timely Orders to Operators. To develop an assessment plan for Timely Orders to Operators, assessors should first review the activity-specific graded-approach document for conduct of operations and determine which guidelines are applicable at the activity. This will allow them to determine the scope of the assessment and which applicable DOE-EM-STD-5505-96 101 activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. Identifying the program that is serving as the activity's timely order program will be the primary task of assessment planning. Additionally, the use of observations will be limited to determining if specific orders were completed by the operators as required and observing how first line supervisors ensure their personnel read the orders as required. 6.15.3 Observation. The use of observations to assess timely orders is limited to watching operators review orders early in the shift and observing them when performing the assigned tasks. Methods used by first line supervisors to ensure operators receive the information can be observed as well.

6.15.4 Interview. The following personnel will normally be interviewed when assessing timely orders: � Operators � First line supervisors � Operations supervisors � Designated timely orders administrator The following questions could be asked of operators: � How often are you required to review timely orders? � Do you think the program is useful? Why? � Tell me about the content of today’s orders A particularly effective method for assessing the quality of Timely Orders programs is for the assessor to gain familiarity with the orders of the last several days and then question the DOE-EM-STD-5505-96 102 personnel administering and using the Timely Orders program concerning their knowledge of those specific orders. This will give information on the process from beginning to end and can be used to assess not only the mechanics of the program, but also the quality and effectiveness of the overall program objectives, or how effectively the program gets pertinent information to the workers and supervisors in a timely manner. The following additional question could be asked of first line supervisors: � How do you ensure that all operators read timely orders? The questions below could be asked during interviews with operations supervisors and designated administrators: � How do you ensure that all operators read timely orders? � How often do you review timely orders? � How does the administrator ensure that the operations supervisor is aware of each issued order? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: � Interviews with several more operators � Interviews with shift supervisors � Interviews with activity managers and operations supervisors � Interviews with training and maintenance department personnel 6.15.5 Document Review. Assessors should review the applicable administrative policy or procedure to familiarize themselves with the requirements of the activity's program. Typically, DOE-EM-STD-5505-96 103 the following documents should be reviewed during the assessment: � Administrative procedure or policy � Applicable timely orders Review of the documents listed above should help determine how the program is administered. From the administrative procedure, an assessor can usually identify: � Who has been designated (if anyone) by the operations supervisor to administer orders � The method of ensuring all operators review and sign orders early in the shift Furthermore, assessors should review the timely orders tracking system to determine whether: � Expired orders are removed � When an order references a modification to a procedure, a procedure change has been approved which covers the information in a timely order. It is vital that the orders do not violate the procedures or the procedure control system. 6.16 Assessing Operations Procedures. 6.16.1 Discussion. Operations procedures provide direction to personnel to ensure that an activity is operated within its design base. Procedures are the activity manager's specific direction to operators when operational direction is needed. It is not intended that every aspect of activity operations, from the job of secretary, to operations manager, be proceduralized. Rather, managers should determine the aspects of operations that require the use of written procedures to

ensure safety, process quality, and control. Managers are also expected to define to what extent a procedure is required to be used. Assessors should specifically note whether such policy and administrative guidance is promulgated in writing to assist the workers in the use of procedures. Terms and extent vary from activity to activity, but normally three levels of procedural use are specified: DOE-EM-STD-5505-96 104 � Step-by-step execution required. "in hand," "reader-worker routine," or "step sign offs" are all methods that ensure step-by-step use of a procedure. This method is used for complex tasks, tasks which have significant consequences of error, infrequently performed tasks, or when the worker is training or has not performed the task for a period of time and is not very familiar with it. � Open for reference. The written procedure is still intended to be followed, but step-by-step referral by the operator is not required. The procedure could be open on a workbench or posted on a wall for operator reference. This method is used for routine evolutions and frequently performed evolutions by fully trained individuals who perform the evolution often. � For training only. Procedures exist but are of such a nature that they are not needed for the actual operation. These procedures are used only to train new operators. Another type of operations procedure is an emergency procedure. These procedures are normally divided into two sections: (1) immediate actions and (2) supplementary actions. When using these procedures during actual emergencies, management might expect operators to have the immediate actions for each postulated problem memorized and then refer to the emergency procedure to conduct supplementary actions. Another type of procedure is the alarm response or annunciator response procedure which provides in a few steps, the actions to be taken when the annunciator or alarm is initiated. 6.16.2 Preparing to Assess Operations Procedures. To develop an assessment plan for Operations Procedures, assessors should first review the activity-specific graded-approach document for conduct of operations and determine which guidelines are applicable at the activity. This will allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment DOE-EM-STD-5505-96 105 techniques. 6.16.3 Observation. One of the most effective methods of assessing operations procedures is to observe as many activities as possible that require the use of procedures. Generally, the observer should review the procedure before observing its use to maximize the benefit of watching the evolution. In some cases, the assessors review can be concurrent with the observation. Observations of Operations Procedures should be conducted during walkthroughs and tours with operators and shift supervisors, and during the conduct of special operations and drills. The activity plan of the day and plan of the week should be reviewed to determine the time and place for observing operations and drills. The following can normally be assessed during the observation of selected procedures: � Operator knowledge of the rules for the use of written procedures � Operator use of written procedures � Operator understanding of and adherence to procedure precautions and prerequisites

� Operators' actions when problems are found in the procedure To observe the use of procedures, start by reviewing the procedure to be observed and look for potential problems in the procedure. If required, can the procedure be followed step-by-step? Some examples of what to look for are included in section 6.16.5 When conducting observations, assessors should bring a copy of the procedure with them and follow along to observe how the operator uses it. When possible, assessors should ask operators questions about what they are doing and how they are doing it. As many observations as possible should be conducted during the assessment period. An effective method for conducting observations is to compare two different operators conducting the same procedure. This will allow an assessor to see any variations between the use of procedures by operators. This methodology will also allow an assessor to uncover DOE-EM-STD-5505-96 106 deviations that are activity wide problems. Once an observation has been completed, compare what was observed to the requirements contained in activity policies and procedures to identify any apparent deviations from requirements. Pursue these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. 6.16.4 Interview. Interviews should be used when assessing procedures to evaluate operator level of knowledge. Interviews of operations and maintenance department personnel should occur during observations and document reviews. Additional scheduled interviews should also be conducted to follow up and clarify leads. The following are individuals who could be interviewed: � Operators � Shift or first line supervisors � Activity managers and operations supervisors � Qualified procedure writers and their supervisors As discussed above, conducting interviews while observing the use of a procedure is an effective means to assess this area of conduct of operations. From an initial observation or interview with an operator, lines of inquiry can be developed. Questions that can be asked of operators include: � What are you required to do when you find procedural mistakes? � How do you know that the procedure being used is the most current, approved copy? � When do you use procedures? � What is your understanding of the rules for the use of written procedures? � How do you use a written procedure when conducting the operation to which it applies? DOE-EM-STD-5505-96 107 Questions for supervisors and managers could include: � How do you ensure operators have input into the development of procedures? � When do you use procedures? � What is your understanding of the rules for use of written procedures? � Are immediate changes made to procedures? How do you compensate for the lack of a normal review process? � What actions are required when an inaccuracy is found in a procedure? � How do you expect operators to use procedures? � How do you ensure that operators are using the correct revision of a procedure? Questions for procedure writers include: � How do you ensure that operators have input into the procedure writing process? � What is the process time for procedure revisions and changes? � Does process time support the users? � How do you ensure consistency in procedures? � How do you ensure references for procedures are accurate and current?

After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: � Observations of and interviews with several more operators � Tours of the activity and interviews with shift supervisors � Interviews with radiological control technicians � Interviews with activity managers and operations supervisors � Reviews of operating and program procedures � Interviews with training and maintenance department personnel DOE-EM-STD-5505-96 Questions for supervisors and managers could include: How do you ensure operators have input into the development of procedures? When do you use procedures? What is your understanding of the rules for use of written procedures? Are immediate changes made to procedures? How do you compensate for the lack of a normal review process? What actions are required when an inaccuracy is found in a procedure? How do you expect operators to use procedures? How do you ensure that operators are using the correct revision of a procedure? Questions for procedure writers include: How do you ensure that operators have input into the procedure writing process? What is the process time for procedure revisions and changes? Does process time support the users? How do you ensure consistency in procedures? How do you ensure references for procedures are accurate and current? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: Observations of and interviews with several more operators Tours of the activity and interviews with shift supervisors Interviews with radiological control technicians Interviews with activity managers and operations supervisors Reviews of operating and program procedures Interviews with training and maintenance department personnel 107 DOE-EM-STD-5505-96 108 6.16.5 Document Review. A list of documents that can be reviewed when assessing operations procedures includes: � The procedure writer's guide � The change and revision procedure � The review and approval procedure � The procedure (or document) control procedure � Individual procedures � The index of procedures � The procedure that addresses the activity's policy on the use of operations procedures. � Procedure history files � Procedure for ensuring safety-related procedures and procedural steps are reviewed by personnel who are knowledgeable of authorization basis (as appropriate) The first four documents listed above may be found in many forms and combinations. These documents are frequently combined into one policy or procedure. Document review is usually most beneficial when conducted prior to starting an observation. This review should focus on gaining familiarity with the procedure system as a whole. Specifically, areas such as document use and control requirements, the change and revision process, and format requirements should be reviewed. The background knowledge gained from this review is useful when evaluating other activities related to procedures. Some common problem areas related to format that can lead to operator errors are listed below: � Caution and warning statements that follow the steps to which they apply (Does the operator

follow a particular statement before conducting the next step in the procedure?) � Reference or required use of other procedures within the procedure being observed. (Does the operator use the procedure as required?) � Very restrictive steps or requirements in the procedure DOE-EM-STD-5505-96 6.16.5 Document ReviewA list of documents that can be reviewed when assessing Operations procedures includes: The procedure writer's guide The change and revision procedure The review and approval procedure The procedure (or document) control procedure Individual procedures The index of procedures The procedure that addresses the activity's policy on the use of operations procedures. Procedure history files Procedure for ensuring safety-related procedures and procedural steps are reviewed by personnel who are knowledgeable of authorization basis (as appropriate) The first four documents listed above may be found in many forms and combinations. These documents are frequently combined into one policy or procedure. Document review Is usually most beneficial when conducted prior to starting an observation. This review should focus on gaining familiarity with the procedure system as a whole. Specifically, areas such as document use and control requirements, the change and revision process, and format requirements should be reviewed. The background knowledge gained from this review is useful when evaluating other activities related to procedures. Some common problem areas related to format that can lead to operator errors are listed below: Caution and warning statements that follow the steps to which they apply (Does the operator follow a particular statement before conducting the next step in the procedure?) Reference or required use of other procedures within the procedure being observed. (Does the operator use the procedure as required?) Very restrictive steps or requirements in the procedure 108 DOE-EM-STD-5505-96 109 � Points at which data is recorded, hold points are designated, and sign-off by another operator or manager is required To determine whether both versions are the same, assessors should review the change revision status for several procedures used by operators. A listing of activity procedures usually provides this information as does each individual procedure. Finally, additional document reviews should be conducted to follow up observations. For example, if during a third observation of the same procedure, the copy used by an operator did not match what was observed during the first two observations, a follow up evaluation of the procedure change and revision system may be necessary. 6.17 Assessing Operator Aid Postings. 6.17.1 Discussion. Operator aid postings provide information useful to operators in performing their duties. The postings can take many forms; posted copies of procedures, graphs, charts, system diagrams, handwritten notes, and information tags could all be operator aids. The most difficult problem to overcome when assessing operator aid postings is consistently determining what constitutes an operator aid. Operator aid postings "provide information useful to operators in performing their duties." This statement is modified somewhat by guideline four, Use of Operator Aids, which states "Operator aids should be viewed as a convenience to the individual using them, not a requirement." These two statements generally define an operator aid,

but cannot be used consistently as a test because assessors must interpret this guidance. This may result in the development of different opinions about what constitutes an operator aid. A conclusive test must be developed to provide a consistent standard. Many postings in facilities are required and controlled by other programs. Examples would include radiological control DOE-EM-STD-5505-96 110 postings required by other DOE Directives, and exit postings required by Federal law. Because the content, format, and use of these postings is controlled by other entities or programs, they are not operator aids. A second aspect of what constitutes an operator aid concerns how the information is used. An operator aid provides information that is useful to operators while performing their duties. This aspect removes from consideration postings, such as bulletin boards or personal notes at one's desk, because these items do not affect activity operations. The third element of the test addresses changes to posted information. In general, if changes to posted information can affect an operator's work, the posting should be controlled as an operator aid. Consider a vendor's nameplate that is welded to a valve in the activity. If the vendor's name changes, the new information would not affect operation of the system, and therefore, this information should not be controlled by the operator aid program. Finally, some postings in an activity can be either copies of operating procedures or contain information specified by procedures. These postings should be controlled by the operator aid program unless the activity controls the information under another program such as the procedure change and revision system. In summary, a posting is an operator aid if: � The information is used by operators to perform their duties, and � Changes to the information would affect the quality of the operator's work, and � The posting is not controlled or required by some other program (e.g., RCRA, Radcon, OSHA, etc.), or � The information is taken from or referenced in an operations procedure. This test provides assessors with a tool to determine if a posting is an operator aid. DOE-EM-STD-5505-96 111 6.17.2 Preparing to Assess Operator Aid Postings. To develop an assessment plan for Operator Aid Postings, assessors should first review the activity-specific graded-approach document for conduct of operations and determine which guidelines are applicable at the activity. This will allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. One aspect to consider when planning the assessment is whether the activity has an official operator aids program. Many facilities choose not to implement a program. If that is the case, the assessment of this area of conduct of operations becomes a walkthrough of the activity to determine whether uncontrolled operator aid postings exist at the activity. 6.17.3 Observation. Observations of operator aids should be conducted during walkthroughs of the activity, and activities in which operators use these aids. Assessors should walk through the activity and look for operator aids. Once found, assessors should review the aids to determine if they contain the information required by the program

procedure and to determine whether they are properly controlled. Furthermore, assessors should look for the following: � Postings that are obscuring equipment, controls, meters, gages, or other postings � Posting that appear to be damaged by their environment � Operators that use postings in place of procedures � Posted material inside cabinets or markings on instruments and equipment may constitute operator aids. Although there are not as obvious as a posting, they constitute a subtle, uncontrolled method to post information. � Markings on instruments and on labels often constitute operator aids. In some cases they are outdated and can lead to misoperation of equipment or systems. DOE-EM-STD-5505-96 112 If an aid is used because an operator cannot actually hold the procedure "in-hand," an assessor should check to see if the procedure is at least open off to the side. While not a lead by itself, not seeing an open procedure should prompt an assessor to interview operators about how they use operator aids. Once an observation has been completed, compare what was observed to the requirements contained in activity policies and procedures to identify any apparent deviations from requirements. Pursue these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. 6.17.4 Interview. The following personnel may be interviewed during the assessment of operator aid postings: � Operators � Activity managers and operations supervisors � Shift supervisors � Operator aid administrators The operator aid administrator could be one person or a group of people who are responsible for running the program on behalf of the activity manager. The activity manager, however, cannot delegate final approval authority for operator aids. Interviews should be used to follow up leads that developed during activity walkthroughs and observations of operators. Some examples of appropriate questions for operators include: � What is an operator aid posting? � Who can develop an operator aid posting at this activity? � If you find a damaged operator aid posting, what do you do? DOE-EM-STD-5505-96 113 Questions for the activity manager include: � Who develops operator aid postings at this activity? � What do you consider to be an operator aid posting? � How do you ensure that damaged or missing operator aid postings are replaced? Finally, questions for the program administrator include: � How often do you conduct an audit of the program? � What does this audit include? � How do you ensure that operators are not using postings to replace procedures? � How do you ensure that postings, derived from procedures, are reviewed when the procedure changes? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: � Tours and interviews with several more operators � Tours of the activity and interviews with shift supervisors � Interviews with activity managers and operations supervisors � Reviews of operating and program procedures � Interviews with the operator aid administrator 6.17.5 Document Review. Because the definition of an operator aid varies across the DOE complex, the administrative policy should be reviewed during assessment planning and the

operator aids test modified accordingly. The following documents should be reviewed during the assessment: � The operator aid administrative control policy or procedure � Operator aid logbooks DOE-EM-STD-5505-96 Questions for the activity manager include: @ Who develops operator aid postings at this activity? @ What do you consider to be an operator aid posting? @ How do you ensure that damaged or missing operator aid postings are replaced? Finally, questions for the program administrator include: e@ How often do you conduct an audit of the program? @ What does this audit include? @ How do you ensure that operators are not using postings to replace procedures? @ How do you ensure that postings, derived from procedures, are reviewed when the procedure changes? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: e Tours and interviews with several more operators @ Tours of the activity and interviews with shift supervisors e Interviews with activity managers and operations supervisors e Reviews of operating and program procedures e@ Interviews with the operator aid administrator 6.17.5 Document ReviewBecause the definition of an operator aid varies across the DOE complex, the administrative policy should be reviewed during assessment planning and the operator aids test modified accordingly. The following documents should be reviewed during the assessment: e@ The operator aid administrative control policy or procedure @ Operator aid logbooks 113 DOE-EM-STD-5505-96 114 � Audits and review records (if separate from the logbook) � Individual operator aid postings When reviewing operator aids, assessors should compare the posting list developed during the walkthrough with operator aids in the logbook to verify that: � The posting is in the logbook listing � The postings are the same revision � There is a copy of the posting with the logbook or file Furthermore, assessors should review the logbook to determine if: � Written evidence of reviews or audits exists � References used to develop the operator aids (i.e. system diagrams, procedures, etc) exist If no written evidence concerning reviews is found, interviews will need to be conducted to determine if the requirements of activity policies or procedures are being met. 6.18 Assessing Equipment and Piping Labeling. 6.18.1 Discussion. The purpose of an equipment and piping labeling program is to ensure that activity personnel are able to identify equipment they operate. An effective labeling program will reduce operator and maintenance errors resulting from incorrect identification of equipment, increase training effectiveness by being able to trace the actual activity system as opposed to tracing a schematic, and reduce operator exposure to radiation and hazardous materials. 6.18.2 Preparing to Assess Equipment and Piping Labeling. To develop an assessment plan for Equipment and Piping Labeling, assessors should first review the activity-specific graded- approach document for conduct of operations and determine which guidelines are applicable at DOE-EM-STD-5505-96 115 the activity. This will allow them to determine the scope of the assessment and which applicable activity policies and procedures to review when developing the assessment plan. Sections 5.4 and

5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. The elements of the program should be defined by activity administrative procedures and policies. The most important policy to note is what components require labeling. If the activity chooses not to have a labeling program, the assessment of this area of conduct of operations becomes a walkthrough of the activity, to identify all damaged or missing labels. It should be noted that even if the decision has been made to not have a labeling program, it may be necessary to assess labeling determine what effect the quality of labeling is having on other areas, such as use of procedures, emergency actions, control of equipment status, maintenance, lock and tag, engineering, etc. 6.18.3 Observation. Observations of equipment and piping labeling should be conducted during walkthroughs of the activity and operations in progress. During the assessment, assessors should walk through the activity and look for: � Systems and equipment that are required to have labels but are not actually labeled � Components that are missing labels or have damaged labels � Labels that do not correlate with diagrams, schematics, drawings, and procedures � Markings on equipment and components that indicate label deficiencies � Markings or corrections made to labels, indicating deficiencies in the labels themselves � Labels that differ in nomenclature from procedures and drawings Once an observation has been completed, compare what was observed to the requirements contained in activity policies and procedures to identify any apparent deviations from DOE-EM-STD-5505-96 116 requirements. Pursue these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. 6.18.4 Interview. During the assessment, the following personnel may be interviewed: � Operators � Maintenance technicians � Shift supervisors � Activity managers and operations supervisors Interviews should be conducted with operators and shift supervisors during activity walkthroughs and tours. Examples of interview questions for operators include: � If you find a label on the floor, what are your actions? � How do you replace a damaged label? � What equipment and systems are labeled in the activity? An additional question for maintenance personnel is: � How do you ensure labels that are removed during maintenance are replaced? Finally, the following additional questions could be asked of an activity manager, operations supervisor, or shift supervisor: � How do you ensure that operators report missing labels? � If temporary labels are used, when do you replace those labels with permanent labels? � What components are required to have labels? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: DOE-EM-STD-5505-96 requirements. Pursue these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. 6.18.4 Interview During the assessment, the following personnel may be interviewed: @ Operators @ Maintenance technicians e@ Shift supervisors e Activity managers and operations supervisors

Interviews should be conducted with operators and shift supervisors during activity walkthroughs and tours. Examples of interview questions for operators include: e lf you find a label on the floor, what are your actions? @ How do you replace a damaged label? @ What equipment and systems are labeled in the activity? An additional question for maintenance personnel is: @ How do you ensure labels that are removed during maintenance are replaced? Finally, the following additional questions could be asked of an activity manager, operations supervisor, or shift supervisor: e@ How do you ensure that operators report missing labels? e@ |f temporary labels are used, when do you replace those labels with permanent labels? @ What components are required to have labels? After completing an interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. These leads can normally be validated or disproved by using one or more of the following: 116 DOE-EM-STD-5505-96 117 � Tours with additional operators � Tours of the activity and interviews with shift supervisors � Interviews with activity managers and operations supervisors � Reviews of program procedures 6.18.5 Document Review. The following documents could be reviewed during the assessment: � Activity equipment and piping labeling program procedure � Supporting documents such as labels, logbooks, and label work order forms � Operations procedures � System drawing or schematics While reviewing the administrative procedure that outlines program requirements, assessors should determine if there is a process for identifying and replacing missing or damaged labels. The process could include items such as a missing label log, copies of work orders for new labels, or a temporary label log. These all can be reviewed for consistency and functionality. Another item to look for during document reviews is a method to promptly identify and correct labeling deficiencies. This portion of the program will typically consist of a governing procedure, a logbook for recording deficiencies, and a method for ordering replacement labels. In addition, the process should have provisions for temporary labeling and supervisory approval of labeling changes. 6.19 Assessing Operations-Related Radiological Control. 6.19.1 Discussion. Effective radiological control is an integral part of safe and efficient activity operations and, therefore, elements of radiological control can be found in several elements of conduct of operations. Operating goals such as "As-Low-As Reasonably DOE-EM-STD-5505-96 118 Achievable" (ALARA) should be used as a management tool for improving operating performance and measuring operating effectiveness. Operations personnel should also follow good personnel protection practices to maintain personnel exposure to radiation “ALARA.” For example, an operator may be required to conduct operations in a radiologically controlled area. The operator’s adherence to radiological control requirements, such as personnel protection requirements and the proper use of radiation work permits, directly impacts the worker’s exposure and safety. For these reasons, operations assessments should include an evaluation of radiological control. An operations assessment of radiological controls focuses on radiological operations rather than a comprehensive radiological control audit. For example, during an operations assessment, it is

more beneficial to observe a maintenance action with significant radiological precautions rather than locating every radiological posting in the activity; the former is operational and the latter is more administrative. The emphasis should be on assessing the radiological operations that require the support of radiological control technicians rather than the administration of a radiological control program. 6.19.2 Preparing to Assess Operations-Related Radiological Control. To develop an assessment plan for operations-related radiological control, assessors should first review the activity’s implementation of radiological control. This will allow assessors to determine which activity policies and procedures apply the guidelines of current radiological protection regulations and directives when developing the assessment plan. Sections 5.4 and 5.8 provide a detailed explanation of assessment planning and the use of assessment techniques. To determine the observations, interviews, and document reviews necessary to assess radiological control, the specific elements of the program to be assessed must be identified. The following are examples of assessable areas in radiological controls: DOE-EM-STD-5505-96 119 � Operations and maintenance personnel knowledge, training, and ability in radiological control, including basic knowledge, job knowledge, worker responsibilities, etc. � Radiological control personnel knowledge, training, and ability to include technicians, staff, and management � Personnel exposure control and use of dosimetry � Use and calibration of radiation measuring equipment � Control of radioactive material and contamination � Functional relationship between the Radiological Control Organization and the Operations Department 6.19.3 Observation. Observations of radiological control should be conducted during walkthroughs and tours with operators, shift supervisors, and radiological control technicians. Walkthroughs with operations department personnel should be used to evaluate their knowledge and skills regarding: � Minimizing personnel exposure � Use of dosimetry � Requirements for personnel monitoring and contamination control � Use of and adherence to the requirements contained in radiological work permits, including the use of personnel protective equipment � Radiological postings Observations of radiological control technicians should focus on the items listed above, but should also include: � The conduct of radiological surveys � Calibration and use of monitoring instruments � Control of radioactive sources � Dosimetry control and operations � Control of radioactive material and contamination DOE-EM-STD-5505-96 Operations and maintenance personnel knowledge, training, and ability in radiological control, including basic knowledge, job knowledge, worker responsibilities, etc. Radiological control personnel knowledge, training, and ability to include technicians, staff, and management Personnel exposure control and use of dosimetry Use and calibration of radiation measuring equipment Control of radioactive material and contamination Functional relationship between the Radiological Control Organization and the Operations Department 6.19.3 ObservatianObservations of radiological control should be conducted during walkthroughs and tours with operators, shift supervisors, and radiological control technicians. Walkthroughs with operations department personnel should be used to evaluate their knowledge

and skills regarding: Minimizing personnel exposure Use of dosimetry Requirements for personnel monitoring and contamination control Use of and adherence to the requirements contained in radiological work permits, including the use of personnel protective equipment Radiological postings Observations of radiological control technicians should focus on the items listed above, but should also include: The conduct of radiological surveys Calibration and use of monitoring instruments Control of radioactive sources Dosimetry control and operations Control of radioactive material and contamination 119 DOE-EM-STD-5505-96 120 The assessment should also include observations of operations and drills. For example, the following operations are good candidates for conducting observations: � Maintenance activities involving contaminated systems and equipment � Routine and special radiological surveys conducted by technicians � Radiological response drills � Inspection, calibration, and testing of radiological monitoring instruments There are several aspects of the radiological control program that can be assessed during these activities. For instance, while observing a maintenance operation on a radiologically contaminated system, the following items might be observed: � Use of and adherence to the radiological work permit by operators, maintenance personnel, and radiological control technicians � Radiological posting of the area � Use of personnel protective equipment and dosimetry by personnel involved in the work � Methods used to control contamination and personnel exposure during the activity � The conduct of radiological surveys by technicians � Personnel monitoring during the activity and upon exit from the posted area � The interface between radiological control technicians and the personnel involved in the work Operators, maintenance personnel, shift supervisors, and technicians from each shift should be observed during the assessment. If possible, the observation of a particular shift position should start at shift turnover since vital information about the operating status of equipment and systems, and operations and maintenance in progress, is passed between shifts at this time. DOE-EM-STD-5505-96 The assessment should also include observations of operations and drills. For example, the following operations are good candidates for conducting observations: Maintenance activities involving contaminated systems and equipment Routine and special radiological surveys conducted by technicians Radiological response drills Inspection, calibration, and testing of radiological monitoring instruments There are several aspects of the radiological control program that can be assessed during these activities. For instance, while observing a maintenance operation on a radiologically contaminated system, the following items might be observed: Use of and adherence to the radiological work permit by operators, maintenance personnel, and radiological control technicians Radiological posting of the area Use of personnel protective equipment and dosimetry by personnel involved in the work Methods used to control contamination and personnel exposure during the activity The conduct of radiological surveys by technicians Personnel monitoring during the activity and upon exit from the posted area The interface between radiological control technicians and the personnel involved in the work

Operators, maintenance personnel, shift supervisors, and technicians from each shift should be observed during the assessment. If possible, the observation of a particular shift position should start at shift turnover since vital information about the operating status of equipment and systems, and operations and maintenance in progress, is passed between shifts at this time. 120 DOE-EM-STD-5505-96 121 To observe a radiological control technician conducting routine radiation and contamination surveys at the activity, the following methodology could be used: � Observe the technician as he performs any required inspection, response checking or testing of monitoring instruments. � Accompany the technician to the first survey area. Observe the performance and documentation of the survey, use of personnel protective equipment, adherence to radiation work permit requirements, etc. � Continue observing the technician until all surveys are completed. Look for differences in how the surveys are conducted and documented at each area, posting of areas, etc. Observe a maintenance task or test calibration performed on a radioactive system or component. Select such a task from the activity plan of the day and arrange to accompany the workers from the time they prepare for the job through job completion. The following steps in the process can be reviewed or observed: � Observe the supervisor and workers as they make preparations, including performance of any required inspection, response checking, or testing of monitoring instruments. � Review the documentation, procedures, and requirements for the job, including radiological preparations such as radiological surveys, radiological work permit, personnel radiological worker training and the work package. � Attend the pre-job brief. � Accompany the workers to the job and observe the performance of the task � Observe performance of radiological controls, including the interaction of workers and the performance of radiological control technicians. � Observe the use of personnel protective equipment such as donning or doffing Anti-C’s. � Review the job area radiological entry controls � Review the work from the standpoint of control of radiological contamination in tools, parts, workers, and the area. � Observe the completion of the task, including post-job briefing and closeout of paperwork. DOE-EM-STD-5505-96 To observe a radiological control technician conducting routine radiation and contamination surveys at the activity, the following methodology could be used: Observe the technician as he performs any required inspection, response checking or testing of monitoring instruments. Accompany the technician to the first survey area. Observe the performance and documentation of the survey, use of personnel protective equipment, adherence to radiation work permit requirements, etc. Continue observing the technician until all surveys are completed. Look for differences in how the surveys are conducted and documented at each area, posting of areas, etc. Observe a maintenance task or test calibration performed on a radioactive system or component. Select such a task from the activity plan of the day and arrange to accompany the workers from the time they prepare for the job through job completion. The following steps in the process can be reviewed or observed: Observe the supervisor and workers as they make preparations, including performance of any required inspection, response checking, or testing of monitoring instruments.

Review the documentation, procedures, and requirements for the job, including radiological preparations such as radiological surveys, radiological work permit, personnel radiological worker training and the work package. Attend the pre-job brief. Accompany the workers to the job and observe the performance of the task Observe performance of radiological controls, including the interaction of workers and the performance of radiological control technicians. Observe the use of personnel protective equipment such as donning or doffing Anti-C’s. Review the job area radiological entry controls Review the work from the standpoint of control of radiological contamination in tools, parts, workers, and the area. Observe the completion of the task, including post-job briefing and closeout of paperwork. 121 DOE-EM-STD-5505-96 122 Once the observation has been completed, compare what was observed to the requirements contained in applicable procedures to identify any deviations from requirements. Pursue these leads using additional observations, interviews, and document reviews to confirm or disprove them and determine if programmatic breakdowns or widespread problems exist. 6.19.4 Interview. Interviews of operations, maintenance, and radiological control department personnel should occur during observations and document reviews. Additional scheduled interviews should also be conducted to follow up and validate leads. The assessment plan should identify the initial interviews necessary to start the assessment. The following personnel might be interviewed during the assessment: � Operators and maintenance department personnel � Shift supervisors or foremen � Activity managers and operations supervisors � Radiological control technicians � Radiological control managers � Training department personnel Most of the interviews accomplished to assess radiological control should be conducted with operators, maintenance personnel, shift supervisors, and radiological control technicians. Whenever possible, these interviews should occur during observations of activities and activity walkthroughs and tours. To pursue leads, scheduled interviews with activity and radiological control managers and training department personnel may also be necessary. While observing a radiological control technician, the following types of questions could be used to assess his level of knowledge: � What radionuclides are of concern at this activity? What types of radiation do these radionuclides emit? What are the effects of these types of radiation? DOE-EM-STD-5505-96 123 � What types of surveys do you conduct at this activity? How are they accomplished, and at what periodicity? � What types of monitoring instruments do you use? How are they calibrated, inspected, and tested? � What are your responsibilities concerning the control of radioactive material and contamination? Describe the process used to control and inventory radioactive material. � How is dosimetry controlled at the activity? Who reads dosimetry and how often is it read? � Who prepares and reviews radiological work permits? Describe the process used to prepare and review permits. How are these permits used at the activity? � What types of radioactive sources are present here? How are they controlled and inventoried? After completing the observation and interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. A similar methodology

to the one described above can be used to observe and interview operations and maintenance personnel. 6.19.5 Document Review. Examples of documents that should be reviewed include: � Radiation Protection Program � Radiation and contamination survey logs and records � Radioactive source control log and inventories � Radioactive material control log and inventories � Logs used to document radioactive releases to the environment � Radiation monitoring equipment calibration and test records � Personnel exposure records � Radiological control procedures � Radiological work permits � Radiological postings DOE-EM-STD-5505-96 @ What types of surveys do you conduct at this activity? How are they accomplished, and at what periodicity? @ What types of monitoring instruments do you use? How are they calibrated, inspected, and tested? @ What are your responsibilities concerning the control of radioactive material and contamination? Describe the process used to control and inventory radioactive material. @ How is dosimetry controlled at the activity? Who reads dosimetry and how often is it read? @ Who prepares and reviews radiological work permits? Describe the process used to prepare and review permits. How are these permits used at the activity? @ What types of radioactive sources are present here? How are they controlled and inventoried? After completing the observation and interview, the information gathered should be evaluated to determine if there are any apparent deviations from activity requirements. A similar methodology to the one described above can be used to observe and interview operations and maintenance personnel. 6.19.5 Document ReviewExamples of documents that should be reviewed include: e@ Radiation Protection Program @ Radiation and contamination survey logs and records e@ Radioactive source control log and inventories e@ Radioactive material control log and inventories e Logs used to document radioactive releases to the environment @ Radiation monitoring equipment calibration and test records e@ Personnel exposure records @ Radiological control procedures e@ Radiological work permits @ Radiological postings 123 DOE-EM-STD-5505-96 124 � General employee and radiological protection personnel training records � Occurrence reports � Logs documenting radiological control activities It is beneficial to review the Radiation Protection Program since it will be the primary governing document for radiological control. An assessor will return to this document many times over the course of the assessment to check on site policies regarding various radiological control activities. Most document reviews necessary to assess radiological control can be conducted during idle time between interviews and observations. Generally, documentation of radiological activities from the past few months should be reviewed. An effective way to conduct the review is to first verify completeness of a document by determining if all required routine entries have been made and if abnormal entries have been recorded and appropriately annotated or explained. Next, determine if the document can be used to accurately reconstruct events. This can be accomplished by comparing the entries in two or more different logs, records, or surveys for the same event. Finally, determine if management periodically reviews and documents meaningful comments for areas needing improvement. As an illustration of reviewing radiological control records, consider the survey program. Check

the Radiation Protection Plan for the requirements regarding survey periodicity, survey techniques, and filling out survey records or maps. The requirements in the site-specific manual should be consistent with the current DOE regulations concerning radiological control. With this background knowledge in mind, find the survey records and maps for the last few months and check to see if all required surveys have been completed, maps and logs are filled out correctly, and any anomalies are fully explained. If any discrepancies are noted on a given survey, look for the same problem on other surveys. Individual deviations from requirements should be used as a starting point to look for more widespread programmatic problems. DOE-EM-STD-5505-96 125 6.20 Assessing Operations-Related Training and Qualification. 6.20.1 Discussion. Since training and qualification are key components in the performance of operations, maintenance and other aspects of work, references to training and qualification can be found in several elements of conduct of operations. For this reason, operations assessments should include an evaluation of training and qualification. An operations assessment of training and qualification focuses on the operational impact of training and qualification rather than a comprehensive audit of the training and qualification program.. For example, during an operations assessment, it is more beneficial to observe an on- shift training event rather than auditing training records; the former is operational and the latter is more administrative. The emphasis should be on assessing operations that are impacted by training and qualification rather than the administration of the training and qualification program. Guidelines for Evaluation of Nuclear Facility Training Programs (DOE-STD-1070-94) provides additional guidance on assessing training and qualification at nuclear activities. 6.20.2 Preparing to Assess Operations-Related Training and Qualification. In order to prevent multiple assessors from following up on similar leads, team members should funnel their training issues to a single assessor who would, in turn, use the tools of observation, document review, and interviews, as discussed in the following sections, to pursue leads. Sections 5.4 and 5.8 provide a detailed explanation of assessment plan development. 6.20.3 Observation. Observation can be an important tool when investigating training. Specific examples include: � Presence of examination and qualification board schedules � Posting of training resources and staffing requirements � Training classes in progress � On-shift training in progress DOE-EM-STD-5505-96 126 � OJT training in progress � Scheduling of training on the plan of the day � Training has a priority and is supported by the line � The formality and atmosphere of training being conducted � Training conducted by the line during lull periods � Logs show training completed that interval with an assessment of accomplishment � Performance or achievement of progress of training is visible � The line and facility has refresher training � Does required reading have training value, Does operations management take advantage of lessons learned in staff meetings � Does training attend event critiques? Is training on distribution for critiques � Are the near misses of a facility factored into training plans, are industry events used. � Do operators retain what the syllabus describes?

While opportunities to observe specific training items depend on the activity being assessed, it is likely that training in one form or another will take place during the assessment. Specific training related events which could be observed might include the following: � An operator undergoing an operational reevaluation as part of their recertification � The recertification of an On-the-Job trainer � An activity drill conducted to determine the effectiveness of the training program � An instructor conducting classroom training � Corrective action training following an abnormal event � The placement or removal of a lockout or tagout � Review of training materials in the hands of the trainees 6.20.4 Interview. Specific questions relevant to the area of conduct of operations in question should be formulated. While it is possible that a cursory review of the activity's training program is all that is required, chances are, a more in-depth review of the training program is DOE-EM-STD-5505-96 OJT training in progress Scheduling of training on the plan of the day Training has a priority and is supported by the line The formality and atmosphere of training being conducted Training conducted by the line during lull periods Logs show training completed that interval with an assessment of accomplishment Performance or achievement of progress of training is visible The line and facility has refresher training Does required reading have training value, Does operations management take advantage of lessons learned in staff meetings Does training attend event critiques? Is training on distribution for critiques Are the near misses of a facility factored into training plans, are industry events used. Do operators retain what the syllabus describes? While opportunities to observe specific training items depend on the activity being assessed, it is likely that training in one form or another will takeapé during the assessment. Specific training related events which could be observed might include the following: An operator undergoing an operational reevaluation as part of their recertification The recertification of an On-the-Job trainer An activity drill conducted to determine the effectiveness of the training program An instructor conducting classroom training Corrective action training following an abnormal event The placement or removal of a lockout or tagout Review of training materials in the hands of the trainees 6.20.4 Interview Specific questions relevant to the area of conduct of operations in question should be formulated. While it is possible that a cursory review of the activity's training program is all that is required, chances are, a more in-depth review of the training program is 126 DOE-EM-STD-5505-96 127 warranted in order to get to the root of a deficiency. The following list provides potential interview questions: � (To Supervisors), are operating standards being communicated to workers through training of operating practices and by supervisory monitoring and guidance of work involving plant operations? � (To Training Manager, Shift Supervisor), how do you ensure that on-shift training adheres to established training programs in order to maintain instructional uniformity? � (To a qualified on-shift trainer), how do you ensure that trainees understand what is required for each training session? � (To Training Manager)Does the training program for operations personnel include in-house events as selected by the operations supervisor?

� (To Training Manager), How do you ensure that the training conducted on systems and equipment remains current with the systems and equipment installed at the activity? The above training related questions, while specific to several different elements of conduct of operations, are varied and mixed. The following example, however, addresses one specific area of conduct of operations, Lockouts and Tagouts in an effort to demonstrate the line of questioning an assessor could use while interviewing the activity training manager. � How do you ensure that operators are receiving the required lockout and tagout training? How is this training tracked? � Do trainers who provide lockout and tagout training possess operational experience? � What administrative support is required by your training department? Are they included on the distribution list for new lockout and tagout procedural revisions? � Are the lockout and tagout requirements incorporated into training guidelines and objectives? If so, are all requirements relative to training of lockouts and tagouts incorporated? � What is the purpose of the lockout and tagout training program at your activity? � How are personnel trained to understand the limitations of the tags? Who receives this training? (operators, managers, secretaries?) DOE-EM-STD-5505-96 warranted in order to get to the root of a deficiency. The following list provides potential interview questions: @ (To Supervisors), are operating standards being communicated to workers through training of Operating practices and by supervisory monitoring and guidance of work involving plant operations? @ (To Training Manager, Shift Supervisor), how do you ensure that on-shift training adheres to established training programs in order to maintain instructional uniformity? @ (TO a qualified on-shift trainer), how do you ensure that trainees understand what is required for each training session? @ (To Training Manager)Does the training program for operations personnel include in-house events as selected by the operations supervisor? @ (To Training Manager), How do you ensure that the training conducted on systems and equipment remains current with the systems and equipment installed at the activity? The above training related questions, while specific to several different elements of conduct of Operations, are varied and mixed. The following example, however, addresses one specific area of conduct of operations, Lockouts and Tagouts in an effort to demonstrate the line of questioning an assessor could use while interviewing the activity training manager. @ How do you ensure that operators are receiving the required lockout and tagout training? How is this training tracked? @ Do trainers who provide lockout and tagout training possess operational experience? @ What administrative support is required by your training department? Are they included on the distribution list for new lockout and tagout procedural revisions? e@ Are the lockout and tagout requirements incorporated into training guidelines and objectives? If so, are all requirements relative to training of lockouts and tagouts incorporated? @ What is the purpose of the lockout and tagout training program at your activity? @ How are personnel trained to understand the limitations of the tags?eddives this training? (operators, managers, secretaries?) 127 DOE-EM-STD-5505-96 128 � Under what circumstances are personnel retrained in the requirements of the lockout and

tagout program? 6.20.5 Document Review. During assessment planning, it is helpful to generate a list of documents to be reviewed when pursuing training related leads. Examples of documents that could be reviewed include: � Training Implementation Matrix � Site and activity training procedures and policies � Qualification cards for operators, technicians, maintenance personnel, supervisors, and managers � Oral and written examination question banks � Oral and written examinations and answer keys � Training program accreditation evaluations � List of qualified operators with assigned operators � The elements of conduct of operations which discuss various aspects of training, specifically: ! Operations Organization and Administration ! Shift Routines and Operating Practices ! Control of On-shift Training ! Investigation of Abnormal Events ! Control of Equipment and System Status ! Lockouts and Tagouts ! Independent Verification ! Operations Aspects of Facility Chemistry and Unique Processes ! Operations Procedures ! Equipment and Piping Labeling DOE-EM-STD-5505-96 @ Under what circumstances are personnel retrained in the requirements of the lockout and tagout program? 6.20.5 Document ReviewDuring assessment planning, it is helpful to generate a list of documents to be reviewed when pursuing training related leads. Examples of documents that could be reviewed include: Training Implementation Matrix Site and activity training procedures and policies Qualification cards for operators, technicians, maintenance personnel, supervisors, and managers Oral and written examination question banks Oral and written examinations and answer keys Training program accreditation evaluations List of qualified operators with assigned operators The elements of conduct of operations which discuss various aspects of training, specifically: A A A Operations Organization and Administration Shift Routines and Operating Practices Control of On-shift Training Investigation of Abnormal Events Control of Equipment and System Status Lockouts and Tagouts Independent Verification Operations Aspects of Facility Chemistry and Unique Processes Operations Procedures Equipment and Piping Labeling 128 DOE-EM-STD-5505-96 129 The documents that should be reviewed will depend on the assessment plan and the leads that are being pursued. As an example, consider a scenario regarding the activity lockout and tagout program. An assessor observes a maintenance operator hanging tags as part of an equipment tagout, and the operator does not seem to know what they are doing. An interview reveals that the operator has been conducting tagouts for two years and has not received any training. To pursue leads, the assessor could: � Check the individual's qualification record to see if lockout and tagout training is a requirement. � Check the site and activity training policies to see if this type of training is necessary. � Check for a list of qualified lockout and tagout personnel to see if this individual is on it, and if not, check to see if their supervisor has been trained to use only qualified personnel when assigning a lockout or tagout. The results of any one of these document reviews could lead down a different investigative path. 6.21 Assessing Drills. 6.21.1 Discussion. A drill is a supervised training session for individuals or teams. DOE 5480.20A requires that drills be conducted in an activity so that personnel and operating teams

maintain proficiency in their ability to respond to abnormal or accident conditions. Therefore, the objective of conducting drills is to train and evaluate managers, supervisors, operators, in a realistic way, to recognize proper response to abnormal or emergency conditions and equipment failures. Drills can be run for several reasons: � Operator initial qualification � Maintain operator proficiency � Self readiness evaluations � Validate training effectiveness DOE-EM-STD-5505-96 130 There are two reasons for requesting that a drill be run during an assessment: � Assess the drill program � Assess the facility’s response to the drill Findings discovered during the drill would only be used as leads for that particular area and should not be reported as findings. The one exception to this situation would be if the deficiency was not detected by the drill team and not discussed at the drill critique. Then the drill team’s failure to detect the deficiency would be noted in the report as a drill program finding. Assessors should understand the difference between an activity drill and a site-wide exercise. While a drill is a supervised training session at an activity, a site-wide exercise is a comprehensive performance test of most aspects of the site's emergency management program. Conducting drills at an activity should be an integral part of the activity's training and qualification program. Proper response to abnormal or emergency conditions is vital to ensure personnel safety, and protection of activity equipment and the environment. The most important aspect to evaluate when assessing drills is the control of equipment, systems, and processes during the drill. 6.21.2 Preparing to Assess Drills. The team leader should attempt to schedule the assessment in conjunction with a scheduled drill if the contractor already has a drill schedule. If the contractor does not routinely schedule drills, then the team leader should request a drill. The scope of assessing drills can be divided into two areas. First, is the drill program itself. Second, is the observation of drills by assessment team members to assist them in their specific areas such as communication, notifications or control area activities. Examples of specific aspects of the drill program or specific elements of conduct of operations that could be assessed include: � Drill program goals, objectives and responsibilities � The incorporation of drills into the continuing training program to enable personnel to DOE-EM-STD-5505-96 131 maintain proficiency in their ability to respond to abnormal or accident situations � Identification, development and scheduling of basic drills � Qualified activity operators and supervisors that are part of a trained drill team � Adequacy of communication system performance and effectiveness for both the drill team and activity operators � The notification process for a reportable event � Operations, training and activity management involvement in the drill program 6.21.3 Observation. Remember that the scope of assessing drills is comprised of two separate areas. First, the drill program itself and second, the conduct of drills at the activity. Both of these aspects of assessing drills may provide leads with respect to specific elements of conduct of operations, radiological controls, or local procedures and program requirements. Keeping in mind this dual scope, the following situations could be observed:

� The pre-drill brief � Control area or room activities during a drill � Normal and emergency communication throughout the activity during a drill � Drill team members actions when conducting drills � The drill evaluation/critique Prior to observing the drill the pre-drill brief should be attended by those team members responsible for assessing the drill program. The following items could be the focus of the assessor(s): � Are all evaluators, controllers, and observers (the drill team) present? � Is the drill scenario reviewed with particular attention given to required initial conditions, conditions requiring drill termination, final activity conditions, controller assignments and general and specific precautions for this drill? � Are lessons learned and areas of noted weaknesses from previous, similar drills discussed? � What is the established communication method amongst the drill team and is there a specific, DOE-EM-STD-5505-96 maintain proficiency in their ability to respond to abnormaacaident situations Identification, development and scheduling of basic drills Qualified activity operators and supervisors that are part of a trained drill team Adequacy of communication system performance and effectiveness for both the drill team and activity operators The notification process for a reportable event Operations, training and activity management involvement in the drill program 6.21.3 ObservatianRemember that the scope of assessing drills is comprised of two separate areas. First, the drill program itself and second, the conduct of drills at the activity. Both of these aspects of assessing drills may provide leads with respect to specific elements of conduct of operations, radiological controls, or local procedures and program requirements. Keeping in mind this dual scope, the following situations could be observed: The pre-drill brief Control area or room activities during a drill Normal and emergency communication throughout the activity during a drill Drill team members actions when conducting drills The drill evaluation/critique Prior to observing the drill the pre-drill brief should be attended by those team members responsible for assessing the drill program. The following items could be the focus of the assessor(s): Are all evaluators, controllers, and observers (the drill team) present? Is the drill scenario reviewed with particular attention given to required initial conditions, conditions requiring drill termination, final activity conditions, controller assignments and general and specific precautions for thidr Are lessons learned and areas of noted weaknesses from previous, similar drills discussed? What is the established communication method amongst the drill team and is there a specific, 131 DOE-EM-STD-5505-96 132 unequivocal method for directing that the drill be stopped? Specific elements of conduct of operations and radiological controls can be observed while conducting drills. For example, communication system performance and effectiveness could be observed by one team member in the control room area and another could be with an operator or emergency response team at a remote location A drill debrief (also referred to as an evaluation or critique) provides an excellent opportunity to observe certain aspects of a drill program, such as: � Is the critique held in a timely manner (immediately following the drill is normal) and chaired by the drill coordinator?

� Are comments solicited (and recorded) from all evaluators and controllers? � How are lessons learned promulgated to ensure that the maximum training value is gained? � Did the debrief ensure that all required actions were reviewed and either verified as completed or noted as a deficiency? � Is the critique more than just a chronology of events? It should be an analysis of what went wrong and what went right. The preceding observations may provide leads to follow in specific areas. 6.21.4 Interview. Once the observations have been completed, it may be necessary to interview the following individuals in order to follow up on leads or clarify initial findings: � Drill team members � Activity managers and operations supervisors � Activity operators and other drill participants � Training department personnel These interviews should focus on the specific elements of the drill program as described in DOE-EM-STD-5505-96 133 section 6.21.2. Leads relating to specific elements of conduct of operations or the radiological controls should be addressed by assessment team members responsible for those areas. The following types of questions could be used to find out more about the drill program: � What type of initial training is provided for drill team members? � How are activity-specific drill scenarios developed? Are the scenarios realistic? � What are the responsibilities of the drill team coordinator? � What are the roles and responsibilities of drill team members? � Are there an adequate number of trained drill team members? � Explain the role of the drill program with respect to continuing training requirements. � What is the relationship between activity management, operations, and training departments regarding drill development and scheduling? � How is a drill session evaluated? � Explain how drill critique lessons learned are promulgated. � When would a drill be terminated early? How is it accomplished? � How does the drill team communicate during a drill? 6.21.5 Document Review. Examples of documents that could be reviewed when assessing drills include: � Activity drill schedule � Drill program implementing procedure � Training Implementation Matrix (TIM) � Activity drill guides and scenarios � Drill critique meeting minutes � Drill critiques from prior drills � Emergency plan DOE-EM-STD-5505-96 section 6.21.2. Leads relating to specific elements of conduct of operations or the radiological controls should be addressed by assessment team members responsible for those areas. The following types of questions could be used to find out more about the drill program: What type of initial training is provided for drill team members? How are activity-specific drill scenarios developed? Are the scenarios realistic? What are the responsibilities of the drill team coordinator? What are the roles and responsibilities of drill team members? Are there an adequate number of trained drill team members? Explain the role of the drill program with respect to continuing training requirements. What is the relationship between activity management, operations, and training departments regarding drill development and scheduling? How is a drill session evaluated? Explain how drill critique lessons learned are promulgated. When would a drill be terminated early? How iadtomplished? How does the drill team communicate during a drill? 6.21.5 Document ReviewExamples of documents that could be reviewed when assessing

drills include: Activity drill schedule Drill program implementing procedure Training Implementation Matrix (TIM) Activity drill guides and scenarios Drill critique meeting minutes Drill critiques from prior drills Emergency plan 133 DOE-EM-STD-5505-96 134 Initial reviews of the above documents should give an assessor a good indication of where the activity stands with regard to conducting drills. The following aspects of a drill program may be evaluated through document reviews: � Drill program goals, objectives and responsibilities are defined and promulgated � Basic drills are identified, developed and scheduled � Drills are incorporated into the continuing training program � "Lessons learned" from evaluated drill sessions are promulgated to ensure the maximum training value is gained DOE-EM-STD-5505-96 APPENDIX A SAMPLE OPERATIONS ASSESSMENT PROGRAM SCHEDULE 135 Activity Month-Year Wastewater Treatment Hazardous Waste Storage Decontamination & Pilot Well Monitoring Plant Area Decommissioning January 1993 # February 1993 * March 1993 April 1993 # May 1993 # June 1993 July 1993 # August 1993 # September 1993 October 1993 * November 1993 # December 1993 January 1994 * February 1994 # March 1994 April 1994 # May 1994 # June 1994 July 1994 # August 1994 # September 1994 October 1994 # November 1994 * December 1994 #-indicates a partial operations assessment. *-indicates a full operations assessment. DOE-EM-STD-5505-96 APPENDIX B OPERATIONS ASSESSMENT FLOW DIAGRAM DEVELOPING FINDINGS AND CONCERNS 136 DOE-EM-STD-5505-96 APPENDIX C SAMPLE OPERATIONS ASSESSMENT PREPARATION CHECKLIST 137 This Assessment Preparation Checklist can be used by assessment team coordinators to prepare for operations assessments of DOE field activities. Assessment preparations should be started far enough in advance to allow adequate time for completion of all pre-assessment items. Assessments should be scheduled using the two-year operations assessment schedules prepared by each Operations Office. The team member coordinating the visit should follow this checklist to ensure that both the team and the site are prepared. Additional items can be added as necessary to support the needs of the upcoming assessment. Site or activity Assessment Start Date Operations Office Point of Contact (POC) Activity POC (name & no.) ____ Assessment Start Date: Establish communications with the activity POC and finalize the dates for the assessment. Provide a list of required resources (activity documentation, computer support, conference room, etc.) ____ Team Leader and Assessment Team Members: Team members should be selected on the basis of their technical ability, expertise, and assessment experi- ence. The number of assessors should reflect the scope and focus of the assessment (i.e., full or partial). Team members should be identified far enough in advance to allow time to prepare for the assessment. ____ Security, Access, Briefing Requirements: If necessary, call the site badging office. Have a list prepared that identifies team member names, clearances, SSNs, birthdays, addresses, and citizenship. Ascertain the security requirements for unrestricted site or activity access. Determine requirements for personal property passes. Obtain necessary forms. Send visit request forms. Set up security briefs.

____ RADCON/HAZWOPER Requirements: Discuss radiological and chemical hazards with the activity POC. Dosimetry badging, site-specific training, etc. should be scheduled for the day the team arrives. Site/activity-specific training required Fastscan, bioassay, and other requirements DOE-EM-STD-5505-96 APPENDIX C SAMPLE OPERATIONS ASSESSMENT PREPARATION CHECKLIST 138 ____ Assessment Schedule: Construct a schedule to identify major assessment events and other pertinent information. The schedule should address, security/radcon training, activity tour, in-brief by the team leader and activity manager, and the assessment out-brief. Fax a copy to the activity POC. If the requested activity-specific information and documentation has not yet been received, request that the information be sent as soon as possible. Provide copies of this information to all team members. ____ Meetings with the Team Leader: When necessary, but at least one week prior to the assessment start date, meet with the team leader to discuss the status of assessment preparations. ____ Meetings with the Assessment Team: When necessary, meet with the entire team. Review the schedule and scope of the assessment. Ensure all team members have copies of the activity information provided. ____ Final Team Meeting: Review the team schedule, scope of the assessment and individual assignments. Ensure all team members have activity information packages, assessment schedule, etc. DOE-EM-STD-5505-96 APPENDIX D SAMPLE GRADED APPROACH MATRIX 139 Activity: Water Treatment Facility (SP=Site Policy, CM=Corp. Manual, TP=Technical DATE: July 4, 1992 Procedure) CHAPTER GUIDELINES Activity IMPLEMENTING DEPTH OF DETAIL DEVIATIONS AND AND COMPANY POLICY PROCEDURES AND REQUIRED EXCEPTIONS DOCUMENTS SECTIONS OF (INCLUDING NON- APPLICABILITY APPLICABILITY) Chapter 2 - SHIFT ROUTINES ERC-CM-2-8 Introduction AND OPERATING PRACTICES ERC-SP-1984, Environmental Restoration Corporation, CONDUCT OF OPERATIONS MANUAL, Chapter 2. C.1 Status Practices ERC-TP-0084 Chapter II Used for Operations Section 5.1 Division, HPT's and Engineering Division C.2 Safety Practices ERC-TP-0084 Chapter II ERC-CM-12-3, MAN- Section 5.1 & 5.2 AGEMENT RE- QUIREMENTS AND PROCEDURES MWP 5.7, Industrial Safety and Fire Protection ERC-CM-8-4, INDUSTRIAL SAFETY MANUAL Vol. 1, Safety Standards Vol. 2, Safety Guides C.3 Operator Inspection Tours ERC-TP-0084 Chapter II II.C.3.c N/A for Salt Yard Section 5.2 C.4 Round/Tour Inspection ERC-TP-0084 Chapter II Sheets Section 5.1 & 5.2 DOE-EM-STD-5505-96 APPENDIX D SAMPLE GRADED APPROACH MATRIX 140 C.5 Personnel Protection ERC-TP-0084 Chapter II ERC-CM-12-3, Section 5.1 & 5.2 MANAGEMENT RE- QUIREMENTS AND PROCEDURES MWP 5.37, ALARA Program MWP 5.38, Radiation Protection ERC-CM-4-8, RADIA- TION PROTECTION C.6 Response to Indications ERC-TP-0084 Chapter II ERC-CM-1-5, STAN- Section 5.6 DARD OPERATING PRACTICES Section 8.4, Alarm Management C.7 Resetting Protective ERC-TP-0084 Chapter II Devices Section 5.7 C.8 Load Changes ERC-TP-0084 Chapter II Operations division II.C.8 For drinking water Section 5.8 operators only. filtration system, operator On-Shift training has authority to change operators can only process rate w/o permission operate equip. under of shift supervisor (low level supervision IAW ERC- of risk) TP-0084 C.9 Authority to Operate ERC-TP-0084 Chapter II Equipment Section 5.9

C.10 Shift Operating Bases ERC-TP-0084 Chapter II II.C.10 [4] Shift turnovers Section 5.10 are conducted in the lunchroom. (Except Control room operator and joint walkdowns.) C.11 Potentially Distractive ERC-TP-0084 Chapter II Written Material and Devices Section 5.11 DOE-EM-STD-5505-96 APPENDIX E SAMPLE OPERATIONS ASSESSMENT PLAN 141 ASSESSMENT PLAN Operations Organization and Administration: CHAPTER 1 Objectives: 1. Assess the measures taken by management to effectively implement and control operations. 2. Assess the success of those actions. Criteria: 1. Establishes written standards for operations. 2. Conducts periodic monitoring and assessment of performance. 3. Holds people accountable for performance. Observations: Observe actual operations in progress. - D&D activities in building AH-1. - Operators participating in on-the-job training. - Emergency Drill. - Routine operations. - Shift Supervisor tours and daily routines. - Sampling operation. - Procedure walkthrough. Observe management's involvement in the training program. - Actual training. Observe management's oversight/involvement in activity operations. - Management tours and daily routines. While doing observations — Are the operations safe? DOE-EM-STD-5505-96 APPENDIX E SAMPLE OPERATIONS ASSESSMENT PLAN 142 Interviews: Interview operators performing work in the activity. - Do they know why they are doing this job? - Do they understand the hazards of the job? - What training have they received and what is required? - Are physical requirements taken into account? - Do they know what to do if a mishap or emergency occurs? Interview first-line supervisors. - What is the guidance on safety of operations? - What are the organization's priorities? - What authority are they given to put safety first? - Are they held accountable for the actions of their shift? Interview management to assess intent of programmatic initiatives. - Is management involved in the training program, policy promulgation, sched- uling, personnel issues, and budgeting? Are goals for improvement used effectively? - Are roles and responsibilities clear? - How does management determine if staffing and resources are adequate? - Are people held accountable? During every interview try to determine: - How is safety of operations ensured? DOE-EM-STD-5505-96 APPENDIX E SAMPLE OPERATIONS ASSESSMENT PLAN 143 Document Reviews: Review activity performance documents regarding operations. - NPDES permits. - Radiological Worker permits. - Safety Work permits. Review activity instructions to supervisors and operators. - Can management use instructions to effectively communicate activity goals to supervisors and operators? Review activity policies and procedures regarding operations. - Is the organizational structure clearly defined? - How does management determine if staffing and resources are adequate? - Are responsibilities and authority clearly defined for all positions? - Are personnel held accountable for their actions? Review performance indicators (PIs). - Are PIs used to improve the quality of operations? - Are PIs tracked, evaluated, and changed to meet the needs of the activity? DOE-EM-STD-5505-96 APPENDIX F SAMPLE SCHEDULE FOR AN OPERATIONS ASSESSMENT 144 BLDG. 001 ASSESSMENT SCHEDULE TIME MONDAY TUESDAY WEDNESDAY THURSDAY 06-08 OBS: HP Tech (6) OBS: Filter Cleaning (4) 08-09 INT: Shift Sup. (3) 09-10 INT: Operations Sup (3) INT: HP Tech (6) INT: Maintenance Sup (3)

10-11 OBS: Evaporator OBS: Chemical INT: H. Stern (3) INT: 2 Operators Startup (2) Analysis (6) (6) 11-12 INT: B. Barker (6) INT: Activity Eng (3) 12-13 OBS: Drum Packing OBS: Valve Maintenance (4) OBS: Activity Drill Operations (4) (ALL) 13-14 INT: Safety Compli- INT: Env. Compliance Mgr INT: Activity Mgr (1) ance Mgr (1) (1) INT: Operator (2) INT: 2 Operators(2) INT: 2 Operators (2) INT: HP Tech (6) INT: J. Hahn (6) OBS: Training Shift (3) 14-15 INT: Operator Trainee INT: Training Mgr (1) INT: 2 Operators (6) (1) INT: Shift Sup. (1) 15-16 INT: On-Shift Operator INT: Maintenance Coordinator Trainer (1) (1) 16-18 Debrief Debrief Debrief Debrief OBS: Observation INT: 1-Hour Interview Team Members: 1 - Bob Green 4 - Paul Mitchell 2 - David Lynch 5 - Gary King 3 - Carl Sipowitz 6 -Steve Little DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 145 OPERATIONS ASSESSMENT Activity Name Location Date of Assessment An Operations Assessment of ACTIVITY NAME at ACTIVITY LOCATION was conducted from DATE OF ASSESSMENT. The members of the assessment team are listed in Attachment 2. The basis for the graded approach is a matrix of applicability as called for in DOE Order 5480.19. A matrix addressing applicability HAS/HAS NOT been produced for ACTIVITY NAME. The findings enumerated in Attachment 1 lead to the following concerns. Although the findings are listed by the chapter in the DOE Order, the concerns are grouped to assist management action. Some findings did not rise to the level of a concern. When reading and interpreting this report the following definitions apply: - A FINDING is an individual item that does not meet requirements. - A CONCERN is a determination of a programmatic breakdown or widespread problem supported by one or more findings. CONCERNS ACTIVITY MANAGEMENT 1. Management oversight processes do not ensure implementation of some tank farm policies and procedures. In some cases, facility practice departs significantly from approved administrative policies and/or from management expectations. Such areas include lockout/tagout, temporary modifications, radiological controls, alarm management, and the facility monitoring program. CONTROL OF OPERATIONS 2. Shift routines at the Computer Automated Surveillance System (CASS) station are informal. CASS operators do not ensure that alarm status and facility configuration are communicated between the CASS station and the cognizant tank farm shift managers. MAINTENANCE 3. Temporary modifications are not controlled in accordance with facility procedures. DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 146 4. Safe condition checks are not consistently performed in accordance with approved Company procedures. However, no immediate hazard to personnel safety was identified during the review. RADIOLOGICAL CONTROL 5. Fundamental contamination control practices need significant improvement. This concern applies to both workers and radiological control technicians. 6. The Radiological Problem Reporting (RPR) program is not effective at identifying and correcting radiological deficiencies. 7. Radiological surveys taken by Radiological Control Technicians are inadequate because the results of the surveys are not available until after the work is completed, in process contamination surveys are not taken, and radiological hold points are not incorporated into work procedures.

8. Radioactive material is not properly controlled because radioactive material is stored outside radioactive material storage areas, not properly marked or labeled, and not enclosed in yellow poly bags. DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 147 OPERATIONS ASSESSMENT ACTIVITY NAME Attachment 1 FINDINGS DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 148 OPERATIONS ORGANIZATION AND ADMINISTRATION REFERENCES: 1. DOE 5480.19, Chapter I 2. IP-0842 Volume II Section 4.1.1 3. IP-0842 Volume II Section 4.14.1 4. IP-0842 Volume II Section 4.3.3 Rev 3 5. IP-0842 Volume IV Section 3.3 Rev 0a 6. IP-0842 Volume II Section 4.1.2 FINDING : There is no documented Management Oversight Program (MOP). The procedure referenced in IP-0842 for the MOP does not exist. (References: 1, 2) [Concern: 1] FINDING : IP-0842, Vol II, Section 4.1.2, Rev 1A, paragraph 5.1.4, Facility Monitoring Program is not being implemented as written. (References: 1, 2) [Concern: 1] DISCUSSION: The monthly reports are not being submitted as required. In order to fully utilize the information obtained from this program, the material needs to be read and analyzed and subsequently reported to the level III manager involved for action/training. This is presently not being done. Only the raw data is being forwarded. FINDING : The Activity does not have written auditable goals established by the line manager as required by company procedure. (References: 1, 2) [Concern: 1] FINDING : The facility applicability matrix contained in reference 2 refers to a superseded procedure for equipment and piping labeling. (References: 1, 4, 5) FINDING : The facility matrix of applicability contained in reference 2 incorrectly refers to a subsection of reference 3 for required reading. (References: 1, 2, 3) DISCUSSION: In the matrix, guideline C.1 for Chapter 14, "Required Reading" refers to subsection 5.1 of reference 3. It should refer to section 5.0.1. DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 149 SHIFT ROUTINES AND OPERATING PRACTICES REFERENCES: 1. DOE 5480.19, Chapter II 2. Company Control Manual 4-41 3. CM-4-3, Industrial Safety Manual, Volume 1, Section E-2 4. IP-0842, Vol II, Section 4.2.1 5. SD-WM-HSP-002, Activity Health and Safety Plan FINDING : The Computer Automated Surveillance System (CASS) operator did not inform the Shift Manager of all alarms received. (References: 1, 4) [Concern: 2] DISCUSSION: In the CASS, no action can be taken to correct the alarm; all information must be passed on the appropriate shift supervisor. FINDING : Two persons working in a construction area where elevated work activities were occurring were not wearing hard hats. (References: 1, 5) DISCUSSION: Correction was provided on the spot by tank farm personnel. FINDING : Individuals reported to work in shorts and/or tank tops. (References: 1, 4) DISCUSSION: Since the average high temperature during the assessment period was approximately 100 degrees, this requirement was not realistic unless there was an underlying safety issue. No person interviewed could specify a safety issue behind this requirement. DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 150 FINDING : One operator on rounds entered the Exhaust System Building without wearing hearing protection. The building was posted as hearing protection required. (References: 1, 5)

FINDING : Scaffolding located near the 105 and 106 exhaust piping was overdue for inspection. (References: 1, 5) FINDING : Numerous inaccurate or out of service alarms and indications do not have a "Service Required" label. (References: 1, 4) [Concern: 1] FINDING : A full five (5) gallon gasoline can was found unattended at the west gate. (References: 1, 2) FINDING : A worker was observed to work near an energized circuit of 110 volts and greater than 1 milliamp without using safety gloves. (References: 1, 3) DISCUSSION: During the operational retest of valves MOV-3 and 5, a worker removed the cage access panel behind the control panel and entered the cage area which contains unguarded and uninsulated energized parts. The worker then attached an ammeter clamp on an energized circuit. When the two MOVs were operated the worker observed the current through the ammeter. When the test was complete the worker removed the clamp and exited the cage. The worker was not wearing rubber gloves as required by reference 3. NOTIFICATIONS REFERENCES: 1. DOE 5480.19, Chapter VII 2. CM-1-5 3. IP-0842 Volume II Section 4.6.2 DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 151 FINDING : Four of 18 Occurrence Reports reviewed were submitted beyond the required time limits.(References: 1, 2, 3) DISCUSSION: Occurrence reports from 17 February 1995 to present were reviewed. The four overdue reports were: - RC-C-Tank Farm-1995-0015 - RC-C-Tank Farm-1995-0025 - RC-C-Tank Farm-1995-0031 - RC-C-Tank Farm-1995-0033 CONTROL OF EQUIPMENT AND SYSTEM STATUS REFERENCES: 1. DOE 5480.19, Chapter VIII 2. CM-1-8, Section 2.0 3. CM-6-1, EP-2.2 4. IP-0842, Volume II, Section 4.8.1 5. IP-0842, Volume IV, Section 4.5 6. IP-0842, Volume IV, Section 5.7 FINDING : Ten of 43 work packages released on 7/18/95 were not listed on the Daily Release Sheet. (References: 1, 2, 4) DISCUSSION: Six of these documents were reported as field work complete and 4 of the documents were deferred or contingency work. FINDING : Two of three Shift Managers interviewed were not familiar with the use of the Facility Status Binder for LCO status. (References: 1, 2, 4) DISCUSSION: The Shift Managers interviewed were knowledgeable of the safety status associated with the activity; however, two were not familiar with the Facility Status Binder which documents the mode for the tanks to ensure activities are not performed outside of the specified safety requirements. DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 152 FINDING : Fifteen of the 16 identified temporary modifications listed in the Operations Modification Log are greater than 90 days old. (References: 1, 3, 5) [Concern: 3] FINDING : Documentation does not exist to officially extend temporary modifications beyond the 90 day limit. (References: 1, 3, 5) [Concern: 3] FINDING : Three active temporary modifications identified in the JCS are not listed in the Operations Modification Log. (References: 1, 4, 5) [Concern: 3] FINDING : The review of temporary modifications is not performed as required by the facility procedure. (References: 1, 5) [Concern: 3] FINDING : Four items in building 2 had expired calibration or functional stickers affixed to them. (References 1, 5) DISCUSSION: Items included the following: - FRC 28 position indicator on control panel due 5/95. This calibration sticker has the wrong due date. It should be 11/28/96. - Magnehelic DP1-7 in ventilation duct due 1/90. This calibration was verified not required by OSR or OSD. - Gage P1-SW-3 on safety shower due 6/92

LOCKOUTS AND TAGOUTS REFERENCES: 1. DOE 5480.19, Chapter IX 2. CM-4-3, Industrial Safety Manual, Volume 1, G-1 3. IP-0842 Volume II Section 4.9.1 DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 153 FINDING : Tag A Complex-95-017-3 (1 of 8 tags checked) does not have a "Verified By" signature on it. (References: 1, 2, 3) DISCUSSION: This tag is outside and exposed to weather, and the verification signature may have faded; however, all other information on the tag was clearly legible. FINDING : Tagout Authorization B Complex-95-029 does not have a Technical Reviewer signature for the second tagout for job ES-95-00041/0. (References: 1, 2, 3) [Concern: 1] FINDING : Tagout Authorization form B Complex-95-017 does not have a date next to the signatures in Block 7 "Work Completed/Date." (References: 1, 2, 3) [Concern: 1] DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 154 FINDING : Some Tagout Authorization forms had missing monthly surveillances. (References: 1, 2, 3) [Concern: 1] DISCUSSION: Examples are: 1. A Complex-95-002 missing 3/95. 2. A Complex-95-007 missing 5/95. 3. A Complex-94-062 missing 2/95. 4. A Complex-94-028 missing 5/95. FINDING : Safe condition checks are not documented in a consistent manner on Tagout Authorization forms. (References: 1, 2, 3) [Concern: 4] DISCUSSION: Examples of Tagout Authorization forms with varying methods of documenting the completion of safe condition checks are: A Complex-95-017 A Complex-95-011 A Complex-95-024 A Complex-95-002 A Complex-94-062 A Complex-94-036 AN-95-009 B Complex-95-017 Some safe condition checks are annotated with zero voltage checks, others are annotated by signing block 26, and others are annotated by lining out block 26 and noting the type of check performed. DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 155 FINDING : Safe condition checks in block 26 on several Tagout Authorization forms do not ensure safe conditions are established for each of the hazard types identified in block 9. (References: 1, 2) [Concern: 4] DISCUSSION: Examples are: 1. A Complex-95-022 states that the hazard is "Contamination" but safe condition check is marked "N/A". 2. Tagouts A Complex-95-015 and A Complex-95-014 state that the hazard is "Personnel Contamination" but safe condition checks are marked "N/A". 3. A Complex-95-013 states that the hazards are "Explosion, Fire" but safe condition check is marked "N/A". 4. A Complex-95-011 states that the hazards are "Electrical shock and flooding of raw water pump out pit 01A." The safe condition check indicated was a zero voltage check only which did not address the safe isolation for low temperature/pressure fluid systems as required in Section 7.3.1 of Reference 2. FINDING : Contrary to Reference 2, caution tags are being used for components or systems that are not functional. (References: 1, 2) DISCUSSION: Examples are: 1. Caution tagout R-95-004, 244-AR Compressed Air, states that the compressor is out-of-service. 2. Caution tagout B-95-028, 241-B Air Compressor, states that the compressor does not operate. FINDING : Caution tagout AR-95-003 does not specify a precaution or item of information necessary before the operation of the components covered. (References: 1, 2) DISCUSSION: The tagout states in block 10, "Leave breaker in OFF position. Do not operate fan. Operation not covered by SARP." This information does not specify a precaution to be met prior to operation.

DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 156 FINDING : A letter from the project manager or delegate designating the lock and tag administrators with lock and tag authority is not located in the Lock and Tag Notebook as required by section 4.1 of reference 2. (References: 1, 3) [Concern: 1] LOGKEEPING REFERENCES: 1. DOE 5480.19, Chapter XI 2. WHC-IP-0842, Vol II, Section 4.11.1. 3. WHC-IP-0842, Vol II, Section 4.2.1 4. WHC-IP-0842, Volume IV, Section 4.5 FINDING : Guidance for the keeping of the CASS narrative log is not provided, consequently conditions that cause alarms at the CASS are not recorded. One condition that caused the alarms that were not recorded was the tank exhauster shutdown by the X Tank Farm. (References: 1, 2) [Concern: 2] FINDING : Red ink was used to fill out an operator shift turnover checksheet. (References: 1, 3) FINDING : The Operations Modification Log is not maintained current and accurate. (References: 1, 4) DISCUSSION: Several issues were identified during a random review of the Operations Modification Log. These issues include a tagout for temporary modification that was cleared; a tagout that was changed without updating the Operations Modification Log; a copy of an out-of date procedure (IP-0842, Section 8.7) in the front of the log; and several other corrections that were made without any initials or date entries. OPERATIONS TURNOVER REFERENCES: 1. DOE 5480.19, Chapter XII 2. IP-0842, Vol II, Section 4.12.1 DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 157 FINDING : Three of three CASS Operators interviewed could not demonstrate how they reviewed CASS status log items referenced in the shift turnover checksheet. The shift turnover checksheet requires use of a status log to review equipment malfunctions and failures, operations/maintenance and work in progress, system/equipment status, etc. (References: 1, 2) [Concern: 2] DISCUSSION: A status log does not exist at the CASS and the status board is not updated to provide the information required to be reviewed at shift turnover. Note that the Operating Procedure, TO-020-755, section 5.3.1 requires that each facility maintain a status log, but this procedure does not cover the CASS. OPERATIONS ASPECTS OF FACILITY CHEMISTRY AND UNIQUE PROCESSES REFERENCES: 1. DOE 5480.19, Chapter XIII 2. IP-0842, Vol II, Section 4.13.1 FINDING : The CASS was not informed that the X Tank Farms tank exhauster was to be shutdown therefore they did not realize that ten (10) alarms would be received. (References: 1, 2) [Concern: 2] FINDING : One of three CASS operators did not know what OSR equipment and associated alarms existed at the CASS. (References: 1, 2) [Concern: 2] REQUIRED READING REFERENCES: 1. DOE 5480.19, Chapter XIV 2. IP-0842 Volume II Section 4.14.1 DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 158 FINDING : The required reading books for A and C shifts in the Shift Operations Office had 12 delinquent items dating from 31 May 1994 to 3 January 1995. Of those 12 items the percentage of personnel completing the item varied from 0% to 83%. (References: 1, 2) DISCUSSION: The delinquent required reading items were removed from the required reading book on 18 July 1995. FINDING : There was one open and delinquent item listed in the required reading log. This item was issued 26 April 1995 and the required completion date was 10 May 1995. T Shift is the only group that has not completed this required reading assignment. (References: 1, 2)

TIMELY ORDERS TO OPERATORS REFERENCES: 1. DOE 5480.19, Chapter XV 2. CM-4-3, Industrial Safety Manual, Volume 1, G-1 3. IP-0842 Volume 2 Section 4.9.1 FINDING : Standing Order 95-05, "Lock and Tag" Revision 1 dated June 2, 1995 contradicts section 4.0 of reference 2. (References: 1, 2, 3) [Concern: 1] DISCUSSION: The standing order states that zero energy checks will be performed in advance of hanging tags, and then annotated on block 26 of the Tagout Authorization form. Reference 2 states that safe condition checks will be performed after the tags have been hung and verified, then indicated on block 26 of the Tagout Authorization form. FINDING : There is no method to verify that personnel read the Standing Orders (Reference: 1) DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 159 OPERATIONS PROCEDURES REFERENCES: 1. DOE 5480.19, Chapter XVI 2. Procedure TO-250-550 Step 4.1 3. IP-0842 Vol V Section 4.1 Para 2.0 4. IP-0842 Vol II Section 5.2 5. IP-0842 Vol II Section 4.10.1 FINDING : Step 4.1 of the Transfer from R to TK-12 Procedure was conducted and signed by Operations Engineer vice the Person in Charge (PIC). (References: 1, 2) FINDING : No safety precautions, job hazards, or potential abnormal events were discussed at the pre-job brief for the Transfer from R to TK-12. (References: 1, 3) FINDING : Transfer from R to TK-12 Procedure references the wrong revision/change for the transfer from R Tank Trailer to Tank Farms Procedure. (Reference: 1, 4) FINDING : Transfer from R Tank Trailer to Tank Farm Procedure references the wrong revision/change for the Transfer from R to TK-12. (Reference: 1, 4) FINDING : Step 5.1.1 of Transfer from R to TK-21 Procedure requires verification that R to TK-12 Checklist - Section A located on page 26 has been completed; no checklist is located on page 26. (Reference: 1, 4) FINDING : Transfer from R Rail Car to Tank Farm Procedure does not require that a tank be lined up to receive the waste as a prerequisite. (Reference: 1, 4) FINDING : Transfer from R Rail Car to Tank Farm Procedure Steps 5.5.21 and 5.5.24, and 5.3.18.2 and 5.3.18.3 are out of order. (Reference: 1, 4) DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 160 FINDING : Transfer from R Rail Car to Tank Farm Procedure Steps 5.5.49.16 and Steps 5.5.47.1. through 5.5.47.9 are not in a logical order. (Reference: 1, 4) FINDING : Transfer from AR Rail Car to Tank Farm Procedure only references actions for alarms at R and does not reference actions for alarms received at CASS, A, or at CR. (Reference: 1, 4) FINDING : Procedure TO-250-550, "Transfer from R to TK-12" did not include the Independent Verification Self Checking Techniques checklist as required in reference 5. (References: 1, 5) DISCUSSION: Independent verification associated with procedure TO-250-550 was performed correctly by the workers involved; however, they did not use the checklist as required. The Cognizant Engineer and workers performing the procedure were not aware of the checklist. OPERATOR AID POSTINGS REFERENCES: 1. DOE 5480.19, Chapter XVII 2. IP-0842 Vol II Section 4.17.1 FINDING : Operator Aids 2E-91-005 and 2E-94-010 are not labeled correctly. (References: 1, 2) FINDING : The Operator Aid Audit Checklist (Attachment C to IP-0842 Vol II Section 4.17.1) is not used. (References: 1, 2) FINDING : The Operator Aid Index (Attachment B to IP-0842 Vol II Section 4.17.1) is not used. (References: 1, 2)

FINDING : Operator Aid 2E-92-003 references a source document which was approved after the Operator Aid. (Reference: 1, 2) DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 161 FINDING : The following unapproved operator aids were discovered: a. TOA Intercom System Use located in building 204-AR; b. How to Respond to an Alarm located in CASS; c. Excerpt from Cross Site Transfer Procedure (TO-140-90 p.57) located in CASS d. Waste Cooling Water Meters TDRC 413,414, and 415 Multiplication Aid located in building 241-A-271. (Reference: 1, 2) EQUIPMENT AND PIPING LABELING REFERENCES: 1. DOE 5480.19, Chapter XVIII 2. IP-0842 Volume IV Section 3.3 Rev 0a FINDING : Two labels in the AP tank farm were overwritten with marker. (References: 1, 2) DISCUSSION: The label for pump P-3 has "Mixer pump" written in marker over the label. The label for the pump rotation motor has "Mixer" written in marker over the label. FINDING : In tank farm valve MK-9518 had no label, although documentation shows a label had been installed. (References: 1, 2) FINDING : Several loose component labels were found (not attached to their components). (References: 1, 2) DISCUSSION: Examples are: 1. The label for the 16-N Conductivity Probe Purge Air Valve in the N tank farm was found on the ground. 2. The labels for the TMACS cabinets in the X/Y tank farms were found on the ground. DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 162 RADIOLOGICAL CONTROLS REFERENCES: 1. Site Radiological Control Manual (SRCM) FINDING : Numerous personnel failed to minimize skin contact with potentially contaminated surfaces. (Reference: 1) [Concern: 5] DISCUSSION: Personnel contamination control work practices were observed during work at Y-Farm, X-Farm, and C-Farm. Approximately fifty percent of the personnel observed exhibited poor contamination control work practices. Examples include: More than ten personnel were observed touching their nose or face with PCS while working inside a Contamination Area (CA), Other problems included wiping nose or face with sleeve of PCS, resting forehead on arm sleeve of PC during rest period, pushing up glasses with potentially contaminated gloves. One person working inside a CA removed his hood, put it in his pocket, and put on a hard hat. Several poor contamination control work practices were observed during doffing of protective clothing: one worker adjusted his personal clothing with potentially contaminated gloves; one person put his potentially contaminated hardhat back on his bare head; one person rested his bare arm on the door inside the contamination area while waiting to perform a whole body frisk; two of three personnel observed used improper technique when removing their respirators (the workers lifted the respirators and pulled them over their head); two workers used improper technique when removing anti-C hoods; one person was observed retrieving material from inside a used anti-C bin wearing only latex gloves; one person was observed walking over a contaminated laydown surface that had been used to prevent the spread of contamination from the highly contaminated bottom surface of the DCRT cover block. FINDING : Most personnel observed performing personnel contamination monitoring at tank farm contamination control points displayed poor frisking practices. (Reference: 1) [Concern: 5]

DISCUSSION: Seven of nine health physics technicians and workers observed performing frisking at contamination control points at N, Y, X, and C farms displayed significant deficiencies in frisking technique. In 15 whole body frisk evolutions observed, significant errors were made in 11 cases. Errors included frisking much too fast, holding the probe too far from the surface being surveyed, not pausing at nose and mouth, and not covering all surfaces of the body. DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 163 FINDING : Thirteen of Fifteen personnel observed did not doff Personal Protective Clothing in accordance with posted instructions. (Reference: 1) [Concern: 5] DISCUSSION: Personnel undressing techniques were observed at C-Farm, Y-Farm, and X-Farm. Of the fifteen personnel observed, thirteen did not remove their PCS in accordance with the posted undressing procedures. Personnel removed hoods, rubber shoe covers and gloves out of sequence. FINDING : One person was observed chewing gum inside a Contamination Area. (Reference: 1) [Concern: 5] DISCUSSION: During work on DCRT cover block removal on 7/18/95, at X-Farm, one person was observed chewing gum inside the Contamination Area. FINDING : One worker entered a Contamination Area without wearing Personal Protective Equipment and Clothing required by the Radiological Work Permit (RWP). (Reference: 1) [Concern: 5] DISCUSSION: A worker installing a step-off pad at X-Farm, entered the Contamination Area (CA) without wearing Personal Protective Clothing (PCS). The worker was on his hands and knees inside the CA. The worker was signed in under RWP E-704. The RWP required a full set of PCS. Discussions with the Radiological Control Technician (RCT) indicated the area had been surveyed earlier and found to be uncontaminated. If the area is not contaminated, the boundary of the Contamination Area should have been collapsed in lieu of having personnel violate RWP requirements. DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 164 FINDING : The Radiological Problem Report (RPR) system is not being effectively used to identify and correct radiological deficiencies in tank farm contamination areas. (Reference: 1) [Concern: 6] DISCUSSION: A large number of radiological posting and housekeeping deficiencies exist in the tank farm contamination areas. Most of these deficiencies have not been entered into the RPR system. Interviews with operators, health physics technicians, and managers indicate that the RPR system is not effective as presently used. Example: Approximately 160 RPRs were used in calendar year 1994. So far this year only 30 RPRs have been issued. Example: A large number of reported problems were not being corrected. Of the approximately 190 RPRs issued during 1994-1995, as of June 1995, 105 RPRs were not closed. FINDING : Housekeeping in tank farm contamination areas is poor. (Reference: 1) [Concern: 6] DISCUSSION: This finding is supported by the following items: In C tank farm: - Open bags of radioactive material were lying inside a sheet metal work shed (which had blown over onto its side). - Various yellow poly bags were adrift inside the CA boundary. - Tools, cable yokes, were adrift. - A large roll of unused yellow poly was adrift. - Several air hoses were lying about. - Several empty 55 gallon drums labeled with faded radioactive material labels were found (operators and health physics technicians interviewed did not know their purpose).

In X tank farm: - A truck was found with used yellow poly bags, used tape, trash, used absorbents, air sample filter heads, wrenches, tools, and a Scott Air Pack lying inside. - Used, aged yellow poly bags were adrift in the cover block area. At diversion box 12-E: - A laundry bag containing used anti-C's located at the contamination area step-off pad was filled to overflowing. - A used latex glove was found strewn about outside the contamination area. - Used poly bags were adrift inside the contamination area. DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 165 At 2-R: - An overfilled RAM waste receptacle was found with contaminated material spilling out of the top. FINDING : A temporary Airborne Radioactivity Area at X Tank Farm was not posted on two sides. (Reference: 1) [Concern: 6] DISCUSSION: The temporary Airborne Radioactivity Area established for DCRT cover block removal at X Tank Farm was not posted properly. After removal of the cover block, personnel decided to expand the posted Airborne Radioactivity to include a larger area down wind of the pit. The expanded area was not posted on two sides. FINDING : The radiological postings and barriers at three locations in C tank farm had fallen to the ground. (Reference: 1) [Concern: 6] DISCUSSION: - A contamination area near the entrance to C-farm (MO-22) had a barrier stanchion that had fallen down. As a result, the barrier rope and contamination area sign were on the ground. - The posting around a contaminated cabinet near tank 110 inside C-farm had fallen down. The barrier rope and one contamination area sign were on the ground. - One side of a High Contamination Area / Radiation Area located inside C-farm is unposted. On the opposite side, the posting is down on the ground. FINDING : Issue and return of filtered respirators are not positively controlled to ensure that only trained and qualified personnel wear respirators. (Reference: 1) DISCUSSION: During cover block removal at the B, X tank farms and DCRT pit, workers were observed passing a box of clean respirators over the fence for use by personnel involved in the cover removal operations. Discussions with health physics supervision verified that tank farms has no respiratory equipment control program. The health physics department has previously identified this issue to tank farms management. DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 166 FINDING : Installed temporary shielding on a pipe in C-Farm was not properly marked. [Concern: 6] DISCUSSION: A pipe in the C-Farm has shielding chained and locked in place. The shielding is not marked in accordance with Article 314.5. FINDING : Barrier tape and rope used for two radiological areas were not yellow and magenta in color. (Reference: 1) [Concern: 6] DISCUSSION: Construction "Caution" tape was used as a barrier tape for the temporary airborne radioactivity area established for DCRT cover block removal at X Tank Farm on 7/18/95. Yellow rope was used as a barrier for the underground radioactive material area adjacent to the entrance trailer for the Y Tank Farm. FINDING : Three radiological areas within C-Farm were posted with out of date signs that do not meet reference 1 specifications. (Reference: 1) [Concern: 6]

DISCUSSION: An out of date High Radiation Area sign was posted on a shielded pipe. The posting states "Caution, High Radiation Area" and does not contain entry requirements. An Airborne Radioactivity/Radiation Area inside a tent was posted with out of date Airborne Radioactivity signs. A High Contamination/Radiation Area was posted with a mixture of out of date signs and new signs. The old signs do not specify entry requirements. DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 167 FINDING: Two Radioactive Material Areas were unnecessarily established inside Contamination Areas. (Reference: 1) DISCUSSION: A Radioactive Material Area (RMA) has been established inside the 106C tent located in C-Farm, a Contamination Area. A RMA was established inside the 244A lift station Contamination Area. This posting is not required by the HSRCM. To minimize unnecessary exposure and generation of radioactive waste, WHC should not post RMAs inside contaminated areas. FINDING : Anti-C doffing procedures posted at four out of four East Tank Farm exits observed, did not reflect all appropriate undressing instructions. (Reference: 1) [Concern: 6] DISCUSSION: The removal sequence of Personal Protective Clothing for the DCRT cover block removal included removal of respiratory protection first. This sequence was not reflected in undressing procedures. Interviews with personnel indicated that a double set of Personal Protective Clothing (PCS) is used in some instances at the tank farm. The standard sign for undressing does not include the sequence for removal of double PCS. The undressing procedure does not address removal of tape on inner gloves that is covered by the outer gloves. The procedure only discusses removal of exposed tape prior to removing outer gloves. Several workers were observed pulling the coveralls off with the inner gloves still taped to the outside of the coverall. FINDING : Buffer Area adjacent to X Tank Farm was not properly posted. (Reference: 1) [Concern: 6] DISCUSSION: The buffer area adjacent to X Tank Farm entrance was posted as a contamination area on two sides. A contamination area sign was posted on the fence bordering the buffer area outside the Tank Farm. The contamination area sign on the rope between the buffer area and Tank Farm contamination area was facing the wrong direction, indicating the buffer area was a contamination area. The buffer area sign on the rope was hanging by one end. DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 168 FINDING : Yellow and magenta radiological boundary chain was observed at two locations without radiological posting identifying the hazard. (Reference: 1) [Concern: 6] DISCUSSION: On the west side of C-Farm, an unposted area in front of the locked gate is bounded by a yellow and magenta chain. A yellow and magenta chain has been used to secure a gate on the south end of A-Farm, adjacent to the entrance. FINDING : Several hoses and cords lying across Contamination Area boundaries were found unsecured. (Reference: 1) DISCUSSION: Several hoses or cords entering C-Farm, an electrical cord entering the 12-ER Diversion Box Contamination Area (CA), and hoses entering the 244A lift station CA were not marked or secured. FINDING : The status board for C-Farm was not updated with the most recent survey results. (Reference: 1) DISCUSSION: The status board at C-Farm was last updated on 7/7/95. The current survey results were not posted. This was observed on 7/18/95.

FINDING : Seven radiological postings were found adrift in the East Tank Farms Area. (Reference: 1) [Concern: 6] DISCUSSION: The following postings were found adrift: - A Soil Contamination Area posting located on the north side of C-Farm. - An Underground Radioactive Material sign on a road north of the railroad tanker car High Radiation Area enclosure. - A Radioactive Material Area sign near 244-AR. - Four radiological posting entry requirements inserts: two inside C-Farm, near a High Contamination/Radiation Area; one near A-Farm, across from 204-AR; and one on the north side of W-Farm, east of 241AW-801, next to the fence. DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 169 FINDING : The radiological control posting inserts that describe entry requirements were found missing on four signs. (Reference 1) [Concern: 6] DISCUSSION: The entry requirements insert for a Contamination Area sign located on the east side of C-Farm was missing. Entry requirement inserts on two High Contamination/Radiation Area signs inside C-Farm were missing. The sliding pieces were adrift near the area. The entry requirement insert was missing for a Contamination Area sign located on the NE side of A-Farm, on a barrier around a valve station. Two Radioactive Material Area signs posted on the entrances to an RMA located near the Bldg.272-AW WRAM station had pieces of paper with hand written information in the insert pocket. FINDING : Several scan surveys using Portable Alpha Meters (PAM) were improperly recorded as having a MDA of 20 DPM/100cm2. (Reference 1) DISCUSSION: A scan survey using a PAM can not detect contamination levels as low as 20 dpm alpha. During an observation of radiological control technician routine surveys, alpha levels measured with a PAM were recorded as less than 20 dpm. A review of other survey records indicated similar survey level errors. FINDING : One potentially contaminated tool was found packaged in clear plastic at the 244-A lift station. (Reference: 1) [Concern: 6] FINDING : During a gasket replacement in a section of contaminated piping, the procedure did not specify radiological hold points. (Reference: 1) [Concern: 7] FINDING : During a gasket replacement in a section of contaminated piping, an RCT only took swipes before and after the work, but the RCT did not take any swipes while the interior surfaces of the pipe and flange were exposed. (Reference: 1) [Concern: 7] DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 170 FINDING : During a gasket replacement in a section of contaminate piping, portable air samples were not measured for airborne contamination levels. (Reference: 1) [Concern: 7] FINDING : RWP B-320 estimated the maximum contact radiation levels of 50 mrem/hr. The survey record for the job stated that actual levels were 200 mrem/hr. (Reference: 1) [Concern: 7] FINDING : During a gasket replacement in a section of contaminate piping, the operator exited the contamination area and deposited his protective clothing in a clear poly bag and carried the bag from the contamination area without sealing or labeling the bag as radioactive material. (Reference: 1) [Concern: 8] FINDING : During a gasket replacement in a section of contaminate piping, contaminated tools were not marked or otherwise identified as radioactive material. (Reference: 1) [Concern: 8] FINDING : Radioactive material is not properly controlled.. (Reference: 1) [Concern: 8]

DISCUSSION: The following improperly controlled radioactive material was identified: - At Plant B, several bags of radioactive material were stored outside the radioactive material storage areas. - In the change room, a table that is marked as radioactive material was found outside of a radioactive material storage area. - Radioactive sources in the custody of the radiological control organization are not inventoried. A list of all sources is not maintained by the facility radiological control organization. - At SF, a potentially contaminated pump had been removed from the pool water circulating system and placed on the deck. There was no indication that the pump was surveyed in order to be released from radiological control. No records for internal surface surveys were available. DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 171 TRAINING REFERENCES: 1. DOE 5480.20A Ch. 1 Para 15B 2. DOE 5480.20A Ch. 1 Sect. 7.d(3) FINDING : Twelve (12) of 14 training records reviewed contained overdue training requirements. (Reference: 1) FINDING : Facility specific drills are not conducted. (Reference: 2) DOE-EM-STD-5505-96 APPENDIX G SAMPLE OPERATIONS ASSESSMENT REPORT 172 U.S. DEPARTMENT OF ENERGY Attachment 2 CONDUCT OF OPERATIONS ASSESSMENT Assessment Team Members B. Bertha DOE-HQ B. Bubble DOE-HQ K. Cobra DOE-HQ P. Zing DOE-Operations Office B. Crenshaw DOE-Operations Office J. Daly DOE-Operations Office C. Pavin DOE-Operations Office S. Elkington ABC, Inc. G. Norman ABC, Inc. DOE-EM-STD-5505-96 INDEX 173 Activity definition 2 Assessment In-brief 14 Assessment Out-brief 30 Assessment Plan Development 12 sample plan 141-143 Assessment Planning 9 Assessment Preparation 7 sample checklist 137-138 Assessment report distribution 30 format 29 generation 29 sample report 145-172 Assessment schedules assessment schedule for an operations assessment 13, 144 two-year schedule 4, 135 Assessment Techniques 15 Communications assessing with Shift Routines and Operating Practices 36-43 assessing separately 48-53 Concern definition 2, 27 developing 29 sample assessment report 145-146 DOE-EM-STD-5505-96 INDEX 174 Control Area Activities assessing with Shift Routines and Operating Practices 36-43 assessing separately 44-48 Control of Equipment and System Status assessing 65-69 Control of On-Shift Training assessing 53-56 Daily Meetings with Activity Management 27 Daily Team Meetings 26 Drills assessing 129-133 Equipment and Piping Labeling assessing 114-117 Activity Briefing and Tour 13 Finding definition 3, 27 developing 23 sample assessment report 145-172 Graded-approach document for conduct of operations 10 assessment report 145 sample matrix 139-140 Independent Verification assessing with Control of Equipment and System Status 65-69 assessing separately 77-80 Investigation of Abnormal Events assessing 57-60 DOE-EM-STD-5505-96 INDEX 175 Lead definition 23 developing 23-25 Lockouts and Tagouts assessing with Control of Equipment and System Status 65-69 assessing separately 70-75 Notifications assessing 61-63, 129 Operations Aspects of Facility Chemistry and Unique Processes assessing with Shift Routines and Operating Practices 36-43 assessing 90-94 Operations Organization and Administration assessing 31-35 Operations Procedures assessing 103-108 Operations Turnover assessing with Shift Routines and Operating Practices 36-43 assessing separately 86-89

Operator Aid Postings assessing 109-113 operator aid test 110 Pre-assessment Information Review 9-12 Pursuing leads example of 24-26 process description 23-24 Radiological Control assessing 117-123 DOE-EM-STD-5505-96 INDEX 176 Required Reading assessing 96-99 Resource Requirements 8 Shift Routines and Operating Practices assessing 36-43 Stop Work Authority 17 Team Composition 8 Team Leader 8 responsibilities 8 Team Members 9 responsibilities 9 Timely Orders to Operators assessing 100-103 Training assessing 125-128 DOE-EM-STD-5505-96 CONCLUDING MATERIAL Review Activity: Preparing Activity: DOE Field Offices DOE-EM-4 EM AL CH Project Number: ID OPER-0002 NV OAK OH OR RF RL SF SR National Laboratories Area Offices FOREWORD CONTENTS 1. INTRODUCTION 2. DEFINITIONS 2.1 Activity 2.2 Concern 2.3 Finding 3. USING THIS LIMITED TECHNICAL STANDARD 4. OPERATIONS ASSESSMENT PROGRAM 4.1 Operations Assessment Program Objective 4.2 Operations Assessment Program Criteria 4.2.1 Operations assessments conducted in accordance with a documented schedule. 4.2.2 Documentation of programmatic breakdowns or widespread problems (concerns) 4.2.3 Communication of concerns and findings to contractor management. 4.2.4 Follow-up to verify concerns are corrected. 4.2.5 Utilization of assessment results as a factor in determining contractor monetary 5. PERFORMING AN OPERATIONS ASSESSMENT 5.1 Operations Assessment Overview. 5.2 Assessment Preparation. 5.2.1 Resource Requirements. 5.3 Assessment Team Composition. 5.3.1 Team Leader. 5.3.2 Team Members. 5.4. Assessment Planning. 5.4.1 Pre-assessment Information Review. 5.4.1.1 Graded-Approach Document. 5.4.1.2 Activity-Specific Information. 5.4.1.3 Prior Assessments, Reports, etc. 5.4.2 Assessment Plan Development. 5.4.2.1 Approach. 5.4.2.2 Content and Format. 5.5 Activity Briefing and Tour. 5.6 Assessment Schedule. 5.7 Assessment In-brief. 5.8 Assessment Techniques 5.8.1 Observation. 5.8.1.1 What to Observe. 5.8.1.2 How to Observe. 5.8.1.3 Where and When to Observe. 5.8.1.4 Stop Work Authority. 5.8.2 Interview. 5.8.2.1 How to Prepare for Interviews. 5.8.2.2 Conducting Interviews. 5.8.2.3 Where and When to Conduct Interviews. 5.8.3 Document Review. 5.8.3.1 What Documents to Review. 5.8.3.2 Reviewing Documents. 5.8.3.3 When and Where to Review Documents. 5.8.4. Pursuing leads. 5.9 Daily Team Meetings. 5.9.1 Daily Meetings with Activity Management. 5.10 Findings and Concerns. 5.11 Developing Concerns. 5.12 Report Generation. 5.12.1 Report Format. 5.12.2 Distribution. 5.13 Assessment Out-brief. 6. GUIDELINES FOR ASSESSING OPERATIONS 6.1 Assessing Operations Organization and Administration (OPS O&A). 6.1.1 Discussion. 6.1.2 Preparing to Assess OPS O&A. 6.1.3 Observation. 6.1.4 Interview. 6.1.5 Document Review. 6.2 Assessing Shift Routines and Operating Practices. 6.2.1 Discussion. 6.2.1.1 Approach. 6.2.2 Preparing to Assess Shift Routines and Operating Practices. 6.2.3 Observation. 6.2.4 Interview. 6.2.5 Document Review. 6.3 Assessing Control Area Activities. 6.3.1 Discussion. 6.3.1.1 Approach. 6.3.2 Preparing to Assess Control Area Activities. 6.3.3 Observation. 6.3.4. Interview. 6.3.5 Document Review. 6.4 Assessing Communications. 6.4.1 Discussion. 6.4.1.1 Approach. 6.4.2 Preparing to Assess Communications. 6.4.3 Observation. 6.4.4 Interview. 6.4.5 Document Review.

6.5 Assessing Control of On-Shift Training. 6.5.1 Discussion. 6.5.2 Preparing to Assess On-Shift Training. 6.5.3 Observation. 6.5.4 Interview. 6.5.5 Document Review. 6.6 Assessing Investigation of Abnormal Events. 6.6.1 Discussion. 6.6.2 Preparing to Assess Investigation of Abnormal Events. 6.6.3 Observation. 6.6.4 Interview. 6.6.5 Document Review. 6.7 Assessing Notifications. 6.7.1 Discussion. 6.7.2 Preparing to Assess Notifications. 6.7.3 Observation. 6.7.4 Interview. 6.7.5 Document Review. 6.8 Assessing Control of Equipment and System Status. 6.8.1 Discussion. 6.8.1.1 Approach. 6.8.2 Preparing to Assess Control of Equipment and System Status. 6.8.3 Observation. 6.8.4 Interview. 6.8.5 Document Review. 6.9 Assessing Lockouts and Tagouts. 6.9.1 Discussion. 6.9.1.1 Approach. 6.9.2 Preparing to Assess Lockouts and Tagouts. 6.9.3 Observation. 6.9.4 Interview. 6.9.5 Document Review. 6.10 Assessing Independent Verification. 6.10.1 Discussion. 6.10.1.1 Approach. 6.10.2 Preparing to Assess Independent Verification. 6.10.3 Observation. 6.10.4 Interview. 6.10.5 Document Review. 6.11 Assessing Logkeeping. 6.11.1 Discussion. 6.11.1.1 Approach. 6.11.2 Preparing to Assess Logkeeping. 6.11.3 Observation. 6.11.4 Interview. 6.11.5 Document Review. 6.12 Assessing Operations Turnover. 6.12.1 Discussion. 6.12.1.1 Approach. 6.12.2 Preparing to Assess Operations Turnover. 6.12.3 Observation. 6.12.4 Interview. 6.12.5 Document Review. 6.13 Assessing Operations Aspects of Facility Chemistry and Unique Processes. 6.13.1 Discussion. 6.13.1.1 Approach. 6.13.2 Preparing to Assess Operations Aspects of Facility Chemistry and Unique Processes. 6.13.3 Observation. 6.13.4 Interview. 6.13.5 Document Review. 6.14 Assessing Required Reading. 6.14.1 Discussion. 6.14.2 Preparing to Assess Required Reading. 6.14.3 Observation. 6.14.4 Interview. 6.14.5 Document Review. 6.15 Assessing Timely Orders to Operators. 6.15.1 Discussion. 6.15.2 Preparing to Assess Timely Orders to Operators. 6.15.3 Observation. 6.15.4 Interview. 6.15.5 Document Review. 6.16 Assessing Operations Procedures. 6.16.1 Discussion. 6.16.2 Preparing to Assess Operations Procedures. 6.16.3 Observation. 6.16.4 Interview. 6.16.5 Document Review. 6.17 Assessing Operator Aid Postings. 6.17.1 Discussion. 6.17.2 Preparing to Assess Operator Aid Postings. 6.17.3 Observation. 6.17.4 Interview. 6.17.5 Document Review. 6.18 Assessing Equipment and Piping Labeling. 6.18.1 Discussion. 6.18.2 Preparing to Assess Equipment and Piping Labeling. 6.18.3 Observation. 6.18.4 Interview. 6.18.5 Document Review. 6.19 Assessing Operations-Related Radiological Control. 6.19.1 Discussion. 6.19.2 Preparing to Assess Operations-Related Radiological Control. 6.19.3 Observation. 6.19.4 Interview. 6.19.5 Document Review. 6.20 Assessing Operations-Related Training and Qualification. 6.20.1 Discussion. 6.20.2 Preparing to Assess Operations-Related Training and Qualification. 6.20.3 Observation. 6.20.4 Interview. 6.20.5 Document Review. 6.21 Assessing Drills. 6.21.1 Discussion. 6.21.2 Preparing to Assess Drills. 6.21.3 Observation. 6.21.4 Interview. 6.21.5 Document Review. 7. APPENDIXES APPENDIX A Sample Operations Assessment Program Schedule APPENDIX B Operations Assessment Flow Diagram APPENDIX C Sample Operations Assessment Preparation Checklist APPENDIX D Sample Graded-Approach Matrix APPENDIX E Sample Operations Assessment Plan APPENDIX F Sample Schedule for Operations Assessment APPENDIX G Sample Operations Assessment Report 8. INDEX CONCLUDING MATERIAL