DOE-ER-STD-6001-92, Implementation Guide for Quality Assurance Programs for Basic and Applied Research

D9E-ER-STD--6001-92 DE92 016352 DOE STANDARD IMPLEMENTATION GUIDE FOR QUALITY ASSURANCE PROGRAMS FOR BASIC AND APPLIED RESEARCH U.S. Department of Energy Washington, D.C. 20585 AREA-QCIC WELCOME This Portable Document Format (PDF) file contains bookmarks, thumbnails, and hyperlinks to help you navigate through the document. All items listed on the contents page are linked to the corresponding sections. In addition, if you click on a section heading while you are reading the text, you will return to the contents page. Click on the DOE seal below to move to the contents page. — — — — —~ This report has been reproduced directly from the best available copy. Available to the public from the National Technical Information Service, U.S. Department of Commerce, 5285 Port Royal Rd., Springfield, VA 22161. DOE-ER-STD-6001-92 ACKNOWLEDGEMENT The Office of Energy Research (ER) wishes to acknowledge and thank the following individuals for their significant contribution to the development of this document: Mark Bodnarczuk, Fermi National Accelerator Laboratory, Primary Author Marvin Shear, Brookhaven National Laboratory, Working Group Chairman Frank Hawkins, Office of Nuclear Safety Policy and Standards ABSTRACT The development of this standard has been a collaborative,effort of a Working Group consisting of representatives from several national laboratories,the ER Program Associate Directorates, the Office of Nuclear Energy, and the ER Office of Assessment and Support. It is applicable to basic and applied research performed at DOE-ER sponsored facilities. The standard is written primarily for scientists and technical managers and purposely uses scientific and technical examples and terminology in an attempt to translate the concepts and requirements of DOE 5700.6C into the language and practices that are familiar to the scientists and technical personnel who manage and do research at DOE-ER facilities. The standard provides the guidance needed to reasonably implement the full intent of 5700.6C in the ER research community and yet preserve the independence of the research community to creatively pursue the advancement of science. Purpose Applicability Background and Format Guidance DOE-ER-STD-6001-92 Table of Contents I. Performing DOE-ER Sponsored Research II. DOE-ER Sponsored Facility Management A. Management Criterion 1 - Program Criterion 2 - Personnel Training and Qualification Criterion 3 - Quality Improvement Criterion 4 - Documents and Records B. Performance Criterion 5 - Work Processes Criterion 6 - Design Criterion 7 - Procurement Criterion 8 - Inspection and Acceptance Testing C. Assessment Criterion 9 - Management Assessment Criterion 10 - IndependentAssessment 1 1 1 1 3 5 6 6 6 7 8 9 9 10 11 11 12 12 12 -— —.— ——..——.——.——— —.. --— — —— — —- _.. . DOE-ER-STD-6001-92 Purpose This ImplementationGuide is intended to assist management at DOE-ER sponsored facilities in the process of developing and implementingQuality Assurance Programs (QAPs) that satisfy the requirements of DOE Order 5700.6C. It should be viewed as guidance, not as a statement of required actions. The Implementation Guide also provides the guidance to be used by the Office of Energy Research to review and approve DOE and DOE contractor QAPs. Applicabilitv

This ImplementationGuide is applicable to the basic and applied research performed at DOE-ER sponsored facilities. Basic and applied research is distinguished from the type of work performed specifically as the first link (the R) of a planned R&D chain by the differences in the objectives and direct products produced. The objective and direct product of a planned R&D chain is the development of equipment, devices, or technology for practical application beyond the new knowledge obtained in basic and applied research. That type of work is beyond the scope of this document and should be based upon the guidance found in Attachment I of DOE 5700.6C. Basic and applied research is defined as work that has the objective of producing new knowledge that is usually published in professional journals and may or may not have immediate application to the first link (the R) of a planned R&D chain. Because much of the research performed at DOE-ER facilities is done in collaboration with faculty as an extension of a university, the graduate students that are trained in the process of performing experiments in the basic and applied sciences are another valuable by-product of this research. Background and Format This ImplementationGuide is written primarily for scientists and technical managers at DOE-ER facilities. It purposely uses scientific and technical examples and terminology in an attempt to translate the concepts and requirements of DOE 5700.6C into the language and practices that are familiar to the scientists and technical personnel who manage and do research at DOE-ER facilities. The guidance is not meant to impose management systems that are not normally a part of performing quality research. The format of this ImplementationGuide is designed so that guidance for implementing DOE 5700.6C is contained in the text while examples that describe the unique application of the 10 Criteria in a scientific environment are contained in footnotes. Because no guidance currently exists for implementing QAPs in basic and applied research activities, the examples are designed to help managers, scientists, and assessment personnel determine how to interpret DOE 5700.6C in DOE-ER sponsored research environments. Guidance Section I of this guidance (PerformingDOE-ER Sponsored Research) should be applied to the performance of basic and applied research at DOE-ER facilities. DOE 5700.6C section 4 e. states that the “work results which undergo peer review for publication are exempt from the scope of DOE 5700.6C.” While peer review is used in virtually all stages of basic and applied research from the evaluation of 1 DOE-ER-STD-6001-92 a research proposal and work-in-progress to the evaluation of research results for publication, the exemption statement is intended to mean that DOE 5700.6C is applicable to all phases of an experiment until data taking and operational activities end. DOE 5700.6C is not applicable to the analysis or publication phases of the experiment which consists of internal peer review performed within the collaboration and external peer review performed by referees prior to publication in a professional journal. Section II of this guidance (DOE-ER Facility Management) should be applied to the construction, operation, and management of facilities sponsored by DOE-ER.

Carrying out the research mission specified in a facility’s contract involves the assessment of the hazards to 1) public health and safety, 2) environmental protection, 3) site personnel safety, and 4) programmatic objectives and property concerns. The hazards to both ES&H and programmatic objectives should be the fundamental consideration in determining how rigorously DOE 5700.6C should be applied to the work performed by DOE-ER sponsored organizations.1 Management should use a graded approach to implementing the requirements of DOE 5700.6C. A graded approach is needed particularly where procedures and methodologies are imprecise due to the state of the art for the specific area of research (e.g., ecological research). The management systems developed should be adequate to ensure that organizations achieve their programmatic objectives given the scale, cost, complexity, and hazards of the work. When implementing DOE 5700.6C at DOE-ER sponsored facilities, the general guidelines listed below should be followed. a. Management should recognize that there are a number of mechanisms for assuring quality in basic and applied research, with two crucial ones being 1) hiring the most qualified personnel and 2) the peer review process. A community of “technical peers” is defined as that competent professional group whose role it is to judge what quality is within a scientific discipline. The technical peers are individuals who meet all of the following criteria: 1) Have an equal or higher level of academic education in the technical discipline in which research is performed or a closely related field. 2) Have experience in proposing and solving experimental or theoretical problems that are recognized as valid by that community of peers. 3) Have contributed to the body of knowledge within a technical discipline by publishing research results in professional journals. 4) Remain active competitive participants in that field of research or remain current in research published in that technical field. 1 Progratnnatic hazards include those events that interrupt or degrade the performance of research due to a failure of a support system or piece of experimental apparatus and consequently adversely affects the outcome of the research. 2 DOE-ER-STD-6001-92 A peer review is an evaluation of the quality and validity of technical work that is conducted by technical peers who did not directly participate in the work being evaluated. Peer reviews may be used at all points along the basic and applied research continuum from the evaluation of a research proposal and work-in-progress,to the evaluation of research results for publication in a professional journal. b. The most essential resource at DOE-ER sponsored facilities is the creativity of scientists and other personnel. The effective implementationof DOE Order 5700.6C should generate management systems that are adequate to ensure programmatic success, but are not so prescriptive that they eliminate human thought or spontaneous problem solving strategies through the inappropriate imposition of step-by-step written procedures. Management should recognize that creativity is important to all aspects of DOE-ER facilities management from the design, construction, and operations of systems to the actual performance of research. c. Quality assurance should be viewed as all activities used to obtain confidence in the reliable performance of the support systems and experimental apparatus used to do research and the creative pursuit of scientific goals through good research practices.

d. Management should clearly define the relationships and responsibilities between the functions of 1) managing the resources necessary to support research work, and 2) performing the research. I. Performing DOE-ER Sponsored Research The basic and applied research performed at DOE-ER sponsored facilities varies from biological and materials science, to applied, nuclear, and high-energy physics. The scale of this research ranges from bench-scale experiments with one or two researchers, to large, costly, complex experiments involving many researchers. When implementing the guidance in Section I, the extent and detail of the management systems should be commensurate with the scale, cost, complexity, and hazards of the research program or experiment.3 2 Defining these relationships and responsibilities is complex because some researchers are also laboratory staff members (they carry out both functions at different times), other researchers come from industry but are . members of research teams, yet other researchers are “outside users” from universities in the U.S. and around the world and are not employed by a DOE facility. 3 The scale of experimental apparatus varies widely from bench scale biological, chemical, or applied physics apparatus that may fit on a table-top to material science apparatus that are room-sized and may weigh less than 1 ton, to high-energy physics detectors that require large dedicated experimental halls and weigh as much as 2,000 tons. The cost of experimental apparatus can range from a few thousand dollars for bench-scale biological or applied physics studies to large high-energy physics detectors that cost over $60 million. The complexity of apparatus also varies widely from biological, chemical, and applied physics apparatus (some of which may be purchased as off-the shelf items) that may have 10-50 channels of electronics to high-energy physics detectors that are specially designed and constructed by scientists that may have over 100,000 channels of electronics. 3 DOE-ER-STD-6001-92 a. A Field Work Proposal or experimental proposal should describe the mission and objectives of the research program or experiment. To an extent commensurate with the type of research (discovery experiments, searches for new phenomena or effects, high-statisticsdetailed measurements), the research objectives should describe calculated predictions or hypotheses of the expected research results that may or may not be corroborated by the actual data accumulated by experiments. Research objectives should be based upon the current state of theoretical models, previous experimental results, and appropriate experimental techniques in that specific scientific field. b. Management should recognize that the outcome of basic and applied research may not be predictable, systematically planned, or defined by written procedures. c. The Field Work Proposal or experimental proposal should identify individuals who have primary responsibility for overseeing the research (Principal Investigator (PI), spokesperson/s,or senior scientist/s). The PI, spokesperson, or senior scientists have primary responsibility for overseeing the research though they may delegate the work to other scientists, post-dots, or graduate students.5 The management systems developed should be adequate to ensure that the research goals are achieved. d. To a level of detail that is adequate to ensure that the research goals are achieved, the PI, spokespersons, or senior scientists are responsible for:

1) Planning the research based upon the appropriate laboratory schedules, including resource requirements, methodology, scheduling, and funding; 2) Ensuring that materials and supplies are properly stored and that reasonable shelf life limitations are observed; 3) Defining policies that ensure that the records associated with the experiment are developed to an adequate level of detail, are legible, complete, correct, retrievable, and protected from loss or damage; 4 A Field Work Proposal defines a program of research where multiple experimenters perform basic studies in a particular scientific field (for example basic materials research in high-Tc conductors, energy research in chemical physics and geophysics, or life sciences research in cell and molecular biology or chemical biodynamics). In research like high-energy physics, the scale, cost, and complexity of experiments may demand that a separate experimental proposal be developed for each experiment. 3 One of the by-products of basic and applied research is graduate students who receive advanced research degrees. Because a graduate level education is designed to develop the intellectual skills needed to pursue careers of independent research in a specific discipline, responsibility for complex scientific and engineering problems is often assigned to students and post-doctoral researchers who are supervised by thesis advisors and other senior scientists in the collaboration. 4 DOE-ER-STD-6001-92 4) Determining calibration requirements of the experimental apparatus. The apparatus may be calibrated against the values of known physical phenomena and effects within that field of study to a degree that is adequate to ensure that the research goals are achieved; 5) Ensuring that the research is conducted in accordance with ES&H and other facility requirements. e. The PI, spokesperson, or senior scientists are responsible todevelop performance criteria for the evaluation of software programs. Software program performance should be validated against the values of known physical phenomena and effects within that field of study. Performance evaluations of the software programs should be at a level of detail that is adequate to ensure that the research goals are achieved. f. The PI, spokesperson, or senior scientists are responsible to define the appropriate human or machine-readable inspections and tests of the experimental apparatus during calibration or data taking. The detail, extent and methods of inspections and tests should be adequate to ensure that the experiment achieves its research goals. g. The work performed by collaboration members should be periodically evaluated by the PI, spokespersons, or senior scientists to a level of detail that will ensure that good research practices are employed to achieve the research goals.7 II. DOE-ER Sponsored Facility Management The mission of DOE-ER facilities is to provide the resources necessary for qualified researchers to conduct a wide variety of research from biological and materials science, to high-energy physics. This research should be performed in accordance with the applicable environment, safety, and health requirements. When implementing the guidance in Section II, the extent and detail of the management systems should be commensurate with the scale, cost, complexity, and hazards of the work being performed. 6 The wide variety of experimental apparatus (bench-scale to large detectors) demands the development of unique

computer software programs for data reconstruction and analysis. In many experimental collaborations, responsibility for the majority of software development is assigned to graduate students or post-doctoral researchers as part of the training/mentoring process that leads to an advanced research degree. 7 In a bench-scale collaboration of 2-3 scientists, evaluation may be performed by informal collaboration meetings and workshops. In larger collaborations involving over 300 scientists, evaluations are more formally documented meetings and workshops that may consist of presentations by collaboration members. 5 DOE-ER-STD-6001-92 A. Management Criterion 1 - Program The QAP should be a total management system that is management’s strategy for successfully carrying out the mission defined in the facility’s contract. The goal should be to effectively utilize the facility’s organizational infrastructureto provide the resources and support necessary to carry out research programs. The basic precepts are: a. b. c. d. e. DOE Orders prescribe a wide variety of management systems to help contractors achieve their mission goals. There is no single DOE Order that has been written specifically to integrate these requirements and other laboratory policies into a total management system. The 10 criteria of DOE 5700.6C can be used as functional categories that interrelate DOE Orders and laboratory policies into a total maniagementsystem. When management at DOE-ER facilities believes the management systems required in other DOE Orders are adequate to fulfillthe intent of one or more of the 10 criteria, they should not interpret DOE Order 5700.6C as requiring the development of duplicate or redundant management systems. Management is responsible for developing and implementing a written QAP which is binding on all personnel and describes organizational structure, functional responsibilities,levels of authority, and organizational interfaces. Achieving quality should be a line responsibility,with personnel being responsible for achieving their assigned performance objectives. Management should clearly define how the requirements described in the applicable QAP translate into the day-to-day work performed by the personnel they supervise. Management should define and adopt a site-specific QA terminology that is representative of the disciplines in their organization. Management should be trained in this terminology to ensure consistent understanding and communication. Management should delegate authority to stop unsafe work or work of inadequate quality. Criterion 2 - Personnel Training and Qualification Management should help to develop the expertise needed for personnel to achieve the mission of DOE-ER sponsored facilities. The type of training should reflect the fact that basic and applied research involves the collaborative effort of personnel who have widely divergent levels of education, skills, and experience (for example, operators, designers, engineers, welders, scientists, technicians, and draftspersons). 6 DOE-ER-STD-6001-92 a. The education that is required for obtaining a university/college degree (or other professional certification) should constitute qualification.for working within the discipline in which the degree was granted. Equivalent work experience and technical activity in a related discipline may also constitute acceptable qualification. Because training by mentoring is crucial to the continued intellectualdevelopment of personnel, management should utilize technically competent mentors to model the problem-solving strategies needed both to achieve the laboratory’smission and enhance the intellectualdevelopment of personnel.

b. For work that does not require an accredited university/college degree or other professional certification,management should develop training that is appropriate to the complexity and hazards involved in the work and utilize technically competent mentors when appropriate. If the complexity of the work or the hazards involved make more formal training programs appropriate, they should be developed to achieve and maintain proficiency. c. All personnel with management responsibilities should receive training in managerial, communication, and interpersonalskills that is appropriately tailored to the organization that they supervise. In disciplines where this training is not included in the manager’s university or college curricula (such as the scientific and engineering disciplines), senior management should require such training as part of the functional responsibilitiesof those managers. d. Laboratory management should provide training in the areas of ES&H for facility personnel and outside users who perform research at the facility. The detail and extent of the training should be commensurate with the hazards associated with the work being performed. Criterion 3 - Quality Improvement a. Quality problems are often inherent in existing management systems and workers have little or no control over eliminating these problems or improving performance. Management should empower personnel to eliminate these ineffectivemanagement systems and improve performance by driving decision making authority to the lowest effective organizational level where the maximum expertise is localized. 8 A large fraction of the personnel in DOE research environments have earned graduate level degrees in science and engineering. A graduate level education develops the intellectual skills needed to pursue careersof independent research in a specific discipline by assigning complex scientific and engineering problems to students with sophisticated problcem-solving strategies being model led by mentors and academic advisors. This type of training is not recipe-like or procedural. 7 b. c. d. e. DOE-ER-STD-6001-92 When appropriate, management should be encouraged to use statistical methods (or other management tools) to help make the organizational decisions necessary to improve quality.9 Management should foster a “no-fault” attitude where all personnel are encouraged to identify and report performance problems to the appropriate level of management and management should take appropriate corrective action. Management should implement systems for documenting failures and nonconformances and for identifying, analyzing, resolving, and following up on recurring programmatic and technical problems. The extent of cause analysis and corrective action should be commensurate with the significance of the problem. Management should utilize a “lessons learned” system to improve performance when appropriate. Laboratory management should implement strategies for improving the quality of DOE-ER sponsored research programs.10 Criterion 4 - Documents and Records a. Management should develop requirements for documenting the organization, functions, policies, decisions, procedures, and essential transactions of organizations at an appropriate level of detail. The objective should be to maximize the usefulness of DOE and contractor records, and minimize the cost of document and records management and the paperwork and record keeping burden within DOE and its contractors.

b. Management should determine which work is of sufficientcomplexity or hazard to require the preparation of controlled documents. When a document is defined as a “controlled document,” procedures that describe the preparation,review, approval, issuance,and revision of the document should be developed. c. Management should implement a records management system to ensure that appropriate records are retained and retrievable. 9 For example, statistical process control (SPC), Pareto analysis, or other appropriate methods. 10 This is comnonly done through the mechanism of scientific and technical advisory committees that provide technical guidance to laboratory management. Other peer review mechanisms can be used by management to improve the performance of research in progress when appropriate. 8 DOE-ER-STD-6001-92 B. Performance Criterion 5 - Work Processes a. Human Resource Management 1) 2) 3) 4) Management should strive for effective human resourcemanagement with the goals of hiring and maintaining an efficient and effective work force and appropriately utilizing personnel skills in the assignment of work responsibilities. The individualworker is the first line in ensuring quality, but management is primarilyresponsiblefor ensuring that people who are assigned to tasks have the appropriate academic qualification, professional certification, or skills and experience to carryout the work successfully. Management is responsible for planning, authorizing, and specifying (to an appropriate level of detail) the conditions under which work is to be performed. This should include the calibration of measuring and test equipment. Management should specify which work is sufficiently complex or involves sufficient hazard to be performed to written procedures. When written procedures are deemed appropriate by management, they should be prepared, revised, approved, and distributed by appropriately knowledgeablemanagers. Management should define the performance objectives for which personnel will be held accountable. Criteria which define acceptable work performance and achievement of performance objectives should be defined for personnel with the goal of acknowledging when work has been performed acceptably and identifying areas for improvement. b. Material Resource Management The laboratory contract defines a variety of management systems to be applied to material resources through the applicable DOE Orders and Code of Federal Regulations (CFR’s). Management should not interpret this portion of Criterion 5 as requiring the development of redundant management systems that are already imposed by these requirements. 1) Management should implement an effective itern resource management system that identifies and controls items in common use stores and warehouse storage to prevent damage, loss, or deterioration. 2) Management should implement an effective management system that ensures that items are properly handled, shipped, and received. 9 DOE-ER-STD-6001-92 Criterion 6 - Design a. Sound engineering/scientific principles and appropriate technical standards should be incorporated into designs to ensure that they will perform as intended. b. Management should define ES&H related design input and design review requirements for apparatus including those designed by “outside users” to ensure compliance with facility ES&H requirements.

c. Management should selectively apply the guidance below to a level of detail that is commensurate with the scale, cost, complexity, and hazards and phase of a design (conceptualto final). Design controls should be defined to ensure that: 1) 2) 3) 4) 5) 6) Design input is correctly translated into specifications and drawings. This should include items such as fire protection requirements, design bases, and reliability requirements. Final designs, field changes, and modifications should be approved by the original design organization or a technically competent designee. Design interfacesand correspondingresponsibilitiesare defined so that design efforts are effectively coordinated among the participating organizations. Design records are incorporated into the records management system. Design inputs, processes, outputs, and changes are validated by qualified individuals or groups other than those who performed the original design, but who may be from the same organization. The level of detail of validation and the methods used should be appropriate to the design. Designs are validated prior to procurement, manufacture, or construction. When this is not possible, designs should be validated prior to the installation and use of the item. d. Management should define and implement procedures for the design and development of computer software to a level of detail that is appropriate to the complexity, cost, and hazards associatedwith the software. 10 — —.. —. —— .. . -- — ————.—.— . .. . DOE-ER-STD-6001-92 Criterion 7 - Procurement The facility contract specifies a variety of management controls to be applied to procurementsand sub-contractsthrough the applicableDOE Orders, Department of Energy Acquisition Regulations (DEAR’s) and Federal Acquisition Regulations (FAR’s).Management shouldn't interpretCriterion 7(Procurement)as requiring the development of redundant management systems that are already imposed by these requirements. a. b. c. d. e. Criterion a. b. Management should implementa procurementand subcontractsmanagement system that complieswith the appropriateprocurementand subcontract procedures as required by the facility contract. Management should require that personnel include the applicable specifications (ES&H and technical) in procurement and subcontract documents. Management should develop qualified suppliers early in design or procurement processes when possible. Management should ensure that specifications and expectations are properly communicated to prospective suppliers and that qualification is based on the appropriate demonstration that they can supply acceptable items and services on schedule. Management should evaluate prospective suppliers to ensure that qualified and responsiblesuppliersare selected.Suppliers should be appropriatelymonitored to ensure that acceptable items and services continue to be supplied. Management should develop requirements for inspection of incoming items. 8 - Inspection and Acceptance Testing Management should define the types of work that require formal inspections and acceptance testing (for example, fabrication, assembly, installation,construction, ES&H, or procurement).When an inspection or acceptance test is performed, the characteristics and processes to be inspected or tested, the inspection techniques to be used, the hold points, and the acceptance criteria should be defined as appropriate. Properlycalibrated and maintained measuring and test equipment should be used for acceptance testing.

Laboratory management should develop requirements for readiness reviews of facility and experimental systems prior to beginning work. The extent and detail of the review should be commensurate with the systems. scale, cost, complexity, and hazards involved in these 11 In larger projects where some perform readiness reviews prior to subsystems are operational while others are being phased in, management should operation of the subsystems that are being completed. 11 DOE-ER-STD-6001-92 c. Assessment Criterion a. b. c. Criterion a. b. c. 9 - Management Assessment Management at all levels should periodically evaluate the effectiveness of the QAP and other management systems. A management assessment should be an introspectiveanalysis that evaluateswhether or not the management infrastructureof the laboratory, institution, or organizational unit is properly focused on achieving its mission objectives,with appropriategoals (includingcost-effectivenessand ES&H) being defined for improving performance. Management at all levels should periodicallyevaluate the performance of their organizations, focusing on how effectively human and material resources are being utilized with respect to the mission goals. An effective management assessment should evaluate 1) the state of worker knowledge, motivation, and morale, 2) the atmosphere of creativity and improvement,3) the level of mutual confidence and collaboration among workers, 4) the adequacy of human and material resources, and 5) the management of ES&H in the performance of work processes. The results of the evaluations should be reported to senior management. Laboratorymanagement should periodically evaluate the on-going work performed as part ofa Field Work Proposal or experiment. They should also evaluate the scientific and technical progress at the conclusion of a research program or experiment. 10 - IndependentAssessment Independentassessment personnel should act as a management advisory function. They should assess how effectively the QAP and other management systems are being implemented in the day-to-day work of personnel. An effective independent assessment should not be limited to the study of documents, but should evaluate the performance of work and actions that cannot be reflected solely by documents. All assessments of the implementationof the QAP should be based on the QAP that has been approved by the Director, Office of Energy Research. Independent assessment personnel that are external to laboratory management should not reinterpret the requirements agreed to in the approved QAP. If laboratory management believes that the implementationof the assessment results will require changes to the approved QAP (other than corrections to punctuation, spelling, or other editorial items), these changes should be approved in writing by the Director, Office of Energy Research prior to implementationby the contractor as required by DOE 5700.6C [9, a, (4)]. Assessment teams should view the organization being assessed as the “customer” of the assessment results and strive to produce high- quality, organizationally meaningful feedback about the achievement of the laboratory mission. 12 _—— —.——— . . DOE-ER-STD-6001-92 d. Assessment teams should include peers who are technically competent to review the work being assessed but who have not participated in that work.12 e. The results of independent assessments should be resolved by the management who have line responsibility for the assessed area in a timely fashion. The actions involved in the resolution of assessment results should be documented and tracked. 12 The use of Visiting Consnittees, Program Advisory Conwnittees, High Energy Physics Advisory Panel, Nuclear Science Advisory Comittee, and reviews by the cognizant DOE Program Managers are examples of independent assessments. 13 DOE-ER-STD-6001-92 Review Activities: CE DP FE NE LLNL MMES BNL SNL-AL FNAL AR BAO LANL SAIC CEBAF SLAC SSRL LBL ID OR RL ANL-E SSCL SERI AMES PNL PPPL AQC Preparing Activity: DOE-ER-8 Project No. QCIC-0001 Table of Contents