DOE-HDBK-1200-97, Guide to Good Practices for Developing Learning Objectives

NOT MEASUREMENT SENSITIVE DOE-HDBK-1200-97 January 1997 Supersedes DOE-STD-1005-92 July 1992 DOE HANDBOOK GUIDE TO GOOD PRACTICES FOR DEVELOPING LEARNING OBJECTIVES U.S. Department of Energy FSC 6910 Washington, D.C. 20585 DISTRIBUTION STATEMENT A. Approved for public release; distribution is unlimited. WELCOME This Portable Document Format (PDF) file contains bookmarks, thumbnails, and hyperlinks to help you navigate through the document. All items listed in the Contents are linked to the corresponding sections. In addition, if you click on a section heading while you are reading the text, you will return to the Contents. Click on the DOE seal below to move to the Contents page. This document has been reproduced directly from the best available copy. Available to DOE and DOE contractors from the Office of Scientific and Technical Information, P.O. Box 62, Oak Ridge, TN 37831; (423) 576-8401. Available to the public from the U.S. Department of Commerce, Technology Administration, National Technical Information Service, Springfield, VA 22161; (703) 487-4650. Order No. DE97000781 DOE-HDBK-1200-97 iii FOREWORD 1. This Department of Energy (DOE) Handbook is approved for use by all DOE Components and their contractors. The Handbook incorporates editorial changes to DOE-STD-1005, Guide to Good Practices for Developing Learning Objectives, and supersedes DOE-STD-1005-92. Technical content of this Handbook has not changed from the original technical standard. Changes are primarily editorial improvements, redesignation of the standard to a Handbook, and format changes to conform with current Technical Standards Program procedures. 2. This technical standard provides guidance to DOE staff and contractors that can be used by training staffs (e.g., instructors, designers, developers and managers) and others for the development of learning objectives in support of facility- and activity-specific training materials and programs. Users are not obligated to adopt all parts of this Handbook; rather, they can selectively use the information to establish or improve facility training programs as applicable. 3. Beneficial comments (recommendations, additions, deletions) and any pertinent data that may improve this document should be sent to the Office of Nuclear Safety Policy and Standards (EH-31), U.S. Department of Energy, Washington, DC 20585, by letter or by using the self- addressed Document Improvement Proposal (DOE F 1300.3) appearing at the end of this document. 4. DOE technical standards, such as this Handbook, do not establish requirements. However, all or part of the provisions in a technical standard can become requirements under the following circumstances: (1) they are explicitly stated to be requirements in a DOE requirements document; or (2) the organization makes a commitment to meet a technical standard in a contract or in a plan or program required by a DOE requirements document. DOE-HDBK-1200-97 iv INTENTIONALLY BLANK DOE-HDBK-1200-97 v TABLE OF CONTENTS FOREWORD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii 1. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2. PARTS OF LEARNING OBJECTIVES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.1 Condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 Action . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.3 Standard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3. TYPES OF LEARNING OBJECTIVES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.1 Cognitive Learning Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 3.2 Psychomotor Learning Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.3 Affective Learning Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4. DEVELOPMENT OF LEARNING OBJECTIVES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 4.1 Sources for Determining Learning Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 4.2 Organization of Learning Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 5. EVALUATION OF LEARNING OBJECTIVES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 APPENDIX Example condition, action, and standard statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 FIGURE FIGURE 1. Example relationships between terminal and enabling objectives . . . . . . . . . . . . . . . 28 TABLE TABLE 1. Terminal and enabling objectives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 DOE-HDBK-1200-97 vi INTENTIONALLY BLANK DOE-HDBK-1200-97 1 1. INTRODUCTION 1.1 Purpose This guide to good practices provides information and guidance on the types of and development of learning objectives in a systematic approach to training (SAT) program. Contractors are encouraged to consider this guidance as a reference when developing new learning objectives or refining existing ones. Training managers, designers, developers, and instructors are the intended audiences. 1.2 Definition A learning objective is defined as "a statement that specifies measurable behavior that a trainee should exhibit after instruction." Properly prepared learning objectives consist of the following parts: & A statement of behavior (action) the trainee must exhibit & The conditions under which the action will take place & The standards of satisfactory performance 1.3 Discussion The development of effective training materials is dependent on the development of learning objectives which adhere to a strict set of criteria. Learning objectives are developed from analysis information obtained during the design phase of the systematic approach to training (SAT) process. It is important that objectives are developed and approved early since they form the foundation for the development of test items and all other training material. Because objectives serve as the design basis of performance-based training programs, they should clearly describe the trainee's desired performance to preclude misinterpretation. Some of the benefits of using learning objectives are:

& Standards of performance are presented in a trainee-accessible way & Criteria for evaluation are defined DOE-HDBK-1200-97 2 & Learning requirements are clarified and explicit & Content, methods, media, and resources are derived and related to objectives & Focus is provided for the instructor and the trainee DOE-HDBK-1200-97 3 2. PARTS OF LEARNING OBJECTIVES All effective learning objectives have certain characteristics. The developer should always take these characteristics into consideration when constructing objectives. & Attainable: Is the objective possible to achieve by the average trainee? & Specific: Is the wording concise? Has unnecessary and confusing verbiage been removed? & Clear: Will everyone interpret the objective in the same way? & Measurable: Can this behavior be measured? How? With what kind of gauge? Effective learning objectives can be stated in a variety of formats. The most common format combines condition, action, and standard statements. This combination explicitly defines the condition under which the performance occurs, what knowledge or skill is exhibited, and the standards of acceptable performance. 2.1 Condition The condition statement establishes the circumstances under which the trainee must be able to perform the specified action. Conditions are derived from information collected during analysis of the task. They also include equipment, tools, and references necessary to perform the task. Condition statements can generally be considered as either aiding or limiting. An aiding condition advises the trainee of things that will be available to assist in performance of the specified action. For example: Given a drawing, label designated areas within the crane operating area... A limiting condition advises the trainee of limitations which are likely to make the action more difficult to perform. For example: DOE-HDBK-1200-97 4 Without the use of a legend, interpret and record symbols on a piping and instrumentation diagram... In either case, the condition statement needs to be relevant to the performance of the action and should include only that which will be provided or denied the trainee during performance evaluation. Condition statements are often implied rather than specified. Using implied condition statements has an advantage of avoiding redundancy when the conditions are identical for a set of learning objectives. The disadvantages of implied conditions stem from a tendency to produce imprecise learning objectives and inefficient instructional designs. For example, an implied condition might be inferred to mean "under normal operating conditions" when the performance would actually be required "under all conditions." The implied condition could conceal from training developers, instructors, and trainees the need to perform at this higher level. If the developer of the learning objective is not certain that the implied condition is clear to everyone who may have the responsibility for learning or teaching that objective, it is best to err on the side of conservatism and include the condition statement. Stated conditions enhance precision and can provide the situational cues associated with the appropriate response. Stated conditions can alert trainees to specific job conditions and prompt the proper response in the presence of the applicable conditions. For example, an implied condition to the learning objective action statement, "perform cardiopulmonary

resuscitation," provides little information on when this action should occur. This action statement becomes more meaningful when the condition, "upon discovering an unconscious person who has no pulse," is stated. Generally, condition statements that call for a response under any situation signal higher-level objectives. Condition statements benefit the instructor and the evaluator as well as the trainee. The conditions will set up the evaluation and also aid the instructor in setting up for instruction. The following are examples of template-type condition statements: & Upon discovering an unconscious person who has no pulse & Using an approved maintenance work request & Upon observing suspected aberrant behavior DOE-HDBK-1200-97 5 & In the presence of an entry condition for an emergency operating procedure and without prompting & Given alternative combinations of alarms and annunciators These condition statements can be combined with template action and standard statements to form complete objectives. Additional template condition statements are provided in the Appendix. Instructional design is affected by the conditions stated in the learning objective. When teaching cognitive, psychomotor, or affective content, the instructional activities should approximate the actual conditions of performance. 2.2 Action The action statement is the nucleus of the objective and should therefore describe exactly the performance required of the trainee on the job. It should be precise, observable, and should be based on the task, element, skill, or knowledge statements resulting from the analysis of the task. The action statement consists of an implied subject (you), a verb, and the verb's object. The following action statements illustrate this format: & Perform cardiopulmonary resuscitation & Verify that the motor-operated valve torque limit switches are set & Evaluate and document the situation & Prescribe and implement the correct immediate action & Diagnose and respond to facility conditions The action statement should ensure that the performance described matches the task. In some cases (as with terminal objectives) it is a restatement of the task itself. In other cases, the action statement may be derived from the element, skill, and/or knowledge statements identified as part of a task analysis. The knowledge items can be translated into cognitive actions that can be measured in the classroom. For example, although the action "Know the xenon concentration of ..." cannot be measured, "State the xenon concentration ..." can be measured. Element and skill items DOE-HDBK-1200-97 6 such as "disassemble the mechanical seal ..." can be translated into performance actions which can be measured in the laboratory, at the simulator, or on the job. An effective way to determine the appropriate action verb is to use a verb list such as the one in DOE Training Program Handbook: A Systematic Approach to Training. This format lists the verb and gives a definition for each verb to aid in the selection. Another way to develop action statements is to use the template approach discussed in Section 4.1. Template-type action statements are listed in the Appendix. The action verb is generally viewed as an indicator of the cognitive, psychomotor, or affective level of the learning objective. In practice, the level of an objective is influenced by the context in which it is used, and is often expressed by the condition, verb, and

standard. For this reason, it is possible for the same action verb to be used to represent more than one level. In a work environment where procedures are used often, it will be rare for tasks to involve cognitive processes above the application level. However, exceptions will be encountered and developers should be alert to this possibility. The higher cognitive and psychomotor levels are most likely to be appropriate for operators responding to abnormal or emergency conditions or to maintenance personnel conducting troubleshooting on complex equipment. Terminal objectives normally reflect a job-related task which is fairly broad in scope. Because job-related tasks are performance-oriented, keep the terminal objective at a performance level. The developer should always keep the terminal objectives at a level that best approximates the job task when the action verb in the task can not be the same as the action verb in the terminal objective due to training limitations. Thus, it is not normally appropriate to write terminal objectives at the knowledge level. Enabling objectives written at the knowledge-level are acceptable if they contribute to the trainee's achievement of the terminal objective. As a hierarchy of enabling objectives is being developed, attention must be given to designing a clear and continuous path from the trainee's entry level to the terminal objective. DOE-HDBK-1200-97 7 2.3 Standard The standard describes what criteria must be met for the performance specified by the action statement to be considered acceptable. The standard for the objective should be derived from the standard required by the task. In some cases, the template approach (as listed in the Appendix) can be a tool for the construction of a standard statement. The developer of learning objectives must always remember to strive for standard statements that closely approximate actual performance criteria. The standard will refer to the quality of the end product or the precision of the process and is usually expressed in terms of time limits, accuracy, quality, or quantity. Time limits should be used as standards only when time is important in the performance of the specified action. They should not be used merely to meet the requirements of including a standard as part of every objective. If time is not important, look for a more meaningful standard. Accuracy and quality are very similar in meaning and are often used interchangeably. Either term might refer to such things as the closeness of a measured value to a true value or the closeness of a machined dimension to a specification. Quantity typically refers to production of a certain number of units or achievement of a certain number of tasks in a given time period. The following standards statements illustrate these criteria: & ...until a pulse is restored & ...at the specified thrust needed to open and close against maximum differential pressure, temperature, and flow & ...in accordance with section 4.1 of administrative procedure 06 & ...within 30 seconds & ...accurately and before any automatic safeguard features actuate Standards should not be implied in terminal objectives. The disadvantage of implied standards stems from invalid assumptions of clarity. Confusion among instructors, trainees, DOE-HDBK-1200-97 8 and evaluators can inappropriately inflate or degrade performance expectations and reduce the reliability of trainee evaluations.

Standards are often implied in enabling objectives when they are the same as those expressed in the terminal objective to which the enabling objective is subordinate. If not included in the terminal objective, the standard should be stated in the enabling objective to preclude misunderstanding. Combining the standard with the example condition and action statements form the following objectives: & Upon discovering an unconscious person who has no pulse, perform cardiopulmonary resuscitation until a pulse is restored. & Using an approved maintenance work request, verify that the motor-operated valve torque limit switches are set at the specified thrust needed to open and close against maximum differential pressure, temperature, and flow. & Upon observing suspected aberrant behavior, evaluate and document the situation in accordance with Section 4.1 of administrative procedure 06. & In the presence of an entry condition for an emergency operating procedure and without prompting, take the correct immediate action within 30 seconds. & Given alternative combinations of alarms and annunciators, diagnose and respond to facility conditions accurately and before any automatic safeguard features actuate. The following strategies can help ensure the development of good learning objectives: & Training developers and subject matter experts should work together to produce learning objectives. & The use of implied conditions and standards should be minimized. Implied conditions and standards can reduce the capability of the learning objective to define trainee behaviors precisely. & The available training settings and methods should be considered as the learning objectives are written. DOE-HDBK-1200-97 9 & Draft learning objectives should be reviewed by facility and training personnel. This strategy confirms technical accuracy and verifies that the training is being developed on a sound foundation. & Draft learning objectives may elicit constructive criticism or questions. This is expected and indicates an aggressive review process. Revising learning objectives at this stage is more cost effective than waiting until later to revise the completed training program. & After learning objectives are approved by management, they are used to guide decisions relative to the selection and development of training materials. Material and training activities should be chosen based on how readily they enable the trainees to meet the learning objectives. DOE-HDBK-1200-97 10 INTENTIONALLY BLANK DOE-HDBK-1200-97 11 3. TYPES OF LEARNING OBJECTIVES Cognitive, psychomotor, and affective learning objectives identify what the trainee will know, be able to perform, and value, respectively, following training. Even though aspects of job performance have been classified into three separate domains, in practice, most tasks include aspects of all three. Each course may have cognitive as well as psychomotor learning objectives to achieve the required job competence. 3.1 Cognitive Learning Objectives Cognitive objectives define what knowledge the trainee must have to perform the job properly. The use of facts, principles, rules, and theory to perform tasks under normal or uncertain conditions can be addressed by the use of cognitive objectives. Training needs analysis, job analysis, and task analysis are sources for determining cognitive learning objectives. The developer should ensure that learning objectives are created to lead the trainee from

some entry level ability to the desired performance. This will require objectives to be written at a variety of levels to reflect simple as well as complex knowledge that is required to perform a task well. A taxonomy comprised of six cognitive levels is listed below. These levels are ordered from the simple (knowledge) to the more complex (evaluation). This taxonomy is only one construct that can be applied to the classification of cognitive levels. & Knowledge & Comprehension & Application & Analysis & Synthesis & Evaluation DOE-HDBK-1200-97 12 The following is a brief description of the six levels: Knowledge This is the first and most basic level. It is defined as the ability to recall key facts. A trainee can be performing at this level when action verbs like define, memorize, recall, label, identify, list, name, or state are used in the learning objective. Example: Define the term "Learning Objective." Comprehension This is the second cognitive level and is evident when the trainee is able to grasp the meaning of the material. Obtaining information from charts, graphs, indicators, and procedures illustrates performance at this level. Identify, locate, discuss, obtain, describe, explain, or record are action verbs used to measure the comprehension level. Example: Given a description of equipment to be aligned and the display from alignment equipment, explain the indications on the display to determine the necessary adjustments. The explanation must lead to the correct adjustment needed to achieve alignment in accordance with the Optalign Operating Manual. Application This is the third cognitive level and identifies the capability to use information learned at the designated knowledge and comprehension levels to solve routine problems. The following types of action verbs can be used at this level: apply, interpret, use, demonstrate, illustrate, indicate, operate, or translate. Example: Given a Goulds 3196ST stuffing box dimension and the blueprint for a John Crane #1 mechanical seal, demonstrate how to find the setting for the seal backing ring placement. The setting must be correct to within 1/32 in. DOE-HDBK-1200-97 13 Analysis This level is also known as diagnosis and is the next level of cognitive objectives. Objectives at this level define those capabilities needed to separate information into its component parts to reveal their organization, relationships, and potential faults. Action verbs used at this level include: analyze, break down, calculate, detect, diagnose, examine, inspect, solve, compare, troubleshoot, or verify. Example: Given the reactor operating conditions, analyze the changes in indicated reactor power caused by raising control rods. The analysis must consider changes resulting from both temperature and xenon concentration changes as described in the reactor physics manual. Synthesis Synthesis refers to the capability to assemble parts into a whole to create new solutions, methods, or procedures. Here, information mastered in prior levels is combined to produce an original result. Action verbs used at the development level include: assemble, construct, create, derive, determine, develop, generate, invent, prescribe, mitigate, plan, or write. Example: Given an operational scenario for an event, prescribe specific components from "Formality of Operations" for which adherence could have prevented problems associated with the event. The components

prescribed will be in agreement with the specifications of RDSI II.8, "Procedures," and RDSI IV.8 "Communications." Evaluation Evaluation is the highest level of cognitive activity. It defines those mental capabilities needed to judge the value of material or information and to maintain effective control in a dynamic, uncertain environment and emergency situations. The following action verbs can signal objectives at this level: assess, avert, compare, conclude, choose, defend, evaluate, judge, justify, lead, or predict. DOE-HDBK-1200-97 14 Example: Given an Unusual Occurrence Report, evaluate its completeness and acceptability in accordance with the subject guidelines. The action verbs listed under each level of cognitive learning are not exhaustive nor are they exclusive of each other. In other words, a verb used for the knowledge level may also be appropriate for the comprehension level depending on the context of the objective. Instructional designers should carefully consider their choice of action verbs due to the importance of the verb to the learning objective. 3.2 Psychomotor Learning Objectives Psychomotor objectives define the physical actions exhibited when performing job tasks and are typically used in on-the-job, laboratory, and simulator training. Job analysis, task analysis, plant references, and procedures are sources for determining psychomotor learning objectives. Psychomotor learning objectives address the following levels of performance: & Observation & Simulate & Performance These levels of performance can be used to identify objectives that, when met by trainees, enable them to achieve competency. Psychomotor objectives are generally sequenced to correspond with procedural/task performance requirements. They are taught in the same sequence that they are performed. Observation Observation of the task to be learned is the first level of psychomotor performance. At this level, trainees position themselves to watch a subject matter expert demonstrate the task or behavior to be learned. During observation, it is helpful for trainees to share the same point of view as the subject matter expert. The following example verbs are appropriate: observe, record, or question. DOE-HDBK-1200-97 15 Example: Given a coolant rupture simulation, record the immediate action steps taken by a Reactor Operator using the simulator. Simulate Simulating the performance of a task enables the trainee to try the behavior in a controlled environment that permits error, provides feedback, and refines performance. Example verbs at this level are simulate, show, or practice. Example: Given the Loss of Flow Simulator and the prescribed action conditions, simulate the Reactor Operator actions required for a coolant rupture in accordance with the procedure. Performance Performance of the task is the third level of psychomotor learning objectives. At this stage, trainees have developed competency through observation, practice, and/or simulation and are capable of demonstrating the learned behavior. The performance of an objective or learned task should occur in the actual setting in which the task normally takes place or at a location as similar as possible to the actual setting. Example verbs include assemble, operate, or repair. Example: Given all necessary tools, materials, and the procedure, disassemble an Air Joy WG09 air compressor in accordance with the procedure.

3.3 Affective Learning Objectives Affective objectives define the attitudes and values associated with reliable job performance. They identify the non-technical aspects of effective performance by communicating company policy and culture. Company policy, ethics, and professional standards are sources of affective learning objectives. The use of affective learning objectives can reinforce the culture of a facility by making company values visible. In so doing, affective learning objectives contribute to DOE-HDBK-1200-97 16 enhanced teamwork and reinforce safe, reliable job performance. For example, new employee orientation, teamwork, and supervisory and management development programs generally include affective dimensions. Affective objectives can be influenced by: & Role models: When the actions of peers, supervisors, and managers support stated corporate values, employees tend to emulate and internalize these values. & The individual’s attitudes and values; those that confirm the value tend to strengthen and sustain it: Experiences that show the value to be ineffective can erode and, eventually, extinguish the value. & The absence of unity between stated corporate values, leader behavior, and operating practices: This absence will undermine desired values and replace them with ones compatible with actual practice. Developers of affective objectives should consider appraising these items before attempting to write affective objectives since achievement of them can be influenced more by forces external to the training environment than achievement of cognitive or psychomotor objectives. Instruction in the affective domain is the most difficult of the three domains and is the most difficult to measure. Teaching in the affective domain can be by example (i.e., if the instructor follows the proper safety precautions when demonstrating a task, the trainee will most likely follow the proper precautions). A method of achieving affective goals (such as safety) in a program is to place the desired values in the standards of the performance objectives. The course designers should incorporate the affective domain with all learning objectives/lessons wherever possible to continually emphasize their importance to effective and reliable job performance. Example: Given all necessary tools, materials, and the procedure, disassemble an Air Joy WG09 air compressor in accordance with the procedure and all relevant site safety precautions. One way of classifying affective objectives is to group them into four levels based on strength of commitment. The categories progress from simple awareness to overt DOE-HDBK-1200-97 17 intervention. They represent one way of classifying objectives and, with the exception of "awareness," can be linked to observable behavior. These four categories are: & Awareness & Reinforcement & Promotion & Defense Awareness Awareness precedes behavior. Prior to adopting attitudes and values, an individual must become aware of them. This occurs through normal experience, training, and exposure to the job and corporate culture. Because of the difficulty of measuring awareness, learning objectives at this level are seldom appropriate. However, an effective movie or videotape can introduce attitudes or values. Through dramatization, these media can elicit a response that deepens awareness. Example action verbs at the awareness level include appreciate, recognize, and sense.

Objective: Recognize the bases for radiation protection practices used at this facility. Assessment: 10 CFR 835 specifies the ALARA requirements for this facility. (True/False) Reinforcement Reinforcement of an attitude is the next affective level. At this stage, the behavior of the individual is compatible with the value. This does not mean that the employee necessarily agrees with the value, but only that his/her behavior conforms to it. A more desirable state exists when the individual both agrees with the value and behaves accordingly. Example action verbs at the reinforcement level include, adhere, exhibit, or follow. DOE-HDBK-1200-97 18 Objective: Follows safe-work practices when setting up rigging equipment. Assessment: Trainee would be observed during a performance test for following all safe work practices in setting up rigging equipment. One of the criteria on the performance test would be: "Checked line strength using ratings chart for load to be lifted." Promotion Promotion of a value is evident when the behavior of an individual conforms to the value and actively encourages others to adopt it. The individual supports the value, respects its importance, and leads others to adopt it through example and support. Objectives related to safety margins, teamwork, and professionalism are generally at this level. Example verbs at the promotion level include advocate, encourage, or model. Objective: Advocates effective communication practices in working with his/her crew members. Assessment: Trainee would be placed in a role play with others that periodically use poor communication practices. One of the criteria on the performance test would be: "Requires feedback from crew member to verify communication message." Defense Defense of a value is the highest affective level. Individuals at this level not only adhere to and promote the value but also support those who believe in the value and respond when someone attacks it. The response can range from a verbal reply to physical intervention. For example, learning objectives at this level are appropriate for personnel responsible for fitness-for-duty, facility safety, public relations, and facility security. Action verbs that may indicate a defense-level objective include arrest, defend, report, intervene, or prevent. DOE-HDBK-1200-97 19 Objective: Prevents security violations in accordance with company policies and procedures. Assessment: Trainee would be placed in various role play scenarios in which security violations were exhibited. Trainee would need to respond to each security violation using company policies and procedures as the basis for his/her actions. DOE-HDBK-1200-97 20 INTENTIONALLY BLANK DOE-HDBK-1200-97 21 4. DEVELOPMENT OF LEARNING OBJECTIVES In the analysis phase of the training program development process, tasks are selected for training from the facility task list. In the design phase, learning objectives are created from that task list with consideration for the categories (types of objectives) discussed in the previous section and the setting required to train for effective task accomplishment (e.g., classroom, laboratory, simulator, OJT, etc.). In any setting, training should prepare the trainee for final job performance. 4.1 Sources for Determining Learning Objectives Sources for determining learning objectives are those activities and products used to

determine the knowledge and skills required for the mastery of the tasks identified and selected for training in the task list. The following are typical sources used as the basis for the development of objectives: Task Lists Task lists contain task statements that must describe an observable and measurable performance so that the mastery of the task can be evaluated. Task statements are written according to established guidelines and have action verbs that describe performance. Task statements are the primary source for the establishment of learning objectives. Task Analysis Task analysis identifies the elements, standards, conditions, knowledge, skills, and abilities required to assess and document adequate performance of a task. When a task statement is analyzed correctly, the supporting learning objectives are created from the knowledge and skills (other than entry-level requirements) specified in the task analysis. NOTE: For certain subject matter, analysis results may not serve as the basis for objectives. An example is fundamentals training, in which analysis of a task may not identify fundamental (e.g., physics, chemistry, math) knowledge unless the analysis is performed to an extremely detailed level. Objectives for fundamental training are DOE-HDBK-1200-97 22 normally determined from subject matter expert (SME) meetings, review of other fundamental training curriculums, and books on the topic for which the learning objectives are being developed. References The references, such as procedures and vendor manuals identified during task analysis, should be available to the developer when constructing learning objectives. Subject matter experts, supervisors, and managers should always be used as additional references during course design. Templates Template-type condition, action, and standard statements are tools for use in writing learning objectives. This approach is based on the premise that technical training has common learning objectives that apply across many areas. Previously developed templates from other facilities or courses may also be used to identify the learning objectives for fundamental knowledge training. For example, individuals who operate or maintain facility systems would be expected to meet the following types of objectives: & State the purpose of the system & Name the major components of the system & Match system parameters to facility mode & Predict the effects of a loss of on & Test a & Diagnose a problem in a & Repair a Used wisely, template statements can simplify development of learning objectives for common tasks and fundamental knowledge. However, users should recognize that most training situations will also require the development of unique learning objectives. Rigidly adhering to a set of template statements in these situations may prevent needed learning objectives from being developed. Learning objectives created using this method, as with any method, should be reviewed and approved for use. DOE-HDBK-1200-97 23 A list of sample condition, action, and standard template type statements is provided in the Appendix. Each facility should revise this list to their own applicable list of statements if this approach is to be used. The list can most easily be constructed by comparing all similar task elements and forming statements around those tasks. 4.2 Organization of Learning Objectives

Relationship of Terminal and Enabling Objectives Learning objectives provide the framework for the development of lesson plans and other training materials. Often, terminal objectives reflect the task statements. They should clearly state the behaviors the trainee will be expected to demonstrate on the job, and should include conditions and standards of performance. Enabling objectives support the terminal objective and address the task elements, skills, and knowledge required to support the task (terminal objective). They are derived from the element, skill, and knowledge statements that result from the analysis of the task. Terminal Objectives Terminal objectives represent performance at the task level and are normally derived from the task statements identified in the job analysis. Task-based terminal objectives should reflect the scope of the tasks upon which they are based. Those which are not derived directly from tasks should address a manageable body of material or topic. Each course lesson should have its own terminal objective that is based on a task. To the extent possible, the condition, action, and standard should be consistent with those associated with actual job performance. As with all objectives, terminal objectives must be measurable, and they should be limited to a single task or topic to help focus the trainee's attention. Basing objectives on tasks is the preferred method, but when tasks can not be used care should be taken to avoid confusion. Terminal objectives should be written to reflect the final desired performance as closely as possible. However, the action verb in the terminal objective for classroom training would be different than the action verb in the terminal objective developed for the laboratory, DOE-HDBK-1200-97 24 All Learning Objectives: & Contain condition, action, standard & Are observable and measurable & Are clearly stated in proper format & Are concise/succinct Terminal Objectives: & Are broad in scope & Are limited to single task & Reflect desired performance for selected training setting Enabling Objectives: & Support the terminal objective & Are derived from the element, skill, or knowledge statement & Are more specific than terminal objective TABLE 1. Terminal and enabling objectives simulator, or on the job training because classroom activities tend to be cognitive (knowledge-based) whereas lab, simulator, and OJT are psychomotor (hands-on or performance-based). An example of a terminal objective is as follows: Given a mechanics tool kit, the applicable procedure, and a disassembled motor operated valve, assemble the valve in accordance with specifications in the motor operated valve maintenance manual. Enabling Objectives Enabling objectives support terminal objectives. As stated previously, tasks on which terminal objectives are based are reasonably broad in scope. Their performance often involves a number of elements requiring the application of skills and knowledge. Enabling objectives are often less broad in scope than terminal objectives. Stated differently, enabling objectives address the steps through which trainees progress on their way to achieving the terminal objective. Like terminal objectives, enabling objectives also have a condition, action, and standard. However, in enabling objectives the condition and standard is often implied rather than expressed because they are the same as those

expressed in the terminal objective. The characteristics of learning objectives are summarized in Table 1. DOE-HDBK-1200-97 25 Consider the following example: Terminal Objective: 1.0 Given a malfunctioning circuit board, hand tools, equipment, and schematic diagrams, repair the circuit board in accordance with IEEE standard practices. Enabling Objectives: 1.1 Determine if the resistance of resistors is within specified limits. 1.2 Determine total resistance of a series/parallel circuit within plus or minus 3% given a schematic diagram. In the first enabling objective (1.1), the conditions and standard are the same as those in the terminal objective and are therefore implied. In the second enabling objective (1.2), the standard is not the same as that in the terminal objective and therefore must be expressed. Sequencing Objectives Complex subjects have an underlying structure that, when recognized, can simplify learning. Determining this structure and using it to sequence learning objectives can help trainees learn more efficiently. Sequencing or organizing learning objectives to take advantage of the relationships within the subject can reduce the time needed for trainees to meet the objectives. Sequencing of objectives establishes the order in which instruction will be organized and presented. This may be the same as the sequence of performance of the job, but sequencing of objectives is often based on some other logical relationship. Objectives can be organized based on any logical progression of the instruction. One of the most fundamental principles involves the use of Bloom's taxonomy. Objectives may be grouped according to their cognitive level; that is, known to unknown or simple to complex. Translated into practical terms, this means that objectives that have been classified as being at the knowledge level would be taught before higher level objectives such as those at the application level. Sometimes it may be necessary to deviate from the actual task DOE-HDBK-1200-97 26 performance sequence during training so that the easy steps are mastered before the difficult ones. After all the individual steps are learned, the task is practiced in the sequence defined in the applicable procedure. Further organization may be based on other relationships that exist among learning objectives. Some objectives may have a common factor in that they may be related because they refer to similar performances or share some basic knowledge. Such objectives should be grouped together to improve learning effectiveness. Other learning objectives may have a dependent relationship. It may be necessary to master one objective before it is possible to master another. When sequencing objectives of this type, it is important to present the objective that is dependent on some prior learning later in training. Still other learning objectives may have a supportive relationship. It may occur that mastery of one objective can help in the mastery of another. These objectives can be grouped in parallel. An example of supportive objectives in parallel is the two paths of enabling objectives shown in Figure 1. Finally, there may be learning objectives dealing with skills and knowledge that are unrelated to each other. In this instance, there may be objectives independent of one another that can be taught in any order. Objectives of this type may be included with dependent or supportive objectives in cases where appropriate.

Developing "organization charts" that show the logical relationships among learning objectives is an effective method of sequencing objectives. Show the completed organization chart to others and revise it as appropriate. Figure 1 shows an example "organization chart" demonstrating the relationships between enabling objectives and the terminal objective. This example uses the concept of personnel protection and work planning to organize learning objectives into two supportive paths. Within these paths, dependent learning objectives move from basic (bottom) to advanced (top) information. Radiation work permit requirements and minimizing radiation exposure learning objectives establish the foundations for the more advanced paths of handling lost or off-scale dosimetry and responding to conditions different from those described in the radiation work DOE-HDBK-1200-97 27 permit. Meeting these dependent learning objectives helps the trainee achieve the terminal learning objective of preventing unnecessary radiation exposure to one's self and to others. DOE-HDBK-1200-97 28 FIGURE 1. Example relationships between terminal and enabling objectives. DOE-HDBK-1200-97 29 5. EVALUATION OF LEARNING OBJECTIVES An assessment of training courses or programs should be performed on a continuing basis. Learning objectives provide a basis for these program evaluations. Quality of performance can be assessed for individual trainees or across classes or programs. Learning objectives should be revised if the results of this assessment, trainee or supervisor input, or trainee performance failures indicate the learning objectives are not adequate. Learning objectives should also be reviewed and revised accordingly when procedure changes, facility modifications, or facility/industry events indicate a need to revise training. Other factors may be the cause of trainee performance problems other than the objectives themselves. For example, the objective may be appropriate, but the sequencing is not, or the instruction is not aligned with the objective. As with the original learning objectives for a course, revised objectives must be reviewed and approved for use. Attention should be given to rewritten objectives to insure they meet the desired intent of the revision. The following are key considerations relative to learning objectives that should be emphasized when evaluating a course or program: & Learning objectives are used to identify training content and satisfactory trainee performance. & Learning objectives identify observable and measurable trainee action or behavior. & Test items are appropriate for learning objectives. & Learning objectives are compatible with expected entry level skills and knowledge of trainees. & Learning objectives are sequenced to assist trainees in making transitions from one skill level to another. & Learning activities are derived from the learning objectives. DOE-HDBK-1200-97 30 & Proposed changes to learning objectives are reviewed and approved by appropriate facility and training personnel. & Changes to learning objectives are tracked to demonstrate that training materials are maintained current with facility operating procedures, requirements, etc. DOE-HDBK-1200-97 31 APPENDIX EXAMPLE CONDITION, ACTION, AND STANDARD STATEMENTS DOE-HDBK-1200-97 APPENDIX 32 INTENTIONALLY BLANK DOE-HDBK-1200-97 APPENDIX 33 PURPOSE The purpose of this Appendix is to provide example condition, action, and standard

statements. These examples can be adapted to support writing learning objectives for operations, maintenance, or support training. The key to using these examples effectively is to recognize the application to the facility requirements and to revise them to fit the training needs of the intended audiences. EXAMPLE CONDITION STATEMENTS The following phrases illustrate several types of condition statements. They are a sample of many different possible combinations and include plant, job, information, and qualitative examples. Fill-in-the-blank spaces are included in statements that can be used in a variety of applications. Facility & During normal conditions & During facility mode & Given a transient & Given a change in & Given failure(s) & Given immediate action conditions & Given entry-level conditions to technical specifications/operational safety requirements & Given entry-level conditions to abnormal (or emergency) procedures, & Under all conditions Job & During shift turnover & While performing facility rounds & While working the shift & While making log entries & While alone in the DOE-HDBK-1200-97 APPENDIX 34 & Given an unlabeled & Using the (tools, equipment, etc.) Information & Using procedures/references & Using available indicators & Using alternative indicators & Using survey results & Using surveillance test results & Upon receiving annunciators/alarms & Given abnormal indications & Given any abnormal indications Qualitative & Upon request & From memory & Through observation & Using only sound & From smell alone & From touch alone & Upon direction & Without prompting EXAMPLE ACTION STATEMENTS Example action statements are provided in the affective, cognitive, and psychomotor performance areas. Each set of examples uses a series of fill-in-the-blank statements arranged from higher to lower levels of performance. They are a sample of many possible combinations. AFFECTIVE & Prevent & Exhibit DOE-HDBK-1200-97 APPENDIX 35 COGNITIVE Purpose & State the purpose of the system Safety Precautions & Instruct others to carry out the following safety precautions: & Predict the damage that each of the following can cause: & List the precautions associated with the system & Identify the personnel hazards or dangers associated with the system Design and Interrelationships & Predict the system response during a transient: & Predict the effects of a loss or malfunction of on & Explain the purpose of each of the following system interlocks: & Identify normal and alternate power supplies to the following: & Describe the functional dependencies that exist between the and systems & Match the following system parameters to facility modes: & Locate the components of the system & Draw a one-line diagram of the system that shows its key components and physical connections with other systems & Name the major components of the system & State the design basis of the system Procedures & Report errors in procedures & Use procedure to & Describe the process for reporting errors or sources of confusion in procedures & List the consequences of improperly performing a & Select the procedure(s) for the activity DOE-HDBK-1200-97 APPENDIX 36 Controls & Evaluate the loss of control to determine alternative means for regaining control

& Evaluate how the control layout, design, and operation limitations might contribute to human performance error & Identify any peculiar features of that might contribute to human error & Relate control adjustments to their effects on the following system parameters: & Identify where the system controls are located Alarms & Verify a alarm & Identify the alarms expected during the following facility activities: & Recognize the set points of the alarms & Locate the alarm annunciator & Identify where the following alarm sensors monitor the system: & Identify the alarms associated with the Indicators & Detect trends displayed by the recorder & Recognize the failure modes of each of the following monitors: & Match indications to specific facility conditions & Obtain information from the recorder & Locate where in the flow path each of the following indicators senses system parameters: & Identify the monitors associated with the system Sampling & Evaluate the need for an additional sample & Record the parameters of a sample & List the factors that can influence analysis results & Identify the labeling information required on samples DOE-HDBK-1200-97 APPENDIX 37 & Determine the flushing/recirculation requirements for sampling the & Identify the sample points in the Teamwork & Critique individual and team performance & Manage conflict through collaboration & Exhibit initiative and leadership & Provide complete input and feedback & Advocate a position or concern & Inquire to obtain needed information Operations & Avert a problem in the & Mitigate the effects of a on the & Evaluate the system response during a event & Evaluate operating limitations of the system & Predict how changing environmental conditions affect the system & Predict the consequences of component failure on the system & Explain the bases for limiting conditions of operations and safety limits of the & Match facility events to the notification requirements of outside agencies & Relate system status to the notification requirements of facility personnel & Relate individual performance responsibilities to each mode of facility operation & Place the in a safe condition & Classify the following system conditions into normal or abnormal: & Relate each system test to the parameters it monitors & Select the applicable technical safety requirements for each of the following facility conditions: & Identify the correct system alignments for each of the following conditions: & Determine the alignment for DOE-HDBK-1200-97 APPENDIX 38 & State the reason for & Describe the normal operation of the system Diagnostics & Evaluate the effects of corrective actions & Implement corrective actions & Evaluate alternatives & Assess the safety implications of each of the following recovery alternatives & Determine the urgency of a condition & Evaluate the potential for to worsen & Predict the effects of on other facility systems & Relate changes in to the need for action & Detect performance errors & Detect abnormal conditions & Detect changes in & Monitor the & Determine an alternative explanation of conditions & Interpret the following conditions: & Use alternative indicators to confirm conditions & Identify the symptoms associated with

& Identify abnormal characteristics PSYCHOMOTOR & (Any task or element statement is a possible psychomotor action statement) & Practice & Observe DOE-HDBK-1200-97 APPENDIX 39 EXAMPLE STANDARDS STATEMENTS The following statements suggest the type of phrases that can identify the performance criteria that trainees must fulfill to meet learning objectives. This is not an exhaustive list. It simply depicts some alternatives. Quantitative, procedural, and qualitative examples are provided. Fill-in-the-blank spaces are included in statements that can be used in a variety of applications. Designers (or developers) are encouraged to use statements that closely approximate actual performance criteria. QUANTITATIVE & With less than errors & To + & Within seconds/minutes/hours & Without producing more than units of waste & Without receiving more than mrems PROCEDURAL & In accordance with ALARA policy & In accordance with the RWP & In accordance with all certification criteria & In accordance with applicable labor agreements & In accordance with steps through of the emergency plan & In accordance with steps through of procedure number QUALITATIVE & Without error & Without spillage & Without breakage & Without loss of material & Without hesitation & With absolute clarity & On schedule & On the first attempt DOE-HDBK-1200-97 APPENDIX 40 & Before proceeding & To minimize time and optimize distance and shielding & To the accuracy of the instrument & Before conditions degrade & Prior to equipment damage & Prior to performing subsequent actions & Without entering a limiting condition of operation & While remaining within technical safety requirements DOE-HDBK-1200-97 41 CONCLUDING MATERIAL Review Activity: Preparing Activity: DOE Operations Offices DOE-EH-31 AD AL DP CH Project Number: EH FN 6910-0062 EM ID ER NV FM OR NE RF RL SAN SR National Laboratories ANL BNL FNAL INEL LANL LLNL MNREL ORNL PNNL SNL DOE-HDBK-1200-97 42 INTENTIONALLY BLANK TABLE OF CONTENTS