Par 13Page 12
c. Training involving communications and the use of automation can be developed for crews operating aircraft with advanced technology cockpits, or for crews transitioning into them.
14. ASSESSMENT OF CRM TRAINING PROGRAMS. It is vital that each program be assessed to determine if it is achieving its goals. Each organization should have a systematic assessment program. Assessment should track the effects of the training program so that critical topics for recurrent training may be identified and continuous improvements may be made in all other respects.
Assessment of the training program should include observation of the training process by program administrators and self-reports by participants using standard survey methods.
a. The emphasis in this assessment process should be on crew performance. The essential areas of CRM-related assessment include communications processes, decisionmaking, team building and maintenance, workload management, and situation awareness, always in balance with traditional technical proficiency. An additional function of such assessment is to determine the impact of CRM training and organization-wide trends in crew performance.
b. For optimal assessment, data on crewmembers’ attitudes and behavior should be collected before CRM indoctrination and again at intervals after the last component of CRM training, to determine both initial and enduring effects of the program. The goal should be to obtain an accurate picture of the organization’s significant corporate personality traits before formal adoption of CRM training, and to continue to monitor those traits after implementation.
c. Reinforcement and feedback are essential to effective CRM training programs. Crewmembers must receive continual reinforcement to sustain CRM concepts. Effective reinforcement depends upon usable feedback to crewmembers on their CRM practices and on their technical performance.
d. Usable feedback requires consistent assessment. Crewmembers and those involved in training and evaluation should be able to recognize effective and ineffective CRM behaviors. CRM concepts should be critiqued during briefing/debriefing phases of all training and checking events.
e. To summarize, the assessment program should:
(1) Measure and track the organization’s corporate culture as it is reflected in attitudes and norms.
(2) Identify topics needing emphasis within the CRM program.
(3) Ensure that all check airmen, supervisors, and instructors are well prepared and standardized.
15. THE CRITICAL ROLE OF CHECK AIRMEN AND INSTRUCTORS.
a. The success of any CRM training program ultimately depends on the skills of the people who administer the training and measure its effects. CRM instructors, check pilots, supervisors, and course
Par 15 Page 13 designers must be skilled in all areas related to the practice and assessment of CRM. These skills comprise an additional level to those associated with traditional flight instruction and checking.
b. Gaining proficiency and confidence in CRM instruction, observation, and measurement requires special training for instructors, supervisors, and check pilots in many CRM training processes. Among those processes are role-playing simulations, systematic crew-centered observation, administering LOFT programs, and providing usable feedback to crews.
c. Instructors, supervisors, and check pilots also require special training in order to calibrate and standardize their own skills.
d. Instructors, supervisors, and check airmen should use every available opportunity to emphasize the importance of crew coordination skills. The best results occur when the crews examine their own behavior with the assistance of a trained instructor who can point out both positive and negative CRM performance. Whenever highly effective examples of crew coordination are observed, it is vital that these positive behaviors be discussed and reinforced. Debriefing and critiquing skills are important tools for instructors, supervisors, and check pilots. (Behavioral markers of effective LOFT debriefings are shown in Appendix 2.)
e. Feedback from instructors, supervisors, and check airmen is most effective when it refers to the concepts that are covered in the initial indoctrination/awareness training. The best feedback refers to instances of specific behavior, rather than behavior in general.
16. EVOLVING CONCEPTS OF CRM.
a. Joint CRM Training. More and more carriers are discovering the value of revising CRM training to reach various employee groups, and sometimes to combine those groups during training.
Their objective is to improve the effectiveness and safety of the entire operations team as a working system.
(1) For many years air traffic controllers have been welcome in the cockpit in order to gain familiarity with procedures by observation from the cockpit jumpseat. Similarly, pilots are welcome to observe operations in air traffic facilities. Using real air traffic controllers during LOFT sessions has been proposed and tried.
(2) Aircraft dispatchers have functioned jointly with flight captains for years. They have been allowed, indeed required to observe cockpit operations from the cockpit jumpseat as part of their initial and recurrent qualification under part 121. Some carriers have included day trips to their aircraft dispatchers’ offices to provide the pilot insight into the other side of the joint function scheme. Those trips have commonly been part of the special training offered to first-time captains. Now, real-life aircraft dispatchers are increasingly being used in LOFT sessions. The training experience gained by the pilot and the dispatcher during LOFT is considered the logical extension of earlier training methods, providing interactivity where CRM (and DRM) principles are applied and discussed.
Par 16Page 14
(3) Maintenance personnel have also had access to the cockpit jumpseat under part 121.
Training of first-time captains has often included day trips to a carrier’s operations control center where a pilot and a maintenance supervisor can meet face to face and discuss issues of mutual interest in a thrumming, real-life setting. Some carriers have included maintenance personnel in LOFT sessions. Dedicated CRM training courses for maintenance personnel have been operating since 1991.
(4) Even broader cross-pollination of CRM concepts has been considered, using other groups such as passenger service agents, mid- and upper-level managers and special crisis teams like hijack and bomb-threat teams.
(5) Cabin attendants are probably the most obvious of the groups other than pilots who may profit from CRM training. Joint CRM training for pilots and flight attendants has been practiced for years. One fruitful activity in joint training has been that each group learns of the other group’s training in shared issues. The joint training has revealed inconsistencies between training for one group and training on the same topic for another group. Examples of shared issues include delays, the use of personal electronic devices in the cabin, and evacuation and ditching. When inconsistencies are identified between the contents of pilots’ manuals and flight attendants’ manuals, for instance, or between widely-held ideas or attitudes in those two populations, those inconsistencies are brought out into the open and often resolved. Other specific topics for joint training include:
(a) Pre-flight briefings.
(b) Post incident/accident procedures.
(c) Sterile cockpit procedures.
(d) Notification procedures pre-takeoff and pre-landing.
(e) Procedures for turbulence and other weather.
(f) Security procedures.
(g) Passenger-handling procedures.
(h) In-flight medical problems.
(i) Smoke/fire procedures.
(j) Passenger-related regulations such as those relating to smoking (section 121.571), exit row seating (section 121.585), and carry-on baggage (section 121.589).
(k) Authority of the pilot in command.
Par 16 Page 15
(6) CRM principles are made more relevant for pilots, flight attendants, and other groups by treating those principles in a familiar job-related context. Furthermore, each group should benefit from concurrent training in CRM that is complemented by usable knowledge of the other’s job.
(7) Communication and coordination problems between cockpit crewmembers and flight attendants continue to challenge air carriers and the FAA. Other measures with positive CRM training value for flightcrews are being considered, such as:
(a) Requiring cockpit observation flights for all new-hire flight attendants; and permitting cockpit observation flights for all other flight attendants;
(b) Including flight attendants as participants during LOFT;
(c) Scheduling month-long pairings of pilots and flight attendants; and
(d) Providing experienced flight crewmembers to teach new-hire flight attendant orientation classes.
b. Error Management. It is now understood that pilot errors cannot be entirely eliminated. It is important, therefore, that pilots develop appropriate error management skills and procedures. It is certainly desirable to prevent as many errors as possible, but since they cannot all be prevented, detection and recovery from errors should be addressed in training. Evaluation of pilots should also consider error management (error prevention, detection, and recovery). Evaluation should recognize that since not all errors can be prevented, it is important that errors be managed properly.
c. Advanced Crew Resource Management. CRM performance requirements or procedures are being integrated into the SOP’s of certain air carriers. Specific callouts, checks, and guidance have been included in normal checklists, the quick-reference handbook, abnormal/emergency procedures, manuals, and job aids. This integration captures CRM principles into explicit procedures used by flightcrews.
d. Culture Issues. While individuals and even teams of individuals may perform well under many conditions, they are subject to the influence of at least three cultures--the professional cultures of the individuals themselves, the cultures of their organizations, and the national cultures surrounding the individuals and their organizations. If not recognized and addressed, factors related to culture may degrade crew performance. Hence, effective CRM training must address culture issues as appropriate in each training population.
17. SUMMARY. Effective Crew Resource Management begins in initial training; it is strengthened by recurrent practice and feedback; and it is sustained by continuing reinforcement that is part of the corporate culture and embedded in every stage of training.
/s/ L. Nicholas Lacey Director, Flight Standards Service
Appendix 1 APPENDIX 1. CREW PERFORMANCE MARKER CLUSTERS Italicized Markers apply to Advanced Technology Flight Decks. These behavioral markers are provided to assist organizations in program and curriculum development and to serve as guidelines for feedback. They are not presented as a checklist for evaluating individual crewmembers.
1. COMMUNICATIONS PROCESSES AND DECISION BEHAVIOR CLUSTER.
a. Briefings. An effective briefing is interesting and thorough. It addresses coordination, planning, and problems. Although briefings are primarily a captain’s responsibility, other crewmembers may add significantly to planning and should be encouraged to do so.
Behavioral Markers.
(1) The briefing establishes an environment for open/interactive communications (for example, the captain calls for questions or comments, answers questions directly, listens with patience, does not interrupt or “talk over,” does not rush through the briefing, and makes eye contact as appropriate).
(2) The briefing is interactive and emphasizes the importance of questions, critique, and the offering of information.
(3) The briefing establishes a “team concept” (for example, the captain uses “we” language, encourages all to participate and to help with the flight).
(4) The briefing covers pertinent safety and operational issues.
(5) The briefing identifies potential problems such as weather, delays, and abnormal system operations.
(6) The briefing provides guidelines for crew actions centered on SOP's; division of labor and crew workload is addressed.
(7) The briefing includes the cabin crew as part of the team.
(8) The briefing sets expectations for handling deviations from standard operating procedures.
(9) The briefing establishes guidelines for the operation of automated systems (for example, when systems will be disabled; which programming actions must be verbalized and acknowledged).
(10) The briefing specifies pilot flying and pilot not flying duties and responsibilities with regard to automated systems.
b. Inquiry/Advocacy/Assertion. These behaviors relate to crewmembers’ promoting the course of action that they feel is best, even when it involves conflict with others.
Appendix 1 Behavioral Markers.
(1) Crewmembers speak up and state their information with appropriate persistence until there is some clear resolution.
(2) “Challenge and response” environment is developed.
(3) Questions are encouraged and are answered openly and nondefensively.
(4) Crewmembers are encouraged to question the actions and decisions of others.
(5) Crewmembers seek help from others when necessary.
(6) Crewmembers question status and programming of automated systems to confirm situation awareness.
c. Crew Self-Critique Regarding Decisions and Actions. These behaviors relate to the effectiveness of a group and/or an individual crewmember in critique and debriefing. Areas covered should include the product, the process, and the people involved. Critique may occur during an activity, and/or after completing it.
Behavioral Markers.
(1) Critique occurs at appropriate times, which may be times of low or high workload.
(2) Critique deals with positive as well as negative aspects of crew performance.
(3) Critique involves the whole crew interactively.
(4) Critique makes a positive learning experience. Feedback is specific, objective, usable, and constructively given.
(5) Critique is accepted objectively and nondefensively.
d. Communications/Decisions. These behaviors relate to free and open communication. They reflect the extent to which crewmembers provide necessary information at the appropriate time (for example, initiating checklists and alerting others to developing problems). Active participation in the decisionmaking process is encouraged. Decisions are clearly communicated and acknowledged.
Questioning of actions and decisions is considered routine.
Behavioral Markers.
(1) Operational decisions are clearly stated to other crewmembers.
(2) Crewmembers acknowledge their understanding of decisions.
Appendix 1
(3) “Bottom lines” for safety are established and communicated.
(4) The “big picture” and the game plan are shared within the team, including flight attendants and others as appropriate.
(5) Crewmembers are encouraged to state their own ideas, opinions, and recommendations.
(6) Efforts are made to provide an atmosphere that invites open and free communications.
(7) Initial entries and changed entries to automated systems are verbalized and acknowledged.
2. TEAM BUILDING AND MAINTENANCE CLUSTER.
a. Leadership Followership/Concern for Tasks. These behaviors relate to appropriate leadership and followership. They reflect the extent to which the crew is concerned with the effective accomplishment of tasks.
Behavioral Markers.
(1) All available resources are used to accomplish the job at hand.
(2) Flight deck activities are coordinated to establish an acceptable balance between respect for authority and the appropriate practice of assertiveness.
(3) Actions are decisive when the situation requires.
(4) A desire to achieve the most effective operation possible is clearly demonstrated.
(5) The need to adhere to standard operating practices is recognized.
(6) Group climate appropriate to the operational situation is continually monitored and adjusted (for example, social conversation may occur during low workload, but not high).
(7) Effects of stress and fatigue on performance are recognized.
(8) Time available for the task is well managed.
(9) Demands on resources posed by operation of automated systems are recognized and managed.
(10) When programming demands could reduce situation awareness or create work overloads, levels of automation are reduced appropriately.
b. Interpersonal Relationships/Group Climate. These behaviors relate to the quality of interpersonal relationships and the pervasive climate of the flight deck.
Appendix 1 Behavioral Markers.
(1) Crewmembers remain calm under stressful conditions.
(2) Crewmembers show sensitivity and ability to adapt to the personalities of others.
(3) Crewmembers recognize symptoms of psychological stress and fatigue in self and in others (for example, recognizes when he/she is experiencing “tunnel vision” and seeks help from the team; or notes when a crewmember is not communicating and draws him/her back into the team).
(4) “Tone” in the cockpit is friendly, relaxed, and supportive.
(5) During times of low communication, crewmembers check in with others to see how they are doing.
3. WORKLOAD MANAGEMENT AND SITUATION AWARENESS CLUSTER.
a. Preparation/Planning/Vigilance. These behaviors relate to crews’ anticipating contingencies and the various actions that may be required. Excellent crews are always “ahead of the curve” and generally seem relaxed. They devote appropriate attention to required tasks and respond without undue delay to new developments. (They may engage in casual social conversation during periods of low workload and not necessarily diminish their vigilance.)
Behavioral Markers.
(1) Demonstrating and expressing situation awareness; for example, the “model” of what is happening is shared within the crew.
(2) Active monitoring of all instruments and communications and sharing relevant information with the rest of the crew.
(3) Monitoring weather and traffic and sharing relevant information with the rest of the crew.
(4) Avoiding “tunnel vision” caused by stress; for example, stating or asking for the “big picture.”
(5) Being aware of factors such as stress that can degrade vigilance and watching for performance degradation in other crewmembers.
(6) Staying “ahead of the curve” in preparing for planned situations or contingencies, so that situation awareness and adherence to SOP's are assured.
Appendix 1 Page 5 (and 6)
(7) Ensuring that cockpit and cabin crewmembers are aware of plans.
(8) Including all appropriate crewmembers in the planning process.
(9) Allowing enough time before maneuvers for programming of the flight management computer.
(10) Ensuring that all crewmembers are aware of initial entries and changed entries in the flight management system.
b. Workload Distributed/Distractions Avoided. These behaviors relate to time and workload management. They reflect how well the crew manages to prioritize tasks, share the workload, and avoid being distracted from essential activities.
Behavioral Markers.
(1) Crewmembers speak up when they recognize work overloads in themselves or in others.
(2) Tasks are distributed in ways that maximize efficiency.
(3) Workload distribution is clearly communicated and acknowledged.
(4) Nonoperational factors such as social interaction are not allowed to interfere with duties.
(5) Task priorities are clearly communicated.
(6) Secondary operational tasks (for example, dealing with passenger needs and communications with the company) are prioritized so as to allow sufficient resources for primary flight duties.
(7) Potential distractions posed by automated systems are anticipated, and appropriate preventive action is taken, including reducing or disengaging automated features as appropriate.
Appendix 2 Page 1 (and 2) APPENDIX 2. LOFT DEBRIEFING PERFORMANCE INDICATORS The effective Line-Oriented Flight Training (LOFT) facilitator leads the flightcrew through a selfcritique of their own behavior and of their crew performance during the simulation. The debriefing and crew analysis include both technical and CRM discussion topics. Positive points of crew performance are discussed, as well as those needing improvement. At the conclusion of the session, key learning points are summarized covering all participants, including the instructor. A strong sense of training accomplishment and learning is taken away from the session.
1. The following performance markers may be used to evaluate the LOFT facilitator’s performance in the debrief/critique phase of LOFT.
a. Actively states the debriefing and critique agenda and solicits topics from the crew on items that they would like to cover; sets time limits.
b. Asks the crew for their appraisal of the mission overall.
c. States his/her own perceptions of the LOFT while guarding against making the crew defensive.
Comments are as objective as possible and focus on performance.
d. Shows appropriate incidents using videotape of the LOFT session, including examples of technical and CRM performance, and selects tape segments for discussion illustrating behaviors that feature the crew performance markers.
e. Effectively blends technical and CRM feedback in the debriefing; does not preach to the crew, but does not omit items worthy of crew discussion.
f. Is patient, and is constructive in probing into key areas where improvement is needed.
g. Ensures that all crewmembers participate in the discussion, and effectively draws out quiet or hostile crewmembers.
h. Provides a clear summary of key learning points.
i. Asks the crew for specific feedback on his/her performance.
j. Is effective in both technical and CRM debriefing.