c. Reinforcement and feedback are essential to effective CRM training. Crewmembers must receive continual reinforcement to sustain CRM concepts. Effective reinforcement depends upon usable feedback to crewmembers on their CRM practices and on their technical performance.
d. Usable feedback requires consistent assessment. Crewmembers and those involved in training and evaluation should be able to recognize effective and ineffective CRM behaviors. CRM concepts should be critiqued during briefing/debriefing phases of all training and checking events.
e. To summarize, the assessment process should:
(1) Measure and track the organization’s corporate culture as it is reflected in attitudes and norms.
(2) Identify topics needing emphasis within the CRM program.
(3) Ensure that all check airmen, supervisors, and instructors are well prepared and standardized.
15. THE CRITICAL ROLE OF CHECK AIRMEN AND INSTRUCTORS.
a. The success of any CRM training ultimately depends on the skills of the people who administer the training and measure its effects. CRM instructors, check pilots, supervisors, and course designers must be skilled in all areas related to the practice and assessment of CRM. These skills comprise an additional level to those associated with traditional flight instruction and checking.
b. Gaining proficiency and confidence in CRM instruction, observation, and measurement requires special training for instructors, supervisors, and check pilots in many CRM training processes. Among those processes are role-playing simulations, systematic crew-centered observation, administering LOFT, and providing usable feedback to crews.
c. Instructors, supervisors, and check pilots also require special training in order to calibrate and standardize their own skills.
d. Instructors, supervisors, and check airmen should use every available opportunity to emphasize the importance of crew coordination skills. The best results occur when the crews examine their own behavior with the assistance of a trained instructor who can point out positive and negative CRM performance. Whenever highly effective examples of crew coordination are observed, it is vital that these positive behaviors be discussed and reinforced. Debriefing and critiquing skills are important tools for instructors, supervisors, and check pilots. (Behavioral markers of effective LOFT debriefings are shown in appendix 2.)
e. Feedback from instructors, supervisors, and check airmen is most effective when it refers to the concepts that are covered in the initial indoctrination/awareness training. The best feedback refers to instances of specific behavior, rather than behavior in general.
Par 16 Page 13
16. EVOLVING CONCEPTS OF CRM.
a. Crew Monitoring and Cross-Checking. Several studies of crew performance, incidents, and accidents have identified inadequate flightcrew monitoring and cross-checking as a problem for aviation safety. Therefore, to ensure the highest levels of safety, each flight crewmember must carefully monitor the aircraft’s flight path and systems and actively cross-check the actions of other crewmembers. Effective monitoring and cross-checking can be the last line of defense that prevents an accident because detecting an error or unsafe situation may break the chain of events leading to an accident. This monitoring function is always essential, and particularly so during approach and landing when controlled flight into terrain (CFIT) accidents are most common. (For more information on SOPs that promote effective monitoring, see AC 120-71, as revised, appendix 19.)
b. Joint CRM Training. More carriers are discovering the value of expanding CRM training to reach various employee groups beyond flightcrew and flight attendants. Dissimilar groups are being brought together in CRM training and in other activities. The objective is to improve the effectiveness and safety of the entire operations team as a working system.
(1) The attacks of September 11, 2001, have caused many restrictions on flightdeck access.
Among those affected are air traffic controllers, for whom revised access procedures are being studied.
Pilots may observe operations in air traffic facilities under certain conditions, and are encouraged to do so. Using real air traffic controllers during LOFT sessions has also proven beneficial to pilots and participating controllers.
(2) Aircraft dispatchers have functioned jointly with flight captains for years. They have been allowed, indeed required to observe cockpit operations from the cockpit jumpseat as part of their initial and recurrent qualification under part 121. Some carriers have included day trips to their aircraft dispatchers’ offices to provide the pilot insight into the other side of the joint function scheme.
Those trips have commonly been part of the special training offered to first-time captains. Now, reallife aircraft dispatchers are increasingly being used in LOFT sessions. The training experience gained by the pilot and the dispatcher during LOFT is considered the logical extension of earlier training methods, providing interactivity where CRM (and DRM) principles are applied and discussed.
(3) Under certain conditions, maintenance personnel have had access to the cockpit jumpseat in air carrier operations under part 121; but that access has come under scrutiny because of security concerns following the attacks of 9/11. Training of first-time captains has often included day trips to a carrier’s operations control center where a pilot and a maintenance supervisor can meet face to face and discuss issues of mutual interest in a thrumming, real-life setting. Some carriers have included maintenance personnel in LOFT sessions. Dedicated CRM training courses for maintenance personnel have been operating since 1991.
(4) Even broader cross-pollination of CRM concepts has been considered, using other groups such as passenger service agents, mid- and upper-level managers, and special crisis teams like hijack and bomb-threat teams.
Par 16 Page 14
(5) Flight attendants are probably the most obvious of the groups other than pilots who may profit from CRM training. Joint CRM training for pilots and flight attendants is not required by FAA regulations, but it is encouraged and has been practiced effectively at some air carriers for years. One fruitful activity in joint training has been that each group learns of the other group’s training in shared issues. The joint training has revealed inconsistencies between training for one group and training on the same topic for another group. Examples of shared issues include delays, the use of personal electronic devices in the cabin, evacuation and ditching, and hijack response. When inconsistencies are identified between the contents of pilots’ manuals and flight attendants’ manuals, for instance, or between widely-held ideas or attitudes in those two populations, those inconsistencies are brought out into the open and often resolved. Other specific topics for joint training include:
(a) Pre-flight briefings;
(b) Post incident/accident procedures;
(c) Sterile cockpit procedures;
(d) Notification procedures (pre-takeoff and pre-landing);
(e) Procedures for turbulence and other weather;
(f) Security procedures;
(g) Passenger-handling procedures;
(h) In-flight medical problems;
(i) Smoke/fire procedures;
(j) Passenger-related regulations such as those relating to smoking (section 121.571), exit row seating (section 121.585), and carry-on baggage (section 121.589); and
(k) Authority of the pilot in command.
(6) CRM principles are made more relevant for pilots, flight attendants, and other groups by treating those principles in a familiar job-related context. Furthermore, each group should benefit from concurrent training in CRM that is complemented by usable knowledge of the other’s job.
(7) Communication and coordination problems between cockpit crewmembers and flight attendants continue to challenge air carriers and the FAA. Other measures with positive CRM training value for flightcrews are being considered, such as:
(a) Including flight attendants as participants during LOFT;
(b) Scheduling month-long pairings of pilots and flight attendants; and Par 16 Page 15
(c) Providing experienced flight crewmembers to teach new-hire flight attendant orientation classes.
c. Error Management. It is now understood that pilot errors cannot be entirely eliminated. It is important, therefore, that pilots develop appropriate error management skills and procedures. It is certainly desirable to prevent as many errors as possible, but since they cannot all be prevented, detection and recovery from errors should be addressed in training. Evaluation of pilots should also consider error management (error prevention, detection, and recovery). Evaluation should recognize that since not all errors can be prevented, it is important that errors be managed properly.
d. Advanced CRM. CRM performance requirements or procedures are being integrated into the SOPs of certain air carriers. Specific callouts, checks, and guidance have been included in normal checklists, the quick-reference handbook (QRH), abnormal/emergency procedures, manuals, and job aids. This integration captures CRM principles into explicit procedures used by flightcrews.
e. Culture issues. While individuals and even teams of individuals may perform well under many conditions, they are subject to the influence of at least three cultures – the professional cultures of the individuals themselves, the cultures of their organizations, and the national cultures surrounding the individuals and their organizations. If not recognized and addressed, factors related to culture may degrade crew performance. Hence, effective CRM training must address culture issues, as appropriate in each training population.
17. SUMMARY. Effective CRM begins in initial training; it is strengthened by recurrent practice and feedback; and it is sustained by continuing reinforcement that is part of the corporate culture and embedded in every stage of training.
/s/ James J. Ballough Director, Flight Standards Service Par 16 Page 16
Appendix 1 APPENDIX 1. CREW PERFORMANCE MARKER CLUSTERS
Italicized Markers apply to Advanced Technology Flight Decks. These behavioral markers are provided to assist organizations in program and curriculum development and to serve as guidelines for feedback. They are not presented as a checklist for evaluating individual crewmembers.
1. COMMUNICATIONS PROCESSES AND DECISION BEHAVIOR CLUSTER.
a. Briefings. An effective briefing is interesting and thorough. It addresses coordination, planning, and problems. Although briefings are primarily a captain’s responsibility, other crewmembers may add significantly to planning and should be encouraged to do so.
Behavioral Markers.
(1) The captain’s briefing establishes an environment for open/interactive communications
(e.g., the captain calls for questions or comments, answers questions directly, listens with patience, does not interrupt or “talk over,” does not rush through the briefing, and makes eye contact as appropriate).
(2) The briefing is interactive and emphasizes the importance of questions, critique, and the offering of information.
(3) The briefing establishes a “team concept” (e.g., the captain uses “we” language, encourages all to participate and to help with the flight).
(4) The captain’s briefing covers pertinent safety and security issues.
(5) The briefing identifies potential problems such as weather, delays, and abnormal system operations.
(6) The briefing provides guidelines for crew actions centered on standard operating procedures (SOP); division of labor and crew workload is addressed.
(7) The briefing includes the cabin crew as part of the team.
(8) The briefing sets expectations for handling deviations from SOPs.
(9) The briefing establishes guidelines for the operation of automated systems (e.g., when systems will be disabled; which programming actions must be verbalized and acknowledged).
(10) The briefing specifies duties and responsibilities with regard to automated systems, for the pilot flying (PF) and the pilot monitoring (PM).
b. Inquiry/Advocacy/Assertion. These behaviors relate to crewmembers promoting the course of action that they feel is best, even when it involves conflict with others.
Appendix 1 Behavioral Markers.
(1) Crewmembers speak up and state their information with appropriate persistence until there is some clear resolution.
(2) “Challenge and response” environment is developed.
(3) Questions are encouraged and are answered openly and nondefensively.
(4) Crewmembers are encouraged to question the actions and decisions of others.
(5) Crewmembers seek help from others when necessary.
(6) Crewmembers question status and programming of automated systems to confirm situation awareness.
c. Crew Self-Critique Regarding Decisions and Actions. These behaviors relate to the effectiveness of a group and/or an individual crewmember in critique and debriefing. Areas covered should include the product, the process, and the people involved. Critique may occur during an activity, and/or after completing it.
Behavioral Markers.
(1) Critique occurs at appropriate times, which may be times of low or high workload.
(2) Critique deals with positive as well as negative aspects of crew performance.
(3) Critique involves the whole crew interactively.
(4) Critique makes a positive learning experience. Feedback is specific, objective, usable, and constructively given.
(5) Critique is accepted objectively and nondefensively.
d. Communications/Decisions. These behaviors relate to free and open communication. They reflect the extent to which crewmembers provide necessary information at the appropriate time (e.g., initiating checklists and alerting others to developing problems). Active participation in the decisionmaking process is encouraged. Decisions are clearly communicated and acknowledged.
Questioning of actions and decisions is considered routine.
Behavioral Markers.
(1) Operational decisions are clearly stated to other crewmembers.
(2) Crewmembers acknowledge their understanding of decisions.
Appendix 1
(3) “Bottom lines” for safety are established and communicated.
(4) The “big picture” and the game plan are shared within the team, including flight attendants and others as appropriate.
(5) Crewmembers are encouraged to state their own ideas, opinions, and recommendations.
(6) Efforts are made to provide an atmosphere that invites open and free communications.
(7) Initial entries and changed entries to automated systems are verbalized and acknowledged.
2. TEAM BUILDING AND MAINTENANCE CLUSTER.
a. Leadership Followership/Concern for Tasks. These behaviors relate to appropriate leadership and followership. They reflect the extent to which the crew is concerned with the effective accomplishment of tasks.
Behavioral Markers.
(1) All available resources are used to accomplish the job at hand.
(2) Flight deck activities are coordinated to establish an acceptable balance between respect for authority and the appropriate practice of assertiveness.
(3) Actions are decisive when the situation requires.
(4) A desire to achieve the most effective operation possible is clearly demonstrated.
(5) The need to adhere to SOPs is recognized.
(6) Group climate appropriate to the operational situation is continually monitored and adjusted (e.g., social conversation may occur during low workload, but not high).
(7) Effects of stress and fatigue on performance are recognized.
(8) Time available for the task is well managed.
(9) Demands on resources posed by operation of automated systems are recognized and managed.
(10) When programming demands could reduce situation awareness or create work overloads, levels of automation are reduced appropriately.
b. Interpersonal Relationships/Group Climate. These behaviors relate to the quality of interpersonal relationships and the pervasive climate of the flight deck.
Appendix 1
Behavioral Markers.
(1) Crewmembers remain calm under stressful conditions.
(2) Crewmembers show sensitivity and ability to adapt to the personalities of others.
(3) Crewmembers recognize symptoms of psychological stress and fatigue in self and in others
(e.g., recognizes when he/she is experiencing “tunnel vision” and seeks help from the team; or notes when a crewmember is not communicating and draws him/her back into the team).
(4) “Tone” in the cockpit is friendly, relaxed, and supportive.
(5) During times of low communication, crewmembers check in with others to see how they are doing.
3. WORKLOAD MANAGEMENT AND SITUATION AWARENESS CLUSTER.
a. Preparation/Planning/Vigilance. These behaviors relate to crews anticipating contingencies and the various actions that may be required. Excellent crews are always “ahead of the curve” and generally seem relaxed. They devote appropriate attention to required tasks and respond without undue delay to new developments. (They may engage in casual social conversation during periods of low workload and not necessarily diminish their vigilance.)
Behavioral Markers.
(1) Demonstrating and expressing situation awareness (e.g., the “model” of what is happening is shared within the crew).
(2) Active monitoring of all instruments and communications and sharing relevant information with the rest of the crew.
(3) Monitoring weather and traffic and sharing relevant information with the rest of the crew.
(4) Avoiding “tunnel vision” caused by stress (e.g., stating or asking for the “big picture”).
(5) Being aware of factors such as stress that can degrade vigilance, and watching for performance degradation in other crewmembers.
(6) Staying “ahead of the curve” in preparing for planned situations or contingencies, so that situation awareness and adherence to SOPs is assured.
(7) Ensuring that cockpit and cabin crewmembers are aware of plans.
(8) Including all appropriate crewmembers in the planning process.
Appendix 1
(9) Allowing enough time before maneuvers for programming of the flight management computer.
(10) Ensuring that all crewmembers are aware of initial entries and changed entries in the flight management system.
b. Workload Distributed/Distractions Avoided. These behaviors relate to time and workload management. They reflect how well the crew manages to prioritize tasks, share the workload, and avoid being distracted from essential activities.
Behavioral Markers.
(1) Crewmembers speak up when they recognize work overloads in themselves or in others.
(2) Tasks are distributed in ways that maximize efficiency.
(3) Workload distribution is clearly communicated and acknowledged.
(4) Nonoperational factors such as social interaction are not allowed to interfere with duties.
(5) Task priorities are clearly communicated.
(6) Secondary operational tasks (e.g., dealing with passenger needs and communications with the company) are prioritized so as to allow sufficient resources for primary flight duties.
(7) Potential distractions posed by automated systems are anticipated, and appropriate preventive action is taken, including reducing or disengaging automated features as appropriate.
Page 5 (and 6)
Appendix 2 APPENDIX 2. LOFT DEBRIEFING PERFORMANCE INDICATORS
The effective line-oriented flight training (LOFT) facilitator leads the flightcrew through a selfcritique of their own behavior and of their crew performance during the simulation. The debriefing and crew analysis include both technical and crew resource management (CRM) discussion topics.
Positive points of crew performance are discussed as well as those needing improvement. At the conclusion of the session, key learning points are summarized covering all participants, including the instructor. A strong sense of training accomplishment and learning is taken away from the session.
The following performance markers may be used to evaluate the LOFT facilitator’s performance in the debrief/critique phase of LOFT.
a. Actively states the debriefing and critique agenda and solicits topics from the crew on items that they would like to cover; sets time limits.
b. Asks the crew for their appraisal of the mission overall.
c. States his/her own perceptions of the LOFT while guarding against making the crew defensive.
Comments are as objective as possible and focus on performance.
d. Shows appropriate incidents using videotape of the LOFT session, including examples of technical and CRM performance, and selects tape segments for discussion illustrating behaviors that feature the crew performance markers.
e. Effectively blends technical and CRM feedback in the debriefing; does not preach to the crew, but does not omit items worthy of crew discussion.
f. Is patient, and is constructive in probing into key areas where improvement is needed.
g. Ensures that all crewmembers participate in the discussion, and effectively draws out quiet or hostile crewmembers.
h. Provides a clear summary of key learning points.
i. Asks the crew for specific feedback on his/her performance.
j. Is effective in both technical and CRM debriefing.
Page 1 (and 2)