For optimal assessment, data on crewmembers' attitudes and behavior should be collected before CRM indoctrination and again at intervals after the last component of CRM training to determine both initial and enduring effects of the program. The goal should be to obtain an accurate picture of the organization's significant corporate personality traits before formal adoption of CRM training, and to continue to monitor those traits after implementation.
Reinforcement and feedback are essential to effective CRM training programs. Crewmembers must receive continual reinforcement to sustain CRM concepts. Effective reinforcement depends upon usable feedback to crewmembers on their human factors practices and on their technical performance.
d. Usable feedback requires consistent assessment.
Crewmembers and those involved in training and evaluation should be able to recognize effective and ineffective human factors behaviors. It is not expected or intended that crewmembers should be formally evaluated and graded on the practice of CRM concepts. . Rather, CRM concepts should be included during briefing/debriefing phases of training.
e. To summarize, the assessment program should:
Measure and track the organization's corporate culture as it is reflected in attitudes and norms.
(2) Identify topics needing emphasis within the CRM program.
(3) Ensure that all check airmen and instructors are well prepared and standardized.
14. THE CRITICAL ROLE OF CHECK AIRMEN AND INSTRUCTORS.
success of any CRM training program ultimately depends on the skills of the people who administer the training and measure its effects. CRM instructors, check pilots, and course designers must be skilled in all areas related to the practice and assessment of CRM. It is important to note that these skills comprise an additional level to those associated with traditional flight instruction and checking.
b. Gaining proficiency and confidence in CRM instruction, observation, and measurement requires special training for instructors and check pilots in many CRM training processes.
Par 13 13
Among those processes are role-playing simulations, systematic crew-centered observation, administering LOFT programs, and providing usable feedback to crews.
C - Instructors and check pilots also require special training in order to calibrate and standardize their own skills.
In order to provide maximum learning for all crewmembers, instructors and check airmen must use every available opportunity to emphasize the importance of crew coordination skiiis.
best results occur when the crews examine their own behavior with the assistance of a trained instructor who can point out both positive and negative CRM performance. Whenever highly effective examples of crew coordination are observed, it is vital that these positive behaviors are discussed and reinforced.
Debriefing and critiquing skills are important tools for instructors and check pilots. (Behavioral markers of effective e . Feedback from instructors and check airmen is most effective when it refers to the concepts that are covered in the initial indoctrination/awareness training. The best feedback refers to specific cases of specific behavior, rather than to generalities about behavior.
15. EVOLVING CONCEPTS OF CRM: EXTENDING TRAINING BEYOND THE COCKPIT. Several air carriers have found it useful to extend CRM training to other operational group's. Their objective has been to improve the effectiveness of these groups and the quality of interactions between them and cockpit crews.
a . Under the FAR, dispatchers share responsibility for the conduct of flight with captains. Both must agree that the flight can be completed safely. The importance of their coordination is the reason for broader and better interactive training of pilots and dispatchers.
Cabin crews are also part of the team responsible for safe flight. The extension of CRM training to cabin crewmembers conditions still other team behaviors needed in normal and emergency situations.
14 Par 14
Maintenance activities are also team efforts that include interactions with the cockpit. CRM training for this group enhances maintenance team performance and joint activities with flightcrews.
Thomas. C. Accardi.
Director, Flight Standards Service Par 15 15
AC 120-51A 2/10/93 Appendix 1 APPENDIX 1. CREW PERFORMANCE MARKER CLUSTERS (Italicized Markers apply to Advanced Technology Flight Decks) These behavioral markers are provided to assist organizations in program and curriculum development and to serve as guidelines for feedback. They are not presented as a checklist for evaluating individual crewmembers.
1. COMMUNICATIONS PROCESSES AND DECISION BERAVIOR CLUSTER.
a. Briefings. The effective briefing is interesting and thorough. It addresses coordination, planning, and problems.
Although briefings are primarily a captain's responsibility, other crewmembers may add significantly to planning and should be encouraged to do so.
Behavioral Markers
(1) The briefing establishes an environment for open/interactive communications (for example, the captain calls for questions or comments, answers questions directly, listens cort edes not inte with patience, does not interrupt or "talk over", does not rush through the briefing, and makes eye contact as appropriate).
The briefing is interactive and emphasizes the importance of questions, critique, and the offering of information.
(3) The briefing establishes a "team concept" (for example, the captain uses "we" language, encourages all to participate and to help with the flight).
(4) The briefing covers pertinent safety and operational issues.
(5) The briefing identifies potential problems such as weather, delays, and abnormal system operations.
(6) The briefing provides guidelines for crew actions;
division of labor and crew workload is addressed.
(7) The briefing includes the cabin crew as part of the team.
(8) The briefing sets expectations for handling deviations from standard operating procedures.
Appendix 1 APPENDIX 1. CREW PERFORMANCE MARKER CLUSTERS (Continued)
(9) The briefing establishes guidelines for the operation of automated systems (for example, when systems will be disabled;
which programming actions must be verbalized and acknowledged).
flying duties and responsibilities with regard to automated (10) The briefing specifies pilot flying and pilot not systems.
Beat Oven when crewmembers' promoting the course of action that they feel is b. Inquiry/Advocacy/Assertion. These behaviors relate to best, even when it involves conflict with others.
Behavioral Markers
(1) Crewmembers speak up and state their information with appropriate persistence until there is some clear resolution.
(2) "Challenge and response" environment is developed.
(3) Questions are encouraged and are answered openly and nondefensively.
(4) Crewmembers are encouraged to question the actions and decisions of others.
(5) Crewmembers seek help from others when necessary.
(6) Crewmembers question status and programming of automated systems to confirm situational awareness.
Crew Self-Critique Regarding Decisions and Actions.
These behaviors relate to the effectiveness of a group and/or an individual crewmember in critique and debriefing. Areas covered should include the product, the process, and the people involved. Critique may occur during an activity, and/or after completing it- Behavioral Markers
(l) Critique occurs at appropriate times, which may be times of low or high workload.
(2) Critique deals with positive as well as negative aspects of crew performance.
(3) Critique involves the whole crew interactively.
• 2
Appendix 1 APPENDIX 1. CREW PERFORMANCE MARKER CLUSTERS (Continued)
(4) Critique makes a positive learning experience.
Feedback is specific, objective, usable, and constructively given.
(5) Critique is accepted objectively and nondefensively.
d. Communications/Decisions. These behaviors relate to free and open communication. They reflect the extent to which crewmembers provide necessary information at the appropriate time (for example, initiating checklists and alerting others to devels is polaged. Acts participation the decisi making acknowledged. Questioning of actions and decisions is considered routine.
Behavioral Markers
(1) Operational decisions are clearly stated to other crewmembers.
(2) Crewmembers acknowledge their understanding of decisions.
(3) "Bottom lines" for safety are established and communicated.
(4) The "big picture" and the game plan are shared within the team, including flight attendants and others as appropriate.
(5) Crewmembers are encouraged to state their own ideas, opinions, and recommendations.
(6) Efforts are made to provide an atmosphere that invites open and free communications.
(7) Initial entries and changed entries to automated systems are verbalized and acknowledged.
2. TEAM BUILDING AND MAINTENANCE CLUSTER.
a. Leadership Followership/Concern for Tasks. These behaviors relate to appropriate leadership and followership.
They reflect the extent to which the crew is concerned with the effective accomplishment of tasks.
Appendix 1 APPENDIX 1. CREW PERFORMANCE MARKER CLUSTERS (Continued) Behavioral Markers
(1) All available resources are used to accomplish the job at hand.
(2) Flight deck activities are coordinated to establish an acceptable balance between respect for authority and the appropriate practice of assertiveness.
(3) Actions are decisive when the situation requires.
(4) A desire to achieve the most effective operation possible is clearly demonstrated.
(5) The need to adhere to standard operating practices is recognized.
Group climate appropriate to the operational situation is continually monitored and adjusted (for example, social conversation may occur during low workload, but not high).
(7) Effects of stress and fatigue on performance are recognized.
(8) Time available for the task is well managed.
(9) Demands on resources posed by operation of automated systems are recognized and managed.
(10) When programming demands could reduce situational awareness or create work overloads, levels of automation are reduced appropriately.
b. Interpersonal Relationships/Group Climate.
behaviors relate to the quality of interpersonal relationships and the pervasive climate of the flight deck.
Behavioral Markers
(1) Crewmembers remain calm under stressful conditions.
(2) Crewmembers show sensitivity and ability to adapt to the personalities of others.
• 4
2/10/93 AC 120-51A Appendix 1 APPENDIX 1. CREW PERFORMANCE MARKER CLUSTERS (Continued)
(3) Crewmembers recognize symptoms of psychological stress and fatigue in self and in others (for example, recognizes when he/sue is experiencing "tunnel vision" and seeks help from the team; or notes when a crewmember is not communicating and draws him/her back into the team).
(4) "Tone" in the cockpit is friendly, relaxed, and supportive.
(5) During times of low communication, crewmembers check in with others to see how they are doing.
3. WORKLOAD MANAGEMENT AND SITUATIONAL AWARENESS CLUSTER.
Preparation/Planning/Vigilance. These behaviors to crews' anticipating contingencies and the various actions that may be required. Excellent crews are always "ahead of the curve" and generally seem relaxed. They devote appropriate attention to required tasks and respond without undue delay to new developments. (They may engage in casual social conversation during periods of low workload and not necessarily diminish their vigilance.)
Behavioral Markers
(1) Demonstrating and expressing situational awareness;
for example, the "model" of what is happening is shared within the crew.
(2) Active monitoring of all instruments and communications and sharing relevant information with the rest of the crew.
(3) Monitoring weather and traffic and sharing relevant information with the rest of the crew.
(4) Avoiding "tunnel vision" caused by stress; for example, stating or asking for the "big picture."
(5) Being aware of factors such as stress that can degrade vigilance and watching for performance degradation in other crewmembers.
Staying "ahead of the curve" in preparing for planned situations or contingencies.
Appendix 2 APPENDIX 2. LOFT DEBRIEFING PERFORMANCE INDICATORS The effective LOFT facilitator leads the flightcrew through a self-critique of their own behavior and of their crew performance during the simulation. The debriefing and crew analysis include both technical and CRM discussion topics. Positive points of crew performance are discussed, as well as those needing improvement. At the conclusion of the session, key learning points are summarized covering all participants, including the instructor. A strong sense of training accomplishment and learning is taken away from the session.
1. The following performance markers may be used to evaluate the LOFT facilitator's performance in the debrief/critique phase of LOFT.
Actively states the debriefing and critique agenda and solicits topics from the crew on items that they would like to cover; sets time limits.
b. Asks the crew for their appraisal of the mission overall.
c. States his/her own perceptions of the LOFT while guarding against making the crew defensive. Comments are as objective as possible and focus on performance.
d. Shows appropriate incidents using videotape of the LOFT session, including examples of technical and CRM performance, and selects tape segments for discussion illustrating behaviors that feature the crew performance markers.
Effectively blends technical and CRM feedback in the debriefing; does not preach to the crew; but does not omit items worthy of crew discussion.
Is patient, and is constructive in probing into key areas where improvement is needed.
g. Ensures that all crewmembers participate in the discussion, and effectively draws out quiet or hostile crewmembers.
h. Provides a clear summary of key learning points.
i. Asks the crew for specific feedback on his/her performance.
j. Is effective in both technical and CRM debriefing.