Par 6-1 Page 31 CHAPTER 6. THE ROLE OF LINE-OPERATIONAL SIMULATION FACILITATORS
6-1. MINIMUM QUALIFICATIONS. Line-Operational Simulation (LOS) facilitators/instructors/evaluators should be trained in the philosophy, skills, and conduct of LOS and Crew Resource Management (CRM). They should be able to effectively observe and critique both individual and crew performance during the scenario. To do this, they should meet the minimum requirements discussed in the following paragraphs.
a. Line Familiar. Facilitators should be line familiar. This will ensure that facilitators accurately perceive and evaluate situations as they arise.
b. Qualified as Instructors/Evaluators. To conduct Line-Oriented Flight Training (LOFT) or Special Purpose Operational Training (SPOT), facilitators should be qualified as defined in part 121, §§ 121.412 and 121.414 and appendix H; or part 135, §§ 135.338 and 135.340, as applicable, and as required by the approved Advanced Qualification Program (AQP), if appropriate. To conduct Line Operational Evaluation (LOE), facilitators should be qualified as defined in §§ 121.411 and 121.413 and part 121 appendix H; or §§ 135.337 and 135.339, as applicable, and as required by the approved AQP.
c. Trained in CRM Skills. Facilitators will receive training in CRM skills in order to observe and critique these areas in LOS. Refer to the current edition of Advisory Circular (AC) 120-51, Crew Resource Management Training, for further information.
d. Trained in Methods for Briefing, Debriefing, and Critique. Facilitators should be trained to conduct the briefing and debriefing/critique phases of LOS, including how to provide effective feedback.
6-2. FACILITATOR RESPONSIBILITIES. The following is a description of the roles and responsibilities of facilitators:
a. Briefing and Preparation. Facilitators should be able to effectively convey the purpose of the LOS and how it is representative of line operations. Facilitators should also explain their role as observers during the training/evaluation, in that they are not considered present unless playing a non-flightcrew member role in the scenario (e.g., air traffic control (ATC), flight attendant (F/A), or dispatch). This does not apply to SPOT, which allows for an interactive role on the part of the facilitator.
b. Flight Segment. Facilitators should be able to observe and perform ancillary roles.
They should be trained in observing and assessing technical and CRM skills. Facilitators should also be trained in proper pacing, proper introduction of abnormal/emergency procedures, and methods of handling unforeseen crew actions.
c. Simulation. Of vital importance to the effectiveness of LOS is the creation of a strong illusion of reality in the simulated flights. This requirement dictates that many routine activities, such as flight paperwork, manuals, and communications should be carefully prepared. Previous experience with LOFT and LOE has shown that overlooking these activities can destroy this illusion.
Par 6-2 Page 32
d. Resources. The facilitator’s goal is to produce crew performance and behavior that is typical for an actual line flight in the same set of circumstances as those developed in the scenario. In keeping with this goal, it is essential that crews have access to all the resources they would have during an actual line flight. The briefing should include mention of the roleplaying aspect of LOS and its importance to overall LOS effectiveness.
e. Facilitator Role. The role of the facilitator in LOFT or LOE should be viewed as that of communicator, observer, and moderator in the debriefing process. He or she is not an instructor in the traditional sense during the flight simulation training device (FSTD) period. He or she is the facilitator or manager of the flight, using appropriate radio calls or responses to direct the flight along the desired path. The facilitator must be prepared to accept and manage alternate courses of action that the crew may wish to follow. The facilitator should remain as unobtrusive as possible within the physical limitations of the FSTD. He or she should resist the temptation to instruct, and must not intrude in any way into the situation.
f. Communication. All communications must be conducted in the manner normally found on a line flight; that is, via radio from outside the “aircraft;” via interphone or normal conversations between flight deck crewmembers; or, in the case of communication between the flight deck and cabin, via the usual aircraft equipment for this purpose. All external communications (e.g., ATC and ground crew) must be credible and realistic.
g. Recorded Feedback. The entire FSTD phase of the flight, including initial flight deck setup, should be recorded, if the equipment is available. The importance of the correct use of recording cannot be overstated: LOS with recording is one of the most powerful tools we have for reinforcing desirable behavior in CRM. During debriefing, the recording should be reviewed and discussed by the flightcrew with emphasis being placed upon crew performance, including their use of CRM behavioral markers and skills. When crewmembers have learned and can appreciate the importance of open and direct critique for purposes of operational review and analysis, a platform is in place for effective post-LOS discussion that reviews more than stick-and-rudder skills or systems knowledge. Following review of the recording, it may be erased.
h. Debriefing and Critique. Facilitators should provide both positive and negative feedback during critiques of individual and crew performance. Prior to the facilitator’s critiques, crewmembers should be encouraged to critique themselves. Facilitators will provide feedback to the crew to encourage the changes needed for improved performance. Facilitators should also provide specific recommendations to improve individual crewmembers’ performance.
Par 7-1 Page 33 CHAPTER 7. LINE-OPERATIONAL SIMULATION SCENARIO DESIGN
7-1. SCENARIO DESIGN METHODOLOGIES. This chapter provides a Line-Operational Simulation (LOS) scenario design methodology that has proven effective. There may be other, equally effective LOS development methods. The important aspect of LOS development is that a disciplined methodology is employed.
7-2. PHILOSOPHY OF LOS DESIGN AND CONDUCT.
a. Scenarios. LOS scenarios are best designed to be operationally relevant, believable, and good tests of the crew’s technical and Crew Resource Management (CRM) skills. LOS training is systematic and is intended to simulate actual problem situations on the line that require good crew skills for effective resolution and decision making.
b. Line Realism. Because LOS requires as much realism as possible, LOS design and evaluation guidelines should maximize line realism. Preflight activities and detailed review of flight paperwork, manuals, and conduct of communications should be included in the scenario.
This requirement does not preclude employing scenarios that use short segments beginning or ending in an en route environment, if the objectives of the LOS can be met. If the scenario is designed to begin in an en route environment, enough quiet time should be present for the crew to become acclimated to the flight routine. These en route segments are identified as Special Purpose Operational Training (SPOT). Line Operations Safety Audit (LOSA) data demonstrate that a flightcrew addresses 40 percent of the risks encountered during a flight prior to takeoff.
Therefore, SPOT should not be employed simply to shortcut this phase of flight, unless the objectives of the scenario warrant it. An operator should not change the name of a Line-Oriented Flight Training (LOFT) to a SPOT simply to avoid the requirement for FAA approval.
c. Open Communication. LOS scenarios should be designed to foster an environment where free and open communication is practiced. This encourages crewmembers to provide necessary information at the appropriate time (e.g., initiating checklists, advocating positions, and problem definition). Furthermore, the LOS design should encourage active participation in the decision-making process and questioning of actions and decisions by all crewmembers.
d. Level of Difficulty. One misconception is the belief that LOS training should continuously increase crew operational workload until the crew becomes overloaded. This is not the purpose or intent of LOS and can actually help to defeat its effectiveness. The difficulty of the LOS should not be designed to saturate a flightcrew or impose an unrealistic level of difficulty or complexity. On the other hand, the LOS must provide enough difficulty to adequately test the flightcrew’s skills and capabilities. LOS scenarios are most effective if they are straightforward. For example, choosing a departure airport that requires an effective preflight briefing might be one way to begin. A scenario that allows the crew to choose from different options is very useful. One scenario can have a wide variety of outcomes and choices, depending on the decision and course of action that a crew undertakes. Again, the scenario should be realistic, and the situation should be one where crewmembers live with whatever problems they have until the situation is either resolved or the aircraft (flight simulation training device (FSTD)) is back on the ground.
Par 7-2 Page 34
e. Briefing and Debriefing. The effective LOS experience begins with a briefing to discuss the LOS objectives and expectations. To be complete, the LOS must include a debriefing to examine the crew performance demonstrated during the LOS. The facilitator draws on personal experience and training in CRM and technical issues to elicit discussions of points of interest and operational relevance. Positive comments regarding crew performance should be emphasized in the debriefing, as well as comments regarding areas of crew performance that need improvement. Crewmembers must be given the opportunity to critique and analyze their own performance and review key points.
7-3. SCENARIO DESIGN PROCESS.
a. Developing the Scenario. The framework described here for developing LOS scenarios is based on the concept of an event set, a group of related tasks and conditions that are part of the scenario and are integrated into the LOS session with specific CRM and technical training objectives. Included in the framework is a new method for identifying specific CRM skills appropriate for the event sets and a tool for developing the CRM category profile of LOS scenarios. The result is an approach that makes LOS sessions more manageable and easier to assess by allowing the facilitator to concentrate on a few CRM categories within any given event set. It also limits the specific crew behavior observations to very specific behavioral markers and indicator behaviors that are either observed or not observed. Not only is the facilitator focusing on a single CRM category, such as “Communication,” but he or she is also looking for a particular, predetermined result driven by the details of the event, such as “Second in command (SIC) communicates Weight and Balance (W&B) for passenger count.”
b. Minimizing Subjectivity. One of the major goals of the use of the event set methodology is to remove as much subjectivity from the grading process as possible, particularly for CRM skills. By engineering the event in such a way that there is a clear indicator of the desired performance, the performance issue is focused down to, for example, whether the SIC either did or did not communicate W&B at the appropriate time and in the appropriate manner.
That is what is being graded, not some subjective notion of “communication.” The assessment is not based on the general adequacy of the SIC’s communication skills, but instead on the adequacy of a very specific behavior. This technique has allowed evaluators to be trained to grade CRM skills with the same consistency with which they grade technical skills.
c. The Event Set. The primary unit of both LOS design and CRM assessment is the event set. The event set is made up of one or more events, including an event trigger, distracters, and supporting events. The event trigger is the condition or conditions under which the event is fully activated. The distractors are conditions inserted within the event set timeframe that are designed to divert the crew’s attention from other events that are occurring or are about to occur. Event sets with extremely disruptive triggers do not necessarily require a distractor. Finally, supporting events are other events taking place within the event set designed to further CRM and technical training objectives, usually the normal events of that phase of activity.
d. Training Objectives. In LOS scenario design, the CRM and technical training objectives should be integrated into the event sets. This event set framework allows the design team to present the appropriate degree of realism in the LOS. Instead of focusing on a single technical issue, the event set integrates the entire complex line environment (e.g., terrain, air
Par 7-3 Page 35 traffic control (ATC), or weather issues) to facilitate and maximize the crew’s performance in response to specified CRM and technical issues. With the LOS scenario now defined by event sets, scenario validation is performed at the event set level rather than limiting validation to the overall LOS. This allows, over time, for a sufficiently large enough library of effective scenarios to build up, such that the operator can begin building new scenarios from the event sets of previously developed scenarios. Rather than building each scenario from scratch, they can begin to mix and match event sets from their event set libraries to build new entire scenarios.
e. Developing Scenarios from the Event Set. The event set framework supports the development of LOS scenarios based on complex events as well as simple events. Simple events have no further consequences on the conduct of the flight once they have been diagnosed and corrected. The hung start in Table 7-2, Selected Scenario Event Set Index With Phases of Flight and Proficiency Objectives, is an example of a simple problem. It is effectively addressed within a single phase of flight. Overuse of simple problems or events detracts from LOS realism.
Routine prestart problems, followed by a start problem, followed by a taxi problem, intrude on the crew’s perception that the LOS is an actual flight. However, one or two of these events can be useful for setting a proper environment to facilitate a CRM LOS when the objectives are stated properly within the event set framework.
f. Complex Events. Complex events have ongoing consequences that must be dealt with during the flight and cannot be solved by simply selecting and executing an abnormal checklist.
Event-set-based scenarios require the coordinated actions of all crewmembers for successful completion. The compressor stall in Table 7-2 is an example of a complex problem. It is never actually solved, but instead it is managed over multiple phases of flight. Complex event set problems tend to be ambiguous, with no simple corrective checklist solution. The properly designed event set does not necessarily have a single solution. Rather, it may have a number of possible and reasonable solutions. Thus, the well-designed event promotes the management of a complex situation.
g. The Development Process. Table 7-1, Line Operational Simulation Design Methodology, describes an LOS development process. The overall purpose of the LOS development methodology is to build event sets that allow for the examination of the crew’s CRM and technical skills.
h. Summary. This design methodology provides a rigorous validation process to assure training and evaluation of all critical technical and CRM tasks identified by a training program.
In addition, it allows for adaptation based on the operational environment of a particular organization. The remainder of this chapter analyzes each component of the design methodology in detail.
Par 7-3 Page 36 TABLE 7-1. LINE-OPERATIONAL SIMULATION DESIGN METHODOLOGY
1) Identification of Primary Crew Resource Management (CRM)/Technical Training Objectives.
1.1 Identify the primary CRM categories (e.g., decision making, communication, workload management) and integrate these with the primary technical training objectives.
1.2 Identify the related skills for the CRM categories identified in 1.1.
1.3 Identify the primary technical training objectives.
2) Identification of Possible Incidents That Will Produce the Training Objectives.
2.1 Identify incidents through a search of the Aviation Safety Reporting System (ASRS) database, own-carrier incident reporting, and own-carrier flight safety programs, such as Aviation Safety Action Program (ASAP) and flight operations quality assurance (FOQA).
2.2 Develop a preliminary list of relevant incidents and events.
2.3 Refine the listing of incidents and events, and correlate with the CRM categories and observable behaviors.
3) Specification and Development of LOS Scenario Event Sets.
3.1 Specify LOS scenario objectives, related proficiency objectives, primary and secondary CRM categories, and observable crew behaviors for each scenario event set.
3.2 Translate incidents and situations into scenario event sets by identifying the event trigger, distracters, and supporting events, and specify the phase of flight.
3.3 Integrate the individual scenario event sets into the overall scenario.
3.4 Administer the LOS validation instrument to ensure event sets are specified and organized consistent with the CRM and technical training objectives or proficiency objectives.
4) Evaluation, Modification, and, if Appropriate, FAA Approval of the LOS Scenario.
4.1 Represent the LOS scenario showing the event sets, event trigger, ATC communications, and the related CRM categories (e.g., use the CRM category
Par 7-3 Page 37 profile and matrix methods to represent the frequency of CRM categories over the entire scenario).
4.2 Fly the LOS scenario using at least two different crews. Invite the Principal Operations Inspector (POI)/Aircrew Program Manager (APM) to participate.
Consider recording some of these sessions for use in developing facilitator training materials.
4.3 Administer the LOS validation instrument form to crews and facilitators instructors that fly the scenario.
4.4 Make required modifications to the revised LOS scenario.
4.5 Submit LOFT or Line Operational Evaluation (LOE) scenario to the POI for approval.
5) Instructor Training Implementation and Evaluation of LOS Scenarios.
5.1 Develop the final representation of LOS for facilitators, with the emphasis on event sets.
5.2 Develop the training plan and materials for recurrent training of facilitators and train the instructors/evaluators.
5.3 Implement the LOS scenario at the fleet level, and evaluate using actual facilitator and crew feedback.
7-4. IDENTIFICATION OF PRIMARY CRM TRAINING OBJECTIVES.
a. Identify Concepts and Definitions. Before an operator can develop a meaningful LOS, it must identify the CRM concepts and definitions that are meaningful within its own culture (refer to the current edition of Advisory Circular (AC) 120-51, Crew Resource Management Training). Several different organizations of CRM concepts can be used, but all of them follow a similar structure:
(1) High-level categories or elements, such as situational awareness and workload management.
(2) Supporting each category are knowledge and skills that should be trained. For example, in workload management, the knowledge would include how to prioritize tasks, while the skill would be the clear assignment of tasks that are understood by all crewmembers.
b. Measure Proficiency. To measure proficiency in the LOS, there should be a set of observable behaviors to look for. For example, in workload management, communicating task priorities is an observable behavior.
7-5. IDENTIFICATION OF PRIMARY TECHNICAL OBJECTIVES. Each carrier’s flight operations department develops technical training objectives. Identification of these objectives,
Par 7-5 Page 38 in the form of proficiency objectives, is a specific requirement of Advanced Qualification Program (AQP). A subset of these objectives can be selected as the technical objectives for each LOS. These objectives will serve an important role in selecting the event sets that will comprise the LOS scenario. Technical objectives might include:
• Origin, routing, and destination.
• Revised departure or arrival procedures.
• Alternate operation of flight management systems (FMS).
• Partial or full loss of integrated FMSs.
• Abnormal and emergency events.
• Adverse weather and environmental conditions.
7-6. DEVELOPING LOS SCENARIOS WITH AUTOMATION THEMES.
a. Automation. Automation is the replacement of a human function, either manual or cognitive, with a machine function. This definition applies to all levels of automation in all aircraft. Effective use of automation means using that level most appropriate to support the priorities of safety, economy, and stated flight operations policies of the individual air carrier.
b. Automation Proficiency. Flightcrew members must be proficient in operating their aircraft in all levels of automation. They must be knowledgeable in the selection of the appropriate degree of automation, and must have the skills needed to move from one level of automation to another.
c. Automation within LOS Scenarios. When developing LOS scenarios with an automation theme, the following items should be considered:
(1) The unique workload distribution between the pilot flying (PF) and pilot monitoring (PM).
(2) The effects of varying levels of automation on situational awareness and workload distribution.
(3) Flightcrew members’ proficiency at dealing with ATC communications, clearance, and weather changes in the automated flight deck.
(4) Company policy and guidelines on high technology procedures.
(5) The effects of lowering levels of automation with decreased levels of crew situational awareness.
(6) Flightcrew members’ proficiency at recognizing automation failures (e.g., autopilot or autothrottle) and, if necessary, taking action to manually control the aircraft.
d. Scenario Design. Scenario designs should be guided by the skills necessary for the individual flightcrew members, as well as the skills necessary for the fully integrated crew.
Scenarios should attempt to engage all crewmembers in CRM activities and should be based on specific training and performance objectives.
Par 7-7 Page 39
7-7. IDENTIFICATION OF POSSIBLE INCIDENTS THAT WILL PRODUCE THE TRAINING OBJECTIVES.
a. Internal Resources. Candidate incidents can be identified through a search of the ASRS database, company incident reports, or company ASAP reports and FOQA events. These incidents can be divided into broad categories of events and these categories then sampled to identify primary issues to be included in new LOS scenarios. Examples of primary issues include rerouting/amended clearance incidents, low fuel during excessive vectoring, and airborne conflicts attributed to flightcrew workload (e.g., delayed approaches, similar call signs, and autoflight incidents). It is common for AQP curriculums to base next year’s scenarios on last year’s ASAP findings. Actual events also provide a template for an event set or even an entire LOS. This prevents the scenario developers from having to begin with a blank sheet each year, and assures the realism of the event.
b. Other Resources. Some other excellent sources for candidate incidents are:
(1) Frequently misused or misunderstood sections of the flight manuals.
(2) Incident reports from other databases, including the International Civil Aviation Organization (ICAO) incident database.
(3) Maintenance difficulty areas identified on the line, during proficiency checks and during training.
(4) Poor performance areas identified on the line, during proficiency checks and during training.
c. Triggers. These sources focus primarily on operational abnormalities and emergencies.
However, CRM skills are required in all aspects of flight operations, including normal operations. By focusing on a specific set of event triggers and associated crew behaviors, a more useful breakdown of these behaviors is obtained. Each type of trigger, whether occurring during normal or abnormal operations, has its own unique CRM requirements:
(1) Normal Operations. CRM behaviors should appear during briefings, crew formation/team building, communications (e.g., inquiry and advocacy), contingency planning, and workload distribution.
(2) Abnormal Operations. Once operations become abnormal or excessively demanding, the required CRM skills will be altered. Some examples include:
(a) Detection of an Abnormal Event.
1. To detect abnormalities, the crew must maintain workload and situational awareness at acceptable levels.
2. Knowledge of checklists, systems, and procedures is required.
(b) Diagnosis and Assessment of an Abnormal Event.
Par 7-7 Page 40
1. Once detected, the abnormal event must be correctly diagnosed and appraised.
2. Appropriate information must be integrated.
3. Essential and nonessential information must be recognized, requiring ongoing situational awareness.
4. The assessment must be communicated to, and acknowledged by, other crewmembers. Challenges should be made when appropriate and all information should be shared.
7-8. EVENT SET SOLUTION VERSUS MANAGEMENT AND EVENT TRIGGER SELECTION.
a. Event Set Selection. Selection of event sets should take into account the types of problems they raise. A mix of simple and complex problems increases the benefits offered by an LOS.
(1) Simple Problems:
(a) Have no further consequences on the conduct of the flight once they have been diagnosed and corrected.
(b) If overused, will detract from realism. Use of one or two of these events can set a proper stage for CRM LOS, but including a number of these events, without logical connection or reason, detracts from the training. If the LOS training objectives are stated properly, they will help to preclude excessive use of nuisance events in the scenario.
(2) Complex Problems:
(a) Have ongoing consequences that must be dealt with in flight, but cannot be fixed.
(b) Add sufficient complexity to the scenario to require the coordinated action of all crewmembers for successful completion, but not to the extent that they induce complete crew failure such as a crash.
(c) Can be compounded by other events, such as weather or ATC-induced complications.
b. Event Set Impacts. The impact of an event set is also influenced by the extent to which the solution to the problem can be solved by means of established standard operating procedures (SOP). The combination of simple versus complex and proceduralized versus nonproceduralized problem characteristics produces three basic categories likely to be found in realistic event sets:
(1) SOP Event Trigger Solution.
Par 7-8 Page 41
(a) These event problems have specific rules or procedures for resolution.
(b) The book procedure for problem solution resolves or reverses the abnormal condition.
(c) After diagnosis, this situation requires no crew decision. The crew selects and uses the appropriate rule (e.g., manual gear extension).
(d) The nature of this type of problem is unlikely to require a high level of CRM skill, unless the event set is already particularly demanding or time compressed, requiring multitask prioritization. In this case, the events are interrelated in the same timeframe.
(2) SOP Event Trigger Management.
(a) This type of event often entails continuous monitoring or system compensation.
(b) The defining characteristic is that the corrective procedure does not solve the problem.
(c) In the case requiring continuous monitoring tasks, the crew will need to prioritize tasks and reduce the effects of distraction (e.g., monitoring a constant speed drive (CSD) outlet temperature in the caution zone).
(d) Problem management should require more inherent CRM skills than SOP problem solution.
(e) On a continuum of difficulty, these types of problems lie between simple and complex problems.
(3) Knowledge-Based Solution/Management.
(a) A book procedure or solution is not available to the crew.
(b) Crews are required to brainstorm a solution or management strategy.
(c) Knowledge-based solutions and management strategies require a decision-making process that often engages multiple crewmembers.
(d) An example of a knowledge-based solution is the selection of an alternate airport when weather or other conditions prohibit landing at the planned destination. There may be no SOP for alternate selection.
c. Summary. Awareness of the types of problems raised by each event set within a LOS scenario will help to ensure that scenario objectives are met and selected CRM skills are used.