AC 120-35D

U.S. Department of Transportation Federal Aviation Administration Advisory Circular Subject: Flightcrew Member Line Operational Simulations: Line-Oriented Flight Training, Special Purpose Operational Training, Line Operational Evaluation Date: 3/13/15 Initiated by: AFS-200 AC No: 120-35D Change:

This advisory circular (AC) presents guidelines for the design and implementation of Line Operational Simulations (LOS), including Line-Oriented Flight Training (LOFT), Special Purpose Operational Training (SPOT), and Line Operational Evaluation (LOE) for flightcrew members. This document does not interpret the regulations; interpretations are issued only under established agency guidelines. As operators develop LOSs, they should develop an interdependent relationship between their human factors, Crew Resource Management (CRM), flight operations, and safety initiatives, because they are linked to a common safety goal.

This AC describes a means by which LOS scenarios are developed, scripted, tested, evaluated, and (in the case of LOFT and LOE) approved by the Administrator for use in an operator’s training program. The methodology set forth also achieves the Federal Aviation Administration’s (FAA) mandate to ensure that each certificate holder provides the highest level of safety in the public interest, while meeting the agency’s responsibility to reduce or eliminate the possibility or recurrence of accidents in air transportation.

ORIGINAL SIGNED by /s/ John Barbagallo Deputy Director, Flight Standards Service

CONTENTS Paragraph Page CHAPTER 1. PURPOSE AND BACKGROUND INFORMATION

1-1. Purpose1

1-2. Cancellation1

1-3. Related Regulations1

1-4. Definitions1

1-5. Related FAA Guidance4

1-6. Related References4

1-7. Background4

1-8. Requirements and Concepts5

1-9. Summary6

CHAPTER 2. BASIC ELEMENTS OF LINE OPERATIONAL SIMULATION

2-1. General7

2-2. LOS Philosophy7

2-3. Crew Composition and Participation7

2-4. Real-World Situations7

2-5. No-Jeopardy Training8

2-6. Phases of LOS8

2-7. Training Hours, LOE, Recurrent LOFT, and Qualification LOFT9

2-8. LOS Scenarios9

2-9. Approval of Scenarios10

2-10. Pre-LOS Briefing11

2-11. CRM Component of the Pre-LOS Briefing12

2-12. LOS CRM Briefing and Crew Orientation14

2-13. Preflight Activities14

2-14. Crewmember Responsibilities15

2-15. Debriefing the LOS15

2-16. LOE17

2-17. Use of Audiovisual Equipment and Animation Software17

2-18. Additional Training/LOFT and SPOT Completion18

2-19. Basic Elements of LOS Summarized18

2-20. FAA Philosophy18

2-21. Safety Management Systems (SMS)19

2-22. CRM/Threat and Error Management (TEM)/Risk and Resource Management

(RRM)19

2-23. CRM Practical Applications20

2-24. CRM (Human Factors) Behavioral Markers20

CHAPTER 3. TYPES OF LINE-ORIENTED FLIGHT TRAINING

3-1. General23

3-2. Recurrent LOFT23

Paragraph Page

3-3. Guidelines For Recurrent LOFT23

3-4. Qualification LOFT24

CHAPTER 4. SPECIAL PURPOSE OPERATIONAL TRAINING

4-1. General27

4-2. Guidelines for SPOT27

4-3. Specific Scenarios for Operators of All-Cargo Aircraft or Combination

Cargo/Passenger (Combi) Aircraft27

CHAPTER 5. LINE OPERATIONAL EVALUATION

5-1. General29

5-2. Elements Resembling Line-Oriented Flight Training (LOFT)29

5-3. Evaluation29

5-4. Evaluator29

5-5. LOE Media29

CHAPTER 6. THE ROLE OF LINE OPERATIONAL SIMULATION FACILITATORS

6-1. Minimum Qualifications31

6-2. Facilitator Responsibilities31

CHAPTER 7. LINE OPERATIONAL SIMULATION SCENARIO DESIGN

7-1. Scenario Design Methodologies33

7-2. Philosophy of LOS Design and Conduct33

7-3. Scenario Design Process34

7-4. Identification of Primary CRM Training Objectives37

7-5. Identification of Primary Technical Objectives38

7-6. Developing LOS Scenarios With Automation Themes38

7-7. Identification of Possible Incidents That Will Produce the Training Objectives39

7-8. Event Set Solution Versus Management and Event Trigger Selection40

7-9. The Event Set Matrix43

7-10. Level of Difficulty43

7-11. Integration And Validation of CRM Skills to Event Sets43

7-12. Scripting and Validating the LOS Scenario48

7-13. Scenario Flexibility50

7-14. Grading Event Sets/Grading Phases of Flight50

LIST OF TABLES

Table 3-1. Recurrent LOFT Substitution Table24

Table 3-2. Qualification LOFT Substitution Table24

Table 7-1. Line Operational Simulation Design Methodology36

Table 7-2. Selected Scenario Event Set Index with Phase of Flight and Proficiency

Objectives42

Table 7-3. Selected Scenario Event Set Index with Phase of Flight and Crew Resource

Management Behaviors45

Table 7-4. The Event Set Matrix46

Table 7-5. The Event Set Assessment/Grade Sheet47

Par 1-1 Page 1 CHAPTER 1. PURPOSE AND BACKGROUND INFORMATION

1-1. PURPOSE. This advisory circular (AC) presents guidelines for the design and implementation of Line Operational Simulations (LOS), including Line-Oriented Flight Training (LOFT), Special Purpose Operational Training (SPOT), and Line Operational Evaluation (LOE) for flightcrew members. This document does not interpret the regulations; interpretations are issued only under established agency guidelines. As operators develop LOSs, they should develop an interdependent relationship between their human factors, Crew Resource Management (CRM), flight operations, and safety initiatives, because they are linked to a common safety goal. This AC describes a means by which LOS scenarios are developed, scripted, tested, evaluated and (in the case of LOFT and LOE) approved by the Administrator for use in an operator’s training program. The methodology set forth also achieves the FAA’s mandate to ensure that each certificate holder provides the highest level of safety in the public interest, while meeting the agency’s responsibility to reduce or eliminate the possibility or recurrence of accidents in air transportation.

1-2. CANCELLATION. AC 120-35C, Line Operational Simulations: Line Oriented Flight Training, Special Purpose Operational Training, Line Operational Evaluation, dated September 27, 2004, is canceled.

1-3. RELATED REGULATIONS. The current editions of the following Title 14 of the Code of Federal Regulations (14 CFR) parts can be located at: http://www.ecfr.gov.

• Part 60, Flight Simulation Training Device Initial and Continuing Qualification and Use.

• Part 121, Operating Requirements: Domestic, Flag, and Supplemental Operations;

Subpart N, Training Program; Subpart O, Crewmember Qualifications; Subpart Y, Advanced Qualification Program; Appendix E, Flight Training Requirements;

Appendix F, Proficiency Check Requirements; Appendix H, Advanced Simulation.

• Part 135, Operating Requirements: Commuter and On Demand Operations and Rules Governing Persons on Board Such Aircraft; Subpart G, Crewmember Testing Requirements; Subpart H, Training.

• Part 142, Training Centers.

1-4. DEFINITIONS. The following terms are used throughout this AC and are defined as follows:

a. Advanced Qualification Program (AQP). AQP provides an alternate method of qualifying and certifying, if required, pilots, Flight Engineers (FE), flight attendants (F/A), aircraft dispatchers, instructors, evaluators, and other operations personnel subject to the training and evaluation requirements of parts 121 and 135. AQP is a systematic methodology for developing the content of training programs. AQP incorporates data-driven quality control (QC) processes for validating and maintaining the effectiveness of curriculum content. AQP encourages innovation in the methods and technology that are used during instruction and evaluation, and efficient management of training systems.

Par 1-4 Page 2

b. Aviation Safety Action Program (ASAP). ASAP is a voluntary safety reporting program that serves to encourage employees of certificate holders or other operators to voluntarily report safety information that may be critical to identifying potential precursors to accidents. The FAA has determined that identifying these precursors is essential to further reducing the already low accident rate. Under an ASAP, safety issues are resolved through corrective action rather than through enforcement or disciplinary action.

c. Aviation Safety Reporting System (ASRS). ASRS is a National Aeronautics and Space Administration (NASA)-administered program that provides for the receipt, analysis, and deidentification of aviation safety reports; in addition, periodic reports of findings obtained through the reporting program are published and distributed to the public, the aviation community, and the FAA.

d. Behavioral Markers. Behavioral markers are very specific, observable, and measurable nontechnical practices that reflect the use of CRM knowledge and skill sets and contribute to safe flightcrew member performance. Operators are encouraged to develop their own classifications of behavioral markers that will function effectively within their unique safety culture.

e. Conditions. Existing circumstances that affect flightcrew performance. The circumstances may be external to the aircraft (e.g., weather, runway conditions, or airport area) or internal to the aircraft (e.g., equipment malfunctions or Weight and Balance (W&B) issues).

f. Crew Resource Management (CRM). The effective use of all available resources

(e.g., human resources, hardware, and information) to achieve safe and efficient flight.

g. Evaluator. A person who assesses or judges the performance of crewmembers, instructors, other evaluators, aircraft dispatchers, and other operations personnel.

h. Event. An integral part of training or evaluation that is task oriented and requires the use of specific procedures. In an AQP, an event is a specific instance of a training or evaluation objective (with its range of conditions, behavior, and standards) in the context of a curriculum.

Once the specific conditions for presenting an objective are selected, that objective becomes an event.

i. Event Set. A relatively independent segment of a scenario made up of several events, including an event trigger, possible distracters, and supporting events.

j. Flight Operational Quality Assurance (FOQA). A program to improve flight safety by providing more information about, and greater insight into, the total flight operations environment through routine recording and analysis of digital flight data generated during flight operations. Analysis of FOQA data can reveal situations that require improved operating, training, and maintenance procedures, practices, equipment, and infrastructure.

k. Flight Simulation Training Device (FSTD). A full flight simulator (FFS) or a flight training device (FTD).

DATE “DRAFT” AC 120-35D Par Page 1

l. Line Familiar. Describes a flightcrew member who is familiar with current line operations. This familiarity may be acquired and maintained either by conducting line operations or by completing an approved line observation program.

m. Line Operational Evaluation (LOE). LOE means a simulated line environment, the scenario content of which is designed to test integrating technical and CRM skills.

n. Line Operational Simulation (LOS). LOS is a training or evaluation session conducted in a simulated “line environment” setting. LOS includes LOFT, SPOT, and LOE.

Instruction and training is based on learning objectives, behavioral observation, and assessment of performance progress and instructor or check pilot/check FE debriefing or critique (feedback).

The training objectives under AQP are proficiency objectives that include both technical and CRM issues identified by a task analysis.

o. Line Operations Safety Audits (LOSA). LOSA is a formal flight deck auditing process that requires expert and highly trained observers conducting observations from the forward observer’s seat during regularly scheduled flights to collect safety-related data on environmental conditions, operational complexity, and flightcrew performance. Confidential data collected in a nonjeopardy environment is used to enhance safety at the operator.

p. Line-Oriented Flight Training (LOFT). Training conducted in an FSTD with a complete flightcrew using representative flight segments that contain procedures that may be expected in line operations. The LOFT includes real-time scenarios that address normal, non-normal, abnormal, or emergency procedures and provides training in CRM. There are two types of LOFT:

(1) Qualification LOFT. Qualification LOFT is conducted to facilitate the transition from a structured flight training environment to line operations. Qualification LOFT is required by part 121 appendix H, and must be conducted in an FFS.

(2) Recurrent LOFT. Recurrent LOFT must be conducted in an FFS and may be used to meet recurrent flight training requirements in accordance with part 121, § 121.441.

q. LOS Facilitator. For LOFT or SPOT, an instructor who administers the training session. For an AQP LOE, a check pilot/check FE who administers the evaluation session.

r. Proficiency Objective. An objective containing the criteria for a required level of performance. A proficiency objective contains a range of conditions that may be encountered by the student, while an event contains the specific conditions selected for a particular training or testing opportunity.

s. Risk and Resource Management (RRM). Like Threat and Error Management (TEM), RRM provides a framework or strategy for employing the traditional CRM skill sets, such as communication, workload management, and decisionmaking, to flight operations. Unlike TEM, it provides a more complex series of strategies and toolsets and integrates more directly to a certificate holder’s Safety Management System (SMS).

Par 1-4 Page 4

t. Safety Management System (SMS). SMS is the formal, top-down business approach to managing safety risk. It includes a systematic approach to risk management, including the necessary organizational structures, accountabilities, policies and procedures.

u. Special Purpose Operational Training (SPOT). SPOT is an FSTD training session designed to address specific training objectives. Training objectives are based on technical and CRM requirements, and include specific training objectives to be critiqued and debriefed on both technical and CRM performance. SPOT may consist of full or partial flight segments, depending on the training objectives for the flight.

v. Task Familiar. Describes a flightcrew member who is familiar with and can satisfactorily accomplish the duties of a particular flightcrew member duty position, though not qualified for that duty position. For example, a second in command (SIC) candidate who performs the duties of the pilot in command (PIC) during FSTD training.

w. Threat and Error Management (TEM). Like RRM, TEM is a framework or strategy for employing the traditional CRM skill sets. TEM, like RRM, promotes safe operations through the continuous process of identifying, avoiding, mitigating, or managing “threats” (external errors, conditions, or situations), internal errors, or undesired safety states.

1-5. RELATED FAA GUIDANCE. The current editions of the following ACs can be located at: http://www.faa.gov/regulations_policies/advisory_circulars/.

• AC 120-51, Crew Resource Management Training.

• AC 120-54, Advanced Qualification Program.

• AC 120-66, Aviation Safety Action Program (ASAP).

• AC 120-82, Flight Operational Quality Assurance.

• AC 120-90, Line Operations Safety Audits.

• AC 120-92, Safety Management Systems for Aviation Service Providers.

• AC 120-109, Stall and Stick Pusher Training.

1-6. RELATED REFERENCES. Facilitating LOS Debriefings: A Training Manual, NASA Technical Memorandum 112192 DOT/FAA/AR-97/6 (http://humansystems.arc.nasa.gov/flightcognition/Publications/Final_Training_TM.pdf).

1-7. BACKGROUND.

a. LOFT. The use of gate-to-gate flight simulator scenarios, known as LOFT, began in the mid-1970s as a means to provide pilot training that is more representative of actual flight operations than maneuver-based training alone. LOFT was soon recognized as a highly effective means of developing and practicing CRM skills. Due to the role of CRM issues in accident causation, it has become evident that training curriculums must develop pilot proficiency in both technical and CRM skills. While LOFT is designed to include all phases of flight, scenario-based training may also include limited portions of flight designed to focus on specific operational training needs, known as SPOT. Air carriers with an approved AQP must also conduct evaluated LOFTs, known as LOE, for jeopardy grading purposes. These three methodologies, LOFT, SPOT, and LOE, are now grouped under the general heading of flightcrew member LOS.

DATE “DRAFT” AC 120-35D Par Page 1

b. CRM. The introductory CRM training that many flightcrew members have experienced is similar to the foundation of a building: it is an essential structural part, but by itself the foundation has limited operational use. If CRM training is to be operationally effective, it must be built into other training steps and activities in a systematic way. A structured LOS design process is employed to specify and integrate the required CRM and technical skills into LOS scenarios.

c. LOS. LOS is an environment that is structured to allow and encourage the application of technical and CRM concepts to a situation that enables conceptual knowledge to become working knowledge. Instead of being programmed with a solution, the students can manage the operational environment and process available information to learn its limits, properties, and operational relevance. LOS can be conducted in an FFS or FTD, depending on whether the LOS is for training or evaluation, and the requisite fidelity of the training/evaluation media.

d. Basis. Much of the information in this AC was taken from a working paper developed by Airlines for America (A4A) (formerly known as the Air Transport Association of America (ATA)) Training Committee, AQP Subcommittee, LOFT Design Focus Group. This AC provides a structured design process for LOS design and implementation and builds upon air carrier experience with developing and implementing scenarios to provide guidelines for LOS programs. The scientific literature on the event set methodology is available from the Air Carrier Training Systems and Voluntary Safety Programs Branch (AFS-280).

1-8. REQUIREMENTS AND CONCEPTS.

a. LOFT. LOFT is a useful training method because it gives flightcrew members the opportunity to practice line operations (e.g., maneuvers, operating skills, systems operations, and the operator’s procedures) with a full flightcrew in a realistic environment. Flightcrew members learn to handle a variety of real-time scenarios that include routine, abnormal, and emergency situations. They also learn and practice CRM by way of operator-developed behavioral markers that may include, but are not limited to, essential elements such as situational awareness, communication, decisionmaking, workload management, and automation management skills.

The overall objective of LOFT is to provide training that improves total flightcrew performance, and thereby preventing incidents and accidents during operational flying.

(1) Section 121.409. Section 121.409(b) delineates the requirements of recurrent LOFT, which may be used to meet recurrent flight training requirements in accordance with § 121.441. Section 121.409(b) requires a complete flightcrew to be used in recurrent LOFT, but does not provide detail on what constitutes a complete flightcrew. The guidance provided in this AC recognizes a complete flightcrew as one that is task familiar and line familiar.

(2) Part 121 Appendix H. Appendix H contains requirements for qualification LOFT for certificate holders who choose to provide flightcrew member training in advanced airplane simulators. This AC presents guidelines for implementing qualification LOFT as required under appendix H or as may be used within any other approved training program. This AC discusses how qualification LOFT is designed to help flightcrew members transition from a training environment to operational flying.

Par 1-8 Page 6

(3) AQP. AQP requires the use of LOS in both qualification and continuing qualification (recurrent) curriculums and encourages the use of SPOT and LOFT.

b. SPOT. New training concepts and training media have provided the opportunity to creatively tailor training sessions to address specific training objectives. Training objectives are based on technical and CRM requirements. SPOT may consist of full or partial flight segments, depending on the training objectives for the flight. This AC presents guidelines in conducting SPOT.

c. LOE. The LOE is the primary means of proficiency evaluation under an AQP. This evaluation addresses the individual’s ability to demonstrate technical and CRM skills appropriate to fulfilling job requirements in a full mission-scenario environment. The intent of an LOE is to evaluate and verify that an individual’s job knowledge, technical skills, and CRM skills are commensurate with AQP qualification standards. The LOE is conducted in an FSTD qualified and approved for its intended use in the AQP.

1-9. SUMMARY. This AC identifies four types of LOSs: (1) Recurrent LOFT (in reference to §§ 121.409, 121.427, 121.433, and 121.441 and part 121 appendix F); (2) Qualification LOFT (in reference to part 121 appendix H); (3) SPOT, which is line-oriented training that addresses specific training objectives; and (4) LOE, which is a line-oriented evaluation designed for flightcrew members qualifying using an AQP. It defines the terms used in describing LOS. It provides guidance for conducting LOFT, SPOT, and LOE. It defines the role of instructors and evaluators, and provides guidance for designing LOS scenarios.

Par 2-1 Page 7 CHAPTER 2. BASIC ELEMENTS OF LINE OPERATIONAL SIMULATION

2-1. GENERAL. Certain elements about Line Operational Simulation (LOS) must be understood to ensure that its primary objective, to provide realistic line-oriented training and evaluation, are met. These elements apply to both recurrent and qualification Line-Oriented Flight Training (LOFT), and are described in this chapter.

2-2. LOS PHILOSOPHY.

a. LOS. The overall objective of LOS is to improve total flightcrew performance by integrating Crew Resource Management (CRM) and technical skills. CRM skills include techniques that allow and encourage crewmembers and aircraft dispatchers to become better problem solvers and resource managers. The LOS context must be structured to enable CRM behaviors to emerge and the crew to become aware of them; that is, the scenario must last long enough for crew traits to become evident and should require CRM skills to be displayed in response to specific circumstances. Similarly, scenario construction should focus on the CRM and technical objectives integrated into a training program.

b. LOFT and SPOT. LOFT and SPOT are training events in which flightcrews can enhance their CRM and technical skills. Line Operational Evaluation (LOE) contains events in which CRM and technical skills are assessed. LOS learning and assessment should not be artificially stress free; crewmembers should maintain performance parameters applicable to their phase of training. This will require crews to handle failures and their consequences. If the LOS facilitator identifies flightcrew member performance deficiencies, additional training should be provided.

2-3. CREW COMPOSITION AND PARTICIPATION. LOSs should take place in a line operational environment with a complete flightcrew. A complete flightcrew concept allows crewmembers to use their full resources and creativity to support a complete learning experience.

During LOS, each flightcrew member performs both as an individual and as a member of a team, as is expected during line operations. Line crewmembers must be scheduled and paired together, as much as practical, in a standard crew configuration (e.g., line captain with line first officer).

Circumstances will occur where the initial composition of the schedule cannot be maintained.

Hiring requirements, high first-officer-to-captain ratios, illness, or failure of a crewmember to progress are all situations that would necessitate providing a seat substitute to complete the training. In all cases, the seat substitute must be task familiar with the flightcrew member duty position.

2-4. REAL-WORLD SITUATIONS. LOS should contain scenarios of real-world, line operational situations, which progress in real time. These scenarios should be representative of flight segments where an entire en route operation is completed. In cases of flights with flight segments involving repetitive events, the en route segments may be compressed. However, if the scenario is compressed, enough time should be allotted for crewmembers to resume or restart the scenario without confusion. In addition, the scenario compression method must be designed and executed to minimize degradation of scenario realism and enhance training or evaluation. It is common practice for scenarios to be based upon actual events reported through the Aviation Safety Action Program (ASAP), Aviation Safety Reporting System (ASRS), flight operational

Par 2-4 Page 8 quality assurance (FOQA), Line Operations Safety Audit (LOSA), and other event-reporting programs.

2-5. NO-JEOPARDY TRAINING. LOFT and SPOT are “no-jeopardy” training (i.e., not pass/fail). If the LOFT or SPOT instructor identifies flightcrew member performance deficiencies, additional training will be provided. This training should be in a form most appropriate to correct the specific performance deficiency, including additional LOFT. Before the crewmember may return to line operations, the performance deficiencies must be corrected, and the instructor must document the training as satisfactorily completed. The no-jeopardy concept allows crewmembers to use their full resources and creativity. At the end of a LOFT session and after debriefing, if the flightcrew member has demonstrated satisfactory performance, the instructor must certify as to the proficiency and knowledge of the crewmember.

This includes situations where the instructor provided additional training following the LOFT in order to correct any performance deficiencies. The certification must be made in accordance with part 121, § 121.401(c) or part 135, § 135.323(c), as applicable. However, if the flightcrew member was not able to demonstrate satisfactory performance, even after additional training following the LOFT, the instructor may not certify the flightcrew member’s proficiency and an additional LOFT session should be scheduled.

2-6. PHASES OF LOS. LOS scenarios should contain the following phases:

a. Briefing. Before the flight begins, the LOS facilitator should brief crewmembers on the LOS scenario, including the training objectives, and the role of the LOS facilitator (i.e., the LOS facilitator is considered not present, except as to perform other roles in support of the scenario, such as flight attendant (F/A), aircraft dispatcher, or air traffic control (ATC)). The role of the flightcrew should be discussed in the briefing (i.e., flightcrew members should perform their duties just as they would in line operations). Information about the environmental setting of the scenario, when appropriate, should also be discussed.

b. Preflight Planning Documents and Activities. Preflight planning documents

(e.g., weather reports and flight plans) should be prepared with the operator’s particular training objectives in mind. For example, the operator may choose to have crewmembers learn how to handle unfavorable weather conditions or how to correct improper fuel loads. Preflight activities include flight deck setup and computation of takeoff data.

c. Flight Segment. The flight segment includes taxi-out, takeoff, climb, cruise, descent, landing, and taxi-in, as appropriate. It should also include the time in which communication with ATC and other ground agencies takes place.

d. Debriefing. Debriefing should include feedback to crewmembers on their performance.

The FAA recommends use of the facilitated or reverse debrief, wherein the LOS facilitator first allows the students to debrief and provide their observations. Positive comments regarding crew performance should be emphasized in the debriefing, as well as crew performance that needs improvement. The debriefing involves LOS facilitator critiques of individual crewmembers and of the crew as a team. Also, it is important that crewmembers be given the opportunity to critique and analyze their own performance and review key points of the recording, if used. The National Aeronautics and Space Administration (NASA) Technical Memorandum 112192, Facilitating