Par 9-13 Page 103
9-13. LINE OPERATIONS AND PROCEDURALIZED CRM. Developing and teaching specific observable actions that would be required in the execution of specific activities at designated points in normal flight operations, as well as during abnormal or emergency conditions, can enhance the crew’s ability to communicate effectively, plan, and manage their workload, and solve problems during flight operations. A procedural approach may raise key aspects of CRM to the level of standard operating procedure (SOP), which increases CRM’s operational significance and provides crews with a standard form of CRM. CRM procedures may be embedded in a range of crew activities through the different phases of flight, reducing distractions to the pilot flying (PF) in both normal and abnormal situations. Also, providing structure to briefings with a checklist format can enhance the crew’s performance and improve the transfer of critical information.
SECTION 3. EVALUATING CRM
9-14. OVERVIEW. The training developed for AQP reflects the corporate philosophy as to how the job is to be performed, including the identification of observable behaviors that serve as the basis for evaluation. The use of a flight management philosophy during the development of training curricula and during the actual training supports an outcome-oriented assessment of skill use. Effective task activity performance is defined within the context of achieving predefined flight objectives. This approach supports the definition of objective standards that can serve as the basis for evaluation, and provides a consistent foundation on which to evaluate both CRM and technical skills.
9-15. OBSERVABLE BEHAVIORS. Observable behavior is a specific action employing a CRM skill in a given situation. Evaluation of individual pilot and crew CRM proficiency is possible through the inclusion of observable behaviors in the evaluation criteria that reflect performance on CRM-related task activities. These observable behaviors are defined for both the proficiency objectives and the events. Evaluation of CRM skills is possible if the event sets used in the evaluation process address these skills and incorporate appropriate observable behaviors in the performance standards.
Par 9-15 Page 104 FIGURE 9-1. SAMPLE CRM SKILLS Exercise Captain’s Authority or Responsibility Distribute workload and prioritize between primary and distracting duties.
Communicate plans and decisions to the crew.
Enforce standardization, policies, and procedure.
Set expectations for maintaining vigilance and avoiding complacency.
Respond to any safety-related concern raised by any crewmember.
Develop and enhance the aviation skill and knowledge of junior crewmembers.
Review operational irregularities and establish bottom lines.
Communicate intentions, “bottom lines,” and decisions to all crewmembers.
Fulfill First Officer or Flight Engineer Responsibility Cross-check and back the captain up. This requires maintaining vigilance and flying proficiency. It also includes effective monitoring of the situation.
Report to the captain any safety-related concern and request a plan or decision if none is articulated.
Support decisions articulated by the captain within the limits of safety, legality, and procedure.
Develop your proficiency and take the best from each captain you work with.
Maintain Situational Awareness Prepare, plan and maintain vigilance—be prepared for what you can reasonably expect.
Carry out actions or decisions based on priorities and crew workload established by the captain.
Identify systemic traps.
Be aware of the limits of human performance and the nature of human error.
Establish Effective Communications Conduct or contribute to briefings—keep your head in the game and work to get ahead of it.
Maintain a communications “loop”—acknowledge commands, statements, and questions of crewmembers.
Use resources appropriately to make informed decisions.
Resolve disagreements or differences in expectations—ensure that all crewmembers are working from the “same page.”
Manage errors appropriately to mitigate consequences.
Continuously review the appropriateness of decisions made and actions taken.
Debrief critical flight events—take the opportunity to learn from unusual events by reviewing the expectations and actions of all cockpit crewmembers at the end of the flight.
Par 9-15 Page 105 FIGURE 9-1. SAMPLE CRM SKILLS (CONTINUED) Develop and Maintain Teamwork Establish appropriate duties and responsibilities by crew position.
Back each other up through effective cross-check and acknowledgment.
Demonstrate motivation appropriate to the situation—transition between casual conversation and focused flight communication based on the need to prepare and execute your flight. This entire range is appropriate at different points in flight.
Protect crewmembers from the consequences of work overload.
Effectively coordinate with other groups: gate agents, dispatch, ground crew.
Apply judgment in use of automated systems and modes.
Operate the airplane using different levels of automation as appropriate to the situation.
Verify that automation is doing what you expect and act to control it when it does not.
Intervene to control autoflight.
When using automation, back each other up (verify settings, state intentions, establish roles).
Par 9-15 Page 106 FIGURE 9-2. SAMPLE EVENT SET WORKSHEET A340 EVENT SET NUMBER 101 WORKSHEET (FROM A340 R L 03-02 LEG 1) OVERVIEW. Low Visibility takeoff and climb with a reroute and a TCAS event.
Phase of Flight: Takeoff through Climb
SUCCESS CRITERIA TPOs and SPOs Conditions Technical Skills and Observables CRM Skills and Observables Trigger:
Departure weather, 200 overcast, RVR 1500.
Distracters:
TCAS RA, shortly after takeoff.
Supporting Events:
Reroute and climb restriction Difficulty Equivalency Rating:
Low Visibility takeoff - IMC – 4 FMS – 1 TCAS – 3 Total – 8 Low Visibility takeoff operations (2.1) Proper cleanup profile (2.1.4) Perform TCAS RA avoidance
(9.1.28) Perform climb operations
(3.1) Take Off Thrust – Normal IMC weather Proficient in use of FMS and Autopilot Flight Director System
(9.1.11) (9.1.13) Accomplishes takeoff/climb procedures IAW SOP (2.1.1) (2.1.2)
(2.1.3) (2.1.4)
(3.1.1) (3.1.2) Appropriate response to TCAS Alert (9.1.28) Crew coordinates for airspeed and altitude changes. (SA 3.4) Crew verbalizes and acknowledges changes in the altitude selector window. (AT 6.4) PF coordinates with PM in the use of automation. (AT 6.6)
Appendix 1 APPENDIX 1.
DEFINITIONS APPLICANT. A Title 14 of the Code of Federal Regulations (14 CFR) part 119 certificate holder that is required to have a training program under 14 CFR part 121, § 121.401, or part 135, § 135.341. A part 142 training center that applies to conduct training and evaluation for an eligible certificate holder under an Advanced Qualification Program (AQP).
ATTITUDE. A persisting internal mental state that influences an individual’s choice of personal action toward some object, person, or event.
CERTIFICATE HOLDER. Holder of an Air Carrier Certificate or Operating Certificate and operations specifications that authorize 14 CFR part 121 or 135 operations.
COGNITIVE SKILLS. Those intellectual skills that are prerequisite to the performance of a task, subtask, element, or sub-element. The three primary categories of cognitive skill are discrimination, concept learning, and rule using.
CONDITION. One of the three primary components of a proficiency objective: performance, condition, and standard. The conditions describe the range of circumstances under which student performance will be measured and evaluated. Conditions may include the natural environment (ceiling, visibility, wind, turbulence, etc.), the operational environment (Navigational Aid (NAVAID) inoperable, conflicting air traffic, gate change, passengers not seated, etc.), or operational contingencies (abnormal situations and emergencies).
COURSEWARE. All instructional material a learner requires to complete a curriculum, in whatever media required, including manuals, visual aids, lesson plans, flight event descriptions, computer software programs, audiovisual programs, workbooks, handouts, etc.
CREW RESOURCE MANAGEMENT (CRM). The effective use of all available resources human resources, hardware, and information to achieve safe and efficient flight. For additional information, refer to the current edition of AC 120-51, Crew Resource Management Training.
CRITICALITY. A terminal proficiency or supporting objective for which the substandard task performance would adversely affect safety. The relative need for awareness, care, exactness, accuracy, or correctness during task performance. Critical tasks must be accomplished more frequently in training and evaluation than non-critical tasks.
CURRENCY. A terminal proficiency or supporting objective for which individuals and/or crews can maintain proficiency by repeated performance of the item in normal line operations.
For pilots, most currency items may be validated during line checks, while most non-currency items must be demonstrated during training, validation, or evaluation events in a simulator or flight training device (FTD).
Appendix 1 CURRICULUM. A portion of an AQP that covers one of three program areas: indoctrination, qualification, and continuing qualification. Qualification and continuing qualification program areas may include upgrade, transition, conversion, and requalification curriculums. The indoctrination program area addresses activities that are general in nature and are not aircraft or duty position specific. The qualification and continuing qualification programs address the required training and qualification activities for each specific make, model, and series aircraft (or variant) and for a specific duty position.
CURRICULUM DESIGN. The activities involved in organizing, clustering, sequencing and otherwise structuring the elements of instruction (objectives, lessons, evaluations, etc.) into an orderly flow of learning experiences to facilitate student performance.
CURRICULUM SEGMENT. An integral part of a curriculum, which can be separately evaluated and individually approved, but by itself does not qualify a person for a duty position.
(e.g., ground training segment, flight training segment, evaluation segment). The first level of curriculum detail (segment, module, lesson, and lesson element).
DE-IDENTIFIED DATA. Data that cannot be linked with a named individual.
DUTY. All the actions (tasks, subtasks, etc.) required by one’s position or occupation.
DUTY POSITION. The operating position of a crewmember, or other person. Includes seat dependent tasks. For operations under parts 121 and 135, duty positions include pilot in command (PIC), second in command (SIC), Flight Engineer (FE), flight navigator, instructor and evaluator, aircraft dispatcher, flight attendant (F/A), or other ground operations personnel such as those dealing in security or hazardous materials.
ELEMENT. A component of training analysis or design. In the case of task analysis, the element may be used as a level of analysis: phase of flight, task, subtask, element, sub-element, etc. In the case of curriculum design, the element may be used as a level of curriculum organization: curriculum, segment, module, lesson, lesson element, etc.
ENABLING OBJECTIVE (EO). A lower level learning objective that helps students master a higher level objective, such as a supporting or terminal level objective. The knowledge and skill prerequisites of maneuvers and procedures are usually trained as enabling proficiency objectives.
EVALUATION. Careful appraisal of an individual’s performance by an evaluator to ascertain whether the standards required for a specified level of proficiency have been demonstrated.
EVALUATION OF PROFICIENCY. In AQP, this is a Line Operational Evaluation (LOE) or an equivalent evaluation acceptable to the Administrator.
EVALUATION PERIOD. A period within the continuing qualification cycle during which each person must receive at least one training session and a proficiency evaluation. The initial continuing qualification cycle for any curriculum will be 24 months (plus or minus the 1 month grace period), and will contain two 12-month (plus or minus the 1 grace month) evaluation periods.
Appendix 1 EVALUATOR. A person who assesses the performance of crewmembers, instructors, other evaluators, aircraft dispatchers, or other operations personnel. An evaluator must have satisfactorily completed the certificate holder’s AQP evaluator training. All evaluators must be approved by the Administrator.
EVENT SET. A relatively independent segment of a scenario made up of several events, including an event trigger, possible distracters, and supporting events.
FLIGHT TRAINING. Training given in the aircraft, flight simulator, FTD, or other cockpit environment. See ground training.
FLIGHT TRAINING EQUIPMENT. Aircraft, flight training devices, or flight simulators that are used for any of the following purposes: (1) Required evaluation of individual or crew proficiency; (2) Training activities that determine if an individual is ready for an evaluation;
(3) Activities used to meet recency of experience requirements; and (4) Line-Operational Simulations (LOS).
FORMATIVE EVALUATION. Process of reviewing courseware for technical accuracy, instructional soundness, and suitability for use by instructor, evaluator, and student. Dry run of the total curriculum with a small group of students to test the effectiveness and efficiency of the training (e.g., small group tryout).
FREQUENCY. Number of occurrences of a task/subtask in a specific period of duty (one flight, one trip, 1 month, 1 year, etc.) How often a task/subtask is performed. Frequency may be used to determine currency (see currency) by comparing the frequency with which activities occur on the line, with the frequency required to maintain proficiency without additional training.
GROUND TRAINING. Aviation/aircraft specific training provided in a classroom, learning center, lecture hall or other traditional educational setting that occurs outside the cockpit environment.
INSTRUCTIONAL ANALYSIS. A process conducted during the design of instruction to identify the presentational components, or learning events, necessary for the student to master the complete range of skills, knowledge, attitudes, abilities, and CRM factors required for proficient performance.
INSTRUCTIONAL SYSTEMS DEVELOPMENT. A systematic methodology for deriving and maintaining qualification standards and associated curriculum content based on a documented analysis of the job tasks, skills, and knowledge required for job proficiency.
JOB. A job is the summation of functions, identified as tasks and subtasks, performed by an individual at work.
KNOWLEDGE. Specific information required to enable a student to develop the skills and attitudes to effectively recall facts, identify concepts, apply rules or principles, solve problems, and think creatively. Because knowledge is covert, students must be assigned overt activities to demonstrate their knowledge base.
Appendix 1 LESSON. A meaningful division of learning consistent with the method of study, learning, or testing of performance (proficiency) objectives. The third level of curriculum definition (segment, module, lesson, lesson element). Lessons usually contain objectives, training events, student materials, instructor materials, and an evaluation scheme or form.
LESSON ELEMENT. A subgroup of activities within a lesson. It is the fourth level of curriculum detail (segment, module, lesson, and lesson element).
LINE OPERATIONAL EVALUATION (LOE). The LOE is the primary proficiency evaluation. This evaluation addresses the individual’s ability to demonstrate technical and CRM skills appropriate to fulfilling job requirements in a full mission scenario environment. The intent of an LOE is to evaluate and verify that an individual’s job knowledge, technical skills, and CRM skills are commensurate with AQP qualification standards. The LOE is conducted in a simulation device approved for its intended use in the AQP. Under extenuating circumstances, the AQP proficiency evaluation may be accomplished in an aircraft, subject to FAA approval.
LINE OPERATIONAL SIMULATION (LOS). A simulator or FTD training session conducted in a “line environment” setting. LOS includes Line-Oriented Flight Training (LOFT), Line Operational Evaluation (LOE), Special Purpose Operational Training (SPOT). For additional information, refer to the current edition of AC 120-35, Flightcrew Member Line Operational Simulations: Line-Oriented Flight Training, Special Purpose Operational Training, Line Operational Evaluation.
LINE-ORIENTED FLIGHT TRAINING (LOFT). A LOS flight scenario designed for training purposes to provide practice in the integration of technical and CRM skills. LOFT is conducted using a complete cockpit flightcrew to the maximum extent feasible and is accomplished in an FAA-certified simulation device. For additional information, refer to AC 120-35.
MEDIA. Physical means for providing the instructional content and experience to the student.
Includes entire set of instructional presentation materials (e.g., workbook, videotape, overheads, computer-based training (CBT), mock-ups, FTDs, simulators, etc.).
MODULE. A group of subject matter under a specific curriculum segment. Second of four curriculum levels of detail (segment, module, lesson, and element). Often corresponds to a day of training or a device event, such as FTD #3 or flight simulator (SIM) #6).
MOTOR SKILL. Physical actions required to perform a specific task (subtask or element).
Students have acquired a motor skill not when they can simply perform a prescribed procedure, but when their movements are smooth, regular, and precisely timed. Those hands-on skills that are prerequisite to the performance of a task, subtask, element, or sub-element.
OBSERVABLE BEHAVIOR. A behavior whose occurrence during the performance of an event is an indicator that the crew is handling the event properly. Observable behaviors are one part of the performance standards identified for each event.
PLANNED HOURS. Planned hours represent the estimated amount of time (as specified in a curriculum outline) that it takes an average student to complete a segment of instruction (to
Appendix 1 include all instruction, demonstration, practice, and evaluation, as appropriate, to reach proficiency). Planned hours replace the programmed hours associated with traditional programs.
The sum of planned hours will not be used by the FAA as a basis for program approval, review, or compliance assessment.
PROTEUS. A software program containing formats, functionality, procedures, and sample documents designed to guide and automate portions of the development of AQPs.
RANDOM LINE CHECK. A no-notice (not scheduled and with no prior notification) proficiency assessment of an entire flightcrew’s operational proficiency conducted by an evaluator during part 121 or 135 revenue flight operations or during operationally oriented flight, such as ferry flights or proving flights.
SKILL. An ability to perform an activity or action. Often divided into motor/hands-on and cognitive categories.
SPECIAL PURPOSE OPERATIONAL TRAINING (SPOT). A portion of an LOS training scenario consisting of flight tasks selected from any phase or phases of flight to provide practice in the integration of technical and CRM skills appropriate to the selected flight tasks. SPOT is conducted using a complete cockpit flightcrew to the maximum extent feasible and is accomplished in a simulation device. For additional information, refer to AC 120-35, as amended.
SUBTASK. Specific separate step or activity required in the accomplishment of a task. May also refer to categories of a task (e.g., non-precision approach – VOR, NDB, LOC, etc.)
SUPPORTING PROFICIENCY OBJECTIVE (SPO). A proficiency objective created at the subtask level.
TASK. A task is a unit of work within a function having an identifiable beginning and ending point, which results in a measurable product. An example of a task applicable to AQP: perform a normal takeoff.
TECHNICAL SKILLS. Within an AQP, technical skills refer to those maneuvers, procedures and other behaviors that have a high psychomotor component, while CRM skills refer to those communication, decision-making and workload management behaviors that have a high cognitive component.
TERMINAL PROFICIENCY OBJECTIVE (TPO). A proficiency objective created at the task level.
TPO/SPO HIERARCHY. The hierarchy of all TPOs and SPOs organized by phase of flight in the Proteus database.
TRAINING SESSION. A contiguously scheduled period of time devoted to training activities at a facility acceptable to the Administrator for that purpose.
Appendix 1 TRAINING TO PROFICIENCY. Training to a performance level that meets or exceeds a qualification standard. This concept must include enough repetition and practice to ensure that each individual can perform at the qualification standard level over the entire evaluation period or continuing qualification cycle.
TRIGGERING CONDITIONS. The conditions whose occurrence defines the beginning of an event.
VALIDATION. Determination that required/desired results were produced. In training systems, the methods and procedures for development, implementation and maintenance as well as performance objectives and results will be validated.
VARIANT. A specifically configured aircraft for which the Administrator has identified training and qualification requirements that are significantly different from those applicable to other aircraft of the same make, model, and series.
Appendix 2 APPENDIX 2.
GENERIC AIRLINES ADVANCED QUALIFICATION PROGRAM
APPLICATION Date, 2005 Subject: Initial Application for Advanced Qualification Program Dear (insert name of the manager of the appropriate Safety Standards office):
The purpose of this letter is to inform you that Airline intends to develop, implement, and operate an Advanced Qualification Program (AQP) curriculum, beginning with our aircraft fleet. We have examined and understand the Advanced Qualification Program requirements, listed in 14 CFR part 121 subpart Y and FAA Advisory Circular 120-54A, Advanced Qualification Program. This letter serves as Airline’s formal application to engage in the five-phase development process of an AQP, which is an alternative regulatory method of training, evaluating and qualifying our flight (crewmembers) (dispatchers).
Our airline will develop its AQP training program in accordance with the performance-based methodology that is described in Advisory Circular 120-54A. The initial development process will include at least the following AQP documents: Initial Job Task Listing; Qualification Standards; Instructional Systems Development Methodology;
Curriculum Outlines, and an Implementation and Operations Plan. All AQP documentation will be submitted to the FAA Extended Review Team for its review and joint approval. It will also show how our proposed AQP curriculum provides an equivalent or enhanced level of safety for each traditional regulation requirement that is replaced by our AQP.
We see our voluntary commitment to AQP as an important safety goal. To achieve that goal, and to facilitate a safe and timely transition to AQP, Airline will commit the appropriate personnel and resources to complete the task, and to continuously maintain the program in the future. Accordingly, in support of this application, we submit the following attachments:
• AQP Organization and Database Management Plan, which describes the multiple resources that our airline plans to employ for its AQP development. This attachment identifies the airline’s leadership person for AQP implementation, and delineates the staff of key subject matter experts who will be tasked with Phase II document and curriculum development, including electronic database management and other computer- related issues. The detailed AQP organization plan reflects our commitment to the goal of achieving each requirement that is unique to a successful AQP implementation.
• A summary of demographic data of our crews, who will be trained under an AQP curriculum. This data includes the current ground and flight instructors and evaluators who are expected to continue these functions under AQP.
• An operating environment description that articulates our airline’s geographic areas of operation, general environmental factors, plus other operational factors which may be critical to development of meaningful proficiency objectives and Line—Operational Simulations (LOS).
• A description of our training equipment, its location, and the organization responsible for its security and maintenance. Also included: the FAA identification number assigned by the National Simulator Program Manager and the level of qualification identifying flight simulators and/or flight training devices.
• A description of our training facilities including the location, type of facility, classrooms, training aids, courseware, and other features that contributes to creating and maintaining a positive learning environment.
Appendix 2 The attached Master AQP Transition Schedule (MATS) describes our timeline for completion of the transition process from our current training program to an AQP. The MATS will be maintained as a current document.
We do/do not intend to apply for a Single Visit Exemption (SVE) to facilitate transition to AQP. The point of contact for our AQP development will be , and he/she may be contacted at , or through email .
Sincerely, VP of Fight Operations or Chief Operations Officer, Generic Airline Attachments: (1) Organization and Database Management Plan; (2) Airline Operational Environment;
(3) Demographic Data; (4) Training Equipment Description and Location; (5) Facilities Description; and (6) MATS.