CONTENTS Paragraph Page
CHAPTER 1. AQP OVERVIEW1
1-1. General1
1-2. AQP Objectives1
1-3. AQP Characteristics2
1-4. Equivalent Level of Safety and AQP Reqiurements2
1-5. Developmental Assistance and Support4
Figure 1-1. Other Resources6
CHAPTER 2. AQP DEVELOPMENT PHASES9
SECTION 1. INTRODUCTION9
2-1. Phases9
2-2. Instructional Systems Development (I SD)9
2-3. Training Systems10
Figure 2-1. AQP Phase Development Process10
SECTION 2. PHASE I: APPLICATION10
2-4. Joint Meeting10
2-5. The AQP Application11
2-6. Approval13
SECTION 3. PHASE II: CURRICULUM DEVELOPMENT13
2-7. Overview13
2-8. J TA14
Figure 2-2. Sample Pilot Job Task Listing16
2-9. Qualification Standards17
F
igure 2-3. AQP Continuing Qualification Critical/Currency Chart19
Figure 2-4. Rating Scale for Procedures Validation, First-Look, Maneuvers
Validation, LOE, or Line Check23
2-10. Instructional Systems Development Methodology Document25
Figure 2-5. Curriculum Development26
2-11. Curriculum Outline Document27
Figure 2-6. Performance Difference Rating Scale28
Figure 2-7. Sample AQP Curriculum Outline29
2-12. Implementation and Operations Plan (I&O Plan) Document30
2-13. Approval32
SECTION 4. PHASE III: SMALL GROUP TRYOUT (IMPLEMENTATION)32
2-14. Overview32
2-15. Approval33
SECTION 5. PHASE IV: INITIAL OPERATIONS33
2-16. Overview33
2-17. Phase IV Activities33
2-18. Revisions34
2-19. Approval34
SECTION 6. PHASE V: CONTINUING OPERATIONS34
2-20. Overview34
2-21. Data’s Impact on Continuing Operations34
2-22. Quality Assurance34
SECTION 7. DOCUMENT AND REPORTING REQUIREMENTS34
2-23. Overview34
Figure 2-8. Document Structures Chart37
CHAPTER 3. AQP CURRICULUMS AND CERTIFICATION39
SECTION 1. CURRICULUMS39
3-1. Background39
3-2. Indoctrination Curriculum39
3-3. Q ualification Curriculum39
Figure 3-1. Flightcrew Member AQP Training, Validation, and Evaluation43
Figure 3-2. AQP Validation/Evaluation Table44
Figure 3-3. Example of AQP Flightcrew Qualification Curriculum Footprint45
3-4. Continuing Qualification Curriculum46
Figure 3-4. Example of An AQP Four-Day Flightcrew Continuing Qualification
Curriculum Footprint49
Figure 3-5. Continuing Qualification Cycle50
3-5. Secondary Curriculums52
Figure 3-6. AQP Secondary Curriculums52
SECTION 2. AIRMAN CERTIFICATION54
3-6. G eneral54
3-7.
AQP Training and Evaluation for Certification54
3-8. Completion of the Qualificatioin Curriculum54
CHAPTER 4. AQP APPROVAL AND DOCUMENTATION55
SECTION 1. DOCUMENTATION REVIEW AND APPROVAL55
4-1. O veriew55
4-2. R eview Process55
4-3. Approva l Process55
Figure 4-1. Five Phases of AQP Review and Approval56
4-4. Phase Approvals56
4-5. Method of Granting Approval59
4-6. R evision Control Process60
SECTION 2. WITHDRAWAL OF APPROVAL61
4-7. G eneral61
4-8. I nitial or Provisional61
4-9. Final Approval61
4-10. Appeal Withdrawal of Final Approval61
CHAPTER 5. INSTRUCTORS AND EVALUATORS63
SECTION 1. INSTRUCTOR/EVALUATOR PROGRAMS63
5-1. G eneral63
5-2. I nstructor/Evaluator Curriculums63
5-3. Authorized Evaluation Personnel by AQP Event67
Figure 5-1. Authorized Evaluation Personnel by AQP Curriculum68
SECTION 2. CERTIFICATION PERSONNEL68
5-4. T raining68
5-5. A uthorized Certification Personnel69
SECTION 3. QUALITY ASSURANCE AND STANDARDIZATION69
5-6. Quality Assurance69
5-7. S tandardization69
CHAPTER 6. TRAINING CENTERS71
SECTION 1. INTRODUCTION71
6-1. Background71
6-2. Training Centers Offering an AQP71
SECTION 2. PROVISIONAL APPROVAL: APPLICATION AND DEVELOPMENT71
6-3. P rovisional AQP71
SECTION 3. REQUEST AND APPROVAL FOR USE OF TRAINING CENTERS72
6-4. R equest72
6-5. Development72
SECTION 4. TRAINING CENTER’S INSTRUCTORS AND EVALUATORS73
6-6. Q ualification73
6-7. I nstructor/Evaluator Curriculum Approval73
SECTION 5. DATA COLLECTION AND RECORDKEEPING73
6-8. D ata Collection73
6-
9. D ata Submission73
6-10. Recordkeeping73
CHAPTER 7. TRANSITION TO AQP75
SECTION 1. OVERVIEW75
7-1. G eneral75
7-2. Entering AQP from an Established Training Program75
7-3. Transitioning to AQP Using the SVE75
SECTION 2. THE PETITION FOR EXEMPTION81
7-4. G eneral81
Figure 7-1. Single Visit Exemption (SVE) Process81
7-5. T he Petition Process81
7-6. T he Petition82
7-7. G ranting of the Petition82
7-8. R enewal Requirements for the SVE83
SECTION 3. SVTP83
7-9. Overview83
7-10. Contents83
7-11. SVTP Proficiency Check83
7-12. Approval of the SVTP84
7-13. Revisions84
7-14. Implementation of the SVTP85
7-15. Data Submission and Reporting Requirements85
7-16. SVTP Review Meeting85
Figure 7-2. Sample of Proficiency Check Fixed and Variable Maneuvers86
CHAPTER 8. AQP DATA MANAGEMENT87
SECTION 1. INTRODUCTION87
8-1. Overview87
8-2. D efinitions87
SECTION 2. DATA MANAGEMENT88
8-3. Overview88
8-4. D ata Collection—SVTP and AQP88
8-5. D a ta Entry88
8-6. D ata Submission88
SECTION 3. DATA ANALYSIS AND REPORTS90
8-7. Overview90
Figure 8-1. Data Submission Requirements91
Figure 8-2. Performance Data Report Table (PDRT)93
Figure 8-3. Skill/Reason Table (SKLRSN)96
Figure 8-4. Training Objectives Report Table (TORT)97
CHAPTER 9. CREW RESOURCE MANAGEMENT99
SECTION 1. INTRODUCTION99
9-1. Overview99
SECTION 2. INTEGRATING CRM INTO AN AQP99
9-2. Scope of Integration99
9-3. CRM/AQP Task List100
9-4. CRM Knowledge and Skills100
9-5. CRM and Proficency Objectives100
Page ix (and x)
9-6. Training Events101
9-7. Event Sets101
9-8. Source101
9-9. C urriculum Design102
9-10. Indoctrination Curriculum102
9-11. Qualification Curriculum102
9-12. Continuing Qualification Curriculum102
9-13. Line Opeartions and Proceduralized CRM103
SECTION 3. EVALUATING CRM103
9-14. Overview103
9-15. Observable Behaiors103
Figure 9-1. Sample CRM Skills104
Figure 9-2. Sample Event Worksheet106
APPENDIX 1. DEFINITIONS (6 pages)1
APPENDIX 2. GENERIC AIRLINES ADVANCED QUALIFICATION PROGRAM APP
LICATION (2 pages)1
APPENDIX 3. REGULATORY COMPARISON CHART/PART OF THE QUALI
FICATIONS STANDARDS DOCUMENT (6 pages)1
APPENDIX 4. ADVANCED QUALIFICATION PROGRAM DOCUMENTATION CHECK
LIST AND REVIEW JOB AID (21 pages)1
APPENDIX 5. ADVANCED QUALIFICATION PROGRAM QUALIFICATION STAN
DARDS DOCUMENTATION (4 pages)1
Par 1-1 Page 1 CHAPTER 1. AQP OVERVIEW
1-1. GENERAL. The Advanced Qualification Program (AQP) integrates a number of training and evaluation features aimed at improving performance. AQPs are systematically developed, continuously maintained, and empirically validated proficiency-based training systems. They allow for the systematic analysis, design, development, implementation, evaluation, and maintenance of self-correcting training programs.
a. Phases of Development. There are five phases for developing, implementing, and maintaining an AQP. A description of each phase is located in Chapter 2. Each phase must be Federal Aviation Administration (FAA)-approved before the applicant may proceed to the next phase. Each phase consists of specific activities, including the documentation of those activities.
These documents must also be FAA-approved.
b. AQP Documentation. In addition to the supporting documents and manuals provided to the FAA in traditional training programs, there are six documents and an annual report requirement that is unique to AQP. The documents are instrumental in managing the AQP and must be maintained throughout the life of the program. The documents can be categorized in two groups, by their function and interrelationship: management and database. The documents, along with the annual report, comprise the Program Audit Database (PADB).
(1) Management Documents. These documents include the Application, Instructional Systems Development Methodology, and Implementation and Operations Plan. These documents are stand-alone in that a change in one will not necessitate a change in another.
(2) Database Documents. These documents include the Task Analysis, Qualification Standards, and Curriculum Outlines. Because a change to one often means a change to the others, it is recommended that they be maintained in an interactive database.
(3) Annual Report. The purpose of the report is to identify changes to the curriculums, training equipment upgrades, and the AQP Maintenance Strategy resulting from feedback and analysis of the information in the Performance/Proficiency Database (PPDB). Details of the report are found in Chapter 2, Section 7.
1-2. AQP OBJECTIVES.
a. Support safe operations by continuously improving training and evaluation.
b. Remain responsive to continuing changes in the industry, including new aircraft technology, changing operational environments, and new training methods and equipment.
c. Remain responsive to continuing changes and best practices relative to training and evaluation.
Par 1-3 Page 2
1-3. AQP CHARACTERISTICS.
a. Participation is voluntary.
b. Employs innovative training and qualification concepts with the regulatory flexibility to tailor training to individual company circumstances.
c. Uses performance data to drive curriculum changes.
d. Qualification is based on individual and team performance, using progressive evaluations of proficiency objectives, and the structure and maintenance of all elements (curriculum, facilities, training equipment, instructors, evaluators, courseware, and quality assurance) of the program.
e. Data collection and analysis to empirically validate individual and team proficiency, and the AQP itself.
f. Training is systematically developed with an audit trail for all training and data requirements.
1-4. EQUIVALENT LEVEL OF SAFETY AND AQP REQUIREMENTS.
NOTE: AQP is a voluntary program that entails a strong commitment from the air carrier to exceed minimum training standards in the greater interest of safety. To determine an equivalent level of safety of an airline’s AQP, the entire training system must be examined as a comprehensive whole rather than considering any one component in isolation. To ensure that the AQP does establish an initial justification and a continuing process to show an equivalent level of safety, mandatory requirements have been established.
Under an AQP, the FAA monitors the process as well as the product. Instead of basing curriculums on prescribed generic maneuvers, procedures and knowledge items, AQP curriculums are based on a detailed analysis of the specific job tasks, knowledge, and skill requirements of each duty position for the individual airline. Compared to traditional training programs, the AQP process provides a systematic basis for establishing an audit trail between training requirements and training methodologies.
AQPs are systematically developed, continuously maintained, and empirically validated, proficiency-based training systems. They allow for the systematic analysis, design, development, implementation, progressive evaluation, and maintenance of self-correcting training programs that include integrated Crew Resource Management (CRM), improved instructor/evaluator standardization, scenario-based evaluation, and a comprehensive data-driven quality assurance system.
Par 1-4 Page 3
a. Comply with all aspects of the approved AQP.
b. The processes used for development, implementation and maintenance of program operations will be continued throughout the life of the program.
c. Accommodate make, model, and series aircraft (or variant).
d. May build upon an existing training program or be completely new.
e. Provide three basic types of curriculums for every duty position; indoctrination (for new hires, new instructors, and new evaluators); qualification; and continuing qualification (see Figure 3-6). Specialty curriculums (transition, conversion, upgrade, requalification, refresher, etc.) will be derivatives from these basic types.
f. Duty positions to be covered must include all flightcrew member positions, instructors, and evaluators, and may include other positions, such as flight attendants (F/A), aircraft dispatchers, and other operations personnel.
g. Provide satisfactory justification to the FAA that the proficiency-based qualification of personnel under AQP meets or exceeds existing 14 CFR part 121 and/or part 135 standards.
h. Each applicable requirement of the Airman Certification Standards (ACS) or practical test standards (PTS) and 14 CFR part 61, 63, 65, 121, or 135 that is not specifically addressed in the AQP continues to apply to the certificate holder.
i. Document the requirements of the ACS or PTS and part 61, 63, 65, 121, or 135, as applicable, which would be replaced by an AQP curriculum.
j. The certificate holder must establish an initial justification and a continuing process to show how the AQP curriculum provides an equivalent level of safety for each requirement to be replaced.
k. Any training or evaluation that is satisfactorily completed in the calendar-month before or the calendar-month after the calendar-month in which it is due is considered to have been completed in the calendar-month it was due.
l. Curriculums must be based on an instructional systems development methodology. This methodology must incorporate a thorough analysis of the certificate holder’s operations, aircraft, line environment and job functions.
m. A list of, and text describing, the knowledge requirements, subject materials, job skills, and qualification standards of each task to be trained and evaluated.
n. A list of, and text describing, supervised operating experience, evaluation/remediation strategies, provisions for special tracking, and how recency of experience requirements will be accomplished.
Par 1-4 Page 4
o. A ll curriculums will include planned (not programmed) hours for ground training, flight training, evaluation, and operating experience.
p. Qualification and continuing qualification curriculums must integrate the training and evaluation of CRM/dispatch resource management (DRM) and technical skills and knowledge.
q. Include line operational training (e.g., full crew LOS).
r. Use LOE or an equivalent evaluation under an AQP acceptable to the FAA for proficiency evaluations.
NOTE: Regardless of number of events sets, a crash (i.e., unsafe individual or crew performance that would result in significant damage, hull loss, or loss of life) during an LOE constitutes a failure of the LOE.
s. Integrate appropriate advanced flight training equipment. Flight-training devices and simulators will be used to support scenario-based training as appropriate.
t. Develop data collection and analysis processes in order to obtain performance information on crewmembers, dispatchers, instructors, evaluators, and other operations personnel that will enable the certificate holder and the FAA to determine whether the form and content of training and evaluation activities are satisfactorily accomplishing the overall objectives of the curriculum.
NOTE: The FAA uses de-identified data for program monitoring not to monitor individual crewmember, dispatcher or other operations personnel.
u. Provide a Master AQP Transition Schedule (MATS) that provides a plan to transition from a traditional program to an AQP. In addition, the MATS should include a plan on how the applicant would return to a traditional program if that becomes necessary or desirable at some later date. This is not to imply that the traditional program must be kept current once AQP is entered, but only how the applicant would update the program and the timeframe needed for implementation.
1-5. DEVELOPMENTAL ASSISTANCE AND SUPPORT. Applicants may avail themselves of a wide range of support organizations, documents and services in the development of their AQPs. There is considerable support available from government and non-government sources.
The total inventory of such support is constantly increasing, and the latest versions of all such support resources are always available by contacting the appropriate Safety Standards office. The following is available directly from the FAA:
a. Flight Standards (FS). The appropriate Safety Standards office is the primary support available to all AQP applicants and participants. This office includes a manager, aviation safety inspectors (ASI), instructional systems development specialists, and data management specialists dedicated to the support of AQP programs. This office works in conjunction with the Principal Operations Inspector (POI) and other FAA personnel to transition applicants into AQP and to maintain the quality of those programs following full implementation.
Par 1-5 Page 5
(1) Role of the Appropriate Safety Standards Office Manager. The FAA manager is the person responsible for ensuring that the regulatory requirements for AQP are met and approved AQP standards are maintained. In this capacity, the appropriate Safety Standards office manager provides oversight of the AQP process and documentation through all phases. The manager will also chair the AQP Extended Review Team (ERT). The manager will retain a quality assurance role and coordinate with the POI any modifications and subsequent approvals throughout the AQP’s lifetime.
(2) Role of the POI. The POI is responsible for ensuring that AQP regulatory requirements are met and, that through surveillance, the AQP provides a satisfactory level of safety and airman competence. The POI will provide oversight of the Instructor/Evaluator program and adherence to approved documentation for all AQP curricula. The POI will coordinate with the manager of the appropriate Safety Standards office any modifications and subsequent approvals throughout the AQP’s lifetime.
(3) ERT. The review and surveillance of an AQP prior to final approval is equivalent to a recertification of the applicants’ entire training program. To ensure maintaining an effective and standardized process, the appropriate Safety Standards office will lead a team to conduct the review and certification of the AQP. The ERT will also include (but not necessarily be limited to) the following:
(a) The POI (or a designated representative).
(b) Training Center Program Manager (TCPM) (if applicable).
(c) The assigned aircrew program manager (APM) (if any).
(d) An appropriate Safety Standards office ASI staff specialist.
(e) A cabin safety inspector (if required).
(f) A dispatcher (if required).
(g) An instructional system design specialist.
(h) A data management specialist.
NOTE: Additional technical personnel can be added to the ERT as required in specific areas of expertise. These would include the focal point, a member of the Flight Standardization Board (FSB) or Flight Operations Evaluation Board (FOEB), as appropriate.
b. Technology Support. The appropriate Safety Standards office includes a technology staff assigned to enhance the effectiveness of the program through the development of websites, databases, collaboration techniques, software training and installation, and other support.
c. Other Resources. The following list (Figure 1-1) provides samples of the types of resources available:
Par 1-5 Page 6 FIGURE 1-1.
OTHER RESOURCES AQP Websites:
http://www.faa.gov/training_testing/training/aqp/ (public access) FAA Resources:
• Title 14 CFR Part 121 Subpart Y, Advanced Qualification Program FAA Advisory Circulars:
• AC 120-35, Flightcrew Member Line Operational Simulations: Line-Oriented Flight Training, Special Purpose Operational Training, Line Operational Evaluation (as amended)
• AC 120-51, Crew Resource Management Training (as amended)
• AC 121-42, Leadership and Command Training for Pilots in Command ATA AQP Working Group: Focus Group Reports:
• Line Operational Simulations: LOFT Scenario Design, Conduct and Validation
• Advanced Qualification Program Instructor/Evaluator Task Analysis
• Applied ISD in AQP Development: Front End Analysis
• Advanced Qualification Program Data Management Guide, Second Release AQP Software Tools:
• RRLOE: Rapidly Reconfigurable Even Set-Based Line Operational Evaluation Software [LOS Scenario-building software toolkit]
• Proteus Database [AQP curriculum & documentation-building software toolkit: AQP in a box]
• Rater Calibration Software Tools [Measures and assesses reliability across evaluators]
• Inter Rater Reliability Assessment Toolkit (workbook & diskette)
• Instructor Evaluator Training & Calibration (IETC) Toolkit AQP Research Products:
• Developing Advanced Crew Resource Management (ACRM) Training: A Training manual
• Developing Operating Documents: A Manual of Guidelines (on CD)
• Facilitating LOS Debriefings: A Training Manual The Appropriate Safety Standards Office Products:
• Advanced Qualification Program Overview, Powerpoint briefing (as amended)
• Advanced Qualification Program (AQP) Inspector Training (Student Guide, Leader Guide, Reference Book, Lab Book, Slides)
• Overview of the Advanced Qualification Program (by T. Longridge)
• Data Analysis and Reporting Tool (DART)
• Data Cleaner (used in conjunction with DART) Generic Documents:
• Generic AQP Application
• Generic Single Visit Training Program Application
• Generic Single Visit Training Plan
• Generic Pilot Job Task List
• Generic Pilot Job Task Analysis
• Generic F/A Job Task List
• Generic Curriculum Development Methodology Document
Par 1-5 Page 7 (and 8) FIGURE 1-1.
OTHER RESOURCES (CONTINUED) Research Papers:
• Scientific Evaluation of Aircrew Performance
• Application of Psychometrics to the Calibration of Air Carrier Evaluators [with UNM]
• Data Warehousing for AQP Databases
• Prototype AQP Data Analysis Process
• Training for Inter-Rater Reliability: Baselines & Benchmarks
• Line Operational Evaluation (LOE) Air Carrier Scenario Based Evaluation
• Performance Measurement Issues and the AQP Database Design
• Skills Development and Assessment in the AQP Environment
• The Importance of Quality Data In Evaluating Aircrew Performance
• Questions Your Database Should Address
• A Guide to the Evaluation Aspects of Entering AQP
• Predicting Behavioral Grades from Physical Flight Data for Rejected Takeoff
• Inferential Statistics and Sample Size: Applications to Air Carrier Training and Assessment
• Initiatives Towards More Affordable Flight Simulators for U.S. Commuter Airline Training
• Simulator Fidelity Requirements: The Case of Platform Motion
• Simulator Platform Motion – The Need Revisited
• Simulator Fidelity: The Effect of Platform Motion
• Relationship Between Objective Measures of Pilot Performance/Behavior and Instructor Grades
• The Effect of Simulator Motion on Pilot Training and Evaluation
• Realistic Radio communications in Pilot Simulator Training
• Air Traffic Control in Airline Pilot Simulator
• Rapidly Reconfigurable Event Set-Based Line Operational Evaluation Scenarios
• Do Three Easy Tasks Make One Difficult One? Studying the Perceived Difficulty of Simulator Scenarios
• Differences in General and Commercial Aviation Automation
• An Information-Processing Model Approach to Identifying Training Issues for Advanced Aviation Automation
• Do Pilots Know How They Manually Control the Vertical Flight Path?
• CRM in the Model AQP: A Preview
• The Use of ASRS Incident Reports in AQP CRM Training
• The Model AQP Database Process: Challenges in Process and Design
• AQPing CRM
• Implementing the Model AQP Database: Lessons Learned
• Data Collection and Analysis: The Next AQP Frontier
• Reconceptualizing Leadership and Followership for Event-Based Training Research Integrations:
• Training Approaches and Considerations For Automated Aircraft: A Summary of Training Development Experiences