Par 4-1 Page 55 CHAPTER 4. AQP APPROVAL AND DOCUMENTATION SECTION 1. DOCUMENTATION REVIEW AND APPROVAL
4-1. OVERVIEW. This section establishes how the FAA reviews and grants approval, or withdraws approval, for all or part of an Advanced Qualification Program (AQP). Specific document content is discussed in Chapter 2.
a. Approval Authority. Except for the three items listed below, notification of approval for AQP documents and their revisions is a joint signature responsibility of the manager of appropriate Safety Standards office and the Principal Operations Inspector (POI), based on recommendations from the FAA Extended Review Team (ERT). The exceptions are:
(1) Application –manager of the appropriate Safety Standards office.
(2) Line Operational Simulation (LOS) scenario content – POI signature approval.
(3) Phase V curriculum outline revisions – POI signature approval.
b. Approval Process. All AQP documents and subsequent revisions are reviewed by the ERT. The ERT reports to the manager of the appropriate Safety Standards office and to the POI with its recommendations. Based on the acceptance of the ERT recommendations, the manager of the appropriate Safety Standards office will originate, sign, and forward electronically a response letter to the POI for signature and delivery to the operator. The operator will ensure that the manager of the appropriate Safety Standards office and the POI receive copies of the approved material.
4-2. REVIEW PROCESS. Chapter 1 of this advisory circular (AC) discusses the roles and responsibility of the ERT personnel involved with AQP. The ERT prefers to interact closely with applicants as AQP documentation is developed. Experience has shown that is more efficient for the ERT to discuss plans and review drafts of document sections early in the development process.
4-3. APPROVAL PROCESS. Applicants develop, implement, and operate the AQP in five sequential phases. FAA approval of each phase marks the completion of that phase and entry into the next. Developing and implementing an AQP requires a commitment of resources for both the FAA and a certificate holder. The FAA and the certificate holder must use their resources in an effective and efficient manner. The FAA has established a transition time criterion in order to provide a clear delineation of FAA expectations regarding reasonable progress towards AQP implementation. Accordingly, continuation in AQP for applicants who do not initiate Phase IV within the 30th month of Phase I approval, as indicated in Figure 4-1, will be subject to a review and analysis of their progression towards completing AQP. If circumstances do not warrant an extension, AQP and the Single Visit Training Program (SVTP) Plan approval (if applicable) may be withdrawn.
FIGURE 4-1.
FIVE PHASES OF AQP REVIEW AND APPROVAL Application Phase I + Application Approved Program Review Curriculum Job Task Analysis Development Qualification Standards Instructional Systems Phase II + Development 3 Program Review By ERI Curriculum Outline Methodology I&O Plan M All Are Approved Curriculum Implementation Phase III Program Review By Small Group Tryout Approved ERT Initial Operations Data Collection & Phase IV Curriculum Maintenance Program Review By Approved ERT Continung Annual AQP Report Phase V Operations Maintenance of AQP Documents
4-4. PHASE APPROVALS. After concluding a joint meeting with the potential applicant, the manager of the appropriate Safety Standards office and the POI/Training Center Program Manager (TCPM) will form the core ERT. The purpose of forming the ERT prior to accepting the Phase I application is to have FAA resources available to help the applicant during the development process. In reviewing all phases of AQP development, the ERT will:
• Evaluate documents based on the guidelines in Chapters 2 and 3 (see Appendix 3).
• Provide suggestions for improvement, if appropriate.
• Make recommendations to the manager of the appropriate Safety Standards office/POI/TCPM regarding approval.
a. Phase I - Initial Application. Approval of the initial application marks the certificate holder's formal entry into the AQP. The steps for the applicant are:
Par 4-3 Page 56
Par 4-4 Page 57
• Simultaneously submit the application electronically to the POI/TCPM and to the manager of the appropriate Safety Standards office.
• Address comments and recommendations of the ERT.
After determining that the applicant’s submittal is satisfactory, the manager of the appropriate Safety Standards office, in collaboration with the POI/TCPM, will approve the application. This approval will permit the applicant to proceed to Phase II.
b. Phase II – Curriculum Development. Curriculum development follows a sequential order of program development and submission: job task analysis, qualification standards, instructional systems development methodology, curriculum outlines, and Implementation and Operations Plan (I&O Plan). After all Phase II-required documents have been reviewed by the ERT, the manager of the appropriate Safety Standards office and the POI/TCPM will issue a joint letter of approval to the applicant, with permission to begin Phase III development.
c. Phase III – Small Group Tryout. In this phase, the applicant will implement the AQP on a limited scale as defined in the I&O Plan. The applicant should provide the ERT a schedule of Phase III activities at least 30 days before starting the events.
(1) Review and Surveillance. ERT activities in this phase will consist of observing, monitoring and participating in the AQP training programs. Particular emphasis will be placed on:
(a) Instructor/evaluator training and observation.
(b) Training scenarios.
(c) Validation.
(d) Line Operational Evaluation (LOE).
(2) Program Operation. Proficiency data, maintenance plan, quality assurance, data collection, analysis, and reporting will be evaluated for consistency and accuracy.
(3) I&O Plan Review. Once Phase III is concluded, the applicant will provide the ERT with an updated I&O Plan. Based on the recommendations of the ERT, the manager of the appropriate Safety Standards office and the POI will approve the completion of Phase III. Initial approval permits the applicant to execute the updated I&O Plan through one complete cycle of the continuing qualification cycle (usually 24 months). See Chapter 2, Section 5, paragraph 2-16 for the qualification and indoctrination curriculums (Phase IV period).
NOTE: The POI and certificate holder must add an AQP entry (A034) into the automated operations specifications (OpSpecs).
d. Phase IV – Initial Operations. In this phase, the applicant will implement the AQP as defined in the updated I&O Plan through one complete cycle of the curriculum.
Par 4-4 Page 58
(1) Review and Surveillance. ERT activities in this phase will consist of surveillance of AQP operations and analysis of data collection results. Joint ERT and applicant reviews will be conducted periodically. These meetings will provide both parties the chance to analyze results and discuss program concerns. At the midpoint in Phase IV, the first annual report will be submitted to the ERT. A final joint review and annual report will be accomplished prior to Phase IV Initial Approval expiration (usually 24 months). Focus areas for these reviews are (see Appendix 3):
(a) Data Management:
1. Collection.
2. Analysis.
3. Standardization.
4. Observations.
5. Additional training.
6. First-look.
7. Program feedback.
(b) Recordkeeping.
(c) Adherence to I&O Plan.
(d) Modifications to the program.
(e) Qualification standards that are reliable and valid.
(f) Maintenance of the AQP.
(g) Currency of the program audit database.
(h) Instructor/evaluator program.
(i) Special tracking.
(j) Transition to AQP (non-AQP fleets).
(2) Once the final joint review has been completed, the applicant will update the I&O Plan to include the changes recommended by the ERT. The successful completion of this phase, and approval of the commensurate revisions to the approved AQP documents and reports, will qualify an applicant for entry into the final phase of the AQP process: Phase V, Continuing Operations. Based on the recommendations of the ERT, the manager of the appropriate Safety Standards office and the POI will approve the completion of Phase IV. The POI and the carrier
Par 4-4 Page 59 will then modify the previously issued OpSpec paragraph A034 to include the Phase V approval date.
e. Phase V – Continuing Operations. In this phase, FAA oversight of the AQP is maintained under an FAA surveillance program. The certificate holder will maintain the AQP through documentation changes, program revisions, data analysis, and will continue to submit monthly data and an annual AQP report.
4-5. METHOD OF GRANTING APPROVAL.
a. Initial (Phase IV) or Provisional (Phase III). The manager of the appropriate Safety Standards office and the POI will grant initial Phase IV AQP approval by a joint signature letter.
For Title 14 of the Code of Federal Regulations (14 CFR) part 142 training centers, the manager of the appropriate Safety Standards office and the TCPM will grant provisional Phase III AQP approval by a joint signature letter. Copies of the approved curriculum documentation and the approval letter are maintained by the POI at the responsible Flight Standards office and the manager of the appropriate Safety Standards office. Copies of a training center’s provisionally approved curriculum material is maintained by the TCPM, the manager of the appropriate Safety Standards office, and the participating air carriers’ responsible Flight Standards office. The approval letter will include at least the following information:
(1) The specific identification of the curriculums for an AQP air carrier or training center curriculum segments, initially or provisionally approved, including a List of Effective Pages (LEP) and revision control dates (date of revision for any page).
(2) A statement that initial or provisional approval is granted, and for initial approval, the effective and expiration dates.
(3) Any specific conditions affecting the approval. A requirement that the applicant provide the FAA with advanced notice of scheduled activities so evaluations may be planned.
b. Final Approval. Based on the results of evaluations accomplished during Phase IV, the period of initial approval, the manager of the appropriate Safety Standards office and the POI will grant or deny final approval of an AQP. Final approval is accomplished with a joint signature approval letter and by stamped endorsement by the POI of the AQP document’s LEP.
Copies of the approved curriculum documentation and the approval letter are maintained by the POI at the responsible Flight Standards office, the manager of the appropriate Safety Standards office, and at the location designated by the participant as its principal training site.
(1) Stamped Approval Endorsement or Acceptable Substitute. For initial and final approval, and revisions thereafter, the original LEP of all AQP Program Audit Database (PADB) documents are stamp-approved, dated, and signed by the POI. The approval stamp should be a duplicate of the following:
Par 4-5 Page 60
FAA FINAL APPROVAL OFFICE DESIGNATOR:
EFFECTIVE DATE:
NAME:
SIGNATURE:
NOTE: As approved changes are made to AQP PADB documents, the “Final Approval” endorsement will be re-accomplished on each LEP.
(2) Approval Letter. All letters of final approval will be a joint signature letter signed by the manager of the appropriate Safety Standards office and the POI. The letter will specifically identify the subject curriculums, contain a statement that “final approval is granted,” and provide the effective date of approval. The final approval does not expire.
4-6. REVISION CONTROL PROCESS. The development and maintenance of an AQP necessitates revisions to documents. Due to the composition and geographic location of the ERT, it is imperative that a systematic revision process be followed.
a. Revisions. AQP document revisions are submitted to the manager of the appropriate Safety Standards office and the POI. In Phases II through IV, the revision is reviewed by the ERT and approved using the notification process for the curriculum phase identified in Figure 4-1. The process is identical in Phase V, except that review and approval of revisions to the curriculum outline are made directly by the POI. Copies of all revisions in all phases, including the curriculum outline, will be provided to both the manager of the appropriate Safety Standards office and the POI. Each revision must indicate which AQP documents are being revised, and must include the following items:
(1) A brief synopsis of each change and what pages/items are affected.
(2) Change bars or acceptable substitute identifying the text/chart that has been changed.
(3) The date of the revision and revision number on each page or graphic.
(4) The LEP.
NOTE: If an operator chooses to integrate the required AQP documents into an established manual structure, each document’s location must be identified and each revision request must identify the AQP document that is being revised and which manual contains the document.
b. Implementation. A revision to an approved portion (initial or final) of the AQP cannot be implemented until the applicant receives FAA approval.
Par 4-7 Page 61 (and 62) SECTION 2. WITHDRAWAL OF APPROVAL
4-7. GENERAL. The FAA may withdraw initial, provisional, or final approval at any time the AQP is not in regulatory compliance, does not provide for safe operations, does not effectively prepare personnel to meet qualification objectives, or when the required data is not being provided or maintained. Before withdrawing approval, the FAA will make reasonable efforts to convince an applicant to correct the deficiencies.
4-8. INITIAL OR PROVISIONAL. The FAA will withdraw initial or provisional approval by letter. The letter will identify the affected curriculums, state the reasons for the withdrawal, and state the effective date of the withdrawal. An applicant who receives a letter of withdrawal may revise or refine the curriculum and resubmit it for initial or provisional approval.
4-9. FINAL APPROVAL. The FAA will withdraw approval by letter. The letter will identify the affected curriculums, state the reasons for the withdrawal, and state the effective date of the withdrawal (except in an emergency, not less than 7 days after receipt of the letter). The letter will advise the certificate holder that withdrawal may be appealed and provide instructions on how to appeal.
4-10. APPEAL WITHDRAWAL OF FINAL APPROVAL. An operator should petition the Director of the Office of Safety Standards for reconsideration within 30 days after receiving withdrawal notification. The petition should be in writing and should explain in detail why the operator believes the withdrawal should not occur.
a. Denial of Appeal. The Director may immediately deny the petition after considering all relevant information presented if the Director believes that an emergency exists that directly affects aviation safety. In this case, the Director will inform the operator, by letter, of the decision to deny the appeal due to the existence of an emergency. The letter will state that an emergency exists and describe the deficiencies and the actions necessary to correct them.
b. Stay of Withdrawal. If the Director does not believe that an emergency exists, the Director shall carefully consider both the operator’s petition for appeal and the FAA’s reason for withdrawal of approval. In this case, the operator’s petition, provided it goes out within 30 days, stays withdrawn, and the operator may continue to use the AQP curriculum pending the decision of the Director. The Director may find it necessary to conduct additional evaluations of the operator’s AQP.
c. Director’s Final Decision. In any case, the Director will make a final decision within 60 days of receiving the operator’s petition. The Director may rescind or modify the letter of withdrawal or uphold the withdrawal. If the decision is to modify or uphold the withdrawal, the operator will be notified by letter. The letter will contain the reasons for denying all or part of the petition.
Par 5-1 Page 63 CHAPTER 5. INSTRUCTORS AND EVALUATORS SECTION 1. INSTRUCTOR/EVALUATOR PROGRAMS
5-1. GENERAL. Instructors, evaluators, and supervisors are the backbone of the Advanced Qualification Program (AQP). The applicant must devote the appropriate time and resources to the qualification and continuing qualification of these key personnel.
5-2. INSTRUCTOR/EVALUATOR CURRICULUMS. Each AQP (including provisional AQP curriculums for training centers) must provide instructor, evaluator, and supervisor indoctrination, qualification, and continuing qualification curriculums. These requirements include a separate job task analysis (JTA), qualification standards, curriculum(s), and curriculum outlines focusing on the instructor/evaluator duty positions. AQP regulatory guidance states, in part, that an AQP provides for approval of an alternate method for qualifying, training, certifying and otherwise ensuring competency of instructors and evaluators required to be trained or qualified under Title 14 of the Code of Federal Regulations (14 CFR) parts 121 and 135. AQP entails no changes to the existing policies and procedures as they apply to the eligibility, approval, and surveillance requirements for instructors, evaluators (check airmen) and Aircrew Program Designees (APD).
NOTE: The qualification standards document for instructor/evaluators does not need to include conditions or a criticality/currency analysis.
a. Definitions. AQP regulatory guidance defines an evaluator as a person who has satisfactorily completed training and evaluation that qualifies that person to evaluate the performance of crewmembers, instructors, other evaluators, aircraft dispatchers and other operational personnel, as appropriate. In an effort to remain aligned with traditional check airman policy and procedure, the AQP term “evaluator” is considered synonymous with check airman. However, for flight attendant (F/A) and dispatcher AQP, the word “evaluator” will replace the traditional use of the word “supervisor” as an individual that evaluates. Part 121 subpart Y does not redefine the qualifications or prerequisites to be an instructor, evaluator, check airman, or APD. Definitions of these positions are found in FAA Order 8900.1.
b. Qualification Training. AQP regulatory guidance contains limited guidance on the instructional content of each curriculum. Training requirements for each curriculum are derived from a thorough and accurate task analysis. Typical subject areas might include as appropriate:
(1) Instructor Indoctrination Curriculum.
(a) The learning process.
(b) Elements of effective teaching.
(c) Student evaluation, quizzing, and testing.
Par 5-2 Page 64
(d) Overview of AQP program development, implementation, and operation policy.
(e) Lesson preparation and application.
(f) Classroom instructing techniques.
(g) Techniques for instructing in the cockpit environment.
(h) Standardization and rater/referent reliability.
(i) Resource management (Crew Resource Management (CRM)/dispatch resource management (DRM)) and human factors training.
(j) How to conduct training modules for students with varying backgrounds and varying levels of experience and ability.
(k) Instructor responsibilities.
(2) Instructor Qualification Curriculum.
(a) Effective use of, and qualification in, specific Flight Training Devices (FTD), flight simulators, and aircraft.
(b) Limitations on use of training equipment.
(c) Evaluation of performance against objective standards.
(d) Effective preflight and postflight instruction.
(e) Effective analysis and correction of common errors.
(f) Teaching/facilitation of CRM/DRM skills.
(g) Performance and analysis of standard flight events and procedures.
(h) Safety considerations in the training environment.
(i) Data-gathering procedures.
(j) Standardization and rater/referent reliability.
(3) Differences Between Traditional and AQP for Existing Instructors.
(a) Overview of AQP program development, implementation, and operation policy.
(b) CRM/DRM and human factors training.
Par 5-2 Page 65
(c) Standardization and rater/reverent reliability.
(d) Data-gathering procedures.
(e) Effective use of, and qualification in, specific FTDs, flight simulators, and aircraft used in the AQP.
(f) Limitations on use of training equipment used in the AQP.
(g) Evaluation of performance against objective standards.
(4) Evaluator Indoctrination Curriculum.
(a) Evaluation policies and techniques.
(b) The role of the evaluator.
(c) Administrative procedures.
(d) General safety considerations.
(e) Evaluating human factors and CRM/DRM skills.
(f) Standardization and rater reliability.
(5) Evaluator Qualification Curriculum.
(a) For each crewmember position requiring a particular evaluation the method of conducting:
1. Line check.
2. In-flight proficiency evaluations if required.
3. Proficiency evaluations in flight simulators and/or flight training devices.
4. Special purpose evaluations (e.g., long-range navigation).
(b) The standards for the evaluations in the previous paragraph.
(c) The methods and standards associated with airman certification evaluation.
(d) How to conduct evaluations while simultaneously serving as pilot in command (PIC), second in command (SIC), or safety pilot.
(e) Safety considerations for the various types of evaluations.
(f) Safety considerations particular to the make, model, and series aircraft (or variant).
Par 5-2 Page 66
(g) How to evaluate instructors/evaluators.
(h) Company/FAA policies with regard to the conduct of evaluations.
(i) Administrative requirements particular to evaluations.
(j) Evaluating CRM/DRM skills.
(k) Briefing and debriefing techniques.
(l) Data-gathering procedures.
(6) Differences Between Traditional and AQP for Existing Evaluators.
(a) Overview of AQP program development, implementation, and operation policy.
(b) CRM/DRM and human factors training/evaluating.
(c) Standardization and rater reliability.
(d) Data-gathering procedures.
(e) Effective use of, and qualification in, specific flight training devices, flight simulators, and aircraft used in the AQP.
(f) Limitations on use of training equipment used in the AQP.
(g) Evaluation of performance against objective standards.
(h) Briefing and debriefing techniques.
(i) How to evaluate instructors/evaluators.
(j) Company/FAA policies with regard to the conduct of evaluations.
(k) Administrative requirements particular to evaluations.
c. Qualification—Evaluation and Observation. Every instructor/evaluator must receive a validation/evaluation at the end of each curriculum. These methodologies must be described in the test and evaluation strategy within the instructor/evaluator qualification standards. The required FAA observation of new instructors/evaluators (check airman) is delineated in Order 8900.1.
d. Continuing Qualification. Instructors/evaluators should be given an opportunity to operate in all positions in which they are authorized to perform duties. Flight instructors/evaluators should demonstrate the ability to brief and debrief, fly and instruct in both pilot seats and operate the simulator, if appropriate. If possible, instructors/evaluators should focus on the training, validations and evaluations that are under development for the next evaluation period. This will allow the instructors/evaluators to become familiar with all aspects
Par 5-2 Page 67 of the periods they will be expected to teach/evaluate the following year, help ensure the development of a polished continuing qualification program and give the instructors/evaluators an opportunity to maintain their own proficiency using scenarios they have not seen. Each training period should be conducted by one of a small core of the most experienced instructors/evaluators. Each instructor/evaluator continuing qualification curriculum segment should include the requirements listed below. Requirements (subparagraphs (3) and (4) below) should be defined in the test and evaluation strategy in the instructor/evaluator qualification standards:
(1) Basic crew position and or specialized flight instructor/evaluator continuing qualification.
(2) Ground and flight training to enhance, upgrade, and maintain each instructor/evaluator’s knowledge, skills, and abilities.
(3) A schedule for recency of instructor/evaluator’s experience.
NOTE: The program must define the minimum requirements each category of instructor/evaluator will accomplish to stay current in their position.
Instructors must maintain a high level of competency, so the minimum standard must be sufficient to maintain their proficiency. Evaluators will perform an established minimum number of each validation/evaluation that they are qualified to administer.
(4) A schedule for critical examination of each instructor/evaluator’s abilities and adherence to prescribed standards.
e. Instructor and Evaluator CRM Training and Evaluation. All instructors and evaluators will receive instruction and evaluation in CRM/DRM objectives and training methods, as appropriate.
5-3. AUTHORIZED EVALUATION PERSONNEL BY AQP EVENT. The table in Figure 5-1 illustrates the level of authorization needed for an individual to either train, validate, or evaluate an AQP event.