Page 70 Par 5-7 instructor/evaluator grading judgments. The program must provide rater reliability training during the qualification, continuing qualification and differences curriculums. There are numerous techniques currently used to address rater reliability. The FAA is involved in research to define acceptable methodologies. Examples are available on the Flight Operational Quality Assurance (FOQA)/AQP Web site.
Par 6-1 Page 71
CHAPTER 6. TRAINING CENTERS
SECTION 1. INTRODUCTION
6-1. BACKGROUND. This chapter provides guidance to any certificated Title 14 of the Code of Federal Regulations (14 CFR) part 142 training center or 14 CFR part 119 certificate holder that intends to provide training, qualification, or evaluation for a part 121 or part 135 certificate holder’s Advanced Qualification Program (AQP). For the purposes of this chapter, a training center is any entity that offers training to a certificate holder by contract or other arrangement.
14 CFR part 121 subpart Y, Advanced Qualification Program, allows certificated air carriers seeking to accomplish training under the AQP to employ the services of a training center for that purpose.
6-2. TRAINING CENTERS OFFERING AN AQP. A training center that applies to offer an AQP curriculum for a specific part 121 or 135 air carrier may be approved under AQP regulatory guidance to provide such training to that specific carrier. There are two types of training centers that may offer AQP.
a. Certificated Part 142 Training Centers. An entity other than a part 121 or part 135 certificate holder may offer an AQP program to air carrier certificate holders. In this case, they are offering an approved, generic AQP with operator specific training to be incorporated into the final product. Any entity serving in this capacity must have been certificated under part 142 after August 2, 1998. Any training program for pilots, instructors, or evaluators approved under AQP is automatically approved under part 142, regardless of whether that approval occurs before or after August 2, 1998.
b. Certificated Air Carrier. A part 121 or part 135 certificate holder may offer a previously approved AQP to other certificate holders. In this case, they are offering their approved AQP with operator-specific training to be incorporated into the final product.
SECTION 2. PROVISIONAL APPROVAL: APPLICATION AND DEVELOPMENT
6-3. PROVISIONAL AQP. The major difference between the development of an AQP by a training center and by an air carrier is that the training center can develop generic AQP documentation and individual curriculum segments through phase II. This documentation is given a provisional approval. A provisional AQP allows a training center to accomplish frontend AQP development through phase II and to offer its services as an approved AQP provider before establishing a contract or other arrangements with a specific certificate holder.
a. Additional Development. A provisional AQP cannot be employed for use with a specific certificate holder without additional development to tailor the program to the needs of a specific air carrier.
Page 72 Par 6-3
b. Multiple Provisional AQPs. Training centers should consider developing multiple provisional AQPs to shorten the time required to adapt the AQP to the needs of an air carrier. For example:
(1) Types -- different makes, models, series.
(2) Trainee demographics – low-time versus high-time pilots.
(3) Flight operations -- short haul regional airlines versus long haul major airlines or supplemental air carriers.
SECTION 3. REQUEST AND APPROVAL FOR USE OF TRAINING CENTERS
6-4. REQUEST. A certificate holder that wishes to use a training center for AQP training must submit a Letter Of Request directly to the FAA manager of Voluntary Safety Programs (VSP), with a copy to the principal operations inspector (POI) and Training Center Program Manager (TCPM). The letter of request should specify the training center, identify the AQP curriculum, and cite the time frame for which such AQP services would be conducted.
6-5. DEVELOPMENT. The AQP analysis and documentation required for a training center to obtain approval to conduct an AQP for a specific certificate holder is similar to that which would be required of an air carrier.
a. Documentation. All documentation must be consistent with the multi-phased development process outlined in this guidance, and must be specific to the certificate holder for whom such training will be conducted.
b. Extended Review Team (ERT). AQP documentation developed by a training center for a specific certificate holder will be submitted directly by the training center to the VSP manager for review and approval. The VSP manager will form an AQP ERT, to include the certificate holder’s POI, the TCPM, and such other personnel as may be appropriate.
c. Notification of Approval. The VSP manager and the POI/TCPM will notify the certificate holder by a joint letter of approval for the training center to conduct AQP training.
(1) For an air carrier, approval for use of the AQP, to include reference to any FAA- specified conditions and limitations, is entered by the POI into the certificate holder’s Operations Specifications (OpSpecs).
(2) Following receipt of notification of approval, the TCPM will notify the training center that it has FAA approval to conduct an AQP program for a specific certificate holder.
Par 6-6 Page 73 (and 74) SECTION 4. TRAINING CENTER’S INSTRUCTORS AND EVALUATORS
6-6. QUALIFICATION. Training center instructors and evaluators are subject to the same training requirements as discussed in Chapter 5.
6-7. INSTRUCTOR/EVALUATOR CURRICULUM APPROVAL. Training centers seeking to offer an AQP curriculum to certificate holders must obtain approval of their AQP qualification and continuing qualification programs for instructors and/or evaluators.
a. Documentation. The documentation required for such approval is similar to that required for certificate holders, except that training centers may obtain provisional approval of AQP instructor and evaluator AQP documentation through AQP phase III.
b. Notification. Notification of provisional approval for instructor and/or evaluator AQPs is initiated by the VSP manager and follows the same process as identified in paragraph 6-5.
SECTION 5. DATA COLLECTION AND RECORDKEEPING
6-8. DATA COLLECTION. Part 121 or 135 certificate holders who obtain an approved AQP through a training center are responsible for verifying compliance with all of the requirements of their approved program, and for reviewing the results of the data collection and analysis on a continuing basis. AQP data collection, analysis, and reporting requirements for training centeradministered AQPs are identical to those that apply to AQPs administered by individual certificate holders for their own employees.
6-9. DATA SUBMISSION. Either the training center or the certificate holder may transmit the required data to the FAA. The party that is delegated for this action must be clearly identified in the training center’s approved AQP Implementation and Operations Plan (I&O Plan) for a specific air carrier. The certificate holder is responsible for complying with the requirements contained in this Advisory Circular (AC) that pertain to the submission of AQP data to the FAA.
6-10. RECORDKEEPING. AQP recordkeeping is identical to that required under traditional parts 121, 135, and 142 training programs. However, for AQP, records must clearly indicate that the training employed to qualify pilots, other flight crewmembers, instructors, or evaluators was accomplished under an AQP curriculum. When a certificate holder arranges for use of a training center to conduct an AQP, the responsibility remains with the certificate holder to assure that the training records required under part 121 or 135 are maintained.
Par 7-1 Page 75 CHAPTER 7. TRANSITION TO AQP
SECTION 1. OVERVIEW
7-1. GENERAL. Most Advanced Qualification Programs (AQP) have been built using one of two approaches. Some applicants have elected to continue training with their traditional program while they developed their AQP. Other applicants have requested an optional exemption to traditional training, commonly called a Single Visit Exemption (SVE).
7-2. ENTERING AQP FROM AN ESTABLISHED TRAINING PROGRAM. Applicants entering AQP may begin with a qualification curriculum or a continuing qualification (CQ) curriculum, or both.
a. Qualification Curriculum. Entering AQP with a qualification curriculum may also require the development of an indoctrination curriculum. Having both curriculums in place allows an applicant to train new hires and provide qualification training for its established crewmembers and other operational personnel (if applicable).
b. CQ Curriculum. If an applicant develops a CQ curriculum using a 12-month training interval, a transition plan must be developed to address the modification of pilots-in-command (PIC), seconds-in-command (SIC) and flight engineers (FE) base months. Participants should transition to a training cycle that puts PICs/SICs/FEs in the same training cycle (annually or more frequently) to maximize simulator resources and to provide effective crew pairing. The transition plan might include:
(1) PICs scheduled for a 6-month flight training session may receive a line check during the 3-month eligibility period in lieu of the training. The AQP CQ will occur during their base month (plus or minus 1 month), 6 months later.
(2) PICs scheduled for a recurrent proficiency check during their base month will receive the AQP CQ.
(3) SICs and FEs (if applicable) scheduled for proficiency checks will receive the AQP CQ.
7-3. TRANSITIONING TO AQP USING THE SVE. The FAA grants an exemption in the public interest and with the assurance that a level of safety equivalent to the regulations from which an exemption is sought will be maintained. The exemption that is issued will contain specific conditions and limitations. It is granted for a 2-year period and may be renewed subject to FAA determination of reasonable progress towards AQP implementation. The SVE allows all fleets to transition to a training cycle that puts PICs, SICs, and FEs in the same training cycle (annually or more frequently). The transition period may require 6 months to place all crewmembers into the Single Visit Training Program (SVTP) and 12 months to complete the training cycle. The SVE applies the following conditions to recurrent training:
Page 76 Par 7-3
a. A certificate holder must maintain an SVTP for crewmembers, instructors, and evaluators.
The certificate holder must submit revisions to its SVTP through its principal operations inspector (POI) to the FAA manager Voluntary Safety Programs (VSP), AFS-230, for approval prior to the execution of privileges under the exemption. Notification of approval for the certificate holder to implement its SVTP will be approved by the POI.
b. A certificate holder must maintain a master AQP transition schedule for FAA approval, and submit quarterly written progress reports thereafter, in a format approved by the Administrator, to the VSP manager with a copy to the POI. The FAA will monitor the progress of the transition toward AQP in accordance with the certificate holder’s approved schedule.
Maintaining reasonable progress toward the certificate holder’s AQP transition is an explicit condition of the exemption.
c. A certificate holder must conduct a pre-training “first-look” evaluation for all pilots that addresses at least a subset of tasks, maneuvers, or procedures included in appendix F to Title 14 of the Code of Federal Regulations (14 CFR) part 121. The certificate holder must collect firstlook data before the repeated execution of any such first-look item in a flight simulator or flight training device (FTD). First-look maneuvers must not be briefed in advance of the first execution of such maneuvers. The certificate holder may elect to obtain first-look evaluation data by measuring the proficiency of first-look items the first time they occur during normally scheduled recurrent training in a flight simulator or FTD, or by scheduling a special first-look evaluation session for first-look items, or a combination thereof. The list of first-look items must be submitted through the certificate holder’s POI to the VSP manager for approval before its use.
The first-look items will be selected for each type of aircraft and will focus on at least one of the following categories:
(1) The execution of maneuvers, tasks, and/or procedures under degraded modes of operation.
(2) The execution of such items under emergency conditions; or
(3) Those maneuvers, tasks, and/or procedures likely to be sensitive to loss of proficiency because of infrequent practice. Unless otherwise authorized by the VSP manager with the concurrence of the POI, first-look items must be maintained therein until this exemption expires. The certificate holder must apply the same measurement methodology and rating criteria to the evaluation of first-look proficiency as that condition specified in paragraph 7-3f below for proficiency evaluations.
d. A certificate holder must ensure all flight crewmembers are trained and evaluated to proficiency at their duty position, except that type-rated first officers may occupy either pilot position for training purposes. This training and evaluation must be conducted during an SVTP that includes at least the following:
• Annual recurrent ground school instruction that includes all 14 CFR part 121-mandated-training requirements and that integrates, where applicable, Crew
Par 7-3 Page 75 Resource Management (CRM) and human factors into recurrent ground school subject lessons.
e. Annual training to proficiency for each flight crewmember in a flight simulator or FTD.
As a minimum, proficiency training must be provided for all tasks, maneuvers, or procedures on which performance is found to be unsatisfactory during the first-look evaluation and must continue until satisfactory performance is observed.
f. Annual recurrent training conducted every 12 months, plus or minus 1 month, in a flight simulator or an FTD approved for its intended use by the POI. This training must be composed of at least two SVTP sessions on 2 separate days, conducted to the maximum extent feasible using a full-crew complement that consists of a captain, first officer, and, where applicable, a second officer. When a full crew is available for training, an SVTP session duration of at least 4 hours per day, including reasonable time for breaks, must be employed, to include line oriented flight training (LOFT), the scenario duration of which must be at least 60 uninterrupted minutes.
Multiple flight segments representative of the certificate holder’s flight operations may be included in the LOFT. When a full-crew complement is not available, the content and duration of the training must be determined in accordance with a crew-pairing decision rule approved by the POI in accordance with the condition spelled out in paragraph 7-3k below, but in no case must the duration of such training be less than 2 hours per day, including a reasonable time for breaks, to include LOFT or Special Purpose Operational Training (SPOT), the scenario duration of which must be at least 60 uninterrupted minutes. Advisory Circular (AC) 120-35, Line Operational Simulations (LOS): Line-Oriented Flight Training, Special Purpose Operational Training, Line Operational Evaluation, as amended, must be used for general guidance with respect to the design and execution of LOFT and SPOT. All LOFT and SPOT scenarios must be approved by the POI for their intended use. SVTP sessions may consist of LOFT, SPOT, line operational evaluation (LOE), support training, and proficiency checks, as well as individual maneuvers or events, as required for training or checking, subject to the conditions specified herein.
g. A certificate holder must ensure all proficiency evaluations consist of maneuvers, tasks, and procedures approved by the Administrator. The certificate holder may elect to propose fixed and variable items to be required on proficiency evaluations, as follows:
(1) Approved fixed items must be included on all proficiency evaluations on a given duty position in a given aircraft for the duration of this exemption, and
(2) Approved variable items may be changed at the discretion of the certificate holder and must be selected based on operational need. Before the execution of this option, the content of fixed and variable items, together with any associated evaluator proficiency assessment rating forms, must be submitted through the POI to the VSP manager, AFS-230, for approval. Any fixed and variable maneuver list proposed by the certificate holder must incorporate specifications of the applicable duty position (e.g., PIC, SIC, or both, and FE) and pilot role (pilot flying, pilot not flying, or both) to be addressed in the proficiency evaluation for each such maneuver.
Page 76 Par 7-3
h. Proficiency evaluations must be accomplished to the maximum extent feasible in a crew-oriented setting with a PIC and SIC. For all tasks, maneuvers, or procedures to be included on proficiency evaluations, a certificate holder must identify the performance standards to be employed. Where appropriate, current FAA practical flight test advisory materials (FAA-S- 8081-5, Airline Transport Pilot and Aircraft Type Rating Practical Test Standards for Airplane, as revised) and pertinent flight standardization board reports for a given type of aircraft should be used for guidance in developing specifications for performance standards. The specification of performance standards must be submitted through the POI to the VSP manager for approval before implementation of the certificate holder’s proposed SVTP.
i. A line check that consists of at least one flight segment and that incorporates both a technical and CRM debriefing facilitated by a line check airman must be conducted for all PICs not more than 6 months, plus or minus 1 month, following the PIC proficiency check.
j. For first-look evaluations, proficiency evaluations, and line checks, a certificate holder must develop and implement the following:
(1) A proficiency rating scale methodology (e.g., Unsatisfactory = 1; Repeated (Once Only) = 2; Debriefed (No Repeats) = 3; Satisfactory = 4; and Outstanding = 5), and
(2) A reason categorization system for use by evaluators to categorize the principal reason for below satisfactory performance (e.g., system knowledge, procedural knowledge, execution, communication/coordination, situation awareness, flight management/flight guidance system operation, and teamwork). A certificate holder must submit its proposed rating scale methodology and reason categorization system through its POI to the FAA AQP manager for approval before the implementation of its SVTP.
k. Notwithstanding any other condition of this exemption, a certificate holder must ensure that each person serving as a required flight crewmember on more than one airplane type complies with the provisions of part 121, sections 121.433(c)(1)(i) and 121.440(a).
l. Any certificate holder PIC who fails a proficiency check must:
(1) Receive additional training to proficiency,
(2) Demonstrate within-standards performance on a subsequent proficiency check before returning to flight duty,
(3) Satisfactorily complete an additional line check not more than 3 months following an unsatisfactory proficiency check, and
(4) Satisfactorily complete an additional proficiency check not more than 6 months following an unsatisfactory proficiency check.
NOTE: The provisions of this condition must not establish a new anniversary date for training or checking.
Par 7-3 Page 77
m. A certificate holder must ensure every flight crewmember who fails to complete any training or evaluation satisfactorily under this exemption does not serve as a required flight crewmember until such deficiencies have been corrected and the crewmember has been retrained or satisfactorily reevaluated as appropriate on those items.
n. Training to be accomplished under the exemption must specifically include CRM conducted in a flight simulator. Certificate holders must refer to AC 120-51, Crew Resource Management Training, as revised, for general guidance in the development of CRM training plans and programs.
o. Before implementation, a certificate holder must submit a documented methodology for developing and administering LOS scenarios (LOFT, SPOT, or LOE) through its POI to the FAA AQP manager for approval. That methodology must ensure that identical LOFT/LOE scenarios are not employed in two successive training sessions for a given crew, and pilots are not exposed to identical LOFT/LOE scenarios in 2 successive years.
p. A certificate holder must ensure that first-look, LOS, and proficiency evaluations are accomplished in a flight simulator meeting the requirements of appendix H to part 121 and are approved for its intended use by its POI. Notwithstanding the provisions of this exemption, if a proficiency evaluation required by this exemption is not accomplished in a level B (phase I) or higher flight simulator, then two landings must be accomplished in the appropriate aircraft in accordance with the provisions of part 121, section 121.441(e). This requirement must be accomplished within the eligibility period for the proficiency evaluation involved.
q. Data on pilot performance conducted under the provisions of this exemption must be collected by the certificate holder and reported to the FAA, as specified in paragraph 7-3v below, beginning on implementation of its SVTP, unless otherwise authorized by the Manager, Air Transportation Division, AFS-200. Performance evaluation and data reporting must be accomplished in accordance with the following categories and the provisions of the condition in paragraph 7-3f above:
(1) By aircraft make, to include separate categorization by series and variant if, according to the criteria specified in AC 120-53, Crew Qualification and Pilot Type Rating Requirements for Transport Category Aircraft Operated Under FAR Part 121, as revised, significant operational differences exist.
(2) By crew position (PIC, SIC, and/or FE) and SVTP flight simulator sessions. Each crewmember’s data must be reported as a single identifiable electronic record, with appropriate identifiers to distinguish crew position to permit tracking individual performance, where possible, from first-look evaluation through proficiency training and evaluation. The certificate holder must report additional training sessions beyond normally programmed training for any crewmember. The certificate holder must report the occurrence of seat position substitution by an instructor or other qualified pilot, or crewmembers in training who are occupying other than their normal seat position.
Page 78 Par 7-3
(3) By individual task, maneuver, or procedure, as applicable.
r. First-Look. Rating scale and reason code data must be obtained on the first occurrence of appendix F to part 121 subset maneuvers in accordance with the provisions of conditions in paragraphs 7-3c and 7-3f above.
s. Proficiency Training. Data must be obtained on the number of maneuver repetitions required during training to proficiency on first-look items on which initial performance is below standard.
t. Proficiency Evaluation. Rating scale data must be obtained for each repetition of a proficiency evaluation maneuver, task, or procedure to include reason codes for each occurrence of performance below standard, in accordance with the provisions of the condition in paragraph 7-3f above. Proficiency must be measured against the specifications for performance standards developed in accordance with the provisions of paragraph 7-3j above. Certificate holder must report unsatisfactory proficiency evaluations.
u. Line Check. Line checks must employ the same evaluation methodology as specified under this exemption for proficiency checks, with the exception that such data will be limited to the PIC tasks, maneuvers, and procedures that occur during the flight segments observed, in accordance with the provisions of the condition in paragraph 7-3e above. The certificate holder must report unsatisfactory line checks.
v. The certificate holder must provide the FAA with de-identified raw data (i.e., individual pilot performance assessment data on which name identification has been removed) on a monthly basis, in accordance with the categorization requirements specified in the condition in paragraph 7-3m above. For this purpose, the certificate holder must convey these data via IBM-compatible diskettes or electronic media. The data format, together with a written description of the electronic record structure (to include database file relationships and data record field definitions), should be forwarded directly to the FAA AQP for approval. The certificate holder must obtain approval for any subsequent changes thereto, not less than 30 days before the implementation of any such changes.
w. In addition to any other reporting requirements specified herein, the certificate holder must submit an annual SVTP report documenting training and checking effectiveness to the certificate holder’s POI using a method and format acceptable to the Administrator. These reports must:
• Document the certificate holder’s own analysis of data acquired under this exemption
• Identify any trends, common problem areas, or potential deficiencies
• Include a description of any corrective steps taken or warranted
During the second year of the exemption, the reports must incorporate comparisons of data between equivalent periods in the preceding year.
Par 7-3 Page 79
x. SVTP review meetings must be scheduled to occur on an annual basis, unless sooner requested by the FAA or by the certificate holder, to be attended by representatives of the certificate holder, the certificate holder’s POI, and AFS-230 at a location and on a date to be determined by the POI. The purpose will be to review:
(1) Compliance with the terms and conditions of this exemption.
(2) Data collection issues.
(3) Data results including, in particular, the identification of trends, as well as any associated action taken or warranted.
(4) The status of the certificate holder’s progress toward transitioning its aircraft fleets out of this SVE into AQP.
SECTION 2. THE PETITION FOR EXEMPTION
7-4. GENERAL. The petition for exemption to allow transition to AQP is made in accordance with 14 CFR part 11, section 11.25. Figure 7-1 illustrates sequence of events for obtaining and applying the SVE.
FIGURE 7-1.
SINGLE VISIT EXEMPTION PROCESS (SVE)
Submit AQP Application
Develop Petition for SVE Develop Single Visit Training Plan
AQP Application Approved
SVE Issued SVTP Approved
Implement SVTP
Submit Performance/ Proficiency Data Monthly
Analyze Data
Quarterly Progress Reports
Annual SVTP Review
7-5. THE PETITION PROCESS. Any interested person may petition the Administrator to issue, amend, or repeal a rule. Each petitioner must submit, unless good cause is shown, the petition at least 120 days before the proposed effective date of the exemption. Each petition