Appendix 2
(e) Identify interpretations of possible test results. The FAA and the applicant will then reach an agreement on specific tests, devices, and schedules to be used for the test program.
(f) Submit proposals for the following items to the FAA, as necessary:
• MDRs;
• Example ODRs;
• Tests and criteria to be used; and
• Other supporting information associated with training, checking, or currency programs.
c. Difference Level Tests. A sequence of five standard tests, described in Appendix 3, is used to set MDRs, minimum acceptable training programs, and other FSB recommendations and define type rating requirements. One or more of these tests is applied, depending on the difference level sought and the success of any previous tests used in identifying MDRs. Only those tests needed are used to establish minimum requirements. The outcome of these tests, and any resulting difference levels that apply, establish minimum requirements for training, checking, currency, and type ratings. The FAA will establish an additional type rating if it is determined during this testing that the assignment of a level E difference training is required.
NOTE: One additional test, the T6 test, can be used to establish CTLC between related aircraft, when not previously demonstrated in a T2 test.
(1) Steps in the Testing Process. The typical steps of the testing process are as follows:
(a) The applicant develops representative training programs, differences and/or related aircraft differences programs, and necessary supporting information, as needed.
(b) The applicant identifies proposed MDRs and example ODRs.
(c) The applicant proposes and the FAA determines which tests and criteria apply.
(d) The applicant proposes and the FAA determines which aircraft, simulation devices, or analyses are needed to support testing.
(e) The applicant makes a proposal to the FAA, and agreement is reached on test procedures, schedules, and specific interpretation of possible results.
(f) Tests are conducted and results evaluated.
(g) The FSB draft minimum requirements are formulated.
NOTE: If the candidate aircraft is anticipated to have no greater than level A or B differences with the base aircraft, the FSB may elect to directly apply a T1 test for equivalency.
Appendix 2
(2) Test Purpose and Application. A summary of the purpose and application of each of the six difference tests is shown in Figure 5.
(3) Test Relationships and Applications. This process is followed whenever a new aircraft is proposed, when significant changes are proposed, or when revisions to existing requirements are needed as a result of requests for change or in-service experience. The test process relationships, the sequence of conducting tests when more than one test is needed, and application of test outcomes are shown in Figure 6. The start of the process is shown at the top of Figure 6.
(a) When a new aircraft is certified and is not compared with an existing aircraft, the aircraft is evaluated using the T5 test to determine the effectiveness of the manufacturer training and evaluation plan. A type rating is determined as a result of the evaluation.
(b) When two aircraft with the same TC are compared, a T1 or T2/T3 test is used to determine the level of differences between the two aircraft. The result can be the assignment of a same type rating if no greater than level D training differences exist or the assignment of a different type rating if level E training differences exist.
(c) When two aircraft with different TCs are compared, a T2/T3 test is used to determine the level of differences between the two aircraft. An example of this is a determination of B, C, D, or E training differences between the two aircraft.
(4) Test Failures and Retesting. Generally, failures do not have paths back to lower levels. T3 test failure at level C can lead to subsequent passage at C or D after modification of the system, operational procedures, or training and retesting. Similarly, failure at level D can subsequently lead to either D or E after modification of the system, operational procedures, or training and retesting, but not C. Failure at level E can only lead to retesting with increased programs, improved programs, or improved devices, since there is no higher level. T5 test failure paths do not lead back to level C or level D. However, subsequent new programs do not preclude making a proposal at a lower difference level if technology changes, aircraft redesign takes place, training methods significantly change, or device characteristics and effectiveness change.
(5) Type Rating Tests. Aircraft seeking a same type rating or type rating in common will follow the path in Figure 6 from the top right “START” of Figure 6 through T1 or T2 and T3 tests, resulting in the assignment of level A, B, C, or D differences.
(6) Currency Tests. T4 currency tests are not shown in Figure 6 because they are necessary only when the applicant seeks relief from system, procedural, and maneuver currency requirements set by the FSB.
(7) Detailed Test Specifications. A detailed specification for the evaluation process and tests to establish difference levels is described in Appendix 3.
d. FSB Assessments and Proposal Formulation. The FSB assesses the applicant’s proposals, test results, analysis, and any other relevant factors to formulate a draft FSB report, which includes MDRs and other pertinent training, checking, and currency requirements.
The FSB either validates the applicant’s proposed MDRs, training programs, and other
Appendix 2 information, or generates alternate requirements, which may include more stringent requirements, additional training, additional testing, etc.
FIGURE 4. MASTER REQUIREMENTS FORMULATION
MASTER REQUIREMENTS FORMULATION (AN FAA/APPLICANT PROCESS) DESIGN
OBJECTIVES
PROPOSED MDRS
EVALUATION & TESTING COMMENTS
FSB REPORT
MDRS AND SAMPLE ODRS
FSB REPORT
APPROVAL
AND FAA
FIELD OFFICE USE FSB
REVISION REQUESTS
OPERATORS
MANUFACTURERS
FAA PRINCIPAL OPERATION
INSPECTORS
OTHER AVIATION ORGANIZATIONS
TYPICAL
MDRS & ODRS
APPLICANT
SUPPORT FAA FLIGHT
STANDARDIZATION BOARD (FSB)
Appendix 2 FIGURE 5. TEST DEFINITIONS
TEST PURPOSE APPLICATION T1 Establishes functional equivalence Sets levels A/B T2 Handling qualities comparison Pass permits T3, and A/B/C/D; failure sets level E and requires T5 T3 Evaluate differences and sets training/checking requirements Pass sets levels A/B/C/D; failure sets level E and requires T5 T4 Revises currency requirements Used to adjust FSB recommendations—if needed T5 Sets training/checking for new or “E” ACFT Sets level E T6 Evaluation for CTLC Sets recency-of-experience requirements Note: Expanded descriptions are contained in Appendix 3.
Appendix 2 FIGURE 6. “T” TESTS 1 THRU 5
“T” TESTS 1 THRU 5 Candidate Level E?T1 Req’d?
T3 Req’d?
T1 T2 T3 T5 LEVEL E No Commonality or T5 LEVEL E With B/C/D Commonality
SAME OR COMMON PILOT TYPE RATING START Level A Differences Level B Differences Level C Differences Level D Differences Level E Differences New T/R NO YES Waive T1 NO Pass Fail YES Fail Pass YES NO Pass Fail LEVEL A or B
A B C D E
Appendix 2
e. Comments Solicited. The FSB proposal is circulated with interested parties representing the manufacturer, operators, and other pertinent FAA organizations, such as engineering, flight test, pilots’ associations, and other aviation representatives for comment, relevant information, and recommendations.
f. FSB Final Determinations and Findings.
(1) FSB Determinations. Any comments submitted to the FAA are reconciled, and final FSB determinations are made. Specification of MDRs, example ODRs, acceptable training programs, and other FSB recommendations are completed. Any necessary pilot checking or currency recommendations are identified. Determination of any necessary type rating(s) is made.
(2) Basis for FSB Judgments. FSB judgments are based on review of the applicant’s supporting documentation, proposed ODR tables, test results, and any other pertinent information, such as FAA policies, in-service experience, and results of other similar FSB evaluations. Specifically, FSB report recommendations are based on the following:
(a) Appropriate Data, Evaluation, or Tests. Testing may include aircraft demonstration, simulation tests, device testing, or analysis.
(b) Direct Experience. The industry may have substantial experience with successful operational programs, which can be useful in the assignment of minimum difference level requirements. This experience may include particular training devices, training/checking/currency requirements, and mixed fleet flying.
(c) Indirect Experience. Applicable experience with foreign operators, military programs, or other programs that can establish the suitability of training, checking, or currency standards may be permitted as a means for FSBs to set differences levels.
(d) Applicant and Industry. FSB recommendations are set following solicitation and review of comments.
(3) FSTD Characteristics. Minimum characteristics for FSTDs for training, checking, or currency are noted using 14 CFR part 60 FSTD definitions. When standard criteria for FSTDs are not appropriate for an aircraft, the FSB identifies suitable criteria to be applied and coordinates with the FAA National Simulator Program (NSP). Standard FSTDs applicable to each difference level are shown in Figure 7.
Appendix 2 FIGURE 7. STANDARD METHOD AND FLIGHT SIMULATION TRAINING DEVICE Difference Level Difference Level Definition Methods FSTD A Self-instruction Bulletins Manual revisions Handout material ------------------------- B Aided instruction Slides/video tapes Stand-up instruction CBT ------------------------- C System devices ------------------------- Level 4 or higher FSTD D Maneuver devices ------------------------- Level 6 or higher FSTD E Level C or D FFS or aircraft ------------------------- Level C or D FFS or aircraft
g. FSB Report Preparation Distribution and FAA Application.
(1) Report Preparation and Approval. After MDRs are finalized, the FSB report is prepared and approved. Sufficient background or explanatory material is provided in the report to permit FAA personnel to apply FSB recommendations. The following are elements of an FSB report:
(a) Record of Revisions;
(b) Introduction;
(c) Highlights of Change;
(d) Background;
(e) Acronyms;
(f) Definitions;
(g) Pilot Type Rating;
(h) Related Aircraft;
(i) Pilot Training;
(j) Pilot Checking;
Appendix 2
(k) Pilot Currency;
(l) Operational Suitability;
(m) Miscellaneous;
(n) Appendix 1, Differences Legends;
(o) Appendix 2, MDR Table; and
(p) Appendix 3, Differences Tables.
(2) FSB Report Distribution. The FSB report is posted on FSIMS under Publications for implementation in approval of particular operators’ programs. The FAA technical requirements described in FSB reports are primarily intended for the operators’ use, to develop programs that will be approved by the FAA.
(3) FSB Report Implementation. FSB recommendations and guidance are provided to FAA field offices through FSB reports for each aircraft. These reports are recommendations to FAA offices to identify acceptable criteria for modifying flightcrew member training and qualification programs under the applicable CFR. FSB recommendations set acceptable standards by which FAA inspectors approve, review, correct, or limit individual operators’ programs. The FSB report is the basis for approval of modifications of training, checking, and currency programs approved by the FAA.
h. FSB Report Revision.
(1) General FSB Revision Process. A general revision process is established to update determinations and findings contained in FSB reports. Revisions may be needed annually for active fleets with numerous change requests. Revisions may be needed infrequently for aircraft not undergoing significant change.
(2) Revisions for New Aircraft. When an applicant proposes to develop or add a series of a TC’d aircraft, MDRs and other FSB recommendations are revised to address that series.
If an applicant initiates this action, the procedures noted in paragraph 5 regarding initial determination of minimum training, checking, currency, and type rating requirements are followed. If an operator proposes to add an aircraft that is not covered within an existing FSB report (e.g., a foreign-manufactured aircraft), POIs should consult with the pertinent AEG.
An FSB will determine the best method of addressing the development of the necessary FSB report. This is particularly important for older aircraft fleets in which differences may be significant, but manufacturer support is no longer available; as well as when aircraft that have been used only by foreign operators are imported into the United States.
(3) Revision for Aircraft Modified by Operators. When an aircraft is to be modified by an operator, the POI determines if the change affects MDRs, Differences Tables, or other FSB report recommendations. The criteria for this assessment includes whether or not the difference affects pilot knowledge, skills, or abilities pertinent to flight safety. If a change meets the criteria, the operator should supply the POI with a difference description, an analysis of the effects of the
Appendix 2 difference, and a proposed training program for the difference. The POI evaluates the proposal and if it is acceptable, seeks concurrence from AFS-200. AFS-200 will advise the applicable AEG/FSB. The AEG/FSB may concur with the POI’s analysis or propose other action. If FSB action is proposed and accepted by AFS-200, the AEG will initiate that action through the FSB chairman. The FSB may require that additional information or analysis be provided or that the entire test process or parts thereof be applied. The AEG may authorize the POI to approve assignment of the difference level. Changes to the MDRs will be made through the normal FSB revision process.
6. OPERATOR’S APPLICATION OF FSB RECOMMENDATIONS, PREPARATION, USE, AND REVISION OF ODRs.
a. General.
(1) Process Overview. FSB reports contain MDRs and other recommendations that are applied by FAA offices in approving operators’ flightcrew member training and qualification programs when operating mixed fleets. MDRs are applied through a particular method that identifies specific ODRs and compliance methods. This is necessary so that regardless of which aircraft is flown, uniform training, checking, and currency standards are met within the constraints of the applicable CFR. Paragraph 6 describes operator application of MDRs and other FSB recommendations for training, checking, and currency. This is done through operator preparation and FAA approval of ODRs for each operator. This AC and FSB reports comprehensively address differences and related aircraft differences training, checking, and currency for each aircraft when aircraft are used in mixed fleet flying. ODRs are used to identify credits or constraints between related aircraft. The overall process for operator application of MDRs and development, approval, use, and revision of ODRs is shown in Figure 8.
(2) Availability and Use of FSB Information. FSB recommendations are made available to operators through FAA CHDOs, applicants, and industry trade associations; and posted on FSIMS under Publications or through other sources. When preparing training programs for specific fleets, individual operators apply the requirements of the applicable FSB report.
b. Application of MDRs and Preparation and Use of ODRs.
(1) Need for ODRs. When operating a mixed fleet, operators prepare the necessary ODR table proposals to describe their particular fleet and show compliance methods. This is done to assess effects of differences, plan compliance methods, and obtain POI approval for that operator’s specific program. ODR tables must be prepared and approved by the FAA for each fleet in which FSB recommendations are established.
(2) Operator Responsibilities. The operator’s responsibilities include:
(a) Specification of a base aircraft.
(b) Identification of differences between the aircraft within a mixed fleet.
(c) Preparation of proposed ODR tables.
Appendix 2
(d) Assessment and description of the effects of the differences on training, checking, and currency.
(e) Proposal of training, checking, and currency methods consistent with MDRs, FSB recommendations, and authorized deviations.
(f) Presentation of proposed ODR tables with necessary supporting information to the FAA POI for approval.
(g) Revision of ODR tables when aircraft are introduced, modified, or phased out;
devices change; or MDRs change.
(3) Use of Standard ODR Format. A common format for ODR tables is used to facilitate preparation, review, use, and comparison with MDRs, and ensure consistency of application and approval by POIs. The common format is used in all cases where ODR tables are required, except when only a few minor differences exist and level A applies. In this event, letters between an operator and FAA containing the necessary information and approval may suffice if acceptable to the POI.
(4) Minimum Threshold for ODR Preparation. Within a mixed fleet, a minimum threshold for preparation of ODR tables occurs when there are differences that potentially affect knowledge, skills, or abilities necessary for flight safety. Differences not related to this criterion need not be addressed in ODR tables.
(5) ODR Description and Examples. ODRs are described in paragraph 4. An example of several pages from an ODR table is shown in Figure 3. Figure 3 shows the application of ODRs to address differences and compliance methods. The “Remarks” column depicts differences and the “Flight Characteristics” and “Procedural Change” columns address effects of differences.
c. Selecting Base Aircraft. An operator chooses a base aircraft from one of the aircraft operated. Base aircraft are defined in Appendix 1. Additional information regarding base aircraft selection is in paragraph 7.
d. Identification of Differences and the Analysis of Effects of Those Differences.
Differences must be described between base aircraft and other related aircraft. Differences may also be described from any related aircraft to each other related aircraft. As long as a complete and clear relationship can be drawn from the base aircraft to each other related aircraft, and all MDR requirements are met from the base aircraft to each other related aircraft, there is no need to describe each possible combination of aircraft. This permits a comprehensive identification of differences that exist in the fleet, determines the effects of those differences, and shows compliance methods. Differences are generally organized to follow an operations manual or flight manual to facilitate use and review, and should be categorized by design and maneuvers. Effects of differences are stated in terms of effects on flight characteristics and procedures. Procedures include normal, non-normal, alternate, emergency, and recall procedures, as applicable. Since complete descriptions may be too lengthy for direct incorporation in ODR tables, appendices or references to other operators’ documents may be used to describe differences or effects. Some differences or effects may be repeated in the analysis. For example,
Appendix 2 an FMS difference may be noted in both a navigation design section and maneuver section associated with preflight setup. The objective is to ensure each difference that pertains to pilot training or checking is identified and addressed.
e. Identification of Compliance Methods. Once differences and difference effects are described, methods of comprehensively addressing each difference (compliance methods) are shown. With the difference descriptions, redundancy may occur. The same training or checking compliance item shown for one item may also be associated with and credited for other items.
The objective for description of compliance methods is to show that each difference is addressed in some appropriate way, to show that the method and level chosen is consistent with the FSB MDRs and Differences Tables.
f. When Proposed ODR Compliance Methods Do Not Meet MDRs. If proposed ODR compliance methods do not satisfy MDRs, Differences Tables, or other FSB report constraints, the following alternatives exist:
(1) Differences may be reduced or eliminated by modification of aircraft, systems, or procedures.
(2) Other training methods or devices that fully comply with MDRs, Differences Tables, and other FSB recommendations may be acquired, leased, or otherwise applied.
(3) Pilot assignments may be separated for a fleet so that mixed flying of related aircraft does not occur.
(4) MDR or Differences Tables change proposals may be requested through FAA POIs to the FSB. If FSB-authorized changes to the MDRs or Differences Tables are made, the operator may then apply the revised criteria.
g. Application, Review, and Approval. Paragraph 7 describes the FAA review and approval process. The process is summarized here to facilitate ODR table preparation.
An operator submits the proposed ODR tables and necessary supporting information to the POI to apply for differences and/or related aircraft differences program approval. The supporting information may include any appendices to the ODR tables necessary for evaluation of the proposal, a transition plan if needed, and a proposed schedule for implementation. POIs may also require review of such pertinent and additional information as copies of bulletins, manuals, or other training materials, before they approve proposed ODRs. If devices are proposed that are not approved by the POI or evaluated by the NSP, a review and approval of those training devices may be necessary before ODR approval. Sufficient lead time must be provided to the FAA for review. Lead time depends on such factors as the complexity of program, proposed difference levels, number of related aircraft, other operator precedents already set, and FAA experience with the proposed aircraft, training devices, and methods. Many noncontroversial level A changes can be reviewed and approved in a few days. Complex programs with many related aircraft can require months for review and approval. It is the operator’s responsibility to consult with the POI to ensure that sufficient lead time is provided to review initial submissions or changes. At least 60 days notice is acceptable for most programs. After the operator submits the program proposal, POIs compare the proposed ODR with the FSB report
Appendix 2 recommendations including the MDRs and Differences Tables. POIs consult pertinent FAA policy directives (orders, notices, Safety Alerts for Operators (SAFO), etc.) for interpretations or guidance in accomplishing the review. If ODRs are consistent with authorized deviations and FAA policies and within the constraints of the MDRs and Differences Tables, the POI will approve the operator’s ODR tables and its proposed differences and/or related aircraft differences program. When approved by the FAA, ODRs establish the basis for training, checking, and currency programs for a given fleet for that operator.
h. Transition Period for ODR Implementation. In certain instances, a transitional period, agreed upon by the POI with FSB concurrence, may be necessary to permit operators to continue operations under previously approved programs until they are able to comply with POI-approved FSB recommendations. This is necessary when these recommendations are initially set or revised and they require lead time for program preparation, device acquisition, or to revise previously approved programs.
i. ODR Revision. ODR revisions are initiated when changes occur in an operator’s fleet relating to differences, difference effects, or compliance methods. ODR revisions are appropriate when such changes affect pilot knowledge, skills, or abilities relevant to flight safety. Examples of program changes or factors that may require ODR revision include:
(1) Addition or deletion of aircraft in a fleet;
(2) Modification of base aircraft or related aircraft in a fleet;
(3) Change of base aircraft;
(4) Discontinuation of use, addition of new or modification of training devices referenced by ODRs;
(5) Revision of training methods with a resulting change in compliance levels;
(6) Changes in effects of differences such as revised procedures, performance, or flight characteristics;
(7) FAA revision of MDRs, Differences Tables, or other FSB recommendations;
(8) Adverse in-service experience or training and checking experience that dictates inadequacy of ODRs, MDRs, or other FSB recommendations;
(9) FAA surveillance results, enforcement actions, or failure of an operator to comply with provisions of their approved ODRs; and
(10) Other factors as determined by the POI.
NOTE: Revisions to ODRs are approved using the same procedures as for initial ODRs approval.