AC 20-167B CHAPTER 6. PERFORMANCE EVALUATION
6.1 EVS/EFVS Performance Demonstration.
The performance demonstration used to establish aircraft system compliance with applicable FAA airworthiness criteria typically includes bench testing, simulation, flight testing, data collection, and data reduction to show that the proposed performance criteria can be met. The performance evaluations should therefore include demonstrations of the kinds of operations and those operations’ flight regimes that are stated as the system’s intended functions. For EFVS, the applicant should demonstrate performance at the lateral and vertical limits for the type of approach (for example, precision, non-precision, and approach with vertical guidance) for which operational credit is sought. Appendix B of this AC provides sample EFVS flight test considerations.
6.1.1 This AC does not prescribe specific test procedures. We recognize that alternative methods can be used. In accordance with §§ 21.17 and 21.97, your test procedures and compliance reports must show compliance with the applicable airworthiness criteria.
System performance tests are the most important tests, as they relate to operational capability. Subsystem tests are used to test subsystems during system buildup to ensure appropriate subsystem performance as it relates to overall system performance.
6.1.2 The applicant should use some or all these four general verification methods as necessary (Refer to AC 23.1309-1E, AC 25.1329-1C, AC 27-1B or AC 29-2C CHG 9 as applicable).
6.1.2.1 Analysis.
Demonstrate compliance using an engineering analysis.
6.1.2.2 Flight Test.
Demonstrate compliance using an aircraft that is fully representative for the purpose of the test in terms of flightdeck geometry, instrumentation, alerts, indications, and controls in the air and on the ground.
6.1.2.3 Laboratory Test.
Demonstrate compliance using an engineering bench representative of the final EVS/CVS/EFVS system being approved.
6.1.2.4 Simulation.
Demonstrate compliance using a flight simulator as accepted by the AIR certification branch in the PSCP.
6.1.3 The applicant should specify the individual verification methods used in the certification plan. Confirm that the responsible AIR certification branch agrees with your plan before you begin. For extensions, features, and design decisions not explicitly specified in this AC, you should conduct human factors evaluations. Conduct these 6-1
AC 20-167B evaluations through appropriate combinations of analyses, bench testing, simulation, or flight testing.
6.1.4 The applicant should verify both the installed system and the individual system components meet the EVS/CVS/EFVS criteria described in Chapter 4 of this AC.
6.1.5 The applicant should demonstrate the system meets performance criteria specified in Chapter 4 by flight test and/or other appropriate means, which can include the use of a flight simulator. An example of a flight test program for EFVS is described in appendix C in this AC. The flight test program assumes that the guidance system utilized to satisfactorily position the aircraft at the DA/DH or MDA has been separately tested and shown to fully perform its intended function. Testing and data collection to demonstrate this is not addressed in this AC.
6.1.6 The applicant should conduct an evaluation of the system used during anticipated operational scenarios, including:
6.1.6.1 EFVS Flight Testing.
Unapproved EFVS systems may be operated on test flights provided that the operation is conducted under § 91.176(d) and the aircraft is issued an experimental certificate under § 21.191.
6.1.6.2 Section 21.191, Experimental Certificates.
Identifies a number of purposes for which an experimental certificate can be issued. Not all of those are eligible for the exception in § 91.176(d).
Section 91.176(d) limits the exception purposes to only those listed in §§ 21.191(a) and 21.191(b). Please note, however, that using an unapproved EFVS to conduct an EFVS operation outside this specific scope would still require a waiver from the applicable operating rules.
For the Administrator to determine that the flight test operations can be conducted safely in accordance with § 91.176(d) and all other applicable regulations, the FAA will evaluate the flight test in accordance with FAA Order 8130.2J, Airworthiness Certification of Aircraft. FAA Order 8130.2J contains detailed instructions about how to obtain an experimental certificate (Chapter 4, Sec. 2 and Chapter 10.) It also describes what the applicant’s program letter must contain (appendix C) and lists what operating limitations must be applied to the experimental certificate (appendix D). As a part of the airworthiness certification process, the FAA determines whether the operations proposed by the applicant can be safely conducted. The FAA issues operating limitations when the program letter is approved, and the experimental certificate is issued.
6.2 Performance Demonstration of EFVS Landing System.
Where appropriate for the performance demonstration, the non-visual conditions can be achieved either by natural obscuration or by the use of a visibility-limiting device in 6-2
AC 20-167B front of the pilot. Caution should be used if the applicant chooses to use a visibilitylimiting device for system performance demonstrations. Visibility-limiting devices may not adequately simulate low visibility conditions for all EFVS landing system landing performance demonstrations because of the unrealistically good external visibility outside the HUD FOV and the unrealistic EFVS landing system image performance in good atmospheric conditions.
6.2.1 Under §§ 23.2610, 25.1523, 27.1523, and 29.1523, the applicant must consider the workload associated with the use of EFVS landing system for approach, landing and rollout.
6.2.2 The system may fly final approach courses with offsets greater than 3 degrees. In this case, the applicant should establish the maximum allowable final approach course offset by flight test. The flight test should address factors related to the offset, such as HUD/EFVS FOV, crosswinds, and the maximum drift angle for a conformal FPV.
6.2.3 The EFVS landing system image with superimposed flight symbology must not mislead, distract or jeopardize the safety of the landing and rollout. Performance should be demonstrated to be no worse than that normally achieved in visual operations for the specific aircraft for all performance parameters measured. See §§ 23.2600, 25.771,
27.771, 29.771 and §§ 23.2500, 23.2505, 25.1309, 27.1309, and 29.1309. The applicant may propose a valid means for establishing performance normally achieved that the FAA may accept as a benchmark for the EFVS demonstrations.
6.2.4 Benchmark data establishing equivalence to normal visual operations with the specific aircraft is not normally necessary. However, if flight test results show deviations for the standard criteria listed above, then benchmark data might be necessary to establish the equivalence of EFVS landing system operations to normal visual operations for that specific aircraft.
6.2.5 Without requiring exceptional pilot skill, alerting, strength, or workload, the EFVS image/symbology should provide the visual cues for the pilot to perform the following items:
6.2.5.1 Speed control within +10/-5 knots of the approach speed, whether manually controlled or with auto-throttle, as proposed by the applicant, up to the point where the throttles are retarded for landing.
6.2.5.2 A smooth transition through flare to landing.
6.2.5.3 Approach, flare, and landing at a normal sink rate for the aircraft.
6.2.5.4 Touchdown within the touchdown zone. The lateral touchdown performance should be demonstrated to be no worse than that achieved in visual landings for the specific aircraft.
6.2.5.5 Prompt and predictable correction of any lateral deviation away from the runway centerline to smoothly intercept the centerline.
6-3
AC 20-167B
6.2.5.6 Touchdowns with a bank angle that is not hazardous to the aircraft.
Demonstrated performance of the installed EFVS at representative visibilities for EFVS approach system and EFVS landing system operations as described in this AC will determine any additional limitation (for example, crosswind and offset).
6.2.5.7 A normal de-rotation.
6.2.5.8 Satisfactory and smooth control of the path of the aircraft along the runway centerline through rollout to a safe taxi speed.
6.2.5.9 A safe go-around anytime including up to touchdown in all aircraft approach/landing configurations to be certified.
6-4
AC 20-167B Appendix A APPENDIX A. FAA EFVS 14 CFR COMPLIANCE (PARTIAL LIST) A.1 Airworthiness Standards.
• Certification Procedures for Products and Articles (14 CFR part 21).
• Normal Category Airplanes (14 CFR part 23).
• Transport Category Airplanes (14 CFR part 25).
• Normal Category Rotorcraft (14 CFR part 27).
• Transport Category Rotorcraft (14 CFR part 29).
A.2 EFVS Compliance.
The following requirements address EFVS compliance. These requirements may also be in addition to those requirements applicable to a HUD and the basic avionics installation. The amount of new test data can be determined by the individual application, availability, and relevance of data.
Table A-1. EFVS Acceptable Methods of Compliance 14 CFR Description Acceptable Method of Compliance
23.2135, 25.143, 27.143,
29.143 General controllability and maneuverability Analysis, Simulation, Flight Test
23.2160, 25.251, 27.251,
29.251 Vibration and buffeting Flight Test
23.2210, 23.2230, 25.301,
27.301, 29.301 Loads Analysis
23.2230, 25.303, 27.303,
29.303 Factor of safety Analysis
23.2235, 25.307, 27.307,
29.307 Proof of structure Analysis A-1
AC 20-167B Appendix A 14 CFR Description Acceptable Method of Compliance
25.561m 25.562(c)(5) Emergency landing conditions; Head Injury Criterion (HIC) Analysis
23.2240, 25.571, 27.571,
29.571 Damage-tolerance and fatigue evaluation of structure Analysis
25.571(e) Damage-tolerance and fatigue evaluation of structure(discrete source) evaluation Analysis and/or Test (See Note)
23.2515, 25.581, 27.610,
29.610 Lightning protection Analysis
23.2250, 25.601, 27.601,
29.601 General Design and Construction Drawing
23.2250, 23.2260, 25.603,
27.603, 29.603 Materials Drawing
23.2260, 25.605, 27.605,
29.605 Fabrication methods Drawing
23.2255, 25.609, 27.609,
29.609 Protection of structure Drawing
23.2255, 25.611, 27.611,
29.611 Accessibility provisions Drawing
23.2260, 25.613, 27.613,
29.613 Material Strength properties and material design values Drawing, Analysis, Test (See Note)
23.2265, 25.619, 27.619,
29.619 Special factors Analysis
23.2265, 25.625, 27.625,
29.625 Fitting Factors Analysis
23.2245, 25.629(d)(8) Aeroelastic stability Analysis, Test A-2
AC 20-167B Appendix A 14 CFR Description Acceptable Method of Compliance
23.2320(d), 25.631,
29.631 Bird strike damage Analysis, or Test if necessary. (Testing should be accomplished unless the analysis shows that the integrity of the original structure has not been compromised by EFVS installation modification.)
23.2600, 25.771, 27.771,
29.771 Pilot compartment Flight Test
23.2600(a), 25.773,
27.773, 29.773 Pilot compartment view Flight Test
23.2600, 23.2610, 25.777,
27.777, 29.777 Flight deck controls Flight Test
23.2500, 23.2505,
25.1301, 27.1301, 29.1301 Function and installation Ground Test and Flight Test
25.1302 Installed systems and equipment for use by the flightcrew Analysis, Simulation, Flight Test
23.2500, 23.2505,
25.1309, 27.1309, 29.1309 Equipment, systems, and installations Analysis and/or Test and Design data
25.1316, 27.1316, 29.1316 System lightning protection Analysis and Ground Test
23.2520, 25.1317,
27.1317, 29.1317 High-intensity Radiated Fields (HIRF) Protection Ground Test and Flight Test Analysis/Data
23.2500, 23.2610,
25.1321, 27.1321, 29.1321 Arrangement and visibility Ground Test and Flight Test
23.2605, 25.1322,
27.1322, 29.1322 Warning, caution, advisory lights Ground Test and Flight Test
25.1323, 27.1323, 29.1323 Airspeed indicating systems Flight Test A-3
AC 20-167B Appendix A 14 CFR Description Acceptable Method of Compliance
23.2500 23.2505, 23.2510,
23.2600, 23.2605,
27.1335, 29.1335 Flight director systems Ground Test and Flight Test
23.2525, 23.2500,
23.2525, 23.2605,
25.1351, 27.1351, 29.1351 Electrical Systems and Equipment;
General Analysis
25.1353, 29.1353 Electrical equipment and installation Analysis
23.2500, 23.2505,
23.2510, 23.2525,
25.1357, 27.1357, 29.1357 Circuit protective devices Analysis and Ground Test
23.2500, 23.2600,
25.1381, 27.1381, 29.1381 Instrument lights Ground Test and Flight Test
23.2165(a)(1), 23.2540(a),
23.2600(a), 25.1419,
27.1419, 29.1419 Ice protection Analysis
23.2510, 25.1431, 29.1431 Electronic equipment Analysis and Ground Test
23.1549(e), 25.1459(e),
27.1459(e), 29.1459(e) Flight data recorders Flight test
23.2610, 25.1501,
27.1501, 29.1501 Operating Limitations and Information;
General Flight Test
23.2610, 25.1523,
27.1523, 29.1523 Minimum flightcrew Flight Test or Simulation
23.2610, 25.1525,
27.1525, 29.1525 Kinds of operation Flight Test
23.1529, 25.1529,
27.1529, 29.1529 Instructions for Continued Airworthiness Design Data A-4
AC 20-167B Appendix A 14 CFR Description Acceptable Method of Compliance
23.2620, 25.1581,
27.1581, 29.1581 Aircraft flight manual; General Design Data and Flight Test
23.2620, 25.1583,
27.1583, 29.1583 Operating limitations Flight Test Data
23.2620, 25.1585,
27.1585, 29.1585 Operating procedures Flight Test Data
26.47 Holders of and applicants for a supplemental type certificate - alterations and repairs to alterations.
Analysis A-5
AC 20-167B Appendix B APPENDIX B. SA MPLE EFVS FLIGHT TEST CONSIDERATIONS B.1 General Flight Test Considerations.
The objectives of the flight test program are to demonstrate the installed EFVS performs its intended function when installed, and to document that the installed EFVS is eligible for operational approval.
B.1.1 Testing should include all phases of flight, approach types, approach course geometries, and aircraft configurations for which the applicant seeks to operate the EFVS. The EFVS minimum performance standards require the assessment of the EFVS display when used in conjunction with the head up flight indications required in § 91.176 and listed in paragraph 4.4.1.3 of this AC. For effective post-flight analysis and evaluation of the results, past experience has shown that recorded time-stamped video of exactly what is displayed on the HUD is recommended.
B.1.2 In general, the test program should demonstrate that the EFVS image, with superimposed flight symbology, does not mislead, distract or jeopardize the safety of the approach, landing and rollout. Without requiring exceptional pilot skill, alerting, strength, or workload, the image/symbology should allow and provide the visual cues for the pilot to perform the EFVS approach or EFVS landing as described in § 91.176.
Minimum performance standards require the assessment of the HUD/EFVS approach system display when used in conjunction with the flight instrumentation required in § 91.176 and listed in paragraph 4.4.1.3 of this AC.
B.1.3 The flight test should demonstrate that the EFVS display of imagery and HUD symbology are readable and usable without mutual interference. Flight testing should demonstrate the brightness and contrast levels of the imagery and the symbology can be effectively controlled and provide satisfactory contrast between the two. Since flightdeck and external light levels can affect the readability of the EFVS display and can change significantly throughout the approach and landing operation, flight testing should demonstrate that control of display brightness and contrast in dynamic conditions does not require excessive workload for the pilot. Additionally, flight tests should demonstrate that controls can clear the EFVS imagery from the HUD in the event that the visual field becomes unusable due to system malfunction or external environmental conditions.
B.1.4 Environmental conditions should be chosen to exercise both automatic and manual control of items such as brightness, contrast, and gain, and any other parameters affecting the image displayed to the pilot.
B.1.5 An applicant may reuse previously collected certification flight test data to meet the criteria specified in this AC if the applicant can show the previously collected certification data is relevant to the current certification project. The FAA and the applicant will agree on the reuse of any previously collected certification flight test data.
For example, if an applicant is certifying a previously certified EFVS approach system as an EFVS landing system, and the applicant can show test data collected to certify the B-1
AC 20-167B Appendix B EFVS approach system meets the criteria specified in this AC, the applicant may only need to conduct the additional EFVS landing system specific test points listed in this appendix.
Figure B- 1. Depiction of Visual Advantage B.2 Evaluation of Visual Advantage.
In addition to demonstrating system safety, the flight test program should demonstrate that the installed EFVS is capable of successfully completing the relevant § 91.176(a) or 91.176(b) operation in a representative sampling of weather and environmental conditions with flight visibilities lower than the published instrument approach visibility minimums as described in section B.3 of this AC.
B.2.1 A successful demonstration of visual advantage in accordance with appendix B is sufficient to support certification of the installed EFVS. However, while not required for certification of the EFVS, quantification of the demonstrated EFVS visual advantage is required for some EFVS operational approvals.
B.2.2 The demonstration of a quantified visual advantage is necessary to authorize the provisions in OpSpec/MSpec/Letter of Authorization (LOA) C048 which allow 14 CFR part 121, 125, and 135 certificate holders to use EFVS-equipped aircraft to:
B.2.2.1 Release a flight under instrument flight rules (IFR) when the forecast weather is equal to or greater than the authorized minimums for use with an EFVS. (§§ 121.613, 125.361, and 135.219); and B-2
AC 20-167B Appendix B B.2.2.2 Continue an approach when the weather is reported to be equal to or greater than authorized minimums for use with an EFVS. (§§ 121.651,
125.325, 125.381, and 135.225.)
B.2.3 If the applicant intends the system to be used for EFVS operations that require OpSpec/MSpec/LOA C048, it is recommended that the applicant also demonstrate and document the EFVS’s quantified visual advantage during the certification testing of the EFVS. Further guidance on how to quantify the visual advantage of the EFVS is found in appendix C.
B.3 Minimum Flight Test Conditions Observed.
Testing should demonstrate fault-free approaches for EFVS approach systems or approach and landings for EFVS landing systems in as many of the conditions listed below as practicable and as applicable to the system’s intended function. Additionally, a sufficient number of approaches will be needed in a representative sampling of weather and environmental conditions with flight visibilities lower than the published instrument approach visibility minimums to demonstrate that the EFVS can provide sufficient enhanced flight visibility to complete the EFVS operation. Past experience has shown that the applicants with a new EFVS should plan for no less than 50 fault-free approaches for EFVS approach systems or approach and landings for EFVS landing systems. Include go-around performance in testing. Typically, at least five satisfactory go-arounds are included in testing.
B.3.1 The applicant and the FAA will agree on the minimum number of successfully demonstrated test approaches.
B.3.2 Testing of the EFVS systems should be conducted in as many of the environmental conditions listed below as practical and applicable to the system’s intended function.
The environmental conditions should be chosen to exercise both the automatic and manual control of items such as brightness, contrast, gain, and any other parameter that affects the image displayed to the pilot. The actual environmental conditions used for the flight test program must be agreed to by the FAA. Address the conditions in the following subsection when developing the flight test program:
B.3.2.1 EFVS approach and EFVS landing systems should be tested under the following conditions:
• Night visual flight rules (VFR) conditions over various topography (urban, rural, snow covered, etc.);
• Day and night IFR conditions over various topography;
• Representative levels of rainfall;
• Representative levels of snowfall;
• Representative levels of fog;
• Haze;
B-3