6-1 CHAPTER 6. PERFORMANCE EVALUATION
6.1 EVS/SVS/EFVS Performance Demonstration
6.1.1 The performance demonstration, establishing aircraft system compliance with applicable FAA regulations, typically includes bench testing, simulation flight testing, data collection, and data reduction to show that the proposed performance criteria can be met. Minimal performance standards necessitate an evaluation of the system used during anticipated operational scenarios. The performance evaluations should therefore include demonstrations of taxi, take-off, missed approaches, failure conditions, cross wind conditions, and approaches into specific airports as appropriate for the system’s intended function. For EFVS, the applicant should demonstrate performance at the lateral and vertical limits for the type of approach (for example, precision, nonprecision, and approach with vertical guidance) for which operational credit being sought. Appendix D provides sample EFVS flight test considerations.
6.1.2 No specific test procedures are cited, as it is recognized that alternative methods can be used. Alternate procedures can be used if it can be demonstrated that they provide all the information needed to show compliance with the applicable regulations. System performance tests as they relate to operational capability are the most important tests.
Subsystem tests are used as subsystems are added during system buildup to ensure appropriate subsystem performance as it relates to overall system performance.
6.1.3 The applicant should use any of these four general verification methods (refer to AC
25.1329-1C, AC 23.1309-1E, AC 27-1B or AC29-2C as applicable)
6.1.3.1 Analysis. Demonstrate compliance using an engineering analysis.
6.1.3.2 Flight Test. Demonstrate compliance using an aircraft that is fully representative for the purpose of the test in terms of flight deck geometry, instrumentation, alerts, indications, and controls (in the air or on the ground).
6.1.3.3 Laboratory Test. Demonstrate compliance using an engineering bench representative of the final EVS/SVS/CVS/EFVS system being certified.
6.1.3.4 Simulation. Demonstrate compliance using a flight simulator.
6.1.4 Specify the individual verification methods that you use in the certification plan.
Confirm the appropriate certification office agrees with your plan before you begin. For extensions, features, and design decisions not explicitly specified in this AC, conduct human factors evaluations. Conduct these evaluations through analyses, bench, simulation, or flight testing.
6.1.5 Verify both the installed system and the individual system components meet the EVS/SVS/CVS/EFVS criteria described in Chapter 4.
6-2
6.1.6 Demonstrate the system meets performance criteria specified in Chapter 4 by flight test and other appropriate means, which can include use of a flight simulator. An example of a flight test program for EFVS is described in Appendix D in this AC. The flight test program assumes that the guidance system utilized to satisfactorily position the aircraft at the DA/DH or MDA has been separately tested and shown to fully perform its intended function. Testing and data collection to demonstrate this is not part of this document.
6.1.7 Airframe and equipment manufacturer based tests or analyses, as applicable, should be developed and conducted to validate the detailed system criteria. No specific test procedures are cited because alternative methods can be used. You can use alternate procedures if you can demonstrate that they provide all the required information.
System performance tests as they relate to operational capability are the most important tests. Subsystem tests are used as subsystems are added during system buildup to ensure appropriate subsystem performance as it relates to overall system performance.
6.1.8 Conduct an evaluation of the system used during anticipated operational scenarios.
6.1.9 For EFVS testing: If the aircraft meets all of the applicable airworthiness requirements specified in 14 CFR §§ 91.176(a)(1)(i), (a)(2)(iii), (b)(1)(i), and (b)(2)(iii) does not aply to operations conducted in an aircraft issued an experimental certificate under 14 CFR § 21.191 for the purposes of research and development or showing compliance with regulations, provided that the operations can be conducted safely in accordance with operating limitations issued for that purpose
6.2 Performance Demonstration of EFVS Landing System.
6.2.1 Where appropriate for the performance demonstration, the non-visual conditions can be achieved either by natural obscuration or by use of a visibility limiting device in front of the pilot. Caution should be used if the applicant chooses to use a visibility limiting device for system performance demonstrations. Visibility-limiting devices may not adequately simulate low visibility conditions for all EFVS Landing System landing performance demonstrations because of the unrealistically good external visibility outside the HUD FOV and the unrealistic EFVS Landing System image performance in good atmospheric conditions.
6.2.2 The workload associated with the use of EFVS Landing System for approach, landing and rollout must be considered when showing compliance with 14 CFR §§ 23/25/27/29.1523.
6.2.3 Final approach course offsets greater than 3 degrees will be subject to additional flight test evaluation. The maximum allowable final approach course offset is established by flight test. These tests should include the factors related to the offset, such as HUD/EFVS field of view, crosswinds, and the maximum drift angle for a conformal flight path vector.
6-3
6.2.4 The EFVS Landing System image, with superimposed flight symbology, must not mislead, distract or jeopardize the safety of the landing and rollout. Performance shall be demonstrated to be no worse than that normally achieved in visual operations for the specific aircraft for all performance parameters measured. The applicant may propose a valid means for establishing performance normally achieved that the FAA may accept as a benchmark for the EFVS demonstrations.
6.2.5 Benchmark data establishing equivalence to normal visual operations with the specific aircraft would not normally be necessary. However, if flight test results show deviations for the standard criteria listed above, then benchmark data might be used to establish the equivalence of EFVS Landing System operations to normal visual operations for that specific aircraft.
6.2.6 Without requiring exceptional pilot skill, alerting, strength, or workload, the image/symbology should provide the visual cues for the pilot to perform the following items.
6.2.6.1 Speed control within +10/-5 knots of the approach speed, whether manually controlled or with auto-throttle, as proposed by the applicant, up to the point where the throttles are retarded for landing.
6.2.6.2 A smooth transition through flare to landing.
6.2.6.3 Approach, flare, and landing at a normal sink rate for the aircraft.
6.2.6.4 All touchdowns in the touchdown zone. Lateral touchdown performance must be demonstrated to be no worse than that achieved in visual operations for the specific aircraft. Longitudinal touchdown performance must be demonstrated within the touchdown zone which is the first one third, or the first 3000ft, of the usable runway, whichever is more restrictive, and demonstrated to be equivalent or better than that achieved in visual operations for the specific aircraft.
6.2.6.5 Prompt and predictable correction of any lateral deviation away from the runway centerline to smoothly intercept the centerline.
6.2.6.6 Touchdowns with a bank angle that is not hazardous to the airplane.
6.2.6.7 Demonstrated performance of the installed EFVS at representative visibilities for EFVS Approach System and EFVS Landing System operations as described in this document, will determine any additional limitation (for example, crosswind and offset).
6.2.6.8 A normal derotation.
6.2.6.9 Satisfactory and smooth control of the airplane from touchdown to a safe taxi speed.
6-4
6.2.6.10 Satisfactory and smooth control of the path of the airplane along the runway centerline through rollout to a safe taxi speed.
6.2.6.11 A safe go around anytime including up to touchdown in all configurations to be certified.
6.3 Environmental Qualification and Design Assurance.
6.3.1 Installed equipment meets the criteria in RTCA/DO-160G (as revised).
6.3.2 Design Assurance.
6.3.2.1 Follow the criteria of RTCA/DO-178C or higher, Software Considerations in Airborne Systems and Equipment Certification, to conduct software design assurance tests. The version must be current at time of application.
6.3.2.2 Follow the criteria of RTCA/DO-254, Design Assurance Guideline for Airborne Electronic Hardware, to conduct hardware design assurance tests, if applicable. The version must be current at time of application.
Appendix A A-1 APPENDIX A. FAA EFVS 14 CFR COMPLIANCE (PARTIAL LIST) A.1 Airworthiness Standards.
A.1.1 Certification Procedures for Products and Parts (14 CFR part 21).
A.1.2 Normal, Utility, Acrobatic, and Commuter Category Airplanes (14 CFR part 23).
A.1.3 Transport Category Airplanes (14 CFR part 25).
A.1.4 Normal Category Rotorcraft (14 CFR part 27).
A.1.5 Transport Category Rotorcraft (14 CFR part 29).
A.2 EFVS Compliance.
The following requirements address EFVS and some of them could be in addition to HUD criteria and the basic avionics installation. The amount of new test data can be determined by the individual application, availability, and relevance of data.
Table 3. EFVS Acceptable Methods of Compliance 14 CFR § Description Acceptable Method of Compliance 23/25/27/29.143 General controllability and maneuverability Analysis, Simulation, Flight Test 23/25/27/29.251 Vibration and buffeting Flight Test 23/25/27/29.301 Loads Analysis 23/25/27/29.303 Factor of safety Analysis 23/25/27/29.307 Proof of structure Analysis
25.561/25.562(c)(5)) Emergency landing conditions;
Head Injury Criterion (HIC) Analysis 25/27/29.571 and
23.573 Damage-tolerance and fatigue evaluation of structure Analysis
Appendix A A-2 14 CFR § Description Acceptable Method of Compliance
25.571(e) Damage-tolerance and fatigue evaluation of structure(discrete source) evaluation Analysis and/or Test (See Note)
25.581 Lightning protection Analysis 23/25/27/29.601 General Design and Construction Drawing 23/25/27/29.603 Materials Drawing 23/25/27/29.605 Fabrication methods Drawing 23/25/27/29.609 Protection of structure Drawing 23/25/27/29.611 Accessibility provisions Drawing 23/25/27/29.613 Material Strength properties and material design values Drawing, Analysis, Test (See Note) 23/25/27/29.619 Special factors Analysis 23/25/27/29.625 Fitting Factors Analysis
25.629 (d)(8) Aeroelastic stability Analysis, Test
25.631 Bird strike damage Analysis, or Test if necessary.
(Testing should be accomplished unless the analysis shows that the integrity of the original structure has not been compromised by EFVS installation modification.)
23/25/27/29.771 Pilot compartment Flight Test 23/25/27/29.773 Pilot compartment view Flight Test 23/25/27/29.777 Flight deck controls Flight Test
Appendix A A-3 14 CFR § Description Acceptable Method of Compliance 23/25/27/29.1301 Function and installation Ground Test and Flight Test
25.1302 Installed systems and equipment for use by the flightcrew Analysis, Simulation, Flight Test 23/25/27/29.1309 Equip, systems, and install Analysis and/or Test and Design data
25.1316 System lightning protection Analysis and Ground Test
23.1308, and 25/27/29.1317 High-intensity Radiated Fields (HIRF) Protection Ground Test and Flight Test Analysis/Data 23/25/27/29.1321 Arrangement and visibility Ground Test and Flight Test 23/25/27/29.1322 Warning, caution, advisory lights Ground Test and Flight Test
25.1323 Airspeed indicating systems Flight Test 23/27/29.1335 Flight director systems Ground Test and Flight Test 23/27.1351 Electrical Systems and Equipment; General Analysis 25/29.1353 Electrical equipment and installation Analysis 23/25/27/29.1357 Circuit protective devices Analysis and Ground Test 23/25/27/29.1381 Instrument lights Ground Test and Flight Test 23/25/27/29.1419 Ice protection Analysis 23/25/29.1431(a)(c) Electronic equipment Analysis and Ground Test 23/25/27/29.1459(e) Flight data recorders Flight test
Appendix A A-4 14 CFR § Description Acceptable Method of Compliance 23/25/27/29.1501 Operating Limitations and Information; General Flight Test 23/25/27/29.1523 Minimum flight crew Flight Test or Simulation 23/25/27/29.1525 Kinds of operation Flight Test 23/25/27/29.1529 Instructions for Continued Airworthiness Design Data 23/25/27/29.1581 Aircraft flight manual; General Design Data and Flight Test 23/25/27/29.1583 Operating limitations Flight Test Data 23/25/27/29.1585 Operating procedures Flight Test Data
26.47 Holders of and applicants for a supplemental type certificate - alterations and repairs to alterations.
Analysis Note: In some cases, previously approved test data for the aircraft may be resubmitted for compliance, if analysis can establish that the test data remains valid for the aircraft with the new modification.
Appendix B B-1 APPENDIX B. EFVS APPROACH SYSTEM AND EFVS LANDING SYSTEM SAFETY
GUIDANCE B.1 General.
B.1.1 Safety Criteria. Safety criteria for approach and landing systems generally consider four elements: accuracy, continuity, availability, and integrity. These criteria apply to both the external navigation systems as well as airborne navigation equipment.
Trajectory management or flight technical error, which can be interpreted as signal structure contributing to roughness, bends and scalloping of ILS-based guidance, should also be considered. They also define how the airspace and aircraft are integrated together to make a safe approach and landing. The FAA developed, in conjunction with the other governments, definitions related to safety and performance for a landing system.
B.1.2 Transport Category Airplanes. For aircraft systems, 14 CFR § 25.1309 defines the safety requirement and AC 25.1309-1A defines the means for verification. The probability of each catastrophic failure condition must be 10 -9 or less per flight hour considering any mode of flight, any combination of failures causing an unsafe condition, and including the probability of the crew to cope with the failures. The FAA accepts this number to assure a negligible adverse effect on accident rates, and to help reduce them as new systems come on line.
B.1.3 General Aviation or Non-Transport Category Airplanes. Similar information can be found in 14 CFR 23.1309 and AC 23.1309. The relationship among airplane classes, probabilities, severity of failure conditions, and software development assurance levels is found in AC 23.1309-1E.
B.1.4 Rotorcraft. Similar information can be found in 14 CFR § 27.1309 and AC 27-1B and 14 CFR § 29.1039 and AC 29-2C.
B.2 Required Level of Safety.
The required level of safety for any aircraft systems, therefore, depends on the ability of the crew to cope with failures as shown in Table 4, which lists the categories of systems and failure probabilities to meet the safety criteria in 14 CFR parts 23 and 25 airplanes.
For part 23 airplanes, similar information can be found in 14 CFR § 23.1309 and AC
23.1309-1E. The relationship among airplane classes, probabilities, severity of failure conditions and software development assurance levels is found in AC 23.1309-1E.
Appendix B B-2 Table 4. Required Level of Safety, 14 CFR part 25 Aircraft Classification Effect Target Probability P Minor Slight reduction in safety margins or functional capabilities.
Slight increase in crew workload.
Some inconvenience to occupants.
10-3 > P > 1-5 Major Significant reduction in safety margins or functional capabilities.
Significant increase in crew workload or in conditions impairing crew efficiency.
Some discomfort to occupants.
10-5 > P > 1-7 Hazardous/ S e v e r e
M a j o r
Large reduction in safety margins or functional capabilities.
Higher workload or physical distress such that the crew could not be relied upon to perform tasks accurately or completely.
Adverse effects upon occupants including serious or fatal injury to a relatively small number of occupants other than the flight crew.
10-7 > P > 1-9 Catastrophic Failure conditions which would prevent continued safe flight and landing, resulting in multiple fatalities, usually with the loss of the airplane.
10-9 > P B.3 Demonstration.
To meet the safety criteria, the EFVS design will be demonstrated through analysis and engineering tests to preclude any critical failure combinations that can cause hazardously misleading information to be presented to the crew, or which can otherwise subsequently cause an unsafe condition. Failures which are self-evident or made obvious to the crew, and with which they can safely cope, need not be specifically monitored.
B.4 State Data.
The aircraft state data are provided by the standard inertial, air data, and radio guidance sensors.
The HUD or display processor will be required to be at a sufficient level of safety for the aircraft type and application to detect critical random or common faults that could otherwise cause an unsafe condition. The ability to continue the approach below the standard Category I
Appendix B B-3 DA/DH/MDA/MDH into the visual segment therefore is strictly borne by the pilot, a safety factor already accounted for in the safety analysis for standard Category I operations. The example below in Table 5 is a model case and cannot be applied for any specific aircraft.
Functional hazard assessments as required by the FAA are aircraft and systems specific.
Table 5. Example EFVS/HUD Functional Hazard Assessment, 14 CFR part 25 Aircraft, ILS Approaches to 100 ft Height Above Touchdown, RVR 1200 ft FHA NO.
FAILURE
CONDITION PHASE OF FLIGHT PILOT ACTION HAZARD
INDEX OBJECTIVE
1.0 LOSS OF
FUNCTION
1.1 Loss of EFVS imagery on HUD Taxi and Takeoff Enroute Terminal Arrival Final Approach - Above 100 ft. EFVS height above touchdown zone elevation Go-Around Crew would revert to standard headup and/or head down procedures MINOR 10-3
1.2 Loss of EFVS imagery on HUD Final Approach - From 100 ft. height above touchdown zone elevation to Landing Crew would perform a goaround or possibly a minimum flare landing MINOR 10-3
1.3 Loss of HUD symbology Taxi and Takeoff Enroute Terminal Arrival Final Approach - Above 100 ft. height above touchdown zone elevation Go-Around Crew would revert to standard head down procedures MINOR 10-3
1.4 Loss of HUD symbology Final Approach - From 100 ft. height above touchdown zone elevation to Landing Pilot would continue the approach and land MINOR 10-3
Appendix B B-4 FHA NO.
FAILURE
CONDITION PHASE OF FLIGHT PILOT ACTION HAZARD
INDEX OBJECTIVE
1.5 Loss of HUD and EFVS Imagery Final Approach - From 100 ft. height above touchdown zone elevation to Landing
Pilot would continue the approach and land MINOR 10-3
2.0 MISLEADING
INFORMATION
2.1 Misleading EFVS imagery on HUD Taxi Enroute Terminal Arrival Final Approach - Above 100 ft height above touchdown zone elevation Go-Around Crew would revert to standard head down procedures MINOR 10-3
2.2 Misleading EFVS imagery on HUD Final Approach - From 100 ft. height above touchdown zone elevation to Landing Crew would perform a goaround or possibly a minimum flare landing MAJOR 10-5
2.3 Misleading HUD symbology Taxi and Takeoff Enroute Terminal Arrival Final Approach - Above 100 ft. height above touchdown zone elevation Final Approach - From 100 ft. height above touchdown zone elevation to Landing Go-Around Pilot would take appropriate action as defined in AFM for standard displays and HUD
Various failure conditions with the highest hazard index being MAJOR Various failure conditions with the highest objective being 10-5
2.4 Misleading EFVS imagery and HUD symbology Final Approach - Above 100 ft. height above touchdown zone elevation
Copilot would recognize condition using copilot's PFD MAJOR 10-5