Appendix E E-1 APPENDIX E. PROCEDURE FOR EVALUATING MATERIAL CONTRIBUTION TO SAFETY
OR IMPRACTICALITY OF APPLYING LATEST REQUIREMENTS
TO A CHANGED PRODUCT E.1 Introduction.
E.1.1 The basic principle of enhancing the level of safety of changed aeronautical products is to apply the latest regulations for significant design changes to the greatest extent practical. In certain cases, the cost of complying fully with a later regulation may not be commensurate with the small safety benefit achieved. These factors form the basis where compliance with the latest standard may be considered impractical, thereby allowing compliance with an earlier regulation. This appendix gives one method of determining if compliance with a later regulation is impractical; however, it does not preclude the use of other methods for improving the safety of aeronautical products.
E.1.2 The FAA recognizes that other procedures can be used and have historically been accepted on a case-by-case basis. The acceptance of results through the use of these procedures may vary from State to State. Consequently, they may not be accepted through all bilateral certification processes. Regardless of which method is used, the process must show that a proposed certification basis is able to achieve a positive safety benefit for the overall product.
E.1.3 In regard to impractical, any method used must encourage incorporating safety enhancements that will have the most dramatic impact on the level of safety of the aircraft while considering effective use of resources. This important point is illustrated graphically in figure E-1 below. This figure notionally shows the interrelation between the total resources required for incorporating each potential safety enhancement with the corresponding net increase in safety benefit.
Appendix E E-2 Figure E-1. Safety Benefits versus Resources
E.1.4 Typically, you will find that, for impractical, there are proposals that can achieve a positive safety benefit that are resource effective. Conversely, there are proposals that may achieve a small safety benefit at the expense of a large amount of resources to implement. Clearly, there will be a point where a large percentage of the potential safety benefit can be achieved with a reasonable expenditure of resources. The focus of the methods used should be to determine the most appropriate regulatory standards relative to the respective incremental cost to reach this point.
E.1.5 This appendix provides procedural guidance for determining the material contribution to the level of safety, or the practicality of applying a requirement at a particular amendment level to a changed product. The procedure is generic in nature and describes the steps and necessary inputs that you can use on any project to develop a position.
E.1.6 The procedure is intended to be used, along with good engineering judgment, to evaluate the relative merits of a changed product complying with the latest regulations.
It provides a means, but not the only means, for you to present your position regarding an exception under § 21.101(b)(3).
E.1.7 The certification basis for a change to a product will not be at an amendment level earlier than the existing certification basis or any requirement found in §§ 23.2, 25.2,
27.2, and 29.2, or part 26, that is related to the change.
Appendix E E-3 E.2 Procedure for Evaluating Material Contribution or Impracticality of Applying Latest Requirements to a Changed Product.
The following are steps to determine the material contribution or impracticality of applying a requirement at a particular amendment level.
E.2.1 Step 1: Identify the Regulatory Change being Evaluated.
In this step, document— E.2.1.1 The specific requirement (e.g., § 25.365), E.2.1.2 The amendment level of the existing certification basis for the requirement, and E.2.1.3 The latest amendment level of the requirement.
E.2.2 Step 2: Identify the Specific Hazard that the Requirement Addresses.
E.2.2.1 Each requirement and subsequent amendments address a hazard or hazards. In this step, the specific hazard(s) is identified. This identification will allow for a comparison of the effectiveness of amendment levels of the regulation at addressing the hazard.
E.2.2.2 In many cases, the hazard and the cause of the hazard will be obvious.
When the hazard and its related cause are not immediately obvious, it may be necessary to review the preamble of the regulation. It may also be helpful to discuss the hazard with the responsible FAA office.
E.2.3 Step 3: Review the Consequences of the Hazard(s).
E.2.3.1 Once the hazard is identified, it is possible to identify the types of consequences that may occur due to the hazard. More than one consequence can be attributed for the same hazard. Typical examples of consequences would include, but are not limited to—
• Incidents where only injuries occurred,
• Accidents where a total hull loss occurred,
• Accidents where less than 10 percent of the passengers died,
• Accidents where 10 percent or more passengers died, and
• Engine- and propeller-specific hazards.
E.2.3.2 The preamble to the regulation may provide useful information regarding the consequences of the hazard that the requirement addresses.
Appendix E E-4 E.2.4 Step 4: Identify the Historical and Predicted Frequency of Each Consequence.
E.2.4.1 Another source for determining impracticality is the historical record of the consequences of the hazard that led to a requirement or an amendment to a requirement. From these data, a frequency of occurrence for the hazard can be determined. It is important to recognize that the frequency of occurrence may be higher or lower in the future. Therefore, it also is necessary to predict the frequency of future occurrences.
E.2.4.2 More than one consequence can be attributed for the same hazard.
Therefore, when applicable, the combination of consequences and frequencies of those consequences should be considered together.
E.2.4.3 The preamble of the regulation may provide useful information regarding the frequency of occurrence.
E.2.5 Step 5: Determine How Effective Full Compliance with the Latest Amendment of the Requirement would be at Addressing the Hazard.
E.2.5.1 When each amendment is issued, it is usually expected that compliance with the requirement would be completely effective at addressing the associated hazard for the designs and technology envisioned at the time. It is expected the hazard would be eliminated, avoided, or mitigated.
However, experience has shown that this may not always be the case. It is also possible that earlier amendment levels may have addressed the hazard but were not completely effective. A product may also contain a design feature(s) that provides a level of safety that approaches the latest regulations, yet is not fully compliant with the latest regulations.
Therefore, in comparing the benefits of compliance with the existing certification basis to the latest amendment level, it is useful to estimate the effectiveness of both amendment levels in dealing with the hazard.
E.2.5.2 It is recognized that the determination of levels of effectiveness is normally of a subjective nature. Therefore, prudence should be exercised when making these determinations. In all cases, it is necessary to document the assumptions and data that support the determination.
E.2.5.3 The following five levels of effectiveness are provided as a guideline:
1. Fully effective in all cases. Compliance with the requirement eliminates the hazard or provides a means to avoid the hazard completely.
2. Considerable potential for eliminating or avoiding the hazard.
Compliance with the requirement eliminates the hazard or provides a means to completely avoid the hazard for all probable or likely cases, but it does not cover all situations or scenarios.
Appendix E E-5
3. Adequately mitigates the hazard. Compliance with the requirement eliminates the hazard or provides a means to avoid the hazard completely in many cases. However, the hazard is not eliminated or avoided in all probable or likely cases. Usually this action only addresses a significant part of a larger or broader hazard.
4. Hazard only partly addressed. In some cases, compliance with the requirement partly eliminates the hazard or does not completely avoid the hazard. The hazard is not eliminated or avoided in all probable or likely cases. Usually this action only addresses part of a hazard.
5. Hazard only partly addressed but action has negative side effect.
Compliance with the requirement does not eliminate or avoid the hazard or may have negative safety side effects. The action is of questionable benefit.
E.2.5.4 If it is determined that compliance with the latest regulations does not contribute materially to the product’s level of safety, skip Step 6 of this appendix and go directly to Step 7 to document the conclusion. If it is determined that complying with the latest amendment of the regulation contributes materially to the product’s level of safety, continue to Step 6 of this appendix.
E.2.6 Step 6: Determine the Incremental Resource Costs and Cost Avoidance.
E.2.6.1 There is always cost associated with complying with a requirement. This cost may range from minimal administrative efforts to the resource expenditures that support full-scale testing or the redesign of a large portion of an aircraft. However, there are also potential cost savings from compliance with a requirement. For example, compliance with a requirement may avoid aircraft damage or accidents and the associated costs to the manufacturer for investigating accidents. Compliance with the latest amendment of a requirement may also help a foreign authority certificate a product.
E.2.6.2 When determining the impracticality of applying a requirement at the latest amendment level, only the incremental costs and safety benefits from complying with the existing certification basis should be considered.
E.2.6.3 When evaluating the incremental cost, it may be beneficial for you to compare the increase in cost of complying with the latest requirements to the cost of incorporating the same design feature in a new airplane. In many cases, an estimate for the cost of incorporation in a new airplane is provided in the regulatory evaluation by the FAA, which was presented when the corresponding regulation was first issued. Incremental costs of retrofit/incorporation on existing designs may be higher than that for production. Examples of costs may include, but are not limited to, the following:
Appendix E E-6 Costs The accuracies of fleet size projections, utilization, etc., may be different than that experienced for derived product designs and must be validated.
• Labor: Work carried out in the design, fabrication, inspection, operation, or maintenance of a product for the purpose of incorporating or demonstrating compliance with a proposed action.
Non-recurring labor requirements, including training, for the applicant supporting development and production of the product, should be considered.
• Capital: Construction of new, modified, or temporary facilities for design, production, tooling, training, or maintenance.
• Material: Cost associated with product materials, product components, inventory, kits, and spares.
• Operating Costs: Costs associated with fuel, oil, fees, training, and expendables.
• Revenue/Utility Loss: Costs resulting from earning/usage capability reductions from departure delays, product downtime, and performance loss due to seats, cargo, range, or airport restrictions.
• The cost of changing compliance documentation and/or drawings in itself is not an acceptable reason for an exception.
Cost Avoidance.
• Avoiding cost of accidents, including investigation of accidents, lawsuits, public relations activities, insurance, and lost revenue.
• Foreign Certification: Conducting a single effort that would demonstrate compliance to the requirements of most certifying Authorities, thus minimizing certification costs.
E.2.7 Step 7: Document the Conclusion.
With the information from previous steps documented and reviewed, the applicant’s position and rationale regarding whether complying with the latest regulations contributes materially to the product’s level of safety or its practicality can be documented. The FAA records the determination of whether the conditions for the proposed exception were met. That determination is based on the information and analysis provided by the applicant in the preceding steps. If the determination to grant the exception is based on the product’s design features, those features are documented at a high level in the TCDS. Documentation in the TCDS is required so that the features are maintained during subsequent changes to the product, therefore, maintaining the product’s agreed level of safety. If the results of this analysis are inconclusive, then further discussions with the FAA are warranted.
Appendix E E-7 E.3 Examples of How to Certify Changed Aircraft.
The following examples illustrate the typical process an applicant follows. The process will be the same for all product types.
E.3.1 Example 1: § 25.963, Fuel Tank Access Covers.
This example is part of a significant change to a transport airplane that increases passenger payload and gross weight by extending the fuselage 20 feet. To accommodate the higher design weights and increased braking requirements and to reduce runway loading, the applicant will change the landing gear from a two-wheel to four-wheel configuration; this changes the debris scatter on the wing from the landing gear. The FAA will require that the new model airplane comply with the latest applicable regulations based on the date of application.
The wing will be strengthened locally at the side of the body and at the attachment of engines and landing gear, but the applicant would not like to alter wing access panels and the fuel tank access covers. Although the applicant recognizes that the scatter pattern and impact loading on the wing from debris thrown from the landing gear will change, the applicant proposes that it would be impractical to redesign the fuel tank access covers.
Note: Sections 121.316 and 21.21(b)(2) may be additional reasons the FAA would require compliance with § 25.963(e), regardless of the significant determination.
E.3.1.1 Step 1: Identify the Regulatory Change being Evaluated.
The existing certification basis of the airplane that is being changed is part 25 prior to Amendment 25-69. Amendment 25-69 added the requirement that fuel tank access covers on transport category airplanes be designed to minimize penetration by likely foreign objects, and that they be fire resistant.
E.3.1.2 Step 2: Identify the Specific Hazard that the Regulation Addresses.
Fuel tank access covers have failed in service due to impact with high-energy objects such as failed tire tread material and engine debris following engine failures. In one accident, debris from the runway impacted a fuel tank access cover, causing its failure and subsequent fire, which resulted in fatalities and loss of the airplane. Amendment 25-69 will ensure that all access covers on all fuel tanks are designed or located to minimize penetration by likely foreign objects, and that they are fire resistant.
E.3.1.3 Step 3: Review the History of the Consequences of the Hazard(s).
There have been occurrences with injuries and with more than 10 percent deaths.
Appendix E E-8 E.3.1.4 Step 4: Identify the Historical and Predicted Frequency of Each Consequence.
In 200 million departures of large jets—
• One occurrence with more than 10 percent deaths, and
• One occurrence with injuries.
There is no reason to believe that the future rate of accidents will be significantly different than the historical record.
E.3.1.5 Step 5: Determine How Effective Full Compliance with the Latest Amendment of the Regulation would be at Addressing the Hazard.
Considerable potential for eliminating or avoiding the hazard. Compliance with Amendment 25-69 eliminates the hazard or provides a means to avoid the hazard completely for all probable or likely cases. However, it does not cover all situations or scenarios.
E.3.1.6 Step 6: Determine Resource Costs and Cost Avoidance.
Costs.
• For a newly developed airplane, there would be minor increases in labor resulting from design and fabrication.
• There would be a negligible increase in costs related to materials, operating costs, and revenue utility loss.
Cost Avoidance.
• There were two accidents in 200 million departures. The applicant believes that it will manufacture more than 2,000 of these airplanes.
These airplanes would average five flights a day. Therefore, statistically there will be accidents in the future if the hazard is not alleviated. Compliance will provide cost benefits related to avoiding lawsuits, accident investigations, and public relation costs.
• There are cost savings associated with meeting a single certification basis for FAA and foreign regulations.
E.3.1.7 Step 7: Document the Conclusion.
It is concluded that compliance with the latest regulation increases the level of safety at a minimal cost to the applicant. Based on the arguments and information presented by the applicant through the issue paper process, the FAA determined that meeting the latest amendment would be practical. Additionally, operators are required to comply with § 25.963(e) under § 121.316. The FAA has also found that fuel tank access covers that are not impact resistant and fire resistant located where a strike is likely is an unsafe feature or characteristic, which precludes issuance of a type certificate under § 21.21(b)(2).
Appendix E E-9 E.3.2 Example 2: § 25.365, Pressurized Compartment Loads.
This example is a passenger-to-freighter conversion STC. This change affects the floor loads on the airplane as well as the decompression venting.
E.3.2.1 Step 1: Identify the Regulatory Change being Evaluated.
The existing certification basis of the airplane that is being changed includes § 25.365 at Amendment 25-00. The initial release of § 25.365 required that the interior structure of passenger compartments be designed to withstand the effects of a sudden release of pressure through an opening resulting from the failure or penetration of an external door, window, or windshield panel, or from structural fatigue or penetration of the fuselage, unless shown to be extremely remote.
Amendment 25-54 revised § 25.365 to require that the interior structure be designed for an opening resulting from penetration by a portion of an engine, an opening in any compartment of a size defined by § 25.365(e)(2), or the maximum opening caused by a failure not shown to be extremely improbable. The most significant change is the “formula hole size” requirement introduced into § 25.365(e)(2) at Amendment 25-54.
Amendment 25-71/72 (Amendments 25-71 and 25-72 are identical) extended the regulation to all pressurized compartments, not just passenger compartments, and to the pressurization of unpressurized areas.
Pressurization of unpressurized areas had previously been identified as an unsafe feature under § 21.21(b)(2).
Amendment 25-87 redefined the pressure differential load factor that applies above an altitude of 45,000 feet. Compliance with Amendment 25-87 is not affected since the airplane does not operate above an altitude of 45,000 feet. The applicant proposes to meet the “pressurization into unpressurized areas” requirement introduced in Amendment 25-71/72.
The applicant does not propose to comply with the formula hole size requirement introduced in § 25.365(e)(2) at Amendment 25-54.
E.3.2.2 Step 2: Identify the Specific Hazard that the Regulation Addresses.
The hazard is a catastrophic structure and/or system failure produced by a sudden release of pressure through an opening in any compartment in flight. This opening could be caused by an uncontained engine failure, an opening of a prescribed size due to the inadvertent opening of an external door in flight, or an opening caused by a failure not shown to be extremely improbable. The opening could be produced by an event that has yet to be identified.
E.3.2.3 Step 3: Review the History of the Consequences of the Hazard(s).
There have been occurrences with injuries, with less than 10 percent deaths and with more than 10 percent deaths.
Appendix E E-10 E.3.2.4 Step 4: Identify the Historical and Predicted Frequency of Each Consequence.
In 200 million departures of large jets—
• Two occurrences with more than 10 percent deaths,
• One occurrence with less than 10 percent deaths, and
• One occurrence with injuries.
There is no reason to believe that the future rate of accidents will be significantly different than the historical record.
E.3.2.5 Step 5: Determine How Effective Full Compliance with the Latest Amendment of the Regulation would be at Addressing the Hazard.
Compliance with the latest amendment eliminates the hazard or provides a means to avoid the hazard completely.
Design changes made to the proposed airplane bring it closer to full compliance with § 25.365 at Amendment 25-54. The original airplane was shown to meet the requirements for a hole size of 1.1 square feet.
Amendment 25-54 would require a hole size of 5.74 square feet, and the current reinforcements for the converted airplane can sustain a hole size of
3.65 square feet in the forward area and 2.65 square feet at the aft area.
This is 3.1 and 2.4 times, respectively, better than the original design condition of Amendment 25-0 and is a significant improvement over the worldwide passenger fleet in service.
E.3.2.6 Step 6: Determine Resource Costs and Cost Avoidance.
Costs.
There would be savings in both labor and capital costs if compliance were shown to Amendment 25-0 instead of Amendment 25-54. Major modifications to the floor beams would be necessary to meet the formula hole size requirement in Amendment 25-54.
Cost Avoidance.
There were four accidents in 200 million departures. The applicant believes that it will manufacture more than 2,000 of these airplanes. These airplanes would average two flights a day. Therefore, statistically there will be accidents in the future if the hazard is not alleviated. Compliance will provide cost benefits related to avoiding lawsuits, accident investigations, and public relation costs.
There are cost savings associated with meeting a single certification basis for FAA and foreign regulations.
E.3.2.7 Step 7: Document the Conclusion Regarding Practicality.
The design complies with § 25.365 at Amendments 25-0, 25-71/72, and 25-87, and it is nearly in full compliance with Amendment 25-54. The
Appendix E E-11 design would adequately address the hazard at an acceptable cost.
Therefore, based on arguments of impracticality discussed in an issue paper, the FAA accepts the applicant’s proposal to comply with § 25.365 at Amendment 25-0.
E.3.3 Example 3: § 25.981, Fuel Tank Ignition Prevention.
This example is part of a significant change to a transport airplane that increases passenger payload and gross weight by extending the fuselage 20 feet. To accommodate the longer fuselage, the applicant will modify systems wiring installations; this includes changing fuel tank system wiring. The new model airplane will be required to comply with the latest applicable regulations based on the date of application.
E.3.3.1 Step 1: Identify the Regulatory Change Being Evaluated.
The existing certification basis of the airplane that is being changed is part 25 prior to Amendment 25-102 but includes Amendment 25-40.
Note: If the original certification basis does not include Amendment 25-40, the certification basis should be considered not adequate for fuel tank ignition prevention.
The 2001 Fuel Tank Safety (FTS) rule adopted Amendment 25-102 to add explicit requirements in § 25.981(a)(3) for demonstrating that the design precludes fuel tank ignition sources that was required but had in several cases not been properly applied in demonstrating compliance to §§ 25.901 and 25.1309. Amendment 25-102, § 25.981(b), added a requirement to develop fuel tank system airworthiness limitations to maintain the ignition prevention features of the design. Section H25.4, Amendment 25-102, requires including those fuel tank system airworthiness limitations in the Airworthiness Limitations section of the Instructions for Continued Airworthiness (ICA).
Since the FAA policy for performing the failure analysis to demonstrate compliance with §§ 25.901 and 25.1309 at Amendment 25-40 and 25-46 was adopted in the explicit fuel tank ignition prevention failure analysis requirements of § 25.981(a)(3), the incremental requirement for demonstrating compliance to the ignition prevention requirements of Amendment 25-102 is to develop and implement the fuel tank system airworthiness limitations instead of developing Certification Maintenance Requirements in accordance with § 25.901(b)(2) at Amendments 25-40 through 25-46 and AC 25-19A.
E.3.3.2 Step 2: Identify the Specific Hazard that the Regulation Addresses.
The FAA issued the 2001 FTS rule to preclude fuel tank ignition sources because of a history of fuel tank explosions. The catastrophic TWA Flight 800 in-flight fuel tank explosion on July 17, 1996, caused the death of all 230 people onboard.
Appendix E E-12 E.3.3.3 Step 3: Review the History of the Consequences of the Hazard(s).
There have been occurrences with injuries, with more than 10 percent deaths, less than 10 percent deaths, and no deaths.
E.3.3.4 Step 4: Identify the Historical and Predicted Frequency of Each Consequence.
The 1998 Aviation Rulemaking Advisory Committee Fuel Tank Harmonization Working Group report documents the historical frequency of fuel tank explosions as 16, which caused a total of 539 fatalities.
There have been two additional fuel tank explosions since that report was issued:
• March 3, 2001—Thai Airways International Flight 114 experienced a fuel tank explosion on the ground that caused 1 fatality and 3 serious injuries. The explosion and subsequent fire destroyed the airplane.
• May 4, 2006—A Malaysia Airlines Boeing 727 experienced a wing tank low pressure explosion during ground operations. There was no fire and no injuries. The wing structure suffered significant damage.
There is no reason to believe that the future rate of accidents will be significantly different than the historical record if fuel tank system airworthiness limitations are not included in the ICA as is permitted in earlier amendment levels.
E.3.3.5 Step 5: Determine How Effective Full Compliance with the Latest Amendment of the Regulation would be at Addressing the Hazard.
Considerable potential for eliminating or avoiding the hazard.
In the 2008 Fuel Tank Flammability Reduction (FTFR) rule, the FAA estimated compliance with the ignition prevention requirements of Amendment 25-102 together with the fuel tank ignition prevention airworthiness directives issued as a result of the Special Federal Aviation Regulation number 88 reviews resulted in the range of effectiveness in preventing fuel tank explosions between 25 to 75 percent with a median value of 50 percent (73 FR 42449).
E.3.3.6 Step 6: Determine Resource Costs and Cost Avoidance.
Costs.
• For newly developed designs, there would be minor increases in costs resulting from identification and implementation of fuel tank system airworthiness limitations.
• There would be no increase in costs related to materials, operating costs, and revenue utility loss.