Revision B 24 13 DEMONSTRATED RELIABILITY.
13.1 Required Reliability.
Section 450.101(c)(3) states that one way in which an operator can protect against a high consequence event in uncontrolled areas for each phase of flight is by establishing that the launch or reentry system has sufficient demonstrated reliability as agreed to by the Administrator based on CEC criteria during that phase of flight. The FAA will use the demonstrated reliability and average ground consequence results from fatal accidents involving U.S. civil aviation aircraft with standard airworthiness certificates to establish what constitutes sufficient demonstrated reliability to protect against a high consequence event based on CEC as described in this section. More specifically, sufficiently high demonstrated reliability will be evaluated based on the principles outlined in paragraphs 13.3.1 and 13.3.2 of this AC.
13.2 Demonstrating Reliability.
An applicant should demonstrate reliability based on history of the vehicle being flown, following equivalent practices to type certification for aircraft. Small modifications of a vehicle may be acceptable without being considered a new vehicle. Based on current licensed operations, the FAA anticipates that, initially, only flight phases where an aircraft carries a rocket would be able to meet the demonstrated reliability standard, and the aircraft would need an airworthiness certificate and extensive flight history.
However, it is anticipated that in the future, rocket systems will also develop history and certification practices that will enable commensurate levels of reliability to be demonstrated. Because demonstrated reliability provides an alternative to flight abort when CEC is greater than 1 x 10-3, it is appropriate to assess it consistent with the approach to flight abort and FSS reliability, which depends on CEC with a 1 x 10-2 threshold. For example, a binomial approach would require data from at 87,000 flights without a failure to demonstrate a reliability corresponding to the 8E-6 failure rate (identified in paragraph 13.3.3 of this AC) at the 50% confidence level.9
13.3 Existing Aviation Data.
Existing data is available and provides findings necessary to relate several classes of aircraft to consequences and reliability requirements. Reliability is simply the converse of failure probability, so reliability = 100% minus failure probability. Consequences for aircraft are not, however, available in terms of ground casualties. This information is only available in terms of ground fatalities. Conditional expected fatalities, 10 CEF, is
9 For example, see equation 13.9 in “An Introduction to Reliability and Maintainability Engineering” by Charles E.
Ebeling, Waveland Press, Inc., Long Grove, IL, 2019, ISBN-13: 978-1577666257.
10 CEF represents conditional expected fatalities and is used to measure the mean number of fatalities predicted to occur given an event with a probability of 1. The FAA found that about one ground fatality resulted on average from one hundred fatal accidents involving U.S.-registered aircraft operated under Part 91 between 1984 and 2013 based on NTSB data. A comparison of CEC to CEF is appropriate here because the CEF values cited here are empirical results from aviation accidents, whereas the CEC values used here are the results of physics-based computer simulations for launch and reentry operations. In addition, the differences between aviation and space operations justify some margin in the tolerability of the conditional risks predicted for space transportation operations.
Revision B 25 therefore used as an equivalent level of safety to compare against CEC, as this is a conservative approach.
13.3.1 Commercial transport aircraft have large casualty areas (up to approximately one million square feet). As a result of rigorous system safety programs and high redundancy of safety critical systems, they are allowed to fly over highly concentrated population centers in large cities. They are allowed to overfly cities and have a demonstrated probability of a failure resulting in a fatal accident per flight of less than 1E-6 (8E-7). These aircraft have a CEF, as measured by fatalities per fatal accident, of approximately 1. This CEF is then compared to the CEC of the applicant’s vehicle for purposes of § 450.101(c)(3). Flight Safety Analysis should protect against high consequence events for purposes of § 450.101(c)(3) by establishing that a launch or reentry vehicle sufficiently demonstrates reliability per the failure probabilities and conditions described in AC 25.1309-1A, System Safety Analysis and Assessment for Part 23 Airplanes.
13.3.2 Fixed wing, non-commercial transport, general aviation aircraft (including business jets) are characterized by a smaller casualty area (typically up to 10,000 square feet).
Their system safety programs are quite rigorous, although they lack the level of redundancy in safety critical systems that commercial transport aircraft have. They are allowed to overfly cities with a demonstrated probability of a failure resulting in a fatal accident per flight of less than 1E-5 (8E-6). These aircraft have a CEF of approximately
0.01 as measured by fatalities per fatal accident. Flight Safety Analyses should protect against high consequence events for purposes of § 450.101(c)(3) by establishing that a launch or reentry vehicle sufficiently demonstrates reliability per the failure probabilities and conditions described in AC 23.1309-1E.
13.3.3 Figure 3 of this AC connects two data points, the estimated failure probability (7.8E-6) and maximum conditional EF (0.01) for Fixed Wing General Aviation and the point with the corresponding values for Commercial Transports (failure probability = 7.8E-7, maximum conditional EF =1.) These points have been connected by a straight line indicating an assumed relationship between allowable failure probability and maximum conditional EC (CEC).
Revision B 26
Figure-3. Tolerable Failure Probability Per Flight vs CEC
13.3.4 If the estimated CEC falls between the CEF data for Fixed Wing General Aviation and that shown for Commercial Transports, then the tolerable failure probability for that segment may be estimated by interpolating between the failure probability for the two classes of aircraft, in logarithmic space, as illustrated in the graph in Figure 3.
OMB Control Number: 2120-0746 Expiration Date: 11/30/2024
FAA Form 1320-73 (09/22) SUPERSEDES PREVIOUS EDITIONS
Advisory Circular Feedback Form
If you find an error in this AC, have recommendations for improving it, or have suggestions for new items/subjects to be added, you may let us know by (1) emailing this form to 9-AST- ASZ210-Directives@faa.gov, or (2) f axing it to (202) 267-5450.
Subject: (insert AC title/number here)(insert AC title/number here) Date: Click here to enter text.
Please check all appropriate line items:
☐ An error (procedural or typographical) has been noted in paragraph Click here to enter text.
on page Click here to enter text..
☐ Recommend paragraph Click here to enter text. on page Click here to enter text. be changed as follows:
Click here to enter text.
☐ In a future change to this AC, please cover the following subject:
(Briefly describe what you want added.)
Click here to enter text.
☐ Other comments:
Click here to enter text.
☐ I would like to discus s the above. Please contact me.
Submitted by: Date:
Paperwork Reduction Act Burden Statement: A federal agency may not conduct or sponsor, and a person is not required to respond to, nor shall a person be subject to a penalty for failure to comply with a collection of information subject to the requirements of the Paperwork Reduction Act unless that collection of information displays a currently valid OMB Control Number. The OMB Control Number for this information collection is 2120-0746. Public reporting for this collection of information is estimated to be approximately 5 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. All responses to this collection of information are voluntary per FAA Order 1320.46D Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to: Information Collection Clearance Officer, 800 Independence Ave, Washington, D.C. 20590.