Revision B 22 12 ALTERNATE MEANS OF COMPLIANCE.
This AC is based on the CEC measure and specific CEC thresholds specified in the regulations, including §§ 450.101(c)(2) and 450.108(b). However, § 450.37 allows applicants to propose alternative approaches to measuring high consequence events that provide an equivalent level of safety, which may be approved by the FAA without a waiver. The FAA added this flexibility because it is aware of methods other than using CEC to measure the potential for high consequence events. If an applicant chooses to propose an alternative means of measuring a high consequence event, the FAA expects the alternative means to account for the potential for any high consequence event using a method that demonstrates an equivalent level of safety to a CEC analysis. In order to demonstrate an equivalent level of safety, the operator must ensure that the alternative means accurately assesses that the operation would not exceed an acceptable threshold for high consequence events. In order to determine whether an alternative threshold for high consequence events is acceptable, the FAA will compare the alternative measurement to the CEC threshold in the regulation.7
12.1 Using Risk Profile Curves.
A tool that may provide an equivalent level of safety to using CEC to measure high consequence events is a risk profile, as per reference number [2] of paragraph 3.4 of this AC. This is a curve that is sometimes used to compute maximum probable loss (MPL) to evaluate the insurance requirements of part 440, Financial Responsibility. Reference number [3] of paragraph 3.4 of this AC defines a risk profile is a plot that shows the probability of N or more casualties (vertical axis) as a function of the number of casualties, N (horizontal axis). A risk profile is discrete (i.e. no fractional casualties) and is the complementary cumulative distribution of the histogram that accounts for the aleatory uncertainty in the discrete number of casualties for each reasonably foreseeable scenario; as such the area under a risk profile is equal to the EC. Reference number [3] of paragraph 3.4 of this AC provides additional details regarding risk profile computations, including an example problem. Note that much more effort may be needed to generate a risk profile as compared to a set of CEC values. Figure 2 is an example of a risk profile showing the probability of third-party fatalities caused by general aviation (GA) accidents.
7 Alternatively, the applicant would be expected to demonstrate that the consequence of any failure during any significant period of flight is at least an order of magnitude less than the average results from a fixed-wing general aviation aircraft fatal accident, as explained below.
Revision B 23
Figure-2 – US General Aviation Ground Fatalities per Fatal Accident 1982-2019
12.2 The General Aviation Risk Profile.
The FAA computed this risk profile using National Transportation Safety Board (NTSB) accident data8 between 1982 and 2019 for only fixed-wing aircraft operated under 14 CFR parts 91, 135, and 137, excluding aircraft types that meet Part 25. The vertical axis of Figure 2 of this AC shows the conditional probability of “N” or more fatalities given a GA aircraft accident, and the horizontal axis shows the number of fatalities “N” for third parties only (uninvolved people located on the ground). The horizontal axis does not count fatal injuries to pilots and passengers. However, to qualify as an “accident” there must have been at least one fatality, including people on-board the aircraft. This conditional risk profile shows that one or more third party fatalities results on average following every 100 GA accidents. This empirical data also indicates that GA accidents have a conditional expected fatality value near 1E-2, which equates to the area under the conditional risk profile.
12.3 Compliance with High Consequence Event Protection.
An equivalent level of safety with § 450.101(c)(2) may be demonstrated by showing that the conditional risk profile for a launch or reentry mission, in terms of casualties, is at least an order of magnitude below the conditional ground fatality risk profile for GA accidents shown in Figure 2 of this AC. The empirical data from aviation accidents demonstrates that ground casualties are about three times more likely than ground fatalities; that difference is deemed appropriate given the uncertainties inherent in any physics-based model results.
8 A total of 12,644 fatal accidents, with 117 accidents that involved one or more ground fatalities.
0.0001
0.001
0.01 1 2 3 4 5 Probability of N or more Fatalities Number of Fatalities
Revision B 24 13 DEMONSTRATED RELIABILITY.
13.1 Required Reliability.
Section 450.101(c)(3) states that one way in which an operator can protect against a high consequence event in uncontrolled areas for each phase of flight is by establishing that the launch or reentry system has sufficient demonstrated reliability as agreed to by the Administrator based on CEC criteria during that phase of flight. The FAA will use the demonstrated reliability and average ground consequence results from fatal accidents involving U.S. civil aviation aircraft with standard airworthiness certificates to establish what constitutes sufficient demonstrated reliability to protect against a high consequence event based on CEC as described in this section. More specifically, sufficiently high demonstrated reliability will be evaluated based on the principles outlined in paragraphs 13.3.1 and 13.3.2 of this AC.
13.2 Demonstrating Reliability.
An applicant should demonstrate reliability based on history of the vehicle being flown, following equivalent practices to type certification for aircraft. Small modifications of a vehicle may be acceptable without being considered a new vehicle. Based on current licensed operations, the FAA anticipates that, initially, only flight phases where an aircraft carries a rocket would be able to meet the demonstrated reliability standard, and the aircraft would need an airworthiness certificate and extensive flight history.
However, it is anticipated that in the future, rocket systems will also develop history and certification practices that will enable commensurate levels of reliability to be demonstrated. Because demonstrated reliability provides an alternative to flight abort when CEC is greater than 1 x 10-3, it is appropriate to assess it consistent with the approach to flight abort and FSS reliability, which depends on CEC with a 1 x 10-2 threshold. For example, a binomial approach would require data from at 87,000 flights without a failure to demonstrate a reliability corresponding to the 8E-6 failure rate (identified in paragraph 13.3.3 of this AC) at the 50% confidence level.9
13.3 Existing Aviation Data.
Existing data is available and provides findings necessary to relate several classes of aircraft to consequences and reliability requirements. Reliability is simply the converse of failure probability, so reliability = 100% minus failure probability. Consequences for aircraft are not, however, available in terms of ground casualties. This information is only available in terms of ground fatalities. Conditional expected fatalities, 10 CEF, is
9 For example, see equation 13.9 in “An Introduction to Reliability and Maintainability Engineering” by Charles E.
Ebeling, Waveland Press, Inc., Long Grove, IL, 2019, ISBN-13: 978-1577666257.
10 CEF represents conditional expected fatalities and is used to measure the mean number of fatalities predicted to occur given an event with a probability of 1. The FAA found that about one ground fatality resulted on average from one hundred fatal accidents involving U.S.-registered aircraft operated under Part 91 between 1984 and 2013 based on NTSB data. A comparison of CEC to CEF is appropriate here because the CEF values cited here are empirical results from aviation accidents, whereas the CEC values used here are the results of physics-based computer simulations for launch and reentry operations. In addition, the differences between aviation and space operations justify some margin in the tolerability of the conditional risks predicted for space transportation operations.
Revision B 25 therefore used as an equivalent level of safety to compare against CEC, as this is a conservative approach.
13.3.1 Commercial transport aircraft have large casualty areas (up to approximately one million square feet). As a result of rigorous system safety programs and high redundancy of safety critical systems, they are allowed to fly over highly concentrated population centers in large cities. They are allowed to overfly cities and have a demonstrated probability of a failure resulting in a fatal accident per flight of less than 1E-6 (8E-7). These aircraft have a CEF, as measured by fatalities per fatal accident, of approximately 1. This CEF is then compared to the CEC of the applicant’s vehicle for purposes of § 450.101(c)(3). Flight Safety Analysis should protect against high consequence events for purposes of § 450.101(c)(3) by establishing that a launch or reentry vehicle sufficiently demonstrates reliability per the failure probabilities and conditions described in AC 25.1309-1A, System Safety Analysis and Assessment for Part 23 Airplanes.
13.3.2 Fixed wing, non-commercial transport, general aviation aircraft (including business jets) are characterized by a smaller casualty area (typically up to 10,000 square feet).
Their system safety programs are quite rigorous, although they lack the level of redundancy in safety critical systems that commercial transport aircraft have. They are allowed to overfly cities with a demonstrated probability of a failure resulting in a fatal accident per flight of less than 1E-5 (8E-6). These aircraft have a CEF of approximately
0.01 as measured by fatalities per fatal accident. Flight Safety Analyses should protect against high consequence events for purposes of § 450.101(c)(3) by establishing that a launch or reentry vehicle sufficiently demonstrates reliability per the failure probabilities and conditions described in AC 23.1309-1E.
13.3.3 Figure 3 of this AC connects two data points, the estimated failure probability (7.8E-6) and maximum conditional EF (0.01) for Fixed Wing General Aviation and the point with the corresponding values for Commercial Transports (failure probability = 7.8E-7, maximum conditional EF =1.) These points have been connected by a straight line indicating an assumed relationship between allowable failure probability and maximum conditional EC (CEC).
Revision B 26
Figure-3. Tolerable Failure Probability Per Flight vs CEC
13.3.4 If the estimated CEC falls between the CEF data for Fixed Wing General Aviation and that shown for Commercial Transports, then the tolerable failure probability for that segment may be estimated by interpolating between the failure probability for the two classes of aircraft, in logarithmic space, as illustrated in the graph in Figure 3.
OMB Control Number: 2120-0746 Expiration Date: 11/30/2024
FAA Form 1320-73 (09/22) SUPERSEDES PREVIOUS EDITIONS
Advisory Circular Feedback Form
If you find an error in this AC, have recommendations for improving it, or have suggestions for new items/subjects to be added, you may let us know by (1) emailing this form to 9-AST- ASZ210-Directives@faa.gov, or (2) f axing it to (202) 267-5450.
Subject: (insert AC title/number here)(insert AC title/number here) Date: Click here to enter text.
Please check all appropriate line items:
☐ An error (procedural or typographical) has been noted in paragraph Click here to enter text.
on page Click here to enter text..
☐ Recommend paragraph Click here to enter text. on page Click here to enter text. be changed as follows:
Click here to enter text.
☐ In a future change to this AC, please cover the following subject:
(Briefly describe what you want added.)
Click here to enter text.
☐ Other comments:
Click here to enter text.
☐ I would like to discus s the above. Please contact me.
Submitted by: Date:
Paperwork Reduction Act Burden Statement: A federal agency may not conduct or sponsor, and a person is not required to respond to, nor shall a person be subject to a penalty for failure to comply with a collection of information subject to the requirements of the Paperwork Reduction Act unless that collection of information displays a currently valid OMB Control Number. The OMB Control Number for this information collection is 2120-0746. Public reporting for this collection of information is estimated to be approximately 5 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. All responses to this collection of information are voluntary per FAA Order 1320.46D Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to: Information Collection Clearance Officer, 800 Independence Ave, Washington, D.C. 20590.