AC 39-8A CHAPTER 4. RISK FACTOR GUIDELINES A ND PRIORITIZATION BASED ON RISK
4.1 Hazard Level 3 and Level 4 Risk Guidelines.
4.1.1 There are long-term and short-term guidelines for risk factors and risk per flight as listed in table 1 of this AC. The risk factor guidelines apply to the aggregate fleet or subfleet suspected of having an unsafe condition. The per-flight guidelines apply to the average of the fleet or subfleet suspected of having an unsafe condition. These guidelines help establish whether immediate action is necessary and establish an acceptable risk for control programs.
Note 1: There are guidelines for HL 3 events and HL 4 events. The HL 3 guidelines cover events predicted to be at least HL 3, such as levels 3, 4, and 5 events. The HL 4 guidelines cover events predicted to be at least HL 4, such as HLs 4 and 5 events. This categorization is necessary to assess the true risk exposure.
Note 2: The per-flight guidelines can also be referred to as individual risk, and risk factor can be referred to as fleet risk (reference FAA Order 8110.107).
4.1.2 Control programs should be acceptable to both the HLs 3 and 4 short-term acceptable risks. Generally, for events unlikely to progress beyond HL 3, such as those that have low HL 4 or 5 hazard ratios, the HL 3 guidelines will be the limiting values. For those with a high hazard ratio for HL 4 and above, the HL 4 guidelines will be the limiting values. Corrective actions should reflect the event’s HL, the probability of an event, and the size of the affected fleet. For large fleet sizes, the risk factor guideline will likely be the limiting value. For small fleets, it is likely that the risk per flight guideline will be the limiting value.
Note: Currently, there are no standardized guidelines for HL 5 events, and the engine community has no plans to develop them. The HL 4 guidelines cover most level 5 threats. Use the HL 4 guidelines when assessing hazards that could potentially result in a level 5 event.
Table 1. Risk Guidelines Hazard Level 3 Hazard Level 4 Guidelines Guidelines Risk Per-flight Risk Per-flight factor factor Long-term - 1 x 10-8 - 1 x 10-9 acceptable risk Short-term 1.0 4 x 10-5 0.1 4 x 10-6 acceptable risk 4-1
AC 39-8A
4.1.3 The risk guidelines in table 1 should not be regarded as targets or typical values. The control program values should usually be lower than these guidelines, unless a lower value would result in extreme resource availability difficulties such as the availability of parts. Any reasonable action that reduces the risk should be included as part of the control program, keeping in mind the principles of prioritization of resources.
4.1.4 The HL 4 risk guidelines are intended to cover exposures to the most severe of serious injuries, such as life-threatening injuries. Consequently, relaxation of these guidelines may be acceptable in cases where the associated serious injuries are clearly not life threatening, such as simple fractures.
4.2 Risk Guidelines for In-Flight Shutdown (IFSD) Rate Concerns.
Typically, manufacturers seek to provide engine reliability improvements that will ensure adequate protection against excessive IFSD rate concerns. For rates below 2x10 -4 failures per cycle, an engine shutdown in which the only consequence is a loss of thrust should not be a cause for concern (other values may be listed elsewhere on a per-flighthour basis; for example, Extended Operations). This exclusion assumes that the event does not result from an initiating cause that can affect multiple engines at the same time.
4.3 Risk Guidelines for Immediate Action.
If the uncorrected risk factor for the affected fleet would exceed the applicable shortterm risk factor within 60 days or the risk per flight to which an airplane would be exposed during that same 60-day period would exceed the applicable risk guideline, then immediate action should be considered. How immediate this action should be could vary from before the next flight to within 60 days depending on the nature and level of risk. If a quantitative assessment of the risk is unavailable, the decision as to the necessity of immediate action should be made based on objective judgment and expert opinion. This initial analysis is not meant to take the place of the complete and in-depth analysis typically performed during the continuing assessment of the risk posed by the identified unsafe condition. It is meant to give a best-estimate relative ranking compared to the overall contributions of all unsafe conditions and to indicate whether continued operation without immediate corrective action is acceptable.
4.4 Prioritization Based on Risk.
Figures 1a and 1b provide a useful means to compare unsafe conditions, based on risk calculations and the HLs 3 and 4 risk guidelines that help to establish priorities.
4.4.1 High Risk.
An HL 3 event is a likely occurrence when its risk plots in the area to the right of the sloped line in Figure 1a, which represents a risk factor of 1.0 for the predicted events.
Furthermore, the malfunction is beginning to contribute more risk than the aggregate risk from all other causes, including contributions from the crew, when it plots in the area above the top horizontal line, 4x10 -5 per flight, displayed in Figure 1a. In instances where the short-term rate is exceeded, immediate actions may be necessary. Similar 4-2
AC 39-8A values can be established for HL 4 events, an order of magnitude below the associated HL 3 values. The high-risk area for HL 4 events is greater than 0.1 predicted events, or above 4x10 -6 per flight (see Figure 1b). These values equate to the short-term acceptable risk guidelines.
4.4.2 Elevated Risk.
Exposure within the enclosed envelope in Figures 1a and 1b, to the left of the sloped line and above the bottom horizontal line, imposes sufficient risk to warrant concern and action. Effective management of the risk may be possible through voluntary compliance to the manufacturer's recommended corrective actions. However, even if the risk becomes mitigated, issuance of an AD may still be necessary to minimize the potential for the unsafe condition being reintroduced in future products
4.4.3 Reasonable Risk.
Risk factors of one HL 3 events in every 100 million airplane flights (1x10 -8) meet the long-term acceptable risk guidelines assuming no other aspects of the original basis of certification have been violated. The equivalent value for HL 4 events is one event occurring in every 1 billion flights (1x10 -9).
Note: The long-term acceptable risk guideline for type certification may sometimes appear less restrictive. However, as noted earlier, certification and continued airworthiness assessments and guidelines are not necessarily equivalent. For example, the additional accuracy that could lead to less data uncertainty and better life prediction potentially afforded to continued airworthiness assessments over certification assessments should be compensated for within the guidelines.
4-3
AC 39-8A Figure 1a. Airplane Threat Comparison (Hazard Level 3)
1.00E-03
1.00E-04 HIGH RISK Level 3 Occurrence Rate (events ELEVATED RISK
1.00E-07 $ INCREASING
CONCERN
1.00E-08
REASONABLE RISK
1.00E-09 1.00E+06+++++
1.00E+03 1.00E+04 1.00E+05 1.00E+07 1.00E+08 1.00E+09 Number of Aircraft Flights (exposure) Figure 1b. Airplane Threat Comparison (Hazard Level 4)
1.00E-04
HIGH RISK 1 100 OS Rate (events 1.00E-06
ELEVATED RISK
1.00E-07
1.00E-08
INCREASING
CONCERN →
1.00E-09
REASONABLE RISK
1.00E-10 1.00E+05++++4 1.00E+06+++++
1.00E+03 1.00E+04 1.00E+07 1.00E+08 1.00E+09 Number of Aircraft Flights (exposure) 4-4
AC 39-8A CHAPTER 5. MISCELLANEOUS C ONDISDERATIONS
5.1 Corrected Risk.
The final goal in addressing an unsafe condition is the development and implementation of corrective actions that, when fully incorporated, minimize the probability of future events to less than the long-term acceptable risk guidelines. Interim measures, such as recurring inspections, are often effective in providing immediate risk reduction to a high-risk problem. However, recurring inspections should not be relied on to serve as a final corrective action unless there is no practicable alternative. If the interim measures minimize the probability of additional events to below the long-term guidelines, the incorporation of the final corrective actions can be delayed. The completion of all actions to correct the unsafe condition should be done as soon as feasible and in accordance with the risk guidelines and principles. The actions should return the product to the level of safety intended by that product's original basis of certification.
5.2 Cumulative Risk.
The cumulative risk of having several unsafe conditions being addressed concurrently on the same powerplant installation may represent an unacceptable risk level for that airplane type, even if each taken individually does not. Furthermore, repeated exposure to risk levels acceptable against any single unsafe condition could reasonably be expected to result in an unacceptable risk of serious injury at some point in the life of the worldwide transport airplane fleet with its various unsafe conditions. The Poisson distribution explains the statistical variation associated with average prediction for rare events; see Figure 2. The 0.1 risk factor guideline for HL 4 equates to a 9.5 percent probability of one or more HL 4 events. If, during the life of a fleet, there are seven different malfunctions, which are each managed to an HL 4 risk factor of 0.1 events, equivalent to a cumulative HL 4 forecast of 0.7 events, then it is likely that an HL 4 event will occur at some point due to one of these malfunctions. Additionally, for an event forecast of 0.7, there is a 16 percent probability of two or more events and a 3 percent probability of three or more events.
Figure 2. Event Probability vs. Event Forecast (Risk Factor) 5-1
AC 39-8A
5.2.1 It is neither expected nor required to calculate cumulative risk, nor track cumulative risk across the life of the fleet. The intent of this paragraph is only to provide recognition that acceptable risk levels should be regarded as upper limits, and stakeholders should reduce the risk further when possible. The goal of risk analysis is not to find the most lenient program possible within acceptable risk levels. The plot of risk factor versus impact on resources follows an asymptotic relationship. This means that at some point, any additional reduction in risk comes only at great increase in the required resources.
This particular point varies from situation to situation. The stakeholders should decide if the reduction in risk warrants the additional burden against the available resources.
5.2.2 Currently, no definitive standards exist for what is an acceptable cumulative risk.
Additionally, no definitive standards currently exist for where the balance should be struck between decreasing risk and increasing burden. Therefore, the FAA should judge what actions best serve the public interest on a case-by-case basis, with the appropriate allocation of resources. Since no guidelines have been developed for cumulative risk, the guidelines published in this AC apply to individual unsafe conditions being addressed by a control program. Although, the individual unsafe condition may apply to several products.
5.3 Lessons Learned.
Throughout the assessment process for any one potential unsafe condition, the stakeholders should apply the experience gained and lessons learned to future certification, including any regulatory action necessary and continued airworthiness monitoring processes. This ensures continuous improvement in the effectiveness of the continued airworthiness assessment process for current products and improves the certification assessment process and instructions for continued airworthiness for new products. Centralized accessible repositories for CAAM “lessons learned” are a valuable resource.
5.4 Alternative Methods of Compliance (AMOC).
The risk assessment process described in this AC can be used to help determine if a proposed AMOC provides an acceptable level of safety. An AMOC should not result in an increase of the total risk assessed against the unsafe condition.
5.5 Airplane and Engine Stakeholder Coordination.
Different risk management objectives may lead to differences of opinions between stakeholders as to the appropriate actions to address identified unsafe conditions, or to the timing of those actions. With most unsafe conditions, there is inter-involvement between the airplane stakeholders, as well as between the engine stakeholders. Close coordination is recommended to resolve any differences that arise, especially those that are highly installation-dependent. However, if these differences persist after coordination and discussion of the assumptions, available data and objectives, the final control program decisions rest with the stakeholders with the most relevant specialized technical expertise. This means, for unsafe conditions caused primarily by shortcomings 5-2
AC 39-8A within the engine type design, the engine stakeholders are responsible for making the final control program decisions. In cases of significant installation-dependency of an engine-related unsafe condition, special care should be taken to address any installation concerns raised by the airplane stakeholders. For unsafe conditions that are caused primarily by shortcomings in the airplane type design, exclusive of the engine type design, the airplane stakeholders are responsible for making the final control program decisions. All stakeholders should respect their primary responsibility and any tendency towards writing competing ADs against the same unsafe condition should be avoided.
5-3
AC 39-8A Appendix A APPENDIX A. POTENTIAL UNSAFE CONDITIONS A.1 Purpose.
This appendix provides the user with an overview of potential unsafe conditions.
Issuance of an AD requires that an unsafe condition exists in a product and the condition is likely to exist or develop in other products of the same type design, per 14 CFR section 39.5. This appendix presents guidance on potential unsafe conditions that may result from design, manufacturing, operational, or maintenance deficiencies as well as unforeseen changes in operations or the operating environment.
A.2 Identification of Potential Unsafe Conditions.
A.2.1 The stakeholders can use at least three areas of information as a guide in identifying potential unsafe conditions. The first and most visible are the conditions, which alone or in combination with other contributing factors, have led to accidents. Such conditions or combinations have been demonstrated to be unsafe. The second includes conditions that have significantly increased the probability of, but not yet directly caused, serious injuries. If such “contributing conditions” occur frequently enough, this too is an unsafe condition. The third area of information involves hazards identified as part of the product’s certification program. Indications that the field experience is worse than that allowed by the design standards may require mitigating action to return the product to the level of safety required by the design standards.
A.2.2 It is normal for the achieved level of safety of a product to vary throughout the lifetime of the fleet. This variation may result in some failure conditions occurring more frequently than permitted by initial certification requirements, in which case it is possible, but not necessarily the case, that an unsafe condition exists. Some assessment of the degree of risk is advisable if the failure condition rates significantly exceed those assumed or intended in the initial certification. If the risk to the airplane, passengers, or crew is much greater than permitted by initial certification standards, an unsafe condition is likely to exist.
A.2.3 It would be unusual for an identified unsafe condition to be limited to a single airplane or engine because of the size and complexity of today’s worldwide air transportation system. Examples of singular events where AD action would not be expected are those caused by gross negligence or rare meteorological phenomenon.
A.2.4 Conditions Specified as Potentially Unsafe.
For transport category airplanes, the FAA has defined certain specific conditions as potentially unsafe, based upon previous service experience and relevant certification assessment.
A.2.4.1 Historical Potentially Unsafe Conditions.
A list of engine-related conditions considered potentially unsafe, along with hazard ratio data is provided for certain conditions in the CAAM A-1
AC 39-8A Appendix A Reports and appendix D. See references listed in paragraph 1.4.5 of this AC.
A.2.4.2 Potential Unsafe Conditions Identified During Certification Assessments.
Certification assessments, such as those performed under § 25.1309, often identify and classify failure and operating conditions according to the severity of the impacts they are expected to have on the continued operational safety of the airplane. Very severe conditions are assigned to categories such as “catastrophic,” “preventing continued safe flight and landing,” or “critical” because of their potential to cause serious injuries to multiple persons. Severe conditions are assigned to categories such as “emergency” or “hazardous” because of their potential either to cause serious injuries to a limited number of persons or to impair the ability of the flight crew to perform their tasks. Therefore, the occurrence of any of these conditions in service is a potential unsafe condition regardless of the actual outcome. Conditions that are more moderate are assigned to categories such as “abnormal” or “major.” The occurrence of any moderate conditions in service at a high frequency may be considered a potential unsafe condition if a reasonable potential exists for it to contribute to a more serious event.
A.3 Categories of Failures Leading to Potential Unsafe Conditions.
A.3.1 Multiple FAA regulations limit the severity and frequency of single failures, such as 14 CFR §§ 33.75(a)(3), 25.901(c), and 25.1309(b)(1). Single failures that could result in a serious injury but are not expected to result in serious injuries to multiple persons are allowed by these regulations provided the frequency of occurrence is sufficiently low.
Prohibition of certain single failures is currently impracticable. These include uncontained engine rotor failure, engine case burst, engine case burnthrough, and propeller separations. For these noted exceptions, regulations such as § 25.903(d)(1) require that the hazards be minimized. When these failures or their precursors occur
(e.g., a flaw is detected in a disk before the disk actually fails), the stakeholder will carefully review the design of the component or engine to determine cause and appropriate action is developed, as necessary, to ensure that the occurrence of similar future events is minimized. The results of the investigation may require AD action to implement more effective monitoring or improved component inspections, shorter component life limits, improved maintenance procedures, or other means to minimize a reoccurrence. In addition, the stakeholder reviews the design of the airplane to ensure that the design covers the likelihood that these failures may continue to occur, and the installation incorporates design considerations to minimize the impact of these failures on the airplane.
A-2
AC 39-8A Appendix A A.3.2 Latent Failures.
Latent failures are failures that are unknown to the flight and maintenance crews.
Certification requirements, such as those contained in §§ 33.75 and 25.1309, assume that any expected latent failure, in combination with the next failure, under any operating and environmental conditions approved for the airplane, should not jeopardize continued safe flight and landing. A simple example is undetected loss of fire containment in a fire zone. If the next failure releases flammable fluid into the zone, a potentially catastrophic condition exists. While the intent of the FAA’s regulations is such that latent failure conditions do not exist, there are, as a practical matter, limitations on how frequently the operators can perform inspections on the powerplant and APU installations to note and correct such conditions. This is particularly true when such inspections require some degree of disassembly or otherwise expose components to potential distress or human error. Automated monitoring and indication should be used to detect and annunciate failure conditions when practical, especially when the next failure could lead to hazardous or catastrophic consequences. The intent of the inspection is not so much to discover the latent failure, but rather to note the proper functioning of the equipment and any safe limits of deterioration so that the equipment can be replaced before any significant failure or malfunction occurs. An additional concern is those latent failures which were either not anticipated at all or were expected to be detectable by either the flight or maintenance crews.
A.3.3 Cascading Failures.
Cascading failures are those for which the probability of occurrence of a subsequent failure is substantially increased by the existence of a previous failure. These types of failures are of particular concern because they can create interdependence between structural and system design elements that are intended or assumed to be independent or even unrelated. This is especially true when the intended means of safely accommodating a failure is affected by that failure. For example, in the structural design area, the failure of one load path should not result in loads that compromise the intended redundancy. Another example is that engine failures, such as fan blade failures, that result in a high vibration condition should not cause loss of the fuel shutoff function. A cascading failure of this sort could lead to a hazardous or catastrophic condition.
Cascading failures in propulsion systems can sometimes be difficult to anticipate. In transport airplanes, failures in the systems of one engine are typically required by § 25.903(b) to be independent of failures in the systems of another engine. Furthermore, a system of one engine may need to be isolated from the effects of failures within another system of that same engine. Engine systems areas where cascading failures are most likely to be of concern are the engine control systems and fuel systems.
A.3.4 Multiple Failures and Probability Estimates.
A.3.4.1 In general, the powerplant and APU installations are designed to be failsafe. That is, one assumes their failure and then ensures the resulting failure condition does not jeopardize continued safe flight and landing. For example, the shutdown of a single engine is assumed to be fail-safe, since transport category airplanes have multiple engines and are certified to A-3
AC 39-8A Appendix A operate safely following the sudden failure of the most critical engine.
Though combinations of failure conditions leading to violation of the failsafe assumption are possible, the consideration of such combinations should, as a practical matter, be limited to those conditions anticipated to occur within the fleet life of the airplane type. To make such determinations, the safety analysis methods associated with § 25.1309(b) are often used. Two examples of such situations are uncontrolled engine overspeed and an adverse frequency of engine shutdowns. The stakeholders usually agree that the first of these is a potential unsafe condition because the engine may liberate parts that could hazard the airplane. For overspeed, the requirement of § 33.28(d)(3) for engine control system certification is that no single failure can cause such a condition, and that the probability of such a condition being caused by multiple failures must be less than 10-8 per flight hour (extremely remote).
A.3.4.2 The second example requires attention because it is recognized that if the engine shutdowns begin occurring at an abnormally high rate, from the same or different failure conditions, the likelihood of multiple independent engine failures should be addressed (See paragraph 4.2). In any case, if an anticipated failure or malfunction can significantly affect the continued safe operation of more than one engine within a given flight, a potentially unsafe condition exists. Typically, business decisions to provide engine reliability improvements provide adequate protection against excessive IFSD rate concerns. In addition, the stakeholder should recognize that certain engine anomalies during critical flight regimes have, on occasion, resulted in accidents due to lack of recognition or appropriate response to a single engine failure, especially in cases of very startling or very subtle failures. Excessive exposure to these events raises the possibility of an inappropriate response. The stakeholders should address any situations where certification assumptions of appropriate responses, and the timing of those responses, have been repeatedly called into question.
A.3.5 Common Cause Failures.
“Common cause failures” refers to multiple failures occurring due to the same event.
These types of failures differ from “cascading failures” in that the multiple failures occur in parallel rather than in series. That is, the same event causes each failure independently rather than the first failure causing the second, and so on. Environmental conditions and human error are examples of occurrences that can give rise to common cause failures. Environmental factors include heavy rain and hail, icing, and bird or wildlife ingestion. Human-caused common cause failures include fuel contamination or mismanagement, procedural deviations, and maintenance errors. There are no regulations specifying that any engine-related maintenance is conducted on only one engine at a time, except for ETOPS maintenance requirements for some airplanes. For example, before long flights, it is common to service engine oil in all engines on the same airplane. Some cases of performing maintenance to multiple engines on the same airplane are likely unavoidable. However, the stakeholder should recognize that there A-4
AC 39-8A Appendix A are many instances (as documented in the CAAM reports) of multiple engine shutdown due to common cause maintenance error, such as chip detector reinstall or O-ring removal, that lead to unsafe conditions.
A-5