Appendix 2 Page 3 (and 4) Content (Standards)
FAA SMS Framework AC 120-92 (2006, App 1) ISO 9001:
2000/AS 9100 ISO 14001 OHSAS 18001 corrective and preventive action Auditing 3.1.3 - 3.1.5 6.3.2, 6.3.3, 6.3.4 8.2.2 4.5.4 4.5.4 Management review 3.3.2 6.7 5.6 4.6 4.6 Continuous Improvement 3.3 6.8 8.5.1 4.3.4 4.3.4
Appendix 3 APPENDIX 3. SAMPLE RISK ASSESSMENT TOOLS
1. PURPOSE OF THIS APPENDIX. The table below is provided to assist those organizations developing and implementing an SMS. It provides examples of potential criteria for the severity of consequences and likelihood of occurrence that may be used by an aviation service provider.
2. SEVERITY AND LIKELIHOOD CRITERIA. The definitions and final construction of the matrix is left to the aviation service provider’s organization to design. The definitions of each level of severity and likelihood will be defined in terms that are realistic for the operational environment. This ensures each organization’s decision tools are relevant to their operations and operational environment, recognizing the extensive diversity in this area. An example of severity and likelihood definitions is shown in Table 1 below. Each operator’s specific definitions for severity and likelihood may be qualitative but quantitative measures are preferable, where possible.
TABLE 1. SAMPLE SEVERITY AND LIKELIHOOD CRITERIA Severity of Consequences Likelihood of Occurrence Severity Level Definition Value Likelihood Level Definition Value Catastrophic Equipment destroyed, multiple deaths 5 Frequent Likely to occur many times 5 Hazardous Large reduction in safety margins, physical distress or a workload such that operators cannot be relied upon to perform their tasks accurately or completely. Serious injury or death. Major equipment damage.
4 Occasional Likely to occur sometimes 4 Major Significant reduction in safety margins, reduction in the ability of operators to cope with adverse operating conditions as a result of an increase in workload, or as result of conditions impairing their efficiency. Serious incident. Injury to persons.
3 Remote Unlikely, but possible to occur 3 Minor Nuisance. Operating limitations. Use of 2 Improbable Very unlikely to occur 2 Page 1
Appendix 3 Severity of Consequences Likelihood of Occurrence emergency procedures. Minor incident.
Negligible Little consequence 1 Extremely Improbable Almost inconceivable that the event will occur 1
a. Risk Acceptance. In the development of its risk assessment criteria, aviation service providers are expected to develop risk acceptance procedures, including acceptance criteria and designation of authority and responsibility for risk management decision making. The acceptability of risk can be evaluated using a risk matrix such as the one illustrated in Figure 1.
The example matrix shows three areas of acceptability. Risk matrices may be color coded;
unacceptable (red), acceptable (green), and acceptable with mitigation (yellow).
(1) Unacceptable (Red). Where combinations of severity and likelihood cause risk to fall into the red area, the risk would be assessed as unacceptable and further work would be required to design an intervention to eliminate that associated hazard or to control the factors that lead to higher risk likelihood or severity.
(2) Acceptable with Mitigation (Yellow). Where the risk assessment falls into the yellow area, the risk may be accepted under defined conditions of mitigation. An example of this situation would be an assessment of the impact of a non-operational aircraft component for inclusion on a minimum equipment list (MEL). Defining an Operational (O) or Maintenance (M) procedure in the MEL would constitute a mitigating action that could make an otherwise unacceptable risk acceptable, as long as the defined procedure was implemented. These situations may also require continued special emphasis in the SA function.
(3) Acceptable (Green). Where the assessed risk falls into the green area, it may be accepted without further action. The objective in risk management should always be to reduce risk to as low as practicable regardless of whether or not the assessment shows that it can be accepted as is. This is a fundamental principle of continuous improvement.
Appendix 3 FIGURE1. SAFETY RISK MATRIX EXAMPLES Severity Likelihood Less More Higher Lower Acceptable UnacceptableAcceptable with Mitigation Severity Likelihood Less More Higher Lower Acceptable UnacceptableAcceptable with Mitigation Example 1 Risk Risk Severity Likelihood Catastrophic A Hazardous B Major C Minor D Negligible E Frequent 5 5A 5B 5C 5D 5E Occasional 4 4A 4B 4C 4D 4E Remote 3 3A 3B 3C 3D 3E Improbable 2 2A 2B 2C 2D 2E Extremely Improbable 1 1A 1B 1C 1D 1E Example 2 NOTE: (The direction of higher/lower and more/less scales on a matrix is at the discretion of the organization).
b. Other Risk Assessment Tools for Flight and Operational Risk Management. Other tools can also be used for flight or operational risk assessment such as the controlled flight into terrain (CFIT), Approach and Landing Accident Reduction (ALAR), operational control, and ground operations risk assessment tools available from the Flight Safety Foundation (FSF) (http://www.flightsafety.org/technical_initiatives.html).
Appendix 3
c. Causal Analysis. Risk analyses should concentrate not only on assigning levels of severity and likelihood but on determining why these particular levels were selected. This is referred to as root cause analysis, and is the first step in developing effective controls to reduce risk to lower levels. Several structured software systems are available to perform root cause analysis. However, in many cases, simple brainstorming sessions among the company’s pilots, mechanics, or dispatchers other experienced subject matter experts is the most effective and affordable method of finding ways to reduce risk. This also has the advantage of involving employees who will ultimately be required to implement the controls developed.
Appendix 4 APPENDIX 4. TABLE OF EXPECTED SAFETY MANAGEMENT SYSTEM OUTPUTS
1. PURPOSE OF THIS APPENDIX. To a large extent, controls are built into the design of the FAA SMS Framework. A general expectation of the policy component is that SMS outputs will be “recorded, monitored, measured, and analyzed” (Component 1.0b(3)). The internal evaluation function of the SA Component calls for evaluations “at planned intervals” of SMS conformance to objectives and expectations (Process 3.1.3b(1)).
2. MEASUREMENTS. Each of the outputs should also have a method of measurement specified by the organization in accordance with Component 1.0b(2) “SMS processes will be…measured…” Measures need not be quantitative where this is not practical. All that should be expected is some method of providing objective evidence of the attainment of the expectation.
NOTE: There is a relationship between controls and process measures. That is, the internal evaluation process is the method of controlling the processes, through the associated data collection, analysis, assessment, and corrective action processes. The individual outputs are the content of the measures.
3. MANAGEMENT REVIEWS. Finally, management reviews are the means of making sure that the appropriate levels of responsibility and authority are brought into the process and that management can be accountable in a proactive way, rather than an after-the-fact attribution.
NOTE: Table 1 below is a complete set of outputs, as a minimum expectation, for the content of internal evaluations of each process area.
TABLE 1. TABLE OF EXPECTED SAFETY MANAGEMENT SYSTEM OUTPUTS Process Reference Output Expectation Component 1.0 - Safety Policy and Objectives This table does not apply to the first component Component 2.0 - Safety Risk Management
2.1.1 System/Task Analysis
2.1.1b(1) System descriptions for following situations:
2.0b(2)(a) • Initial designs of systems, organizational procedures, and products
2.0b(2)(b) • Development of operational procedures
2.0b(2)(d) • Planned Changes
2.1.2 Hazard Identification
2.1.2b(1)b) Hazards documented
2.1.2b(2)(a) Hazards tracked
2.2.1 Risk Analysis 2.2.1b(1)(c) Assignment of severity and likelihood for each hazard (as documented in 2.1.2)
2.2.2 Risk Assessment
2.2.2b(1) Assessment of risk acceptability for each hazard (as documented in 2.1.2)
2.2.3 Risk Control 2.2.3b(1) Risk control/mitigation plans for each hazard with an unacceptable risk (as assessed in 2.2.2) Component 3.0 - Safety Assurance Page 1
Appendix 4 Process Reference Output Expectation
3.1.1 Continuous Monitoring
3.1.1 Objective evidence of monitoring activities in accordance with company policy
3.1.2 Internal Audit 3.1.2b(5)(b)(1) Plans
3.1.2b(5)(b)(3) &
(4) Reports/records
3.1.3 Internal Evaluation
3.1.3b(3)(d)(2)(a) Plans
3.1.3b(3)(d)(2)
(c) & (d) Reports/records
3.1.4 External Evaluation
3.1.4 Objective evidence of external audit findings
(e.g., International Air Transport Association Operational Safety Audit (IOSA), International Business Aviation Council (IS-BAO), Air Charter Safety Foundation (ACSF), and FAA)
3.1.5 Investigations 3.1.5b(1) Data collected (e.g. records, reports) for investigations of:
3.1.5b(1)(a) Incidents
3.1.5b(1)(b) Accidents
3.1.5b(1)(c) Potential regulatory violations (e.g., Voluntary Disclosure Reporting Program (VDRP) records)
3.1.6 Employee Reporting System (ERS)
3.1.6b(1) Evidence of system (e.g. report file, log, database)
3.1.6b3) Evidence of monitoring of ERS data for hazards
3.1.6b(4) Evidence of analysis of ERS data
3.1.7 Analysis of Data 3.1.7b
3.1.7b(1) Objective evidence of analysis processes for each data type
3.1.8 System Assessment
3.1.8b(4) Records of system assessments
3.3.1 Preventive/Corrective Action
3.3.1b(1) Corrective action plans
3.3.1b(5) Records of disposition and status of corrective actions
3.3.2 Management Review
3.3.2b(1) Objective evidence of management reviews (e.g., minutes, log) Component 4.0 - Safety Promotion
4.1.1 Competency Requirements
4.1.1b(1) Documented competency requirements in accordance with 1.2 b(3) & 1.3b(1)
4.1.2 Training 4.1.2b(1) Plans/requirements
4.1.2b(3) Records
4.1.2b(4) Reviews