3. The above example of adding aircraft to your fleet would entail consideration of several systems and a variety of procedures due to the number of processes that would be affected. However, in a simpler case, such as a change in the procedure for arming cabin doors prior to pushback, only those elements of the systems that would be affected by the change would need to be considered. The carrier would not be expected to perform a comprehensive analysis of the entire cabin safety system.
4. The system description and analysis process frequently includes representatives from management, safety staff, subject matter experts (SME), employees, and representation groups (e.g., unions) formed into workgroups such as safety committees, safety roundtables, safety action groups, or similar titles. Since many, if not most, system changes involve allocation of resources, the accountable executive or other managers with the authority to commit resources should be included in the process.
5. The outputs could be recorded in a simple recording medium such as a worksheet or a notebook, common desktop software, or a Web-Based Application Tool (WBAT), (see Appendix 6). Appendix 2 provides an example of a set of worksheets that could be used as paper records or converted to a variety of software applications, including desktop spreadsheets or WBATs.
(g) Scalability.
1. Small. System description and analysis could be performed by the owner/manager and/or another assigned employee(s). An analysis could consist of a discussion among managers such as the Director of Operations (DO) and/or Chief Pilot or other individuals designated by them.
2. Medium. System description and analysis could be performed by a member of management or one of the designated SMS management representatives with a small workgroup of company SMEs and stakeholders.
3. Large. System description and analysis might be performed at multiple organizational levels (e.g., corporate, division, or department) and facilitated by the Safety Department/division or its equivalent. The organization might have standing committees of SMEs and stakeholders participating at various levels.
(2) Hazard Identification.
(a) SRRs. Section 5.53(c).
(b) Related Regulatory Requirements.
• Section 5.23(a)(2)(i), Accountabilities of Members of Management.
• Section 5.25(c)(2), Responsibilities of Safety Management Personnel.
(c) ICAO/SMSPP. Element 2.1/Process 2.1.2.
Par 3-4 Page 21
(d) Objective. To examine operational systems, operations, processes, and the operational environment in order to identify conditions that could result in an aircraft accident.
(e) Part 5 Requirements.
§ 5.53 System analysis and hazard identification.
(c) The certificate holder must develop and maintain processes to identify hazards within the context of the system analysis.
(f) Discussion. The hazard identification process flows from the system analysis. In hazard identification, you would ask: What could go wrong with your processes, under typical or abnormal operational conditions, that could cause an accident?
1. Most often the same individuals or groups conducting the system analysis process (safety committees, safety roundtables, etc.) would conduct hazard identification. You use your experience, FAA requirements, manufacturers’ technical data, and knowledge of your operations to identify hazards to your operation. For example, a newly modified component of an aircraft cabin door might require new arming and disarming procedures by flight attendants and new signaling procedures for station personnel upon aircraft arrival at a passenger gate.
Hazards could include the effectiveness of new procedure training, employees missing training, failing to read or understand newly published procedures, supervisors failing to monitor the new procedures, etc. While identification of every conceivable hazard is unlikely, you are expected to exercise due diligence in identifying hazards that could foreseeably lead to an aircraft accident.
2. The output of hazard identification could be recorded in a simple recording medium such as a spreadsheet or paper files, or software such as a WBAT.
(g) Scalability.
1. Small. Hazard identification could be performed by the owner/manager and/or another employee(s), often as part of the system analysis.
2. Medium. As in the system analysis, hazard identification could be performed by a member of management or a designated SMS management representative with a small workgroup of company SMEs and stakeholders.
3. Large. Hazard identification might be performed at multiple organizational levels (e.g., corporate, division, or department levels) and facilitated by a Safety Department/division or its equivalent. The organization might have standing committees of SMEs and stakeholders participating at various levels.
d. Safety Risk Analysis and Control. This section of the regulation addresses the following three processes: (1) Safety Risk Analysis, (2) Safety Risk Assessment, and (3) Safety Risk Control.
Par 3-4 Page 22
(1) Safety Risk Analysis.
(a) SRRs. Section 5.55(a).
(b) Related Regulatory Requirements. Section 5.25(c)(2), Responsibilities of Safety Management Personnel.
(c) ICAO/SMSPP. Element 2.2/Process 2.2.1.
(d) Objective. To estimate the severity and likelihood of a potential accident due to exposure to an identified hazard.
(e) Part 5 Requirements.
§ 5.55 Safety risk assessment and control.
(a) The certificate holder must develop and maintain processes to analyze safety risk associated with
the hazards identified in § 5.53(c).
(f) Discussion. For each identified hazard, define the potential for injury and damage that may result from an accident related to operating while exposed to the hazard. In order to determine potential for injury and damage, you need to define the likelihood of occurrence of an accident and severity of the injury or damage that may result from the aircraft accident. It is important to remember that the likelihood and severity do not refer to the hazard but of a potential occurrence (accident or incident) related to the hazard.
1. The risk analysis also needs to consider the basis for the estimates of severity and likelihood. What is it about the factors analyzed in § 5.53, individually or in combination, which could result in an accident? Have you changed equipment that your employees must use, the procedures for using it, the layout of the facility, etc., in ways that could increase the likelihood of errors resulting in an accident? For example, if, in the process of a merger, flight deck procedures from one of the partner airlines become the standard across the merged carrier and if the change in procedures has been identified as a hazard, what is it about the new procedures that would lead to errors?
2. This is one reason why the system analysis is an essential foundational step in risk management. If risk analysis is not based upon a thorough understanding of the system, you may miss important details that could cause the system to fail. The knowledge gained in the system analysis and subsequent risk analysis will later be used to develop a mitigating strategy.
Risk controls will target the conditions that we think will cause an accident or affect its severity or likelihood.
3. Risk analyses in operational contexts are often based on expertise and expert judgment, but they should also use data from the carrier’s own experience or those of others in the industry where available. Review of accident statistics, failure data, error data (e.g., runway incursion reports or information from the National Aeronautics and Space Administration (NASA)’s Aviation Safety Reporting System (ASRS)) or equipment reliability data may help in determining likelihood.
Par 3-4 Page 23
4. The type of consequence (e.g., error, failure, accident, or incident) that is envisioned normally drives the estimate of severity. For example, if the hazard could result in controlled flight into terrain (CFIT), the severity of this outcome is normally major, if not catastrophic. Conversely, tire failures, while potentially leading to a fatal accident, more often lead only to aircraft damage.
5. Even where the best estimate has to be based on reasonable expert judgment, effective risk management can be accomplished by applying a disciplined analysis. The outputs could be recorded in a simple recording medium such as a notebook, basic desktop software, a WBAT,5 or a spreadsheet application.
(g) Scalability.
1. Small. Risk analysis could be performed by the owner/manager, and/or another employee(s). It might be performed in conjunction (by the same individual/group) with system description and analysis, hazard identification, risk assessment, and risk control.
2. Medium. Risk analysis could be performed by a member of management and/or the designated management representative with a small workgroup of company SMEs and stakeholders.
3. Large. Risk analysis might be performed at multiple organizational levels,
(e.g., the corporate, division, or department levels) and facilitated by the Safety Department or specially trained analytical personnel shared with other departments. The organization might have standing committees of SMEs and stakeholders participating at various levels.
(2) Safety Risk Assessment.
(a) SRRs. Section 5.55(b).
(b) Related Regulatory Requirements.
• Section 5.23(a)(2)(i), Accountabilities of Members of Management.
• Section 5.23(b), Decision Authority Regarding Safety Risk Acceptance.
(c) ICAO/SMSPP. Element 2.2/Process 2.2.2.
(d) Objective. To make a decision regarding the acceptability of operation in the presence of an identified hazard.
5 The Web-Based Analysis Tool (WBAT) is a federally developed and funded software system that may be used to assist air carriers with data management. WBAT also contains an SMS implementation plan manager module, which supports the air carrier’s implementation of SMS.
Par 3-4 Page 24
(e) Part 5 Requirements.
§ 5.55 Safety risk assessment and control.
(b) The certificate holder must define a process for conducting risk assessment that allows for the determination of acceptable safety risk.
(f) Discussion. Once the risk is analyzed, you must assess whether the risk is acceptable. A common tool used in risk assessment decisions is a risk matrix. A risk matrix provides you with a way to integrate the effect of severity of the outcome and the probability of occurrence, which enables you to assess risks, compare potential effectiveness of proposed risk controls, and prioritize risks where multiple risks are present.
1. If a risk matrix is used, the carrier should develop criteria for severity and likelihood that are appropriate for their type of operations and their operational scenario. For example, severity levels are sometimes defined in terms of a dollar value of potential damage. In this case, different types of airplanes operated and their relative values would dictate different scales between carriers. Likewise, the method that the carrier uses to estimate likelihood will have an effect on how likelihood scales are defined. If the carrier prefers to use quantitative estimates (e.g., probability), the scales would be different than one that prefers to use qualitative estimates. Figure 3-2 below depicts a sample risk matrix.
FIGURE 3-2. SAMPLE RISK MATRIX
2. Risk assessment is based on judgment, experience, and input from data collection tools (DCT) and previous processes. If the risk is acceptable, it may be done with the SRM process. If this is the case, then the system may be placed into operation and monitored in the SA process. If you decide the risk is not acceptable, you will need to follow the next step in SRM, which is developing risk controls.
3. Risk assessments must also involve the levels of management with the authority to make risk decisions, deciding what is or is not an acceptable risk for the systems within their area of operational responsibility. For example, dispatching a flight that presents Par 3-4 Page 25
a medium or high risk might require the Chief Pilot or DO to approve or authorize the flight. For large scale operational decisions, the accountable executive may be the only appropriate person to make these risk acceptance decisions. Thus, the person responsible for making these risk acceptance decisions will depend on the scope of the proposed change to the operation and the level of risk presented to the operation.
4. The outputs of this process could be recorded on paper or via an electronic medium, such as a WBAT.
(g) Scalability.
1. Small. Risk assessment could be performed by the owner/manager, and/or another employee(s) making the risk decisions. Risk acceptance would also probably be conducted by this individual/group. These processes could be similar to a flight risk management process, or you could use a risk matrix.
2. Medium and Large. SRM should be coordinated across the divisional and geographic units of the company to assure integrated decisionmaking and communication.
Decisions involving multiple systems may require joint decisionmaking among departments or managers responsible for those systems. Many companies have standing committees made up of senior managers, who are the decision makers, supported by working groups of technical personnel. For example, the accountable executive could make company-level decisions and department managers could make the decisions for their operations. A risk matrix may be useful to determine who makes the risk decision, whether the risk is acceptable, or to determine the priority for mitigating risk.
(3) Safety Risk Control.
(a) SRRs. Sections 5.55(c) and (d).
(b) Related Regulatory Requirements. None.
(c) ICAO/SMSPP. Element 2.2/Process 2.2.3.
(d) Objective. To develop a risk control that reduces risk to an acceptable level.
(e) Part 5 Requirements.
§ 5.55 Safety risk assessment and control.
(c) The certificate holder must develop and maintain processes to develop safety risk controls that
are necessary as a result of the safety risk assessment process under paragraph (b) of this section.
(d) The certificate holder must evaluate whether the risk will be acceptable with the proposed safety
risk control applied, before the safety risk control is implemented.
(f) Discussion.
1. After hazards and associated risk are fully understood, risk controls must be designed for risks that the carrier deems unacceptable. This is accomplished using their risk Par 3-4 Page 26
assessment process, as specified in § 5.55(b). Examples of risk controls include new processes, equipment, training, new supervisory controls, new equipment or hardware, new software, changes to staffing arrangements, or any of a number of other system changes. In short, anything that would lessen the likelihood or severity of a potential incident/accident.
2. Next, look at the system with the proposed control in place to determine if the level of risk is now acceptable and the proposed control does not introduce unintended consequences or new hazards. This is commonly referred to as substitute risk. Section 5.55(d) requires you to evaluate whether the risk will be acceptable with the proposed safety risk control applied. The risk that remains is often called residual risk.
(g) Scalability.
1. Small. The risk control process could be a documented activity performed by the owner/manager and/or another employee(s) designing and evaluating the risk controls. It might be performed in conjunction (by the same individual/group) with system description and analysis, hazard identification, risk analysis, and risk assessment.
2. Medium and Large. The risk control process could be performed by a member of management or SMS management representatives with a small workgroup of company SMEs and stakeholders to design the risk controls. There would be interdepartmental coordination before the controls are implemented. After the control is approved, it is implemented and documented through the company’s publication system. Implementation of risk controls may include distribution of manual revisions and training of company personnel.
e. SRM Example. The scenario below illustrates how these SRM processes would be applied in an operational context.
(1) An airline receives notification that the instrument landing system (ILS) at one of its destination airports is scheduled to be out of service for a period of time and, therefore, the flights will have to use a less familiar Nonprecision Approach (NPA) procedure. Concern is expressed by the flightcrews about the procedure and the differences between fleets that will affect operational procedures and the currency of training. Under § 5.51, this triggers the SRM processes and procedures.
(2) Pursuant to § 5.53(a) and (b), the SRM process starts with an analysis of the systems involved, with the addition of a new procedure for the aircraft fleet. This would include analysis of the flight operations and training program. In this case, the company has two fleets of airplanes that are operated to the destination airport in question. The equipment, and therefore the procedures and flightcrew training in these two fleets, differs. By analyzing the system, they identify a hazard (per § 5.53(c)), in that the pilots are not properly trained to fly the procedures that are required for that specific airport.
(3) To analyze the risk associated with this hazard (as required by § 5.55(a)), a team is assembled with representatives from management, the training and standardization organization, the flight safety organization, the organization responsible for dispatch and operational control, and the pilot’s union. Representatives of the FAA Certificate Management Team (CMT) also Par 3-4 Page 27
attend the work team’s meetings. The risk analysis consists of documented discussions among the assembled SMEs.
(4) As a result of the risk analysis, a risk assessment is done (per § 5.55(b)). As part of the assessment, it is determined that the existing operational procedures and training used by one of the fleets are not acceptable for safe operation into the airport without revision and additional training. A management decision is made that the procedures and associated training currently in use will not provide an acceptable level of safety (ALoS) for operations into the airport.
(5) After determining that the risk is unacceptable, risk controls are designed under § 5.55(c). The risk control in this case consists of a change to operational procedures, including programming of the flight management system (FMS) and associated approach procedure training for flightcrew members. Prior to implementation of the risk controls, the company checks the resulting procedures in flight simulators to ensure that the risk control will mitigate risk to an acceptable level, as required by § 5.55(d).
(6) Per § 5.23(b), the solution is submitted to the fleet Chief Pilots for management review and acceptance of the mitigated risk. Division-level management and top management are briefed on the operation at regular management reviews. Once the controls are approved by all concerned, they are implemented operationally. Per the SA requirements, these controls are specifically targeted for performance monitoring by the operational control/dispatch organization and the flight operations organization line checks.
3-5. SUBPART D: SAFETY ASSURANCE.
a. Introduction.
(1) SRR. Part 5, subpart D.
(2) Related Regulatory Requirements.
• Section 5.23(a)(2), Accountabilities of Members of Management.
• Section 5.25(a)(4), Accountable Executive—Ultimate Responsibility for Safety Performance.
• Section 5.25(b)(1), Accountable Executive—Responsibility to Ensure SMS Implementation and Performance.
(3) ICAO/SMSPP. Component 3.0.
(4) Summary. SA consists of the following five processes, divided into three sections of the regulation.
(a) Safety Performance Monitoring and Measurement § 5.71.
• System monitoring,
• Data acquisition, and
• Analysis.
Par 3-4 Page 28
(b) Safety Performance Assessment § 5.73. System assessment.
(c) Continuous Improvement § 5.75. Continuous improvement.
NOTE: Figure 3-3, Safety Assurance Processes and Regulatory Requirements, depicts the SA processes and their associated regulatory requirements.
FIGURE 3-3. SAFETY ASSURANCE PROCESSES AND REGULATORY
REQUIREMENTS SA 5.71Safety (Subpart c) Performance 5.71(a)(1)&(2) Monitoring Monitoring Inputs From SRM:
5.71(a) and Measurement
5.71(a)(3) Internal Audits
5.71(a)(4) Internal Evaluation *
5.71(a)(5)&(6) Investigations
5.71(a)(7) Employee Reporting How is this going to be 5.71(b) Analysis of Data analyzed? By whom?
5.75 Continuous System Operation/ Monitoring Analysis Data Acquisition & Process Preventive/ Corrective Action System Assmt Improvement
5.73(a) System Assessment 5.73 Safety
5.73(a) Management Review Performance Outputs To SRM:
Assessment 5.73(b) Note: Each data source should be traceable * through analysis 5.71(b) assessment 5.73(a) 5.75 And Corrective Action 5.75 where necessary.
b. Safety Performance Monitoring and Measurement: Overview. This section of the regulation consists of the following processes: Monitoring of Operational Processes (subparagraph 3-5c(1)), Monitoring of the Operational Environment (subparagraph 3-5c(2), Data Acquisition (subparagraphs 3-5c(3)–(7)), and Analysis of Data (subparagraph 3-5d).
(1) SRR. Section 5.71.
(2) Related Regulatory Requirements. None.
(3) ICAO/SMSPP. Element 3.1.
Par 3-5 Page 29
c. Safety Performance Monitoring and Measurement: Required Data Acquisition Processes. This subsection consists of the following seven data acquisition processes:
Monitoring of Operational Processes, Monitoring of the Operational Environment, Auditing of Operational Processes, Evaluation of the SMS and Operational Processes, Investigations:
Accidents and Incidents, and Investigation: Potential Noncompliance and Confidential Employee Reporting System.
(1) Monitoring of Operational Processes.
(a) SRR. Section 5.71(a)(1).
(b) Related Regulatory Requirements.
• Part 121, § 121.703, Service Difficulty Reports.
• Section 121.705, Mechanical Interruption Summary Reports.
(c) ICAO/SMSPP. Element 3.1/Process 3.1.1.
(d) Objective. To observe the day to day, flight by flight, job by job performance of operational systems and their associated risk controls.
(e) Part 5 Requirements.
§ 5.71 Safety performance monitoring and measurement.
(a) The certificate holder must develop and maintain processes and systems to acquire data with
respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, the following:
(1) Monitoring of operational processes.
(f) Discussion. Monitoring operational processes is what supervisors do on a day-to-day basis (e.g., direct supervision of employee activities, monitoring of pilot currency, and monitoring Minimum Equipment List (MEL) status). Monitoring also involves reviewing data that is collected for operational purposes to look for anything of safety significance (e.g., duty logs, crew reports, work cards, process sheets, and reports from the employee safety feedback system). This may include monitoring products and services from outside sources that are used in the certificate holder’s operations.
(g) Scalability.
1. Small. Most of the data/information-gathering for monitoring of operational processes will likely occur as a normal business process by the management personnel who are directly involved in the day-to-day operations. For example, regularly reviewing (e.g., weekly, monthly, or quarterly) the flight dispatch logs and crewmember duty records is a form of monitoring and could be conducted during the normal course of duties.
2. Medium. Line managers and departmental or key management personnel may observe and review day-to-day activity, noting work task inconsistencies and potential Par 3-5 Page 30
safety issues. Flight operations quality assurance (FOQA) and Line Operations Safety Audit (LOSA) programs may also be sources of information to monitor operations.
3. Large. Monitoring may involve multiple levels of management, safety professionals, functional area managers (such as DO, DOM, Chief Inspector, and Chief Pilot), trained auditors/analysts, and teams/groups of line managers. As in medium-size carriers, FOQA and LOSA programs may be employed. Operational processes may need to be coordinated across adjacent work function boundaries, so effective monitoring may also need to be coordinated.
(2) Monitoring of Operational Environment.
(a) SRR. Section 5.71(a)(2).
(b) Related Regulatory Requirements. None.
(c) ICAO/SMSPP. Element 3.1/Process 3.1.1.
(d) Objective. To monitor the operational environment to identify new or changed conditions.
(e) Part 5 Requirements.
§ 5.71 Safety performance monitoring and measurement.
(a) The certificate holder must develop and maintain processes and systems to acquire data with
respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, the following: …
(2) Monitoring of the operational environment to detect changes.
(f) Discussion. Monitoring of the operational environment involves practices that are similar to those of monitoring operational processes. The context for monitoring the operational environment of a system is developed from the system analysis that is conducted under SRM. Once the scope of the operational environment is defined under SRM, the operational environment must be monitored to assess impacts on aviation safety. For example, increases in the price of fuel may require airlines to change their scheduling, routes, and aircraft utilization.
(3) Auditing of Operational Processes and Systems.
(a) SRR. Section 5.71(a)(3).
(b) Related Regulatory Requirements. Section 5.23(a)(2)(ii).
(c) ICAO/SMSPP. Element 3.1/Processes 3.1.2 and 3.1.4.
(d) Objective. To provide the process owners with a means to obtain information about the performance of systems in their area of responsibility.
Par 3-5 Page 31
(e) Part 5 Requirements.
§ 5.71 Safety performance monitoring and measurement.
(a) The certificate holder must develop and maintain processes and systems to acquire data with respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, the following: …
(3) Auditing of operational processes and systems.
(f) Discussion.
1. Audits are a means of collecting data to confirm whether or not actual practices are being followed within a department. Audits should typically involve the operational management responsible for the system(s) being audited. Procedures for auditing should describe your audit process, criteria, scope, frequency, method for selecting auditors, and methods of documentation and recordkeeping. Audit planning should take into account the safety criticality of the processes to be audited and the results of previous audits. Auditors should not audit their own work, but may audit the work of others around them in the same department. Audit procedures should include the responsibilities and expectations for planning, conducting, reporting results of audits, maintaining records of audit results, and processes for auditing contractors and vendors, as necessary.
2. The results of audits can be recorded in paper format such as a common logbook-style binder, or in electronic media such as a desktop spreadsheet program or a program such as a WBAT.
(g) Scalability.
1. Small. The auditing process could be carried out periodically by the accountable executive/owner, key management person, or a trained employee as a collateral duty. Audits may also be carried out as a subfunction of normal business processes. For example, comparisons of deferred maintenance logs and repair part receipts are a form of safety auditing that are probably already accomplished routinely.
2. Medium. In a medium-size organization, the auditing process can be accomplished by operational departmental personnel, on a periodic basis, as determined by the needs of operational decision makers.
3. Large. In a large organization, the auditing processes are typically fulfilled by divisional auditors on a consistent basis. Large companies may already have safety and safety/quality auditors who perform this function or, in smaller divisions, they may be performed by personnel from inside the divisions.
(4) Evaluation of SMS and Operational Processes and Systems.
(a) SRR. Section 5.71(a)(4).
(b) Related Regulatory Requirements. Section 5.25(c)(3), Responsibilities of Safety Management Personnel.
Par 3-5 Page 32