assessment process, as specified in § 5.55(b). Examples of risk controls include new processes, equipment, training, new supervisory controls, new equipment or hardware, new software, changes to staffing arrangements, or any of a number of other system changes. In short, anything that would lessen the likelihood or severity of a potential incident/accident.
2. Next, look at the system with the proposed control in place to determine if the level of risk is now acceptable and the proposed control does not introduce unintended consequences or new hazards. This is commonly referred to as substitute risk. Section 5.55(d) requires you to evaluate whether the risk will be acceptable with the proposed safety risk control applied. The risk that remains is often called residual risk.
(g) Scalability.
1. Small. The risk control process could be a documented activity performed by the owner/manager and/or another employee(s) designing and evaluating the risk controls. It might be performed in conjunction (by the same individual/group) with system description and analysis, hazard identification, risk analysis, and risk assessment.
2. Medium and Large. The risk control process could be performed by a member of management or SMS management representatives with a small workgroup of company SMEs and stakeholders to design the risk controls. There would be interdepartmental coordination before the controls are implemented. After the control is approved, it is implemented and documented through the company’s publication system. Implementation of risk controls may include distribution of manual revisions and training of company personnel.
e. SRM Example. The scenario below illustrates how these SRM processes would be applied in an operational context.
(1) An airline receives notification that the instrument landing system (ILS) at one of its destination airports is scheduled to be out of service for a period of time and, therefore, the flights will have to use a less familiar Nonprecision Approach (NPA) procedure. Concern is expressed by the flightcrews about the procedure and the differences between fleets that will affect operational procedures and the currency of training. Under § 5.51, this triggers the SRM processes and procedures.
(2) Pursuant to § 5.53(a) and (b), the SRM process starts with an analysis of the systems involved, with the addition of a new procedure for the aircraft fleet. This would include analysis of the flight operations and training program. In this case, the company has two fleets of airplanes that are operated to the destination airport in question. The equipment, and therefore the procedures and flightcrew training in these two fleets, differs. By analyzing the system, they identify a hazard (per § 5.53(c)), in that the pilots are not properly trained to fly the procedures that are required for that specific airport.
(3) To analyze the risk associated with this hazard (as required by § 5.55(a)), a team is assembled with representatives from management, the training and standardization organization, the flight safety organization, the organization responsible for dispatch and operational control, and the pilot’s union. Representatives of the FAA Certificate Management Team (CMT) also Par 3-4 Page 27
attend the work team’s meetings. The risk analysis consists of documented discussions among the assembled SMEs.
(4) As a result of the risk analysis, a risk assessment is done (per § 5.55(b)). As part of the assessment, it is determined that the existing operational procedures and training used by one of the fleets are not acceptable for safe operation into the airport without revision and additional training. A management decision is made that the procedures and associated training currently in use will not provide an acceptable level of safety (ALoS) for operations into the airport.
(5) After determining that the risk is unacceptable, risk controls are designed under § 5.55(c). The risk control in this case consists of a change to operational procedures, including programming of the flight management system (FMS) and associated approach procedure training for flightcrew members. Prior to implementation of the risk controls, the company checks the resulting procedures in flight simulators to ensure that the risk control will mitigate risk to an acceptable level, as required by § 5.55(d).
(6) Per § 5.23(b), the solution is submitted to the fleet Chief Pilots for management review and acceptance of the mitigated risk. Division-level management and top management are briefed on the operation at regular management reviews. Once the controls are approved by all concerned, they are implemented operationally. Per the SA requirements, these controls are specifically targeted for performance monitoring by the operational control/dispatch organization and the flight operations organization line checks.
3-5. SUBPART D: SAFETY ASSURANCE.
a. Introduction.
(1) SRR. Part 5, subpart D.
(2) Related Regulatory Requirements.
• Section 5.23(a)(2), Accountabilities of Members of Management.
• Section 5.25(a)(4), Accountable Executive—Ultimate Responsibility for Safety Performance.
• Section 5.25(b)(1), Accountable Executive—Responsibility to Ensure SMS Implementation and Performance.
(3) ICAO/SMSPP. Component 3.0.
(4) Summary. SA consists of the following five processes, divided into three sections of the regulation.
(a) Safety Performance Monitoring and Measurement § 5.71.
• System monitoring,
• Data acquisition, and
• Analysis.
Par 3-4 Page 28
(b) Safety Performance Assessment § 5.73. System assessment.
(c) Continuous Improvement § 5.75. Continuous improvement.
NOTE: Figure 3-3, Safety Assurance Processes and Regulatory Requirements, depicts the SA processes and their associated regulatory requirements.
FIGURE 3-3. SAFETY ASSURANCE PROCESSES AND REGULATORY
REQUIREMENTS SA 5.71Safety (Subpart c) Performance 5.71(a)(1)&(2) Monitoring Monitoring Inputs From SRM:
5.71(a) and Measurement
5.71(a)(3) Internal Audits
5.71(a)(4) Internal Evaluation *
5.71(a)(5)&(6) Investigations
5.71(a)(7) Employee Reporting How is this going to be 5.71(b) Analysis of Data analyzed? By whom?
5.75 Continuous System Operation/ Monitoring Analysis Data Acquisition & Process Preventive/ Corrective Action System Assmt Improvement
5.73(a) System Assessment 5.73 Safety
5.73(a) Management Review Performance Outputs To SRM:
Assessment 5.73(b) Note: Each data source should be traceable * through analysis 5.71(b) assessment 5.73(a) 5.75 And Corrective Action 5.75 where necessary.
b. Safety Performance Monitoring and Measurement: Overview. This section of the regulation consists of the following processes: Monitoring of Operational Processes (subparagraph 3-5c(1)), Monitoring of the Operational Environment (subparagraph 3-5c(2), Data Acquisition (subparagraphs 3-5c(3)–(7)), and Analysis of Data (subparagraph 3-5d).
(1) SRR. Section 5.71.
(2) Related Regulatory Requirements. None.
(3) ICAO/SMSPP. Element 3.1.
Par 3-5 Page 29
c. Safety Performance Monitoring and Measurement: Required Data Acquisition Processes. This subsection consists of the following seven data acquisition processes:
Monitoring of Operational Processes, Monitoring of the Operational Environment, Auditing of Operational Processes, Evaluation of the SMS and Operational Processes, Investigations:
Accidents and Incidents, and Investigation: Potential Noncompliance and Confidential Employee Reporting System.
(1) Monitoring of Operational Processes.
(a) SRR. Section 5.71(a)(1).
(b) Related Regulatory Requirements.
• Part 121, § 121.703, Service Difficulty Reports.
• Section 121.705, Mechanical Interruption Summary Reports.
(c) ICAO/SMSPP. Element 3.1/Process 3.1.1.
(d) Objective. To observe the day to day, flight by flight, job by job performance of operational systems and their associated risk controls.
(e) Part 5 Requirements.
§ 5.71 Safety performance monitoring and measurement.
(a) The certificate holder must develop and maintain processes and systems to acquire data with
respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, the following:
(1) Monitoring of operational processes.
(f) Discussion. Monitoring operational processes is what supervisors do on a day-to-day basis (e.g., direct supervision of employee activities, monitoring of pilot currency, and monitoring Minimum Equipment List (MEL) status). Monitoring also involves reviewing data that is collected for operational purposes to look for anything of safety significance (e.g., duty logs, crew reports, work cards, process sheets, and reports from the employee safety feedback system). This may include monitoring products and services from outside sources that are used in the certificate holder’s operations.
(g) Scalability.
1. Small. Most of the data/information-gathering for monitoring of operational processes will likely occur as a normal business process by the management personnel who are directly involved in the day-to-day operations. For example, regularly reviewing (e.g., weekly, monthly, or quarterly) the flight dispatch logs and crewmember duty records is a form of monitoring and could be conducted during the normal course of duties.
2. Medium. Line managers and departmental or key management personnel may observe and review day-to-day activity, noting work task inconsistencies and potential Par 3-5 Page 30
safety issues. Flight operations quality assurance (FOQA) and Line Operations Safety Audit (LOSA) programs may also be sources of information to monitor operations.
3. Large. Monitoring may involve multiple levels of management, safety professionals, functional area managers (such as DO, DOM, Chief Inspector, and Chief Pilot), trained auditors/analysts, and teams/groups of line managers. As in medium-size carriers, FOQA and LOSA programs may be employed. Operational processes may need to be coordinated across adjacent work function boundaries, so effective monitoring may also need to be coordinated.
(2) Monitoring of Operational Environment.
(a) SRR. Section 5.71(a)(2).
(b) Related Regulatory Requirements. None.
(c) ICAO/SMSPP. Element 3.1/Process 3.1.1.
(d) Objective. To monitor the operational environment to identify new or changed conditions.
(e) Part 5 Requirements.
§ 5.71 Safety performance monitoring and measurement.
(a) The certificate holder must develop and maintain processes and systems to acquire data with
respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, the following: …
(2) Monitoring of the operational environment to detect changes.
(f) Discussion. Monitoring of the operational environment involves practices that are similar to those of monitoring operational processes. The context for monitoring the operational environment of a system is developed from the system analysis that is conducted under SRM. Once the scope of the operational environment is defined under SRM, the operational environment must be monitored to assess impacts on aviation safety. For example, increases in the price of fuel may require airlines to change their scheduling, routes, and aircraft utilization.
(3) Auditing of Operational Processes and Systems.
(a) SRR. Section 5.71(a)(3).
(b) Related Regulatory Requirements. Section 5.23(a)(2)(ii).
(c) ICAO/SMSPP. Element 3.1/Processes 3.1.2 and 3.1.4.
(d) Objective. To provide the process owners with a means to obtain information about the performance of systems in their area of responsibility.
Par 3-5 Page 31
(e) Part 5 Requirements.
§ 5.71 Safety performance monitoring and measurement.
(a) The certificate holder must develop and maintain processes and systems to acquire data with respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, the following: …
(3) Auditing of operational processes and systems.
(f) Discussion.
1. Audits are a means of collecting data to confirm whether or not actual practices are being followed within a department. Audits should typically involve the operational management responsible for the system(s) being audited. Procedures for auditing should describe your audit process, criteria, scope, frequency, method for selecting auditors, and methods of documentation and recordkeeping. Audit planning should take into account the safety criticality of the processes to be audited and the results of previous audits. Auditors should not audit their own work, but may audit the work of others around them in the same department. Audit procedures should include the responsibilities and expectations for planning, conducting, reporting results of audits, maintaining records of audit results, and processes for auditing contractors and vendors, as necessary.
2. The results of audits can be recorded in paper format such as a common logbook-style binder, or in electronic media such as a desktop spreadsheet program or a program such as a WBAT.
(g) Scalability.
1. Small. The auditing process could be carried out periodically by the accountable executive/owner, key management person, or a trained employee as a collateral duty. Audits may also be carried out as a subfunction of normal business processes. For example, comparisons of deferred maintenance logs and repair part receipts are a form of safety auditing that are probably already accomplished routinely.
2. Medium. In a medium-size organization, the auditing process can be accomplished by operational departmental personnel, on a periodic basis, as determined by the needs of operational decision makers.
3. Large. In a large organization, the auditing processes are typically fulfilled by divisional auditors on a consistent basis. Large companies may already have safety and safety/quality auditors who perform this function or, in smaller divisions, they may be performed by personnel from inside the divisions.
(4) Evaluation of SMS and Operational Processes and Systems.
(a) SRR. Section 5.71(a)(4).
(b) Related Regulatory Requirements. Section 5.25(c)(3), Responsibilities of Safety Management Personnel.
Par 3-5 Page 32
(c) ICAO/SMSPP. Element 3.1/Process 3.1.3.
(d) Objective. To provide a source of information to the organization regarding the safety performance of operational systems and the SMS.
(e) Part 5 Requirements.
§ 5.71 Safety performance monitoring and measurement.
(a) The certificate holder must develop and maintain processes and systems to acquire data with respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, the following: …
(4) Evaluations of the SMS and operational processes and systems.
(f) Discussion. An evaluation is typically an independent review of the company’s processes, procedures, and systems. The evaluation process builds on the concepts of audit and inspection. An evaluation is an internal oversight tool that provides the accountable executive with a snapshot of the safety performance of the carrier’s operational processes and systems, as well as SMS processes. The evaluation should include all available data about the organization, including information from the audits conducted by the operational management.
1. Conducting evaluations at planned intervals will help the carrier’s management to determine if its safety management methods and practices are meeting safety objectives and expectations set out in the safety policy. Evaluation planning should take into account the safety criticality of the processes that are being evaluated and the results from previous evaluations. The scope, content, and frequency of evaluations should be based on the decision maker’s need for information to assess the health of operational processes and the SMS.
You will also need to define criteria for selecting evaluators.
2. The results of evaluations can be recorded in a paper or electronic medium in a common logbook-style binder, an electronic file folder, or a secure email account.
(g) Scalability.
1. Small. The evaluation process could be carried out periodically by the accountable executive/owner, a key management person, or designated employees as a collateral duty under the direction of the accountable executive.
2. Medium. This process could be accomplished by the Director of Safety (DOS) or Safety Department on a monthly, quarterly, or other periodic basis, as determined by the information needs of the accountable executive or other senior management decision makers.
Personnel resources to perform the observations and data collection for evaluations could be from a small, dedicated department or selected line personnel as a collateral duty.
3. Large. Evaluations could be accomplished by a Safety Department or an Internal Evaluation Program (IEP) office on a quarterly, annual, or other periodic basis, as determined by the information needs of the accountable executive or other senior management decision makers. Most part 121 companies have IEPs, and their outputs can be integrated into the SMS. Analysis of evaluations is typically performed by a Safety Department. The resulting data Par 3-5 Page 33
would be acted upon by the appropriate operational department with the Safety Department managing the data and assisting the responsible division(s) in resolving their issues. Most large organizations have standing management committees that consider results of evaluations and any corrective action needed.
(5) Investigation of Incidents and Accidents.
(a) SRR. Section 5.71(a)(5).
(b) Related Regulatory Requirements. None.
(c) ICAO/SMSPP. Element 3.1/Process 3.1.5.
(d) Objective. To gather information on accidents and incidents to identify potential weaknesses in operational systems.
(e) Part 5 Requirements.
§ 5.71 Safety performance monitoring and measurement.
(a) The certificate holder must develop and maintain processes and systems to acquire data with
respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, the following: …
(5) Investigations of incidents and accidents.
(f) Discussion. Investigations should be treated as an opportunity for organizational learning to prevent a repeat of errors and/or change company processes so that mistakes do not recur. Investigations should focus on what went wrong rather than who caused the error and emphasize improvement of safety performance. The organization should include investigation data, if available, from outside sources such as FAA or National Transportation Safety Board (NTSB) investigations and may, where appropriate, participate as a party to official investigations. The results of investigations can be recorded in paper or electronic medium in a common logbook-style binder, an electronic file folder, or other electronic system such as a WBAT or another suitable system.
(g) Scalability.
1. Small. Investigations can be conducted by the accountable executive or assigned employees.
2. Medium. Investigations can be conducted by a Safety Department, with additional assigned line personnel providing technical expertise.
3. Large. Investigations, depending on the situation, can be conducted by safety teams with specialized disciplines.
(6) Investigation of Potential Noncompliance.
(a) SRR. Section 5.71(a)(6).
Par 3-5 Page 34
(b) Related Regulatory Requirements. None.
(c) ICAO/SMSPP. Element 3.1/Process 3.1.5.
(d) Objective. To obtain information to determine compliance with regulations as well as underlying policies and procedures.
(e) Part 5 Requirements.
§ 5.71 Safety performance monitoring and measurement.
(a) The certificate holder must develop and maintain processes and systems to acquire data with respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, the following: …
(6) Investigations of reports regarding potential non-compliance with regulatory standards or other safety risk controls established by the certificate holder through the safety risk management process established in subpart B of this part.
(f) Discussion. This subject is very similar to the one described above in that the focus of the investigation should reveal information that, when utilized correctly, will concentrate on objective facts to identify system deficiencies, help prevent future recurrences, and improve system reliability. It is not as important to identify “who did it” as it is for you to learn why it happened. Within this process, it is important to distinguish between error and intentional/willful noncompliant actions. Investigations of reports regarding potential noncompliance with regulatory standards or of inadequate safety risk controls established by the certificate holder should be mitigated through the SRM process established in part 5, subpart B.
Instances of noncompliance with an FAA regulation may be reported through the Voluntary Disclosure Reporting Program (VDRP), where applicable. For instances involving individual employee noncompliance with FAA regulations, these employees may use an Aviation Safety Action Program (ASAP), if one is available.
(g) Scalability. Methods of conducting investigations of potential noncompliance can be accomplished in a manner similar to that for investigations of accidents and incidents.
(7) Confidential Employee Reporting System.
(a) SRR. Section 5.71(a)(7).
(b) Related Regulatory Requirements. None.
(c) ICAO/SMSPP. Element 3.1/Process 3.1.6.
(d) Summary. To provide a means for employees to communicate safety information to management.
Par 3-5 Page 35
(e) Part 5 Requirements.
§ 5.71 Safety performance monitoring and measurement.
(a) The certificate holder must develop and maintain processes and systems to acquire data with respect to its operations, products, and services to monitor the safety performance of the organization. These processes and systems must include, at a minimum, the following: …
(7) A confidential employee reporting system in which employees can report hazards, issues, concerns, occurrences, incidents, as well as propose solutions and safety improvements.
(f) Discussion. The workforce is an important information source that should be included in the data-gathering process. Audits and evaluations bring decision makers important information and are essential to a proactive SA approach. However, these tools can be limited by the scope and content of their design. Front-line employees may observe aspects of the operation or environment that were not expected and were not included in audit or evaluation protocols. In this respect, the employee reporting system can fill in important gaps in the company’s data collection process.
1. In order to be effective, the organization needs to establish and maintain an environment in which employees feel comfortable to report hazards, issues, and concerns, as well as occurrences, incidents, etc., and propose safety solutions and improvements. The accountable executive and management team need to encourage employees to report safety issues and not fear reprisals from management. Policies that assure employees of fair treatment and clear standards of behavior are an essential part of the reporting process.
2. A key aspect of the confidential reporting system is that it is confidential.
Therefore, you must define methods for employee reporting and de-identification of sources without losing essential information. As you develop and employ the confidential reporting procedures and include this invaluable input in safety decisionmaking, employees will begin to trust the system to work toward elimination of systemic problems. This, in turn, will stimulate greater participation in employee reporting of safety concerns.
3. ASAPs can be used as part of the employee reporting system for the employee groups covered by the ASAPs. However, the confidential employee reporting system required by part 5 must include all employees in the company. Companies that do not have ASAPs may consider providing employees with a hotline, a suggestion box, or information and forms for the NASA ASRS. If a WBAT is used, this system provides a portal for ASRS reporting. ASRS provides certificated employees with limited immunity in the form of waivers of sanctions for reported events with certain restrictions.
(g) Scalability.
1. Small. An employee reporting system for a small company need not be highly sophisticated to be effective. The employees might report a hazard either orally or in a note or email to their supervisor. Collection tools could be a suggestion box, voice mail “hotline,” etc.
Par 3-5 Page 36
2. Medium and Large. A medium- or large-sized company will most likely have an existing online employee reporting system or ASAPs for some employee groups. Data collection for the reporting system can take many forms, from a simple suggestion box to company Web sites or intranets, or a dedicated email address. Data management can be accomplished with common desktop spreadsheet or database software or specialized software, such as a WBAT.
d. Analysis of Data.
(1) SRR. Section 5.71(b).
(2) Related Regulatory Requirements. Section 5.25(c)(2), Responsibilities of Safety Management Personnel.
(3) ICAO/SMSPP. Element 3.1/Process 3.1.7.
(4) Objective. To organize and examine data to determine its meaningfulness in making safety decisions.
(5) Part 5 Requirements.
§ 5.71 Safety performance monitoring and measurement.
(b) The certificate holder must develop and maintain processes that analyze the data acquired through the processes and systems identified under paragraph (a) of this section and any other relevant data with respect to its operations, products, and services.
(6) Discussion. Analysis involves examining data acquired from various sources in § 5.71(a) in order to make inferences about the safety performance of operational systems and the SMS.
(a) It is common for organizations to treat each employee report, audit finding, or investigation in isolation. Often, system problems may not be seen if data points are examined in isolation. Thus, analysis processes should also look across individual reports and among various data sources for patterns or trends.
(b) For example, in a recent case, examination of data from an airline’s FOQA program showed several instances of unstabilized approaches and exceeding of flap and landing gear speeds on approach. It would be easy to assume that pilot technique was the cause and then counsel or retrain the pilots. However, upon closer analysis and comparison of the events, the airline found that all of the instances were at one specific airport. Contacts with other airlines indicated similar experiences at the same airport. After further conversations with air traffic control (ATC), the parties concluded that, due to the traffic handling practices, it was common for flights to be vectored close and high, which resulted in the approach problems. After further conversations with ATC management, the problem was resolved.
Par 3-5 Page 37
(c) The following is a starting point for an analysis process.
1. Establish the context: Understand the safety performance objectives of the system, operations, or SMS. For system impacts, and to analyze risk controls developed under SRM, you would also need to review the system analysis conducted under SRM.
2. Identify the objective of the analysis: Are you analyzing the safety performance of a system, of an operation, or the SMS itself?
3. Secure appropriate data: Section 5.71(a) provides a framework for data sources. The data needed may be already on hand, or additional data-gathering may be needed, such as conducting a special audit with focus on a specific problem.
4. Select an appropriate data analysis method: Analysis need not be sophisticated to yield valuable results. For example, analysis of employee reports or qualitative analysis by SMEs may be the best method. If desired, several classification systems exist to help convert subjective, qualitative data into quantitative data for tracking and trend analysis. For routine reporting, analysis may consist of tracking such things as dispatch reliability per month, system or part failure rates, crew utilization/duty time, and events such as minor incidents, diversions, and precautionary engine shutdowns.
5. Recommendation: At this point, the person conducting the analysis may compare performance against relevant company safety objectives. Unless the decision maker is personally conducting the analysis, an assessment recommendation may be made. In the case that a potential regulatory violation is discovered during analysis, the carrier may initiate a self-disclosure under voluntary reporting procedures.
6. Documentation: Prepare reports and records in a format appropriate to your operation.
(d) The outputs from data analysis could be recorded in a simple recording medium such as a notebook, paper files, common desktop software, or specialized systems such as a WBAT.
(7) Scalability.
1. Small. Analysis of the data gathered could be done by the accountable executive, the DOS, other individual managers, or other designated employees as a collateral duty.
2. Medium. Analysis of data could be done by the DOS, other individuals within a Safety Department, or a person(s) within each department, and then shared with other departments and management during regularly scheduled meetings.
3. Large. Operational departments may have their own data analysis group reviewing data and analyzing the data by SMEs within the respective department, possibly supported and coordinated by a Safety Department.
Par 3-5 Page 38