Par 3-5 Page 37
(f) Discussion. The workforce is an important information source that should be included in the data-gathering process. Audits and evaluations bring decision makers important information and are essential to a proactive SA approach. However, these tools can be limited by the scope and content of their design. Front-line employees may observe aspects of the operation or environment that were not expected and were not included in audit or evaluation protocols. In this respect, the employee reporting system can fill in important gaps in the company’s data collection process.
1. In order to be effective, the organization needs to establish and maintain an environment in which employees feel comfortable to report hazards, issues, and concerns, as well as occurrences, incidents, etc., and propose safety solutions and improvements. The accountable executive and management team need to encourage employees to report safety issues and not fear reprisals from management. Policies that assure employees of fair treatment and clear standards of behavior are an essential part of the reporting process.
2. A key aspect of the confidential reporting system is that it is confidential.
Therefore, you must define methods for employee reporting and de-identification of sources without losing essential information. As you develop and employ the confidential reporting procedures and include this invaluable input in safety decisionmaking, employees will begin to trust the system to work toward elimination of systemic problems. This, in turn, will stimulate greater participation in employee reporting of safety concerns.
3. ASAPs can be used as part of the employee reporting system for the employee groups covered by the ASAPs. However, the confidential employee reporting system required by part 5 must include all employees in the company. Companies that do not have ASAPs may consider providing employees with a hotline, a suggestion box, or information and forms for the NASA ASRS. If a WBAT is used, this system provides a portal for ASRS reporting. ASRS provides certificated employees with limited immunity in the form of waivers of sanctions for reported events with certain restrictions.
(g) Scalability.
1. Small. An employee reporting system for a small company need not be highly sophisticated to be effective. The employees might report a hazard either orally or in a note or email to their supervisor. Collection tools could be a suggestion box, voice mail “hotline,” etc.
2. Medium and Large. A medium- or large-sized company will most likely have an existing online employee reporting system or ASAPs for some employee groups. Data collection for the reporting system can take many forms, from a simple suggestion box to company Web sites or intranets, or a dedicated email address. Data management can be accomplished with common desktop spreadsheet or database software or specialized software, such as a WBAT.
Par 3-5 Page 38
d. Analysis of Data.
(1) SRR. Section 5.71(b).
(2) Related Regulatory Requirements. Section 5.25(c)(2), Responsibilities of Safety Management Personnel.
(3) ICAO/SMSPP. Element 3.1/Process 3.1.7.
(4) Objective. To organize and examine data to determine its meaningfulness in making safety decisions.
(5) Part 5 Requirements.
(6) Discussion. Analysis involves examining data acquired from various sources in § 5.71(a) in order to make inferences about the safety performance of operational systems and the SMS.
(a) It is common for organizations to treat each employee report, audit finding, or investigation in isolation. Often, system problems may not be seen if data points are examined in isolation. Thus, analysis processes should also look across individual reports and among various data sources for patterns or trends.
(b) For example, in a recent case, examination of data from an airline’s FOQA program showed several instances of unstabilized approaches and exceeding of flap and landing gear speeds on approach. It would be easy to assume that pilot technique was the cause and then counsel or retrain the pilots. However, upon closer analysis and comparison of the events, the airline found that all of the instances were at one specific airport. Contacts with other airlines indicated similar experiences at the same airport. After further conversations with air traffic control (ATC), the parties concluded that, due to the traffic handling practices, it was common for flights to be vectored close and high, which resulted in the approach problems. After further conversations with ATC management, the problem was resolved.
(c) The following is a starting point for an analysis process.
1. Establish the context: Understand the safety performance objectives of the system, operations, or SMS. For system impacts, and to analyze risk controls developed under SRM, you would also need to review the system analysis conducted under SRM.
2. Identify the objective of the analysis: Are you analyzing the safety performance of a system, of an operation, or the SMS itself?
§ 5.71 Safety performance monitoring and measurement.
(b) The certificate holder must develop and maintain processes that analyze the data acquired through the processes and systems identified under paragraph (a) of this section and any other relevant data with respect to its operations, products, and services.
Par 3-5 Page 39
3. Secure appropriate data: Section 5.71(a) provides a framework for data sources. The data needed may be already on hand, or additional data-gathering may be needed, such as conducting a special audit with focus on a specific problem.
4. Select an appropriate data analysis method: Analysis need not be sophisticated to yield valuable results. For example, analysis of employee reports or qualitative analysis by SMEs may be the best method. If desired, several classification systems exist to help convert subjective, qualitative data into quantitative data for tracking and trend analysis. For routine reporting, analysis may consist of tracking such things as dispatch reliability per month, system or part failure rates, crew utilization/duty time, and events such as minor incidents, diversions, and precautionary engine shutdowns.
5. Recommendation: At this point, the person conducting the analysis may compare performance against relevant company safety objectives. Unless the decision maker is personally conducting the analysis, an assessment recommendation may be made. In the case that a potential regulatory violation is discovered during analysis, the carrier may initiate a self-disclosure under voluntary reporting procedures.
6. Documentation: Prepare reports and records in a format appropriate to your operation.
(d) The outputs from data analysis could be recorded in a simple recording medium such as a notebook, paper files, common desktop software, or specialized systems such as a WBAT.
(7) Scalability.
1. Small. Analysis of the data gathered could be done by the accountable executive, the DOS, other individual managers, or other designated employees as a collateral duty.
2. Medium. Analysis of data could be done by the DOS, other individuals within a Safety Department, or a person(s) within each department, and then shared with other departments and management during regularly scheduled meetings.
3. Large. Operational departments may have their own data analysis group reviewing data and analyzing the data by SMEs within the respective department, possibly supported and coordinated by a Safety Department.
e. Safety Performance Assessment.
(1) SRR. Section 5.73.
(2) Related Regulatory Requirements. Sections 5.23(a)(2)(ii) and 5.25(b)(1).
(3) ICAO/SMSPP. Element 3.1/Processes 3.1.8, 3.2, and 3.3.2.
Par 3-5 Page 40
(4) Objective. To make decisions regarding safety performance with reference to safety objectives and regulatory compliance.
(5) Part 5 Requirements.
(6) Discussion. Under § 5.71, collected safety performance data is analyzed and the results are used for informed decisionmaking. The assessment process is where these decisions are made. Decisions are made by personnel with assigned responsibility and authority. The SA process should consider who makes the decisions regarding whether the company’s safety performance is effective and whether the company is meeting its safety objectives and expectations that are identified in the safety policy required by § 5.21. The conclusions of the safety assessments are reported to the accountable executive, who possesses ultimate authority to act on such conclusions, as necessary.
(a) Assessments can have one of the following general outcomes:
• Performance is acceptable and objectives are being met.
• Performance is not acceptable, and analysis suggests that the problem lies with conformity with regulations or company policy and procedures, or necessary resources have not been provided. In the event this occurs, corrective action under § 5.75 would be warranted.
• Conformity with the risk controls and regulations appears to be satisfactory;
however, desired results are not being obtained. In the event that this occurs, the SRM processes would be triggered.
• New or uncontrolled hazards are discovered. This may be due to new hazards having arisen since the system was designed or discovery of factors that were overlooked. In this case, as in the previous, the SRM processes must be followed.
(b) The results of assessments can be recorded in a paper or electronic medium in a common logbook-style binder, an electronic file folder, common desktop software, or specialized system, such as a WBAT.
§ 5.73 Safety performance assessment.
(a) The certificate holder must conduct assessments of its safety performance against its safety objectives, which include reviews by the accountable executive, to:
(1) Ensure compliance with the safety risk controls established by the certificate holder;
(2) Evaluate the performance of the SMS.
(3) Evaluate the effectiveness of the safety risk controls established under § 5.55(c) and identify any ineffective controls.
(4) Identify changes in the operational environment that may introduce new hazards.
(5) Identify new hazards.
(b) Upon completion of the assessment, if ineffective controls or new hazards are identified under paragraph (a)(2) through (a)(5) of this section, the certificate holder must use the safety risk management process described in subpart C of this part.
Par 3-5 Page 41
(7) Scalability.
(a) Small and Medium. As an organization grows in size, it is normal to have additional personnel performing safety, quality, or internal evaluation functions. An SMS does not change the number and types of personnel in these situations as much as it may change the way in which these persons and organizations work and interact. For example, safety performance and assessment could be a documented activity performed by the accountable executive or a coordinated activity between the accountable executive and other operational managers, supported and coordinated by the DOS. Risk acceptance would also normally fall to managers within this group.
(b) Large. This process is best addressed at the highest level in the organization and involves the accountable executive, division vice presidents, and other defined leaders and decision makers. At each level, the company would define who is responsible for making risk acceptance decisions and what actions should be taken to either correct the problem or design new risk controls. Larger companies typically have standing management committees at the functional organization level (e.g., flight operations, technical operations/maintenance, in-flight services, dispatch/operational control) and a second body at the corporate level to assure integration, coordination, and review by the accountable executive.
f. Continuous Improvement.
(1) SRR. Section 5.75.
(2) Related Regulatory Requirements. None.
(3) ICAO/SMSPP. Element 3.3/Process 3.3.1.
(4) Objective. To provide the organization’s decision makers with a means to correct safety performance deficiencies identified in the safety performance assessment process.
(5) Part 5 Requirements.
(6) Discussion.
(a) The final step within SA is continuous improvement. This process is designed to ensure that you are correcting substandard safety performance identified during the safety performance assessment in order to continuously improve safety performance.
(b) As discussed in subparagraph 3-5e, Safety Performance Assessment), the corrective action process of § 5.75 is triggered when conformity with risk controls has been found to be deficient. In this case, it is not necessary to conduct a new safety risk analysis. Risk has already been assessed as being unacceptable without satisfactory completion of the risk control. For example, if it has been found that an Airworthiness Directive (AD) has not been § 5.75 Continuous improvement.
The certificate holder must establish and implement processes to correct safety performance deficiencies identified in the assessments conducted under § 5.73.
Par 3-5 Page 42 applied to a particular aircraft, the only correct action is to comply with the risk control (in this case, the AD). The risk of flying the airplane without the AD has already been assessed as unacceptable.
(7) Scalability. Continuous improvement decisionmaking is an output of the performance assessment process. Therefore, the arrangements discussed for assessment apply to scalability for continuous improvement activities. The managers, committees, or working groups that make assessment decisions would also lay out courses of corrective action.
g. An SA Example. This example builds on the scenario discussed in the SRM example, which can be found in the SRM section, subparagraph 3-4e. In that example, an airline addressed the unacceptable risk of an airport approach procedure by imposing risk controls consisting of a change to operational procedures, including programming of the FMS and associated approach procedure training for flightcrew members.
(1) As required by §§ 5.71(a)(1) and 5.73(a)(1) and (a)(3), the air carrier specifically monitored the flight operations and the associated controls at the airport by the operational control/dispatch organization as part of their normal activities. This organization was tasked with assuring that crews assigned to the specific flights were appropriately qualified (thus verifying that this part of the risk control was being employed), and that the airplanes scheduled for the flights had appropriate operational equipment. As required by § 5.71(a)(2), the air carrier also monitored the airport to determine whether the altered approach procedure was still necessary.
(2) Another part of the risk control monitoring and auditing was conducted using structured line checks pursuant to § 5.71(a)(3). This method was used to conduct audits of specific flight operations (in addition to the airline’s normal checking procedures), thus providing data to verify that the procedures put in place provided risk controls that are effective, as required by § 5.73(a)(3). At the same time, per § 5.71(a)(7), the airline’s employee reporting system was available for pilots to submit their concerns to the company. Line check and employee reporting data were collected and analyzed under § 5.71, and then assessed by management under § 5.73.
(3) If any deficiencies were found, an analysis would have been conducted to ascertain whether the deficiencies should be addressed through SRM (per § 5.73(b)) or a correction (per § 5.75). If the root cause of the deficiencies was shortfalls in compliance with the designed operational process (e.g., required training was not completed), the deficiency should be addressed through a correction, per § 5.75. However, if the deficiency was associated with the basic design of the control (e.g., inadequate training curriculum), the SRM process would be repeated to revise the procedure in the areas where it was found deficient, per § 5.73(b).
3-6. SUBPART E: SAFETY PROMOTION.
a. Introduction.
(1) SRR. Part 5, subpart E, Safety Promotion
(2) Related Regulatory Requirements. Section 5.23(a)(2)(ii), Accountabilities of Members of Management.
Par 3-6 Page 43
(3) ICAO/SMSPP. Component 4.0.
(4) Objective. To provide employees with effective SMS training commensurate with their safety responsibilities, and to create a means to deliver organization-wide safety communication.
(5) Discussion. Because a key component of SMSs is the effective control of risk, every member in your organization must understand and take responsibility for the role they play in controlling risk by their actions and behavior. These members should also have access to up-to-date safety information so that they can properly fulfill their roles.
b. Competencies and Training.
(1) SRR. Section 5.91.
(2) Related Regulatory Requirements. None.
(3) ICAO/SMSPP. Element 4.1/Processes 4.1.1 and 4.1.2.
(4) Objective. To assure that employees are competent to perform their SMS-related duties.
(5) Part 5 Requirements.
(6) Discussion. Your organization will need to train its employees. Your employees may receive initial safety training for them to understand and perform their safety duties.
Recurrent training may also be necessary to reinforce these skills. For example, a Safety Department employee will probably need more detailed SMS training (such as safety risk analysis, system evaluation, system assessment, and data mining, auditing, and inspections) than will a baggage handler. Line managers may need to know the potential consequences of safety failures and system failure modes more so than line employees will. Line employees may only need an overview of safety management fundamentals for their operational department (e.g., instructions on how to identify and report safety concerns).
(a) Competency is an observable, measurable set of skills, knowledge, abilities, behaviors, and other characteristics that individuals exhibit as they successfully perform work functions. Competencies are typically required at different levels of proficiency depending on the work roles or occupational function. By training your employees, you should establish competencies for all employees, commensurate with their duties relevant to the operation and performance of the SMS. Competence can be assessed at the completion of training by written, oral, or demonstration tests, and then measured periodically during the performance of that individual’s work by way of periodic evaluations or supervisor/management observations. As § 5.91 Competencies and training.
The certificate holder must provide training to each individual identified in § 5.23 to ensure the individuals attain and maintain the competencies necessary to perform their duties relevant to the operation and performance of the SMS.
Par 3-6 Page 44 part of SA, organizations should periodically review their training program(s) to ensure that those programs meet the objectives set out in the safety policy.
(b) It is the responsibility of your organization to determine its own training needs based on competency requirements. Management personnel, specifically designated by the accountable executive to ensure the SMS is fully implemented, may need to be trained first and may also need specialized training to fulfill their responsibilities. Determining the organization’s training needs starts with a careful review of the safety policy, processes, and objectives.
Everyone working within the scope of SMS should receive training commensurate with their position in the organization.
(7) Scalability. Organizations of all sizes may choose to either train their employees in house or to contract out the training to outside vendors. Whichever option is taken, the training must be specific to your company’s SMS and operations.
c. Safety Communication.
(1) SRR. Section 5.93.
(2) Related Regulatory Requirements. None.
(3) ICAO/SMSPP. Element 4.2.
(4) Objective. To assure that employees have current and pertinent safety information.
(5) Part 5 Requirements.
(6) Discussion.
(a) Effective communication involves adjusting the content of the communication and manner in which the information is delivered to match the target employee’s role in the organization. The accountable executive must ensure that communication mechanisms are available and are effectively utilized. The delivery system should be appropriate according to the size and complexity of the organization.
(b) Safety policy and information could be provided as text, visual media such as posters or short videos, orally, or through examples. Messages should be consistent and in a form employees at each level can relate to, and be delivered using whichever media the organization has available. For example, hazard communications regarding birds for flightcrew members § 5.93 Safety communication.
The certificate holder must develop and maintain means for communicating safety information that, at a minimum:
(a) Ensures that employees are aware of the SMS policies, processes, and tools that are relevant to their responsibilities.
(b) Conveys hazard information relevant to the employee’s responsibilities.
(c) Explains why safety actions have been taken.
(d) Explains why safety procedures are introduced or changed.
Par 3-6 Page 45 (regarding new bird strike avoidance techniques) may be in a “Notices” section of the Flight Operations Manual (FOM), and may be reinforced by recurrent training. Hazard communications made to line maintenance technicians (regarding birds roosting and nesting in flight controls, auxiliary power unit (APU) intakes, and engine cowlings) may be conveyed by posters and changes to daily inspection procedures. Hazard communications regarding birds made to ground service personnel may be in the form of posters, videos, and demonstrations (cleaning and removing bird droppings from windshields).
(7) Scalability.
(a) Small. Communicating safety considerations to employees will probably be simple and direct. For example, the company owner or the organization could conduct regular all-hands/employee meetings, such as “hangar talk sessions.” Additionally, communication could include regular and periodic briefings to the employees, posting the status of safety issues on bulletin boards, emails to employees, and face-to-face meetings with division management teams.
(b) Medium. Communication methods may be more structured than in small organizations. Safety information may be published throughout the company by printed or electronic means. Safety meetings are likely more structured and formal. Communication and feedback may be formalized in order to provide information to individual employees as well as organization-wide information for cross-boundary issues and/or common hazards.
(c) Large. Communication is more likely to be formal and a tracking system may be used to ensure that the appropriate safety messages are delivered to the appropriate personnel.
Information technology (IT) approaches, such as email broadcasts or intranet Web sites, may be considered to facilitate directing the flow of safety information and recording its accomplishment for evaluation and auditing purposes.
3-7. SUBPART F: SMS DOCUMENTATION AND RECORDKEEPING.
a. Introduction.
(1) SRR. Part 5, Subpart F.
(2) Related Regulatory Requirements. None.
(3) ICAO/SMSPP. Element 1.5.
(4) Discussion. You must establish and maintain SMS information, in either paper or electronic form, describing the safety-related processes and procedures. Additionally, you must maintain the SMS records discussed below for a specified period of time.
b. SMS Documentation.
(1) SRR. Section 5.95.
(2) Related Regulatory Requirements. None.
Par 3-7 Page 46
(3) ICAO/SMSPP. Element 1.5.
(4) Part 5 Requirements.
(5) Discussion. You must establish and maintain SMS information, in either paper or electronic form, describing the safety-related processes and procedures and interfaces between these. You should also implement a distribution system to ensure that the documents dealing with these processes and procedures are promptly updated whenever there is a change in one or more of these processes or procedures.
(6) Scalability.
(a) Small and Medium. The owner/manager or designee may be responsible for maintaining and distributing current versions of guidance documents. Documentation may consist of a set of typewritten documents, spreadsheets, and forms that are kept in file cabinets or binders. Managers of medium-size companies need the same type of information to make decisions, although the volume is typically larger than that of a small company and smaller than that of a large organization.
(b) Large. Documentation and recordkeeping processes for a large organization may use a WBAT, unique software applications, or development of new database tools to support risk reporting and analysis. These organizations should examine existing tools and infrastructure, as it is likely that these can be leveraged (modified) to meet SMS requirements.
c. SMS Records.
(1) SRR. Section 5.97.
(2) Related Regulatory Requirements. None.
(3) ICAO/SMSPP. Element 1.5.
§ 5.95 SMS documentation.
The certificate holder must develop and maintain SMS documentation that describes the certificate holder’s:
(a) Safety policy.
(b) SMS processes and procedures.
Par 3-7 Page 47 (and 48)
(4) Part 5 Requirements.
(5) Discussion. The timeline associated with the retention of the documents required by the regulation for outputs of SRM processes would be for as long as they remain relevant to the operation. For risk assessments, this may mean for as long as the air carrier engages in the activity for which the risk assessment was conducted. For risk controls, it may mean for as long as the risk control remains in effect.
(6) Scalability.
(a) Small. The owner/manager or designee may be responsible for maintaining auditable records. Documentation may consist of handwritten records, spreadsheets, and completed forms that are kept in file cabinets or binders.
(b) Medium. An individual or small staff may coordinate document maintenance and retention. This staff may use a combination of paper and electronic media to administer the process. Some records may be retained by department heads in accordance with a procedure delegating this responsibility.
(c) Large. The organization may have a dedicated records staff or department whose duties include document distribution and records retention. Due to the size and complexity of the organization, the use of technology is probably more pronounced.
§ 5.97 SMS records.
(a) The certificate holder must maintain records of outputs of safety risk management processes as described in subpart C of this part. Such records must be retained for as long as the control remains relevant to the operation.
(b) The certificate holder must maintain records of outputs of safety assurance processes as described in subpart D of this part. Such records must be retained for a minimum of 5 years.
(c) The certificate holder must maintain a record of all training provided under § 5.91 for each individual.
Such records must be retained for as long as the individual is employed by the certificate holder.
(d) The certificate holder must retain records of all communications provided under § 5.93 for a minimum of 24 consecutive calendar-months.