Par 5-4 Page 31 task requirements, ensure that individuals maintain and improve their skills, and enhance the work environment. This knowledge can help your CASS analysts perform RCA. Continuing with the previous example of inadequate training, with insufficient awareness of human factors issues, you might trace a maintenance error to a mechanic or technician who appears to have insufficient training for the task, and determine that the solution is more technical training. However, further analysis may reveal that there are contributing flaws in equipment design, job cards, manuals, work environment, or such organizational procedures as shift turnover reports that more training will not satisfactorily overcome. Or, it may turn out that a different kind of training, perhaps involving decision-making skills, is called for.
(d) The FAA is involved in cooperative efforts with the industry and academia in promoting human factors in aviation. This field is rapidly evolving, particularly in its application to aviation maintenance. According to a study conducted for the FAA, maintenance error contributes to a significant portion of air carrier accidents, with shift-turnover errors and work interruptions standing out as leading types of underlying causes. Based on human factors’ growing importance and the human factors information available to the industry, the FAA expects that you will apply concepts of human factors to your CASS surveillance and analysis.
(e) Your CASS surveillance should ensure that an RCA that considers human factors is part of the investigation of individual events by any personnel designated by you to respond to maintenance-related events, such as rejected takeoffs. Otherwise, data reviewed in a CASS could be incomplete.
(f) One challenge presented by the increasing emphasis on human factors is how to balance two seemingly contradictory purposes. On the one hand, the FAA and industry need to encourage personnel to cooperate in addressing system organization and design issues without inhibitions caused by fear of discipline or enforcement. On the other hand, in some cases, individual employees or even senior management may bear a degree of culpability (for example, in deliberately bypassing important controls or committing a serious regulatory infraction in the commission of a maintenance error). In some instances, disciplinary action or even FAA administrative or legal enforcement may be indicated. This is a common issue in the industry. FAA programs designed to promote the greater good of the system, instead of the blame culture described earlier, have become available. These programs encourage voluntary reporting of errors and infractions by aviation personnel and operators without threat of disciplinary action or penalty. Your specific CASS, in any event, should be concerned specifically with identifying and correcting deficiencies in your maintenance program. You should design your CASS with that objective in mind, rather than specific event resolution, even if your CASS analysts research specific events.
b. Analytical Tools and Processes. While it is not necessary that you implement any specific externally developed system, specific analytical tools or processes are available to assist you in your analysis process. Examples of these follow:
(1) Maintenance Error Decision Aid (MEDA). Developed by the Boeing Human Factors Engineering group in collaboration with the FAA, airlines, and the International Association of Machinists, MEDA analyzes human performance issues related to maintenance errors and trends. MEDA is used to track events, investigate and prevent maintenance errors, and
Page 32 Par 5-4 identify contributing factors, corrective actions, and prevention strategies. The group of collaborators has also developed a software analysis package to work with this aid and facilitate analysis of systemic issues.
(2) Managing Engineering Safety Health. Developed by the University of Manchester in collaboration with British Airways (BA) Engineering. It is a system geared toward researching the workplace and organizational environment in aircraft maintenance to find the issues with greatest potential to contribute to human factors problems. The system uses software, diagnostic, and sampling tools. Managing Engineering Safety Health conducts anonymous, survey-like assessments among personnel at the work location, which are then analyzed. This is a more structured, data-intensive approach to determining and monitoring personnel attitudes toward the system than the interview process discussed earlier. The industry has far less practical experience with Managing Engineering Safety Health than with MEDA.
(3) Human Factors Accident Classification System Maintenance Extension.
Developed by the U.S. Naval Safety Center in collaboration with the FAA for use in the air carrier industry as well as naval aviation, this comprehensive system incorporates a number of analytical tools and has profiled maintenance errors and contributing conditions, permitting development of potential prevention measures. While the Human Factors Accident Classification System Maintenance Extension may be more sophisticated than many operators would need, it demonstrates principles and techniques of software-aided analysis that could apply to a CASS.
c. Corrective Action Options.
(1) Determining Whether or Not to Proceed with a Corrective Action. Once your CASS auditors and analysts have identified a problem or deficiency, you must determine if a corrective action is warranted and, if so, the details of the corrective action
(i.e., the corrective action plan).
(2) CASS Procedures. CASS procedures regarding the determination of whether you should proceed with a corrective action. Your CASS procedures should outline:
• How such a determination will be made,
• Who will make the determination, and
• What levels of review, if any, you will perform. A review is a system safety control.
(3) Developing a Corrective Action Proposal. Your technical area personnel should have primary responsibility for developing the corrective action proposal, as they would be most familiar with the technical workings of the area in question and would be sensitive to the possibility of creating new problems as a result of the corrective action. Your CASS procedures should emphasize a team approach. Team members should include your CASS auditors or analysts, your technical area personnel in the affected maintenance disciplines, and perhaps other affected areas such as training or flight operations.
Par 5-4 Page 33
(4) Types of Corrective Actions. There are several possible types of general corrective actions or responses, depending on the outcome of your risk assessment:
(a) Prevent recurrence through engineering or system changes designed to eliminate the risk.
(b) Accept the underlying cause of a trend or discrepancy, but reduce the risk through implementing controls or countermeasures. Examples are training, policy or procedure revisions, or warning devices. Other countermeasures might be modifying or introducing new equipment or technology.
(c) Accept that under certain conditions a discrepancy may occur, but be prepared to contain or mitigate the results of that situation. Your CASS does not necessarily have to implement corrective actions for every apparently negative trend or finding. Analysis of findings or trends may identify problem areas that do not present safety hazards that you are willing to accept, in accordance with your risk assessment process. For example, you might find that a higher than average number of component removals occur at a particular location with “no fault found” as a consequence of a non-safety-related failure. You might determine that the reason for this situation is that your aircraft spends insufficient time on the ground for line maintenance to completely isolate the fault. You might prefer to continue the brief turn times and simply change components. This would be a business decision for you to make. However, more comprehensive corrective actions would be mandatory if your CASS detects that your maintenance program lacks adequate procedures and standards to meet the requirements of part 121 or 135, as applicable.
d. Written Procedures for Developing and Implementing Corrective Actions. Your CASS should provide relatively detailed written procedures for developing and implementing corrective action. Your written procedures should:
(1) Result in a specific corrective action plan that addresses the following:
• Development and proposal of the corrective action.
• Analysis and final approval level of the corrective action, including who is responsible for approval of the corrective action.
• Who will implement the corrective action.
• How the responsible person will implement the corrective action.
• When the corrective action completion due date is.
• Who will evaluate the outcome and how, including identification of data requiring collection, awareness of the possibility of unintended consequences, and events that should trigger a response.
• Who will monitor the status of the corrective action and how.
Page 34 Par 5-4
• Reporting the status of the corrective action (to whom, with what frequency).
(2) Maintain the appropriate role of auditors in developing responses to findings, so that they continue to remain independent from the corrective actions they may subsequently audit.
(3) Distinguish clearly between your technical area personnel’s responsibility for developing and implementing corrective actions, and your CASS personnel’s responsibility for producing the findings and analysis and making sure the technical area involved develops and implements appropriate corrective actions.
(4) Designate the position or organization responsible for evaluating and approving proposed corrective actions. Your CASS director or other designated manager may appoint a corrective action team to design and propose a corrective action. The team—which typically represents a cross section of the departments involved in audits, operational data collection, analysis, and production—oversees the implementation of the corrective action. Technical and reliability control boards are most often used in conjunction with FAA-approved reliability programs; however, a similar concept applies to a CASS, even if no FAA-approved reliability program exists.
e. Corrective Action Risk Assessment.
(1) CASS Procedures Regarding Risk Assessment. Your CASS procedures should:
(a) Specify that your personnel will analyze a proposed corrective action carefully before its selection and implementation to ensure that corrective action is necessary and will actually fix the problem and not lead to unintended, negative consequences.
(b) Remind both CASS and technical area personnel of the need to consider the impact of the proposed corrective action on other aspects of your operation. This includes other areas of your maintenance program, such as manuals. The corrective action may require coordination with other areas, such as flight operations, that it could affect.
(2) Personnel Involved in Risk Assessment.
(a) Your technical area personnel play the key role in risk assessment, but the process should include your CASS analysts, who will act as resources in support of your technical area managers and bring risk assessment and systems analysis techniques to the process. Your auditor and analyst should be qualified (through training or experience) in systems analysis and can contribute to the evaluation of a proposed corrective action by determining if the basic system elements have been considered. However, your technical personnel have the expertise to actually develop and implement the corrective action, and to evaluate it in practical terms. Thus, the corrective action is a result of cooperation between your technical personnel and your CASS personnel.
(b) Personnel working on the proposed corrective actions should ensure they consider issues of a timetable for the corrective action implementation, and the safety attributes of authority, responsibility, procedures, controls, process measurement, and interfaces.
Par 5-4 Page 35
f. Corrective Action Plan.
(1) With the RCA complete, corrective action options identified, and risk assessment performed as appropriate, you can make a final decision on the proposed corrective action plan.
The corrective action plan should address all relevant issues, including a timetable for completion of the action, with milestones if appropriate. The appropriate technical department and, if the corrective action goes beyond your maintenance organization, other departments (such as flight operations) should then implement the plan.
(2) Your CASS procedures should identify:
• How this plan will be approved and at what level of your company.
• The parties responsible for implementing, monitoring, and ensuring that all affected parties are notified, both within your maintenance organization and externally, if necessary.
5-5. FOLLOWUP.
a. Corrective Action Followup. Your CASS procedures should:
(1) Specify how you will evaluate your corrective actions for effectiveness. That is, did your corrective action do what you wanted it to do? This may require the following:
• Followup audits of a specific area,
• Regular communication from the affected technical area about the effects of the corrective action and,
• Other forms of verification by your auditors or analysts tracking the implementation.
(2) Identify the person or entity (such as a CASS board) responsible for determining if any changes in the status of a corrective action are acceptable. Your CASS auditors or analysts have the duty of ensuring that the person implementing the corrective action did so in accordance with the established timetable, or, if not, determining why the timetable has changed.
(3) Include responsibilities and guidelines for:
• Tracking the implementation of corrective actions in accordance with the timeline.
• The role of your auditors, managers, management committees, and senior management.
• How your CASS will use automation or computerized systems.
Page 36 Par 5-5
• How your CASS will use risk assessment and systems analysis to guard against unintended consequences.
• Measures to evaluate the effect of the corrective action.
• The affected technical area to communicate the status of the corrective action to the person responsible for monitoring implementation.
b. Getting Help from a Manufacturer. In some cases, you may require data or assistance from a manufacturer in correcting a deficiency detected by your CASS. Manufacturers may not always assign these issues the same priority as you do. You should provide guidance in your CASS procedures, based on your own particular experience, about how your CASS should address assistance or information from manufacturers, and how to proceed in case of unsatisfactory or slow responses. This may include developing a standardized letter citing the need for this information or assistance to satisfy the requirements of § 121.373, § 135.431, or other pertinent regulations. It may also include working with your FAA principal inspector (PI) to find solutions.
c. Followup Surveillance Plan.
(1) Your CASS procedures should include how to determine the level of followup audits for verifying corrective action implementation. For example, based on the risk assessment or the complexity of your corrective action, the designated CASS analyst or team may schedule special or more frequent audits. They may also change the data collection process or institute other means of verification. The FAA expects you to have a well-defined and logical process to design the followup actions.
(2) The information and analysis performed through the closed-loop, continuous cycle of surveillance, investigations, analysis, and corrective action, permits you to refine your audit and data collection priorities through the risk assessment process.
Par 6-1 Page 37 CHAPTER 6. PERSONNEL WHO PERFORM CASS FUNCTIONS
6-1. PERSONNEL MANAGING CASS FUNCTIONS.
a. Decision-making Group.
(1) Your CASS should include a decision-making group at a relatively high organizational level to oversee or carry out your CASS functions. This oversight group could include:
• Technical boards concerned with performance and other technical issues,
• Administrative boards that may have broader decision-making authority to act on technical recommendations, or
• A single board combining both functions.
(2) The key concept is that there is a decision-making group at a relatively high level to monitor your CASS and to make critical decisions in a timely manner. Typically, if you are a smaller certificate holder, this committee or board might be composed of the president of the company, and the DOM and flight operations. If you are a larger certificate holder, participants might be managers from several departments, such as maintenance and engineering, quality assurance (QA), and flight operations.
b. Group Members.
(1) If you use committees or boards as major decision-making groups for CASS issues, members of these groups should:
• Have an appropriate technical background, and
• Be thoroughly familiar with the role and functioning of CASS, systems analysis, and how to evaluate the RCA and proposed corrective actions submitted for their review.
(2) You should consider requiring participants in such committees or boards to receive training or orientation on these issues to ensure that they can provide critical evaluation. The membership of such boards and committees as well as the basic operating procedures and records should be described in your CASS document.
6-2. CASS PERSONNEL TRAINING AND EXPERIENCE.
a. Maintenance. You should determine the precise mix of training and experience your CASS auditors and analysts need. In general, auditors and analysts should:
(1) Have sufficient maintenance background applicable to your program to ensure that they are familiar with maintenance procedures, technical documents, and aircraft systems.
Page 38 Par 6-2
(2) Be able to understand and interpret the data they see, as well as evaluate facilities, equipment, and processes they observe. While they are unlikely to have specialized knowledge in all of the areas over which they conduct surveillance, a foundation of technical expertise is important.
b. Surveillance and Analysis. Your auditors and analysts should have training and experience in the functions they are responsible for looking at and analyzing. However, it is also essential that they have training and experience in the following areas:
• Systems analysis,
• Auditing techniques,
• Risk assessment and risk management (RM),
• RCA, and
• Human factors.
c. Quality Processes and Systems. Additionally, you may seek specialized training in specific quality processes or systems for your CASS personnel, such as:
(1) The International Organization for Standardization (ISO) developed ISO 9000, which is a quality system set of standards that seeks to standardize processes into organized and documented systems.
(2) Six Sigma, which is process-oriented, but from an intensively data-oriented, statistical approach.
d. Technical. Persons who collect and analyze operational data may require specialized technical backgrounds. This will depend on the level of complexity of the operational data that you collect. These personnel may work in the unit conducting an FAA-approved reliability program or in an independent data collection and analysis system. You may also wish to consider auditor certification. There are a number of auditor certifications available such as those under ISO 9001 and the RABQSA International Certification AS9110, Aerospace Auditor.
e. Summary of Experience and Training for Your CASS Personnel. Your CASS document should reflect that you have considered the type of experience and training, initial and recurrent, appropriate to your auditors and analysts in your operation. Areas to consider include those listed in the following table:
Par 6-2 Page 39 (and 40) Table 6-1. Subject Training Subject Area Auditors Audit Analysts Operational Data Analysts Title 14 of the Code of Federal Regulations part 121/135 (as applicable);
operations specifications.
Initial and recurrent. Initial and recurrent. Initial and recurrent.
Systems analysis training.
General training in quality standards.
General training in quality standards, statistics, and/or training in widely used industry courses such as ISO 9000.
General training in quality standards, statistics, and/or training in widely used industry courses such as ISO 9000.
Audit training. Init ial and recurrent.
Coordinating Agencies for Supplier’s Evaluation (C.A.S.E.) training if applicable.
Experience or training in conducting and reporting results of audits.
Statistical investigation experience or training.
Risk assessment training.
Initial and recurrent. Initial and recurrent. Initial and recurrent.
Root cause analysis training, including human factors.
Initial and recurrent. Initial and recurrent. Initial and recurrent.
Technical competence.
FAA mechanic certificate, engineering, or other maintenance background.
FAA mechanic certificate, engineering, or other maintenance background.
FAA mechanic certificate, engineering, or other maintenance background.
Educational background.
Related education or training may partially fulfill similar qualification requirements set by you.
Related education or training may partially fulfill similar qualification requirements set by you.
Related education or training may partially fulfill similar qualification requirements set by you.
Par 7-1 Page 41 CHAPTER 7. COMMUNICATIONS BETWEEN YOUR CASS PERSONNEL AND OTHER
DEPARTMENTS
7-1. COMMUNICATING SPECIFIC CASS RESULTS AND ACTIONS. The procedures that you would use for communicating CASS information and results to interested parties, internally and, as applicable, externally (for example, vendors, or the FAA) varies depending on factors such as the size and nature of your operation, the level of automation, and your CASS procedures themselves. The number and complexity of the standardized communications processes, such as forms or electronic mail messages with standard distribution, should be appropriate to the overall size and scope of your operation and CASS.
a. Standard Communication Processes. You should develop appropriate standard communication processes for all aspects of your CASS to assist in standardizing procedures, including the following:
(1) Audit checklists and results.
(2) Analysis procedures and results.
(3) Records of audit/analysis findings: internal.
(4) Records of audit/analysis findings: external.
(5) Corrective action forms and corrective action plans. These forms should address system considerations to ensure that there is a clear understanding of when the person implementing the corrective action will do so, who is responsible, and what the impact will be on written procedures.
(6) Information for monitoring and followup of corrective action. The processes should also assist in tracking the implementation of corrective actions once underway.
(7) Periodic status reports to senior management and to the FAA.
b. CASS Description. Your CASS description should address issues such as:
(1) Who is responsible for keeping these standard communication processes up-to-date and available.
(2) Who is responsible for completing the standard communication processes.
(3) Where it sends communications, who must respond, and how it tracks responses.
(4) How, where, and for how long it retains completed records.
7-2. EDUCATING PERSONNEL ON CASS. Your CASS should include procedures and responsibility to create some form of communication between the area responsible for the CASS, other areas of the company, and the FAA. You may accomplish this through training, newsletters, bulletins, meetings, or another format that you determine is suitable. One purpose of
Page 42 Par 7-2 such communication is to educate your mechanics and other departments that feed information and data into the CASS about why these data are necessary, what you do with the data, and how this process benefits the operation.
7-3. COMMUNICATIONS WITH PERSONNEL OUTSIDE THE CASS. The FAA expects a good communication system to meet the objectives in this section. You must determine which system is best for your particular operation.
a. Outside Resources. Your CASS should provide for regular, structured communications within the CASS structure and between your CASS and any other resources involved in decision-making for you. Examples of these other resources include:
(1) Avionics and other shops,
(2) Cabin safety organization,
(3) Engineering department and FAA-approved reliability program department,
(4) FAA certificate management office (CMO) or PI,
(5) Flight operations,
(6) Ground operations,
(7) Inspection department,
(8) Internal Evaluation Program (IEP),
(9) Maintenance control,
(10) Maintenance operations,
(11) Manufacturers’ technical representatives,
(12) Purchasing,
(13) QA,
(14) Receiving inspection,
(15) Maintenance recordkeeping department,
(16) Safety program,
(17) Senior management,
(18) Stores department, and
(19) Training departments.
Par 7-3 Page 43 (and 44)
b. Feedback Loop. The communications mechanisms should include a feedback loop designed to ensure that any changes implemented as a result of corrective actions are functioning as intended and improving the process. You may accomplish this level of communications through a variety of means, including the following:
(1) Periodic (weekly, monthly, quarterly) statistical and narrative CASS reports on trends, findings, and the status of corrective actions.
(2) Periodic CASS meetings to discuss trends or specific problem areas. If you are a smaller certificate holder, where your relevant managers work in close proximity, such meetings might be informal but frequent, or if you are a larger certificate holder using more formal and structured meetings, you may designate specific boards or committees.
(3) CASS board or committee meetings, including senior management, possibly on a monthly or bimonthly basis. Even if meetings are somewhat informal, you should keep minutes.
c. Monthly Report. Typically, if you use a program incorporating statistical performance standards (alert-type programs), you develop a periodic (monthly) report with appropriate data displays, summarizing the previous month’s activity. To help evaluate the effectiveness of the total maintenance program, the report should cover all aircraft systems controlled by the FAA-approved reliability program. A certificate holder without an FAA-approved reliability program may find that using a similar report can enhance its CASS.