Par 5-4 Page 35
f. Corrective Action Plan.
(1) With the RCA complete, corrective action options identified, and risk assessment performed as appropriate, you can make a final decision on the proposed corrective action plan.
The corrective action plan should address all relevant issues, including a timetable for completion of the action, with milestones if appropriate. The appropriate technical department and, if the corrective action goes beyond your maintenance organization, other departments (such as flight operations) should then implement the plan.
(2) Your CASS procedures should identify:
• How this plan will be approved and at what level of your company.
• The parties responsible for implementing, monitoring, and ensuring that all affected parties are notified, both within your maintenance organization and externally, if necessary.
5-5. FOLLOWUP.
a. Corrective Action Followup. Your CASS procedures should:
(1) Specify how you will evaluate your corrective actions for effectiveness. That is, did your corrective action do what you wanted it to do? This may require the following:
• Followup audits of a specific area,
• Regular communication from the affected technical area about the effects of the corrective action and,
• Other forms of verification by your auditors or analysts tracking the implementation.
(2) Identify the person or entity (such as a CASS board) responsible for determining if any changes in the status of a corrective action are acceptable. Your CASS auditors or analysts have the duty of ensuring that the person implementing the corrective action did so in accordance with the established timetable, or, if not, determining why the timetable has changed.
(3) Include responsibilities and guidelines for:
• Tracking the implementation of corrective actions in accordance with the timeline.
• The role of your auditors, managers, management committees, and senior management.
• How your CASS will use automation or computerized systems.
Page 36 Par 5-5
• How your CASS will use risk assessment and systems analysis to guard against unintended consequences.
• Measures to evaluate the effect of the corrective action.
• The affected technical area to communicate the status of the corrective action to the person responsible for monitoring implementation.
b. Getting Help from a Manufacturer. In some cases, you may require data or assistance from a manufacturer in correcting a deficiency detected by your CASS. Manufacturers may not always assign these issues the same priority as you do. You should provide guidance in your CASS procedures, based on your own particular experience, about how your CASS should address assistance or information from manufacturers, and how to proceed in case of unsatisfactory or slow responses. This may include developing a standardized letter citing the need for this information or assistance to satisfy the requirements of § 121.373, § 135.431, or other pertinent regulations. It may also include working with your FAA principal inspector (PI) to find solutions.
c. Followup Surveillance Plan.
(1) Your CASS procedures should include how to determine the level of followup audits for verifying corrective action implementation. For example, based on the risk assessment or the complexity of your corrective action, the designated CASS analyst or team may schedule special or more frequent audits. They may also change the data collection process or institute other means of verification. The FAA expects you to have a well-defined and logical process to design the followup actions.
(2) The information and analysis performed through the closed-loop, continuous cycle of surveillance, investigations, analysis, and corrective action, permits you to refine your audit and data collection priorities through the risk assessment process.
Par 6-1 Page 37 CHAPTER 6. PERSONNEL WHO PERFORM CASS FUNCTIONS
6-1. PERSONNEL MANAGING CASS FUNCTIONS.
a. Decision-making Group.
(1) Your CASS should include a decision-making group at a relatively high organizational level to oversee or carry out your CASS functions. This oversight group could include:
• Technical boards concerned with performance and other technical issues,
• Administrative boards that may have broader decision-making authority to act on technical recommendations, or
• A single board combining both functions.
(2) The key concept is that there is a decision-making group at a relatively high level to monitor your CASS and to make critical decisions in a timely manner. Typically, if you are a smaller certificate holder, this committee or board might be composed of the president of the company, and the DOM and flight operations. If you are a larger certificate holder, participants might be managers from several departments, such as maintenance and engineering, quality assurance (QA), and flight operations.
b. Group Members.
(1) If you use committees or boards as major decision-making groups for CASS issues, members of these groups should:
• Have an appropriate technical background, and
• Be thoroughly familiar with the role and functioning of CASS, systems analysis, and how to evaluate the RCA and proposed corrective actions submitted for their review.
(2) You should consider requiring participants in such committees or boards to receive training or orientation on these issues to ensure that they can provide critical evaluation. The membership of such boards and committees as well as the basic operating procedures and records should be described in your CASS document.
6-2. CASS PERSONNEL TRAINING AND EXPERIENCE.
a. Maintenance. You should determine the precise mix of training and experience your CASS auditors and analysts need. In general, auditors and analysts should:
(1) Have sufficient maintenance background applicable to your program to ensure that they are familiar with maintenance procedures, technical documents, and aircraft systems.
Page 38 Par 6-2
(2) Be able to understand and interpret the data they see, as well as evaluate facilities, equipment, and processes they observe. While they are unlikely to have specialized knowledge in all of the areas over which they conduct surveillance, a foundation of technical expertise is important.
b. Surveillance and Analysis. Your auditors and analysts should have training and experience in the functions they are responsible for looking at and analyzing. However, it is also essential that they have training and experience in the following areas:
• Systems analysis,
• Auditing techniques,
• Risk assessment and risk management (RM),
• RCA, and
• Human factors.
c. Quality Processes and Systems. Additionally, you may seek specialized training in specific quality processes or systems for your CASS personnel, such as:
(1) The International Organization for Standardization (ISO) developed ISO 9000, which is a quality system set of standards that seeks to standardize processes into organized and documented systems.
(2) Six Sigma, which is process-oriented, but from an intensively data-oriented, statistical approach.
d. Technical. Persons who collect and analyze operational data may require specialized technical backgrounds. This will depend on the level of complexity of the operational data that you collect. These personnel may work in the unit conducting an FAA-approved reliability program or in an independent data collection and analysis system. You may also wish to consider auditor certification. There are a number of auditor certifications available such as those under ISO 9001 and the RABQSA International Certification AS9110, Aerospace Auditor.
e. Summary of Experience and Training for Your CASS Personnel. Your CASS document should reflect that you have considered the type of experience and training, initial and recurrent, appropriate to your auditors and analysts in your operation. Areas to consider include those listed in the following table:
Par 6-2 Page 39 (and 40) Table 6-1. Subject Training Subject Area Auditors Audit Analysts Operational Data Analysts Title 14 of the Code of Federal Regulations part 121/135 (as applicable);
operations specifications.
Initial and recurrent. Initial and recurrent. Initial and recurrent.
Systems analysis training.
General training in quality standards.
General training in quality standards, statistics, and/or training in widely used industry courses such as ISO 9000.
General training in quality standards, statistics, and/or training in widely used industry courses such as ISO 9000.
Audit training. Init ial and recurrent.
Coordinating Agencies for Supplier’s Evaluation (C.A.S.E.) training if applicable.
Experience or training in conducting and reporting results of audits.
Statistical investigation experience or training.
Risk assessment training.
Initial and recurrent. Initial and recurrent. Initial and recurrent.
Root cause analysis training, including human factors.
Initial and recurrent. Initial and recurrent. Initial and recurrent.
Technical competence.
FAA mechanic certificate, engineering, or other maintenance background.
FAA mechanic certificate, engineering, or other maintenance background.
FAA mechanic certificate, engineering, or other maintenance background.
Educational background.
Related education or training may partially fulfill similar qualification requirements set by you.
Related education or training may partially fulfill similar qualification requirements set by you.
Related education or training may partially fulfill similar qualification requirements set by you.
Par 7-1 Page 41 CHAPTER 7. COMMUNICATIONS BETWEEN YOUR CASS PERSONNEL AND OTHER
DEPARTMENTS
7-1. COMMUNICATING SPECIFIC CASS RESULTS AND ACTIONS. The procedures that you would use for communicating CASS information and results to interested parties, internally and, as applicable, externally (for example, vendors, or the FAA) varies depending on factors such as the size and nature of your operation, the level of automation, and your CASS procedures themselves. The number and complexity of the standardized communications processes, such as forms or electronic mail messages with standard distribution, should be appropriate to the overall size and scope of your operation and CASS.
a. Standard Communication Processes. You should develop appropriate standard communication processes for all aspects of your CASS to assist in standardizing procedures, including the following:
(1) Audit checklists and results.
(2) Analysis procedures and results.
(3) Records of audit/analysis findings: internal.
(4) Records of audit/analysis findings: external.
(5) Corrective action forms and corrective action plans. These forms should address system considerations to ensure that there is a clear understanding of when the person implementing the corrective action will do so, who is responsible, and what the impact will be on written procedures.
(6) Information for monitoring and followup of corrective action. The processes should also assist in tracking the implementation of corrective actions once underway.
(7) Periodic status reports to senior management and to the FAA.
b. CASS Description. Your CASS description should address issues such as:
(1) Who is responsible for keeping these standard communication processes up-to-date and available.
(2) Who is responsible for completing the standard communication processes.
(3) Where it sends communications, who must respond, and how it tracks responses.
(4) How, where, and for how long it retains completed records.
7-2. EDUCATING PERSONNEL ON CASS. Your CASS should include procedures and responsibility to create some form of communication between the area responsible for the CASS, other areas of the company, and the FAA. You may accomplish this through training, newsletters, bulletins, meetings, or another format that you determine is suitable. One purpose of
Page 42 Par 7-2 such communication is to educate your mechanics and other departments that feed information and data into the CASS about why these data are necessary, what you do with the data, and how this process benefits the operation.
7-3. COMMUNICATIONS WITH PERSONNEL OUTSIDE THE CASS. The FAA expects a good communication system to meet the objectives in this section. You must determine which system is best for your particular operation.
a. Outside Resources. Your CASS should provide for regular, structured communications within the CASS structure and between your CASS and any other resources involved in decision-making for you. Examples of these other resources include:
(1) Avionics and other shops,
(2) Cabin safety organization,
(3) Engineering department and FAA-approved reliability program department,
(4) FAA certificate management office (CMO) or PI,
(5) Flight operations,
(6) Ground operations,
(7) Inspection department,
(8) Internal Evaluation Program (IEP),
(9) Maintenance control,
(10) Maintenance operations,
(11) Manufacturers’ technical representatives,
(12) Purchasing,
(13) QA,
(14) Receiving inspection,
(15) Maintenance recordkeeping department,
(16) Safety program,
(17) Senior management,
(18) Stores department, and
(19) Training departments.
Par 7-3 Page 43 (and 44)
b. Feedback Loop. The communications mechanisms should include a feedback loop designed to ensure that any changes implemented as a result of corrective actions are functioning as intended and improving the process. You may accomplish this level of communications through a variety of means, including the following:
(1) Periodic (weekly, monthly, quarterly) statistical and narrative CASS reports on trends, findings, and the status of corrective actions.
(2) Periodic CASS meetings to discuss trends or specific problem areas. If you are a smaller certificate holder, where your relevant managers work in close proximity, such meetings might be informal but frequent, or if you are a larger certificate holder using more formal and structured meetings, you may designate specific boards or committees.
(3) CASS board or committee meetings, including senior management, possibly on a monthly or bimonthly basis. Even if meetings are somewhat informal, you should keep minutes.
c. Monthly Report. Typically, if you use a program incorporating statistical performance standards (alert-type programs), you develop a periodic (monthly) report with appropriate data displays, summarizing the previous month’s activity. To help evaluate the effectiveness of the total maintenance program, the report should cover all aircraft systems controlled by the FAA-approved reliability program. A certificate holder without an FAA-approved reliability program may find that using a similar report can enhance its CASS.
Par 8-1 Page 45 CHAPTER 8. HOW YOUR CASS DIFFERS FROM AND RELATES TO OTHER PROGRAMS
8-1. SUMMARY OF OTHER PROGRAMS. Your description of your CASS should identify other related programs where you participate and explain how your CASS relates to those programs or differs from them. Experience has shown that certain other programs are potential sources of information for your CASS. Some programs have been mistakenly assumed to be so similar to a CASS that you might neglect an important aspect of your CASS. Therefore, your CASS documentation should describe the relationship between your CASS and programs, such as the:
• FAA-approved reliability program,
• IEP,
• Safety program,
• Voluntary Disclosure Reporting Program (VDRP),
• Coordinating Agencies for Supplier’s Evaluation (C.A.S.E.),
• Aviation Safety Action Program (ASAP), and
• Aviation Safety Reporting Program.
8-2. DISCUSSION OF INDIVIDUAL PROGRAMS.
a. FAA-Approved Reliability Program. According to the current edition of AC 120-17, Maintenance Control by Reliability Methods, the FAA developed the concept of reliability control to maintain an acceptable level of reliability. The concept evolved based on FAA and airline efforts to develop more responsive methods of controlling scheduled maintenance without sacrificing safety or FAA regulatory responsibility. An FAA-approved reliability program includes systems for data collection and analysis, corrective action, statistical performance standards, data display and reporting, scheduled maintenance adjustments, and process changes.
AC 120-17 defines an acceptable level of reliability as maintaining failure rates below a predetermined value. However, AC 120-17 does not address modern-day, task-based scheduled maintenance programs.
(1) Typically, if you are a larger certificate holder, you will have an FAA-approved reliability program, but the operational statistical data collection and analysis requirements of a program that conforms to AC 120-17 usually exceed the resources or requirements of smaller and even most medium-sized operators, and generally are greater than what would be necessary for those operators’ CASS. However, if you do have an approved reliability program, you should incorporate it into your CASS as the means of performing operational data collection and analysis to monitor the effectiveness of your maintenance program. Your CASS procedures should describe how you will integrate the approved reliability program into your CASS. You should be aware that an FAA-approved reliability program cannot substitute for a CASS because the reliability program addresses only the scheduled maintenance element and does not include
Page 46 Par 8-2 the broader auditing surveillance and analysis of the full range of all 10 elements of your maintenance program, nor the complete processes for developing and implementing corrective actions. An FAA-approved reliability program should be a part of your CASS; however, it cannot replace or substitute for a CASS.
(2) This AC is not intended to describe FAA-approved reliability programs. However, CASS operational data collection needs are typically similar to, but much more comprehensive than, those of an approved reliability program. You should, within your CASS, establish a program similar to an FAA-approved reliability program for the purpose of collecting and analyzing operational data. However, you must ensure its operational data collection program meets the needs of your CASS.
(3) Within the aviation community, it is common to refer to “reliability,” in a generic sense, as meaning dispatch availability of equipment or in relation to equipment failure rates. If your CASS manual or document uses this terminology, you should distinguish whether the reference is to an FAA-approved reliability program or generic reliability.
b. IEP.
(1) An IEP is a voluntary program that you can use to provide measurement of your internal processes and procedures to assess whether they are adequate and functioning properly.
An IEP should be independent of all of your other programs and systems, and could be a useful tool to evaluate your CASS, as well as other systems or programs, such as your safety program.
An IEP is a very high-level review to provide information to your senior management as to how well critical programs, such as your CASS, are working. It is not a substitute for a CASS. An IEP is a broader, system evaluation program and is less “audit-oriented” than your CASS, although both use a system evaluation approach. An IEP would not focus on evaluating the maintenance program, because such a focus would duplicate what the CASS is supposed to do. If management believes they need to have additional evaluation of your maintenance program or certain elements of your maintenance program, they should direct the CASS to accomplish that work. An IEP should produce information to determine if the CASS is being performed and is effective.
(2) The current edition of AC 120-59, Air Carrier Internal Evaluation Programs, describes the IEP. You should not misunderstand the IEP as a program that replaces existing regulatory auditing requirements such as a CASS. Audits are a very minor part of an effective IEP.
c. Safety Program.
(1) According to 14 CFR part 119, § 119.65, if you are conducting operations under 14 CFR part 121, you must have a full-time Director of Safety (DOS) or equivalent position unless you ask for, and the FAA authorizes, a deviation from the management position requirements. Consistent with the objectives of part 119, the DOS is responsible for keeping your highest management officials fully informed about the safety status of your entire operation.
Your DOS should oversee a safety program that addresses the wide range of risks involved in operations in air transportation, including flight operations, maintenance operations, and ground
Par 8-2 Page 47 operations. The safety program should be a comprehensive program with a variety of elements, such as investigations of and a reporting system for accidents and incidents, safety audits and inspections, operational risk assessment, and trend analysis.
(2) If you conduct operations under 14 CFR part 135, the regulations do not require you to have a DOS position. Nonetheless, the FAA encourages you to designate a company management official or manager to monitor and evaluate your flight, maintenance, and ground safety practices, procedures, and programs and to keep your highest management official fully informed about the safety status of your entire operation.
d. VDRP. The current edition of AC 00-58, Voluntary Disclosure Reporting Program, provides guidance for you on procedures to use when voluntarily disclosing to the FAA violations of Federal aviation regulations. Your participation in the program may reveal important information regarding maintenance issues and lead to the development of comprehensive fixes relevant to the maintenance program that your CASS oversees.
(1) Under this program, you may voluntarily report violations of regulations that you discover and avoid certain enforcement consequences.
(2) While the regulations do not require that your CASS addresses disclosures made under the VDRP, the FAA recommends that you consider whether to include information from voluntary disclosures in your CASS in any fashion. For example, your CASS personnel may be the same personnel as those who handle voluntary disclosures. They may therefore be able to use “de-identified” information from voluntary disclosures to point to areas where additional auditing may be necessary. Your CASS personnel should be aware of comprehensive fixes developed in conjunction with the VDRP. These are, after all, precisely the types of systems or procedural modifications that an effective CASS is seeking to avoid adverse audit findings or unwanted operational performance.
e. C.A.S.E.
(1) C.A.S.E. is a cooperative effort within the airline industry to audit suppliers and vendors and to analyze, control, and determine the acceptability of vendors supplying parts and maintenance services to participating airlines. According to FAA Order 8900.1, Flight Standards Information Management System (FSIMS), the FAA has determined that the use of C.A.S.E.
audits satisfies some of the requirements of §§ 121.373 and 135.431. Operations specifications (OpSpecs) must authorize the use of the C.A.S.E. program to satisfy some of these requirements.
(2) If you participate in C.A.S.E., your CASS procedures should address whether or how the CASS will use C.A.S.E. audits and the basis for that decision. If the functions being audited depend on the specifics of your program, a C.A.S.E. audit would probably not suffice.
The important point is that you have a written policy and procedures for when and how to use C.A.S.E. audit results in your CASS.
f. ASAPs. The current edition of AC 120-66, Aviation Safety Action Program (ASAP), describes this program for the voluntary reporting of safety issues and events by employees, such as crewmembers and mechanics. ASAPs involve the collection, analysis, and retention of safety data that would otherwise be unobtainable. Such data can be important input to your CASS.
Page 48 Par 8-2
g. Aviation Safety Reporting Program. The current edition of AC 00-46, Aviation Safety Reporting Program, describes the Aviation Safety Reporting Program, which uses the National Aeronautics and Space Administration (NASA) as a third party to receive aviation safety reports. The Aviation Safety Reporting Program invites crewmembers, maintenance personnel, and others to report to NASA actual or potential discrepancies and deficiencies involving aviation safety. NASA designed and administers the Aviation Safety Reporting System to facilitate the program. These reports may help your CASS personnel identify areas of potential concern within their own company based on industry-wide trends or experiences identified by NASA.