followed. For example, a program element that has failed and is not working, or a program element that has faults and is not working as it should, are deficiencies.
(7) Effective. Producing or capable of producing a result. The maintenance program is producing the desired results when it realizes the following objectives:
• Airworthy aircraft have been properly maintained for operations in air transportation;
• Personnel are competent;
• Facilities and equipment are adequate; and
• All maintenance, preventive maintenance, and alterations are always performed in accordance with your maintenance program and manual.
(8) Establish and Maintain. To establish means that you develop a CASS that is appropriate for the type and scope of your operation. To maintain means that you keep your CASS current and appropriate in response to changes in the type and scope of your operation.
(9) Maintenance Program. The programs outlined in §§ 121.367 and 135.425 are outlined in other sections of part 121 subpart L and part 135 subpart J, and the current edition of AC 120-16 describes them in some detail.
(10) Maintenance. Inspection, overhaul, repair, preservation, and the replacement of parts, excluding preventive maintenance.
(11) Performance. The act of doing something successfully; the successful execution of an action. In CASS, performance means your maintenance program is being accomplished or executed as outlined in your air carrier manual.
(12) Person. An individual, firm, partnership, corporation, company, association, joint-stock association, or governmental entity. It includes a trustee, receiver, assignee, or similar representative of any of them.
(13) Preventive Action. Action to eliminate or mitigate the cause or to reduce the effects of potential nonconformity or other undesirable situation.
(14) Program. An organized list of procedures.
(15) Responsibility. The obligation to ensure that a task or function is successfully carried out. Responsibility includes accounting for actions related to the task or function. This is a key attribute of operational control.
(16) Risk. Risk is the degree of probability that hurt, injury, or loss will occur over a specific period of time or number of operational cycles. Risk has two elements: severity and likelihood. With regard to air carrier maintenance operations standards, the relationship between these two elements must be inverse.
Par 2-3 Page 5
(a) Severity is the type of harm inflicted if a particular event occurs. For air carrier maintenance programs, severity should be expressed in qualitative terms as a consequence of failure, such as catastrophic, hazardous, etc.
(b) Likelihood is the estimated probability or frequency, in quantitative or qualitative terms, of an occurrence related to the hazard; it is an expression of the probability that a specific unsafe event will occur. Likelihood is always an estimate.
(17) Risk Mitigation. A risk control measure. It refers to the process of modifying the system in order to reduce the risk.
(18) Risk Management. A formal process composed of identifying hazards, analyzing risk, assessing risk, and controlling risk. This process, embedded within the processes used to provide the product/service, is not a separate process.
(19) Root Cause Analysis (RCA). The analysis of deficiencies to determine their underlying root cause.
(20) System. A functionally related group of elements. In CASS, the elements are:
• Surveillance,
• Analysis,
• Corrective action, and
• Followup.
2-4. PURPOSE OF A CASS.
a. High Level Purpose. If you do not accomplish your maintenance program according to your manuals and applicable requirements, or if your maintenance program itself has deficiencies, your aircraft might be approved for return to service (RTS) when it is not airworthy.
The high-level purpose of a CASS is to reduce or eliminate the likelihood of your aircraft being approved for RTS when it is not airworthy through the continuous, system safety-based, closed-loop cycle of surveillance, investigation, data collection, analysis, corrective action, monitoring, and feedback of a CASS.
b. CASS Design. The FAA expects you to design your CASS to ensure that you conduct your maintenance program according to regulations and your manuals, and that your program is effective in achieving the desired result of consistently having airworthy aircraft approved for RTS. For the CASS to yield this safety benefit, the FAA expects your senior management to establish safety as its top organizational priority. All personnel need to embrace your organizational goals and act jointly to achieve them.
2-5. STRUCTURE OF A CASS.
a. Overall Objective. The overall objective of the air carrier regulations governing maintenance programs is to ensure that you maintain at least the level of safety originally designed into an aircraft and that each aircraft used in air transportation is always in an airworthy condition.
Page 6 Par 2-3
b. Questions a CASS Addresses. There are two basic questions the regulations require a CASS to address:
(1) Are you following your maintenance program as you have documented in your maintenance manuals and procedures? The CASS of the performance of a maintenance program refers to the process of collecting and evaluating information to determine that the execution of the maintenance program is done according to the regulations, your manuals, and other applicable requirements. This portion of the CASS consists of conducting audits and analyzing audit findings and trends to verify that you are following your maintenance program as you have written it and are properly performing maintenance on your aircraft. The analysis conducted in this area of your CASS should also identify weaknesses, if any, in the systems and procedures used to carry out your maintenance program.
(2) In following your manuals and procedures, are you producing the results that you want? The CASS of the effectiveness of the maintenance program refers to the process of collecting and evaluating operational data that you need to verify that your maintenance program is producing the desired results.
(a) Under the regulations, your primary desired results should be those defined results found in §§ 121.367 and 135.425. There are three:
1. You, or any of your maintenance providers, perform all maintenance, preventive maintenance, and alterations in accordance with your manual.
2. You and your maintenance providers provide competent personnel and adequate facilities and equipment for the proper performance of maintenance, preventive maintenance, and alterations.
3. Each of your aircraft released to service is airworthy and properly maintained for operations in air transportation.
(b) Another desired result is that your aircraft operate with a level of reliability that is consistent with the goals of your maintenance program. “Reliability” is a broad term in this context and is an expression of dependability and the probability that an item, including an aircraft, engine, propeller, or component, will perform its required function under specified conditions without failure, for a specified period of time. Testing for effectiveness usually consists of collecting and analyzing operational performance data, such as:
• Maintenance-related delays and cancellations,
• Failure rates of parts and components after they are approved for RTS,
• Discrepancy rates of aircraft after heavy maintenance, and
• Related trend analysis.
Par 2-5 Page 7
c. Correcting Deficiencies. The regulations require you to include, as part of your CASS, provisions to correct any deficiencies in your maintenance program, regardless of whether your maintenance program or any part of it is actually conducted by you or by your maintenance provider. The regulations also provide authority for the FAA to require you to make necessary changes in the program if you do not make them on your own.
d. Systems Safety. A well-structured CASS can assist you in taking a systems safety approach to your maintenance program through recognition of the interaction of all the elements within your program systems and subsystems. The systems consist of interrelated processes that comprise of personnel, procedures, materials, tools, equipment, facilities, and software operating in a specific environment to perform a specific task or to achieve a specific purpose, support, or mission requirement for an air carrier.
e. Organizational Functions. An unairworthy aircraft can be the result of the actions of a wide variety of organizational functions, in addition to those associated with maintenance. These organizational functions include senior management decisions, flight operations, ground operations, and others. An effective CASS would consider the potential role of these organizational functions through effective surveillance and complete RCA. Chapter 5, paragraphs 5-2 through 5-4 covers these issues in greater detail.
f. System Safety Categories. When performing their surveillance and analysis functions, we encourage individuals responsible for a CASS to use the system safety categories of safety attributes, safety culture, communication, accountability, training programs, and potential problem areas when identifying hazards and managing risks. The FAA defines safety attributes as the following:
(1) Authority. You have a clearly identifiable, qualified, and knowledgeable individual with the authority to establish and modify your CASS or a process within your CASS. Authority means the power to design or change fundamental policy or procedures without having to seek a higher level approval. Authority is permission; it is a right coupled with an autonomous power to accomplish certain acts or order others to act. Often one individual grants another authority to act, such as an employer to an employee, a corporation to its officers, or a governmental empowerment to perform certain functions.
(2) Responsibility. You have a clearly identifiable, qualified, and knowledgeable individual who is accountable for the quality and accomplishment of your CASS or a process within your CASS. Responsibility means the obligation to ensure a task or function is successfully carried out. Responsibility includes accountability for the action to carry out a task or function. There is overall responsibility and direct responsibility.
(3) Procedures. You have documented methods for accomplishing a process. The procedure description should answer the basic questions of who, what, when, where, and why, as appropriate.
(4) Controls. You design checks and restraints into a process to ensure you get the results that you want. Controls may not be necessary for every process; however, if a process is Page 8 Par 2-5
critical to meeting your overall system goals, specific controls to ensure proper accomplishment of that process generally are indicative of a well-designed system.
(5) Process measurement. This is where you measure and assess your processes to identify and correct problems or potential problems. You should identify specific measures and the data that you require to be collected.
(6) Interfaces. You identify and manage the interactions between processes, and the interfaces between your CASS and other systems or programs, whether within or outside your maintenance organization.
g. Risk Management. System safety and, therefore, CASS functions are built around principles of what is commonly referred to as risk management. Risk has two elements: severity and probability. Risk management includes identifying hazards, evaluating how severe the hazards’ consequences would be, and how likely they are to occur (probability). After you do that, you can develop, implement, and evaluate measures to address the identified risks and program deficiencies throughout a system’s life cycle to achieve the level of risk that you deem acceptable. All operators currently perform these functions on some level, although the degree of formality and sophistication depends on the size and scope of the operation as well as the level of training your personnel have in risk management. In your CASS, the FAA expects a formal risk management process (system safety) with safety, regulatory, and program compliance as the top priorities. Your formal risk management process should be structured, but not necessarily complex or expensive.
h. Safety Enhancement versus Disciplinary Action. The intention of a CASS is to give you (and the FAA) a realistic picture of the frequency and nature of deficiencies occurring in your maintenance program, and the opportunity to correct them. If your personnel at any level perceive that their jobs are at risk by collaborating in this system, they are likely to withhold information or bias the analysis for self-protection. You should design your CASS to emphasize the end goal of enhancing safety by evaluating and improving your maintenance program. You should not carry out your analysis and surveillance so that others perceive it or its intention as a method of identifying individuals who have committed errors simply to take some sort of disciplinary action. Human error is inevitable, but the question for your CASS to answer is how to better design your maintenance program to preclude errors from encroaching on operational safety or resulting in noncompliance.
i. Complacency. A particular challenge for your CASS is to overcome complacency that may be caused by the high degree of redundancy and dependability in modern aircraft systems.
You need to place high priority on the CASS of your maintenance program because the potential consequences of deficiencies in your programs can be very serious.
j. Unique Program. Because of the wide range of size and aircraft types of air carriers, it would be unrealistic to set forth a single means of compliance for all to follow. Just as each of you have your own maintenance procedures manuals, each of you should have your very own unique CASS. You should design your CASS appropriate to the size and sophistication of your operation. This is consistent with the performance-based regulatory approach described in the second paragraph of the Foreword of this AC.
Par 2-5 Page 9 (and 10)
CHAPTER 3. USING THIS AC TO DESIGN A CASS
3-1. TYPES OF OPERATORS THIS AC HELPS.
a. Types of Operators. The CASS applies to many types of operators, ranging from small operators of one or two propeller-driven aircraft, to operators with several hundred turbojet aircraft. Your aircraft may include helicopters or airplanes, and you may provide scheduled or unscheduled service and operate under 14 CFR part 121, part 135, or both. These factors have a significant effect on the size and structure of your maintenance organization. Additionally, if you conduct operations under 14 CFR part 91, 125, 129, 133, 135 (part 135, § 135.411(a)(1) applicability), or 137, you may also find the information in this AC useful if you decide to implement a CASS. If, under your operation, the regulations do not require you to have a CASS, you do not have to obtain FAA approval or authorization to develop or use a CASS as long as the use of a CASS does not conflict with any of the regulations that you must comply with.
b. Personnel Assigned to Accomplish CASS-Related Duties. In addition, you should be aware that a primary difference in CASS design between air carriers involves the number of personnel assigned to accomplish CASS-related duties. If you are a smaller certificate holder, you may assign fewer personnel and may have to draw upon personnel that you normally have assigned to other functions to fulfill CASS functions on a collateral duty basis. You may even need to use outside resources such as contract personnel to supply special expertise or independent review if your organization lacks the necessary special skills or training, or if you have an insufficient number of personnel to fulfill your CASS functions. If you are a larger certificate holder, you may have a significant number of personnel assigned full-time to CASS functions.
3-2. APPROACH OF THIS AC.
a. Main Text. This AC primarily addresses the functions of a CASS. The main text of this AC (chapter 5) presents the basic functions that you should have in any CASS.
b. Appendices 1, 2, and 3. In appendices 1, 2, and 3, we present examples of how three different types of operators might accomplish these CASS functions and satisfy the requirements. The intention of these examples is to show the different ways to comply; they are not a menu from which you may select a CASS. Rather, the intention of the examples is to demonstrate how you might design a CASS suitable for your particular, unique operation. Each appendix contains descriptions of procedures based on likely available resources for a different type and size of certificate holder. For example, if you are a smaller, on-demand air carrier operating under part 135 with two SAAB 340B airplanes, you probably meet the requirements in essentially the same way as a small certificate holder operating one Boeing 737 under part 121.
You will need to develop your own procedures and use the terminology (for example, designating the personnel or organizations involved in different aspects of the CASS) that best fit your particular and unique operation. For that reason, any job titles in this AC are for illustration;
they are not requirements or even recommendations.
Par 3-1 Page 11
Page 12 Par 3-2
c. Appendices 4 and 5. Appendix 4 is a list of sample CASS manual/document contents.
Appendix 5 lists related publications and the publications that we used as reference material or source documents to produce this AC.
CHAPTER 4. CASS IN YOUR ORGANIZATION
4-1. CASS DOCUMENTATION. You should describe your CASS policy and procedures in writing. You can produce these documents in a paper or electronic form, or another comparable record. For example, you could put them in a detailed chapter or section within your general maintenance manual or you could put them in a separate CASS manual associated with your general maintenance manual. In all cases, your policy and procedures should be written and not simply be verbal understandings.
4-2. WRITTEN POLICY AND PROCEDURES. Your CASS policy and procedures should:
a. Recognize CASS as a Coordinated System. You should recognize and treat your CASS as a coordinated system rather than just audit and data collection activities dispersed within your maintenance program. You do not have to contain your CASS personnel within a single department or office. However, your policy and procedures should identify all functions related to your CASS. You should not assume that because an audit or data collection function exists somewhere within your organization, it automatically satisfies your CASS requirement with no further coordination necessary.
b. Identify Major Programs. It should identify any programs, such as an optional reliability program, that you use to satisfy a major portion of your CASS. Your CASS documentation may refer to the documentation for that other program rather than restate the documentation again in another manual. The relationship/interfaces between the CASS and that other program should be clear and address responsibility and feedback issues to ensure that the other program meets CASS objectives.
c. Address Communication and Coordination Needs. It should address the need for fluid communications and coordination among the individuals who you have assigned authority, responsibility, and duties related to your CASS.
d. Address Principles of System Safety. You should base it on principles of system safety.
e. Identify Positions of Authority and Responsibility. It should clearly identify the positions within your maintenance organization with authority and responsibility for the CASS.
Remember that authority means permission and responsibility means obligation. Consistent with the system safety attributes, you must identify a position that has autonomous decision-making authority for your overall CASS process. There also must be a position that has, or positions that have, direct responsibility for carrying out the various functions of your CASS. If you have a larger organization, you may have a number of positions that have direct responsibility for carrying out functions of your CASS. However, there should be only one position that has the autonomous decision-making authority to authorize change for any of the CASS procedures for the CASS activities or functions.
(1) Authority. For purposes of this AC, “authority,” with regard to a CASS, means the power to create or modify fundamental policies or procedures without higher level review or approval. The individual with authority for the CASS may design or change the CASS without Par 4-1 Page 13
Page 14 Par 4-2 having to seek approval from a higher level of management. The CASS procedures should include how to modify the CASS.
(2) Responsibility. For purposes of this AC, “responsibility,” with regard to a CASS, means an obligation, with the expected accountability, for ensuring that others successfully accomplish tasks and functions in accordance with applicable policies, procedures, and standards. The person with direct responsibility may directly accomplish this work, or you may delegate the work. The individual with overall responsibility for your CASS has the obligation to carry out the functions of the CASS, including overseeing and managing any personnel to whom you have delegated CASS functions and duties. If your organization is smaller where personnel share duties and only carry out CASS functions part-time, this oversight and management responsibility relates only to those part-time tasks.
(a) You should designate a single person or position within your organization with the overall authority for your CASS, and you should designate an individual, position, or management committee within your organization with overall responsibility for managing and implementing your CASS. If you are a small certificate holder, an individual may have both authority and responsibility for CASS activities. That individual might also have responsibility for other functions as well as the CASS. It would be common for the person with overall responsibility for CASS functions to delegate some or much of this work to others within your organization, depending on the size and staffing of the certificate holder. The FAA expects you to assign clear responsibility for your CASS activities so that you do not end up with a fragmented system with a high risk of confusion over who is responsible for a given task.
(b) You should ensure that personnel with CASS responsibilities and duties are as independent as possible from the day-to-day operations of your maintenance program. Ideally, personnel conducting audits would work in separate departments from the departments performing your actual maintenance activities. However, if you are a small certificate holder, this may not necessarily be feasible. If you are a small certificate holder, personnel performing your CASS functions, particularly audits, may consist of one or more of the following:
1. “Borrowed” personnel from other shops or departments. Your procedures should include ways to avoid having these individuals assigned to audit areas where they normally work.
2. The company owner or chief executive officer (CEO), particularly if there are no other employees and the CASS audits focus on outside vendors and maintenance providers because maintenance providers accomplish all or most of your actual maintenance work.
3. Outside resources that you make arrangements with to perform audits and analysis for you.
4. Others that you have determined are qualified to provide independent audit, operational data collection, and analysis services that fulfill the requirements of a CASS as described in this AC.
CHAPTER 5. MAJOR CASS ACTIVITIES
5-1. SUMMARY OF A CASS. The regulations require that your CASS accomplish surveillance and analysis of all elements of your maintenance program from two perspectives:
verifying performance and verifying effectiveness. There are four activities in the CASS process:
surveillance, analysis, corrective action, and followup. You can carry out the first two activities (surveillance and analysis) in two different ways. One audit-based, and the other is operational data and collection and analysis-based. The results of the two types of surveillance and analysis feed into the third and fourth major CASS activities: corrective action and followup. Figure 5-1 summarizes the flow of the four basic activities of CASS, which paragraphs 5-2 through 5-5 describe in further detail.
FIGURE 5-1. THE FOUR BASIC CASS ACTIVITIES:
SURVEILLANCE, ANALYSIS, CORRECTIVE ACTION, FOLLOWUP Verify Performance of Maintenance Program Verify Effectiveness of Maintenance Program 1a. Surveillance: Audit process.
- Create an audit plan based on risk assessment.
- Perform work-in-progress audits.
- Perform transaction audits.
- Perform systems audits.
- Identify hazards.
1b. Surveillance: Data collection process.
- Select data sets.
- Collect operational data.
- Collect failure data.
- Identify trends, anomalies, and potential hazards.
2a. Analysis: Identify hazards and accomplish risk analysis and assessment.
2b. Analysis: Identify hazards, investigate adverse indicators, and accomplish risk analysis and assessment.
3. Corrective Action: Accomplish Root Cause Analysis (RCA); develop, implement, and monitor a corrective action plan, as appropriate.
4. Followup (Performance Measurement): Verify the corrective action is effective, and initiate risk-based followup surveillance planning, as appropriate.
a. Risk Management in CASS.
(1) In concert with the attributes of a good organization, you should prioritize personnel and resources for CASS as part of the overall risk management process. Risk management facilitates the balancing act between assessed risks and practical risk mitigation.
(2) Risk management serves to focus safety efforts on those hazards posing the greatest risks. Essentially, any methodology used to prioritize surveillance personnel and resources (as well as to formulate corrective action decisions later in the process) involves principles of risk management.
(3) A formal risk management process is composed of the following elements:
identifying hazards, analyzing risk, assessing risk, and controlling risk. This process, embedded within the processes used to provide the product/service; it is not a separate process. The flow Par 5-1 Page 15
Page 16 Par 5-1 chart in Figure 5-2 summarizes an overview of the risk management process. The elements of a risk management process encompass the four major CASS activities (Figure 5-1).
FIGURE 5-2. RISK MANAGEMENT PROCESS
5-2. VERIFYING PERFORMANCE OF YOUR MAINTENANCE PROGRAM.
a. Surveillance of the Performance of Your Maintenance Program.
(1) Definition of a CASS “Audit.” Your main tool for assessing whether you are properly performing (executing) your maintenance program is audits, particularly work-in-progress audits. Work-in-progress audits are consistent with the plain language meaning of “performance” found in 14 CFR part 121, § 121.373 and 14 CFR part 135, § 135.431. The main direction of your audits should address your maintenance program objectives found at §§ 121.367 or 135.425, as appropriate. For purposes of CASS, a CASS audit is a formal examination of the activities of a department or area of your maintenance program based on an established standard such as the regulations or your manual. The intention of work-in-progress
audits is to ensure that your maintenance personnel, as well as your maintenance providers, comply with your manual, program, and all other requirements.
(2) Audit Procedures. You should have written procedures to guide your auditing process, including the scheduling of audits. The CASS addresses both internal audits and external audits. Internal audits are audits that you conduct within your organization. External audits are audits that you conduct of vendors who supply parts and services to you. Your CASS procedures should include a methodology for determining priorities and for establishing and adjusting audit cycles (for example, 12, 18, 24, 36-month cycles) so that you focus your resources on the most pressing issues. This is a risk assessment and risk management process.
(See paragraph 5-2(a)(3) for further explanation of risk assessment and risk management.)
(a) Although the majority of the generated inputs to this process are internal, one additional input may be the result of outside audits conducted by entities other than yourself or your maintenance providers. For example, the results of audits or inspections conducted by the FAA or the Department of Defense (DOD) may be useful by providing you with:
• Specific findings requiring RCA and possible corrective action (activities discussed later in this AC), and
• Information useful in focusing your own audits and operational data collection.
(b) You may approach this initial scheduling task in many different ways, ranging from resource allocation based on company experience and very basic analysis to the use of a sophisticated, software-supported risk analysis process. Within this range of possible methodologies, your CASS procedures should contain a process to systematically make those decisions that are compatible with the size and complexity of your operations. The FAA encourages you to make this process as structured as possible. You should place priority first on safety and regulatory compliance, and second on issues of operational efficiency. However, if your CASS is effective, you will meet all three of these objectives.
(3) Prioritizing Surveillance Resources. Essentially, any methodology that you select to prioritize your surveillance resources (as well as to formulate corrective action decisions later in the process) involves principles of risk assessment. Risk assessment is a concept applicable in many aspects of an aviation operation. (See FAA Order 8040.4, Safety Risk Management, for additional information.) Order 8040.4 is an example and is not the only source of risk assessment procedures. However, it provides insight into FAA expectations. The FAA encourages operators to incorporate the principles of this systematic process to:
• Establish a plan, including the scope of the process and priorities (for example, detect and prevent noncompliance);
• Specify the areas of concern for surveillance and analysis (personnel, the maintenance program and organization, operations, aircraft, facilities, and systems);
• Identify hazards or potential threats to the operation;
• Determine how likely the consequences of a hazard will occur and actually cause harm;
Par 5-2 Page 17