5-2
4. Confidence testing of expected and achieved results.
5. Studies of life-bands and survival characteristics.
6. Investigative testing/sampling programs.
7. Structural review/analysis. To monitor and control structural integrity, competent personnel should review structural service bulletins and industry reports for applicability and urgency. Structural discrepancies should be reviewed with an emphasis on major structural discrepancies.
8. Weibull analysis. Weibull analysis determines the failure distribution profile and the predictability of failure, and is used to determine whether a component or system shows signs of deteriorating performance with age. This statistical method helps in determining if a component may benefit from a restoration task or regular preventative maintenance.
Note: When appropriately applied, Weibull analysis methods are not subject to yield limitations as described below.
9. Pareto analysis. Pareto analysis is a simple rank ordering of the number or rate of failures or defects for a given data source. This graphical representation may be effectively used for aircraft out-of-service events, delay and cancellation events, MEL applications/extensions, and other metrics as deemed appropriate. Pareto charts are used to identify the top subjects, which are often targeted for performance improvements.
10. Failure Mode and Effects Analysis (FMEA).
11. MSG analysis methods.
12. Other root cause analysis tools.
5.2 Analysis of Task Effectiveness. You should routinely perform an analysis to determine the applicability and effectiveness of the tasks contained within the maintenance schedule. This is in addition to the analysis of the task and related operational data performed during optimization. The evolution of your maintenance schedule is not necessarily to simply increase individual task intervals, but to substantiate the accomplishment of an effective task at the appropriate interval, without compromising safety or negatively impacting operational performance beyond that which is acceptable.
5.2.1 Data Selection Criteria. An operator should include in its reliability program a defensible method for determining the relevant type and amount of data that adequately represents its fleet with respect to the maintenance task under consideration. The following describes a defensible method for data selection:
• Consistent,
• Measurable,
• Unbiased,
• Factual,
5-3
• Accurate, and
• Repeatable.
5.2.2 Data Sampling. In compliance with your CASS obligation under 14 CFR §§ 91.1431,
121.373, and 135.431 to determine effectiveness of your maintenance schedule, 100 percent of your performance standard data is collected and analyzed. Because of this CASS requirement, the concept of sampling may be applied as an effective and efficient tool when analyzing other data types or sets in support of your program’s requirements.
Data sampling allows you to analyze a defined portion of data from a relevant data set, rather than analyzing the entire amount of data that may be available. As with any effective data analysis process, the FAA expects you to collect and analyze additional data types or different data sets as the particular circumstances dictate.
5.2.2.1 When executing these CASS requirements in the context of data sampling, you will encounter variables where differences between the following aspects of your program become significant. Examples include:
1. The type of task being evaluated.
2. The finding or type of finding considered unacceptable for a given task (see paragraph 1.6).
3. Related vs. unrelated data.
4. Significant vs. nonsignificant findings.
5. Data variables:
• Discrete data (i.e., can only take on certain values, can be counted);
• Continuous data (i.e., can take on infinite number of values, must be measured); and
• Data quality/error rate.
6. Qualitative vs. quantitative analysis.
7. Other differences in analysis methods and tools:
• Those designed to predict failures or future performance, and
• Those designed to monitor real-time performance.
5.2.2.2 Due to these variations and the wide range of analysis tools you may choose to employ, it is not possible in this AC to provide prescriptive details or criteria for you to apply when determining what data types/sets to collect and analyze, your data sample set, or which analysis methods you should use.
5.2.2.3 Regarding data sampling specifically, it is likewise not possible to provide a one-size-fits-all sampling method or formula that is effective for every data type and analysis method available. You should not attempt to define or rely on any one method or formula that establishes a sample size to be used in
5-4 every case. You must respond to the needs of the analysis being performed, which includes performing any additional analyses as discussed above.
Therefore, you should design and execute your processes and methods, including your sampling methods, to be agile enough to respond to the specific data and analysis needs of each situation. The keys in executing an effective analysis process are:
1. Recognizing the significance and the effect of variations in your operations and data;
2. Utilizing the appropriate data types/sets, amount of data, and analysis methods and tools for the given situation;
3. Following through with the actions necessary to ensure program goals and objectives are met; and
4. Using competent personnel to perform these functions to ensure a consistent outcome.
Note: Personnel competencies are discussed in paragraph 2.1.4; in AC 120-79, Developing and Implementing an Air Carrier Continuing Analysis and Surveillance System, Chapter 6, Personnel Who Perform CASS Functions; and in AC 120-16, Air Carrier Maintenance Programs, Chapter 10, Personnel Training.
5.2.2.4 Regardless of the method used to determine your sample data requirements, your sampling process must result in confidence that the sample data is applicable and representative of the population. Your data sampling process should consider data provided by outliers within your population, while preventing the possibility that a decision is inappropriately influenced by data from aircraft that are at the extremes of the data set. The following is provided for reference only, and is not intended to suggest any single method described herein as the recommended method or formula to determine sample size in every situation.
1. The process for selecting a data set that reflects the fleet composition should consider the following criteria:
• Operational history,
• Configuration differences,
• Time elapsed since major conversions or refurbishments,
• Utilization differences,
• High time/cycle and low time/cycle aircraft,
• Seasonality and environmental conditions,
• Sustained storage periods, and
• Task yield.
5-5 Note: Aircraft that don’t meet minimum task yield standards should not be included in the random sample. Findings from scheduled maintenance tasks actually performed at a lower yield may be an indicator that the published interval is too high. Further analysis will be necessary, including analysis of related in-service data.
2. Include one or more of the following means when defining a data selection method. Data should be selected from the available data pool generated using the criteria above, such as:
• A defined number of aircraft (typically used with smaller fleets).
• A percentage of fleet size (e.g., a minimum percentage of randomly selected aircraft), plus outlier aircraft from the criteria applicable under paragraph 5.2.2.4, item 1 (e.g., high utilization and low utilization, high flight-hours and low flight-hours, high cycles and low cycles, etc.).
• A statistical formula (primarily used for quantitative data analysis).
• Data collected over a defined period of time. Here, the sample is all data from the data set that is collected during the specified period, rather than all data that may be available or that has been collected (particularly useful when identifying performance trends).
3. Depending on the task being analyzed, your sampling method should allow flexibility to account for benefits in analyzing the task performed repetitively on specified aircraft vs. analyzing tasks performed once on a greater number of aircraft. As stated above, collecting additional data and/or different data sets may be necessary.
4. You should ensure fleet representation is maintained in the data throughout your analysis process (see Figure 5-1, Data Selection Example).
5-6 Figure 5-1. Data Selection Example
5.2.3 Task Yield.
5.2.3.1 Any scheduled maintenance task analysis that includes either defects generated or a lack of defects generated, should consider the respective interval yield. This is because the current task interval is established with the intent of detecting degradation or potential failure at that interval. If the task is performed significantly earlier than the specified interval, the data may not be valid to support an evaluation of the specified interval.
Note: Operators should include in their analysis process defects discovered at relatively low yield intervals since this may be an indication that the current task interval is not effective.
5.2.3.2 To arrive at a yield solution, the defined interval and the interval at which the task was actually accomplished must be calculated using the same parameter (flight-hours, flight cycles, or calendar). In some cases, you may need to perform a conversion of values to arrive at the same usage parameter.
5-7 The yield calculation divides the interval at which the task was actually accomplished by the defined interval.
• A minimum acceptable value for average yield of all task accomplishments contributing data to the task analysis is 90 percent.
• No single task accomplishment within the data set used for task analysis should have a yield less than 80 percent.
Note: Analysis of consecutive task accomplishments may be useful when evaluating task and interval effectiveness. Failures discovered at a relatively low yield should be considered as such in the overall analysis. However, tasks performed at a relatively low yield with no findings should not be used to determine or analyze failure rates at the published interval, or as zero findings data when substantiating escalations from that interval. The second or subsequent task, by itself resulting in a higher overall yield, may be useful for these purposes provided in-service failures are identified and incorporated into the data.
5.2.3.3 Example:
• Actual task interval: 7,100 flight-hours.
• Task defined interval: 7,500 flight-hours.
• Yield calculation: 7,100 flight-hours/7,500 flight-hours = 94.7 percent task yield.
5.2.4 Data Preparation and Related Considerations. Typically, operational data are collected and consistently coded so correlation to a maintenance schedule task is evident to analysts or engineers performing the analysis (see paragraph 3.3). To ensure comprehensiveness and relevance of data when performing analysis, the operator should ensure all data findings are directly related to the task being analyzed.
5.2.4.1 Related Data. Related data is a maintenance defect discovered during scheduled or unscheduled maintenance to which the corrective action is directly associated to the task being analyzed or its consequence of functional failure.
5.2.4.2 Unrelated Data. Unrelated data is a maintenance defect discovered during scheduled or unscheduled maintenance for which the corrective action is not directly associated to the task being analyzed or its consequence of functional failure. Depending on risk, it may be necessary to investigate unrelated data findings separately from the evaluation of the given task and interval.
Note: Unrelated findings that would otherwise be considered significant as defined below should be thoroughly investigated to determine overall program effectiveness.
5-8
5.2.4.3 Significant Finding. A significant finding is a maintenance record considered directly related to the task being analyzed that indicates a functional failure or significant degradation/wear has occurred. These are primary analysis concerns. Examples of significant findings include:
1. Systems that:
• Affect airworthiness or safety on the ground or in flight.
• Are undetectable or unlikely to be detected during operations.
• Have significant operational impact.
• Have significant economic impact.
2. Structures that:
• Affect airworthiness or safety on the ground or in flight.
• Have reportable defects under the SDR system.
• Have Level 2 corrosion.
• Have Level 1 corrosion found other than at the scheduled maintenance interval.
5.2.4.4 Non-Significant Finding. A non-significant finding is a maintenance defect considered directly related to the task being analyzed that does not indicate a functional failure or significant degradation or wear and which does not fall into an example category for significant finding (e.g., Level 1 corrosion found at the scheduled maintenance interval). Depending on risk, these may or may not be primary analysis concerns.
5.2.5 Summary of Data. Each interval adjustment should include a summary of the data that was used for the analysis. Relevant, accurate, and a sufficient amount of data constitute the foundation for an operator’s standards determining their scheduled maintenance time limitations. This data summary should demonstrate that the processes and procedures of the operator’s reliability program regarding data selection were followed.
5.2.6 Evaluation of Data Sources. Table 5-1, Data Sources by Task Type, contains recommended data sources that you may use for analysis to determine time limitations for maintenance schedule tasks. The data is divided into primary and secondary sources.
1. Primary data elements should be used as the main data source for analysis by the operator to substantiate an effective task interval.
2. Secondary data sources should be used to further support or validate analysis conclusions, or when further investigation is warranted. This data should not be used as the sole source for determining time limitations of a task. Operators may identify additional data sources for analysis.
5-9 Table 5-1. Data Sources by Task Type Task Type Primary Data Secondary Data Systems tasks1 (excluding Failure Effect Categories (FEC) 8 & 9) In-service operational data Routine task findings Propulsion tasks1 (excluding FEC 8 & 9) In-service operational data Routine task findings Structures tasks2 Routine task findings3 In-service operational data Zonal tasks Routine task findings3 In-service operational data FEC 8 & 9 tasks Routine task findings/sampling at proposed interval3 FEC 8 ONLY: MSG-derived functional failure data/expand data set/previous accomplishment of task Enhanced zonal analysis procedure (EZAP)/electrical wiring interconnection system (EWIS) tasks Routine task findings3 In-service operational data Lube tasks Volume of consumables, rates/sampling at proposed interval, and/or degree of wear noted on component teardown reports Operational reliability data Filter tasks Analysis of remaining life at current interval/sampling at proposed interval Operational reliability data4 1 The primary data source for systems and powerplant tasks (excluding FEC 8 & 9 tasks) is the operator’s operational reliability data. The effectiveness of these tasks is measured by the ability of the maintenance schedule task to detect and correct defects in the system before the defect impacts the operational environment.
2 Include any damage (fatigue cracks or corrosion), whether or not it was discovered as a result of a scheduled maintenance task.
3 It is important to note that because failures associated with these tasks are normally found during the accomplishment of the routine task, this data may not be captured as part of an operator’s normal reliability data elements. Therefore, some means of capturing those failures for analysis is necessary.
4 Only for those tasks where failures would be evident to the flightcrew or maintenance personnel and would be recorded in the operational data (such as logbook writeups).
5-10
5.2.7 Analysis Process. Once the operator has identified and obtained the data, the operator must analyze the data in order to substantiate interval adjustment or task revision.
To perform the correct level of analysis, the operator should develop standardized decision logic based on the type of task and data being analyzed, and should be performed by appropriately trained and competent personnel. For examples of this decision logic, see the appendices within this AC.
5.2.8 Result of Analysis. If the analysis shows the impact on operations for a given system is low, the task may be considered effective regardless of the number of scheduled maintenance findings. Conversely, a high degree of operational impact could indicate an ineffective task, even if the number of scheduled maintenance task findings is low.
Other combinations are also possible. Table 5-2, Review of Systems/Powerplant Tasks, represents examples of situations that may be identified during the analysis. This table is relative; conclusions should be based on your unique operations. It is not definitive or exhaustive, but is intended to only provide guidance to operators when evaluating tasks and findings. The existence of scheduled maintenance findings by itself is not an indication of an ineffective program.
Table 5-2. Review of Systems/Powerplant Tasks1 Low Number of Unscheduled Maintenance Defects High Number of Unscheduled Maintenance Defects Low Impact on Operations High Impact on Operations Low Impact on Operations High Impact on Operations Low Number of Scheduled Maintenance Defects Effective but not optimized Ineffective Note 2 Ineffective High Number of Scheduled Maintenance Defects Effective and optimized Ineffective Effective but not optimized Ineffective 1 FEC 8 and 9 tasks are designed to detect failures that are not evident during normal operations. Therefore, these failures do not typically result in an impact to in-service operations. (Refer to Maintenance Steering Group - 3rd Task Force (MSG-3).)
2 Additional analysis is required to determine task effectiveness.
5.2.9 Data Standards. You should develop a standard for how to collect, validate, display and archive task data, along with the results of any analysis performed, including conclusions and recommendations to be considered during your internal approval process. You should consider adding a special emphasis task section to your Summary of Data that contains information related to certain tasks that would warrant special analysis or consideration, such as:
5-11
1. FEC 5 and 8 tasks,
2. Level 2 or higher corrosion findings,
3. EWIS and fuel tank safety tasks, or
4. Lightning/High Intensity Radiated Field (L/HIRF) tasks.
5.3 Recommendations. You are expected to investigate and develop risk-based recommendations (which may include “no action” recommendations), in response to variations from a performance standard. The results of the investigation, including the data collected and analyzed, root cause(s), the risk assessment, and recommended action should be documented and retained.
5.3.1 Recommended Maintenance Schedule Actions. See Appendix A, Task/Interval Adjustments—Top-Level Chart. The purpose of a reliability program is to allow an operator a means to determine effective time limitations, and authorization to publish and use those time limitations as your maintenance schedule without prior FAA involvement.
Unless otherwise restricted, an operator’s revisions to the maintenance schedule could include:
1. Escalating (increase) or de-escalating (decrease) the current maintenance schedule interval;
Note 1: You may not escalate airworthiness limitation items (ALI) using your reliability program.
Note 2: Except as provided in the Airworthiness Limitations Section (ALS), operators may not escalate tasks that are “tagged” or otherwise identified as precluding a Certification Maintenance Requirement (CMR), beyond the System Safety Assessment (SSA) interval. Refer to AC 25-19, Certification Maintenance Requirements.
2. Deleting an existing task;
3. Adding a new task;
4. Revising task procedures; or
5. Accepting or validating current performance and continue to monitor without further action.
5.3.2 Other Recommendations. Your reliability program is part of your CASS; however, it does not replace or substitute for a CASS. As part of CASS, recommendations resulting from your data collection and analysis may go beyond a maintenance schedule adjustment, and result in an input to the operator’s broader CASS. Some recommended examples are:
5-12
1. Changes within other elements of the CAMP:
• Airworthiness responsibility,
• Air carrier maintenance manual,
• Air carrier maintenance organization,
• Accomplishment and approval of maintenance and alterations,
• Maintenance schedule,
• Required inspection items,
• Maintenance recordkeeping system,
• Contract maintenance,
• Personnel training, and
• CASS.
2. Fleet modifications or configuration changes.
3. Changes to maintenance, operational procedures, or techniques. For example:
• One-time special maintenance for the fleet,
• Changes to provisioning of spare parts for maintenance, and
• Changes to manpower and equipment planning.
4. Requests for support from Original Equipment Manufacturer (OEM).
5. Coordinated efforts with other divisions (Flight Operations, Ground Operations, etc.).
Note: The operator’s CASS procedures may be followed for recommendations outside the scope of an adjustment to the maintenance schedule.
6-1 CHAPTER 6. MAINTENANCE SCHEDULE CHANGES—INTERNAL APPROVAL
AND IMPLEMENTATION
6.1 Approval Process. In addition to the appropriate substantiation data, the operator’s maintenance schedule adjustment process should define internal approval and implementation procedures. An approval plan should be comprised of procedures initiated by the competent review and disposition of a recommendation by the designated decision authority. The party responsible for the implementation of the change should be identified and defined. The operator should also develop methods to:
• Resolve nonconcurrence.
• Ensure closure of all proposals.
• Archive the disposition of a recommendation.
6.1.1 Revisions Without Direct FAA Involvement. Unless otherwise restricted (such as airworthiness limitations (AL), Certification Maintenance Requirements (CMR), electronic wiring interconnection system (EWIS), or Airworthiness Directives (AD)), you may make revisions to the maintenance schedule without direct FAA involvement.
However, you should coordinate significant revisions with the PMI prior to publishing your revision. Prior coordination with the PMI is recommended for all maintenance schedule revisions. You should be prepared to provide documentation to the FAA demonstrating that the process used to determine maintenance schedule revisions was accomplished in accordance with your standards for determining time limitations.
Note: Your entire process is subject to continual FAA oversight to verify you are executing your process in accordance with your program, and that your data-based decisions are in the interest of safety and program objectives.
Therefore, your program should include sufficient documentation and recordkeeping requirements.
6.2 Implementation. Your process should result in a comprehensive plan for implementing changes to the maintenance schedule. The plan should include procedures for notifying FAA representatives assigned to the air carrier that maintenance schedule changes are planned or have been made. The implementation plan should be documented and be included with other information considered during the change approval process. In most cases, any change to the maintenance schedule will require significant coordination with maintenance support organizations such as resource allocation, logistics, and planning groups, especially when the interval for an entire packaged check is revised.
6.2.1 Escalated Intervals and Task Deletions. These changes will normally become effective immediately upon attaining internal approval per the operator’s defined task amendment process. The implementation of individually escalated task intervals is relatively straightforward.
Note: Special considerations must be given to time/date check tasks that evaluate remaining life-limits. An in-service, life-limited item near its life-limit that passes