Par 12-1 Page 47 CHAPTER 12. IMPLEMENTATION AND OPTIMIZATION OF
TASKING INTERVALS
12-1. INTRODUCTION. The guidance in this document is intended for use by original equipment manufacturer(OEM)/type certificate holder (TCH) and Maintenance Review Board (MRB)/Industry Steering Committee (ISC) members who are involved with the evolution/optimization or deletion/addition of tasks in an initial/current MRB Report (MRBR).
Apply this guidance for evolution/optimization or deletion/addition activities where no official correspondence has been forwarded to the airworthiness authorities, or for activities to be finalized (MRBR proposal submittal) after April 2009. The following is guidance for developing and assessing proposals to the MRBR.
a. The initial MRBR for any new aircraft is developed in the absence of actual in-service experience. As a result, the tendency is to be conservative in the decisionmaking process. As service experience is accumulated, task intervals (thresholds/repeats) should be adjusted to reflect the results of actual in-service data.
b. The OEM/TCH evolution/optimization process does not assume any operational control over an operator’s maintenance program.
NOTE: When intervals are stated in this document it includes both threshold and repeat values.
NOTE: If this chapter is not followed, the OEM/TCH will be limited to no more than 10 percent escalation with approved data. Further escalation is not allowed until a task is repeated and sufficient data are available.
12-2. PURPOSE. While this guidance is not exhaustive, use it as the basis for a Policy and Procedures Handbook (PPH) procedure when the OEM/TCH, MRB, and Industry Steering Committee (ISC) wish to proceed with evolution/optimization regarding the MRBR process.
Evolution/optimization or deletion/addition of a task through the management of data is a means to assure the continued applicability and effectiveness of the task, while simultaneously improving the integrity of the MRB process. This policy allows the OEM/TCH to develop and use a process that serves as a continuous analysis and evolution/optimization or deletion/addition for the MRBR. It is based on performance data and experience for model-specific fleets flown by multiple operators under a variety of operating conditions and environments.
12-3. POLICY DESCRIPTION. The OEM/TCH must meet the policy requirements defined by the regulatory authorities of the country of origin; and will define further details and procedure clarifications in the PPH. As the PPH is a living document, a response must be given within 60 days after ISC acceptance/OEM submission. The following should occur:
a. The MRB/ISC must coordinate and approve PPH revisions, where applicable.
b. Evaluate in-service data, both scheduled and unscheduled maintenance findings related to the intent of the Maintenance Steering Group (MSG) task.
c. Weigh the relevance and significance of findings.
d. Standardize data format and content (ATA SPEC 2000 Chapter 11 or equivalent).
e. Ensure data quality, integrity, completeness, and clarity.
f. Consider each task individually.
g. Review original design and engineering specs, as required.
h. Review all information related to continue airworthiness (Airworthiness Directives, Service Bulletins, in-service reports/letters, modifications/repairs, etc.).
i. Base MRBR task evolution/optimization or deletion/addition on worldwide representative samples that span the operating environment and age groupings of the aircraft.
j. Base interval evolution/optimization or deletion/addition on risk management. Risk management is the systematic application of management policies, procedures, and practices to the tasks of identifying, analyzing, evaluating, treating, and monitoring risk.
k. Use safety management principles at the OEM level. Safety management is the application of engineering and management principles, criteria and techniques to optimize safety.
It is an integrated and comprehensive engineering effort.
l. Apply statistical models to support the evolution/optimization or deletion/addition exercise. In a data-driven statistical decisionmaking process, data size is determined based on the level of confidence. Confidence level refers to the likelihood that the overall fleet performance lies within the range specified by the sample fleet performance. The confidence level is usually expressed as a percentage. For example, a 95 percent confidence level implies that the probability that the fleet parameter lies within the confidence interval is 0.95. For a given confidence level, data size may vary depending on the fleet size and variability of in-service data.
m. The OEM/TCH will collect sufficient data that would support the expected confidence level. However, engineering judgment will remain a part of the evaluation. Statistical analysis should be supported and validated by engineering judgment.
n. Measure task effectiveness should be measured and demonstrated, that is, the ability to:
• Detect and prevent defects before loss of function/structural integrity.
• Mitigate risk of exposure to hidden defects.
o. Record and respond to operators’ and regulators’ feedback.
p. Ensure the effectiveness and integrity of the process by: 1) collecting in-service data in an ATA SPEC 2000 Chapter 11 format or equivalent, 2) analyzing it, and 3) comparing the results with existing MRBR task requirements.
Page 48 Par 12-3
Par 12-3 Page 49
q. This policy allows for evolution/optimization or deletion/addition of MRBR, scheduled maintenance tasks, intervals, and enhances the use of reliability-driven maintenance analysis processes.
r. Adjust the MRBR based on performance data and analysis processes. However, operator reliability programs should continue to ensure continuous evolution/optimization of their maintenance programs.
s. The OEM/TCH evolution/optimization process does not assume any operational control over an operator’s maintenance program.
12-4. RESPONSIBILITIES. The OEM/TCH applicant must notify the approving authorities in writing of their intent to begin an evolution/optimization or deletion/addition process. This will be in the form of an official correspondence as defined by the approving authorities.
NOTE: Approving authorities are those authorities that approve the MRBR.
The approving authorities will respond, in writing, to the OEM/TCH of their intent to participate in the evolution/optimization or deletion/addition exercise for a given fleet or model.
a. OEM/TCH (PPH Amendment and ISC/MRB Acceptance/Approval). The OEM/TCH will include within the PPH the policy requirements and criteria as contained within this document. The OEM/TCH will further define the details and procedural actions necessary to conduct the evolution/optimization or deletion/addition exercise. The MRB/ISC will coordinate and approve this plan. Where documents that support the evolution/optimization or deletion/addition are incorporated by reference within the PPH, the current document number and revision number must be stated.
b. OEM/TCH Data Collection. The OEM/TCH system must include a data quality, data integrity, data quantity, audit system, and historical data tool as defined in the next steps.
c. Data Format. The OEM/TCH will use in-service data in a standardized format (ATA SPEC 2000 Chapter 11 format, or equivalent), as deemed acceptable by the regulatory authority, to ensure data quality and integrity. ATA SPEC 2000 Chapter 11 is an industry-sanctioned maintenance reliability data communication format. In order to use this format, operators would have to transition to this type of format, or the OEM/TCH would have to convert the operator data into this standardized format.
d. Regulatory Authorities. It is incumbent on the OEM/TCH to demonstrate to the regulatory authorities compliance with these guidelines for all and any evolution/optimization or deletion/addition MRB task adjustment.
12-5. DATA QUALITY. The OEM/TCH should have a system in place that allows for the collection of data found during operator’s task accomplishment to be delivered to the OEM/TCH and then entered in a standardized format into their data collection system. The data collected and used by the OEM/TCH regarding evolution/optimization will include the following information:
a. Aircraft Age. Aircraft age (since delivery) is measured in calendar days, flight hours, or flight cycles, as applicable. MRB task evolution will be based on in-service data collected from a representative sample of older and newer aircraft incorporating more current production standards and modifications. Fleet age representation will be summarized in the analysis report.
b. Geographical or Operational Environment Representation. MRB task interval adjustments will be based on in-service data collected from a representative sample that spans all operating environments. The data will be in proportion to the specific model fleet size of each geographical area; however, it is not necessary to sample all geographical regions, nor is it required to collect data from all extreme operating conditions (e.g., extremely hot and sandy (desert ), extremely cold (arctic). A brief summary of the operating environments of the sampled aircraft will be provided in the report.
c. Number of Tasks Accomplished. The number of times the task has been accomplished including “nil/no findings” will be captured and used in the evaluation. Participating operators should provide task findings and nonroutine write-ups for the related tasks of the sample fleet for the evolution/optimization or deletion/addition exercise reporting period.
d. Interval of Tasks findings Applied. Actual task interval of each participating operator will be captured and evaluated.
NOTE: The actual intervals may vary between operators and may be different from MRBR requirement. The impact of these variations will be assessed and accounted.
e. Component Data (Shop Findings, No-Fault-Found Removals and Failures).
Information regarding component removal and replacement activity and vendor repair documents should be evaluated, as applicable, where available. This information provides the data necessary to perform component failure-mode and life-cycle analysis which is necessary to support the evolution/optimization or deletion/addition of the tasks associated with the component.
f. Correct Mapping to the MRBR Task. Nonroutine write-ups and in-service findings should be linked to appropriate MRBR tasks, as applicable. Only findings related to the MSG-3 task intent are relevant.
g. Failure Effect Category Considerations. MRBR task interval optimization is based on principles that reflect the criticality of airplane systems components identified during MSG analysis. Account for Failure Effect Categories during the analysis.
h. Operational Representation Flight Hour vs. Cycles, Calendar Time. Aircraft utilization (flight-hours or cycles, as applicable) should be captured and evaluated. Include a summary of fleet-wide service experience, high-time aircraft (hours, cycles, years), time in service, daily utilization (high, low, average), etc in the analysis report.
i. Consecutive Tasking Requirements. To the extent possible, capture consecutive task check data to assess reliability of aircraft systems, components, or structural elements related to the MRBR task.
Page 50 Par 12-5
Par 12-5 Page 51 NOTE: This requirement may be applied to lower interval tasks.
Consecutive check data can be impractical for higher interval tasks.
j. Unscheduled Maintenance Findings. Mechanical irregularities and the resulting corrective actions captured from pilot reports and maintenance reports should be reviewed, as applicable. Unscheduled maintenance is a prime indicator of the effectiveness of the scheduled maintenance program.
k. Scheduled Maintenance Findings.
(1) Routine Maintenance Tasks That Generate No Findings. Tasks that generate no findings are as important as tasks that generate findings in determining failure-mode and life-cycle analysis.
(2) Routine Maintenance Tasks That Generate Nonroutine Cards. These findings, which require corrective action, involve structures, area/zonal, and aircraft systems categorized by ATA chapter.
l. Unrelated Significant Findings, If Applicable. Operators should capture significant nonroutine write-ups generated in the course of an unrelated maintenance task, if applicable.
These findings, which require corrective action, may not correlate to a routine maintenance task.
m. Four-Digit ATA Code, If Available. To the extent possible, operators should provide four-digit ATA code for scheduled/unscheduled maintenance write-ups to facilitate transfer of findings to appropriate MRBR tasks.
n. Serial Number of Aircraft. The operator should provide an aircraft manufacturer serial number that uniquely identifies each aircraft in the sample fleet.
12-6. DATA INTEGRITY. Data integrity is the quality of correctness, completeness, and compliance with the intention of the creators of the data. It is the condition in which data are identically maintained during any operation, such as transfer, storage, and retrieval. Data integrity is achieved by preventing accidental or deliberate, but unauthorized insertion, modification, or destruction of data in a database.
a. Data Validation. OEM/TCH must have a data validation process that does the following:
(1) Verifies that operator data are converted to ATA SPEC 2000 Chapter 11 or equivalent standard format.
(2) Ensures that all required data elements and attributes are satisfied for submitted data.
b. Audit System. The audit system must ensure that all data be traceable to the original task.
12-7. DATA REVIEW.
a. Analysis Schedule—Evolution/Optimization or Deletion/Addition Timeline. MRB task interval adjustments should be considered after sufficient service experience is accumulated since entry into service. Subsequent task interval adjustments should be considered after additional service experience has been accumulated since the last interval adjustment. In both cases, data sufficiency is measured by the level of confidence as stipulated in these guidelines.
b. Statistical Analysis. The OEM/TCH will develop and implement a statistical analysis system to provide justification that a 95 percent confidence level has been achieved for the evolution/optimization or deletion/addition exercise on a task-by-task basis. Exceptions can be presented and may be approved at the discretion of the approving airworthiness authorities.
c. Engineering Analysis. Engineering analysis will verify that findings are relevant to the scheduled task under evaluation. Nonroutine write-ups will be evaluated to determine the significance or severity of findings. Pilot reports and component reliability reports will also be examined to account for line maintenance activities that may be relevant to the task under evaluation. The severity of the findings must be considered and evaluated.
NOTE: Scheduled servicing (e.g., lubrication/oil replenishment) task data do not result in reported related findings, therefore, engineering assessment must be conducted to support an evolution/optimization or deletion/addition.
Negative long-term effects (e.g., corrosion) resulting from inappropriate servicing intervals must be considered.
d. Modification Status, Airworthiness Directives, Service Bulletins, Service Letters, etc. Review all information related to the task (Service Bulletins, Airworthiness Directives, service letters, and other in-service reports/resolutions, as applicable). Also, assess fleet configuration.
e. Internal Review. The OEM/TCH will develop and implement internal quality procedures to review and validate MRBR revision process as defined in the PPH. OEM/TCH will develop and implement internal processes to validate MRBR revised tasks and/or intervals resulting from evolution, or demonstrate that an equivalent written internal process already exists to reach the same intent (not required before April 2010).
12-8. DATA CORRELATION. Correlate mean time between unscheduled removal, mean time between failures, pilot reports, nonroutines, technical followup on open technical issue, and all other pertinent data, as applicable.
a. Working Group Activity (WG)—Interval Recommendation to the ISC (e.g., increase, decrease, remain the same, introduction of new task, or task deletion).
(1) MRB task intervals can be escalated based on the results of in-service experience. In addition, tasks should be de-escalated when in-service data support interval reductions. A task may also be deleted when it is determined that it is ineffective, or the failure mode for which the task was selected never developed due to effective design provisions.
Page 52 Par12-7
Par 12-7 Page 53 (and 54)
(2) Task deletion, addition, or modification of intent requires new/revised/amended MSG-3 analysis. However, complete reanalysis of the MSG package is not required. Any decision together with justification will be recorded and traceable in the associated MSG-3 analysis. Applicable and effective criteria as specified in MSG will be observed.
(3) The intervals of potential failure finding tasks (i.e., those looking for degradation) should be less than the shortest likely interval between the point at which a potential failure becomes detectable and the point at which it degrades into a functional failure. If the specific failure data are available, this interval may be referred to as the “P to F” interval. Assess consecutive task accomplishments to show that failures are not occurring before the new initial interval.
(4) Interval determination should be validated with a maintenance engineering analysis based on consideration of all the items listed in the quality and quantity of data. The process will be referred or mentioned in the PPH for ISC and regulatory acceptance.
b. ISC Review Acceptance of MRBRP. The ISC will ensure all PPH guidance has been followed and applied. The MRB will ensure all PPH guidance has been followed and applied.
The MRBR is released.
Par 13-1 Page 55 CHAPTER 13. STATISTICAL ANALYSIS TASKING OPTIMIZATION
13-1. GENERAL. Statistical Analysis Tasking Optimization (SATO) is a term to describe an original equipment manufacturer (OEM)/type certificate holder (TCH) customized program for optimization of scheduled maintenance. It is data driven, combining data for model-specific fleets from multiple operators that use the type of aircraft under a variety of operating conditions and environments. Its intention is to use a larger fleet sampling data to optimize tasking intervals in operators’ maintenance programs. SATO is not intended to replace an operator’s Reliability Program (if applicable). Its use by operators will be voluntary. OEMs/TCHs that wish to offer SATO programs must design a framework for its operation. The OEM/TCH will work with the appropriate Aircraft Evaluation Group (AEG) to evaluate the program at the Maintenance Review Board (MRB) level.
13-2. APPLICABILITY. An OEM/TCH SATO program should contain these minimum elements:
a. A delivery system of data from the operator to the OEM in an ATA SPEC 2200 format (or equivalent).
b. An in-service data program to catalog the data into a useable form.
c. An aircraft reliability management database. The OEM/TCH must have a quality assurance system that ensures the data are clean and valid before accomplishing the engineering analysis.
d. A statistical model that will optimize the data.
13-3. OPERATOR PARTICIPATION.
a. Prerequisites for Participation. If an operator wishes to participate in an OEM/TCH SATO program, the OEM/TCH will have to accept it. This means the operator must meet the OEM/TCH prerequisites that may include supplying specific historical data about their fleet in a format acceptable to the OEM/TCH program. Complete (from birth) data would be the best;
however, that will not always be possible. The OEM/TCH program will set a baseline of minimum data.
b. Maintenance Planning Document (MPD). Data will be combined with other participant operators’ data to “optimize” the MPD by including a larger sample of data. The AEG chairperson or the designee for the affected aircraft will review the OEM/TCH recommendation for an optimized task and resolve any issues. The principal inspectors (PI) who have operators with affected aircraft will have an opportunity to comment on the proposed optimized task before final consideration. When satisfied with the technical merits, the AEG will send the OEM/TCH recommendation to AFS-302 for policy review and final concurrence. The OEM/TCH will then distribute the accepted recommendation to all participating operators. Only operators participating in the SATO program will be allowed to use the optimized MPD. The participating operators would then incorporate the new revised MPD task into their maintenance program and submit the revision to the PI.
Appendix 1 APPENDIX 1. ACRONYM/ABBREVIATION LISTING FOR THE MRB REPORT The following is a recommended listing of acronyms and abbreviations that may be contained in each MRB Report.
AC Advisory Circular ACO Aircraft Certification Office AD Accidental Damage/Airworthiness Directive ADR Accidental Damage Rating AEG Aircraft Evaluation Groups AEG-BOS Boston-AEG, Engines, Propellers AEG-DFW Dallas Ft. Worth-AEG, Rotorcraft AEG-LGB Long Beach-AEG, Transport Aircraft AEG-MKC Kansas City-AEG, Small Airplane AEG-SEA Seattle-AEG, Transport Aircraft AFS Flight Standards Services AEP Age Exploration Program AFRP Aramid Fiber Reinforced Plastic ALI Airworthiness Limitations AMM Aircraft Maintenance Manual AMOC Alternate Means of Compliance ATA Air Transport Association of America, Inc.
CAM Canadian Airworthiness Manual CAA Civil Airworthiness Authority CFR Code of Federal Regulations CFRP Carbon Fiber Reinforced Plastic CMCC Certification Maintenance Coordination Committee CMM Component Maintenance Manual CMO Certificate Management Office CMR Certification Maintenance Requirement CP Corrosion Program CPCP Corrosion Prevention and Control Program DAH Design Authority Holder DET Detailed Inspection DIS Discard DSO Design Service Objective DTA Design Tolerance Assessment DTR Damage Tolerance Rating DVI Detailed Visual Inspection DY Daily EASA European Aviation Safety Agency EAPAS Enhanced Airworthiness Program for Airplane Systems ECO Engine Certification Office ED Environmental Deterioration Page 1