(2) Introduction. The introduction section should state the goals and objectives of the airline’s FOQA program.
(3) FOQA Program Stakeholders. This section should identify the key stakeholders in the FOQA program. There may also be stakeholders outside the company that should be identified here.
(4) Protective Provisions, Pilot Association Agreement (if Applicable), and Corporate Policy Statement. Summarize the salient points of the protective provisions that the airline and its pilots are afforded from FAA certificate action or civil penalties from information and data collected and analyzed by the FOQA program. In addition, the protective provisions from the pilots association agreement should be summarized. Provide a copy of the agreement in Appendix 1 of the I&O Plan. If a pilot association agreement is not applicable to the airline, a corporate policy statement should be included in the I&O Plan that establishes protective provisions to its pilots against disciplinary or other pejorative action from the airline from data or information produced by the FOQA program.
(5) Data Protective Provisions and Security.
(a) In this section, include a summary of the protective provisions to be incorporated into the FOQA program that will gain acceptance by all participants, including the pilot association (if applicable).
(b) Describe the pilot association agreement (if applicable) as it pertains to individual protection and data usage. At a minimum, discuss the following security considerations:
• The team member(s) responsible for data protection and security
• Data protection methods (including those provided by the GDRAS, physical security of FOQA media and facilities and information dissemination safeguards, etc.)
• De-identification requirements and procedures
• Methods for ensuring confidentiality
• Data retention policies and procedures
• Data storage policies and procedures
• Procedures for auditing and refining the security policy, methods, and procedures
(6) FOQA Program Components. Describe the specific technology components proposed for use in the FOQA program. The I&O Plan will need to be amended when changes or additions to the fleet types or changes to other technology components are made in the airline’s FOQA program. Program components described should include the following:
(a) Aircraft Fleet. Describe the following:
Page 24 Par 9
• How the initial aircraft fleet(s) were selected for participation in the FOQA program
• Number of parameters to be collected from each fleet
• Future plans for program expansion to additional fleets and aircraft within the current fleet
(b) Airborne Data Acquisition System. Describe the selection criteria and product selection process.
• What technical criteria were evaluated (recording capacity, media handling, and download capabilities)?
• How were service and support considered (warranty, repair station locations )?
• What compatibility issues with existing systems were raised (power considerations, size, weight)?
(c) Describe the airborne system configuration and provide information concerning the following:
• Strategy for acquiring airborne data
• Equipment to be installed in the aircraft (including vendor, part number, and other pertinent information) and the technology to be used
• Availability of the STC for each component installed on the aircraft or the plan for obtaining an STC
• Method used for loading and maintaining the LFL
• Fleet installation plan, including equipment installation requirements and schedule
• Support to be provided by the vendor (including a description of repair facilities and warranty policies)
(7) Airborne System Maintenance and Support. Describe whose responsibility it is within the operator to maintain the airborne acquisition and recording system, including parameter maps and configuration. This should also include interface between the FOQA FMT and the responsible party.
(8) GDRAS. Describe the GDRAS selection criteria and product selection process.
Describe the GDRAS to be used for the FOQA program, including:
• Specify the vendor, product name, hardware, software, operating system configuration, and communication network
• Summarize the proposed GDRAS’s functionality
• Describe vendor support and training
• Specify how LFLs, events, and parameters are defined, configured, and maintained
• Describe user configuration capabilities Par 9 Page 25
• Describe how the system complies with de-identification and security requirements defined by the air carrier and pilot association
(9) Other Equipment. Describe any other FOQA components, such as:
• Software for trend analysis, statistical analysis, and flight animation
• Remote data collection systems and communication infrastructure, as applicable
• The vendor, product name, and associated hardware, software, communication, and operating system requirements
• The selection criteria and product selection process
(10) Equipment Upgrade, Modification, or Replacement. Describe the procedures, including criteria, which will be used for upgrading, modifying, or replacing the FOQA program components once those components have been approved for use.
(11) FOQA Organization. This section explains the context of the FOQA program within the air carrier’s departmental settings and the individuals who will serve on the FOQA program team. The following subsections contain topics that should be incorporated into the I&O Plan:
(a) Organization Structure. Describe the following:
• The organization and management of the FOQA program, including the organizational entity responsible for the FOQA program
• Any oversight body (such as a steering committee or FOQA committee), including information on membership, charter, duties, meeting schedule.
• Provide an organizational chart that illustrates the organization and management structure of the FOQA program
(b) Personnel. Describe the skills, knowledge, duties, and responsibilities of the following anticipated key personnel associated with the FOQA program:
• Person providing corporate oversight
• FOQA manager
• Gatekeepers
• FOQA analyst
• FOQA intern, if applicable
• Members of the FMT, along with any other committees in the organizational structure
(12) FOQA Program Implementation.
(a) Describe the air carrier’s concept of the FOQA program. Include operational procedures for:
• Data processing and analysis Page 26 Par 9
• Investigating results
• Determining corrective actions to be taken for significant events
• Communicating findings to all effected stakeholders
• Obtaining feedback and follow-up for corrective actions
• Data trending
• Generating periodic reports
• Providing local FAA with aggregate FOQA information on the operator’s premises
(b) The FOQA Implementation Process. Describe the following:
1 Schedule and timeline for FOQA implementation, including required resources.
2 Training that will be provided to team members and key stakeholders.
3 Location of FOQA facilities, including central processing and any remote sites.
4 Program startup criteria, including milestones (e.g. on system training, education, infrastructure, data validation) that should be met before collected data are used for analysis of line operations. If a user needs assessment has been conducted, summarize the methods used, users interviewed, and results.
NOTE: When developing a schedul ed timeline for starting a FOQA program, many of these tasks may require considerable time for completion
(i.e., beyond the month in w hich they are listed as occurring) and that preliminary w ork will have commenced on these tasks prior to w here they appear on the checklist. Obtaining financial approvals and commitments for equipment expenditures and perso nnel, contracting for equipment purchases, and delivery of equipment may all have very long lead times.
(13) Education and training. Describe the following:
• How officers, senior management, team members, and stakeholders will be educated about the FOQA program
• How pilots will be educated about the program
• How team members will be trained
(14) Data Analysis Procedures.
(a) Data Usage and Management. Describe the anticipated usage of FOQA data for safety, operations, training, and maintenance/engineering. Also describe the framework/technology architecture that will be used for managing the data.
(b) Flight Data Collection and Analysis. Describe the following:
• Procedures for the physical retrieval of data from aircraft Par 9 Page 27
• Procedures for transferring data from airborne systems to the GDRAS, including media logistics and schedule for data retrieval
• Manual and automated methods to verify the quality and integrity of collected data, including any data quality standards
• Methods for logging and tracking airborne-collected data
• Procedures for handling invalid data and diagnosing airborne equipment problems
• Include a process flow diagram that shows the transit and direction of FOQA data through the system, including key systems, entities, and decision points
(c) Data Classifications and Definitions. Describe the following:
• Team member(s) involved in developing the event set
• Methods used to develop the event set
• Source documents
• Event categories, classifications, and severity levels
NOTE: Provide initial event set cla ssifications by operational mode in Appendix B.
(d) Data Definition Maintenance.
1 Describe how event definitions will be validated, reviewed, and defined by the FMT. The following boilerplate can be used as a baseline, but should be modified, as appropriate, to the air carrier’s specific situation:
NOTE: Documentation of event definiti on, validation, and modification w ill be maintained in the FOQA office and will be made available on request.
2 Also, describe the following:
• Procedures for validating, reviewing, and refining event definitions
• Procedures for creating and verifying new events
• Procedures for tracking modifications to event definitions
(e) Data Review and Evaluation. Describe the following:
• Procedures for periodic review of FOQA event data and trends, including personnel responsible and proposed schedule for review of data and trends
• Procedures for joint FAA/air carrier periodic review of aggregate trend data
• Procedures for notifying appropriate personnel (e.g., flightcrews, engineering/maintenance, and training) about events requiring immediate action
• Processes for maintaining event information for trend analysis, including databases and methods to have invalid events and associated data removed Page 28 Par 9
• Procedures for generating periodic reports to convey FOQA trends and findings
• Methods for detecting and analyzing data trends
• Procedures for crew contact and follow-up
• Procedures for determining corrective actions to be taken for identified events and/or trends
• Methods for obtaining feedback/follow-up for resolution
(f) Data Trending and Record Retention. Describe the following:
• The data retention policy for FOQA data and trend analyses, which should include requirements by maintenance to satisfy manufacturer warranty claims
• Archiving procedures and process for archiving and retrieval of archived data
(15) Program and Data Documentation. Describe how this I&O Plan will be maintained, who the review process will involve, how changes will be tracked, and how revisions will be submitted to the FAA. With the approval of the POI, the air carrier may submit quarterly updates of the I&O Plan to reflect changes that were made during the preceding quarter.
(a) Operational Development. Describe the overall development of FOQA program documentation in support of the program in cases of personnel transitions or program changes.
(b) Information and Data Control. Describe how changes to the FOQA Program will be documented. This should include the following:
• A description of the identified use
• Analysis that was accomplished
• Specific corrective actions or recommendations taken or made
• Personnel who were notified
• Resolution of actions or recommendations
(c) I&O Plan Revision Control. A revision control methodology should be incorporated into the I&O plan. This revision control methodology should include controls for page revisions, deletions, or replacements. The I&O Plan should also stipulate that the operator will submit all revisions to the POI and AFS-230.
(16) FAA Access.
(a) Procedures for joint FAA/air carrier periodic reviews of FOQA program effectiveness.
(b) Procedures for the operator's handling and marking of company proprietary and confidential information submitted to the FAA.
Par 9 Page 29
(17) I&O Appendices.
(a) Appendix I (Pilot Association Agreement, if Applicable). If the implementation of the FOQA program depends on obtaining a signed pilot association agreement, insert the agreement or side letter in this section. The agreement should describe how the pilot association will be involved in the operation of the FOQA program.
(b) Appendix II (Event Parameters and Definitions). This appendix should contain list of events and minimum parameters to be monitored and analyzed in a basic FOQA program. When establishing a new FOQA program, the events and parameters in this appendix should provide a good starting point. Note that there are many more parameters available for recording on modern aircraft than are listed in the appendix. Although many such parameters are not used in FOQA event creation, they can be useful to develop a more complete understanding of the causes of events and for aircraft maintenance troubleshooting purposes. Following initial establishment and validation of its program, operators are encouraged to consider expanding upon the basic parameter list provided in this appendix. Ideally, the aircraft fleet initially involved in starting a FOQA program should have a data bus capable of supplying the parameters listed in this appendix and a means of capturing and recording those parameters. It is important to understand, however, that the list of available parameters on any given aircraft will be a result of those provided by the airframe manufacturer, ordered by the air carrier, or a combination of both.
1 Event Selection. Depending on the GDRAS used, selecting events to be included and analyzed can be simple or complex. Developing a list of events from scratch can be extremely time-consuming, particularly during event validation. GDRAS vendors can also create and deliver event sets, but this may entail considerable expense. If an event list is available from another carrier or the list contained in this appendix is adequate, then the event selection process will be relatively simple and much less expensive.
NOTE: The event list should be ta ilored to the specific air carrier and aircraft type. The parameters used to me asure the event need to be recorded on that aircraft type. Next, the tolerances that trigger the events should be set to account for applicable federal regulations, aircraft limitations, and company policies and procedures. The FMT and FOQA analyst should w ork together to evaluate and adjust event triggers. Since maintenance w ill also be an important stakeholder, creating events that maintenance w ould be interested in analyzing would be advantageous.
2 Event Standardization.
(aa) If an operator has multiple aircraft type or model variance within its fleet, attempting to standardize events within the GDRAS analysis function may be advantageous. Numerous events will be common to all types and models of aircraft in an air carrier’s fleet (e.g., VMO, MMO, and VLE exceedences). These “common” events can be monitored and analyzed across fleets. However, there will be differences in the triggering limits, which will Page 30 Par 9
be dictated by an aircraft’s specific operating limitations. Analysis of these events across fleet or model types must account for the difference in triggering.
(bb) To assist in standardization of the analysis, the events can be organized or grouped in different categories. One method of organizing the events could be by “phase of flight.” Another method may be to use the sections in the operator’s manuals where aircraft limitations or flight procedures are outlined or defined. Users should choose the method that best suits their needs.
(cc) The events may be categorized by the flight phase in which they are most likely to occur, although they may also occur in other phases. However, the flight phase is not used as a discriminator in the creation of these events. Each event is designed to work even if the software used does not use flight phase in its analysis process. The use of flight phase is a common method and may speed the processing of raw data.
(dd) Maintaining documentation of event definitions used in an operator’s analysis may be useful. Documentation of the events can assist the operator in validating trend analysis and serves as a guide to program history as events change or are modified during the maturation of the program. Documentation could include the name for specific events, a description that clarifies what the event is designed to measure, suggested event-triggering conditions, the needed parameters to create the event, a possible way of defining the event, and explanatory notes or comments that go beyond the event description. In the event definition, user-defined variables that need to be extracted from Approved Flight Manual (AFM), Flight Operations Manual, Flight Standards/training guidance, or other applicable corporate guidance should also be included.
(ee) Some parameters used in events are derived, meaning they are not directly measured by sensors in the aircraft, but are calculated as part of the processing done by the analysis software. Height Above Takeoff/Touchdown (HAT) is a good example of this.
Many events require an altitude component in relation to the airport elevation. This is most accurately done by using the elevation at takeoff or touchdown, calculating altitudes above this elevation, and then measuring actual aircraft performance at these points. There are many possible ways to calculate events, and those presented here may not be compatible with parameters available on a particular aircraft or the capabilities of particular analysis software.
Maintaining documentation of events will assist the carrier in trend analysis and the development of events as the program matures. Appendix II of the example I&O plan in Appendix A of this AC contains a representative sample of event documentation.
CAUTION: Each air carrier should review all events to see that the limits defining each event account for applicable federal regulations, airplane flight manual limitations, and company policies and procedures.
(c) Appendix III (Glossary). Definitions of all acronyms used in the document should be included. The definitions should cover more than just what the acronym stands for, the reader should be able to understand what the acronym means. In constructing the glossary, Par 9 Page 31
assume that the reader of the I&O Plan is unfamiliar with FOQA, as might be the case with senior management, union officers, legal staff, or local FAA representatives.
(d) Appendix IV (References). Include citations for all referenced documents including, but not limited to:
• Title 14 CFR Part 193, Protection of Voluntarily Submitted Information
• Title 14 CFR Part 13, Section 13.401, Flight Operational Quality Assurance Program
• AC 00-46 (as amended), Aviation Safety Reporting System (ASRS)
• AC 00-58 (as amended), Voluntary Disclosure Reporting Program
• AC 120-59 (as amended), Air Carrier Internal Evaluation Programs: Air Carrier Internal Evaluation-Model Program Guide
• AC 120-66 (as amended), Aviation Safety Action Program
• Flight Safety Foundation, FAA Contract Report, Air Carrier Voluntary Flight Operational Quality Assurance (FOQA) Program, 1992
• General Accounting Office, GAO/RCED-98-10, Aviation Safety—Efforts to Implement Flight Operational Quality Assurance Programs, December 1997
10. APPENDICES.
• Appendix A: Example of a FOQA Implementation and Operations Plan
• Appendix B: FOQA I&O Plan Checklist
11. ABBREVIATIONS
ABBREVIATIONS Abbreviation Term AAC Airline Administrative Control AC Advisory Circular ACARS Aircraft Communications Addressing and Reporting System ACDF Airline Common Data Format ACMS Aircraft Condition Monitoring System ACSF Airline Common Statistical Format AEEC Airlines Electronic Engineering Committee AFE Above Field Elevation AFM Approved Flight Manual AFS Aviation Flight Standards Service AFS-230 Volunteer Safety Program Branch ALPA Air Line Pilots Association AOC Aircraft Operational Control AQP Advanced Qualification Program ARC Aviation Rulemaking Committee ARINC Aeronautical Radio Incorporated ASAP Aviation Safety Action Program ASCII American Standard Code for Informational Interchange ASRS Aviation Safety Reporting System Page 32 Par 9
ABBREVIATIONS Abbreviation Term ATC Air Traffic Control BASIS British Airways Safety Information System CAS Computed Air Speed CMO Certificate Management Office COTS Commercial Off-The-Shelf CSV Comma Separated Value CVR Cockpit Voice Recorder DAR Digital ACMS Recorder DAS Designated Alteration Station DAU Data Acquisition Unit DBMS Database Management System DFDAR Digital Flight Data Acquisition Recorder DFDAU Digital Flight Data Acquisition Unit DFDMU Digital Flight Data Management Unit DFDR Digital Flight Data Recorder DMU Data Management Unit EGT Exhaust Gas Temperature EGT Exceedance Guidance Team EMT Event Monitoring Team ERC Event Review Committee FAA Federal Aviation Administration FAR Federal Aviation Regulations FDAU Flight Data Acquisition Unit FDR Flight Data Recorder FMT FOQA Monitoring Team FOD Foreign Object Damage FOIA Freedom of Information Act FOQA Flight Operational Quality Assurance FSDO Flight Standards District Office g Gravity (G-Force) Par 11 Page 33
ABBREVIATIONS Abbreviation Term GB Gigabyte GDL Ground Data Link GDRAS Ground Data Replay and Analysis System GPWS Ground Proximity Warning System HAA Height Above Airport HAT Height Above Takeoff/Touchdown HQ Headquarters HTML Hypertext Markup Language I&O Implementation and Operations IT/IS Information Technology/Information Systems LAN Local Area Network LFL Logical Frame Layout LRU Line Replaceable Unit MB Megabyte MEL Minimum Equipment List MMO Maximum Mach Operating Speed MTBF Mean Time Between Failure N1 Low Pressure Compressor N2 High/Intermediate Pressure Compressor NAS National Airspace System NASA National Aeronautics and Space Administration NTSB National Transportation Safety Board ODBC Open Database Connectivity OQAR Optical Quick Access Recorder Order 8400.10 Air Transportation Operation’s Inspectors Handbook PAI Principal Avionics Inspector PCMCIA Personal Computer Memory Card International Association PDF Portable Document Format PMI Principal Maintenance Inspector POI Principal Operations Inspector PPH Pounds Per Hour PPM Partial Program Manager PSI Pounds Per Square Inch QA Quality Assurance QAR Quick Access Recorder RA Traffic Alert and Collision Avoi dance System (TCAS) Resolution Advisories RFP Request for Proposal ROM Routine Operational Measurement SRU Shop Replaceable Unit SSFDR Solid State DFDR STC Supplemental Type Certificate Page 34 Par 11
ABBREVIATIONS Abbreviation Term TA Traffic Alert and Collision Avoidance System (TCAS) Traffic Advisories TC Type Certificate TCAS Traffic Alert and Collision Avoidance System TE Triggered Event V1 Critical Engine Failure Speed V2 Takeoff Safety Speed VLE Maximum Landing Gear Extend Speed VMO Maximum Operating Limit Speed VREF Reference Velocity WAN Wide Area Network WDL Wireless Data Link 14 CFR Title 14 of the Code of Federal Regulations 49 U.S.C. Title 49 of the United States Code
/s/ James J. Ballough Director, Flight Standards Service Par 11 Page 35 (and 36)