(b) Forwarding of Data to the GDRAS Location. Depending on the size of the operator’s route structure, the location of where the data is removed in relation to the location of the GDRAS can be great. Methods for transferring the data to the GDRAS may consist of the following:
1 Ground-Based Transportation. The storage medium can be mailed from the maintenance location using regular mail, company mail, or private overnight forwarding companies. If this type of process is used, a tracking system should be developed so that the recording medium removal timing and location can be verified and documentation of aircraft data retrieval can be maintained. This will prevent a loss of recording medium so that the timing of data acquisition into the GDRAS can be tracked.
2 Electronic Transmission. This is a remote data transmission from the aircraft maintenance location to the GDRAS by the use of download equipment or milking-type machines that interface with the aircraft or by removal of the storage media from the onboard system. This process, while more efficient, requires a larger capital outlay and requires sufficient data transmitting capability from the remote maintenance locations to the location of the GDRAS. Coordination with an operator’s information services department will be needed to accomplish this. Data security issues must be considered when incorporating this process.
3 Wireless Transmission. This is an emerging technology that enables direct transmission of aircraft flight data to a network that interfaces with the GDRAS using wireless technology. The download is accomplished automatically, thus removing the requirements for maintenance involvement. Incorporation of this technology involves aircraft and ground-based data transfer systems to be installed. Data security issues must be considered when incorporating this process. Close coordination with an operator’s engineering and information services departments will be needed.
b. FOQA Program Description.
(1) The improvement of flight safety is the driving force behind the implementation of FOQA programs. A FOQA program is used to reveal operational situations in which risk is increased in order to enable early corrective action before that risk results in an incident or accident. FOQA should interface and be coordinated with the operator’s other safety programs.
The FOQA program should be part of the operator’s overall operational risk assessment and prevention program as described in part 119, section 119.65 and FAA guidance materials. Being proactive in discovering and addressing risk will enhance air safety.
(2) In a FOQA program, data are collected from the aircraft by using special acquisition devices, such as QARs, or directly from the FDR. Using one of several available transmission methods, data are periodically retrieved and sent to the air carrier’s FOQA office for analysis.
This office usually resides within the flight safety organization at the air carrier. The data are then validated and analyzed using specialized processing and analysis software, known as GDRAS, designed to convert the flight data into usable information.
Par 6 Page 7
NOTE: The quality and capability of a carrier’s FOQA program w ill be directly dependent on the number of parameters available. The carrier should see that sufficient parameters are available for collection from the acquisition device or FDR (see a ppendix A, Example of a FOQA Implementation and Operations Plan).
(3) The GDRAS transforms the data into an appropriate format for analysis and generates reports and visualizations to assist personnel in analyzing the collected data. It extracts FOQA events from the raw digital data stream based on parameters, threshold values
(e.g., descent rate in excess of 1,000 feet per minute on approach), and/or routine operational measurements that are specified by the air carrier. The analysis may focus on events that fall outside normal operating boundaries, event categories, or ROMs, as determined by the air carrier’s operational standards (as well as the manufacturer’s aircraft operating limitations). The FOQA FMT then reviews the events to assess their validity and potential significance. FOQA events are then marked for appropriate handling.
(4) In terms of determining the root causes of systemic problems that need correction, aggregate FOQA data have proven to be of greater value than detailed parameter data gathered during a single flight. Individual data records are typically aggregated into categories to assist the analyst in looking for trends and patterns. For example, an analysis may be conducted on the average maximum rate of descent below 2,000 feet by airport by fleet type. This may be useful to better understand the meaning of the data once related events indicate that this is an area requiring investigation. This analysis may suggest that all fleets are experiencing high descent rates at a certain airport or just a specific aircraft type. This type of information can be used to pinpoint the potential source of the problem and, hopefully, suggest the nature of appropriate corrective action.
(5) Data that could be employed to determine flight crewmember identity are removed from view in the electronic record as part of the initial processing of the airborne data. However, air carrier FOQA programs typically provide for a gatekeeper, who is provided with a secure means of determining identifying information for a limited period of time, in order to enable follow-up inquiry with the specific flightcrew associated with a particular FOQA event. Such contact is usually limited to situations when further insight into the circumstances surrounding an event is needed. The gatekeeper is typically a line captain designated by the air carrier’s pilot association (if applicable). The concurrence of the gatekeeper is required in order to initiate a follow-up with an individual pilot. Follow-up inquiries with individual crewmembers concerning FOQA events will normally be accomplished by a line captain designated as a gatekeeper by the air carrier’s pilot association (if applicable).
Page 8 Par 6
c. FOQA Analysis Process.
(1) Overview. The FOQA analysis process must be developed based on the objective and scope of the intended program. At a minimum, the process will be determined depending on whether information will be used to evaluate or effect change in any or all of the following areas:
• Operational Safety
• Aircraft Performance
• Aircraft System Performance
• Crew Performance
• Company Procedures
• Training Programs
• Training Effectiveness
• Aircraft Design
• ATC System Operation
• Airport Operational Issues
• Meteorological Issues
NOTE: Data analysis may be different for each of these groups, depending on the intended use of the information. What type of analysis is available will be a function of the aircraft reco rding capability, available parameters, and GDRAS hardw are and softw are capabilities. Extensive coordination between the FOQA FMT and other ai rline departments is crucial in maximizing analysis capabilities within the FOQA program.
(2) Data Recording. The available parameters and their associated sample rates and recording accuracies will directly affect FOQA analysis. The minimum core recorded parameters are those specified in part 121, sections 121.343 and 121.344 for FDRs. Aircraft that have been further outfitted with programmable FDAUs or DMUs may have parameters in excess of the minimum required. FDAUs or DMUs can be programmed to provide these additional parameters dependent upon storage medium capability. These FDAUs and DMUs may also be able to modify the sample rate through reprogramming. This will be a function of the parameter sensor on the aircraft and recorder medium size. Close coordination with operator engineering personnel will be required to identify available parameters.
(3) Analysis Techniques. Two types of analysis techniques can be applied to FOQA data. They are parameter exceedence analysis and statistical analysis.
(a) Exceedence Analysis. This involves setting a specific limit for the GDRAS to detect for a particular parameter. For example, the GDRAS can be programmed to detect each time the aircraft roll angle exceeds 45 degrees. This data can be trended over multiple flights to determine the number of exceedence occurring per flight segment. In addition, the data can be trended to determine which phase of flight, airport, or runway, if appropriate, depending on the event type. Levels of exceedence can be programmed for particular events based on the operator’s risk assessment to assist in focusing resources on implementing corrective action on the highest perceived operational risk area. A higher level of risk may be associated with an Par 6 Page 9
occurrence where the bank angle reached or exceeded 60 degrees. The FMT, through the gatekeeper, may choose to contact the crew or conduct a more detailed investigation of the event for this type of exceedence in addition to just maintaining and monitoring the trends where bank angle exceedences reach 45 degrees or greater. Exceedence levels will have to be developed through assessment of a carrier’s operations manuals, training programs, and risk assessment process as part of the overall safety program.
(b) Statistical Analysis. This is used to create profiles of flight, maintenance, or engineering operational procedures. The profiles can use several measurements to build distributions of various criteria. A distribution of data will show all flights and enable a carrier to determine risk based on mean and standard deviations from the mean. One procedure a carrier may look at is approach tracks. A profile would be designed to measure the different criteria of an approach, like airspeed, rate of descent, configuration, or power setting. For example, the GDRAS will capture the maximum airspeed of every flight on final approach. A series of distributions will show a picture of how all flights are performing. The carrier can then determine when an approach track may lead to an unstable approach or landing. Similar to exceedence analysis, statistical analysis can use distributions to drill down into the data to look at phase of flight, airports, or aircraft type, if appropriate. Each individual airline working with its FOQA team could establish or modify airline policy and training programs based on the performance of all its flights. Once a baseline is established, the data could be monitored to track the trend of what is occurring. The value of using statistical analysis is that data from all flights is used to determine risk for an airline without focusing on specific event exceedences.
The use of data distributions can develop a risk assessment process by establishing a baseline for trending data and determining critical safety concerns. Statistical analysis is a tool to look at the total performance of an airline’s operation.
(c) Validated Trend Information. This is reviewed to determine the nature of any required action. Such actions might include the immediate notification of maintenance personnel if limits were exceeded that require inspection of the aircraft, reviews of the event to identify possible corrective measures, or a determination that further information is needed through crew feedback. Depending on the particular event, the flightcrew may be contacted to gather more information about the circumstances and causes of the event. Corrective measures can range from modifications of flightcrew training to revisions of the operating procedures to equipment redesign. Information on valid events is also stored in databases for use in trend analysis.
7. FOQA PROGRAM ESTABLISHMENT AND IMPLEMENTATION. This section presents guidelines for designing, developing, implementing, and evaluating a FOQA program.
These guidelines do not reflect any single FOQA program in operation today. Rather, the guidelines describe the best practices culled from various air carriers that currently operate highly effective FOQA programs. The FAA does not require these guidelines to be followed in order for an air carrier to receive approval for its FOQA program. A successful FOQA program should be customized to address an air carrier’s individual needs and situation. Air carriers that are considering establishing a FOQA program should visit with air carriers that have already established FOQA programs. Such meetings are intended to foster a clear understanding of what is involved in the entire process. These discussions can provide useful information and practical know-how regarding lessons learned, obstacles to success, and potential benefits. The three Page 10 Par 6
phases of a FOQA program are Planning and Preparation (Phase I), Implementation and Operations (Phase II), and Continuing Operations (Phase III).
NOTE: Each phase contains specific elements, as illustrated in figure 1. Each of these phase elements w ill be furth er discussed in the follow ing sections.
Activities in each phase may occur in parallel. Also, because implementing a FOQA program is an iterative process, tasks in all phases may be open-ended and continue for the duration of the F OQA program. However, the transition to Phase II begins definitively when the FAA approves the air carrier’s I&O Plan.
FIGURE 1. FOQA PHASES S t a r t 6 m o n t h s 12 months 18 month s 2 4 months P h a s e I I : I mplementation & Operations z z z z z z z z z I m p l e m e n t and audit security mechanisms I n s t a l l e q u i p ment T r a i n p e r s o nnel I n v o l v e s t a k eholders C o l l e c t a n d process a i r b o r n e d a t a A n a l y z e a n d validate data D e v e l o p a n d document F O Q A s y s t em procedures D e v e l o p I n f ormation S u b m i t t a l P r ocedures D e f i n e s t a r t - up criteria Phase III: Continuing Oper a t i o n s z z z z z Conduct periodic reviews Track costs and benefits Evaluate emerging technologies Expand data usage Market the FOQA program P h a s e I : P l a n n i n g a n d P r e p a r a t i o n z z z z z z z z z z E s t a b l i s h a s t e e r i n g c o m m it t e e D e f i n e g o a l s a n d o b j e c t i v e s I d e n t i f y s t a k e h o l d e r s S e l e c t t e c h n o l o g y S e l e c t p e r s o n n e l D e f i n e s a f e g u a r d s D e f i n e e v e n t s N e g o t i a t e p i l o t a g r e e m e n t D e f i n e F O Q A I n f o r m a t i o n S u b m i s s i o n P l a n G e n e r a t e F O Q A a n d I & O p l a n s z Conduct periodic meetings with company stakeholding departments
a. Phase I—Planning and Preparation.
(1) Overview. Phase I is the foundation of a FOQA program. This phase begins when the air carrier decides to establish a FOQA program. A FOQA plan that defines the type of system that will meet user needs and how that system will integrate with other areas of the company and stakeholders is then written. Because much of Phase I requires interdepartmental cooperation and communication, many air carriers establish a FOQA steering committee or similar oversight body. During this phase, the air carrier should also establish mechanisms for communicating the current status and progress of the program. These updates should be tailored to the interests of the various users. Planning, organizing, and obtaining resources for a FOQA program can be extremely challenging. The effort required should not be underestimated.
Realistic assessments of the required time and resources indicate that designing, developing, and implementing a FOQA program takes months. Educating users and fully realizing the benefits takes even longer.
Par 7 Page 11
(2) Establish a Steering Committee. The formation of a FOQA steering committee is optional. However, it is a recommended first step in developing a FOQA program. The steering committee should define its members, meet regularly, and identify all applicable stakeholders early in the process. A typical steering committee might include a senior management member and representatives from flight operations, maintenance, safety, training, and the pilot association, if applicable. As Phase I progresses, the steering committee might invite the purchasing and legal departments to participate. Key issues to be addressed in Phase I are the size and scope of the FOQA program, organizational issues, resources requirements, and support from upper management. Determining where FOQA program responsibility will reside within the organization and ensuring participation of that group or department are important.
Experience has shown that the establishment of a steering committee is an important step for the following reasons:
(a) It formalizes the operator’s intent to initiate a FOQA program by creating a recognized, corporate-sponsored standing committee. Executive sponsorship demonstrates commitment and fosters this commitment through ongoing communication, thereby ensuring corporate-level understanding of the FOQA program’s value as well as its costs. Executive participation also makes FOQA a priority within the organization, supports allocation of required resources, and aids in overcoming any organizational resistance or departmental conflicts.
(b) It provides an early opportunity to identify and include the appropriate stakeholders from various air carrier departments in the FOQA program development process.
(c) It may conceive and articulate the vision for the air carrier’s FOQA program. The committee is also a vehicle for communicating that vision to the stakeholders and developing a consensus regarding key program issues.
(d) It can guide the FOQA program through Phase I and the development of the I&O Plan.
(3) Define Goals and Objectives. A key step in Phase I is to clearly define the vision, goals, and objectives of the FOQA program. These goals should be meaningful and measurable, define the expected uses for the FOQA data, identify critical success factors, and be prioritized.
Well-defined goals are tools for convincing stakeholders and management why the air carrier should invest in a FOQA program. At the beginning of Phase I, goals will likely be defined broadly, because the air carrier is still determining what the FOQA program will accomplish. By the conclusion of Phase I, a specific set of goals and objectives to achieve in Phase II should be defined.
(4) Involve Stakeholders. To correctly formulate the program’s expected output, stakeholders should be identified and involved early in the overall process. Each department is likely to require data analysis/reporting capabilities that are unique to its own needs. Identifying the stakeholders will help to identify their data analysis/reporting requirements, which will facilitate selection of the technology for recording, transporting, analyzing, and disseminating FOQA data. Initial stakeholders should include representatives from safety, flight operations, training, maintenance, engineering, airfield operations or ATC liaison, and the pilot association.
Page 12 Par 7
(5) Identify User Needs. The Steering committee should conduct a user needs assessment to develop a better understanding of stakeholder needs and to establish user requirements for the operations, training, engineering, maintenance, or other relevant departments. This study elicits and analyzes users’ needs to assist in selecting the proper technology for the program. A clear and comprehensive understanding of the users’ needs is essential to the program’s success. FOQA’s primary benefits are directly related to the usefulness of the data analyses to the stakeholder. Further, the FOQA data analyses need to be distributed intelligently. Determine what pieces of information are valuable to the stakeholder and consider how FOQA data can enhance current processes. An effective user needs assessment may identify user groups reluctant to participate in the FOQA program and help bring them into the program.
(6) Select Technology. Determining the technology and vendors to use is a critical decision. FOQA programs are comprised of at least three interdependent specialized systems.
These three systems are airborne data collection, ground data replay and analysis, and data management and analysis. The first generation of programs used by U.S. air carriers was designed around commercial off-the-shelf (COTS) systems similar to those used in European and Pacific Rim FOQA programs. An increase in the number of air carriers implementing FOQA programs has led to an increasing number of vendors and products supporting FOQA programs.
The approach taken by these vendors varies widely and the technological solutions offered should be carefully evaluated to assess their suitability to the air carrier’s needs. Special consideration should be given to technical issues, such as whether a particular product is compatible with the avionics and bus configurations already on the aircraft. Vendor training, warranty, and support policies are also important considerations in product selection.
(7) Select Personnel. Selecting personnel to staff the FOQA program depends on the program’s scope, the size and organization of the air carrier, and the technology that will be implemented to record and analyze information. A typical program includes a FOQA manager, one or more FOQA analysts, and a FMT composed of experienced pilots. FMT members should be technically proficient on the aircraft types used in the FOQA program and have excellent communication and problem-solving skills.
NOTE: In order to obtain perspectives from various interested departments, an air carrier may “share” employees a cross normal departmental lines. For example, two part-time people (one from the safety department and one from the engineering department) may provi de a more synergistic approach than a single full-time analyst devoted only to FOQA.
(8) Define Safeguards. FOQA requires vigilant security and privacy protection for confidentiality of the data and to protect data against unauthorized disclosure, alteration, misuse, or destruction. The issue of data protection and security is sensitive and focuses on the confidentiality of a particular air carrier, flight, date, or flightcrew and a recorded event. The security policy should balance users’ needs to access the data against the need to keep the data confidential. From the outset, air carrier policies and procedures for all security and protective aspects of the FOQA program should be carefully designed, documented, implemented, and Par 7 Page 13
periodically reviewed. The person responsible for implementing the security policy, such as the gatekeeper, should be identified.
(9) Define Events/ROMs. The events/ROMs that can be defined are dependent upon the available parameters that are recorded on a given aircraft type. Event/ROM definition and modification starts in this phase and continues for the duration of the FOQA program. Typically, the first phase of a FOQA program focuses on a single aircraft type. Information on defining events/ROMs and associated parameters is contained in appendix A.
(10) Negotiate Pilot Agreement (if necessary). Establishing an air carrier FOQA program may necessitate the negotiation of an agreement between the air carrier and its pilots’ collective bargaining agent. This agreement defines the specifics of the FOQA program and its objectives and administration. This agreement is crucial for obtaining buy-in from the pilot community and for ensuring that line pilots play an integral part in the process.
(11) Generate FOQA I&O Plan. The FOQA I&O Plan is the most important output of Phase I. The I&O Plan describes key aspects of the FOQA program. Preparing this document is one of the last steps of Phase I. Work done on the FOQA Plan can serve as a basis for the I&O Plan. The I&O Plan must be submitted to the FAA for review and approval in order to obtain protection from FAA civil enforcement actions. See section 13.401(c) for specific guidance for the development of the I&O Plan, including a template and checklist, is provided in appendices A and B.
b. Phase II—Implementation and Operations.
(1) Work-Intensive. Phase II is the most work-intensive phase of a FOQA program.
During Phase II, the FMT performs the tasks outlined in the I&O Plan. The airborne and GDRAS equipment selected in Phase I are installed. Phase II initially focuses on a single fleet and a limited number of equipped aircraft (usually 15 to 25). A major milestone in this phase is the first time that the air carrier records FOQA data, processes the data, and performs an analysis. During this phase, activities transition from designing the FOQA program to implementing and administering the program. The FOQA steering committee will begin to function in more of an advisory capacity as the FMT begins to assume its leadership role.
NOTE: Experience has show n that it takes approximately three to six months from when data are first recorded until official program startup can commence. The transition to Phase III occurs w hen the air carrier is convinced that FOQA data are accurate, reliable, and secure.
(2) Implement and Audit Security Mechanisms. The security policies and procedures defined in Phase I should be implemented and thoroughly tested to see that they are effective.
The actual mechanisms for protecting the data will be based on the capabilities of the hardware and software used in the program. The gatekeeper should be trained on how to implement and manage these mechanisms to protect data and control access. All GDRAS users should receive instruction about the protective provisions and how to handle problems. Periodic audits of the Page 14 Par 7
security mechanisms should be conducted, and the results should be used to fine-tune the policies and procedures.
(3) Install Equipment. A schedule should be established for installation of all equipment, including airborne and GDRAS. If installation of airborne equipment requires obtaining a Supplemental Type Certificate (STC), additional time and resources must be budgeted for this process. Coordination with maintenance and vendors will be required to track progress and resolve problems.
(4) Train Personnel. Personnel who install airborne systems may require training before or during equipment installations. For the GDRAS, the different levels of users may benefit from separate training classes tailored to their needs. The individual responsible for the GDRAS, typically the FOQA analyst, should receive training on system installation, configuration, and administration. Training for end-users should be tailored to their analysis needs and be provided as close to the initial exposure to the system as possible. End-users should be educated on product usage and the data. The training should occur after the system is fully operational and when the air carrier’s actual data are available so that users learn to effectively utilize the system with their data. Additional documentation may need to be developed to supplement materials provided by the vendors and to cover carrier and fleet-specific topics.
(5) Involve Stakeholders. To realize value from the FOQA investment, information derived from FOQA data must reach the appropriate user groups. Stakeholders will value the FOQA program only if they can obtain useful information that was not available before or obtain information faster than they can by using their current methods. The stakeholders will need to know what information is available to them. Education sessions should be held to promote user awareness of the information available, the program’s capabilities, and the information’s potential uses. Initial education should explain the FOQA program, including concepts, technology, benefits, and implementation schedule. Just because FOQA data are available does not guarantee widespread usage. Follow-up educational sessions may be required and should focus on instructing stakeholders on how to access and use the information available from FOQA data. Using a variety of methods and media (newsletters, e-mail, corporate Intranet, formal presentations, one-on-one meetings, team meetings, and videos) may attract a wider audience to the program than using only a single vehicle.
(6) Collect and Process Airborne Data. Procedures for retrieving the media on which FOQA data are recorded, such as optical disks or Personal Computer Memory Card International Association (PCMCIA) cards, or for downloading data from solid-state recorders using handheld readers, are needed. These procedures need to be developed and tested for accuracy, completeness, and resource requirements. The procedures should address the retrieval of recorded media, storage and distribution of unrecorded (blank) media, and installation of unrecorded media. Typically, media containing the flight data are removed from the recording device during a scheduled maintenance check. Retrieved media are sent to a central location for transmission or processing. New media are then inserted into the devices for the next round of flights. Schedules for retrieving the media are determined by the capacity of the media, the amount of data recorded, and the schedule of maintenance checks. The media retrieval schedule may range from 3 to 20 days. The same kind of schedule would apply if hand-held download Par 7 Page 15
devices were being used. Wireless data link systems, which transmit information directly to a ground system, eliminate the scheduling and staffing logistics associated with media or data retrieval. Using wireless data links may also reduce the potential for data loss when recording media reach capacity.
(7) Analyze and Validate Data. Stakeholder confidence in the FOQA program is directly proportional to the data’s accuracy, reliability, and completeness.
(a) Data reliability is determined by validating the integrity of the airborne and ground systems’ hardware and software. “Reasonableness” and consistency checks need to be performed on the recorded data. These checks can be accomplished by a variety of means, including:
• Validation by the vendor
• Comparison of Digital Flight Data Recorder (DFDR) data readings with FOQA readings
• Sensor validation
• Comparison of FOQA data with onboard, in-flight observations
(b) Data integrity and validity standards should be established to see that the data and associated analysis and reporting are performed in a consistent, standardized manner.
NOTE: Data validation activities tend to take much longer than anticipated.
1 Event sets will probably need to be fine-tuned after data from the first flights are analyzed to determine that what is being recorded is exactly what is needed and that appropriate data are being recorded at the proper resolution. Faulty sensors, modified LFLs, or missing software updates for acquisition units or analysis programs can cause errors. Fine-tuning event sets is a time-consuming and reiterative process. Failure to properly fine-tune the event sets can yield information of no use to stakeholders or worse, unreliable and invalid data.
Appendix A contains a suggested list of events to use (with modifications, as needed) and analyze in a basic FOQA program.
2 Any modifications to event sets or their associated parameters should be carefully tracked and documented to preserve the integrity of the process. Be aware that changes to event definitions may diminish the usefulness of trend or aggregate data if those data were captured under a combination of old and new event definitions.
(8) Store Data. FOQA programs yield vast quantities of raw data. The average amount of FOQA data collected from a single, digital aircraft is approximately 7.2 megabytes (MB) per day, resulting in 2.6 gigabytes (GB) per year. The air carrier that fails to plan ahead for data storage from all the aircraft covered will soon be awash in data. Although air carriers may choose to retain only a small portion of these data, establishing and maintaining a data storage program is critical for success.
Page 16 Par 7
(9) Develop and Document FOQA Program Procedures. As the program transitions through shakedown to production status, the FMT should develop and document procedures for operating and managing the program.
(a) Manual and/or automated procedures should be developed and documented for data security and data management (including backup and recovery, data archiving and restoration, monitoring and fine-tuning databases, defining and fine-tuning event sets, and data de-identification). Written procedures describe how to:
• Define, update, and delete user accounts
• Manage libraries of reports and graphics created by users
• Control security and access permissions for users and groups
(b) Documentation of all procedures is important for promoting consistent administration of the program. The importance of good documentation becomes evident when there are personnel changes on the FMT.
(10) Satisfy Startup Criteria. The criteria that indicate when a FOQA program is official and can transition from shakedown status into a mode of formal continuing operation should be established. The official start date defines when FOQA data will be used for formal analysis and trending. The FOQA manager and members of the FMT should periodically review the startup criteria to identify and correct problem areas, as well as certify criteria completion.
Satisfaction of the startup criteria heralds the start of Phase III.
c. Phase III—Continuing Operations.
(1) Startup Criteria. Phase III begins once the startup criteria have been satisfied.
Airborne and ground-based data systems must have been tested and confirmed, data accuracy and integrity must have been checked, and methods of analysis must be validated. At this phase, the FOQA program has stability, reliably providing high-quality, readily usable data to the FOQA program’s stakeholders.
(2) Iteration and Review. Phase III shifts the focus from implementing the technology to optimizing available data and the processes required to obtain the desired information.
Periodic reviews of all aspects of the FOQA program will determine whether the program is working as well as it could or whether revisions are required. These reviews will also identify when the program needs to be updated and modernized. Air carriers typically go through several iterations of experimentation and learning before mature FOQA processes are achieved.
Following the full cycle of analysis, design, implementation, operation, and evaluation for each iteration of upgrade or change is important. At the end of each iteration, the lessons the team has learned should be captured and documented so that subsequent efforts benefit from the team’s experience.
(3) FOQA Program Changes. Changes are likely to occur in an air carrier’s FOQA program as air carriers assimilate new technologies, modify event definitions, and change structures to meet the stakeholders’ growing needs. When changes to an air carrier’s FOQA Par 7 Page 17
program result in disparities between the program as implemented and the program as documented in the approved I&O Plan, the I&O Plan should be modified accordingly and changes must be submitted for acceptance by the FAA in order for section 13.401(c) to continue to apply to the FOQA program.
(4) FOQA Program Expansion. A FOQA program is neither static nor finite. It is meant to undergo controlled expansion and evolution as stakeholder demand for information grows and new technologies become available. The program should be able to accommodate new uses for FOQA data. These may require new equipment to capture and analyze the data.
Additional stakeholders may wish to make use of available FOQA information. Once successes are achieved, the process of expanding usage of FOQA information will likely accelerate. The real payback begins when FOQA capabilities are widely recognized and used throughout the air carrier. Phase III has no distinct end-point. It is deliberately “open-ended” to allow for enhancements to airborne, ground, and processing system technology; to allow program expansion to other fleets; and to allow broadened data usage. The program’s long-term plans will span several years, but incremental evolution should occur in carefully planned and welldocumented 6- to 12-month cycles.
(5) Communicate FOQA Program Benefits. A comprehensive program to continuously promote FOQA should be established, along with mechanisms for implementation.
The FOQA promotion effort should focus on gaining widespread support for the program. A variety of communication methods should be considered. For example, a newsletter can be used to inform users of significant program accomplishments, additional capabilities, and program evolution. Bulletin boards in pilot crew rooms are useful in disseminating FOQA information.
Video presentations on FOQA findings might be useful in pilot annual recurrent training classes.
Speaking at departmental and staff meetings and publishing testimonials about FOQA successes are also useful methods to spread the word about the program’s progress.
(6) Conduct Program-Wide Periodic Reviews. Periodic reviews and assessments should be conducted to determine that the program stays relevant to stakeholders’ new and existing interests and to identify areas for potential improvements. These assessments should determine if anticipated benefits are being realized and whether the information provided to endusers is accurate, timely, and usable. Ongoing user feedback mechanisms can be a valuable tool for capturing comments on the efficacy, usefulness, perceived shortcomings, and desired improvements of the current program. Likewise, efforts should be made to maintain current awareness of new technological alternatives and product enhancements. An audit of the quality of the aggregate data should also be performed along with an assessment of the accuracy of the reference and descriptive information. The tools employed to create aggregate and trend data should be periodically reviewed to determine if new technology would be more effective.
(7) Track Costs and Benefits. Justifying the investment in a FOQA program is an ongoing task. Capturing the initial acquisition and recurring costs is straightforward, as long as all categories of recurring costs are identified. Tracking the less tangible benefits is much more difficult. Benefits of a FOQA program, savings achieved and costs avoided, are spread across many departments. The safety department is an obvious beneficiary. However, placing a dollar value on an unknown number of aircraft accidents or incidents that were prevented because of Page 18 Par 7