Appendix 2 risk monitoring and management. This report will be available to all interested parties, including the FAA. If the performance indicators are acceptable, the operation will revert to routine monitoring.
(2) If the controls and mitigations perform to an acceptable standard, they become part of normal operations and are monitored by the FRMS SA processes. If the controls and mitigations do not perform to an acceptable standard, then it will be necessary to reenter the FRM processes at the appropriate step. This could require:
• Gathering additional information and data;
• Reevaluation of the fatigue hazard and the associated risks; and/or
• Identification, implementation, and evaluation of new or revised controls and mitigations.
Appendix 2 SECTION 6. FATIGUE RISK MANAGEMENT SYSTEM SAFETY ASSURANCE
PROCESSES
1. INTRODUCTION TO FRMS SA PROCESSES. The FRMS processes is the part of the day-to-day FRMS operations focusing on identifying fatigue hazards, assessing safety risks, putting in place controls and mitigation strategies, and monitoring their effectiveness. This section works through the basic steps in FRMS SA processes, which form another layer in a certificate holder’s defenses against fatigue-related risk.
a. SA Processes. Figure 2-5, Fatigue Risk Management and Safety Assurance Processes, illustrates the combined elements of the FRM and FRMS SA processes. FRMS SA processes are also part of the routine operation of the FRMS, and they monitor how well the entire FRMS is functioning. The FRMS SA processes:
(1) Check that the FRMS is functioning as intended;
(2) Check that it is meeting the safety objectives defined in the FRMS policy;
(3) Check that it is meeting regulatory requirements;
(4) Identify where changes in the operating environment have the potential to increase fatigue risk; and
(5) Identify areas for improvement in the management of fatigue risk (continuous improvement of the FRMS).
b. Continuous Monitoring. The certificate holder shall develop and maintain FRMS SA processes to provide for continuous FRMS performance monitoring, analysis of trends, and measurement to validate the effectiveness of the fatigue safety risk controls. The sources of data may include, but are not limited to:
(1) Hazard reporting and investigations,
(2) Operational data related to fatigue,
(3) Audits and surveys, and
(4) Reviews and fatigue studies.
c. Management Process. The certificate holder shall provide a formal process for the management of changes which shall include, but is not limited to:
(1) Identification of changes in the operational environment that may affect the FRMS;
(2) Identification of changes within the organization that may affect the FRMS; and
(3) Consideration of available tools that could be used to maintain or improve FRMS performance prior to implementing changes.
Appendix 2
d. Continuous Improvement. The FRMS SA process must provide for the continuous improvement of the FRMS. This shall include, but is not limited to:
(1) The elimination and/or modification of risk controls that have had unintended consequences or that are no longer needed due to changes in the operational or organizational environment;
(2) Routine evaluations of facilities, equipment, documentation, and procedures;
(3) Trending of safety performance indicators to determine if there is a need to introduce new processes and procedures to mitigate emerging fatigue-related risks; and
(4) As an option, retrospective and prospective biomathematical modeling of schedules to assess potential schedule-related fatigue risk.
e. Safety Performance Indicators. The FRMS SA processes use a variety of data and information as safety performance indicators that can be measured and monitored over time.
Having a variety of safety performance indicators, plus a safety target for each, is expected to give better insight into the overall performance of the FRMS than having a single measure.
Safety performance targets must fall in the tolerable region defined in the risk assessment process and they may need to be revised as operational circumstances change. The information, data, and safety performance indicators from the FRMS processes provide a source of information for the FRMS SA processes. In addition, the FRMS SA processes:
(1) Use information and expertise from other sources, both from within the certificate holder’s organization and external to it, to evaluate the functioning of the FRMS;
(2) Evaluate trends in safety performance indicators to identify emerging or changed hazards and refer these back to the FRM processes;
(3) Identify changes in the operating environment that could affect fatigue risk and refer these back to the FRM processes; and
(4) Provide input for improving the operation of the FRMS.
f. FRMS SA FIVE-STEP PROCESS. The FRMS SA processes consist of a five-step process that focuses on identifying fatigue hazards, assessing safety risks, putting in place controls and mitigation strategies, and monitoring their effectiveness. The objectives of each individual step are outlined below.
g. Step 1: Collect and Review Information.
(1) Safety Performance Indicators. This process involves collecting and reviewing information gained through the FRMS processes to examine the overall performance of the FRMS. Performance of the FRMS should be examined through indentifying a variety of safety performance indicators. This should include information specific to the FRMS as well as safety performance indicators.
Appendix 2
(2) Examples. Examples of safety performance indicators specific to an FRMS will include measures obtained through the FRMS processes, such as:
• The number of exceeded maximum duty days in operations covered by the FRMS,
• The number of voluntary fatigue reports per month,
• The average “fatigue call” rate by flightcrews on a specific pairing (trip),
• The ratio of fatigue reports from operations covered by the FRMS to fatigue reports from operations covered by the prescriptive flight and duty time regulations,
• Attendance at FRMS training sessions,
• Results on FRMS training assessments,
• The level of crewmember participation in fatigue-related data collection; and
• The number of times fatigue is identified as an organizational factor contributing to an event.
(3) Sources of Data. The sources of data for monitoring the safety performance of the FRMS may include (but are not limited to):
• Hazard reporting and investigations,
• Audits and surveys, and
• Reviews and fatigue studies.
(4) Hazard Reporting and Investigations. Trends in voluntary fatigue reports by crewmembers and others can provide valuable insights into the effectiveness of the FRMS.
Safety events in which crewmember fatigue has been identified as a contributing factor should be less common than fatigue reports. However, regular review of these events may also highlight areas where functioning of the FRMS could be improved. The value of both these sources of information depends on appropriate methods to identify the role of fatigue.
(5) Audits and Surveys. Audits and surveys can provide measures of the effectiveness of the FRMS without having to rely on fatigue levels being high enough to trigger fatigue reports or fatigue-related safety events (both of which are relatively rare events).
(a) Audits focus on the integrity of, and adherence to, the FRMS processes. These audits should answer questions such as:
• Are all departments implementing the recommendations of the FSAG?
• Are crewmembers using mitigation strategies as recommended by the FSAG?
• Is the FSAG maintaining the required documentation of its activities?
(b) Audits can also periodically assess the effectiveness of the FRMS (e.g., by looking at the status of FRMS safety performance indicators and targets). Audits are external to the FSAG, but may still be internal to the certificate holder (i.e., conducted by other groups within the organization). In addition, feedback from regulatory audits can provide useful information for FRMS safety performance monitoring. Another type of audit that can be used in this context is a review by an independent scientific review panel that periodically reviews the
Appendix 2 activities of the FSAG and the scientific integrity of their decisions. A scientific review panel can also provide the FSAG with periodic updates on new scientific developments relevant to the FRMS.
(c) Surveys can provide information on the effectiveness of the FRMS. For example, they can document how schedules and assignments are affecting crewmembers, either by asking about their recent experiences (retrospective) or tracking them across time (prospective). Surveys for this purpose should include validated measures, such as standard rating scales for fatigue and sleepiness and standard measures of sleep timing and quality. Remember that a high response rate (ideally, more than 70%) is needed for survey results to be considered representative of the entire group, and response rates tend to decline when people are surveyed too frequently (participant fatigue).
(6) Reviews and Fatigue Studies. In general, safety reviews are used to ensure that safety performance is adequate during times of change (e.g., during the introduction of a new type of operation or a significant change to an existing operation covered by the FRMS).
(a) A review would start by identifying the change (e.g., moving a trip to a crew base in a different time zone, changes in on-board crew rest facilities, significant changes in the total trip, or a change of equipment being used for the trip). It would then evaluate the appropriateness and effectiveness of the FRMS activities relative to the change (for example, proposed methods for fatigue hazard identification, the risk assessment process, proposed controls and mitigations to address the fatigue hazard(s), and measures of their effectiveness to be used during the implementation of the change).
(a) Fatigue studies, as part of FRMS SA processes, are undertaken when a certificate holder is concerned about a broad fatigue-related issue for which it is appropriate to look at external sources of information. These could include the experience of other certificate holders, industry-wide or nation-wide studies, and scientific studies. External sources of information are particularly helpful when an internal consensus for a course of action cannot be reached or to supplement the limited experience and knowledge within that certificate holder’s organization. Fatigue studies in this context are mainly used for gathering information about large-scale issues related to the FRMS, rather than for identifying specific fatigue hazards.
(7) FRMS Safety Performance Indicators. Trends in FRMS safety performance indicators are also an important source of information about the effectiveness of the FRMS. This may include indicators identified by the FSAG as part of the FRMS processes. They may also include indicators that capture more global aspects of the safety performance of the FRMS.
h. Step 2: Evaluate FRMS Performance.
(1) This process is intended to validate the effectiveness of the fatigue controls and mitigations by analyzing the data collected and reviewed in Step 1 of the FRMS SA processes to determine whether:
(a) All specified FRMS safety performance targets are being met;
Appendix 2
(b) All specified FRMS safety performance indicators remain in the tolerable region defined in the risk assessment process;
(c) The FRMS is meeting the safety objectives defined in the FRMS policy; and
(d) The FRMS is meeting all regulatory requirements.
(2) The following are examples of safety performance targets and indicators that could be used in FRMS SA processes and that correspond with the safety performance indicators identified above:
(a) The length of the maximum duty days in operations covered by the FRMS does not exceed the limits defined in the FRMS policy. This is reviewed monthly by a computer algorithm and trends across time are evaluated every 3 months.
(b) By the fourth month after the introduction of a new operation, there must be a stable, low number of voluntary fatigue reports per month or a clear downward trend in the number per month (allowing time for crewmembers and other affected personnel to adjust to the new operation).
(3) The FSAG is responsible for providing a written report on the validation phase of the new operation, including analysis of all fatigue-related events and voluntary fatigue reports, and documentation of the corresponding adjustments made in fatigue controls and mitigations.
Additionally, the following information that should be included in the report:
(a) The rate of fatigue calls under the FRMS operation in comparison to the rate under non-FRMS operations.
(b) Actions taken when the rate of fatigue reports for operations conducted under the FRMS exceed the rate of fatigue reports for non-FRMS operations. The actions by the certificate holder taken must be reported to the FAA.
(c) Evidence that, in the last quarter, designated management has provided adequate resourcing for the FRMS, as specified in the FRMS policy.
(d) Evidence that, in the last quarter, the FSAG has met as often as is required in the FRMS policy and has maintained all the documentation of its activities required for internal and regulatory auditing.
(e) Evidence that all personnel responsible for schedule design and assignments have met annual FRMS training requirements as specified in the FRMS promotion processes.
(f) Measures of the effectiveness of FRM training and education programs.
(g) Evidence that quarterly levels of absenteeism are below the target specified for each operation covered by the FRMS.
Appendix 2
(4) When FRMS safety performance targets are not met or when safety performance indicators are not at an acceptable level, the controls and mitigations in use may need to be modified by reentering the FRMS processes at Step 2 or beyond. It may also be appropriate to seek additional information from outside the organization (e.g., by looking at fatigue studies). It may be necessary to undertake a review of compliance of crewmembers and other departments with the recommendations of the FSAG. It may also sometimes be necessary to review the functioning of the FSAG itself to find out why the FRMS is not working as intended.
i. Step 3: Identify Emerging Hazards.
(1) Analysis of trends in safety performance indicators may indicate the emergence of fatigue hazards that have not previously been recognized through the FRMS processes. For example, changes in one part of the organization may increase workload and fatigue risk in another part of the organization. Identifying emerging fatigue-related risks is an important function of FRMS safety performance processes, which take a broader system perspective than FRMS processes.
(2) Any newly identified fatigue risk, or combination of existing risks for which current controls are ineffective, should be referred back to the FSAG for evaluation and management using FRMS processes (risk assessment and design and implementation of effective controls and mitigations).
j. Step 4: Identify Changes Affecting the FRMS.
(1) In our dynamic aviation environment, changes are a normal part of flight operations.
They may be driven by external factors (e.g., new regulatory requirements, changing security requirements, or changes to air traffic control) or by internal factors (e.g., management changes or new routes, aircraft, equipment, or procedures). Changes can introduce new fatigue hazards into an operation, which need to be managed. Changes may also reduce the effectiveness of controls and mitigations that have been implemented to manage existing fatigue hazards.
(2) During Step 4, the objective of FRMS SA processes is to identify new hazards that may be a result of change. The current edition of ICAO Annex 6, Part I, Appendix 7 requires that a certificate holder has FRMS SA processes that provide a formal methodology for the management of change. These must include (but are not limited to):
(a) Identification of changes in the operational environment that may affect the FRMS;
(b) Identification of changes within the organization that may affect the FRMS; and
(c) Consideration of available tools that could be used to maintain or improve FRMS performance prior to implementing changes.
(3) A change management process is a documented strategy to proactively identify and manage the safety risks that can accompany significant change. When a change is planned, the following steps can be followed.
Appendix 2
(a) Use the FRM processes to identify fatigue hazards, assess the associated risk, and propose controls and mitigations; and
(b) Obtain appropriate management and/or regulatory signoff that the level of residual risk is acceptable.
(4) During the period of implementation of the change, use the FRMS SA processes to provide periodic feedback to line managers that the FRMS is functioning as intended in the new conditions. Documentation of the change management strategy in relation to fatigue management is also the responsibility of the FSAG. Changes in the operational environment may also necessitate changes in the FRMS itself. Examples include bringing new operations under the scope of the FRMS, collecting different types of data, and adjusting training programs. The FSAG should propose such changes and obtain approval for them from the FAA for implementation.
k. Step 5: Improve Effectiveness of the FRMS.
(1) Ongoing Evaluations. Ongoing evaluation by the FRMS SA processes not only enables the FRMS to be adapted to meet changing operational needs: it also allows the FRMS to continuously improve the management of fatigue risk. In doing so, risk controls that have unintended consequences or that are no longer needed due to changes in the operational or organizational environment can be identified and then modified or eliminated through the FRMS processes. Examples include:
(a) Routine evaluations of facilities, equipment, documentation, and procedures; and
(b) The determination of the need to introduce new processes and procedures to mitigate emerging fatigue-related risks.
(2) Documentation. It is important that changes made to the FRMS are documented by the FSAG so that they are available for internal and regulatory audits.
(3) Assigning Responsibility for FRMS SA Processes.
(a) To deliver effective oversight of the functioning of the FRMS, the FRMS SA processes need to operate in close communication with the FSAG, but with a degree of independence. The objective is to avoid the FSAG from reviewing its own performance.
(b) Primary responsibility for the FRMS SA processes is assigned to a quality assurance (QA) person or team (as appropriate) that is accountable to the executive management team. In smaller operations, responsibility for the FRMS SA processes might reside with an individual rather than a team. This individual may also have a variety of other QA responsibilities.
Appendix 2 FIGURE 2-5. FATIGUE RISK MANAGEMENT AND SAFETY ASSURANCE
PROCESSES
SAFETY ASSURANCE
FRM PROCESSES PROCESSES
1. Identify 1. Collect and Operation(s) where Review FRMS will be Information Applied
2. Collect and Not OK 2. Evaluate Analyze Data Overall FRMS Performance Not OK 3. Identify
3. Identify Emerging Fatigue Hazards Hazards
4. Identify
4. Assess Safety Not OK Changes Affecting FRMS Risk
5.1 Select and Implement Not OK Controls and Mitigations 5. Improve Effectiveness of
5.2 Set Safety Performance FRMS OK Indicators
5.3 Flightcrew FRMS Operations Procedure Development
6. Monitor Effectiveness of Not Controls and Effective Effective mitigations Page 39
Appendix 2 SECTION 7. FRMS AUTHORIZATION PROCESS
1. HOW AN FRMS IS APPLIED. An FRMS is a data-driven system based upon scientific principles that must be evaluated and validated by the FAA for safety and effectiveness.
Typically, a certificate holder will utilize an FRMS authorization as a means to apply an AMOC to a prescriptive rule. Therefore, the certificate holder’s AMOC must be evaluated and validated for safety and effectiveness. The certificate holder must also develop flightcrew member FRMS operations procedures that are supported by the validated data and evaluated and approved by the FAA as demonstrating compliance with the FRMS authorization.
2. DEFINITION OF TERMS. The following terms are applied to FRMS and the approval process:
a. Actigraph. A wristwatch-like device containing an accelerometer to detect movement.
Activity counts are recorded every minute. The patterns of movement can be analyzed using purpose-built software to estimate when the wearer of the actigraph is asleep, and to provide some indication of how restless a sleep period is (i.e., to measure sleep quality). Actigraphs are designed to record continuously for several weeks, so they are valuable tools for monitoring sleep patterns before, during, and after a trip.
b. Biomathematical Model. A computer program (a fatigue model) designed to predict crewmember fatigue levels, based on scientific understanding of the factors contributing to fatigue. All biomathematical models have limitations that need to be understood for their appropriate use in an FRMS and the determination of predicted fatigue levels.
c. Controls. System-level defensive strategies designed to minimize fatigue risk on an ongoing basis.
d. Data. Routine or planned collection of sleep, performance, and alertness measurements.
Other sources of data include flightcrew member reports of fatigue, reports of fatigue-related events, actions taken by the certificate holder to mitigate future fatigue events, and continuous monitoring of performance indicators to determine the overall effectiveness of the FRMS.
e. Data Collection. Application of a scientific methodology during flight operations designed to acquire the information (data) necessary for comparisons between groups or conditions or across time to assess the relative levels of fatigue experienced by flightcrew members both before FRMS operations and after FRMS operations combined with mitigating strategies.
f. Data Package. A description of data collection methods, a compilation of collected data, the results of the analysis of the data with comparisons between groups or conditions or across time, and the FRMS operations procedures that are constructed to support the results of data analysis for FRMS approvals and continuous monitoring.
g. Fatigue Risk Management (FRM). The management of fatigue in a manner appropriate to the level of risk exposure and the nature of the operation in order to minimize the adverse effects of fatigue on the safety of operations.
Appendix 2
h. Fatigue Risk Management (FRM) Processes. FRM processes are one part of the day-to-day operations of the FRMS. They are designed to enable the certificate holder to achieve the safety objectives defined in its FRMS policy, and are managed by the FSAG.
i. Fatigue Safety Action Group (FSAG). A group comprised of representatives from all stakeholder groups that are responsible for coordinating all fatigue management activities in the organization.
j. Flightcrew Member FRMS Operations Procedures. Flightcrew member operations policy and procedures demonstrating compliance during operations where FRMS is applied.
k. Mitigations. System-level interventions designed to reduce a specific identified fatigue risk.
l. Reporting Intervals. Specific timeframes for when the certificate holder is required to provide reports to the FAA regarding data collection, analysis, and the demonstration of the effectiveness of their overall FRMS.
m. Safety Assurance (SA) Processes. SA processes monitor the entire FRMS to check that it is functioning as intended and meeting the safety objectives in the FRMS policy and regulatory requirements. SA processes also identify operational and organizational changes that could potentially affect the FRMS, and identify areas where the safety performance of the FRMS could be improved (continuous improvement).
n. Safety Performance. The level of safety achieved in a risk-controlled environment, measured against a safety level deemed as low as reasonably practicable.
o. Study Design. The definition of the goals or aims of the study, the description of the data collection methods, the groups or conditions that will be studied and compared, the measurements to be taken, the frequency and timing of those measurements, the crewmembers to be studied, the timeframe of the data collection, the methods of analysis, the criteria to be applied to evaluate the findings relative to the goals or aims of the study, and the proposed approach to establish that an FRMS provides an effective AMOC.
3. FRMS APPROVAL PROCESS OVERVIEW. The FRMS authorization process is a systematic and progressive approach to obtain approval of a certificate holder’s proposed FRMS.
This process provides a means for the certificate holder to demonstrate their proposed FRMS provides an AMOC for managing and mitigating fatigue along with a continuous monitoring output.
a. Meet with the FAA. Prior to development of the FRMS application package, the certificate holder should contact the Air Transportation Division to schedule time to discuss their plans for operating under an FRMS. This meeting may be conducted as either an in-person meeting or a teleconference. During this meeting, the FAA will review the approval process with the certificate holder and outline all of the items required for the process. Additionally, this meeting will serve as an opportunity for the certificate holder to ask the FAA any questions relative to any part of the approval process.
Appendix 2
b. Approval Process. The FRMS approval process consists of five phases and nine gates, all of which must be satisfactorily completed in succession. See Figure 2-6, Fatigue Risk Management System Approval Process, for a graphic representation of the FRMS approval process.
(1) The five phases of the approval process are:
• Phase 1: Preapplication, Planning, and Assessment.
• Phase 2: Formal Application.
• Phase 3: Documentation and Data Collection Plan.
• Phase 4: Demonstration and Validation.
• Phase 5: Authorization, Implementation, and Monitoring.
(2) Phase 1 has four gates, Phase 2 has two gates, and Phases 3 through 5 have one gate per phase. Each gate must be satisfactorily completed in succession before the certificate holder may move to the next phase.
(3) The certificate holder is responsible for developing the requirements outlined in each gate and satisfactorily demonstrating the effectiveness of those items. The FAA is responsible for reviewing, evaluating and validating the effectiveness of each phase completed by the certificate holder.
c. Submit Application. Each part 121 certificate holder must develop their draft FRMS application package in a manner acceptable to the FAA for review. When the draft FRMS application package is ready for FAA submission, the certificate holder will electronically submit it to the Air Transportation Division via email at 9-AFS-200-Correspondence@faa.gov and provide its Principal Operations Inspector (POI) with a copy. Upon receipt of the package, the Air Transportation Division will acknowledge receipt to the sender, via automatic email reply.
d. Primary Objectives. Upon satisfactory conclusion, this process will yield four primary objectives relative to the certificate holder’s proposed FRMS, including:
• Validation,
• Authorization,
• Phased implementation, and
• Continuous improvement monitoring.
e. Basic Steps for the Approval of an FRMS Application Package.
(1) The certificate holder develops their plan for an AMOC to a prescriptive rule.
(2) The certificate holder presents their fatigue modeling results or another form of data acceptable to the FAA that supports their proposed AMOC.
(3) The certificate holder presents their study design to verify the AMOC.