25 Company ABC will employ a PHA and an Event Tree Analysis (ETA) for identifying the safety-critical systems and events. Additional safety-critical systems will also be derived from company operating practices, standard industry practices, and regulations. Outputs from the identification of safety-critical systems include, but are not limited to, top-level safety requirements as well as safety-critical systems, scenarios, and events (see figure 3.1).
Figure 3.1. Identification of Safety-Critical Systems
Preliminary Hazard Analysis will be started as soon as design work has begun so that safety considerations are used to evaluate design alternatives and trade studies. The PHA is used to identify hazards and aid in establishing safety requirements as early in the program as possible. A common format for the PHA will be used to aid in tracking and transfer of risk reports between Company ABC and associate contractors where necessary. The PHA will serve as the baseline for performing future analyses. See appendix B for details and examples of the risk report format.
The PHA and resulting descriptions of the safety-critical systems and events will be presented at the Company ABC Preliminary Design Review (PDR). Associate contractors are responsible for performing the analyses and submitting these analyses to Company ABC for that portion of the XYZ vehicle design or operation for which they have accepted responsibility. Company ABC will consolidate results and keep them available for program management and FAA review.
3.2 HAZARD ANALYSES AND RISK ASSESSMENT Analyses will be performed to identify hazards, their causes and effects, controls to eliminate or mitigate the hazards, risk assessment, and status of hazard resolution (see figure 3.2). For the XYZ Vehicle Program, the intent is to have a database containing all of the hazards involved with the total system throughout its life cycle. Within this database, individual hazards will be identified as relating to the areas normally covered in the traditional hazard analyses.
Hazard analyses are performed to identify hazards during ground and flight tests. They include the hazards in the equipment, procedures, hardware, and software necessary to complete safe and successful tests in all areas of testing (development, qualification, and acceptance). These types of hazards are included in the database.
Tools: PHL, PHA, ETA, Mishap Data, Industry Guidelines Objectives, approaches, scope, system, and process descriptions Safety-critical scenarios and events Safety-critical systems Top-level safety requirements
Safety-Critical Systems, Events Identification
3.2.1 SUBSYSTEM HAZARD ANALYSIS Immediately after the PDR, Company ABC will perform a FMECA. Company ABC, associate contractors, or both, will perform the analysis on the portion of the system for which they are responsible, in the format outlined in appendix B. This analysis will be based on detailed subsystem design data, PHA results, and safety design requirements for the subsystem. The FMECA must be complete by the Critical Design Review (CDR) to the point that all subsystem failure modes and hazards have been identified, mitigating actions have been planned, and verification requirements for these hazards have been identified. The FMECA and resulting failure modes and effects will be briefed at the CDR and made available to XYZ program management and FAA.
Figure 3.2. Hazard Analysis and Risk Assessment
3.2.2 SYSTEM HAZARD ANALYSIS Company ABC will perform a Fault Tree Analysis (FTA). It will incorporate each company’s FMECA into a system level analysis, with emphasis on system interfaces and interactions. Company ABC and associate contractors will ensure that all hazards discovered since the CDR have been added to their respective FMECA. The FTA will be presented at the First Flight Test Readiness Review. Work on the analysis will continue past this time, however, until all actions required on identified hazards have been completed.
3.2.3 SOFTWARE SAFETY Software and firmware hazard analysis is imbedded in the XYZ vehicle hazard analysis process through the system PHA, FMECA, and FTA. Potential software causes for all hazards are to be addressed on each risk report (see appendix B). Additional or detailed software hazard analysis requirements may be identified through the FMECA and a software-specific PHA or FTA.
3.2.4 RISK ASSESSMENT Risk is a function of the combination of the severity (consequences) and the likelihood (frequency of occurrence) levels illustrated in tables 3.1 and 3.2. The risk assessment will generally be qualitative and will be performed with respect to risk to public health and safety. Qualitative measures of hazard severity categories are defined in table 3.1, and table 3.2 lists hazard likelihood categories.
Safety-critical scenarios and events Safety-critical systems Failure modes, effects, mitigating measures Design-level safety requirements Subsystem Hazard Analysis, Risk Assessment Tools: PHA, ETA, FTATools: FMECA, Software Hazard Analysis, Operating & Support Hazard Analysis System Hazard Analysis, Risk Assessment
3.2.5 RISK ACCEPTABILITY CRITERIA The acceptability of risk will be determined for individual hazards by comparing the Hazard Risk Index (HRI) with the HRI acceptability criteria. The HRI is a number from 1 to 20, which ranks the risk, with 1 representing the maximum risk. Hazards with the lowest HRIs will receive priority for corrective action.
The matrix in figure 3.3 shows HRI values and acceptability criteria corresponding to the qualitative severity and likelihood categories.
Table 3.1. Hazard Severity
DESCRIPTION CATEGORY MISHAP DEFINITION Catastrophic I Death to uninvolved public or safety-critical system loss.
Critical II Severe injury or illness to the uninvolved public, or major safety-critical system damage.
Marginal III Minor injury, illness, or safety-critical system damage.
Negligible IV Less than minor injury, illness, or safetycritical system damage.
Table 3.2. Hazard Likelihood
DESCRIPTION LEVEL INDIVIDUAL ITEM Frequent (X> 10-1) A Likely to occur often in the life of an item, with a probability of occurrence greater than 10-1 in any one mission.
Probable (10-1 >X> 10-2) B Will occur several times in the life of an item, with a probability of occurrence less than 10-1 but greater than 10-2 in any one mission.
Occasional (10-2 >X>10-3) C Likely to occur sometime in the life of an item, with a probability of occurrence less than 10-2 but greater than 10-3 in any one mission.
Remote (10-3 >X>10-6) D Unlikely but possible to occur in the life of an item, with a probability of occurrence less than 10-3 but greater than 10-6 in any one mission.
Improbable (10-6 >X) E So unlikely, it can be assumed occurrence may not be experienced, with a probability of occurrence less than 10-6 in any one mission.
28 Severity
Frequency
Catastrophic I
Critical II
Marginal III
Negligible IV Frequent (A) 1 3 7 13 Probable (B) 2 5 9 16 Occasional (C) 4 6 11 18 Remote (D) 8 10 14 19 Improbable (E) 12 15 17 20
Level Index Hazard Risk Acceptability Criteria High 1 - 6 Corrective/controlling actions must be taken to reduce the hazard severity below “II” or reduce the likelihood of occurrence below “C”.
Medium 7 - 10 If not controlled, the risk must be accepted by Program Management and FAA.
Low 11 - 20 Project Management decides on actions, if any.
Figure 3.3. Hazard Risk Index Matrix
3.2.6 RISK MITIGATION MEASURES To reduce the risk to the uninvolved public during conduct of operations, Company ABC will implement additional risk mitigation measures, including performing a flight hazard area analysis and using a flight safety system (FSS). This system will be designed to limit or restrict hazards by initiating and accomplishing a controlled ending to flight, thereby preventing the vehicle from reaching a populated area in the event of a failure. The XYZ vehicle will use a thrust termination system as the FSS to end the flight whenever the vehicle strays outside of a predefined envelope. By identifying specific safety-critical failure modes, timelines of safety-critical events, consequences, and risks to the public, the system safety process will determine the circumstances when an FSS is necessary as a mitigation measure.
The XYZ vehicle flight hazard area analysis will identify any regions of land, sea, or air that must be monitored, publicized, controlled, or evacuated to control risks from debris impact hazards. This analysis will establish the ship and aircraft hazard areas for Notices to Mariners and Notices to Airmen. In addition, the system safety process will identify when the public is potentially at risk based on safety-critical failure modes and events and if a flight hazard area analysis is necessary. This analysis will account for the following items:
• Regions of land, sea, and air potentially exposed to debris resulting from normal flight events and from potential malfunctions.
• Waterborne vessels or aircraft exposed to debris from events resulting from potential normal or abnormal flight events, including vehicle malfunction.
• Operational controls implemented to control risk to the public from debris hazards.
• Debris identified from debris analysis.
• Vehicle trajectory dispersion effects in the surface impact domain.
3.3 VALIDATION AND VERIFICATION The V&V process shall apply to all levels of safety requirements captured throughout the life cycle of the system. The process will determine that the correct safety-critical system is being built (validation), as represented by a correct set of safety-critical requirements, and that XYZ vehicle design solutions meet all the safety-requirements (verification). The V&V process used by Company ABC to manage and document the set of safety requirements is in alignment with the FAA/AST Guide to Reusable Launch Vehicle Safety Validation and Verification Planning, Version 1.0, September 2003.
Figure 3.4. Validation and Verification
3.4 OPERATIONS AND MAINTENANCE The Operations and Maintenance (O&M) process applies to all levels of safety requirements captured throughout the life cycle of the system. This process will determine the O&M programs and safety data necessary to ensure that safety is maintained before and during each flight of the XYZ vehicle. Company ABC will provide a maintenance program that addresses safety-critical system and subsystem maintenance and refurbishment considerations. This process will identify and track limited life items throughout the life cycle of the XYZ Vehicle Program.
The operations program will address operating procedures and processes for conducting safe operations over the lifetime of the XYZ Vehicle Program. The O&M process implemented by Company ABC is in alignment with the FAA/AST Guide to Commercial Reusable Launch Vehicle Operations and Maintenance, Version 1.0, May 2005.
3.5 ANOMALY REPORTING AND CORRECTIVE ACTION Company ABC will establish an anomaly reporting and corrective action system to support anomaly identification, tracking, reporting, and disposition. This system will allow Company ABC to implement design improvements and corrections as part of the design process. Data collected will support tracking the root cause of problems. The corrective action system will continue to be used to support upgrading system O&M performance. Anomalies with potential safety implications occurring in a system or subsystem of a Design-level safety requirements Reduction of risk to acceptable levels Top-level safety requirements
Validation and Verification
30 launch vehicle during verification activities, prelaunch processing, launch, flight, or post-launch processing will be reported to the FAA/AST for review. Company ABC will notify the FAA/AST of all anomaly reviews, meetings, or both. In addition, Company ABC will provide the FAA/AST copies of the briefings, reports, meeting minutes, and actions identified and taken to address these anomalies.
3.6 SAFETY RISK TRACKING All hazards discovered, including those resolved informally, shall be recorded on a risk report (see appendix B). To ensure that the actions taken to control the risks are acceptable to system safety and program management, the risk reports will be used to track identified hazards. The signatures of the originator, Company ABC System Safety Manager, and Company ABC Program Manager will be required to dispose of all risk reports.
3.7 ADDITIONAL CONSIDERATIONS Company ABC has listed additional scheduled safety-related tasks and activities that have system safety imbedded within them. Table 3.3 lists the tasks and activities usually performed by other organizations or disciplines, including associate contractors, which apply most directly to the system safety tasks.
Table 3.3. Additional Safety-Related Tasks
ORGANIZATION/ DISCIPLINE/
ACTIVITIES
SAFETY RELATED TASK System Safety Data Documentation of safety data. Company ABC will document safety data. Examples of this data include anomaly reports, procedures, test data, hazard reports, lessons learned, and associate contractor safety effort deliverables. The system safety process will include the means for describing, collecting, and processing this safety data.
Configuration Management Tracking of changes. Development of any system requires that changes be made throughout the life cycle. Changes to the vehicle, especially on safety-critical systems, can have significant impacts on public safety. At a minimum, Company ABC will implement a configuration control process to identify components, subsystems, and systems; establish baselines and traceability; and track changes to the configuration and system safety documentation.
Quality Assurance (QA) Establishment of a quality assurance program. Company ABC will establish a QA program. This program will be suitable to Company ABC system safety objectives; QA program management; vehicle and hardware acceptance; QA engineering; supplier selection, quality surveillance, and audits; production quality performance and evaluation;
verification; configuration assurance; calibration; metrology;
test assurance; material, nonconformance, and process reviews; and corrective action identification, quality data collection, and reporting.
Reliability Performance of reliability analyses. Company ABC will perform reliability analysis, predictions, critical item
ORGANIZATION/ DISCIPLINE/
ACTIVITIES
SAFETY RELATED TASK identification, testing and demonstration, and FMECA.
Results of these analyzes will be used to develop parts selection and derating criteria and to identify and resolve reliability issues on safety-critical systems.
Training Development of training plans. Designing safety into the system requires that personnel involved in system development, production, and operation understand and practice operations and procedures that protect the public safety. Training can help ensure that personnel can produce a safe system or operation.
Training used as a risk mitigation measure will be identified as such. Training is a critical element in helping ensure the safety of the public.
APPENDIX B
RISK REPORT FORMAT
This appendix provides a format for reporting all hazards associated with the operation of reusable launch, reentry, or both vehicles. The format is designed to allow for ease of hazard tracking, flexibility in analysis presentation, and reduction of time and effort in report preparation. Such reports should include the following information:
• Report number
• Company name
• Hazard title
• Analysis phase
• Initial date of the report
• Closeout date of the report
• Originator
• Hazard status (initial and closeout frequency, severity, and Hazard Risk Index)
• Mission phase
• Risk level (frequency and severity)
• Hazard Risk Index acceptability criteria
• System
• Subsystem
• Controlling documents
• System function
• Hazard description
• Failure modes and causes
• Effects
• Mitigation of hazard and controls
• Verification methods and documentation
• Closure signatures (originator, safety system manager, and program manager) The FAA does not generate or require use of specific forms for submission of risk reports. Elements from this format have been used to show a simple form that a company might create for its risk reporting functions. Although this appendix demonstrates use of the format as a form, other approaches to presenting this information are acceptable. Regardless of the approach used in presenting the needed information, closure of a risk report should have the signature of the report originator, system safety manager, and program manager.
Examples of risk reports Company ABC might submit as part of an RLV license application presented in this appendix include two examples from a Failure Modes, Effects, and Criticality Analysis and one example from a Preliminary Hazard Analysis. In addition, a blank copy of a sample risk report form is provided.
RISK REPORT NO.: 1.0 Company: Company ABC, Inc.
Hazard Title: Primary Load Structural Failure: Vehicle Airframe Failure Analysis Phase: Failure Modes, Effects, and Criticality Analysis (FMECA)
Initial Closeout Date 12/ 5/ 00 4/15/02 Originator John Doe, XYZ
Hazard Status:
Frequency* Severity* HRI* Initial D I 8 Closeout D I 8 *See FAA Advisory Circular 431.35-2A Prelaunch to Launch X Descent X Ascent Landing/Recovery Mission Phase:
X On-Orbit to Reenty
Post-launch Risk Level: (See FAA Advisory Circular 431.35-2A for explanation of risk level.)
Severity
Frequency
Catastrophic I
Critical II
Marginal III
Negligible IV Frequent (A) 1 3 7 13 Probable (B) 2 5 9 16 Occasional (C) 4 6 11 18 Remote (D) 8 10 14 19 Improbable (E) 12 15 17 20
Level Index Hazard Risk Index Acceptability Criteria High (Red) 1 - 6 Corrective/controlling actions must be taken to reduce the hazard severity below “II” or reduce the likelihood of occurrence below “C”.
Medium (Yellow) 7 - 10 If not controlled, must be presented to Program Management and FAA as accepted risk.
Low (Green) 11 - 20 Project Management decides on actions, if any.
34 RISK REPORT NO.: 1.0 System:
Main Structure: Vehicle Airframe
Subsystem:
Wings, booms, stabilizers, fuselage Controlling Doc.
(Name, #, etc.):
Systems Requirements Database XXX Function:
Primary load-carrying structure
Hazard Description:
Overload failure of primary structural load path—most critical at or near maximum dynamic pressure or maximum acceleration—consider wings, booms, stabilizers.
Failure Mode(s)/Cause(s):
Structural failure caused by aerodynamic loading, inadequate maintenance, or improper flight test procedures Effect:
Loss of control, loss of vehicle—show by simulation likely worst-case trajectories of intact vehicle
Mitigation of Hazard/Controls:
Structural analysis showing minimum 1.5 factor of safety on design limit loads on critical structures; see Vehicle Design Document (VDD) for specific results. Pre- and post-flight vehicle inspections in accordance with approved maintenance plan and specific checklists. Static proof tests of specific structural elements.
Verification Method(s) & Documentation:
(1) Loads are defined in VDD, sec. 1.0, Jan 2004 Structural Analysis PowerPoint package (ref 39). (2) Crew chief Pre- & Post Checklist & Systems Requirements Database Inspection Plan. (3). VDD, sec. 1.0; proof test plans for critical structures (most); flight load comparison plots between maximum test loads experienced to date and the maximum expected flight loads. (4) Process described and representative plots at the Jan 2002 Technical Interchange Meeting (TIM). See TIM minutes (ref. 14).
Additional Remarks:
Closure Signatures Originator: _____________________ System Safety Manager: _____________________ Program Manager: _____________________
RISK REPORT NO.: 2.1 Company: Company ABC, Inc.
Hazard Title: Uncontrolled Crash: Propulsion Shutdown System Analysis Phase: Safety-Critical System Identification: Preliminary Hazard Analysis (PHA)
Initial Closeout Date 12/ 5/ 01 4/15/02 Originator John Doe, XYZ
Hazard Status:
Frequency* Severity* HRI* Initial B II 5 Closeout D II 10 *See FAA Advisory Circular 431.35-2A Prelaunch to Launch Descent X Ascent Landing/Recovery Mission Phase:
On-Orbit to Reenty
Post-launch Risk Level: (See FAA Advisory Circular 431.35-2A for explanation of risk level.)
Severity
Frequency
Catastrophic I
Critical II
Marginal III
Negligible IV Frequent (A) 1 3 7 13 Probable (B) 2 5 9 16 Occasional (C) 4 6 11 18 Remote (D) 8 10 14 19 Improbable (E) 12 15 17 20
Level Index Hazard Risk Index Acceptability Criteria High (Red) 1 - 6 Corrective/controlling actions must be taken to reduce the hazard severity below “II” or reduce the likelihood of occurrence below “C”.
Medium (Yellow) 7 - 10 If not controlled, must be presented to Program Management and FAA as accepted risk.
Low (Green) 11 - 20 Project Management decides on actions, if any.
36 RISK REPORT NO.: 2.1 System:
Vehicle propulsion system
Subsystem:
Propulsion shutdown system Controlling Doc.
(Name, #, etc.):
Vehicle Operations Handbook Function:
The propulsion shutdown system terminates thrust during nominal and off-nominal operations.
Hazard Description:
Component failure, software faults, human error, environmental impacts, or any combination thereof, could lead to the inability to shut down the engine with the potential for an uncontrolled crash in populated areas.
Failure Mode(s)/Cause(s):
Not applicable to this analysis phase. Preliminary Hazard Analysis does not normally include failure modes and causes.
Effect:
Not applicable to this analysis phase. Preliminary Hazard Analysis includes the effect in the hazard description.
Mitigation of Hazard/Controls:
(1) Use redundant engine shutdown systems with different methods of operation, such as an automated system (valve with software-driven controller) and a manual system (manually operated valve). (2) Incorporate operating and training procedures to operate manual shutdown system.
Verification Method(s) & Documentation:
Not applicable to this analysis phase.
Additional Remarks:
See risk report No. 3.2 for description of one safety-critical system failure mode for this hazard.
Closure Signatures Originator: _____________________ System Safety Manager: _____________________ Program Manager: _____________________