30 the applicant, how each part of each standard is used in the proposed development and operation, and how each part of each standard supports public safety. Development standards include requirements, design, coding, and safety standards, as follows:
Requirements standards may include methods for developing requirements and a description of how the requirements flow down to coding.
Design standards may include restrictions on the use of scheduling and interrupts, specification of usable code libraries, or rules for conditional branches to reduce complexity.
Coding standards may include specifications for the programming language;
naming conventions for modules, variables, and constants; and constraints on the use of tools.
Safety standards may include approaches for analyzing risk and classifying hazards, such as MIL-STD-882 or GEIA-STD-0010A. See references [1] and [13] of paragraph 3.3 of this AC.
8.3.3 Quality Assurance.
Quality assurance may support the achievement of the performance objectives in § 450.141. Quality assurance verifies that the objectives and requirements of the software system safety program are satisfied and confirms that deficiencies are detected, evaluated, tracked, and resolved. An acceptable quality assurance function should include audits and inspections of elements and processes, such as plans, standards, and problem tracking and configuration management systems. In addition, the quality assurance function could evaluate the validity of system safety data. NASA Software Assurance and Software Safety Standard (NASA-STD-8739.8), and NASA Software Engineering and Assurance Handbook (NASA-HDBK-2203) provide examples of acceptable software quality assurance methods. See references [8] and [11] of paragraph
3.3 of this AC.
8.3.4 Formal Inspections.
Formal inspections are well thought out technical reviews that provide a structured way to find and eliminate defects in documentation products, ranging from a requirements document to the actual source code. These inspections differ from informal reviews or walkthroughs since there are specified steps to be taken and roles assigned to individual reviewers. As required by § 450.141(c)(2), a process must be defined for internal review and approval such that no person approves their own work. Further information regarding formal inspections can be found in NASA-STD-8739.9, Software Formal Inspections Standard. See reference [9] of paragraph 3.3 of this AC.
8.3.5 Anomaly Reports.
To help prevent recurrence of computing system safety-related anomalies, the applicant could develop a standardized process to document anomalies, analyze the root cause, and determine corrective actions. Software anomaly reports (also known as problem reports) are a means to identify and record:
Computing system anomalous behavior and its resolution, including failure to respond properly to nominal and off-nominal conditions.
Development process non-compliance with software, requirements, plans, and standards, including improperly implemented safety measures.
Deficiencies in documentation and safety data, including invalid requirements.
8.3.6 Maintenance and Repair of Computing System Hardware.
While software does not wear out, maintenance engineering ensures that systems and subsystems will remain at the design safety level by minimizing computing system hardware wear-out failures through replacement of failed items and surveillance for possible degradation due to environments. Maintenance engineering personnel also participate in analyzing the safety implications of proposed maintenance procedures on the ground and in flight. Therefore, the applicant could perform activities to aid maintenance and repair of computing system hardware.
8.3.7 Maintenance of Computing System Software.
Software maintenance differs from hardware maintenance because software does not wear out or degrade in the same way that hardware does. However, software maintenance corrects defects; adds or removes features and capabilities; compensates for or adapts to hardware changes, wear-out, or failure; and accommodates changes in other computing system safety items or system components. Changes to both the hardware and software after deployment can produce computing system anomalies. An applicant could identify a process for verifying the integrity of the safety-critical software and computing systems after deployment. Examples of such verification methods include the use of checksums and parity bit checks to ensure proper data transfer, built-in or external measures for evaluating the software and its data, inspections to detect unauthorized modification of the software or its data, and regression testing.
8.3.8 Building Maintainable Software.
Because software changes can be expected over the lifecycle of the product, an applicant could build maintainable software to facilitate those changes and reduce the likelihood of introducing new hazards. NASA-GB-8719.13 provides additional information on software maintainability. See reference [10] of paragraph 3.3 of this AC.
Considerations for building maintainable software include:
Planning early for expected changes.
Using strong configuration management practices.
32 Using modular design, where appropriate, to minimize the overall impact of changes.
Implementing naming conventions for variables to improve code readability.
Using comment and style coding standards to improve code readability.
Implementing documentation standards to make important information easy to find.
Using a standardized set of development tools to reduce the chance of introducing errors in code changes.
Assuring that design and verification documentation, such as regression tests and test cases, are updated and maintained.
33 9 APPLICATION MATERIALS.
This chapter provides guidance on satisfying the application requirements set forth in § 450.141(d). In accordance with § 450.141(d), an application must include:
Descriptions of all computing system safety items involved in the proposed operations;
All safety requirements for each computing system safety item;
Documentation of the software development process that meets § 450.141(c);
Evidence of the execution of appropriate development processes for each computing system safety item; and Evidence of the implementation of each safety requirement.
9.1 Computing System Safety Items.
An applicant must identify and describe all computing system safety items involved in its proposed operation, in accordance with § 450.141(d)(1). These descriptions should include the severity of hazards associated with each computing system safety item and the computing system safety item’s level of criticality regarding each hazard. The descriptions of computing system safety items and their criticality allow the applicant to know, and the FAA to verify, how each computing system safety item influences public safety when combined with the other requirements in § 450.141.
9.2 Safety Requirements.
An applicant must identify the safety requirements for each computing system safety item, in accordance with § 450.141(d)(2). Although § 450.141(d)(2) only requires applicants to identify safety requirements, an applicant may, and is encouraged to, submit all computing system requirements for its computing system safety items in order to effectively convey the software’s intended functionality to the FAA. State diagrams, user manuals, flow charts, and other documents are helpful to communicate computing system requirements.
9.3 Development Process.
Section § 450.141(d)(3) requires applicants to submit documentation of the development process that meets § 450.141(c), including the minimum attributes required by the regulation. Similarly, § 450.141(d)(4) requires an applicant to provide evidence of the execution of the appropriate development process for each computing system safety item. An applicant must demonstrate its development process for each computing system safety item in accordance with § 450.141(d)(3), and each development process must meet § 450.141(c). If a development process contains more than one distinct set of development process requirements, such as a set of development requirements for applicant-developed computing system safety items and another set for third-party products, then the applicant should specify which set was executed for each computing system safety item. The documentation required of an applicant to meet these requirements will vary among applications, depending on the complexity of the computing system safety item, but will generally include some combination of audit
34 results, milestone results, and outputs of automated code checking tools. Substantive guidance on demonstrating compliance with § 450.141(c) is provided in paragraph 8.2 of this AC. A software hazard analysis is often needed to ensure that test cases address all necessary nominal and off-nominal conditions, and it expands in scope throughout the development process. Although the test cases trace to requirements, the hazard analysis validates the content of the tests.
9.4 Providing Evidence of the Implementation of Each Safety Requirement.
Section 450.141(d)(5) requires the submission of evidence of the implementation of each safety requirement. An applicant should submit the combination of test descriptions, test plans, test outputs, and analyses that verifies that each computing system safety item meets each applicable safety requirement. Applicants that plan repeated missions with identical computing system safety items may define a standard set of test reports.
9.5 Examples of Lessons Learned from Previous Space Vehicle Failures.
Appendices C and D of this AC provide examples of lessons learned from previous system failures in which software and its associated computing system hardware have played a significant role. The information provided in these appendices is intended to provide an understanding of the types of failures that have been observed in software and computing systems and provide lessons learned for the design of future systems.
Appendix A
35 APPENDIX A. GENERIC COMPUTING SYSTEM SAFETY REQUIREMENTS.
This appendix provides generic computing system safety requirements that an applicant may use in the design and development of software and computing systems. These generic design requirements represent best practices used in the aerospace community, and are offered here as examples of safety requirements for any application under § 450.141. Additional information and sample requirements can be found in the following references:
FAA System Safety Handbook (2002) Joint Services Software Safety Committee Software System Safety Handbook (1999) NASA Software Safety Guidebook (2004) Range Safety User Requirements Manual: Air Force Space Command Range Safety Policies and Procedures (2004) Sample General Computing System Requirements.4 A.1.1 Computer systems should be validated for operation in the intended use and environment. Such validation should include testing under operational conditions and environments.
A.1.2 Under maximum system loads, CPU throughput should not exceed 80 percent of its design value.
A.1.3 Computer system architecture should be single fault tolerant. No single software fault or output should initiate a hazardous operation, cause a critical accident, or cause a catastrophic accident.
A.1.4 Safety-critical computer system flight architecture that will be exposed to cosmic radiation should protect against CPU single event upset and other single event effects. A single event upset occurs when an energetic particle travels through a transistor substrate and causes electrical signals within a component.
A.1.5 Sensitive components of computer systems should be protected against the harmful effects of electromagnetic radiation, electrostatic discharges, or both.
A.1.6 The computer system should verify periodically that safety-critical computer hardware and software safety-critical functions, including safety data transmission, operate correctly.
4 The requirements in this appendix are computing system design requirements as opposed to regulatory requirements.
Appendix A 36 A.1.7 The computer system should verify periodically the validity of real-time software safety data, where applicable.
A.1.8 Software should process the necessary commands within the time-to-criticality of a hazardous event.
A.1.9 Memory allocation should occur only at initialization.
A.1.10 If the system begins to use areas of memory that are not part of the valid program code, the system should revert to a safe state.
A.1.11 Memory partitions, such as RAM, should be cleared before loading software.
A.1.12 Prerequisite conditions for the safe execution of an identified hazardous command should exist before starting the command. Examples of these conditions include correct mode, correct configuration, component availability, proper sequence, and parameters in range. If prerequisite conditions have not been met, the software should reject the command and alert the crew, ground operators, or controlling executive.
A.1.13 Provisions to protect the accessibility of memory region instructions, data dedicated to critical software functions, or both, should exist.
A.1.14 Software should provide proper sequencing, including timing, of safety-critical commands.
A.1.15 Where practical, software safety-critical functions should be performed on a standalone computer. If that is not practical, software safety-critical functions should be isolated to the maximum extent practical from non-critical functions.
A.1.16 Documentation describing the software and computing system should be developed and maintained to facilitate maintenance of the software.
A.1.17 The software should be annotated, designed, and documented for ease of analysis, maintenance, and testing of future changes to the software.
A.1.18 Interrupt priorities and responses should be specifically defined, documented, analyzed, and implemented for safety impact.
A.1.19 Critical software design and code should be structured to enhance comprehension of decision logic.
A.1.20 Software code should be modular in an effort to reduce logic errors and improve logic error detection and correction functions.
A.1.21 The software should be initiated and terminated in a safe state.
Appendix A 37 A.1.22 Critical hardware controlled by software should be initialized to a known safe state.
A.1.23 NASA Software Engineering Requirements (NPR 7150.2C) requirement SWE-134 provides an alternative set of requirements that is helpful in determining a full set of software requirements for safety critical applications.
A.2 Computing System Power.
A.2.1 Computer systems should be powered up, restarted, and shutdown in a safe state.
A.2.2 A computer system should not enter a hazardous state as a result of an intermittent power transient or fluctuation.
A.2.3 If a single failure of primary power to a computer system or computer system component occurs, then that system or some cooperating system should take action automatically to transition to a stable state.
A.2.4 Software used to power up safety-critical systems should power up the required systems in a safe state.
Anomaly and Failure Detection.
A.3.1 Single event system failures should be protected against by employing mitigating approaches, as appropriate, such as redundancy, error-correcting memory, and voting between parallel CPUs.
A.3.2 Before initiating hazardous operations, computer systems should perform checks to ensure that they are in a safe state and functioning properly. Examples include checking safety-critical circuits, components, inhibits, interlocks, exception limits, safing logic, memory integrity, and program loads.
A.3.3 Failure of software safety-critical functions should be detected, isolated, and recovered from in a manner that prevents catastrophic and critical hazardous events from occurring.
A.3.4 Software should provide error handling to support software safety-critical functions. The following hazardous conditions and failures, including those from multiple sources, should be detected:
Input errors. Data or sequences of data passed to software modules, either by human input, other software modules, or environmental sensors, that are outside a specified range for safe operation.
Output errors. Data output from software modules that are outside a specified range for safe operation.
Appendix A 38 Timing errors. The state when software-timed events do not happen according to specification.
Data transmission errors.
Memory integrity loss.
Data rate errors. Greater than allowed safe input data rates.
Software exceptions. “Divide by zero” or “file not found.”
Message errors. Data transfer messages corrupted or not in the proper format.
Logic errors. Inadvertent instruction jumps.
A.3.5 Watchdog timers or similar devices should be used to ensure that the microprocessor or computer operates properly. For example, a watchdog timer should be used to verify events within an expected time budget or to ensure that cyclic processing loops complete within acceptable time constraints.
A.3.6 Watchdog timers or similar devices should be designed, so the software cannot enter an inner loop and reset the timer or similar device as part of that loop sequence.
Anomaly and Failure Response.
A.4.1 Software should provide fault containment mechanisms to prevent error propagation across replaceable unit interfaces.
A.4.2 All anomalies, software faults, hardware failures, and configuration inconsistencies should be reported to the appropriate system operator, safety official, or both, consoles in real time, prioritized as to severity, and logged to an audit file. The display should:
Distinguish between read and unread anomaly alerts;
Support reporting multiple anomalies;
Distinguish between anomaly alerts for which corrective action has been taken and those that still require attention; and Distinguish between routine and safety-critical alerts.
A.4.3 Upon detecting anomalies or failures, the software should:
Remain in or revert to a safe state;
Provide provisions for safing the hardware subsystems under the control of the software;
Reject erroneous input; and
Appendix A 39 Ensure the logging of all detected software safety-critical function system errors.
A.4.4 Upon detecting a failure during vehicle processing, the software should maintain the Flight Safety System (FSS) in its current state and meet the requirements in paragraph A.4.3 of this appendix. The software should maintain the FSS in the safe state. After the FSS is readied, the software should retain the FSS in the readied state. When the FSS receiver is on internal power, the software should maintain the FSS receiver on internal power. During flight, all detected FSS-related system errors should be transmitted to the safety official.
A.4.5 Details of each anomaly should be accessible with a single operator action.
A.4.6 Automatic recovery actions taken should be reported to the crew, operator, or controlling executive. There should be no necessary response from crew or ground operators to proceed with the recovery action.
A.4.7 Override commands should require multiple operator actions.
A.4.8 Software that executes hazardous commands should notify the initiating crew, ground operator, or controlling executive upon execution or provide the reason for failure to execute a hazardous command.
A.4.9 Hazardous processes and safing processes with a time-to-criticality such that timely human intervention may not be available should be automated. Such processes should not require human intervention to begin, accomplish interim tasks, or complete.
A.4.10 The software should notify the crew, ground operators, or controlling executive during or immediately after completing an automated hazardous or safing process.
A.4.11 After correction of erroneous entry, the software should provide positive confirmation of a valid data entry. The software should also provide an indication that the system is functioning properly.
Appendix A 40 Maintenance, Inhibits, and Interlocks.
A.5.1 Systems should include hardware and software interlocks and software controllable inhibits, as necessary, to mitigate hazards when performing maintenance or testing.
A.5.2 Interlocks should be designed to prevent an inadvertent override.
A.5.3 Interlocks that are required to be overridden should not be autonomously controlled by a computer system, unless dictated by a timing requirement.
A.5.4 Interlocks that are required to be overridden and are autonomously controlled by a computer system should be designed to prevent an inadvertent override.
A.5.5 The status of all interlocks should be displayed on the appropriate operator consoles.
A.5.6 An interlock should not be left in an overridden state once the system is restored to operational use.
A.5.7 A positive indication of interlock restoration should be provided and verified on the appropriate operator consoles before restoring a system to its operational state.
A.5.8 Software should make available status of all software controllable inhibits to the crew, ground operators, or controlling executive.
A.5.9 Software should accept and process crew, ground operator, or controlling executive commands to activate and deactivate software controllable inhibits.
A.5.10 Software should provide an independent and unique command to control each software controllable inhibit.
A.5.11 Software should incorporate the ability to identify and display the status of each software inhibit associated with hazardous commands.
A.5.12 Software should make available the current status on software inhibits associated with hazardous commands to the crew, ground operators, or controlling executive.
A.5.13 All software inhibits associated with a hazardous command should have a unique identifier.
A.5.14 If an automated sequence is already running when a software inhibit associated with a hazardous command is executed, the sequence should complete before the software inhibit is started.
Appendix A 41 A.5.15 Software should have the ability to resume control of an inhibited operation after deactivation of a software inhibit associated with a hazardous command.
A.5.16 The state of software inhibits should remain unchanged after the execution of an override.
Human-Computer Interface.
A.6.1 The system should be designed such that the operator may exit current processing to a known stable state with a single action and have the system revert to a safe state.
A.6.2 Computer systems should minimize the potential for inadvertent actuation of hazardous operations.
A.6.3 Only one operator at a time should control safety-critical computer system functions.
A.6.4 Operator-initiated hazardous functions should require two or more independent operator actions.
A.6.5 Software should provide confirmation to the operator of valid command entries, data entries, or both.
A.6.6 Software should provide feedback to the operator that indicates command receipt and status of the operation commanded.
A.6.7 Software should provide the operator with real-time status reports of operations and system elements.
A.6.8 Error messages should distinguish safety-critical states and errors from nonsafety-critical states and errors.
A.6.9 Error messages should be unambiguous.
A.6.10 Unique error messages should exist for each type of error.
A.6.11 The system should ensure that a single failure or error cannot prevent the operator from taking safing actions.
A.6.12 The system should provide feedback for any software safety-critical function actions not initiated.
A.6.13 Safety-critical commands that require several seconds or longer to process should provide a status indicator to the operator indicating that processing is occurring.