18 Table 9 – NASA-GB-8719.13 Software Control Categories Software Control Categories Descriptions IA
Partial or total autonomous control of safety-critical functions by software.
Complex system with multiple subsystems, interacting parallel processors, or multiple interfaces.
Some or all safety-critical functions are time critical.
IIA & IIB*
Control of hazard but other safety systems can partially mitigate.
Detects hazards, notifies human operator of need for safety actions.
Moderately complex with few subsystems and/or a few interfaces, no parallel processing.
Some hazard control actions may be time critical but do not exceed time needed for adequate human operator or automated system response.
IIIA & III B* * A = software control of hazard. B = Software generates safety data for human operator
Several mitigating systems prevent hazard if software malfunctions.
Redundant sources of safety-critical information.
Somewhat complex system, limited number of interfaces.
Mitigating systems can respond within any time critical period.
IV
No control over hazardous hardware.
No safety-critical data generated for a human operator.
Simple system with only 2-3 subsystems, limited number of interfaces.
Not time-critical.
6.2.4.5 Option 5: Fault Tolerance.
An applicant could perform a detailed analysis to specify the system’s tolerance to faults in computing system safety items. The system’s tolerance to software faults should be the basis for degree of control assessments at the following levels, with additional levels as appropriate:
1. Zero fault tolerant – a single fault in the computing system safety item could result in an adverse consequence.
2. Single fault tolerant – a fault in the computing system safety item requires a second, independent fault in order to result in an adverse consequence.
3. Dual fault tolerant – a fault in the computing system safety item requires two or more independent faults in order to result in an adverse consequence.
4. Critical informational – a fault in the computing system safety item results in erroneous information that the operator could not readily perceive as erroneous, which can cause an operator to take actions that result in an adverse consequence.
5. Informational – a fault in the computing system safety item results in evidently erroneous information that, if not detected, could cause an operator to take actions that result in an adverse consequence.
6. Non-safety – a fault in the computing system safety item has no potential to result in an adverse consequence.
6.2.4.5.1 A fault tolerance analysis should first identify the conditions under which each level of fault tolerance is acceptable for the proposed operation. For example, fault tolerance acceptability criteria might include “hazards that are catastrophic must be dual fault tolerant to computing system faults,” or “hazards that are critical must be single fault tolerant.” The FAA would evaluate any such conditions as part of the license evaluation. The analysis should identify each computing system safety item and document its fault tolerance, then describe the measures in place to limit risk, linked to tests and analyses that demonstrate their adequacy.
6.2.4.5.2 These categories are not arranged in a hierarchical order; instead, each category represents a degree of control that a computing system safety item could exert on the system through nominal or faulty operation. For example, a computing system safety item that can cause an adverse outcome with a single fault, and for which no other system could credibly prevent an adverse outcome as a result of that fault, is zero fault tolerant.
A computing system safety item that can cause an adverse outcome with a single fault, but that requires a hardware component to fail or a human to make a mistake in order for the software fault to cause an adverse outcome, may be one fault tolerant if the software fault is independent of the second fault. A computing system safety item that could misinform a
20 system operator in a way that the operator could independently and credibly detect and correct would be informational. A computing system safety item that could misinform a system operator in a way that the operator could not independently detect or correct would be critical informational.
7 SAFETY REQUIREMENTS.
7.1 Identification of Safety Requirements.
7.1.1 In accordance with § 450.141(b)(1), applicants are required to develop safety requirements for each computing system safety item identified under § 450.141(a).
Safety requirements specify the implementation of public safety-related functions, capabilities, or attributes in a computing system safety item. Safety requirements are not necessarily obvious, which makes their methodical and explicit identification an important step in understanding a system. In general, an increase in either the severity of hazards, or the computing system’s degree of control over those hazards, will increase the level of rigor applied to that computing system to protect the public. The requirements in § 450.141 are based on this relationship between a computing system’s potential consequences, its degree of control in the causal chain for those consequences, and the rigor applied to development and testing. Software requirements and safety requirements are frequently inherited or derived from system requirements. In addition to the examples listed in Appendix A of this AC, common safety requirements for computing system safety items might include:
Shall use metric units.
Shall compute vacuum instantaneous impact point.
Shall contain a boundary polygon that ensures that the consequence of any reasonably foreseeable failure mode, in any significant period of flight, is no greater than 1 × 10-3 conditional expected casualties.
Shall compare vacuum instantaneous impact point to boundary polygon.
Shall not use flight safety system memory or CPU.
Shall accept and execute “abort” command from ground control at any time.
7.1.2 Deficient requirements are the largest single factor in software and computing system project failure, and deficient requirements have led to a number of software-related aerospace failures and accidents, some of which are described in Appendices C and D.
Faults in requirements can originate from the adoption of requirements that are incomplete, unnecessary, contradictory, unclear, unverifiable, untraceable, incorrect, in conflict with system performance requirements, otherwise poorly written, or undocumented. It is important that operators properly identify and document safety requirements, and per industry standards, ensure that safety requirements are internally consistent and valid at the system level for the resulting computing system to work safely. Applicants should implement a process for managing safety requirements
21 throughout the lifecycle. IEEE 1012-2016 with Corrigendum 1-2017 (see reference [4] of paragraph 3.3 of this AC) provides examples of approaches that can assist in managing requirements. The IEEE 1228-1994 (reference [5]) and NASA GB 8719.13 (reference [10]) also provide methods for managing and analyzing safety requirements.
7.2 Ensuring Safety Requirements are Complete and Correct.
Section 450.141(b)(2) requires that the applicant ensure that safety requirements are complete and correct. The applicant may use standards, such as NASA-STD-8739.9 (see reference [9] of paragraph 3.3 of this AC) or similar formal inspection standards for software, to meet this requirement.
7.2.1 Ensuring Safety Requirements are Complete.
A complete set of safety requirements for a given computing system safety item is the set of requirements that includes all functionality and attributes associated with public safety. To ensure complete safety requirements, an applicant should have a robust interface between system safety and software safety, robust software documentation, and a process to close any gaps in requirements identified by testing or operation of the system.
7.2.2 Ensuring Safety Requirements are Correct.
A set of safety requirements is correct if it specifies exactly and only the functions and attributes intended by the applicant, and when that intention aligns with the requirements for a safe mission. To ensure correct safety requirements, an applicant should have processes for identifying and reviewing safety requirements at the system level and at the software level with coordination between levels, independent validation of safety requirements for safety-critical computing system safety items, and a process to close any gaps in requirements identified by testing or operation of the system.
Reviews of safety requirements need not be single events but can be accomplished progressively as individual computing system safety items mature.
7.3 Implementation and Verification of Safety Requirements.
Section 450.141(b)(3) requires the applicant to implement each safety requirement.
There need not be a separate implementation process for safety requirements; the applicant’s normal process for implementing software requirements is sufficient. This step is required as a bridge between §§ 450.141(b)(2) and 450.141(b)(4), and records produced during this step are important for meeting § 450.141(d)(5).
7.3.1 Independent Validation and Verification.
Applicants are required to validate and verify the implementation of each safety requirement by using a method appropriate for the level of criticality of the computing system safety item, in accordance with § 450.141(b)(4). This regulation requires that when testing a safety-critical computing system safety item, its validation and verification must include testing by a test team independent of the software development division or organization. This validation and verification should take place within the development cycle and contribute iterative findings to the design of the computing system safety item.
7.3.2 Choosing Appropriate Validation and Verification Methods.
Validation and verification methods should be proportional to the level of criticality of the computing system safety item, meaning the method should produce a degree of certainty that leaves little risk of fault in the computing system safety item. One acceptable method of implementing validation and verification in proportion to criticality is provided by MIL-STD-882E (see reference [1] of paragraph 3.3 of this AC), in its discussion of “level of rigor tasks.” Acceptable methods of verification include analyses, formal inspections, and testing. In most cases, testing is the preferred verification approach. These methods are often used in combination, depending on the feasibility of the method and the maturity of the vehicle and operations.
8 DEVELOPMENT PROCESS.
8.1 Development Process Rigor.
Once the applicant has determined the safety requirements needed for each computing system safety item based on the level of criticality of each item, the applicant can assign the appropriate development tasks to its computing system safety items.
Section 450.141(c) requires an applicant to implement and document a development process for computing system safety items that is appropriate for the level of criticality of each computing system safety item. The FAA defines the performance objectives for software development tasks in § 450.141(c) but relies on the applicant to design the tasks that fit its development process while achieving the performance objectives. The tasks required to demonstrate the necessary rigor for a computing system safety item’s criticality should be assigned at the item level. The required tasks should be based on the highest criticality function of a computing system safety item and assigned through the software development process required by § 450.141(c).
8.2 Development Process Requirements.
8.2.1 Responsibility Assignments.
An applicant must define development responsibilities for each task associated with a computing system safety item, in accordance with § 450.141(c)(1). Applicants may achieve this requirement in a wide variety of ways, including documentation of the engineers responsible for computing system safety items, logging of approvals for changes to requirements and software, and definition of software build or release processes. An applicant has met this requirement when the applicant can determine who conducted and approved each step in the development of a computing system safety item retrospectively.
8.2.2 Review and Approval Processes.
In accordance with § 450.141(c)(2), an applicant must define processes for internal review and approval, including reviews that evaluate the implementation of all safety requirements, such that no person approves that person’s own work. This requirement is related to § 450.141(c)(1) because responsibility assignments need to be known in order to determine whether each review was conducted with the appropriate degree of independence. At a minimum, to meet that requirement, reviews and approvals should include reviews of safety requirements and approvals for their implementation, and reviews and approvals of validation and verification evidence.
8.2.3 Training.
Development personnel are required to be trained, qualified, and capable of performing their role, in accordance with § 450.141(c)(3). This training should include learning and practicing operations and procedures that protect the public, including operations and procedures for computing system safety item development. Training can be included as a risk mitigation measure in hazard analyses, as it limits the potential for a range of errors in computing system safety item development. The applicant should develop plans that describe its training process. This training should include, but is not limited to: training for development tools, development methods, installation and testing, hazard analysis approaches, computing system use, and software maintenance. Since training needs and methods are specific to each applicant, FAA’s application evaluation will verify that an applicant’s training process meets the performance objectives in this regulation. In its application review, the FAA does not intend to verify the qualifications of individual development personnel, but rather to verify that the operator has a process in place to put appropriately trained and experienced personnel in public safety roles.
8.2.4 Traceability.
An applicant must define processes that trace requirements to validation and verification evidence, in accordance with § 450.141(c)(4). This traceability enables the applicant to demonstrate that its validation and verification of each safety requirement is sufficient. An applicant should connect the computing system requirements to the analytical and test evidence that demonstrates their implementation in a manner suited to its development process. The connections should be verifiable and human-readable, and the connections for safety requirements should be included in the application materials. FAA does not prescribe the technical methods for making these traceability connections but will evaluate the selected method for possibilities of error.
8.2.5 Configuration Management.
An applicant must define processes for configuration management that specify the content of each released version of a computing system safety item, in accordance with § 450.141(c)(5). The applicant must also comply with § 450.103(c) in its configuration management approach.3 This is the minimum performance requirement for configuration management, but an applicant may need to use other aspects of configuration management (e.g., hardware supply chain requirements or administrative procedures) to achieve the performance requirement. The development process should produce a record of each version of its resultant software or data and each hardware component on which each software or data component is installed. The applicant should retain a record of the system configuration for each computing system safety item in order to demonstrate compliance with the regulatory requirements. Changes to the computing system, especially on safety-critical systems, can have significant impacts on public safety. This configuration management and control process should be in force during the entire life cycle of the program, from initiation of development through retirement, and should include control of project documentation, source code, object code, data, development tools, test tools, environments (both hardware and software), and test cases. Configuration management processes should be ready when each component reaches an initial baseline configuration and should be applied from that point forward. More information on configuration management can be found in paragraph A.Error! Reference source not found. of this AC. As required by §
450.141(c)(5), the applicant must implement a process for configuration management that specifies the content of each released version of a computing system safety item. At a minimum, the process should:
Identify components, subsystems, and systems;
Establish baselines and traceability; and Track changes to the software configuration and system documentation.
8.2.6 Validation and Verification.
Safety analyses generate top- and design-level safety requirements that the applicant uses to meet its system safety goals. These requirements typically result from implementation of mitigation measures, or operational controls to reduce risk. Other sources of safety requirements may include operating practices, standard industry practices, lessons learned, and regulations. Those of the system’s safety requirements that apply to computing systems will need validation and verification consistent with § 450.141. Regardless of the source, effective management of the complete set of safety requirements is an essential component of system safety engineering, and safety requirements validation and verification depends on the integrity of the set of safety requirements.
3 Section 450.103 sets forth the requirements for implementing and documenting a system safety program throughout the lifecycle of a launch or reentry system. This system safety program includes configuration management and control requirements under § 450.103(c).
8.2.6.1 Testing.
In accordance with § 450.141(c)(6), applicants are required to define processes for testing that validation and verify all safety requirements to the extent required by § 450.141(b)(4). Testing should verify the correct functionality of a computing system safety item and validate its performance in the system. Tests should check the implementation of safety requirements. To demonstrate that an applicant sufficiently tested its computing systems based on their criticality, an applicant must trace its validation and verification evidence to the requirements, as required by § 450.141(c)(4). Applicants should describe the components of its testing process, such as the test plan, test cases, test logs, and the procedures for testing computing systems in representative environments. Validation and verification testing in appropriately representative environments may include tests performed on flight-like hardware, Monte Carlo simulations, branch and boundary tests, mathematical validation, fault injection testing, or other methods appropriate for the criticality of the computing system safety item. The degree of testing required by § 450.141(c)(6) will depend on the complexity of the system—that is, the nature of the computing system safety items and their criticality, as identified in § 450.141(b)(4).
Computing system testing is conducted as part of a larger system and vehicle verification program. The system’s responses to computing system safety item faults should be tested whenever the system is part of the mitigation strategy for a fault or failure of a computing system safety item.
8.2.6.2 Test Plan.
A computing system testing process begins with a test plan that demonstrates how the results of testing will be used to validate and verify all safety requirements. An applicant should develop its test plan before verification testing begins. A plan normally prescribes the scope, approach, resources, and schedule of the testing activities. The applicant’s plan should include a description of the test environments, including software tools and hardware test equipment. Tests may include, but are not limited to those contained in Table 5, Testing Types, of this AC.
26 Table 10 – Testing Types Test Comments Unit Demonstrates correct functioning of critical software elements.
Interface Shows that critical computer software units execute together as specified.
System Demonstrates the performance of the software within the overall system throughout the planned mission duration.
Stress Confirms the software will not cause hazards under abnormal circumstances, such as unexpected input values, overload conditions, or off-nominal mission timelines.
Regression Demonstrates changes made to the software did not introduce conditions for new hazards.
8.2.6.3 Test Cases.
The applicant should also define specific test cases with pass and fail criteria. Test cases describe the inputs, predicted results, test conditions, and procedures for conducting the test. The applicant should design test cases to ensure that all safety requirements are covered. These test cases should include scenarios that demonstrate the ability of the software to respond to both nominal and off-nominal inputs and conditions.
Off-nominal and failure test scenarios often come from the hazard analysis.
8.2.6.4 Test Log.
The applicant should record the results of the tests; this is often done in a test log. Anomalies discovered during testing should also be recorded.
ISO/IEC/IEEE 29119-3:2013 provides additional information on test documentation. The recorded test results need to include the version of the software that was tested. See reference [2] of paragraph 3.3 of this AC.
8.2.6.5 Verification Tests.
Testing traditionally has been relied on to verify that computing system requirements have been met and have been implemented correctly. There are several types of tests available, shown in Table 6, and verification testing is normally a combination of many or all types. The applicant should use a combination of verification approaches that are appropriate for its software system (analysis, inspection, and test), including testing to the extent practicable. The applicant should also use proven methods to verify the software requirements, which include, but are not limited, to the verification tests listed in Table 6 of this AC.
27 Table 11 – Verification Tests Test Comments Equivalence partitioning Identifying valid and invalid classes of input conditions. If, for example, an input field calls for values between 1 and 10, inclusive, then a valid equivalence class would be all values between and including 1 and 10. Invalid equivalent classes would be values less than 1 and values greater than 10.
Boundary value Testing at the extremes of an input condition, values close to those extremes, and crossing those boundaries. If, for example, an input field calls for values between 0 and 100 with a precision of 0.01, then test inputs could include 0, 100, -0.01,
100.01, 0.01, and 99.99. Test inputs that exceed the implicit boundaries of allocated memory, such as 129 or -128 for an 8-bit signed integer, should also be included.
Error guessing Using empty or null lists and strings, negative numbers, null characters in a string, and incorrect entry types.
Statement coverage Ensuring that each instruction is executed at least once and instruction execution produces the expected response.
Decision coverage Ensuring that each decision takes on all possible outcomes at least once. For example, assuring that all “if” and “while” statements are evaluated to both true and false.
Function coverage Determining whether each function or procedure was invoked.
Call coverage Verifying that each function call has been completed at least once and produced the expected results.
8.2.7 Previously Developed Software and Computing Systems.
As required by § 450.141(c)(7), an applicant must validate and verify the safety requirements for reused computing system safety items. In addition, an applicant must validate and verify the safety requirements for third-party products, as required by § 450.141(c)(8). Using previously developed computing system safety items can reduce development time, because those components have already undergone design and testing. However, analysis of accidents where software was a contributing factor shows the risks in this approach. Previously-developed computing system safety items include commercial off-the-shelf (COTS) software, government off-the-shelf (GOTS) software, and “reused” software. Although another vendor may have developed the software or product, reducing the risks of using third-party products remains the responsibility of the applicant. These risk reduction efforts should include evaluating the differences
28 between the computing system safety item’s role in the new system and its use in the previous system, including assessment of any identified issues found during use in the previous system and implementation of all preconditions for its use in the new system.
For third-party computing system safety items, risk reduction efforts should include verification of compliance with the developer’s specified uses for third-party products and verification of safety requirements for its use in the system.
8.3 Development Process Considerations.
The rest of this chapter outlines considerations that may guide applicants in formulating development processes that satisfy § 450.141(c) and, more generally, the performancebased requirements of § 450.141.
8.3.1 Analysis.
Analyses to verify that the software requirements are implemented correctly could include the components described in Table 7 of this AC. Additional information about software analysis methods is available in IEEE 1228-1994 and the Joint Services Software Safety Committee (JSSSC) Software System Safety Handbook. See references [5] and [7] of paragraph 3.3 of this AC.
29 Table 12 – Software Implementation Analysis Analysis Comments Logic Evaluates the sequence of operations represented by the coded program and detects programming errors that might create hazards.
Data Evaluates the data structure and usage in the code to ensure each is defined and used properly by the program. Analysis of the data items used by the program is usually performed in conjunction with logic analysis.
Interface Ensures compatibility of program modules with each other and with external hardware and software.
Constraint Ensures that the program operates within the constraints imposed upon it by requirements, design, and target computer. Constraint analysis is designed to identify these limitations, ensure that the program operates within them, and make sure that all interfaces have been considered for out-of-sequence and erroneous inputs.
Programming style Ensures that all portions of the program follow approved programming guidelines.
Non-critical code Examines portions of the code that are not considered safety-critical code to ensure that they do not cause hazards. As a general rule, safety-critical code could be isolated from non-safety-critical code. The intent of this analysis is to prove that this isolation is complete and that interfaces between safetycritical code and non-safety-critical code do not create hazards.
Timing and sizing Evaluates safety implications of safety-critical requirements that relate to execution time, clock time, and memory allocation.
8.3.2 Development Standards.
The applicant may identify development standards that define the rules and constraints for the development process in accordance with § 450.141(c). The use of development standards can enable uniformly designed and implemented computing system safety items. They can prevent the use of methods that are incompatible with safety requirements. Referencing a standard may produce a compelling rationale for the acceptance of a development process. RCC 319-19 defines a development standard that is sufficient for a flight safety system. An applicant can meet § 450.141(c) by demonstrating compliance with RCC 319-19. See reference [12] of paragraph 3.3 of this AC. Regardless of the standard or its use in the proposed operation, the application could establish an understanding of the parts of each standard that has been adopted by