13 Figure 1 – Position of COTS IP within a “simplified” design representation flow The availability of a COTS IP does not guarantee that it is suitable to be used in a custom device for aircraft systems. Some COTS IP may have been developed using ED-80/DO-254 and will therefore have the necessary life cycle data to demonstrate satisfaction of ED-80/DO-254. However, most COTS IP are not developed to meet aviation development assurance standards and, therefore, there are risks associated with their use in a custom device for aircraft systems or equipment.
The risks of using COTS IP may include:
• Incomplete or missing documentation/data regarding:
o The behavioral operation of the COTS IP, o How to integrate it into the design;
• Insufficient verification performed by the COTS IP provider;
• Deficient quality of the COTS IP.
The potential for design errors may be increased by the lack of development assurance, and/or by insufficient service experience.
Possible design errors within COTS IP or in the use of COTS IP may lead to a failure mode. Risk factors for these types of errors include:
• Unknown level of rigor of the COTS IP design and verification process;
• Misalignment between the intended usage of the COTS IP by the IP provider and the usage in the custom device by the IP user;
• Incomplete or missing details regarding the detailed operation of the COTS IP;
• Incorrect integration of the COTS IP with the rest of the custom device design;
• Integrator lacking expertise with the function of the IP.
Additionally, the COTS IP user completes the development of the integrated COTS IP up to the physical implementation of the device. The COTS IP user may introduce a design error while completing the physical implementation of the COTS IP because of the user’s incomplete knowledge of the internal design of the COTS IP.
5.11.2 Applicability to COTS IP Section 5.11 is applicable to COTS IP used in a custom device that meets the definition of “commercial-off-the-shelf intellectual property” in the Appendix A Glossary. This scope encompasses digital, analog, and mixed-signal COTS IP.
Note: Analog COTS IP is within the above-mentioned scope, as it could be instantiated within a custom mixed-signal device.
Section 5.11 is applicable to COTS IP contributing to hardware DAL A, B, or C functions.
Section 5.11 is applicable to Soft IP, Firm IP, and Hard IP that are inserted within a custom device by the applicant. However, Section 5.11 does not apply to Hard IP that is embedded in the silicon of an FPGA or a PLD by the FPGA/PLD device manufacturer.
This type of IP is considered to be part of the COTS device, and is covered in Section 6, Use of Commercial Off-the-Shelf Devices.
5.11.3 Development Assurance for COTS IP A COTS IP development assurance approach should be based on the category of the COTS IP (soft, firm, hard) and on the identified risks of failure due to a design error in the COTS IP itself or an error in the way it is used in the custom device.
This section provides objectives addressing development assurance when using COTS IP. These objectives are intended to cover the particular aspects of development when using COTS IP, and are expressed in connection with the custom device development process that follows ED-80/DO-254 and the custom device objectives of this document.
The development aspects related to COTS IP start from the custom device process that captures the allocated requirements for the function that will be performed by the COTS IP. From this entry point, the following aspects provide a basis to define the development assurance objectives for the use of COTS IP:
• Selection of the COTS IP,
• Assessment of the IP provider and the IP data,
• Planning activities, including the verification strategy,
• Definition of the requirements/derived requirements,
• Design integration, implementation, and verification of the COTS IP in the custom device.
5.11.3.1 Selection of the COTS IP to implement the function COTS IP can be available in different forms/source formats and various levels of quality. Some COTS IP may not be acceptable for use in airborne systems. The selection criteria below are intended to address the essential characteristics that are considered a minimum for the use of IP in custom AEH devices.
15 Objective IP-1 The applicant should select a COTS IP that is considered to be an acceptable solution, based on at least the following criteria:
1. The IP is technically suitable for implementing the intended function;
2. The description of the COTS IP architecture or IP design concept provides an understanding of the functionality, modes, and configuration of the IP. The description should also include an understanding of the source format or combination of source formats of the COTS IP;
3. The availability and quality of data and documentation allow the understanding of all aspects of the COTS IP functions, modes, and behavior, and enable the integration and verification of the COTS IP (e.g., datasheets, application notes, user guide, knowledge of errata, etc.);
4. Information exists for the IP user to be able to create the physical implementation of the COTS IP (e.g., synthesis constraints, usage and performance limits, physical implementation, and routing instructions);
5. It can be demonstrated that the COTS IP fulfills its intended function.
5.11.3.2 Assessment of the COTS IP Provider & COTS IP Data Objective IP-2 The applicant should assess the COTS IP provider and the associated data of the COTS IP based on at least the following criteria:
1. The IP provider provides all the information necessary for the integration of the COTS IP within the custom device and to support the implementation of the COTS IP within the device (e.g., synthesis constraints, usage domain, performance limits, physical implementation, and routing instructions);
2. The configurations, selectable options, and scalable modules of the COTS IP design are documented so that the implementation of the COTS IP can be properly managed;
3. The COTS IP has been verified by following a trustworthy and reliable process, and the verification covers the applicant’s specific use case for the COTS IP (including the used scale for scalable IP and the IP functions selected for selectable functions);
4. The known errors and limitations are available to the IP user, and there is a process to provide updated information to the IP user;
5. The COTS IP has service experience data that shows reliable operation for the applicants specific use case for the COTS IP.
The assessment should be documented. The results of the assessment should be submitted for certification.
5.11.3.3 Planning of the Hardware Development Assurance Approach related to COTS IP
5.11.3.3.1 Complementary Development Assurance Objective IP-3 When the IP-2 objective criteria items 1, 2, 4, or 5 cannot be completely met using the IP provider’s data, the applicant should define an appropriate development assurance activity to mitigate the criteria that were not met and address the associated risk of development errors. The development assurance activity should be based on the ED-80/DO-254 objectives.
Note: The results of the assessment of objective IP-2, item 3 are considered in Section 5.11.3.3.2.
5.11.3.3.2 Verification Strategy for COTS IP Functions In addition to the verification of the custom device functions supported by the COTS IP, there is a need to ensure that the aspects related to the COTS IP and its usage are addressed. This section focuses on defining a verification strategy to cover those aspects.
The verification performed by the COTS IP provider typically does not follow the ED-80/DO-254 verification process but may provide some credit to be used for the verification strategy. However, the verification process for COTS IP generally differs from one IP vendor to another, and the level of assurance varies depending on the IP provider’s development practices.
The verification strategy may combine different means to complement the traditional requirements-based testing approach.
Based on the applicant’s assessment of the IP provider and the IP data through objective IP-2, the applicant is expected to establish a verification strategy. The aim of this verification strategy is to cover all three of the following aspects:
• The COTS IP – the purpose is to ensure that the COTS IP is verified, addressing the risk identified from the IP-2, Item 3 objective;
• Its implementation – the purpose is to ensure that the COTS IP still performs its allocated function, and that no design errors have been introduced by the design steps performed by the applicant (e.g., synthesis/place and route);
• Its integration within the custom device – the purpose is to ensure that the COTS IP has been properly connected, configured, and constrained within the custom device.
17 The strategy may accomplish more than one aspect within a common verification step.
This section identifies a general objective for the verification of COTS IP used in a custom device, enabling various verification approaches.
Objective IP-4 The applicant should describe in the hardware verification plan, PHAC, or any related planning document, a verification strategy that should encompass all three of the following aspects:
1. The verification of the COTS IP itself, addressing the risk identified from the IP-2, Item 3 objective,
2. The verification of the COTS IP after the design steps performed by the applicant (e.g., synthesis/place and route),
3. The verification of the integrated COTS IP functions within the custom device.
Note 1: Reliable and trustworthy test data, test cases or procedures from the COTS IP provider may be used as part of the verification strategy to satisfy this objective.
Note 2: If the COTS IP implements functions based on an industry standard, proven standardized test vectors verifying compliance with the standard may be used in the verification strategy of the COTS IP.
Note 3: The verification strategy covers at a minimum the used functions of the COTS IP and ensures that the unused functions are correctly disabled or deactivated and do not interfere with the used functions.
5.11.3.3.3 COTS IP and Planning Aspects The applicant has to define the activities that are needed for the hardware development assurance approach related to COTS IP.
Objective IP-5 The applicant should describe in the PHAC, or any related planning document, a hardware development assurance approach for using the COTS IP that at least includes:
1. Identification of the selected COTS IP (version) and its source format(s) associated with the point(s) in the design flow where the COTS IP is integrated into the custom device,
2. A summary of the COTS IP functions,
3. The development assurance process that the applicant defines to satisfy the objectives of Section 5.11.3,
4. The process related to the design integration and to the usage of the COTS IP in the development process of the custom device,
5. Tool assessment and qualification aspects when the applicant uses a tool to perform design and/or verification steps for the COTS IP.
5.11.3.4 Requirements for COTS IP Function and Validation Custom device requirements typically contain requirements that relate to the function supported by the COTS IP. The granularity of these requirements may be very different depending on the COTS IP function and the visibility of the functions supported by the IP at the custom device level.
Depending on the extent of requirements-based testing as a part of the chosen verification strategy of the COTS IP, the level of detail and the granularity of the AEH custom device requirements may need to be refined to specifically address the COTS IP functions and the implementation of the COTS IP.
In addition, requirements should be captured to encompass all the necessary design details used to connect, configure, and constrain the COTS IP and properly integrate it into the AEH custom device.
Objective IP-6 The requirements related to the allocated COTS IP functions should be captured to an extent commensurate with the verification strategy.
In addition, derived requirements should be captured to cover the following aspects associated with the integration of the COTS IP into the custom device design:
1. COTS IP used functions (including parameters, configuration, selectable aspects),
2. Deactivation or disabling of unused functions,
3. Correct control and use of the COTS IP, in accordance with the data from the COTS IP provider.
When the applicant chooses a verification strategy (see Section 5.11.3.3.2) that solely relies on requirements-based testing, the “extent commensurate with the verification strategy” corresponds to a complete requirement capture of the COTS IP following ED-80/DO-254.
Regarding the validation aspects, the COTS IP requirements should be validated as a part of the validation process of the AEH custom device.
5.11.3.5 Verification The applicant should ensure that the COTS IP is verified as a part of the overall custom device verification process per ED-80/DO-254 and based on the verification strategy for the COTS IP that has been described in the PHAC or a related planning document.
19 For the requirements-based verification part, the applicant should satisfy ED-80/DO-254, Section 6.2 for the verification of the requirements related to the COTS IP (see Section 5.11.3.4 above). This can be performed as a part of the overall custom device process, therefore there is no separate objective.
5.11.3.6 DO-254 Appendix B Considerations When developing a hardware DAL A or B custom device, ED-80/DO-254, Appendix B is applicable.
Code coverage analysis that is recognized as part of elemental analysis (refer to Section 5.7 of this document) might not be possible for the COTS IP part of the design. However, ED-80/DO-254, Appendix B offers other acceptable methods, including safety-specific analysis. The following objective further clarifies the expectations when using safety-specific analysis.
Objective IP-7 For COTS IP used in DAL A or DAL B hardware, the applicant should satisfy ED-80/DO-254, Appendix B.
The applicant may choose safety-specific analysis methods to satisfy Appendix B on the COTS IP function and its integration within the custom device functions. This safety-specific analysis should identify the safetysensitive portions of the COTS IP and the potential for design errors in the COTS IP that could affect the hardware DAL A and DAL B functions in the custom device or system.
For unmitigated aspects of the safety-sensitive portions of the IP, the safety-specific analysis should determine which additional requirements, design features, and verification activities are required for the safe operation of the COTS IP in the custom device.
Any additional requirements, design features, and/or verification activities that result from the analysis should be fed back to the appropriate process.
6 USE OF COMMERCIAL OFF-THE-SHELF DEVICES Applicants are increasingly using COTS electronic devices in aircraft/engines/ propellers/airborne systems, which may have safety implications for the aircraft, engines/propellers, or systems.
Section 6 addresses the use of COTS devices through objectives that support the demonstration of compliance with the applicable airworthiness regulations for hardware aspects of airborne systems and equipment certification when using complex COTS devices. Section 6.2, “Applicability to COTS devices,” enables applicants to identify the COTS devices that are within the scope of Section 6.
20 Note: The term “COTS device” used in this document applies to a semiconductor product that is fully encapsulated in a package. This term does not apply to circuit board assemblies.
6.1 Background COTS devices continue to increase in complexity and are highly configurable. COTS devices provide “off-the-shelf” already developed functions, some of which are highly complex. Their development and production processes undergo a semiconductor industry qualification based on their intended market (consumer, automotive, telecom, etc.). Their usage by the aerospace industry provides additional integration and higher performance capabilities than were possible in the past.
The design data for these COTS devices are usually not available to the COTS user.
Since these devices are generally not developed for airborne system purposes, assurance has not been demonstrated that the rigor of a COTS manufacturer’s development process is commensurate with the aviation safety risks.
ED-80/DO-254 introduces a basis for the development assurance for the use of COTS devices in Section 11.2, “COTS components usage.” This section states that “the use of COTS components will be verified through the overall design process, including the supporting processes.”
Since ED-80/DO-254 was released in the year 2000, the number of functions embedded and integrated in a single COTS device has significantly increased. Functions which were previously split into various components, making the interface between those components accessible for verification, are now embedded within a single chip. While there are clearly some benefits of integrating more functions within a device, the increased level of integration makes it difficult for the user to verify the different hardware functions in the device due to lack of access to the interfaces between functions. Since these devices are more complex and highly configurable than the older separate devices, the risk is greater that the COTS device will not achieve the intended function in particular use cases over the required operating conditions.
Furthermore, some additional assurance is needed because design errors may still be discovered after the COTS device is released to the market, or when an applicant extends the use of the device beyond the manufacturer’s specifications.
6.2 Applicability to COTS devices Section 6 is applicable to digital, hybrid, and mixed-signal COTS devices that contribute to hardware DAL A, B, or C functions. For COTS devices contributing to hardware DAL C functions, a limited set of the objectives of this section will apply.
Section 6 is also applicable to FPGA and PLD devices that embed Hard IP (see definition) in their produced/manufactured silicon, but only for the COTS part of the FPGAs/PLDs.
Section 6.4 only applies to COTS devices that are complex, as determined by the following COTS complexity assessment.
6.3 COTS Complexity Assessment In order to define which COTS devices are complex, the following high-level criteria should be used, considering all functions of the device, including any functions intended to be unused:
A COTS device is complex when the device:
1. Has multiple functional elements that can interact with each other; and
2. Offers a significant number of functional modes; and
3. Offers configurability of the functions, allowing different data/signal flows and different resource sharing within the device.
Or when the device:
4. Contains advanced data processing, advanced switching, or multiple processing elements (e.g. multicore processors, graphics processing, networking, complex bus switching, interconnect fabrics with multiple masters, etc.)
For complex COTS devices, it is impractical to completely verify all possible configurations of the device, and it is difficult to identify all potential failures.
Objective COTS-1 The applicant should assess the complexity of the COTS devices used in the design according to the high-level criteria of Section 6.3, and document the list of relevant devices (see note 1), including the classification rationale, in the PHAC or any related hardware planning document.
Note 1: The applicant is not expected to assess the complete bill of material to satisfy the above objective, but only those devices that are relevant for the classification, including devices that are at the boundary between simple and complex. The resulting classification (simple or complex) for those devices that are at the boundary and those that are definitely complex should be documented.
Note 2: A classification rationale is required for those devices that are at the boundary (meeting a part of the high-level criteria) and are classified as simple.
Some examples of classification are provided in AC 00-72 for illustration.
6.4 Development Assurance for Use of Complex COTS ED-80/DO-254, Section 11.2.1, identifies some electronic component management process items when using a COTS device. ED-80/DO-254, Section 11.2.2, and Section
6.1 of this document, identify some concerns with using a COTS device. The following objectives acknowledge and supplement ED-80/DO-254, Section 11.2 in clarifying how to gain certification credit when using complex COTS devices.
6.4.1 Electronic Component Management Process As stated in ED-80/DO-254, Section 11.2, “the use of an electronic component management process, in conjunction with the design process, provides the basis for COTS components usage.”
22 Objective COTS-2 The applicant should ensure that an electronic component management process exists to address the selection, qualification, and configuration management of COTS devices.
The electronic component management process should also address the access to component data such as the user manual, the datasheet, errata, installation manual, and access to information on changes made by the component manufacturer.
As part of the electronic component management process, for devices contributing to hardware DAL A or B functions, the process for selecting a complex COTS device should consider the maturity of the COTS device and, where risks are identified, they should be appropriately mitigated.
Note: Recognized industry standards describing the principles of electronic component management may be used to support the development of the electronic component management process. See AC 00-72.
6.4.1.1 Using a Device outside Ranges of Values Specified in its Datasheet The device reliability is established by the device manufacturer through the device qualification process. (See definition of “qualification of a device” in the glossary.) ED-80/DO-254, Section 11.2.1, Item 6 mentions that a device is selected based on the technical suitability of the device for the intended application.
In some cases, the applicant may need to use the device outside the specified operating conditions guaranteed by the device manufacturer.
ED-80/DO-254, Section 11.2.1, Item 4 and Item 6 should be addressed when the device is used outside its guaranteed specification. The following objective describes what to achieve when using a device outside the ranges of values specified in its datasheet.
Objective COTS-3 When the complex COTS device is used outside the limits of the device manufacturers specification (such as the recommended operating limits), the applicant should establish the reliability and the technical suitability of the device in the intended application.
6.4.1.2 Considerations when the COTS Device has Embedded Microcode COTS devices may need microcode to execute some hardware functions.
When those functions are used by the applicant, there is a risk if the microcode has not been verified by the device manufacturer during the COTS device qualification, or if the microcode is proposed to be modified by the applicant.
If the microcode is delivered by the device manufacturer, is controlled by the device manufacturer’s configuration management system, and is qualified together with the device by the device manufacturer, it is accepted that the microcode is part of the qualified COTS device. If the microcode is not qualified by the device manufacturer or if it is modified
23 by the applicant, the microcode will not be considered to be part of the qualified COTS device.
Objective COTS-4 If the microcode is not qualified by the device manufacturer or if it is modified by the applicant, the applicant should ensure that a means of compliance for this microcode integrated within the COTS device is proposed by the appropriate process and is commensurate with the usage of the COTS device.
Note: the PHAC (or any other related planning document) should document the existence of the microcode and refer to the process (hardware, software, system) where it is addressed.
6.4.2 COTS Device Malfunctions Some COTS devices may contain errors that may or may not have been detected by the device manufacturer.
Objective COTS-5 The applicant should assess the errata of the COTS device that are relevant to the use of the device in the intended application and identify and verify the means of mitigation for those errata. If the mitigation means is not implemented in hardware, the mitigation means should be fed back to and verified by the appropriate process.
Note: The above objective refers to any mitigation means (such as hardware, software, system, or other means.)
Objective COTS-6 The applicant should identify the failure modes of the used functions of the device and the possible associated common modes, and feed both of these back to the system safety assessment process.
6.4.3 Usage of COTS Devices This section focuses on the usage of complex COTS devices, while Section 7 covers the overall circuit board assembly development process. This Section refers to the term “intended function of the hardware,” which is considered to be defined through the CBA development process.
Complex COTS devices can have multiple functions and many configurations of those functions. The configuration of a device should be managed in order to provide the ability to consistently apply the required configuration settings, to replicate the configuration on another item, and to modify the configuration in a controlled manner, when modification is necessary.
The configuration of the device addresses at least the following topics:
• Used functions (e.g., identification of each function, configuration characteristics, modes of operation),
• Unused functions and the means (internal/external) used to deactivate them,
• Means to control any inadvertent activation of the unused functions, or inadvertent deactivation of the used functions,
• Means to manage device resets,
• Power-on configuration,
• Clocking configuration (e.g., identification of the different clock domains), and
• Operating conditions (e.g., clock frequency, power supply level, temperature, etc.).
Objective COTS-7 The applicant should ensure that the usage of the COTS device has been defined and verified according to the intended function of the hardware. This also includes the hardware-software interface and the hardware to (other) hardware interface.
When a COTS device is used in a hardware DAL A or B function, the applicant should show that unused functions of the COTS device do not compromise the integrity and availability of the COTS device’s used functions.
Note 1: For unused functions of the COTS device, it is recommended that an effective deactivation means is used and verified, when available.
Note 2: Verification should be performed at an appropriate level (hardware, software, equipment).
ED-80/DO-254, Section 10.3.2.2.4 introduces hardware/software (HW/SW) interface data, which can be used as a reference to define the software interface data of the COTS device.
Some additional consideration should be given to the critical configuration settings.
Those are defined as the settings that are deemed necessary by the applicant for the proper usage of the hardware, which, if inadvertently altered, could change the behavior of the COTS device, causing it to no longer fulfill the hardware intended function.
Objective COTS-8 If the complex COTS device contributes to DAL A or B functions, the applicant should develop and verify a means that ensures an appropriate mitigation is specified in the event of any inadvertent alteration of the “critical configuration settings” of the COTS device.
Note: The mitigation means might be defined at the hardware, software, or system level, or a combination of these. The mitigation means may also be defined by the safety assessment process.
7 DEVELOPMENT ASSURANCE OF CIRCUIT BOARD ASSEMBLIES (CBA) This section provides guidance for the development assurance of CBA (a board or a collection of boards).