Code Coverage Metrics Recommended by ISO 26262

ISO 26262, the international standard for the functional safety of road vehicles, mandates the measurement of structural coverage. Both at the software unit level as well as the  architectural level. The main goal is the completeness of test cases. As a positive side effect unwanted (and possibly harmful) functionality will be discovered and can be disabled or removed completely.

Various Coverage Levels exist. Learn which one to select below.

Metrics at the Unit Level

Part 6 of the standard covers the product development at the software level. Structural coverage needs to be measured at both unit and architectual level. See sections 9.4.5 and 10.4.6, respectively.

The recommended structural coverage metrics at unit level are:

The choice of one or several of these methods needs to take the applicable ASIL into account. The degree of risk (and controllability) is grouped between A and D. Whereas higher levels require more thorough measures for controlling the risk. The recommendations applying to your concrete ASIL can be looked up in Table 12 of Part 6. Even without spending time considering the individual situation one can simply follow the rule: statement coverage is implied by (or inferior to) branch coverage which again is implied by MC/DC. In other words: once full MC/DC coverage is achieved one has already achieved branch and statement coverage.

Metrics at the Architectural Level

Table 15 of ISO 26262 lists the coverage metrics recommended at the software architectual level:

While the choices appear to by synonmous they are not: Function Coverage is about measuring the completeness of functions being covered. In contrast to that Call Coverage is about the function invocations. While each function may have been invoked at least once, only part of the calling sites may have been executed. In other words: 100% Function Coverage does not necessarily mean 100% Call Coverage.

When reading up the Interpretations of tables in the standard section 4.2 one will discover that the entry marked with letters can be considered alternatives. And since both levels have the same degree of recommendation for each ASIL there’s also no reason to prefer one over the other. Since Function Coverage support is much more wide-spread it’ll be the most typical choice however.

Untestable Code

Ideally, one should strive for 100% coverage. Code may however be written in a defensive manner (think of MISRA), contain debugging facilities or simply be impossible to exercise in a test environment. In that cases a rationale should be given for untested code. To keep track of code exempted from testing froglogic’s code coverage tool Coco features a manual validation process. Code segments can be marked and commented either in an interactive code browser or directly in the source code. Those segments will be reported separately and don’t require further testing anymore.