Justification of Missing Coverage

In real-life test situations, 100% coverage is hardly achieved for several reasons. You can justify missing coverage with Testwell CTC++, using structured comments in source code or justifications collected in text files.

Justifications always refer to counters relevant for decision coverage, for example:
  • function begins
  • true or false branches of if statements
  • cases of a switch statement

The justifications provided are documented in the HTML report. For all coverage measures, the impact of justifications is calculated. Hence a clear indication is given if "everything" was tested or at least justified.

For dependent coverage counters, statements and lines, the property to be justified is derived by ctcreport. In the following piece of code, the default counter is justified by the user. The two statements below and the break counter are in consequence also justified:

Please note that there is an exception for ternary-? operators: They can be justified directly, but justifications are not passed to their counters.

Structure of Justifications

A justification looks like: 
if (world_still_exists) // CTC++ Justify False | EFFORT: Simulation too expensive, else-branch reviewed instead.
or, inside a justification file: 
182 False | EFFORT: Simulation too expensive, else-branch reviewed instead.
The examples show the common structure of justifications:
type
"False" in example above. Available types are:
  • True: For true counters in if or loop decisions.
  • False: For false counters in if or loop decisions.
  • Function: Whole functions can be justified (Note that this does not refer to unreachable code).
  • empty: Without a specification, single counters like return or case are referred.
  • FullLine: All counters on the line are justified. This can be useful for complex macro code expanded on one line.
tag
"EFFORT": A short text or tag to categorize justifications according to your requirements. Justification information in overview and detail pages is grouped by these tags.
explanation
"Simulation too expensive...": A long explanation for every justification. This explanation is shown in source code view together with the tag for each justified counter.

Separator | is used between type and short text, and : between tag and explanation.