Code Coverage: Testwell CTC++ vs. GCov
Often people ask us about the advantages of Testwell CTC++
Test Coverage Analyzer for C/C++ compared to the GNU coverage testing tool GCov.
Below you will find a short and non-exhaustive comparaison of both tools.
If you have comments, please feel free to contact us: quality(AT)verifysoft.com
Your experience is important to us and might be published here to improve this page.
Code Coverage Level
Testwell CTC++ supports all coverage levels
up to Modified Condition Coverage (including MC/DC coverage as requested for critical projects) and can be used to get certifications, i.e. DO-178C
, IEC 61508
, ISO 26262
On some GCov related web sites is claimed that GCov supports Decision Coverage, but there is some danger(!) and finally we do not agree with this statements.
Have a look to the following:
GCov instruments its code at a late step into your code. It is possible here, that some coverage gets lost because the compiler just optimises it out!
GCov is based on Line Coverage: it can not handle multiple stuff at the same line: try to shrink a function to a one-liner and you will see...
As GCov makes the instrumentation at a late step, it has no control which branch of source condition has been mapped to that intermediate code.
It gives just "branches" and you have really to guess, what source condition maps to.
Let´s have a look to the outputs taken from prime example
using this input vector
First the CTC++ report:
You see, "if" was 2 times true and 7 times false.
And with the nice logical table, the first condition was once true, etc....
Now have a look to the GCov report:
The if() statement has been executed 9 times, but how often was it "true" and how often was it "false"?
"Branch 3" corresponds here to to "val==2", but according to the level of compiler optimisation things might change...
Anyway with (multi)condtion coverage this results are unusable... and they will be even worse with a real world project.
If you compile our example project with -O2 optimisation (which is the default optimisation in most projects) you will get the following with GCov:
... only two branches remaining. How to map them into our "CTC++ truth table"???
GCov supports gcc compiler.
Testwell CTC++ works with any compiler
and btw with any target (even very small targets) when used with embedded systems.
GCov supports C and C++.
Testwell CTC++ supports C and C++ as well as Java and C#. To extend the "basic"CTC++ to Java and C# only a small "CTC++ for Java and C# add-on" is needed.
As a result you have one tool for several languages.
Price vs. Cost
(Brian Marick: "The most important thing is that the tool´s reports be easy for programmers to understand.")
Testwell CTC++ has a much more intuitive usable/clickable HTML report.
This HTML report is created out of a textual report. You can put the textual report into your repository, and at any time in the future, you can re-create the HTML-report.
Testwell CTC++ has several reports, i.e.: CTC++ Coverage Report (HTML format, hierarchical with 4 levels: Directory Summary, Files Summary, Functions Summary, Execution Profile), Execution Profile Listing, Untested Code Listing, Coverage Summary Listing, Execution Time Listing.
To learn more about Testwell Code Coverage reports, please have a look to CTC++ mainpage or explore the reports by clicking the picture beside.
GCov is "free".
Testwell CTC++ is a professional commercial tool.
Both floating and machine locked licenses are available at very reasonable prices. Although you have to pay for Testwell CTC++, you will most likely save money thanks to CTC++ features and benefits.
Depending on your projects the cost of Testwell CTC++ is probably much lower than the cost of GCov usage.
You will find some information below.
But please try yourself:
professional developers can ask us for a free time limited license (with full functionalities) to evaluate
Testwell CTC++ and to learn about its advantages.