About Dynamic Analysis in Testwell CTC++

The examination of the run-time behavior of a program is called dynamic analysis. Beyond code coverage, Testwell CTC++ can analyze the execution time of functions.

When measuring code coverage, Testwell CTC++ shows how many times each measurement point was executed, and not only the information if it was executed. These execution counters already give a reasonable picture of the program behavior.

Additionally, Testwell CTC++ supports dynamic analysis through timing instrumentation. It measures how many times each function in the instrumented file was called and what was the total, average and maximum (considered to be the “worst”) execution time of the function. This information can be used for finding program execution bottlenecks and for code optimizing.

Actually what is measured here does not need to be “execution time”. It can be more generally “execution cost”. It is a measurable amount of some resource that the function execution consumes. By default, Testwell CTC++ is configured to use clock() from <time.h>, i.e. to measure execution time, as the function by which the function resource consumption is measured. You can set Testwell CTC++ to use some other function (and provide its implementation file to the linkage) for measuring something else, for example, CPU time or number of I/O operations.

When you select timing instrumentation, you also need to select whether the time measuring is inclusive or exclusive. Inclusive timing means that the time spent in the called functions is counted also to the time of the caller function. Exclusive timing means that the time spent in the called instrumented functions is counted away from the time of the caller function.

It should, however, be kept in mind that in the first place Testwell CTC++ is a code coverage tool. Execution time measurements produced by Testwell CTC++ are rough estimates that can be used for finding the program bottlenecks but they are not recommended for measuring actual and exact performance. At least, you should measure in your environment what overhead the timing instrumentation brings. Calling operating system service clock() may be a costly operation, if done in a tight loop of your program.

Testwell CTC++ supports program dynamic analysis also by a function call trace. You can adjust Testwell CTC++’s instrumentation so that whenever a function is called and when it exits, a trace function (which you can specify) is also called. In the trace function you can display the name of the function to the screen for example, or do whatever recording you find useful to analyze the dynamic program call flow.