Cover Python programs by soft&curly tests
Table of Contents
There are several subconcepts of Code coverage, which is just a quantitative measure of finding out how much of the code has been executed. The concept can be deceptive, though, if one doesn't know exactly what those figures mean.
For example, all of the following are subconcepts of code coverage:
- Statement coverage
- Line coverage
- Condition coverage
- Decision coverage
- Multiple condition coverage
- Path coverage
100% line coverage doesn't guarantee anything (well, except that the interpreter has traversed each line, nothing more :). For example, consider the line
if a.bar() and b.frob() or c.frob(): do_something()
a.bar() evaluates to false,
b.frob() isn't usually executed at all
due to ShortCircuit evaluation. b.frob() can be as broken as ever, but
still line coverage can read 100% thus lulling developer into false sense
of security. Condition coverage is a bit better, but it still doesn't try
everything: what about loops? And even if a loop is executed no times at
all and maximum amount of iterations, it still doesn't test everything.
Certainly one discovers quickly that to really test all execution paths, you have to test all possible paths, and even that leaves questions (what about if some loop is executed only max/2 iterations? what about all possible inputs?).
Still, even line coverage is not useless. For developer, it allows one to notice those parts of codes that haven't been even glanced at yet. In interpreted languages, those parts can contain even typos which usually lead to immediate crashes (calling a non-existing method, for example). Part coverage tests much, much more, but unfortunately, the amount of test paths rises exponentially in each decision. For a simple module with 20 if statements, each having 2 possible alternatives, there are 220 ~ 1 million different paths of execution. Add to that a few loops and one more complex condition there and another elsewhere, and you quickly run into billions of test paths. In any real-life project, even 10% path coverage can be just impossible to achieve.
Still, path coverage may be useful for testing only single methods which contain complex algorithms. For Python, it seems like we have only statement coverage so far, thanks to efforts of Gareth Rees and Ned Batchelder. More advanced tools are under development, however.
pip install coverage pip freeze | grep coverage # pip install --upgrade pip
Requirement already satisfied: coverage in /usr/local/share/DVCS/lib/Python/venv/lib/python3.4/site-packages coverage==4.3.4
Python in suite
Python in elegant suite
blog comments powered by Disqus