Jon Williams Blog Upcoming Resume More…

Making Sure All Your Test Classes Are Loaded by Your Test Suite

20 May 2013

We were having an issue with developers ommiting or removing tests from our test harness. So I kluged together a PHP/Bash monotrosity that prints a list of files not loaded during the execution of the test harness as determined by strace’s log of file access. Although this is a little coarse, it does provide a list of outdated fixtures, new unadded tests and defunct tests to be removed/fixed. You probably want to tee this into a logfile.

<?php
define('TEST_BASE_PATH', realpath(realpath(dirname(__FILE__)) . '/../library/Mmf/Test/'));
define('SCRIPT_PATH', realpath(dirname(__FILE__)) . '/CommitTest.php');

echo SCRIPT_PATH,"\n";
echo "This is SLOW!\n";
$tmp = tempnam(sys_get_temp_dir(), "TestSuiteCoverage-");
$path = TEST_BASE_PATH;
$files = explode("\n",`find $path -type f`);
echo "Logging strace to $tmp\n";
passthru("strace -o $tmp -eopen -f php ".SCRIPT_PATH." ".escapeshellcmd(implode(' ',array_slice($argv,1))));

$lines = explode("\n",`cut -d \" -f 2 $tmp | grep $path | sort | uniq`);
$diff = array_diff($files,$lines);
echo count($lines), " files encountered of ", count($files), "; ", count($diff), " missing\n\n";
foreach($diff as $file) {
    echo "$file\n";
}
unlink($tmp);

Example output:

php util_scripts/TestSuiteCoverage.php
/home/jon/build/Spam/util_scripts/CommitTest.php
This is SLOW!
Logging strace to /tmp/TestSuiteCoverage-z0EBOL

phpunit   /home/jon/build/Spam/util_scripts/../library/SpacelySprockets/Test/Suite/Integration/Frontend.php 

PHPUnit 3.6.3 by Sebastian Bergmann.

...............................................................  63 / 528 ( 11%)
............................................................... 126 / 528 ( 23%)
............................................................... 189 / 528 ( 35%)
............................................................... 252 / 528 ( 47%)
............................................................... 315 / 528 ( 59%)
............................................................... 378 / 528 ( 71%)
............................................................... 441 / 528 ( 83%)
............................................................... 504 / 528 ( 95%)
........................

Time: 19:56, Memory: 558.25Mb

OK (528 tests, 5557 assertions)
301 files encountered of 362; 361 missing

/home/jon/build/Spam/library/SpacelySprockets/Test/Suite/Integration/CodeLibrary.php
/home/jon/build/Spam/library/SpacelySprockets/Test/Suite/Integration/Fizbuzz.php
/home/jon/build/Spam/library/SpacelySprockets/Test/Suite/Integration/Service.php
/home/jon/build/Spam/library/SpacelySprockets/Test/Suite/Integration/Model.php
/home/jon/build/Spam/library/SpacelySprockets/Test/Suite/Unit/All.php
/home/jon/build/Spam/library/SpacelySprockets/Test/Library/Constant/DefaultProfile.php
/home/jon/build/Spam/library/SpacelySprockets/Test/Library/Autoloader.php
/home/jon/build/Spam/library/SpacelySprockets/Test/Mock/Acl/Frontend.php
/home/jon/build/Spam/library/SpacelySprockets/Test/Mock/View/Helper/TabbedPane.php
/home/jon/build/Spam/library/SpacelySprockets/Test/Fixtures/video-fizbuzz.sql
/home/jon/build/Spam/library/SpacelySprockets/Test/Fixtures/content_comment.sql
/home/jon/build/Spam/library/SpacelySprockets/Test/Fixtures/member-usage.sql
/home/jon/build/Spam/library/SpacelySprockets/Test/Fixtures/stats-test.sql
blog comments powered by Disqus