= Caca Labs test suite =

All our major projects feature an extensive test suite that checks the software for consistency, source code quality, features and regressions.

The testsuite can be run by typing `make check` in the project’s top directory.

== Source code tests ==

Most source code checks rely on the project’s ability to list all source files. This is done by putting the following `Makefile.am` snippet in all directories that contain source code:

{{{
#!makefile
echo-sources: ; echo $(SOURCES)
}}}

... and the following snippet in the top directory `Makefile.am` (example taken from libcaca):

{{{
#!makefile
echo-dirs: ; echo caca kernel src cxx examples ruby tools
}}}

'''check-source''' performs a source code quality check. It checks for the following coding style errors:
 * no source code file should contain tabs
 * no source code file should contain trailing spaces

'''check-copyright''' performs a copyright statement check. The following tests are performed:
 * all source code files must contain a “`Copyright (c)`” statement
 * all source code files must mention the year of the last modification (this check is only performed when in a git-svn repository)

'''check-win32''' performs a Win32 port consistency check. The following tests are performed:
 * all preprocessor tokens in `config.h.in` must appear in `msvc/config.h`

== Documentation tests ==

'''check-doxygen''' checks for problems in the Doxygen-built documentation:
 * no warnings should appear in the Doxygen build log

== libcaca-specific test suite ==

[wiki:libcaca libcaca] uses !CppUnit for its unit tests. Coverage is pretty low, but increases with time. The following features are already tested:
 * canvas creation, resizing, and basic character blitting
 * dirty rectangle features
 * driver list
 * “caca” format import/export

== zzuf-specific test suite ==

[wiki:zzuf zzuf] uses several custom scripts to test the program’s features.

'''check-div0''' runs a misbehaving executable doing a division by zero when its input is corrupted, and checks that zzuf properly catches the failure.

'''check-overflow''' runs a misbehaving executable accessing an unallocated memory area when its input is corrupted, and checks that zzuf properly catches the failure.

'''check-utils''' is the most extensive zzuf test. It runs various system utilities and `zzat` programs with known 64kB files and checks that the input of all programs is fuzzed in exactly the same way. System utilities include the following:
 * `dd` with different block sizes
 * `grep -a ''`
 * `sed -e n`
 * `tail -n +1`
`zzat` programs include the following:
 * `repeat(-1,fread(1,1000),feof(1))`
 * `repeat(-1,getc(),feof(10))`
 * `fread(1,33000) rewind() repeat(-1,fseek(1,SEEK_CUR),fread(1,1),feof(1))`
 * `repeat(8000,getc_unlocked()) fread(1,33000)`

'''check-zzuf-A-autoinc''' is a check for `zzuf -A|--autoinc`: it runs `zzat` with two files, and checks that the second file read is fuzzed differently with `-A`.

'''check-zzuf-f-fuzzing''' is a check for `zzuf -f|--fuzzing`: it checks that `xor` always changes the input file, that `set` changes all files except when the input already has all bits set, and so on.

'''check-zzuf-M-max-memory''' is a check for `zzuf -M|--max-memory`: it runs a misbehaving executable allocating enormous massive chunks of memory when its input is corrupted, and checks that zzuf properly catches the failure.

'''check-zzuf-m-md5''' is a check for `zzuf -m|--md5`: it checks that zzuf properly computes known MD5 checksums such as `d41d8cd98f00b204e9800998ecf8427e` (the empty string) or `5eb63bbbe01eeed093cb22bb8f5acdc3` (hello world).

'''check-zzuf-r-ratio''' is a check for `zzuf -r|--ratio`: it checks for consistency with the expected average number of flipped bits with a given ratio:
{{{
#!latex
$n_{expected} = \dfrac {n_{total}r^{1 - \frac{2}{n_{total}}}}{2}$
}}}

'''check-zzuf-s-seed''' is a check for `zzuf -s` (`--seed`): it runs zzuf up to 200000 times on the same file and checks that the seed distribution causes each bit to be flipped at least once.

== toilet-specific test suite ==

[wiki:toilet toilet] also has a custom test script.

'''check-fonts''' checks that all shipped fonts can be opened with toilet.