= Caca Labs test suite = All our major projects feature an extensive test suite that checks the software for consistency, source code quality, features and regressions. The testsuite can be run by typing `make check` in the project’s top directory. == Source code tests == Most source code checks rely on the project’s ability to list all source files. This is done by putting the following `Makefile.am` snippet in all directories that contain source code: {{{ #!makefile echo-sources: ; echo $(SOURCES) }}} ... and the following snippet in the top directory `Makefile.am` (example taken from libcaca): {{{ #!makefile echo-dirs: ; echo caca kernel src cxx examples ruby tools }}} '''check-source''' performs a source code quality check. It checks for the following coding style errors: * no source code file should contain tabs * no source code file should contain trailing spaces '''check-copyright''' performs a copyright statement check. The following tests are performed: * all source code files must contain a “`Copyright (c)`” statement * all source code files must mention the year of the last modification (this check is only performed when in a git-svn repository) '''check-win32''' performs a Win32 port consistency check. The following tests are performed: * all preprocessor tokens in `config.h.in` must appear in `msvc/config.h` == Documentation tests == '''check-doxygen''' checks for problems in the Doxygen-built documentation: * no warnings should appear in the Doxygen build log == libcaca-specific test suite == [wiki:libcaca libcaca] uses !CppUnit for its unit tests. Coverage is pretty low, but increases with time. The following features are already tested: * canvas creation, resizing, and basic character blitting * dirty rectangle features * driver list * “caca” format import/export == zzuf-specific test suite == [wiki:zzuf zzuf] uses several custom scripts to test the program’s features. '''check-div0''' runs a misbehaving executable doing a division by zero when its input is corrupted, and checks that zzuf properly catches the failure. '''check-overflow''' runs a misbehaving executable accessing an unallocated memory area when its input is corrupted, and checks that zzuf properly catches the failure. '''check-utils''' is the most extensive zzuf test. It runs various system utilities and `zzat` programs with known 64kB files and checks that the input of all programs is fuzzed in exactly the same way. System utilities include the following: * `dd` with different block sizes * `grep -a ''` * `sed -e n` * `tail -n +1` `zzat` programs include the following: * `repeat(-1,fread(1,1000),feof(1))` * `repeat(-1,getc(),feof(10))` * `fread(1,33000) rewind() repeat(-1,fseek(1,SEEK_CUR),fread(1,1),feof(1))` * `repeat(8000,getc_unlocked()) fread(1,33000)` '''check-zzuf-A-autoinc''' is a check for `zzuf -A|--autoinc`: it runs `zzat` with two files, and checks that the second file read is fuzzed differently with `-A`. '''check-zzuf-f-fuzzing''' is a check for `zzuf -f|--fuzzing`: it checks that `xor` always changes the input file, that `set` changes all files except when the input already has all bits set, and so on. '''check-zzuf-M-max-memory''' is a check for `zzuf -M|--max-memory`: it runs a misbehaving executable allocating enormous massive chunks of memory when its input is corrupted, and checks that zzuf properly catches the failure. '''check-zzuf-m-md5''' is a check for `zzuf -m|--md5`: it checks that zzuf properly computes known MD5 checksums such as `d41d8cd98f00b204e9800998ecf8427e` (the empty string) or `5eb63bbbe01eeed093cb22bb8f5acdc3` (hello world). '''check-zzuf-r-ratio''' is a check for `zzuf -r|--ratio`: it checks for consistency with the expected average number of flipped bits with a given ratio: {{{ #!latex $N_{expected} = \dfrac {N_{total}}{2}R^{1 - \frac{2}{N_{total}}}$ }}} '''check-zzuf-s-seed''' is a check for `zzuf -s|--seed`): it runs zzuf up to 200000 times on the same file and checks that the seed distribution causes each bit to be flipped at least once. == toilet-specific test suite == [wiki:toilet toilet] also has a custom test script. '''check-fonts''' checks that all shipped fonts can be opened with toilet.