Changes between Version 1 and Version 2 of testsuite


Ignore:
Timestamp:
02/16/2010 10:58:33 PM (15 years ago)
Author:
Sam Hocevar
Comment:

more test suite information

Legend:

Unmodified
Added
Removed
Modified
  • testsuite

    v1 v2  
    55The testsuite can be run by typing `make check` in the project’s top directory.
    66
    7 == Source code checks ==
     7== Source code tests ==
    88
    99Most source code checks rely on the project’s ability to list all source files. This is done by putting the following `Makefile.am` snippet in all directories that contain source code:
     
    1414}}}
    1515
    16 ... and the following snippet in the top directory `Makefile.am`:
     16... and the following snippet in the top directory `Makefile.am` (example taken from libcaca):
    1717
    1818{{{
     
    3131'''check-win32''' performs a Win32 port consistency check. The following tests are performed:
    3232 * all preprocessor tokens in `config.h.in` must appear in `msvc/config.h`
     33
     34== Documentation tests ==
     35
     36'''check-doxygen''' checks for problems in the Doxygen-built documentation:
     37 * no warnings should appear in the Doxygen build log
     38
     39== libcaca-specific test suite ==
     40
     41[wiki:libcaca libcaca] uses !CppUnit for its unit tests. Coverage is pretty low, but increases with time. The following features are already tested:
     42 * canvas creation, resizing, and basic character blitting
     43 * dirty rectangle features
     44 * driver list
     45 * “caca” format import/export
     46
     47== zzuf-specific test suite ==
     48
     49[wiki:zzuf zzuf] uses several custom scripts to test the program’s features.
     50
     51'''check-div0''' runs a misbehaving executable doing a division by zero when its input is corrupted, and checks that zzuf properly catches the failure.
     52
     53'''check-overflow''' runs a misbehaving executable accessing an unallocated memory area when its input is corrupted, and checks that zzuf properly catches the failure.
     54
     55'''check-utils''' is the most extensive zzuf test. It runs various system utilities and `zzat` programs with known 64kB files and checks that the input of all programs is fuzzed in exactly the same way. System utilities include the following:
     56 * `dd` with different block sizes
     57 * `grep -a ''`
     58 * `sed -e n`
     59 * `tail -n +1`
     60`zzat` programs include the following:
     61 * `repeat(-1,fread(1,1000),feof(1))`
     62 * `repeat(-1,getc(),feof(10))`
     63 * `fread(1,33000) rewind() repeat(-1,fseek(1,SEEK_CUR),fread(1,1),feof(1))`
     64 * `repeat(8000,getc_unlocked()) fread(1,33000)`
     65
     66'''check-zzuf-A-autoinc''' is a check for `zzuf -A` (`--autoinc`): it runs `zzat` with two files, and checks that the second file read is fuzzed differently with `-A`.
     67
     68'''check-zzuf-f-fuzzing''' is a check for `zzuf -f` (`--fuzzing`): it checks that `xor` always changes the input file, that `set` changes all files except when the input already has all bits set, and so on.
     69
     70'''check-zzuf-M-max-memory''' is a check for `zzuf -M` (`--max-memory`): it runs a misbehaving executable allocating enormous massive chunks of memory when its input is corrupted, and checks that zzuf properly catches the failure.
     71
     72'''check-zzuf-m-md5''' is a check for `zzuf -m` (`--md5`): it checks that zzuf properly computes known MD5 checksums such as `d41d8cd98f00b204e9800998ecf8427e` (the empty string) or `5eb63bbbe01eeed093cb22bb8f5acdc3` (hello world).
     73
     74'''check-zzuf-r-ratio''' is a check for `zzuf -r` (`--ratio`): it checks for consistency with the expected average number of flipped bits with a given ratio, ''n/2*R^(1 - 2/n)''.
     75
     76'''check-zzuf-s-seed''' is a check for `zzuf -s` (`--seed`): it runs zzuf up to 200000 times on the same file and checks that the seed distribution causes each bit to be flipped at least once.