NAME - tester hints

DESCRIPTION

The tester manual overview refers to all other sections:

• tester Tester overview

NOTES

HINTS (FAQ)

These hints will be split into two columns. The left column explains what to do; the right column gives the specific commands.

How to commands

backup

backup all expected data see save a test revision expected files are *.R_* backup a test revision tar & compress a copy of: tar -cvf EO_<testname>_rev.tar <testname> *.result *.FAILED *.R_* compress EO_<testname>_rev.tar create "results" test to auto-collect it put utility tests at end of test spec backup data not expected specify with "save" keyword To not write over previous result, use -nocleanup to not execute after. backup test ordering by default tester creates a database database files are: db.<testspec>.dir db.<testspec>.pag

test specification

create a test specification start with template t.form

benchmark

create benchmarks use the -bench -verbose options tests with Hours/Minutes/Seconds reported to Standard Error Use a save output test to run the tests, e.g. "save stderr"

parallel tests

create parallel tests Write tests to run in test document use the end of the tests area use the -queuejobs options Invoke the main test with -execute don't use -queuejobs then all tests go in parallel Use the main test with -queuejobs to do them serially

delete files

delete files removes all system generated files /bin/rm -f *.SI *.[SR]_* *.result *.FAILED *.CMP *.se *.so *.seso OR removes files for t.program /bin/rm -f t.program[0-9a-z]* t.program.result

find

find all expected output more *.R_* > output; vi output find all saved output more *.S_* > saved; vi saved find all Failed diffs more *.CMP > compares; vi compares find all Failed Results more *.FAILED > FAILED; vi FAILED find all Results more *.result > results; vi results find failed tests vi *.FAILED <filename>.result has all failures <testname><id><num>.FAILED has specific test failures

ttags

        
        find specific tests             use tag facility (ttags)
                                        numbers are in 001 .. 999 range
                                        e.g.: for test 005 in t.remark
                                          ttags t.remark > tags
                                          vi -t 005
        
        find test in multiple specs     use tag facility (ttags)
                                        use a keyword for each test spec
                                        numbers are in 001 .. 999 range
                                        e.g.: for test 025 in t.remark
                                          ttags -t remark t.remark >> tags
                                          vi -t remark025
        

Keep files around

keep all files around use -keep to keep expect files use -nocleanup to not do after, afterall do /bin/rm logic in after, afterall do copies / moves in prune

manage tests

manage known failures create a class called "kf" use -c regres -u kf for testing use -c kf to check out bugs use -c regres to see it all add "fix" for "kf" when fixed use -c fix+kf to check bug fixes manage tests use class include, -c regres and class exclude, -u kf+error regres - all regression tests kf - known failure tests fix - known failure fix suite - typical test runs error - error tests all - invoke all of class "suite" put utility tests at end of test spec

reproduce results

create results save test spec (t.testname) save input data (d.testname<id>) create "suite" class for test runs run tests to verify logic tester -v t.spec run tests to generate data tester -c suite -g -ex t.<testname> run tests to check data tester -c suite -ex t.<testname> verify results and output save expected data (*.R_*) save test ordering database files database files are: db.<testspec>.dir db.<testspec>.pag reproduce results restore files run tests to check data tester -c suite -ex t.<testname> look for *FAILED tests ls *.FAILED

restore files

restore a test revision untar & uncompress a copy of: uncompress EO_<testname>_rev.tar.Z tar -xvf EO_<testname>_rev.tar

.

---

SEE ALSO

Steps To Effective Testing