nitunit: document black box (.res) testing

Signed-off-by: Jean Privat <jean@pryen.org>

diff --git a/share/man/nitunit.md b/share/man/nitunit.md
index 4e4ea82..0576201 100644 (file)
--- a/share/man/nitunit.md
+++ b/share/man/nitunit.md
@@ -169,6 +169,34 @@ class TestFoo
 end
 ~~~~
 
+## Black Box Testing
+
+Sometimes, it is easier to validate a `TestCase` by comparing its output with a text file containing the expected result.
+
+For each TestCase `test_bar` of a TestSuite `test_mod.nit`, if the corresponding file `test_mod.sav/test_bar.res` exists, then the output of the test is compared with the file.
+
+The `diff(1)` command is used to perform the comparison.
+The test is failed if non-zero is returned by `diff`.
+
+~~~
+module test_mod is test_suite
+class TestFoo
+       fun test_bar do
+               print "Hello!"
+       end
+end
+~~~
+
+Where `test_mod.sav/test_bar.res` contains
+
+~~~raw
+Hello!
+~~~
+
+If no corresponding `.res` file exists, then the output of the TestCase is ignored.
+
+## Configuring TestSuites
+
 `TestSuites` also provide methods to configure the test run:
 
 `before_test` and `after_test`: methods called before/after each test case.

diff --git a/tests/sav/nitunit_args9.res b/tests/sav/nitunit_args9.res
index 8ff31d6..9cbce91 100644 (file)
--- a/tests/sav/nitunit_args9.res
+++ b/tests/sav/nitunit_args9.res
@@ -3,15 +3,9 @@ Tested method
 After Test
 Runtime error: Assert failed (test_nitunit4/test_nitunit4_base.nit:31)
 
-diff: missing operand after 'expected:'
-diff: Try 'diff --help' for more information.
-test_nitunit4/test_nitunit4.nit:28,2--30,4: ERROR: test_bar (in file .nitunit/gen_test_nitunit4.nit): Diff
-
-diff: missing operand after 'expected:'
-diff: Try 'diff --help' for more information.
 test_nitunit4/test_nitunit4.nit:32,2--34,4: ERROR: test_baz (in file .nitunit/gen_test_nitunit4.nit): Diff
---- expected: test_nitunit4/test_baz.res
-+++ got: .nitunit/gen_test_nitunit4_test_baz.out1
+--- expected:test_nitunit4/test_nitunit4.sav/test_baz.res
++++ got:.nitunit/gen_test_nitunit4_test_baz.out1
 @@ -1 +1,3 @@
 -Bad result file
 +Before Test
@@ -23,15 +17,14 @@ No doc units found
 Entities: 12; Documented ones: 0; With nitunits: 0; Failures: 0
 
 TestSuites:
-Class suites: 1; Test Cases: 3; Failures: 3
+Class suites: 1; Test Cases: 3; Failures: 2
 <testsuites><testsuite package="test_nitunit4&gt;"></testsuite><testsuite package="test_nitunit4::nitunit4"></testsuite><testsuite package="test_nitunit4"><testcase classname="nitunit.test_nitunit4::test_nitunit4.test_nitunit4::TestTestSuite" name="test_nitunit4::TestTestSuite::test_foo"><system-out></system-out><system-err>Before Test
 Tested method
 After Test
 Runtime error: Assert failed (test_nitunit4&#47;test_nitunit4_base.nit:31)
-</system-err><error message="Runtime Error"></error></testcase><testcase classname="nitunit.test_nitunit4::test_nitunit4.test_nitunit4::TestTestSuite" name="test_nitunit4::TestTestSuite::test_bar"><system-out></system-out><system-err>Diff
-</system-err><error message="Runtime Error"></error></testcase><testcase classname="nitunit.test_nitunit4::test_nitunit4.test_nitunit4::TestTestSuite" name="test_nitunit4::TestTestSuite::test_baz"><system-out></system-out><system-err>Diff
---- expected: test_nitunit4&#47;test_baz.res
-+++ got: .nitunit&#47;gen_test_nitunit4_test_baz.out1
+</system-err><error message="Runtime Error"></error></testcase><testcase classname="nitunit.test_nitunit4::test_nitunit4.test_nitunit4::TestTestSuite" name="test_nitunit4::TestTestSuite::test_bar"><system-out></system-out><system-err></system-err></testcase><testcase classname="nitunit.test_nitunit4::test_nitunit4.test_nitunit4::TestTestSuite" name="test_nitunit4::TestTestSuite::test_baz"><system-out></system-out><system-err>Diff
+--- expected:test_nitunit4&#47;test_nitunit4.sav&#47;test_baz.res
++++ got:.nitunit&#47;gen_test_nitunit4_test_baz.out1
 @@ -1 +1,3 @@
 -Bad result file
 +Before Test