Merge: Added contributing guidelines and link from readme

[nit.git] / share / man / nitunit.md

diff --git a/share/man/nitunit.md b/share/man/nitunit.md
index e80e80e..2bec157 100644 (file)
--- a/share/man/nitunit.md
+++ b/share/man/nitunit.md
@@ -129,18 +129,25 @@ This flag can be used by libraries and program to prevent (or limit) the executi
 
 ## Working with `TestSuites`
 
-TestSuites are Nit files that define a set of TestCases for a particular module.
+TestSuites are Nit modules that define a set of TestCases.
 
-The test suite must be called `test_` followed by the name of the module to test.
-So for the module `foo.nit` the test suite will be called `test_foo.nit`.
+A test suite is a module that uses the annotation `is test_suite`.
+
+It is common that a test suite focuses on testing a single module.
+In this case, the name of the test_suite is often `test_foo.nit` where `foo.nit` is the tested module.
 
 The structure of a test suite is the following:
 
 ~~~~
 # test suite for module `foo`
-module test_foo
+module test_foo is test_suite
+
+import test_suite
 import foo # can be intrude to test private things
+
 class TestFoo
+       super TestSuite
+
     # test case for `foo::Foo::baz`
     fun test_baz do
         var subject = new Foo
@@ -153,11 +160,13 @@ Test suite can be executed using the same `nitunit` command:
 
     $ nitunit foo.nit
 
-`nitunit` will execute a test for each method named `test_*` in a class named `Test*`
-so multiple tests can be executed for a single method:
+`nitunit` will execute a test for each method named `test_*` in a class
+subclassing `TestSuite` so multiple tests can be executed for a single method:
 
 ~~~~
 class TestFoo
+       super TestSuite
+
     fun test_baz_1 do
         var subject = new Foo
         assert subject.baz(1, 2) == 3
@@ -173,14 +182,33 @@ end
 
 Sometimes, it is easier to validate a `TestCase` by comparing its output with a text file containing the expected result.
 
-For each TestCase `test_bar` of a TestSuite `test_mod.nit`, if the corresponding file `test_mod.sav/test_bar.res` exists, then the output of the test is compared with the file.
+For each TestCase `test_bar` of a TestSuite `test_mod.nit`, a corresponding file with the expected output is looked for:
+
+* "test_mod.sav/test_bar.res". I.e. test-cases grouped by test-suites.
+
+       This is the default and is useful if there is a lot of test-suites and test-cases in a directory
+
+* "sav/test_bar.res". I.e. all test-cases grouped in a common sub-directory.
+
+       Useful if there is a lot of test-suites OR test-cases in a directory.
+
+* "test_bar.res" raw in the directory.
+
+       Useful is there is a few test-suites and test-cases in a directory.
+
+All 3 are exclusive. If more than one exists, the test-case is failed.
+
+If a corresponding file then the output of the test-case is compared with the file.
 
 The `diff(1)` command is used to perform the comparison.
 The test is failed if non-zero is returned by `diff`.
 
 ~~~
 module test_mod is test_suite
+
 class TestFoo
+       super TestSuite
+
        fun test_bar do
                print "Hello!"
        end
@@ -195,15 +223,19 @@ Hello!
 
 If no corresponding `.res` file exists, then the output of the TestCase is ignored.
 
+To helps the management of the expected results, the option `--autosav` can be used to automatically create and update them.
+
+
 ## Configuring TestSuites
 
-`TestSuites` also provide methods to configure the test run:
+`TestSuite`s also provide methods to configure the test run:
 
 `before_test` and `after_test`: methods called before/after each test case.
 They can be used to factorize repetitive tasks:
 
 ~~~~
 class TestFoo
+       super TestSuite
     var subject: Foo
     # Mandatory empty init
     init do end
@@ -268,10 +300,13 @@ Output name (default is 'nitunit.xml').
 `nitunit` produces a XML file compatible with JUnit.
 
 ### `--dir`
-Working directory (default is '.nitunit').
+Working directory (default is 'nitunit.out').
 
 In order to execute the tests, nit files are generated then compiled and executed in the giver working directory.
 
+In case of success, the directory is removed.
+In case of failure, it is kept as is so files can be investigated.
+
 ### `--nitc`
 nitc compiler to use.
 
@@ -286,8 +321,15 @@ Only run test case with name that match pattern.
 
 Examples: `TestFoo`, `TestFoo*`, `TestFoo::test_foo`, `TestFoo::test_foo*`, `test_foo`, `test_foo*`
 
-### `-t`, `--target-file`
-Specify test suite location.
+### `--autosav`
+Automatically create/update .res files for black box testing.
+
+If a black block test fails because a difference between the expected result and the current result then the expected result file is updated (and the test is passed).
+
+If a test-case of a test-suite passes but that some output is generated, then an expected result file is created.
+
+It is expected that the created/updated files are checked since the tests are considered passed.
+A VCS like `git` is often a good tool to check the creation and modification of those files.
 
 ## SUITE GENERATION