nitweb/angular: show linearization for mclasses, mclassdefs, mproperties, mpropdefs

[nit.git] / share / man / nitunit.md

diff --git a/share/man/nitunit.md b/share/man/nitunit.md
index 4f9ad61..23b7382 100644 (file)
--- a/share/man/nitunit.md
+++ b/share/man/nitunit.md
@@ -169,6 +169,53 @@ class TestFoo
 end
 ~~~~
 
 end
 ~~~~
 

+## Black Box Testing
+
+Sometimes, it is easier to validate a `TestCase` by comparing its output with a text file containing the expected result.
+
+For each TestCase `test_bar` of a TestSuite `test_mod.nit`, a corresponding file with the expected output is looked for:
+
+* "test_mod.sav/test_bar.res". I.e. test-cases grouped by test-suites.
+
+       This is the default and is useful if there is a lot of test-suites and test-cases in a directory
+
+* "sav/test_bar.res". I.e. all test-cases grouped in a common sub-directory.
+
+       Useful if there is a lot of test-suites OR test-cases in a directory.
+
+* "test_bar.res" raw in the directory.
+
+       Useful is there is a few test-suites and test-cases in a directory.
+
+All 3 are exclusive. If more than one exists, the test-case is failed.
+
+If a corresponding file then the output of the test-case is compared with the file.
+
+The `diff(1)` command is used to perform the comparison.
+The test is failed if non-zero is returned by `diff`.
+
+~~~
+module test_mod is test_suite
+class TestFoo
+       fun test_bar do
+               print "Hello!"
+       end
+end
+~~~
+
+Where `test_mod.sav/test_bar.res` contains
+
+~~~raw
+Hello!
+~~~
+
+If no corresponding `.res` file exists, then the output of the TestCase is ignored.
+
+To helps the management of the expected results, the option `--autosav` can be used to automatically create and update them.
+
+
+## Configuring TestSuites
+

 `TestSuites` also provide methods to configure the test run:
 
 `before_test` and `after_test`: methods called before/after each test case.
 `TestSuites` also provide methods to configure the test run:
 
 `before_test` and `after_test`: methods called before/after each test case.

@@ -237,13 +284,22 @@ With the `--full` option, all imported modules (even those in standard) are also
 ### `-o`, `--output`
 Output name (default is 'nitunit.xml').
 
 ### `-o`, `--output`
 Output name (default is 'nitunit.xml').
 

-### `nitunit` produces a XML file comatible with JUnit.
+`nitunit` produces a XML file compatible with JUnit.

 
 ### `--dir`
 
 ### `--dir`

-Working directory (default is '.nitunit').
+Working directory (default is 'nitunit.out').

 
 In order to execute the tests, nit files are generated then compiled and executed in the giver working directory.
 
 
 In order to execute the tests, nit files are generated then compiled and executed in the giver working directory.
 

+In case of success, the directory is removed.
+In case of failure, it is kept as is so files can be investigated.
+
+### `--nitc`
+nitc compiler to use.
+
+By default, nitunit tries to locate the `nitc` program with the environment variable `NITC` or heuristics.
+The option is used to indicate a specific nitc binary.
+

 ### `--no-act`
 Does not compile and run tests.
 
 ### `--no-act`
 Does not compile and run tests.
 

@@ -255,6 +311,16 @@ Examples: `TestFoo`, `TestFoo*`, `TestFoo::test_foo`, `TestFoo::test_foo*`, `tes
 ### `-t`, `--target-file`
 Specify test suite location.
 
 ### `-t`, `--target-file`
 Specify test suite location.
 

+### `--autosav`
+Automatically create/update .res files for black box testing.
+
+If a black block test fails because a difference between the expected result and the current result then the expected result file is updated (and the test is passed).
+
+If a test-case of a test-suite passes but that some output is generated, then an expected result file is created.
+
+It is expected that the created/updated files are checked since the tests are considered passed.
+A VCS like `git` is often a good tool to check the creation and modification of those files.
+

 ## SUITE GENERATION
 
 ### `--gen-suite`
 ## SUITE GENERATION
 
 ### `--gen-suite`

@@ -271,6 +337,36 @@ Also generate test case for private methods.
 ### `--only-show`
 Only display the skeleton, do not write any file.
 
 ### `--only-show`
 Only display the skeleton, do not write any file.
 

+
+# ENVIRONMENT VARIABLES
+
+### `NITC`
+
+Indicate the specific Nit compiler executable to use. See `--nitc`.
+
+### `NIT_TESTING`
+
+The environment variable `NIT_TESTING` is set to `true` during the execution of program tests.
+Some libraries of programs can use it to produce specific reproducible results; or just to exit their executions.
+
+Unit-tests may unset this environment variable to retrieve the original behavior of such piece of software.
+
+### `SRAND`
+
+In order to maximize reproducibility, `SRAND` is set to 0.
+This make the pseudo-random generator no random at all.
+See `Sys::srand` for details.
+
+To retrieve the randomness, unit-tests may unset this environment variable then call `srand`.
+
+### `NIT_TESTING_ID`
+
+Parallel executions can cause some race collisions on named resources (e.g. DB table names).
+To solve this issue, `NIT_TESTING_ID` is initialized with a distinct integer identifier that can be used to give unique names to resources.
+
+Note: `rand` is not a recommended way to get a distinct identifier because its randomness is disabled by default. See `SRAND`.
+
+

 # SEE ALSO
 
 The Nit language documentation and the source code of its tools and libraries may be downloaded from <http://nitlanguage.org>
 # SEE ALSO
 
 The Nit language documentation and the source code of its tools and libraries may be downloaded from <http://nitlanguage.org>