From: Jean Privat <jean@pryen.org>
Date: Sun, 5 Jun 2016 16:10:19 +0000 (-0400)
Subject: Merge: nitunit with black boxes: alternative sav files and --autosav
X-Git-Url: http://nitlanguage.org

Merge: nitunit with black boxes: alternative sav files and --autosav

Two new things:

* alternative .res location for better scalability
* option --autosav to create/update the .res files

For each TestCase `test_bar` of a TestSuite `test_mod.nit`, a corresponding file with the expected output is looked for:

* "test_mod.sav/test_bar.res". I.e. test-cases grouped by test-suites.
    This is the default and is useful if there is a lot of test-suites and test-cases in a directory
* "sav/test_bar.res". I.e. all test-cases grouped in a common sub-directory.
    Useful if there is a lot of test-suites OR test-cases in a directory.
* "test_bar.res" raw in the directory.
    Useful is there is a few test-suites and test-cases in a directory.

All 3 are exclusive. If more than one exists, the test-case is failed.
If a corresponding file then the output of the test-case is compared with the file.

To helps the management of the expected results, the option `--autosav` can be used to automatically create and update them.

With the option, if a black block test fails because a difference between the expected result and the current result then the expected result file is updated (and the test is passed).
If a test-case of a test-suite passes but that some output is generated, then an expected result file is created.

It is expected that the created/updated files are checked since the tests are considered passed.
A VCS like `git` is often a good tool to check the creation and modification of those files.

Pull-Request: #2158
---

e013ac6370c69098c5a85f5ec5963405cbb290cd