## Working with `TestSuites`
-TestSuites are Nit files that define a set of TestCases for a particular module.
+TestSuites are Nit modules that define a set of TestCases.
-The test suite must be called `test_` followed by the name of the module to test.
-So for the module `foo.nit` the test suite will be called `test_foo.nit`.
+A test suite is a module that uses the annotation `is test`.
+
+It is common that a test suite focuses on testing a single module.
+In this case, the name of the test suite is often `test_foo.nit` where `foo.nit` is the tested module.
The structure of a test suite is the following:
~~~~
# test suite for module `foo`
-module test_foo
+module test_foo is test
+
import foo # can be intrude to test private things
+
class TestFoo
+ test
+
# test case for `foo::Foo::baz`
- fun test_baz do
+ fun baz is test do
var subject = new Foo
assert subject.baz(1, 2) == 3
end
$ nitunit foo.nit
-`nitunit` will execute a test for each method named `test_*` in a class named `Test*`
-so multiple tests can be executed for a single method:
+`nitunit` will execute a test for each method with the `test` annotation in a class
+also annotated with `test` so multiple tests can be executed for a single method:
~~~~
class TestFoo
- fun test_baz_1 do
+ test
+
+ fun baz_1 is test do
var subject = new Foo
assert subject.baz(1, 2) == 3
end
- fun test_baz_2 do
+ fun baz_2 is test do
var subject = new Foo
assert subject.baz(1, -2) == -1
end
Sometimes, it is easier to validate a `TestCase` by comparing its output with a text file containing the expected result.
-For each TestCase `test_bar` of a TestSuite `test_mod.nit`, if the corresponding file `test_mod.sav/test_bar.res` exists, then the output of the test is compared with the file.
+For each TestCase `test_bar` of a TestSuite `test_mod.nit`, a corresponding file with the expected output is looked for:
+
+* "test_mod.sav/test_bar.res". I.e. test-cases grouped by test-suites.
+
+ This is the default and is useful if there is a lot of test-suites and test-cases in a directory
+
+* "sav/test_bar.res". I.e. all test-cases grouped in a common sub-directory.
+
+ Useful if there is a lot of test-suites OR test-cases in a directory.
+
+* "test_bar.res" raw in the directory.
+
+ Useful is there is a few test-suites and test-cases in a directory.
+
+All 3 are exclusive. If more than one exists, the test-case is failed.
+
+If a corresponding file then the output of the test-case is compared with the file.
The `diff(1)` command is used to perform the comparison.
The test is failed if non-zero is returned by `diff`.
~~~
-module test_mod is test_suite
+module test_mod is test
+
class TestFoo
- fun test_bar do
+ test
+
+ fun bar is test do
print "Hello!"
end
end
If no corresponding `.res` file exists, then the output of the TestCase is ignored.
-## Configuring TestSuites
+To helps the management of the expected results, the option `--autosav` can be used to automatically create and update them.
-`TestSuites` also provide methods to configure the test run:
-`before_test` and `after_test`: methods called before/after each test case.
+## Configuring TestSuites
+
+`TestSuite`s also provide annotations to configure the test run:
+`before` and `after` annotations can be added to methods that must be called before/after each test case.
They can be used to factorize repetitive tasks:
-~~~~
+~~~
class TestFoo
- var subject: Foo
- # Mandatory empty init
- init do end
- # Method executed before each test
- fun before_test do
- subject = new Foo
- end
- fun test_baz_1 do
- assert subject.baz(1, 2) == 3
- end
- fun test_baz_2 do
- assert subject.baz(1, -2) == -1
- end
+ test
+
+ var subject: Foo is noinit
+
+ # Method executed before each test
+ fun set_up is before do
+ subject = new Foo
+ end
+
+ fun baz_1 is test do
+ assert subject.baz(1, 2) == 3
+ end
+
+ fun baz_2 is test do
+ assert subject.baz(1, -2) == -1
+ end
end
-~~~~
+~~~
-When using custom test attributes, an empty `init` must be declared to allow automatic test running.
+When using custom test attributes, a empty init must be declared to allow automatic test running.
-`before_module` and `after_module`: methods called before/after each test suite.
-They have to be declared at top level:
+At class level, `before_all` and `after_all` annotations can be set on methods that must be called before/after all the test cases in the class:
-~~~~
+~~~
+class TestFoo
+ test
+
+ var subject: Foo is noinit
+
+ # Method executed before all tests in the class
+ fun set_up is before_all do
+ subject = new Foo
+ end
+
+ fun baz_1 is test do
+ assert subject.baz(1, 2) == 3
+ end
+
+ fun baz_2 is test do
+ assert subject.baz(1, -2) == -1
+ end
+end
+~~~
+
+`before_all` and `after_all` annotations can also be set on methods that must be called before/after each test suite when declared at top level:
+
+~~~
module test_bdd_connector
+
import bdd_connector
+
# Testing the bdd_connector
class TestConnector
- # test cases using a server
+ test
+ # test cases using a server
end
+
# Method executed before testing the module
-fun before_module do
- # start server before all test cases
+fun setup_db is before_all do
+ # start server before all test cases
end
+
# Method executed after testing the module
-fun after_module do
- # stop server after all test cases
+fun teardown_db is after_all do
+ # stop server after all test cases
end
-~~~~
+~~~
+
+When dealing with multiple test suites, niunit allows you to import other test suites to factorize your tests:
+
+~~~
+module test_bdd_users
+
+import test_bdd_connector
+
+# Testing the user table
+class TestUsersTable
+ test
+ # test cases using the db server from `test_bdd_connector`
+end
+
+fun setup_table is before_all do
+ # create user table
+end
+
+fun teardown_table is after_all do
+ # drop user table
+end
+~~~
+
+Methods with `before*` and `after*` annotations are linearized and called in different ways.
+
+* `before*` methods are called from the least specific to the most specific
+* `after*` methods are called from the most specific to the least specific
+
+In the previous example, the execution order would be:
+
+1. `test_bdd_connector::setup_db`
+2. `test_bdd_users::setup_table`
+3. `all test cases from test_bdd_users`
+4. `test_bdd_users::teardown_table`
+5. `test_bdd_connector::teardown_db`
+
+## Accessing the test suite environment
+
+The `NIT_TESTING_PATH` environment variable contains the current test suite
+file path.
+Nitunit define this variable before the execution of each test suite.
+It can be used to access files based on the current test suite location:
+
+~~~
+class TestWithPath
+ test
+
+ fun test_suite_path do
+ assert "NIT_TESTING_PATH".environ != ""
+ end
+end
+~~~
## Generating test suites
Examples: `TestFoo`, `TestFoo*`, `TestFoo::test_foo`, `TestFoo::test_foo*`, `test_foo`, `test_foo*`
-### `-t`, `--target-file`
-Specify test suite location.
+### `--autosav`
+Automatically create/update .res files for black box testing.
+
+If a black block test fails because a difference between the expected result and the current result then the expected result file is updated (and the test is passed).
+
+If a test-case of a test-suite passes but that some output is generated, then an expected result file is created.
+
+It is expected that the created/updated files are checked since the tests are considered passed.
+A VCS like `git` is often a good tool to check the creation and modification of those files.
+
+### `--no-time`
+Disable time information in XML.
+
+This is used to have reproducible XML results.
+
+This option is automatically activated if `NIT_TESTING` is set.
## SUITE GENERATION
Note: `rand` is not a recommended way to get a distinct identifier because its randomness is disabled by default. See `SRAND`.
+### `NIT_TESTING_PATH`
+
+Only available for test suites.
+Contains the module test suite path.
# SEE ALSO
nitlanguage / nit.git / blobdiff