3 nitunit - executes the unit tests from Nit source files.
7 nitunit [*options*] FILE...
11 Unit testing in Nit can be achieved in two ways:
13 * using `DocUnits` in code comments or in markdown files
14 * using `TestSuites` with test unit files
16 `DocUnits` are executable pieces of code found in the documentation of groups, modules,
17 classes and properties.
18 They are used for documentation purpose, they should be kept simple and illustrative.
19 More advanced unit testing can be done using TestSuites.
21 `DocUnits` can also be used in any markdown files.
23 `TestSuites` are test files coupled to a tested module.
24 They contain a list of test methods called TestCase.
26 ## Working with `DocUnits`
28 DocUnits are blocks of executable code placed in comments of modules, classes and properties.
29 The execution can be verified using `assert`.
36 # assert foo.bar == 10
42 Everything used in the test must be declared.
43 To test a method you have to instantiate its class:
48 # assert foo.bar == 10
51 # assert foo.baz(1, 2) == 3
52 fun baz(a, b: Int) do return a + b
56 In a single piece of documentation, each docunit is considered a part of a single module, thus regrouped when
58 Therefore, it is possible (and recommended) to split docunits in small parts if it make the explanation easier.
61 # Some example of grouped docunits
63 # Declare and initialize a variable `a`.
67 # So the value of `a` can be used
71 # even in complex operations
77 Sometime, some blocks of code has to be included in documentation but not considered by `nitunit`.
78 Those blocks are distinguished by their tagged fences (untagged fences or fences tagged `nit` are considered to be docunits).
90 The special fence-tag `nitish` could also be used to indicate pseudo-nit that will be ignored by nitunit but highlighted by nitdoc.
91 Such `nitish` piece of code can be used to enclose examples that cannot compile or that one do not want to be automatically executed.
97 # var a: Int = someting
99 # if a == 1 then something else something-else
102 # Some code to not try to execute automatically
109 The `nitunit` command is used to test Nit files:
113 Groups (directories) can be given to test the documentation of the group and of all its Nit files:
117 Finally, standard markdown documents can be checked with:
121 When testing, the environment variable `NIT_TESTING` is set to `true`.
122 This flag can be used by libraries and program to prevent (or limit) the execution of dangerous pieces of code.
125 # NIT_TESTING is automatically set.
127 # assert "NIT_TESTING".environ == "true"
130 ## Working with `TestSuites`
132 TestSuites are Nit modules that define a set of TestCases.
134 A test suite is a module that uses the annotation `is test`.
136 It is common that a test suite focuses on testing a single module.
137 In this case, the name of the test suite is often `test_foo.nit` where `foo.nit` is the tested module.
139 The structure of a test suite is the following:
142 # test suite for module `foo`
143 module test_foo is test
145 import foo # can be intrude to test private things
150 # test case for `foo::Foo::baz`
152 var subject = new Foo
153 assert subject.baz(1, 2) == 3
158 Test suite can be executed using the same `nitunit` command:
162 `nitunit` will execute a test for each method with the `test` annotation in a class
163 also annotated with `test` so multiple tests can be executed for a single method:
170 var subject = new Foo
171 assert subject.baz(1, 2) == 3
174 var subject = new Foo
175 assert subject.baz(1, -2) == -1
182 Sometimes, it is easier to validate a `TestCase` by comparing its output with a text file containing the expected result.
184 For each TestCase `test_bar` of a TestSuite `test_mod.nit`, a corresponding file with the expected output is looked for:
186 * "test_mod.sav/test_bar.res". I.e. test-cases grouped by test-suites.
188 This is the default and is useful if there is a lot of test-suites and test-cases in a directory
190 * "sav/test_bar.res". I.e. all test-cases grouped in a common sub-directory.
192 Useful if there is a lot of test-suites OR test-cases in a directory.
194 * "test_bar.res" raw in the directory.
196 Useful is there is a few test-suites and test-cases in a directory.
198 All 3 are exclusive. If more than one exists, the test-case is failed.
200 If a corresponding file then the output of the test-case is compared with the file.
202 The `diff(1)` command is used to perform the comparison.
203 The test is failed if non-zero is returned by `diff`.
206 module test_mod is test
217 Where `test_mod.sav/test_bar.res` contains
223 If no corresponding `.res` file exists, then the output of the TestCase is ignored.
225 To helps the management of the expected results, the option `--autosav` can be used to automatically create and update them.
228 ## Configuring TestSuites
230 `TestSuite`s also provide annotations to configure the test run:
231 `before` and `after` annotations can be added to methods that must be called before/after each test case.
232 They can be used to factorize repetitive tasks:
238 var subject: Foo is noinit
240 # Method executed before each test
241 fun set_up is before do
246 assert subject.baz(1, 2) == 3
250 assert subject.baz(1, -2) == -1
255 When using custom test attributes, a empty init must be declared to allow automatic test running.
257 At class level, `before_all` and `after_all` annotations can be set on methods that must be called before/after all the test cases in the class:
263 var subject: Foo is noinit
265 # Method executed before all tests in the class
266 fun set_up is before_all do
271 assert subject.baz(1, 2) == 3
275 assert subject.baz(1, -2) == -1
280 `before_all` and `after_all` annotations can also be set on methods that must be called before/after each test suite when declared at top level:
283 module test_bdd_connector
287 # Testing the bdd_connector
290 # test cases using a server
293 # Method executed before testing the module
294 fun setup_db is before_all do
295 # start server before all test cases
298 # Method executed after testing the module
299 fun teardown_db is after_all do
300 # stop server after all test cases
304 When dealing with multiple test suites, niunit allows you to import other test suites to factorize your tests:
307 module test_bdd_users
309 import test_bdd_connector
311 # Testing the user table
314 # test cases using the db server from `test_bdd_connector`
317 fun setup_table is before_all do
321 fun teardown_table is after_all do
326 Methods with `before*` and `after*` annotations are linearized and called in different ways.
328 * `before*` methods are called from the least specific to the most specific
329 * `after*` methods are called from the most specific to the least specific
331 In the previous example, the execution order would be:
333 1. `test_bdd_connector::setup_db`
334 2. `test_bdd_users::setup_table`
335 3. `all test cases from test_bdd_users`
336 4. `test_bdd_users::teardown_table`
337 5. `test_bdd_connector::teardown_db`
339 ## Accessing the test suite environment
341 The `NIT_TESTING_PATH` environment variable contains the current test suite
343 Nitunit define this variable before the execution of each test suite.
344 It can be used to access files based on the current test suite location:
350 fun test_suite_path do
351 assert "NIT_TESTING_PATH".environ != ""
356 ## Generating test suites
358 Write test suites for big modules can be a repetitive and boring task...
359 To make it easier, `nitunit` can generate test skeletons for Nit modules:
361 $ nitunit --gen-suite foo.nit
363 This will generate the test suite `test_foo` containing test case stubs for all public
364 methods found in `foo.nit`.
370 Process also imported modules.
372 By default, only the modules indicated on the command line are tested.
374 With the `--full` option, all imported modules (even those in standard) are also precessed.
377 Output name (default is 'nitunit.xml').
379 `nitunit` produces a XML file compatible with JUnit.
382 Working directory (default is 'nitunit.out').
384 In order to execute the tests, nit files are generated then compiled and executed in the giver working directory.
386 In case of success, the directory is removed.
387 In case of failure, it is kept as is so files can be investigated.
390 nitc compiler to use.
392 By default, nitunit tries to locate the `nitc` program with the environment variable `NITC` or heuristics.
393 The option is used to indicate a specific nitc binary.
396 Does not compile and run tests.
398 ### `-p`, `--pattern`
399 Only run test case with name that match pattern.
401 Examples: `TestFoo`, `TestFoo*`, `TestFoo::test_foo`, `TestFoo::test_foo*`, `test_foo`, `test_foo*`
404 Automatically create/update .res files for black box testing.
406 If a black block test fails because a difference between the expected result and the current result then the expected result file is updated (and the test is passed).
408 If a test-case of a test-suite passes but that some output is generated, then an expected result file is created.
410 It is expected that the created/updated files are checked since the tests are considered passed.
411 A VCS like `git` is often a good tool to check the creation and modification of those files.
414 Disable time information in XML.
416 This is used to have reproducible XML results.
418 This option is automatically activated if `NIT_TESTING` is set.
423 Generate test suite skeleton for a module.
426 Force test generation even if file exists.
428 Any existing test suite will be overwritten.
431 Also generate test case for private methods.
434 Only display the skeleton, do not write any file.
437 # ENVIRONMENT VARIABLES
441 Indicate the specific Nit compiler executable to use. See `--nitc`.
445 The environment variable `NIT_TESTING` is set to `true` during the execution of program tests.
446 Some libraries of programs can use it to produce specific reproducible results; or just to exit their executions.
448 Unit-tests may unset this environment variable to retrieve the original behavior of such piece of software.
452 In order to maximize reproducibility, `SRAND` is set to 0.
453 This make the pseudo-random generator no random at all.
454 See `Sys::srand` for details.
456 To retrieve the randomness, unit-tests may unset this environment variable then call `srand`.
460 Parallel executions can cause some race collisions on named resources (e.g. DB table names).
461 To solve this issue, `NIT_TESTING_ID` is initialized with a distinct integer identifier that can be used to give unique names to resources.
463 Note: `rand` is not a recommended way to get a distinct identifier because its randomness is disabled by default. See `SRAND`.
465 ### `NIT_TESTING_PATH`
467 Only available for test suites.
468 Contains the module test suite path.
472 The Nit language documentation and the source code of its tools and libraries may be downloaded from <http://nitlanguage.org>