| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
gnattest
gnattest is an ASIS-based utility that creates unit-test stubs
as well as a test driver infrastructure (harness). gnattest creates
a stub for each visible subprogram in the packages under consideration when
they do not exist already.
In order to process source files from a project, gnattest has to
semantically analyze the sources. Therefore, test stubs can only be
generated for legal Ada units. If a unit is dependent on other units,
those units should be among the source files of the project or of other projects
imported by this one.
Generated stubs and harnesses are based on the AUnit testing framework. AUnit is
an Ada adaptation of the xxxUnit testing frameworks, similar to JUnit for Java
or CppUnit for C++. While it is advised that gnattest users read the AUnit
manual, deep knowledge of AUnit is not necessary for using gnattest. For correct
operation of gnattest, AUnit should be installed and aunit.gpr must be
on the project path. This happens automatically when Aunit is installed at its
default location.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
gnattest
gnattest has a command-line interface of the form
$ gnattest -Pprojname [--harness-dir=dirname] [switches] [filename] [-cargs gcc_switches] |
where
gcc. These switches will be passed on to all compiler invocations
made by gnatstub to generate a set of ASIS trees. Here you can provide
`-I' switches to form the source search path,
use the `-gnatec' switch to set the configuration file,
use the `-gnat05' switch if sources should be compiled in
Ada 2005 mode, etc.
gnattest results can be found in two different places.
test_driver.gpr. Tests can be compiled and run using a command
such as:
gnatmake -P<harness-dir>/test_driver test_runner |
Note that you might need to specify the necessary values of scenario variables when you are not using the AUnit defaults.
Note that if the project already has both my_unit.ads and my_unit-tests.ads, this will cause a name conflict with the generated test package.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
gnattest
gnattest creates a harness for all
sources, treating them as test packages.
`--separate_root' and `--subdir' switches are mutually exclusive.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
gnattest
Most of the command-line options can also be passed to the tool by adding special attributes to the project file. Those attributes should be put in package gnattest. Here is the list of attributes:
Each of those attributes can be overridden from the command line if needed.
Other gnattest switches can also be passed via the project
file as an attribute list called GNATtest_Switches.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Let's take a very simple example using the first gnattest example
located in:
<install_prefix>/share/examples/gnattest/simple |
This project contains a simple package containing one subprogram. By running gnattest:
$ gnattest --harness-dir=driver -Psimple.gpr |
a test driver is created in directory "driver". It can be compiled and run:
$ cd driver $ gprbuild -Ptest_driver $ test_runner |
One failed test with diagnosis "test not implemented" is reported. Since no special output option was specified, the test package Simple.Tests is located in:
<install_prefix>/share/examples/gnattest/simple/src/tests |
For each package containing visible subprograms, a child test package is generated. It contains one test routine per tested subprogram. Each declaration of a test subprogram has a comment specifying which tested subprogram it corresponds to. All of the test routines have separate bodies. The test routine located at simple-tests-test_inc_5eaee3.adb contains a single statement: a call to procedure Assert. It has two arguments: the Boolean expression we want to check and the diagnosis message to display if the condition is false.
That is where actual testing code should be written after a proper setup. An actual check can be performed by replacing the Assert call with:
Assert (Inc (1) = 2, "wrong incrementation"); |
After recompiling and running the test driver, one successfully passed test is reported.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Besides test routines themselves, each test package has an inner package Env_Mgmt that has two procedures: User_Set_Up and User_Tear_Down. User_Set_Up is called before each test routine of the package and User_Tear_Down is called after each test routine. Those two procedures can be used to perform necessary initialization and finalization, memory allocation, etc.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Bodies of test routines and env_mgmt packages are never overridden after they have been created once. As long as the name of the subprogram, full expanded Ada names, and the order of its parameters is the same, the old test routine will fit in its place and no test stub will be generated for the subprogram.
This can be demonstrated with the previous example. By uncommenting declaration
and body of function Dec in simple.ads and simple.adb, running
gnattest on the project, and then running the test driver:
gnattest --harness-dir=driver -Psimple.gpr cd driver gprbuild -Ptest_driver test_runner |
the old test is not replaced with a stub, nor is it lost, but a new test stub is created for function Dec.
The only way of regenerating tests stubs is to remove the previously created tests.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The generated test driver can treat unimplemented tests in two ways: either count them all as failed (this is useful to see which tests are still left to implement) or as passed (to sort out unimplemented ones from those actually failing).
The test driver accepts a switch to specify this behavior: --stub-default=val,
where val is either "pass" or "fail" (exactly as for gnattest).
The default behavior of the test driver is set with the same switch as passed to gnattest when generating the test driver.
Passing it to the driver generated on the first example:
test_runner --stub-default=pass |
makes both tests pass, even the unimplemented one.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Creation of test stubs for primitive operations of tagged types entails a number of features. Test routines for all primitives of a given tagged type are placed in a separate child package named according to the tagged type. For example, if you have tagged type T in package P, all tests for primitives of T will be in P.T_Tests.
Consider running gnattest on the second example (note: actual tests for this example already exist, so there's no need to worry if the tool reports that no new stubs were generated):
cd <install_prefix>/share/examples/gnattest/tagged_rec gnattest --harness-dir=driver -Ptagged_rec.gpr |
Taking a closer look at the test type declared in the test package Speed1.Controller_Tests is necessary. It is declared in:
<install_prefix>/share/examples/gnattest/tagged_rec/src/tests |
Test types are direct or indirect descendants of AUnit.Test_Fixtures.Test_Fixture type. In the case of nonprimitive tested subprograms, the user doesn't need to be concerned with them. However, when generating test packages for primitive operations, there are some things the user needs to know.
Type Test_Controller has components that allow assignment of various derivations of type Controller. And if you look at the specification of package Speed2.Auto_Controller, you will see that Test_Auto_Controller actually derives from Test_Controller rather than AUnit type Test_Fixture. Thus, test types mirror the hierarchy of tested types.
The User_Set_Up procedure of Env_Mgmt package corresponding to a test package of primitive operations of type T assigns to Fixture a reference to an object of that exact type T. Notice, however, that if the tagged type has discriminants, the User_Set_Up only has a commented template for setting up the fixture, since filling the discriminant with actual value is up to the user.
The knowledge of the structure of test types allows additional testing without additional effort. Those possibilities are described below.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Since the test type hierarchy mimics the hierarchy of tested types, the inheritance of tests takes place. An example of such inheritance can be seen by running the test driver generated for the second example. As previously mentioned, actual tests are already written for this example.
cd driver gprbuild -Ptest_driver test_runner |
There are 6 passed tests while there are only 5 testable subprograms. The test routine for function Speed has been inherited and run against objects of the derived type.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Tagged Types Substitutability Testing is a way of verifying the Liskov substitution principle (LSP) by testing. LSP is a principle stating that if S is a subtype of T (in Ada, S is a derived type of tagged type T), then objects of type T may be replaced with objects of type S (that is, objects of type S may be substituted for objects of type T), without altering any of the desirable properties of the program. When the properties of the program are expressed in the form of subprogram preconditions and postconditions (let's call them pre and post), LSP is formulated as relations between the pre and post of primitive operations and the pre and post of their derived operations. The pre of a derived operation should not be stronger than the original pre, and the post of the derived operation should not be weaker than the original post. Those relations ensure that verifying if a dispatching call is safe can be done just by using the pre and post of the root operation.
Verifying LSP by testing consists of running all the unit tests associated with the primitives of a given tagged type with objects of its derived types.
In the example used in the previous section, there was clearly a violation of LSP. The overriding primitive Adjust_Speed in package Speed2 removes the functionality of the overridden primitive and thus doesn't respect LSP. Gnattest has a special option to run overridden parent tests against objects of the type which have overriding primitives:
gnattest --harness-dir=driver --liskov -Ptagged_rec.gpr cd driver gprbuild -Ptest_driver test_runner |
While all the tests pass by themselves, the parent test for Adjust_Speed fails against objects of the derived type.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
gnattest supports pragmas Precondition, Postcondition, and Test_Case.
Test routines are generated, one per each Test_Case associated with a tested
subprogram. Those test routines have special wrappers for tested functions
that have composition of pre- and postcondition of the subprogram with
"requires" and "ensures" of the Test_Case (depending on the mode, pre and post
either count for Nominal mode or do not count for Robustness mode).
The third example demonstrates how this works:
cd <install_prefix>/share/examples/gnattest/contracts gnattest --harness-dir=driver -Pcontracts.gpr |
Putting actual checks within the range of the contract does not cause any error reports. For example, for the test routine which corresponds to test case 1:
Assert (Sqrt (9.0) = 3.0, "wrong sqrt"); |
and for the test routine corresponding to test case 2:
Assert (Sqrt (-5.0) = -1.0, "wrong error indication"); |
are acceptable:
cd driver gprbuild -Ptest_driver test_runner |
However, by changing 9.0 to 25.0 and 3.0 to 5.0, for example, you can get a precondition violation for test case one. Also, by using any otherwise correct but positive pair of numbers in the second test routine, you can also get a precondition violation. Postconditions are checked and reported the same way.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
gnattest can add user-written tests to the main suite of the test
driver. gnattest traverses the given packages and searches for test
routines. All procedures with a single in out parameter of a type which is
derived from AUnit.Test_Fixtures.Test_Fixture and that are declared in package
specifications are added to the suites and are then executed by the test driver.
(Set_Up and Tear_Down are filtered out.)
An example illustrates two ways of creating test harnesses for user-written tests. Directory additional_tests contains an AUnit-based test driver written by hand.
<install_prefix>/share/examples/gnattest/additional_tests/ |
To create a test driver for already-written tests, use the --harness-only option:
gnattest -Padditional/harness/harness.gpr --harness-dir=harness_only \ --harness-only gnatmake -Pharness_only/test_driver.gpr harness_only/test_runner |
Additional tests can also be executed together with generated tests:
gnattest -Psimple.gpr --additional-tests=additional/harness/harness.gpr \ --harness-dir=mixing gnatmake -Pmixing/test_driver.gpr mixing/test_runner |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
gnattest can be used to generate the test harness for platforms
and run-time libraries others than the default native target with the
default full run-time. For example, when using a limited run-time library
such as Zero FootPrint (ZFP), a simplified harness is generated.
Two variables are used to tell the underlying AUnit framework how to generate
the test harness: PLATFORM, which identifies the target, and
RUNTIME, used to determine the run-time library for which the harness
is generated. For example, the following options are used to generate the
AUnit test harness for a PowerPC ELF target using the ZFP run-time library:
gnattest -Psimple.gpr -XPLATFORM=powerpc-elf -XRUNTIME=zfp |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The tool currently does not support following features:
| [ << ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |