UPDATE: We now primarily use the standard Yocto image test framework which contains several basic tests.
Please refer to the parent page for information. The test runner/exporter is part of Yocto/Poky so we're reusing that instead of test_harness.pl now.
Output using the TAP protocol for reporting could however be useful. That's something to be discussed with Yocto community.
Historical information kept here.
This test suite for the GENIVI Demo platform currently is a proof of concept.
The test suite is composed of a test harness and a number of tests to run. The harness runs the tests and reports back the results in a final report. There are currently two different tests but they're just designed to demonstrate the PoC and how tests can be added. Example tests in different programming languages will be included as well. The test harness uses a protocol called "Test Anything Protocol" or TAP. TAP is supported by many libraries in different programming languages making it quite flexible to create new tests with your favorite language. More info on TAP here: http://testanything.org/
Test suite requirements
The test suite is currently written in perl and requires a fairly new version of perl as well as potentially additional perl modules depending on your set up. If these requirements are problematic, then its possible that the test suite can be re-written in shell for easier use and deployment.
You have set up a running GDP with needed components as follows;
Using this approach to testing requires adding additional components to the GDP. This is because the required TAP functionality is not installed by default on the GDP.
To get the components we need, we'll need to edit two files;
In the first file, genivi-demo-platform, we'll need to add a recipe so that our test harness dependencies get brought in. Add this line;
to the file under IMAGE_INSTALL_append line. In the second file (bblayers.conf.sample), add this line;
to the file under the BBLAYERS line.
Note that the GDP still needs additional modules to complete the tests! This is still a work in progress.
You have configured ssh for your test user on the target and on the host for passwordless logins
- Get the test suite
The test suite is kept in a git repository along with the GENIVI Demo platform. It is available separately here: http://github.com/GENIVI/gdp-test-suite.git
- Run the tests
The tests are kept in a directory called "t". To run the test harness, which automatically scoops up the test directory, simply run
./test-harness.plwhich is a perl executable. It makes some assumptions about the environment its running so it will likely fail if it doesn't have needed dependencies. Installation information is provided.
- Analyze the output
The report is created in plain text, here's an example;
<pre> Report 09/01/15
ok 1 - require Test::Harness;
ok 2 - Perl version 5.20.0
ok 3 - Operating system is correct
ok 1 ssh to porter
ok 2 NSM is active
The report above shows TAP protocol output which is "ok" or "not ok" for passed or failed tests respectively. The first line is the report's date, then the next line (/home/jeremiah/code/perl/get_os.pl results:) holds the name of the specific test file. Below that line is the output of that test file in "ok" or "not ok" as well as the enumeration of the number of tests, in this case 3. Finally, there is a summary which lists the number of passed and failed tests.
- Test against Qemu emulated platform