Graphics HAI Test Overview

Scope

Components under test

The test suite tests the Screen Driver using the GDI and Colour Palette components.

Test scope

The test suite is used as a validation of the Graphics Device Interface.

Screen Driver testing

The test suite includes test cases for basic functionality, such as changing the display mode, draw mode and orientation. All tests are automated. For details, see Graphics Screen Driver Test Suite.

Test environment

Device under test setup

The device under test (hardware reference platform or phone) must contain a Symbian platform ROM.

PC setup

To build and run the test code, the PC must have installed a suitable Symbian Product Developer Kit (PDK) or equivalent, the Symbian Product Developer Toolkit (PDT) and relevant compilers (such as RVCT or GCCE).

You build and run the Graphics HAI test suite using the TEFLite framework. TEFLite (also known as TEF Lite) is a variant of the Test Execution Framework (TEF). Unlike TEF, TEFLite does not have a dependency on the Window Server. This means that tests can run on a base ROM that simply has the text shell rather than the full Window Server. TEF and its variant TEFLite are included in the PDT.

Test dependencies

There are no dependencies for the test suites.

Pre-build setup

The file ...\graphics\graphicsapitest\screendriverhaitest\group\device.cfg needs to be set up for the device under test. To do this, remove the comment (double slash) on the #define for the device on which you want to build the tests.

Building and execution

Compiling the test code

Screendriver test suites can be built from:

...\graphics\graphicsapitest\screendriverhaitest\group\

To compile the test code, perform the following steps:

1. Set the default device with this command:

   devices -setdefault @device-identifier

   where device-identifier identifies the SDK or PDK.

2. Create your batch file and .make files using the following command:

   bldmake bldfiles

3. Clean up all files created in a previous build using the following command:

   abld test reallyclean

4. Build the current project using the following command:

   abld test build armv5 [urel | udeb]

Building and executing the tests manually (tests built into ROM)

To build ROM image for an S60 SDK:

1. Navigate to the directory \epoc32\rom.

2. Make sure master.oby includes the following:

   #include <../haitests/t_screendriver.iby>

3. Run BuildS60rom.bat. Make sure to use the s key until you get the image type set to subcon. This file creates the image at epoc32/rom.

4. Ensure that the size of your image is equal to or greater than 60 MB. This is the minimum size of an image that works.

5. No errors should be generated to continue flashing the image. Errors are kept in sp_rnd_imagecreation.log file at epoc32/rom.

Here is the procedure for a NORMAL flashing with Phoenix software:

• Flash the MCU SW engineering English ROM image by performing the following steps:

  1. Select Connection type –> USB.

  2. Select File –> Scan Product or press Ctrl+R.

  3. Select Flashing –> Prommer tool.

  4. Select Init.

  5. Select Program Phone.

  6. Select the correct *.fpsx file in the MCU Image file field.

  7. Click the Flash button.

• Flash the Symbian user area erase file by performing the following steps:

  1. Select File –> Scan Product or press Ctrl+R.

  2. Select Flashing -> Prommer tool.

  3. Select Init.

  4. Select Program Phone.

  5. Select the correct *_udaerase.fpsx file in the MCU Image file field.

  6. Click the Flash button.

Executing the tests on the device

To execute the tests on the device:

1. Open the eShell application.

2. Type z: to go the z drive.

3. Type cd <test suite name> to go <test suite name> directory.

4. Type t_<test suite name>.bat to execute test cases of the test suite name.

Example:

z:

cd graphics

t_screendriver.bat

The batch files must contain one or more lines like this:

testexecute \graphics\<script name>.script -tcx z:\graphics\t_<test suite name>.tcs

The following code snippet shows the content of the t_screendriver.bat file:

testexecute z:\graphics\graphics-screendriver-cfbsdrawdevice-automated.script 
            -tcx z:\graphics\t_screendriver.tcs

Building and executing tests using TestDriver

1. Configure the TestDriver

   TestDriver can be configured using the batch file testdriversetup.bat located in the following location:

   ...\graphics\graphicsapitest\screendriverhaitest\group\.....

   Run the following command:

   testdriversetup

   testdriversetup.bat calls testdriversetup.pl which applies an appropriate set of configuration parameters to TestDriver.

2. Build test code for TestDriver

   TestDriver commands may be executed from any location.

   To build the test code, run the following command:

   testdriver build –p <armv5> –b <urel/udeb> –s screendriverhai.[<specific test suite path>]

   Note: Refer to the driver file structure for the possible test paths in the test suite.

   To look at the TestDriver file structure, look at the screendriverhai.driver file found under ...\graphics\graphicsapitest\screendriverhaitest\testsuites\graphics\...

   Alternatively, it can be seen in the following location after building the test suite:

   %EPOCROOT%\epoc32\testdriver\...

   The hierarchical test suite structure facilitates easy build and execution of the complete test suite or part of it. For example, if the path specified is <screendriverhai.screendriver>, the entire screendriver suite is executed.

3. Communication between PC and hardware

   TestDriver uses the STAT application on the device to communicate with the PC. Therefore, the device under test must have STAT installed. The device should be connected to the PC through USB, serial cable, BT or TCP/IP connection. The ROM image is built on the PC and transferred to the hardware board through the MMC card. TestDriver commands are used to execute the tests on the device.

   Note: For more information, see Symbian Verification Suite » TestDriver V2 » TestDriver requirements » Communicating with the device » Introduction .

4. Build ROM image

   For an H4HRP device:

   rom –I=armv5 –v=h4hrp –b=urel –t=td_salt

   Note: STAT must be installed on the device.

   For other devices (such as a prototype device):

   In user_input.oby, remove all references to the base test suite: binaries, DLLs, .iby, .oby.script and .ini files, (including any sub components). This excludes the test suite in the image, as TestDriver installs a SIS package for the suite automatically.

   Note: STAT must be installed on the device.

5. Execute the tests

   Run the following command to execute the tests:

   testdriver run

   Note: All scripts and test data are automatically removed from the system drive of the device under test after the execution.

Test results

Test results log file location

After tests complete execution, result logs can be found on the system drive of the device in the \logs\testexecute\ directory.

Each test case in the script consists of test blocks (usually just one), which may pass or fail. Each test block executes a number of commands. Each of the commands performs an action on the tested APIs (calling a certain function of tested classes). A test case passes if all its test blocks pass. A test block passes if all its commands and other instructions pass.

The log displays the percentage of test cases that have passed successfully. Ideally the script should have a 100% pass rate.

How to interpret test results

Each test suite produces its own HTML log file containing the results of the test run. When examining the log file, check the results at the bottom first. This gives a summary of the number of tests in the suite that have passed or failed. In the body of the log, passing tests are highlighted in green and failing tests in red.

When you identify a failing test, you can find out the reason it failed by examining each test block in the test. Test blocks are listed between the START_TESTCASE and END_TESTCASE tags. In a test block that passes, these tags are green and Result = PASS appears after the END_TEST_BLOCK. A failing test block is in red or blue (depending on the reason of failure) and the END_TEST_BLOCK is followed by Result = FAIL and the reason for the failure (such as ABORT in case of a timeout, PANIC followed by a panic string and code).

If the reason of a failure is not displayed after the END_TEST_BLOCK it can be seen within instructions contained by the test block and in logs printed in black within the test block. Failed instructions are displayed in red and end with Result = FAIL.

When the test suite is run on a real device, some of the tests may fail due to limitations of the device. This is because of the device lacking a hardware feature or software plug-in required to run the test. This can be spotted if the log displays that a test has failed with a -5 (KerrNotSupported) error. This error means that the test has failed because the functionality in this test is not supported on the device used. It is important that you check that this functionality is supported by the device prior to raising any defects. These kinds of tests are excluded in the test suite by using TCS files. For more information, see the TCS file located in TestData directory.