<?xml version="1.0" encoding="utf-8"?>

<!-- Copyright (c) 2007-2010 Nokia Corporation and/or its subsidiary(-ies) All rights reserved. -->

<!-- This component and the accompanying materials are made available under the terms of the License 

"Eclipse Public License v1.0" which accompanies this distribution, 

and is available at the URL "http://www.eclipse.org/legal/epl-v10.html". -->

<!-- Initial Contributors:

    Nokia Corporation - initial contribution.

Contributors: 

-->

<!DOCTYPE concept

  PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd">

<concept xml:lang="en" id="GUID-824CE245-C692-5B8C-8C21-18F7CB8E7B11"><title>Graphics HAI Test Overview</title><shortdesc>This topic describes the Graphics HAI test suite and explains how to set up the test environment. The tests target device driver acceptance testing. </shortdesc><prolog><metadata><keywords/></metadata></prolog><conbody><section><title>Scope</title> <p><b>Components under test </b> </p> <p>The test suite tests the <xref href="GUID-9E7D563E-9FFB-5FA9-8944-9CBAC281FDD2.dita">Screen Driver</xref> using the GDI and Colour Palette components. </p> <p><b>Test scope </b> </p> <p>The test suite is used as a validation of the Graphics Device Interface. </p> <p><b>Screen Driver testing </b> </p> <p>The test suite includes test cases for basic functionality, such as changing the display mode, draw mode and orientation. All tests are automated. For details, see <xref href="GUID-6C1CE0B7-4DA5-5F33-BCA5-6E74B645DA31.dita">Graphics Screen Driver Test Suite</xref>. </p> </section> <section id="GUID-2C5964D4-ABB4-516F-A17F-FFF30BA6F9E1"><title>Test environment</title> <p><b>Device under test setup </b> </p> <p>The device under test (hardware reference platform or phone) must contain a Symbian platform ROM. </p> <p><b>PC setup </b> </p> <p>To build and run the test code, the PC must have installed a suitable Symbian Product Developer Kit (PDK) or equivalent, the Symbian Product Developer Toolkit (PDT) and relevant compilers (such as RVCT or GCCE). </p> <p>You build and run the Graphics HAI test suite using the TEFLite framework. TEFLite (also known as TEF Lite) is a variant of the Test Execution Framework (TEF). Unlike TEF, TEFLite does not have a dependency on the Window Server. This means that tests can run on a base ROM that simply has the text shell rather than the full Window Server. TEF and its variant TEFLite are included in the PDT. </p> <p><b>Test dependencies </b> </p> <p>There are no dependencies for the test suites. </p> <p><b>Pre-build setup </b> </p> <p>The file <filepath>...\graphics\graphicsapitest\screendriverhaitest\group\device.cfg</filepath> needs to be set up for the device under test. To do this, remove the comment (double slash) on the <codeph>#define</codeph> for the device on which you want to build the tests. </p> </section> <section id="GUID-6D3C3986-814D-58F3-A702-99923486271C"><title>Building and execution</title> <p><b>Compiling the test code </b> </p> <p>Screendriver test suites can be built from: </p> <p> <filepath>...\graphics\graphicsapitest\screendriverhaitest\group\</filepath>  </p> <p>To compile the test code, perform the following steps: </p> <ol id="GUID-F51A7BE9-FCF7-5BDA-A833-0B47D68CC914"><li id="GUID-552CC891-67AD-5D17-8B6E-B0C83ED86B06"><p>Set the default device with this command: </p> <p><userinput>devices -setdefault @device-identifier</userinput> </p> <p>where <codeph>device-identifier</codeph> identifies the SDK or PDK. </p> </li> <li id="GUID-17FD930C-5CC3-545E-A9C5-7EE6205FA3A3"><p>Create your batch file and .<filepath>make</filepath> files using the following command: </p> <p><userinput>bldmake bldfiles</userinput> </p> </li> <li id="GUID-8430684F-1D5F-5BBD-BE06-8CA4EB467B15"><p>Clean up all files created in a previous build using the following command: </p> <p><userinput>abld test reallyclean</userinput> </p> </li> <li id="GUID-7FA0389B-3334-5D69-9D80-489A140864BC"><p>Build the current project using the following command: </p> <p><userinput>abld test build armv5 [urel | udeb]</userinput> </p> </li> </ol> <p><b>Building and executing the tests manually (tests built into ROM) </b> </p> <p>To build ROM image for an S60 SDK: </p> <ol id="GUID-6E38B253-4142-5F18-9741-D58CFDEC107E"><li id="GUID-AB32D829-0D99-5D03-A934-619B6AD08FE1"><p>Navigate to the directory <filepath>\epoc32\rom</filepath>. </p> </li> <li id="GUID-75BAAA62-F891-5A06-A6FB-E61467D01FA1"><p>Make sure <filepath>master.oby</filepath> includes the following: </p> <p> <codeph>#include &lt;../haitests/t_screendriver.iby&gt;</codeph>  </p> </li> <li id="GUID-8116E17B-655D-587E-B6BE-7BDFFC084DF5"><p>Run <filepath>BuildS60rom.bat</filepath>. Make sure to use the <codeph>s</codeph> key until you get the image type set to <codeph>subcon</codeph>. This file creates the image at <filepath>epoc32/rom</filepath>. </p> </li> <li id="GUID-151B44A9-9574-5C61-9BEF-68FB3F988288"><p>Ensure that the size of your image is equal to or greater than 60 MB. This is the minimum size of an image that works. </p> </li> <li id="GUID-65A15A4B-1D41-5061-9104-A508756206EF"><p>No errors should be generated to continue flashing the image. Errors are kept in s<filepath>p_rnd_imagecreation.log</filepath> file at <filepath>epoc32/rom</filepath>. </p> </li> </ol> <p>Here is the procedure for a NORMAL flashing with Phoenix software: </p> <ul><li id="GUID-C066D92F-2501-5E7F-96C4-5A9F09F70FF9"><p>Flash the MCU SW engineering English ROM image by performing the following steps: </p> <ol id="GUID-EBD86C3F-2D67-581A-B2DC-8619B37A8EA8"><li id="GUID-BE3F5159-43B8-595F-B4BF-66566F4026BE"><p>Select <b>Connection type –&gt; USB</b>. </p> </li> <li id="GUID-5EBC97AC-C0AC-5CA2-B78D-B788C7AFBA8E"><p>Select <b>File –&gt; Scan Product </b> or press <codeph>Ctrl+R</codeph>. </p> </li> <li id="GUID-251C6A60-4A72-5D65-AA0E-DB21F0CBB0C5"><p>Select<b> Flashing –&gt; Prommer tool</b>. </p> </li> <li id="GUID-D96278A6-0E1C-5A99-A40B-534A5262BDF5"><p>Select <b>Init</b>. </p> </li> <li id="GUID-1FAC6012-2CD7-5975-B3E4-0283894DA81E"><p>Select <b>Program Phone</b>. </p> </li> <li id="GUID-2F51D65A-6985-54DA-931E-3731B3E7CC66"><p>Select the correct <codeph>*.fpsx</codeph> file in the <b>MCU Image file</b> field. </p> </li> <li id="GUID-2F5EEB5E-2BEB-53DF-94FE-AC7D71AE37A0"><p>Click the <b>Flash</b> button. </p> </li> </ol> </li> <li id="GUID-20B31ED1-CC05-5144-BB01-3A5A7C2261EA"><p>Flash the Symbian user area erase file by performing the following steps: </p> <ol id="GUID-548B41FF-4EF9-5C8C-B46B-98B066ECE4B0"><li id="GUID-73895DC1-D34D-5A24-9D37-11FF6594D187"><p>Select <b>File –&gt; Scan Product</b> or press <codeph>Ctrl+R</codeph>. </p> </li> <li id="GUID-3CA3BDA6-7001-5259-A7E8-9CBDA231E580"><p>Select <b>Flashing -&gt; Prommer tool</b>. </p> </li> <li id="GUID-B64790FE-0107-502A-BF1E-742145327A9C"><p>Select <b>Init</b>. </p> </li> <li id="GUID-AB2F48B9-F715-5C70-B632-8E96782D8476"><p>Select <b>Program Phone</b>. </p> </li> <li id="GUID-1795ED62-4365-5CA4-90F0-ED2556BE0CF7"><p>Select the correct <codeph>*_udaerase.fpsx</codeph> file in the <b>MCU Image file</b> field. </p> </li> <li id="GUID-55EB6B82-1F4F-58CE-A30D-C1E764C8ABA1"><p>Click the <b>Flash</b> button. </p> </li> </ol> </li> </ul> <p><b>Executing the tests on the device </b> </p> <p>To execute the tests on the device: </p> <ol id="GUID-7934DB14-F92B-5234-B067-ACE740FBF152"><li id="GUID-B9A49F04-3F9F-5135-817E-D3722AA411AC"><p>Open the <b>eShell</b> application. </p> </li> <li id="GUID-E2B1A601-06FE-5DAC-80AA-CB4225F121FA"><p>Type <codeph>z:</codeph> to go the <codeph>z</codeph> drive. </p> </li> <li id="GUID-A8DF53F3-33EB-5DCE-9F26-0BE40E844427"><p>Type <codeph>cd &lt;test suite name&gt;</codeph> to go <codeph>&lt;test

                suite name&gt;</codeph> directory. </p> </li> <li id="GUID-032EC98A-8EF3-5103-A83A-211B7A25F880"><p>Type <codeph>t_&lt;test suite name&gt;.bat</codeph> to execute test cases of the test suite name. </p> </li> </ol> <p> <b>Example</b>: </p> <p><userinput>z:</userinput> </p> <p><userinput> cd graphics</userinput> </p> <p><userinput>t_screendriver.bat</userinput> </p> <p>The batch files must contain one or more lines like this: </p> <p> <codeph>testexecute \graphics\&lt;script name&gt;.script -tcx

          z:\graphics\t_&lt;test suite name&gt;.tcs</codeph>  </p> <p>The following code snippet shows the content of the <filepath>t_screendriver.bat</filepath> file: </p> <codeblock id="GUID-0354D419-2C05-5C23-9B53-827A92C46718" xml:space="preserve">testexecute z:\graphics\graphics-screendriver-cfbsdrawdevice-automated.script 

            -tcx z:\graphics\t_screendriver.tcs</codeblock> <p><b>Building and executing tests using TestDriver </b> </p> <ol id="GUID-465C2463-D04F-56BB-9539-7790ADE1EBF4"><li id="GUID-22F9AFFE-BC4F-5B6C-911D-B75B5DE5B743"><p> <b> Configure the TestDriver </b>  </p> <p>TestDriver can be configured using the batch file <filepath>testdriversetup.bat</filepath> located in the following location: </p> <p> <filepath>...\graphics\graphicsapitest\screendriverhaitest\group\.....</filepath> </p> <p>Run the following command: </p> <p><userinput>testdriversetup</userinput> </p> <p> <filepath>testdriversetup.bat</filepath> calls <filepath>testdriversetup.pl</filepath> which applies an appropriate set of configuration parameters to TestDriver. </p> </li> <li id="GUID-350BEB9C-39AA-540F-A788-7254A3CEAEB1"><p> <b> Build test code for TestDriver </b>  </p> <p>TestDriver commands may be executed from any location. </p> <p>To build the test code, run the following command: </p> <p><userinput>testdriver build –p &lt;armv5&gt; –b &lt;urel/udeb&gt; –s screendriverhai.[&lt;specific test suite path&gt;] </userinput> </p> <p> <i>Note</i>: Refer to the driver file structure for the possible test paths in the test suite. </p> <p>To look at the TestDriver file structure, look at the <filepath>screendriverhai.driver</filepath> file found under <filepath>...\graphics\graphicsapitest\screendriverhaitest\testsuites\graphics\...</filepath> </p> <p>Alternatively, it can be seen in the following location after building the test suite: </p> <p> <filepath> %EPOCROOT%\epoc32\testdriver\...</filepath>  </p> <p>The hierarchical test suite structure facilitates easy build and execution of the complete test suite or part of it. For example, if the path specified is <codeph>&lt;screendriverhai.screendriver&gt;</codeph>, the entire screendriver suite is executed. </p> </li> <li id="GUID-14B553E2-52E9-563D-9A68-11F663478940"><p> <b>Communication between PC and hardware</b>  </p> <p>TestDriver uses the STAT application on the device to communicate with the PC. Therefore, the device under test must have STAT installed. The device should be connected to the PC through USB, serial cable, BT or TCP/IP connection. The ROM image is built on the PC and transferred to the hardware board through the MMC card. TestDriver commands are used to execute the tests on the device. </p> <p> <b>Note</b>: For more information, see <b>Symbian Verification Suite » TestDriver V2 » TestDriver requirements » Communicating with the device » Introduction </b>. </p> </li> <li id="GUID-19AF43DC-C316-5B56-990E-1866BBEB37C9"><p> <b> Build ROM image </b>  </p> <p> <b>For an H4HRP device</b>: </p> <p> <codeph>rom –I=armv5 –v=h4hrp –b=urel –t=td_salt</codeph>  </p> <p> <b>Note</b>: STAT must be installed on the device. </p> <p> <b>For other devices (such as a prototype device)</b>: </p> <p>In <filepath>user_input.oby</filepath>, remove all references to the base test suite: binaries, DLLs, .<filepath>iby</filepath>, .<filepath>oby</filepath>.<filepath>script</filepath> and .<filepath>ini</filepath> files, (including any sub components). This excludes the test suite in the image, as TestDriver installs a SIS package for the suite automatically. </p> <p> <b>Note</b>: STAT must be installed on the device. </p> </li> <li id="GUID-F4211FF5-3235-51B0-86D5-89AA186C313E"><p> <b> Execute the tests </b>  </p> <p>Run the following command to execute the tests: </p> <p><userinput>testdriver run </userinput> </p> <p> <b>Note</b>: All scripts and test data are automatically removed from the system drive of the device under test after the execution. </p> </li> </ol> </section> <section><title>Test results</title> <p><b>Test results log file location </b> </p> <p>After tests complete execution, result logs can be found on the system drive of the device in the <filepath>\logs\testexecute\</filepath> directory. </p> <p>Each test case in the script consists of test blocks (usually just one), which may pass or fail. Each test block executes a number of commands. Each of the commands performs an action on the tested APIs (calling a certain function of tested classes). A test case passes if all its test blocks pass. A test block passes if all its commands and other instructions pass. </p> <p>The log displays the percentage of test cases that have passed successfully. Ideally the script should have a 100% pass rate. </p> <p><b>How to interpret test results </b> </p> <p>Each test suite produces its own HTML log file containing the results of the test run. When examining the log file, check the results at the bottom first. This gives a summary of the number of tests in the suite that have passed or failed. In the body of the log, passing tests are highlighted in green and failing tests in red. </p> <p>When you identify a failing test, you can find out the reason it failed by examining each test block in the test. Test blocks are listed between the <codeph>START_TESTCASE</codeph> and <codeph>END_TESTCASE</codeph> tags. In a test block that passes, these tags are green and <codeph>Result = PASS</codeph> appears after the <codeph>END_TEST_BLOCK</codeph>. A failing test block is in red or blue (depending on the reason of failure) and the <codeph>END_TEST_BLOCK</codeph> is followed by <codeph>Result = FAIL</codeph> and the reason for the failure (such as <codeph>ABORT</codeph> in case of a timeout, <codeph>PANIC</codeph> followed by a panic string and code). </p> <p>If the reason of a failure is not displayed after the <codeph>END_TEST_BLOCK</codeph> it can be seen within instructions contained by the test block and in logs printed in black within the test block. Failed instructions are displayed in red and end with <codeph>Result = FAIL</codeph>. </p> <p>When the test suite is run on a real device, some of the tests may fail due to limitations of the device. This is because of the device lacking a hardware feature or software plug-in required to run the test. This can be spotted if the log displays that a test has failed with <codeph>a -5