<?xml version="1.0" encoding="utf-8"?>
<!-- Copyright (c) 2007-2010 Nokia Corporation and/or its subsidiary(-ies) All rights reserved. -->
<!-- This component and the accompanying materials are made available under the terms of the License 
"Eclipse Public License v1.0" which accompanies this distribution, 
and is available at the URL "http://www.eclipse.org/legal/epl-v10.html". -->
<!-- Initial Contributors:
    Nokia Corporation - initial contribution.
Contributors: 
-->
<!DOCTYPE concept
  PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd">
<concept id="GUID-C824D5A1-3330-4293-9975-496D4D6F7D45" xml:lang="en"><title>Analyzing
Test Results</title><abstract><p>After tests complete execution, result logs can be found on the
system drive of the device in the <filepath>\logs\testexecute\</filepath> directory.
On the PC, if tests are executed using testdriver, results can be found in
the following location: </p> <p> <filepath>%EPOCROOT%\epoc32\TestDriver\results\</filepath> or </p> <p> <filepath>%EPOCROOT%\epoc32\winscw\c\logs\testexecute </filepath>  </p> <p>if executing the tests manually. </p> <p>When using testdriver,
a new test result folder is created for each test run that is, results are
not overwritten each time a test run is executed. A .htm log file can be found
for each script contained in the suite, in its own subfolder. </p> <p>Each
test case in the script will consists of test blocks (usually just one), which
will pass or fail. Each test block will execute a number of commands. Each
of the commands performs an action on the tested APIs (calling a certain function
of tested classes). A test case passes if all its test blocks pass. A test
block passes if all its commands and other instructions pass. </p> <p>The
log will display the percentage of test cases that have passed successfully.
The script should ideally have a 100% pass rate. </p></abstract><prolog><metadata><keywords/></metadata></prolog><conbody>
<section><title>How to interpret test results</title>       <p>Each
test suite produces its own HTML log file containing the results of the test
run. When examining the log file, the results at the bottom of the log should
be checked first. This gives a summary of the number of tests in the suite
that have passed or failed. In the body of the log, passing tests are highlighted
in green and failing tests in red. </p> <p>If a failing test has been identified,
its reason for failing can be seen by examining each test block in the test,
listed between the “<codeph>START_TESTCASE</codeph> ” and “<codeph>END_TESTCASE</codeph> ”
tags. Starting and ending tags that is, “<codeph>START_TEST_BLOCK</codeph> ”
and “<codeph>END_TEST_BLOCK</codeph> ” of a test block which passes will be
reflected in green with text “<codeph>Result = PASS</codeph> ” after the “<codeph>END_TEST_BLOCK</codeph> ”.
A failing test block will instead be in red or blue (depending on the reason
of failure), its “<codeph>END_TEST_BLOCK</codeph> ” will be followed by “<codeph>Result
= FAIL</codeph> ” and will refer to the reason of the failure (“<codeph>ABORT</codeph> ”
in case of abort by timeout, “<codeph>PANIC</codeph> ” followed by a panic
string and a code in case of an unexpected panic, etc.). If the reason of
a failure is not displayed after the “<codeph>END_TEST_BLOCK</codeph> ” then
it can be seen within instructions contained by the test block and/or in logs
printed in black within the test block. Failed instruction will be displayed
in red and will end with “<codeph>Result = FAIL</codeph> ”. </p> <p>When this
test suite is run on a real device, some of the tests may fail due to limitations
of the device. This will be because of the device not having the correct hardware
(for example, no modem exists for a test which needs one), or lacking the
right software plug-ins to run the test. This can be spotted if the log displays
that a test has failed with a -5 error (which means <codeph>KErrNotSupported</codeph>).
This -5 error basically means that the test has failed because the functionality
in this test is not supported on the device used. Therefore, any tests failing
with -5 are not due to defects, but to the limitations of the device used. </p> 
   </section>
</conbody></concept>