Symbian3/PDK/Source/GUID-C824D5A1-3330-4293-9975-496D4D6F7D45.dita
changeset 1 25a17d01db0c
child 3 46218c8b8afa
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/Symbian3/PDK/Source/GUID-C824D5A1-3330-4293-9975-496D4D6F7D45.dita	Fri Jan 22 18:26:19 2010 +0000
@@ -0,0 +1,56 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Copyright (c) 2007-2010 Nokia Corporation and/or its subsidiary(-ies) All rights reserved. -->
+<!-- This component and the accompanying materials are made available under the terms of the License 
+"Eclipse Public License v1.0" which accompanies this distribution, 
+and is available at the URL "http://www.eclipse.org/legal/epl-v10.html". -->
+<!-- Initial Contributors:
+    Nokia Corporation - initial contribution.
+Contributors: 
+-->
+<!DOCTYPE concept
+  PUBLIC "-//OASIS//DTD DITA Concept//EN" "concept.dtd">
+<concept id="GUID-C824D5A1-3330-4293-9975-496D4D6F7D45" xml:lang="en"><title>Analyzing
+Test Results</title><abstract><p>After tests complete execution, result logs can be found on the
+system drive of the device in the <filepath>\logs\testexecute\</filepath> directory.
+On the PC, if tests are executed using testdriver, results can be found in
+the following location: </p> <p> <filepath>%EPOCROOT%\epoc32\TestDriver\results\</filepath> or </p> <p> <filepath>%EPOCROOT%\epoc32\winscw\c\logs\testexecute </filepath>  </p> <p>if executing the tests manually. </p> <p>When using testdriver,
+a new test result folder is created for each test run that is, results are
+not overwritten each time a test run is executed. A .htm log file can be found
+for each script contained in the suite, in its own subfolder. </p> <p>Each
+test case in the script will consists of test blocks (usually just one), which
+will pass or fail. Each test block will execute a number of commands. Each
+of the commands performs an action on the tested APIs (calling a certain function
+of tested classes). A test case passes if all its test blocks pass. A test
+block passes if all its commands and other instructions pass. </p> <p>The
+log will display the percentage of test cases that have passed successfully.
+The script should ideally have a 100% pass rate. </p></abstract><prolog><metadata><keywords/></metadata></prolog><conbody>
+<section><title>How to interpret test results</title>       <p>Each
+test suite produces its own HTML log file containing the results of the test
+run. When examining the log file, the results at the bottom of the log should
+be checked first. This gives a summary of the number of tests in the suite
+that have passed or failed. In the body of the log, passing tests are highlighted
+in green and failing tests in red. </p> <p>If a failing test has been identified,
+its reason for failing can be seen by examining each test block in the test,
+listed between the “<codeph>START_TESTCASE</codeph> ” and “<codeph>END_TESTCASE</codeph> ”
+tags. Starting and ending tags that is, “<codeph>START_TEST_BLOCK</codeph> ”
+and “<codeph>END_TEST_BLOCK</codeph> ” of a test block which passes will be
+reflected in green with text “<codeph>Result = PASS</codeph> ” after the “<codeph>END_TEST_BLOCK</codeph> ”.
+A failing test block will instead be in red or blue (depending on the reason
+of failure), its “<codeph>END_TEST_BLOCK</codeph> ” will be followed by “<codeph>Result
+= FAIL</codeph> ” and will refer to the reason of the failure (“<codeph>ABORT</codeph> ”
+in case of abort by timeout, “<codeph>PANIC</codeph> ” followed by a panic
+string and a code in case of an unexpected panic, etc.). If the reason of
+a failure is not displayed after the “<codeph>END_TEST_BLOCK</codeph> ” then
+it can be seen within instructions contained by the test block and/or in logs
+printed in black within the test block. Failed instruction will be displayed
+in red and will end with “<codeph>Result = FAIL</codeph> ”. </p> <p>When this
+test suite is run on a real device, some of the tests may fail due to limitations
+of the device. This will be because of the device not having the correct hardware
+(for example, no modem exists for a test which needs one), or lacking the
+right software plug-ins to run the test. This can be spotted if the log displays
+that a test has failed with a -5 error (which means <codeph>KErrNotSupported</codeph>).
+This -5 error basically means that the test has failed because the functionality
+in this test is not supported on the device used. Therefore, any tests failing
+with -5 are not due to defects, but to the limitations of the device used. </p> 
+   </section>
+</conbody></concept>
\ No newline at end of file