diff -r ebc84c812384 -r 46218c8b8afa Symbian3/PDK/Source/GUID-C824D5A1-3330-4293-9975-496D4D6F7D45.dita --- a/Symbian3/PDK/Source/GUID-C824D5A1-3330-4293-9975-496D4D6F7D45.dita Thu Mar 11 15:24:26 2010 +0000 +++ b/Symbian3/PDK/Source/GUID-C824D5A1-3330-4293-9975-496D4D6F7D45.dita Thu Mar 11 18:02:22 2010 +0000 @@ -1,56 +1,56 @@ - - - - - -Analyzing -Test Results

After tests complete execution, result logs can be found on the -system drive of the device in the \logs\testexecute\ directory. -On the PC, if tests are executed using testdriver, results can be found in -the following location:

%EPOCROOT%\epoc32\TestDriver\results\ or

%EPOCROOT%\epoc32\winscw\c\logs\testexecute

if executing the tests manually.

When using testdriver, -a new test result folder is created for each test run that is, results are -not overwritten each time a test run is executed. A .htm log file can be found -for each script contained in the suite, in its own subfolder.

Each -test case in the script will consists of test blocks (usually just one), which -will pass or fail. Each test block will execute a number of commands. Each -of the commands performs an action on the tested APIs (calling a certain function -of tested classes). A test case passes if all its test blocks pass. A test -block passes if all its commands and other instructions pass.

The -log will display the percentage of test cases that have passed successfully. -The script should ideally have a 100% pass rate.

-
How to interpret test results

Each -test suite produces its own HTML log file containing the results of the test -run. When examining the log file, the results at the bottom of the log should -be checked first. This gives a summary of the number of tests in the suite -that have passed or failed. In the body of the log, passing tests are highlighted -in green and failing tests in red.

If a failing test has been identified, -its reason for failing can be seen by examining each test block in the test, -listed between the “START_TESTCASE ” and “END_TESTCASE ” -tags. Starting and ending tags that is, “START_TEST_BLOCK ” -and “END_TEST_BLOCK ” of a test block which passes will be -reflected in green with text “Result = PASS ” after the “END_TEST_BLOCK ”. -A failing test block will instead be in red or blue (depending on the reason -of failure), its “END_TEST_BLOCK ” will be followed by “Result -= FAIL ” and will refer to the reason of the failure (“ABORT ” -in case of abort by timeout, “PANIC ” followed by a panic -string and a code in case of an unexpected panic, etc.). If the reason of -a failure is not displayed after the “END_TEST_BLOCK ” then -it can be seen within instructions contained by the test block and/or in logs -printed in black within the test block. Failed instruction will be displayed -in red and will end with “Result = FAIL ”.

When this -test suite is run on a real device, some of the tests may fail due to limitations -of the device. This will be because of the device not having the correct hardware -(for example, no modem exists for a test which needs one), or lacking the -right software plug-ins to run the test. This can be spotted if the log displays -that a test has failed with a -5 error (which means KErrNotSupported). -This -5 error basically means that the test has failed because the functionality -in this test is not supported on the device used. Therefore, any tests failing -with -5 are not due to defects, but to the limitations of the device used.

-
+ + + + + +Analyzing +Test Results

After tests complete execution, result logs can be found on the +system drive of the device in the \logs\testexecute\ directory. +On the PC, if tests are executed using testdriver, results can be found in +the following location:

%EPOCROOT%\epoc32\TestDriver\results\ or

%EPOCROOT%\epoc32\winscw\c\logs\testexecute

if executing the tests manually.

When using testdriver, +a new test result folder is created for each test run that is, results are +not overwritten each time a test run is executed. A .htm log file can be found +for each script contained in the suite, in its own subfolder.

Each +test case in the script will consists of test blocks (usually just one), which +will pass or fail. Each test block will execute a number of commands. Each +of the commands performs an action on the tested APIs (calling a certain function +of tested classes). A test case passes if all its test blocks pass. A test +block passes if all its commands and other instructions pass.

The +log will display the percentage of test cases that have passed successfully. +The script should ideally have a 100% pass rate.

+
How to interpret test results

Each +test suite produces its own HTML log file containing the results of the test +run. When examining the log file, the results at the bottom of the log should +be checked first. This gives a summary of the number of tests in the suite +that have passed or failed. In the body of the log, passing tests are highlighted +in green and failing tests in red.

If a failing test has been identified, +its reason for failing can be seen by examining each test block in the test, +listed between the “START_TESTCASE ” and “END_TESTCASE ” +tags. Starting and ending tags that is, “START_TEST_BLOCK ” +and “END_TEST_BLOCK ” of a test block which passes will be +reflected in green with text “Result = PASS ” after the “END_TEST_BLOCK ”. +A failing test block will instead be in red or blue (depending on the reason +of failure), its “END_TEST_BLOCK ” will be followed by “Result += FAIL ” and will refer to the reason of the failure (“ABORT ” +in case of abort by timeout, “PANIC ” followed by a panic +string and a code in case of an unexpected panic, etc.). If the reason of +a failure is not displayed after the “END_TEST_BLOCK ” then +it can be seen within instructions contained by the test block and/or in logs +printed in black within the test block. Failed instruction will be displayed +in red and will end with “Result = FAIL ”.

When this +test suite is run on a real device, some of the tests may fail due to limitations +of the device. This will be because of the device not having the correct hardware +(for example, no modem exists for a test which needs one), or lacking the +right software plug-ins to run the test. This can be spotted if the log displays +that a test has failed with a -5 error (which means KErrNotSupported). +This -5 error basically means that the test has failed because the functionality +in this test is not supported on the device used. Therefore, any tests failing +with -5 are not due to defects, but to the limitations of the device used.

+
\ No newline at end of file