diff -r ebc84c812384 -r 46218c8b8afa Symbian3/PDK/Source/GUID-3C26B5B2-1B90-5F96-9342-1D53F6EDD94F.dita --- a/Symbian3/PDK/Source/GUID-3C26B5B2-1B90-5F96-9342-1D53F6EDD94F.dita Thu Mar 11 15:24:26 2010 +0000 +++ b/Symbian3/PDK/Source/GUID-3C26B5B2-1B90-5F96-9342-1D53F6EDD94F.dita Thu Mar 11 18:02:22 2010 +0000 @@ -1,526 +1,526 @@ - - - - - -F32 Test -Technology DescriptionDescribes the high level test philosophy, scope and the types of -test used in F32 technology. -

This topic provides a high level information about the Base test technology -area. It also provides an overview of the test technology area, its coverage, -test environment setup and how to find further information about the test -suites offered by this test technology area.

-

The aim is to provide and deliver a product quality set of test suites -that can be used internally within Symbian and by external groups who have -access to the Symbian test source code.

-
Scope

Components under test

The Base subsystem is the lowest layer -in the Symbian platform software stack. It contains the core components, such -as the kernel, user library, file server, and device drivers, for every supported -hardware platform.

This suite tests the F32_EKA2 (file server) and -FAT32 components.

Test -types

The suite has three sub-suites: validation, performance -and conformance.

Validation -test suite

The Validation -Test Suite provides a series of tests to verify that published APIs -perform as described in the Symbian platform library. The tests are intended -for regression and compatibility verification.

The validation suite -uses Dual Driver tests to test the dual driver feature on the phone. DUAL -Drive tests are configured to run on two drives of the same type having common -interface.

Performance -Test Suite

The Performance -Test Suite is intended for assessing and benchmarking file server performance.

Conformance Test Suite

The FAT32 -Conformance Test Suite ensures the file server's compliance with FAT32.

-
Test Environment

Device Setup

The device under test (a hardware platform, -a phone or an emulator) must contain an appropriate Symbian platform build -or licensee UI build.

PC -Setup

The PC must have a Symbian Developer Kit (or licensee equivalent), -an PDT and appropriate compilers to build the test code (for example Code -Warrior, RVCT or GCCE).

Test -Dependencies

The base test suite has no dependencies other than -those common to all test suites.

Any mention of “epoc32” is -offset by %EPOCROOT%, the full path where the epoc tree is installed.

Building and Execution

For -general information about test suite structure refer to Symbian Verification -Suite » TestDriver.

The following illustrates the F32 test suite -structure:

<Component> - base - <Validation suite> - validation - <Sub-component> - f32 - <Manual tests> - f32-manual - <RAM-drive specific tests> - ram-manual - <Suite> - pbase-f32-sfsrv-publicapi-rem-manual-suite - <Dual Drive Tests>- f32-dualdrive> - <Suite> - pbase-f32-sfsrv-Publicapi-ram-DualDrive-suite - <Automated tests> - f32-automated - <Suite> - pbase-f32-sfsrv-publicapi-any-suite - <System drive specific tests> - os - <Suite> - pbase-f32-sfsrv-publicapi-os-suite - <ROM-drive specific tests> - rom - <Suite> - pbase-f32-publicapi-sfsrv-rom-suite - <NAND-drive specific tests> - nand - <Suite> - pbase-f32-sfsrv-publicapi-nand-suite - <RAM-drive specific tests> - ram - <Non-removable RAM-drive specific tests> - fixed - <Suite> - pbase-f32-publicapi-sfsrv-ram-suite - <Suite> - pbase-f32-publicapi-sfsrv-rem-suite - <Performance suite> - performance - <Sub-component> - f32 - <Scenario variant> - small - <Suite> - pbase-f32-rfile-performance-small - <Suite> - pbase-f32-rfs-performance-small - <Scenario variant> - medium - <Suite> - pbase-f32-rfile-performance-medium - <Suite> - pbase-f32-rfs-performance-medium - <Scenario variant> - large - <Suite> - pbase-f32-rfile-performance-large - <Suite> - pbase-f32-rfs-performance-large - <Conformance suite> - conformance - <Sub-component> - f32 - <Sub-component> - FAT32 - <Suite> - f32-fat32-conformance

This structure can -be seen in the following location after building the test suite: %EPOCROOT%\epoc32\testdriver\testproduct.

The -hierarchical test suite structure facilitates easy build and execution of -the complete test suite or part of it. For example, if the path specified -is:

    -
  • base: -The entire suite is built/run.

  • -
  • base.validation.f32: -All the suites F32 are built/run.

  • -
  • base.validation.f32.f32-automated: -Only F32 automated test is built/run.

  • -
  • base.validation.f32.f32-manual: -Only F32 manual test is built/run.

  • -
  • base.validation.f32.f32-dualdrive: -Only f32 dual drive tests is built/run.

  • -
  • base.validation.f32.f32-automated.ram: -Only tests on RAM are built/run.

  • -
  • base.validation.f32.f32-automated.ram.fixed: -Only tests on non-removable RAM are built/run.

  • -
  • base.validation.f32.f32-automated.rom: -Only tests on ROM are built/run.

  • -
  • base.validation.f32.f32-automated.os: -Only OS suite is built/run.

  • -
  • base.validation.f32.f32-automated.ram.fixed: -Only tests on non-removable RAM are built/run.

  • -

General -environment setup

    -
  1. Refer to specific sub-component -test suite description in Base -Verification Suite for details about configuration and building additional -components before building the test suite.

  2. -
  3. The entire Base Test -product can be built from ...\baseapitest\basesvs\group\.

  4. -
  5. Individual test components -can be built from the respective group directories. For example, ...\baseapitest\basesvs\validation\f32\group\.

    bldmake bldfiles

    abld test -build <WINSCW/ARMV5> <udeb/urel>

  6. -

Steps -to build and execute tests using TestDriver

Communication between -the PC and the device under test (Symbian platform device) is enabled using -STAT. So, the device under test must have STAT installed.

For WINSCW platform:

    -
  1. Configure TestDriver -using the following script:

    ...\baseapitest\basesvs\group\testdriversetup.bat

  2. -
  3. Build the test code:

    testdriver -build –p <winscw/armv5> –b <urel/udeb> –s base[.<specific test suite -path>]

    Testdriver commands can be run from any location.

  4. -
  5. To execute the Tests -:

    testdriver run

  6. -

For ARMV5 platform:

Communication between PC and hardware:

The -hardware reference board (for example H4) must be connected to the PC through -USB or serial cable. The ROM image is built on the PC and transferred to the -hardware board through MMC card.

    -
  1. Configure TestDriver -using the following script:

    ...\baseapitest\basesvs\group\testdriversetup.bat

  2. -
  3. Build the test -code:

    testdriver build –p <winscw/armv5> –b <urel/udeb> -–s base[.<specific test suite path>]

    Testdriver commands -can be run from any location.

  4. -
  5. Build the ROM image:

    buildrom --D_STARTUPMODE2 -D_EABI=ARMV5 -DRVCT -DUSE_STRONG_CRYPTOGRAPHY <h2/h4hrp> -techview_statapi td_base

    Note The ROM building -commands are different for NAND images and enabling Dual Drive.

  6. -
  7. Transfer the ROM to -the device and start-up.

  8. -
  9. Execute the Tests:

    testdriver -run -t <transport service>

    Transport service may be -“serial<COM port number that is connected to the board>”, “bt<COM port -number for BT dongle>” or “usb<mapped port number>”

  10. -

All scripts and test data are automatically removed from -the system drive of the device under test after the execution.

Steps to build and execute -tests manually (Tests built into ROM)

For WINSCW platform:

    -
  1. Select your Setup -Script:

    For example setup-PBASE-F32-Sfsrv-PublicApi.script is -used to run only the F32_EKA2 validation test suite.

  2. -
  3. Change directory -to:

    %EPOCROOT%\epoc32\release\winscw\<urel/udeb>

  4. -
  5. Execute the Tests:

    testexecute -z:\base\setup_<test suite name>.script

    setup_t_base.script is -used to copy the test data files on to c:\drive.

    testexecute -c:\base\<test suite name>.script

    t_base.script is -used to run the entire base test suite.

  6. -

For ARMV5 platform:

    -
  1. Build ROM image:

    buildrom --D_EABI=ARMV5 –DRVCT -DUSE_STRONG_CRYPTOGRAPHY <h2/h4hrp> techview t_base

    The -ROM image is built on the PC and transferred to the hardware board through -MMC card. Note: The ROM building commands is different for NAND images. -Refer to Base Verification -Suite for more details.

  2. -
  3. Select your Setup -Script:

    The setup-PBASE-F32-Sfsrv-PublicApi.script is -used to run only F32_EKA2 validation test suite.

  4. -
  5. Running Scripts:

    Open -eshell

  6. -
  7. Execute the Tests:

    testexecute -z:\base\setup_<test suite name>.script

    setup_t_base.script is -used to copy the test data files on to c:\drive.

    testexecute -c:\base\<test suite name>.script

    t_base.script is -used to run the entire base test suite.

  8. -

Steps -to build and execute tests on phone directly using .pkg file

For ARMV5 platform:

    -
  1. EShell and TestExecute:

    The -phone must have eshell and testexecute installed.

  2. -
  3. Certificate files:

    Ensure .cer and .key files are placed into /epoc32/pkg/ directory on the -PC and rename aszeus.cer and SymbianACS.key correspondingly.

  4. -
  5. Create and install -SIS-file:

    Run makesisfiles.bat from /epoc32/pkg/ directory.

    makesisfiles.bat

    This creates t_sfsrv.sis file. The file must -be copied to an MMC card, transferred to the phone and executed.

    When -the t_sfsrv.sis file is run, the base test suite is installed -on the device.

  6. -
  7. Running Scripts:

    Run -the test scripts. For more information refer to Steps -to build and execute tests manually (Tests built into ROM).

  8. -

Selective -Execution of Test Cases using TCS File

Test cases can be excluded -from a test run by using a .tcs file. The .tcs file -contains IDs of test cases to be excluded.

The test case IDs can be -specified as a range, a list, or a combination of both. Each range or individual -test case ID is delimited by a new-line character.

For example, range -of test case IDs to be excluded can be specified as:

pbase-f32-rfile-performance-large-0001: - pbase-f32-rfile-performance-large-0005

This -excludes all test cases from 1 to 5 (1 and 5 included) during execution.

A -list of test cases IDs to be excluded can be:

COMINF-ESOCK-RSubConnection-PublicAPI-0001

COMINF-ESOCK-RSubConnection-PublicAPI-0005

COMINF-ESOCK-RSubConnection-PublicAPI-0007

A -combination of range and list can be:

pbase-f32-rfile-performance-large-0001: - pbase-f32-rfile-performance-large-0005

pbase-f32-rfile-performance-large-0010: - pbase-f32-rfile-performance-large-0015

pbase-f32-rfile-performance-large-0020

pbase-f32-rfile-performance-large-0021

There is only one .tcs file located at …\baseapitest\basesvs\config\t_base.tcs

For -more information on location of .tcs files, see TCS -File Source Location in F32 Performance Test Suite, TCS File Source Location in FAT32 Conformance Test Suite, and TCS -File Source Location in F32_EKA2 Validation Test Suite.

The .tcs file -must be modified when using the TestDriver for test execution. When executing -the tests manually, the following testexecute command illustrates -the usage of the .tcs file:

testexecute -%EPOCROOT%\epoc32\release\winscw\<urel/udeb>\<test suite name>.script -–[tcx|tci] %EPOCROOT%\epoc32\release\winscw\<urel/udeb>\t_base.tcs

The -tcx switch -is used to execute all the tests in the specified script excluding the test -cases specified in the .tcs file. The –tci switch -is used to execute only the tests specified in the .tcs file -from the specified script.

Test -Results

Test -Results log file location

After tests complete execution, result -logs can be found on the system drive of the device in the \logs\testexecute\ directory.

On -the PC, if tests are run using testdriver, results can be found in the following -location: %EPOCROOT%\epoc32\TestDriver\results\. If executing -the tests manually the results can be found in the following location: %EPOCROOT%\epoc32\winscw\c\logs\testexecute

When -using testdriver, a new test result folder is created for each test run that -is, results are not overwritten each time a test run is run. A .htm log -file for each script contained in the suite can be found in its own subfolder.

Each -test case in the script consists of test blocks (usually just one), which -either pass or fail. Each test block runs some of the commands. Each of the -commands performs an action on the tested APIs. A test block passes if all -its commands and other instructions pass. A test case passes if all its test -blocks pass.

The log displays the percentage of test cases that have -passed successfully.

How -to interpret test results

Each test suite produces its own HTML -log file containing the results of the test run. When examining the log file, -the results at the bottom of the log should be checked first. This gives a -summary of the tests in the suite that have passed, failed or panicked.

In -the body of the log tests which passed are highlighted in green, tests which -failed in red and tests which caused a panic in blue. The reason for any failure -or panic is also given.

When this test suite is run on a real device -some tests may fail due to limitations of the device. Either the device does -not have the correct hardware (for example, no modem exists for a test which -needs one) or it lacks the right software plug-ins to run the test. In such -cases the log displays that a test has failed with a -5 error (which means KErrNotSupported). -Tests appearing to fail with -5 do not indicate defects, but limitations of -the device.

Refer to Symbian Verification Suite » TestDriver » -Using TestDriver » Results » Viewing the test results for more information -on test driver results.

-
OS Focussed -Components

The following tables illustrate the components and classes -tested on each Symbian platform baseline:

Table 1: Validation -testing of Base, F32_EKA2 component on each Symbian platform baseline

- - - - - 9.1 -9.2 -9.3 -9.4 -9.5 - - - RFs - - - CDir - - - CDirScan - - - CFileBase - - - CFileMan - - - MFileManObserver - - - RDir - - - RFile - - - RFormat - - - RRawDisk - - - TDriveUnit - - - TEntry - - - TEntryArray - - - TFindFile - - - TFileText - - - TOpenFileScan - - - TParse - - - TParseBase - - - TParsePtr - - - TparsePtrC - - - TVolumeInfo - - - FileNamesIdentical() - - - - -

X

-

X

-

X

-

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - - - -

X

-

X

-
- - -

Table 2: Performance testing of Base, F32_EKA2 on each Symbian -platform baseline

- - - - - 9.1 -9.2 -9.3 -9.4 -9.5 - - -RFs - - -RFile - - - - -

X

-

X

-

X

-

X

-

X

-
- -

X

-

X

-

X

-

X

-

X

-
- - -

The FAT32 conformance test suite is not API specific and is available -on Symbian platform versions 9.4 and 9.5.

+ + + + + +F32 Test +Technology DescriptionDescribes the high level test philosophy, scope and the types of +test used in F32 technology. +

This topic provides a high level information about the Base test technology +area. It also provides an overview of the test technology area, its coverage, +test environment setup and how to find further information about the test +suites offered by this test technology area.

+

The aim is to provide and deliver a product quality set of test suites +that can be used internally within Symbian and by external groups who have +access to the Symbian test source code.

+
Scope

Components under test

The Base subsystem is the lowest layer +in the Symbian platform software stack. It contains the core components, such +as the kernel, user library, file server, and device drivers, for every supported +hardware platform.

This suite tests the F32_EKA2 (file server) and +FAT32 components.

Test +types

The suite has three sub-suites: validation, performance +and conformance.

Validation +test suite

The Validation +Test Suite provides a series of tests to verify that published APIs +perform as described in the Symbian platform library. The tests are intended +for regression and compatibility verification.

The validation suite +uses Dual Driver tests to test the dual driver feature on the phone. DUAL +Drive tests are configured to run on two drives of the same type having common +interface.

Performance +Test Suite

The Performance +Test Suite is intended for assessing and benchmarking file server performance.

Conformance Test Suite

The FAT32 +Conformance Test Suite ensures the file server's compliance with FAT32.

+
Test Environment

Device Setup

The device under test (a hardware platform, +a phone or an emulator) must contain an appropriate Symbian platform build +or licensee UI build.

PC +Setup

The PC must have a Symbian Developer Kit (or licensee equivalent), +an PDT and appropriate compilers to build the test code (for example Code +Warrior, RVCT or GCCE).

Test +Dependencies

The base test suite has no dependencies other than +those common to all test suites.

Any mention of “epoc32” is +offset by %EPOCROOT%, the full path where the epoc tree is installed.

Building and Execution

For +general information about test suite structure refer to Symbian Verification +Suite » TestDriver.

The following illustrates the F32 test suite +structure:

<Component> - base + <Validation suite> - validation + <Sub-component> - f32 + <Manual tests> - f32-manual + <RAM-drive specific tests> - ram-manual + <Suite> - pbase-f32-sfsrv-publicapi-rem-manual-suite + <Dual Drive Tests>- f32-dualdrive> + <Suite> - pbase-f32-sfsrv-Publicapi-ram-DualDrive-suite + <Automated tests> - f32-automated + <Suite> - pbase-f32-sfsrv-publicapi-any-suite + <System drive specific tests> - os + <Suite> - pbase-f32-sfsrv-publicapi-os-suite + <ROM-drive specific tests> - rom + <Suite> - pbase-f32-publicapi-sfsrv-rom-suite + <NAND-drive specific tests> - nand + <Suite> - pbase-f32-sfsrv-publicapi-nand-suite + <RAM-drive specific tests> - ram + <Non-removable RAM-drive specific tests> - fixed + <Suite> - pbase-f32-publicapi-sfsrv-ram-suite + <Suite> - pbase-f32-publicapi-sfsrv-rem-suite + <Performance suite> - performance + <Sub-component> - f32 + <Scenario variant> - small + <Suite> - pbase-f32-rfile-performance-small + <Suite> - pbase-f32-rfs-performance-small + <Scenario variant> - medium + <Suite> - pbase-f32-rfile-performance-medium + <Suite> - pbase-f32-rfs-performance-medium + <Scenario variant> - large + <Suite> - pbase-f32-rfile-performance-large + <Suite> - pbase-f32-rfs-performance-large + <Conformance suite> - conformance + <Sub-component> - f32 + <Sub-component> - FAT32 + <Suite> - f32-fat32-conformance

This structure can +be seen in the following location after building the test suite: %EPOCROOT%\epoc32\testdriver\testproduct.

The +hierarchical test suite structure facilitates easy build and execution of +the complete test suite or part of it. For example, if the path specified +is:

    +
  • base: +The entire suite is built/run.

  • +
  • base.validation.f32: +All the suites F32 are built/run.

  • +
  • base.validation.f32.f32-automated: +Only F32 automated test is built/run.

  • +
  • base.validation.f32.f32-manual: +Only F32 manual test is built/run.

  • +
  • base.validation.f32.f32-dualdrive: +Only f32 dual drive tests is built/run.

  • +
  • base.validation.f32.f32-automated.ram: +Only tests on RAM are built/run.

  • +
  • base.validation.f32.f32-automated.ram.fixed: +Only tests on non-removable RAM are built/run.

  • +
  • base.validation.f32.f32-automated.rom: +Only tests on ROM are built/run.

  • +
  • base.validation.f32.f32-automated.os: +Only OS suite is built/run.

  • +
  • base.validation.f32.f32-automated.ram.fixed: +Only tests on non-removable RAM are built/run.

  • +

General +environment setup

    +
  1. Refer to specific sub-component +test suite description in Base +Verification Suite for details about configuration and building additional +components before building the test suite.

  2. +
  3. The entire Base Test +product can be built from ...\baseapitest\basesvs\group\.

  4. +
  5. Individual test components +can be built from the respective group directories. For example, ...\baseapitest\basesvs\validation\f32\group\.

    bldmake bldfiles

    abld test +build <WINSCW/ARMV5> <udeb/urel>

  6. +

Steps +to build and execute tests using TestDriver

Communication between +the PC and the device under test (Symbian platform device) is enabled using +STAT. So, the device under test must have STAT installed.

For WINSCW platform:

    +
  1. Configure TestDriver +using the following script:

    ...\baseapitest\basesvs\group\testdriversetup.bat

  2. +
  3. Build the test code:

    testdriver +build –p <winscw/armv5> –b <urel/udeb> –s base[.<specific test suite +path>]

    Testdriver commands can be run from any location.

  4. +
  5. To execute the Tests +:

    testdriver run

  6. +

For ARMV5 platform:

Communication between PC and hardware:

The +hardware reference board (for example H4) must be connected to the PC through +USB or serial cable. The ROM image is built on the PC and transferred to the +hardware board through MMC card.

    +
  1. Configure TestDriver +using the following script:

    ...\baseapitest\basesvs\group\testdriversetup.bat

  2. +
  3. Build the test +code:

    testdriver build –p <winscw/armv5> –b <urel/udeb> +–s base[.<specific test suite path>]

    Testdriver commands +can be run from any location.

  4. +
  5. Build the ROM image:

    buildrom +-D_STARTUPMODE2 -D_EABI=ARMV5 -DRVCT -DUSE_STRONG_CRYPTOGRAPHY <h2/h4hrp> +techview_statapi td_base

    Note The ROM building +commands are different for NAND images and enabling Dual Drive.

  6. +
  7. Transfer the ROM to +the device and start-up.

  8. +
  9. Execute the Tests:

    testdriver +run -t <transport service>

    Transport service may be +“serial<COM port number that is connected to the board>”, “bt<COM port +number for BT dongle>” or “usb<mapped port number>”

  10. +

All scripts and test data are automatically removed from +the system drive of the device under test after the execution.

Steps to build and execute +tests manually (Tests built into ROM)

For WINSCW platform:

    +
  1. Select your Setup +Script:

    For example setup-PBASE-F32-Sfsrv-PublicApi.script is +used to run only the F32_EKA2 validation test suite.

  2. +
  3. Change directory +to:

    %EPOCROOT%\epoc32\release\winscw\<urel/udeb>

  4. +
  5. Execute the Tests:

    testexecute +z:\base\setup_<test suite name>.script

    setup_t_base.script is +used to copy the test data files on to c:\drive.

    testexecute +c:\base\<test suite name>.script

    t_base.script is +used to run the entire base test suite.

  6. +

For ARMV5 platform:

    +
  1. Build ROM image:

    buildrom +-D_EABI=ARMV5 –DRVCT -DUSE_STRONG_CRYPTOGRAPHY <h2/h4hrp> techview t_base

    The +ROM image is built on the PC and transferred to the hardware board through +MMC card. Note: The ROM building commands is different for NAND images. +Refer to Base Verification +Suite for more details.

  2. +
  3. Select your Setup +Script:

    The setup-PBASE-F32-Sfsrv-PublicApi.script is +used to run only F32_EKA2 validation test suite.

  4. +
  5. Running Scripts:

    Open +eshell

  6. +
  7. Execute the Tests:

    testexecute +z:\base\setup_<test suite name>.script

    setup_t_base.script is +used to copy the test data files on to c:\drive.

    testexecute +c:\base\<test suite name>.script

    t_base.script is +used to run the entire base test suite.

  8. +

Steps +to build and execute tests on phone directly using .pkg file

For ARMV5 platform:

    +
  1. EShell and TestExecute:

    The +phone must have eshell and testexecute installed.

  2. +
  3. Certificate files:

    Ensure .cer and .key files are placed into /epoc32/pkg/ directory on the +PC and rename aszeus.cer and SymbianACS.key correspondingly.

  4. +
  5. Create and install +SIS-file:

    Run makesisfiles.bat from /epoc32/pkg/ directory.

    makesisfiles.bat

    This creates t_sfsrv.sis file. The file must +be copied to an MMC card, transferred to the phone and executed.

    When +the t_sfsrv.sis file is run, the base test suite is installed +on the device.

  6. +
  7. Running Scripts:

    Run +the test scripts. For more information refer to Steps +to build and execute tests manually (Tests built into ROM).

  8. +

Selective +Execution of Test Cases using TCS File

Test cases can be excluded +from a test run by using a .tcs file. The .tcs file +contains IDs of test cases to be excluded.

The test case IDs can be +specified as a range, a list, or a combination of both. Each range or individual +test case ID is delimited by a new-line character.

For example, range +of test case IDs to be excluded can be specified as:

pbase-f32-rfile-performance-large-0001: + pbase-f32-rfile-performance-large-0005

This +excludes all test cases from 1 to 5 (1 and 5 included) during execution.

A +list of test cases IDs to be excluded can be:

COMINF-ESOCK-RSubConnection-PublicAPI-0001

COMINF-ESOCK-RSubConnection-PublicAPI-0005

COMINF-ESOCK-RSubConnection-PublicAPI-0007

A +combination of range and list can be:

pbase-f32-rfile-performance-large-0001: + pbase-f32-rfile-performance-large-0005

pbase-f32-rfile-performance-large-0010: + pbase-f32-rfile-performance-large-0015

pbase-f32-rfile-performance-large-0020

pbase-f32-rfile-performance-large-0021

There is only one .tcs file located at …\baseapitest\basesvs\config\t_base.tcs

For +more information on location of .tcs files, see TCS +File Source Location in F32 Performance Test Suite, TCS File Source Location in FAT32 Conformance Test Suite, and TCS +File Source Location in F32_EKA2 Validation Test Suite.

The .tcs file +must be modified when using the TestDriver for test execution. When executing +the tests manually, the following testexecute command illustrates +the usage of the .tcs file:

testexecute +%EPOCROOT%\epoc32\release\winscw\<urel/udeb>\<test suite name>.script +–[tcx|tci] %EPOCROOT%\epoc32\release\winscw\<urel/udeb>\t_base.tcs

The -tcx switch +is used to execute all the tests in the specified script excluding the test +cases specified in the .tcs file. The –tci switch +is used to execute only the tests specified in the .tcs file +from the specified script.

Test +Results

Test +Results log file location

After tests complete execution, result +logs can be found on the system drive of the device in the \logs\testexecute\ directory.

On +the PC, if tests are run using testdriver, results can be found in the following +location: %EPOCROOT%\epoc32\TestDriver\results\. If executing +the tests manually the results can be found in the following location: %EPOCROOT%\epoc32\winscw\c\logs\testexecute

When +using testdriver, a new test result folder is created for each test run that +is, results are not overwritten each time a test run is run. A .htm log +file for each script contained in the suite can be found in its own subfolder.

Each +test case in the script consists of test blocks (usually just one), which +either pass or fail. Each test block runs some of the commands. Each of the +commands performs an action on the tested APIs. A test block passes if all +its commands and other instructions pass. A test case passes if all its test +blocks pass.

The log displays the percentage of test cases that have +passed successfully.

How +to interpret test results

Each test suite produces its own HTML +log file containing the results of the test run. When examining the log file, +the results at the bottom of the log should be checked first. This gives a +summary of the tests in the suite that have passed, failed or panicked.

In +the body of the log tests which passed are highlighted in green, tests which +failed in red and tests which caused a panic in blue. The reason for any failure +or panic is also given.

When this test suite is run on a real device +some tests may fail due to limitations of the device. Either the device does +not have the correct hardware (for example, no modem exists for a test which +needs one) or it lacks the right software plug-ins to run the test. In such +cases the log displays that a test has failed with a -5 error (which means KErrNotSupported). +Tests appearing to fail with -5 do not indicate defects, but limitations of +the device.

Refer to Symbian Verification Suite » TestDriver » +Using TestDriver » Results » Viewing the test results for more information +on test driver results.

+
OS Focussed +Components

The following tables illustrate the components and classes +tested on each Symbian platform baseline:

Table 1: Validation +testing of Base, F32_EKA2 component on each Symbian platform baseline

+ + + + + 9.1 +9.2 +9.3 +9.4 +9.5 + + + RFs + + + CDir + + + CDirScan + + + CFileBase + + + CFileMan + + + MFileManObserver + + + RDir + + + RFile + + + RFormat + + + RRawDisk + + + TDriveUnit + + + TEntry + + + TEntryArray + + + TFindFile + + + TFileText + + + TOpenFileScan + + + TParse + + + TParseBase + + + TParsePtr + + + TparsePtrC + + + TVolumeInfo + + + FileNamesIdentical() + + + + +

X

+

X

+

X

+

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + + + +

X

+

X

+
+ + +

Table 2: Performance testing of Base, F32_EKA2 on each Symbian +platform baseline

+ + + + + 9.1 +9.2 +9.3 +9.4 +9.5 + + +RFs + + +RFile + + + + +

X

+

X

+

X

+

X

+

X

+
+ +

X

+

X

+

X

+

X

+

X

+
+ + +

The FAT32 conformance test suite is not API specific and is available +on Symbian platform versions 9.4 and 9.5.

\ No newline at end of file