Multimedia -test technology

This topic provides information about the Multimedia test technology area. -It describes the test philosophy, the types of test that this test technology -area contains, what is being tested and describes the test environment that -the test are run in. This topic is intended to provide an overview about the -test technology area, its coverage, test environment setup and find further -information about the test suites offered by this test technology area.

Multimedia test technology area scope

Components -under test

The Multimedia test suites test the following components -under the Multimedia technology area:

Camera functionality:

ECam -is an API which allows an application to access and control any camera hardware -which is attached to the device.
Image conversion -library (ICL):

ICL provides a lightweight Image Conversion Library -which supports the decoding and encoding of image files.
Multimedia Framework -(MMF):

MMF including audio and video recording and playback, -tone playback and MIDI playback.
- Controller framework:
  
  Some -of the client-side functionality is implemented as part of the framework. -Base classes are provided to help device creators implement plugins.
- Audio:
  - Audio playback utility
  - Audio recording utility
  - Audio conversion utility
  - Tone playback utility
- Video:
  
  Video -player utility - APIs for video playback and recording.
- MIDI:
  
  MIDI -Client Utility - APIs for MIDI playback and recording.

Test -scope

Published -API testing

This technology area provides a number of test suites -to test published APIs. These test suites can be used as a regression test -suite and also as a part of compatibility testing program.

For more -details about the individual classes, see API -test suites for Multimedia.

Test -omissions

The Multimedia test suites test only the common audio, -video, and image codecs that the Symbian platform implements.

For -more details about the individual classes, see API -test suites for Multimedia.

Multimedia test technology environment

Device -under test setup

The device under test (hardware platform/phone/emulator) -should have an appropriate Symbian platform or device creator UI build (S60, -UIQ, MOAP for example).

PC -setup

The PC should have a Symbian or equivalent device creator -developer kit, PDT and relevant compilers (Code Warrior, RVCT, GCCE) installed -to build and execute the test code.

Network -environment setup

None

Test -dependencies

Note: Any mention of “epoc32” -is offset by %EPOCROOT%. That is, the full path where -the EPOC tree is installed.

Test data

The -test data for each test suite is in the following location:

…\os\mm\mmapitest\mmsvs\suite\ecam\T_Camera\testdata
…\os\mm\mmapitest\mmsvs\suite\icl\T_ImageEncoder\testdata
…\os\mm\mmapitest\mmsvs\suite\icl\T_ImageDecoder\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_MdaAudioConvertUtility\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_MdaAudioInputStream\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_MdaAudioOutputStream\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_MdaAudioPlayerUtility\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_MdaAudioRecorderUtility\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_MdaAudioToneUtility\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_MidiClientUtility\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_VideoPlayerUtility\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_VideoRecorderUtility\testdata

For more details about the individual classes, see API -test suites for Multimedia.

Also, there are two global environment -files, t_multimedia.ini and t_multimedia_location.ini, -which contain configurable parameters and their default values. Test that -are device setup specific have these details defined in t_multimedia.ini. -Test that are dependent on location have these details defined in t_multimedia_location.ini.

Your -can configure these parameters by modifying their default values in these -files. Details regarding each parameter can be found in the file comments. -These files are in the following location:

…\os\mm\mmapitest\mmsvs\suite\testdata

For -more details about the individual classes, see API -test suites for Multimedia.

Some of the test scripts use an -audio file as a seed. As each test case runs the test data is converted into -the test data format required for input to the subsequent test case. The approach -is taken to help reduce the size on disk that the test data occupies and speed-up -transfer of data to device under test.

Testing with corrupt files -is difficult, as they are reported by most virus checker applications as viruses, -when saved on the PC. To get around this the files are stored in the source -tree as complete files, modified during the test cases on the device under -test and cleaned up afterwards. Such files can still potentially cause problems -while executing tests on the emulator if the test run is terminated before -the test data has been cleaned up.

Building and Execution

The following structure illustrates -the Multimedia test suite hierarchical structure after the build process. -This structure can be seen in the following location after building the test -suite:

%EPOCROOT%\epoc32\testdriver\testproduct

For -more information on for general information about testdriver test suite structure, -see Symbian Verification Suite » TestDriver V2 » TestDriver Output » Viewing -the test results.

<Component> - Multimedia - <Sub-component> - ICL - <suite> - MM-ICL-DECDE-PublicApi-suite - <suite> - MM-ICL-ENCDE-PublicApi-suite - <Sub-component> - MMF - <suite> - MM-MMF-ACLNT-CNVRT-PublicAPI-suite - <suite> - MM-MMF-ACLNT-INPT-PublicAPI-suite - <suite> - MM-MMF-ACLNT-OUTPT-PublicAPI-suite - <suite> - MM-MMF-ACLNT-PLYR-PublicAPI-suite - <suite> - MM-MMF-ACLNT-RCRDR-PublicAPI-suite - <suite> - MM-MMF-ACLNT-TONE-PublicAPI-suite - <suite> - MM-MMF-MIDI-PublicAPI-suite - <suite> - MM-MMF-VCLNT-PLYR-PublicAPI-suite - <suite> - MM-MMF-VCLNT-RCRDR-PublicAPI-suite - <Sub-component> - ECAM - <suite> - MM-ECM-PublicApi-suite - <suite> - MM-MMF-VCLNT-PLYR-PublicAPI-suite - <suite> - MM-MMF-VCLNT-RCRDR-PublicAPI-suite -

The hierarchical test suite structure facilitates easy build -and execution of the complete test suite or part of it. For example, if the -path specified is:

<multimedia> - -The entire suite is built/executed.
<multimedia.audio> - -All the six suites under audio are built/executed.
<multimedia.audio.MM-MMF-ACLNT-CNVRT-PublicAPI-suite> - -MM-MMF-ACLNT-CNVRT-PublicAPI-suite is built/executed.

The following list show the sequence of commands to be executed on -emulator and hardware:

General environment setup for WINSCW and ARMV5
Steps to build and execute tests using test driver
Steps to build and execute tests manually (tests built into ROM)

General environment setup -for WINSCW and ARMV5

Ensure that all the -parameters in the files t_multimedia.ini and t_multimedia_location.ini are -set to the correct values. For more details, see Test data.
Build additional components. -For details regarding downloading and building additional components before -building the test suite, see the specific sub-component test suite in API test suites for Multimedia.
The entire Multimedia -test component can be built from …\os\mm\mmapitest\mmsvs\suite\group\….

Individual -test components can be built from the respective group directory:
- …\os\mm\mmapitest\mmsvs\suite\ecam\t_<sub-component>\group -
- …\os\mm\mmapitest\mmsvs\suite\icl\t_<sub-component>\group
- …\os\mm\mmapitest\mmsvs\suite\mmf\t_<sub-component>\group -
Execute the following commands:
1. bldmake bldfiles
2. abld test build -[winscw|armv5] [urel|udeb]

Steps to build and execute -tests using test driver

On -WINSCW platform

Communication between -the PC and the device under test (Symbian platform device) is enabled using -STAT. Hence, the device under test should have STAT installed.
TestDriver can be configured -using the automated scripts located at, …\os\mm\mmapitest\mmsvs\suite\group\… using -the following command:

testdriversetup

testdriversetup.bat calls testdriversetup.pl, which applies appropriate set of configuration parameters.
Build test code using -the following command:

testdriver build [–p [winscw|armv5]] -[–b [udeb|urel]] –s multimedia[.audio|image|video[.<test suite name >]]

Note: You -can execute TestDriver commands from any location.

For the multimedia -test suite hierarchical structure, refer to Building and Execution section.
Execute the tests on -the emulator using the following command:

testdriver run

Note: All scripts and test data are automatically removed -from the system drive of the device under test after the execution.

On ARMV5 platform

Communication between -the PC and the device under test (Symbian platform device) is enabled using -STAT. Hence, the device under test should have STAT installed.
Communication between -PC and hardware. For more details, refer to Symbian Verification Suite -» TestDriver » Using TestDriver » Communicating with the device. The ROM -image is built on the PC and transferred to the hardware board via MMC card.
TestDriver can be configured -using the automated scripts located at, …\os\mm\mmapitest\mmsvs\suite\group\… using -the following command:

testdriversetup

testdriversetup.bat calls testdriversetup.pl, which applies appropriate set of configuration parameters.
Build test code using -the following command:

testdriver build [–p [winscw|armv5]] -[–b [udeb|urel]] –s multimedia[.mmf|icl|ecam[.<test suite name >]]

Note: You -can execute TestDriver commands from any location.

For the multimedia -test suite hierarchical structure, refer to Building and Execution section.
Build the ROM image -using the following command:

buildrom -D_STARTUPMODE2 -D_EABI=ARMV5 --DRVCT -DUSE_STRONG_CRYPTOGRAPHY -D_CUSTOM_COMMSDAT <h2/h4hrp> techview_statapi -td_multimedia
Execute the tests on -the device using the following command:

testdriver run --t <transport service used>

The transport service used -can be one of the following:
- serial<COM port number -that is connected to the board>
- bt<COM port number -for BT dongle>
- usb<mapped port number>
For more details, refer to Symbian Verification Suite » TestDriver -» Using TestDriver » Communicating with the device.

Note: All scripts and test data are automatically removed -from the system drive of the device under test after the execution.

Steps to build and execute -tests manually (tests built into ROM)

On -WINSCW platform

The following setup -scripts are used to execute tests:
- Execute the entire multimedia -test suite using the setup script, setup_t_multimedia.script
- Execute the test suite -for a specific class using one of the following setup scripts:
  - setup-MM-ECM-PublicApi.script
  - setup-MM-ICL-DECDE-PublicApi.script
  - setup-MM-ICL-ENCDE-PublicApi.script
  - setup-MM-MMF-ACLNT-CNVRT-PublicAPI.script
  - setup-MM-MMF-ACLNT-INPT-PublicAPI.script
  - setup-MM-MMF-ACLNT-OUTPT-PublicAPI.script
  - setup-MM-MMF-ACLNT-PLYR-PublicAPI.script
  - setup-MM-MMF-ACLNT-RCRDR-PublicAPI.script
  - setup-MM-MMF-ACLNT-TONE-PublicAPI.script
  - setup-MM-MMF-MIDI-PublicAPI.script
  - setup-MM-MMF-VCLNT-PLYR-PublicAPI.script
  - setup-MM-MMF-VCLNT-RCRDR-PublicAPI.script
Run scripts on the emulator -by changing the directory to, %EPOCROOT%\epoc32\release\winscw\<urel/udeb>.
Execute tests using -the following commands:

testexecute z:\multimedia\setup_<test -suite name>.script

setup_t_multimedia.script is -used to execute the entire multimedia setup script.

testexecute -c:\multimedia\<test suite name>.script

t_multimedia.script can -be used to execute the entire multimedia test suite. The scripts mentioned -in the t_multimedia.script can be run individually.

On -ARMV5 platform

Build the ROM image -using the following command:

buildrom -D_EABI=ARMV5 –DRVCT --DUSE_STRONG_CRYPTOGRAPHY -D_CUSTOM_COMMSDAT <h2/h4hrp> techview t_multimedia
The following setup -scripts are used to execute tests:
- Execute the entire multimedia -test suite using the setup script, setup_t_multimedia.script
- Execute the test suite -for a specific class using one of the following setup scripts:
  - setup-MM-ECM-PublicApi.script
  - setup-MM-ICL-DECDE-PublicApi.script
  - setup-MM-ICL-ENCDE-PublicApi.script
  - setup-MM-MMF-ACLNT-CNVRT-PublicAPI.script
  - setup-MM-MMF-ACLNT-INPT-PublicAPI.script
  - setup-MM-MMF-ACLNT-OUTPT-PublicAPI.script
  - setup-MM-MMF-ACLNT-PLYR-PublicAPI.script
  - setup-MM-MMF-ACLNT-RCRDR-PublicAPI.script
  - setup-MM-MMF-ACLNT-TONE-PublicAPI.script
  - setup-MM-MMF-MIDI-PublicAPI.script
  - setup-MM-MMF-VCLNT-PLYR-PublicAPI.script
  - setup-MM-MMF-VCLNT-RCRDR-PublicAPI.script
Run scripts on the device -using eshell.
Execute tests using -the following commands:

testexecute z:\multimedia\setup_<test -suite name>.script

setup_t_multimedia.script is -used to execute the entire multimedia setup script.

testexecute -c:\multimedia\<test suite name>.script

t_multimedia.script can -be used to execute the entire multimedia test suite. The scripts mentioned -in the t_multimedia.script can be run individually.

Steps -to build and execute tests on phone directly using .pkg file

For ARMV5 platform:

EShell and TestExecute:

The -phone must have eshell and testexecute installed.
Certificate files:

Ensure .cer and .key files are placed into /epoc32/pkg/ directory on the -PC and renamed aszeus.cer and SymbianACS.key correspondingly.
Create and install -SIS-file:

Execute makesisfiles.bat from /epoc32/pkg/ - directory.

makesisfiles.bat

This -creates files.
- t_camera.sis
- t_imagedecoder.sis
- t_imageencoder.sis
- t_mdaaudioconvertutility.sis
- t_mdaaudioinputstream.sis
- t_mdaaudiooutputstream.sis
- t_mdaaudioplayerutility.sis
- t_mdaaudiorecorderutility.sis
- t_mdaaudiotoneutility.sis
- t_midiclientutility.sis
- t_videoplayerutility.sis
- t_videorecorderutility.sis
The files must be copied to an MMC card, transferred to the phone -and executed.

When the above .sis files are executed, all multimedia -test suites are installed on the device.
Running Scripts:

Run -the test scripts. For more information refer to Steps -to build and execute tests manually (tests built into ROM) .

Excluding -Test Cases from Test Run using TCS File

Test cases can be executed -selectively using a .tcs file. The .tcs file -contains the IDs of test cases:

to be excluded from -a test run
to be executed on their -own. Therefore, excluding all other test cases in the test script from a test -run.

The test case IDs can be specified as a range, a list, or a combination -of both, where each range or individual test case ID is delimited by a new-line -character.

For example, range of test case IDs to be excluded can -be specified as:

MM-ICL-DECDE-PublicApi-0003: MM-ICL-DECDE-PublicApi-0012

Test -cases from 3 to 12 are (3 and 12 included) either excluded or are the only -ones to be executed during a test run.

A list of test cases IDs to -be excluded can be:

MM-ICL-ENCDE-PublicApi-0007

MM-ICL-ENCDE-PublicApi-0010

MM-ICL-ENCDE-PublicApi-0012

A combination of range and -list can be:

MM-MMF-ACLNT-CNVRT-PublicAPI-0005

MM-MMF-ACLNT-CNVRT-PublicAPI-0008

MM-MMF-ACLNT-CNVRT-PublicAPI-0010: MM-MMF-ACLNT-CNVRT-PublicAPI-0015

There -is only one .tcs file, which is located at the following -location:

…\os\mm\mmapitest\mmsvs\suite\config\t_multimedia.tcs

For -more information on location of .tcs files, see t_camera -TCS files source location, , image -conversion TCS files source location, and multimedia framework TCS files source location.

Only the .tcs files -has to be modified when using the test driver for test execution. When executing -the tests manually, the following testexecute command illustrates the usage -of the .tcs file:

testexecute %EPOCROOT%\epoc32\release\winscw\<urel/udeb>\<test -suite name>.script –[tcx|tci] %EPOCROOT%\epoc32\release\winscw\<urel/udeb>\t_multimedia.tcs

The -tcx switch -is used to execute all the tests in the specified script excluding the test -cases specified in the .tcs file. While the –tci switch -is used to execute only the tests specified in the .tcs file -from the specified script.

Test -Results

Test -Results log file location

After tests complete execution, result -logs can be found on the system drive of the device in the \logs\testexecute\directory. -On the PC, if tests are executed using test driver, results can be in any -of the following locations:

%EPOCROOT%\epoc32\TestDriver\results\
%EPOCROOT%\epoc32\winscw\c\logs\testexecute

If the tests are manually run, when using test driver, a new test -result folder is created for each test run. That is, results are not overwritten -each time a test run is executed. In its own subfolder, an .html log -file can be found for each script contained in the suite.

Each test -case in the script contains one test block, which performs a number of actions -on an API. The log will display the percentage of test cases that have passed -successfully. The script should ideally have a 100% pass rate.

How to interpret test results

Each test suite produces its -own HTML log file containing the results of the test run. When examining the -log file, the results at the bottom of the log should be checked first. This -gives a summary of the number of tests in the suite that have passed, failed -or panicked. In the body of the log, passing tests are highlighted in green, -and failing tests in red, while tests which cause a panic are shown in blue.

If -a failing or panicking test has been identified, its reason for failing or -panicking can be seen by examining each step in the test, listed between the -“START_TESTCASE” and “END_TESTCASE” tags. The steps which passes have the -word “INFO” followed by some information regarding the step. A failing step -will instead have the word “ERROR”, followed by an explanation of the error -(usually with an error code). If a panic occurs there may be no textual explanation -of the problem, but a panic code will be given (within the blue highlighted -text).

When this test suite is run on a real device, some of the tests -may fail due to limitations of the device. This happens because of the device -not having the correct hardware (for example, no modem exists for a test which -needs one), or lacking the right software plugins to run the test. This can -be spotted if the log displays that a test has failed with a -5 error (which -means KErrNotSupported). The -5 error basically means that the test has failed -because the functionality in this test is not supported on the device used. -Therefore, any tests failing with -5 are not due to defects, but to the limitations -of the device used.

For more information on test driver results, see -R4Error! Reference source not found in Symbian Verification Suite » TestDriver -V2 » TestDriver Output » Viewing the test results.

OS Focussed Components

The following table illustrates -the sub-components and classes available on each Symbian platform.

Table -1: Component Multimedia sub-components and classes tested on each Symbian -platform baseline

- - - -Symbian platform baseline -9.1 -9.2 -9.3 -9.4 -9.5 - - - - -

Sub-component1 - ECam

- - -

Class1 – CCamera

- - -

Sub-component2 - ICL

- - -

Class1 - CImageEncoder

- - -

Class2 – CImageDecoder

- - -

Sub-component3 – MMF

- - -

Class1 – CMdaAudioConvertUtility

- - -

Class2 - CMdaAudioInputStream

- - -

Class3 – CMdaAudioOutputStream

- - -

Class4 – CMdaAudioPlayerUtility

- - -

Class5 – CMdaAudioRecorderUtility

- - -

Class6 - CMdaAudioToneUtility

- - -

Class7 – CMidiClientUtility

- - -

Class8 - CVideoPlayerUtility

- - -

Class9 – CVideoRecorderUtility

- - - -

+ + + + + +Multimedia +test technology +

This topic provides information about the Multimedia test technology area. +It describes the test philosophy, the types of test that this test technology +area contains, what is being tested and describes the test environment that +the test are run in. This topic is intended to provide an overview about the +test technology area, its coverage, test environment setup and find further +information about the test suites offered by this test technology area.

Multimedia test technology area scope

Components +under test

The Multimedia test suites test the following components +under the Multimedia technology area:

Camera functionality:

ECam +is an API which allows an application to access and control any camera hardware +which is attached to the device.
Image conversion +library (ICL):

ICL provides a lightweight Image Conversion Library +which supports the decoding and encoding of image files.
Multimedia Framework +(MMF):

MMF including audio and video recording and playback, +tone playback and MIDI playback.
- Controller framework:
  
  Some +of the client-side functionality is implemented as part of the framework. +Base classes are provided to help device creators implement plugins.
- Audio:
  - Audio playback utility
  - Audio recording utility
  - Audio conversion utility
  - Tone playback utility
- Video:
  
  Video +player utility - APIs for video playback and recording.
- MIDI:
  
  MIDI +Client Utility - APIs for MIDI playback and recording.

Test +scope

Published +API testing

This technology area provides a number of test suites +to test published APIs. These test suites can be used as a regression test +suite and also as a part of compatibility testing program.

For more +details about the individual classes, see API +test suites for Multimedia.

Test +omissions

The Multimedia test suites test only the common audio, +video, and image codecs that the Symbian platform implements.

For +more details about the individual classes, see API +test suites for Multimedia.

Multimedia test technology environment

Device +under test setup

The device under test (hardware platform/phone/emulator) +should have an appropriate Symbian platform or device creator UI build (S60, +UIQ, MOAP for example).

PC +setup

The PC should have a Symbian or equivalent device creator +developer kit, PDT and relevant compilers (Code Warrior, RVCT, GCCE) installed +to build and execute the test code.

Network +environment setup

None

Test +dependencies

Note: Any mention of “epoc32” +is offset by %EPOCROOT%. That is, the full path where +the EPOC tree is installed.

Test data

The +test data for each test suite is in the following location:

…\os\mm\mmapitest\mmsvs\suite\ecam\T_Camera\testdata
…\os\mm\mmapitest\mmsvs\suite\icl\T_ImageEncoder\testdata
…\os\mm\mmapitest\mmsvs\suite\icl\T_ImageDecoder\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_MdaAudioConvertUtility\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_MdaAudioInputStream\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_MdaAudioOutputStream\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_MdaAudioPlayerUtility\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_MdaAudioRecorderUtility\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_MdaAudioToneUtility\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_MidiClientUtility\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_VideoPlayerUtility\testdata
…\os\mm\mmapitest\mmsvs\suite\mmf\T_VideoRecorderUtility\testdata

For more details about the individual classes, see API +test suites for Multimedia.

Also, there are two global environment +files, t_multimedia.ini and t_multimedia_location.ini, +which contain configurable parameters and their default values. Test that +are device setup specific have these details defined in t_multimedia.ini. +Test that are dependent on location have these details defined in t_multimedia_location.ini.

Your +can configure these parameters by modifying their default values in these +files. Details regarding each parameter can be found in the file comments. +These files are in the following location:

…\os\mm\mmapitest\mmsvs\suite\testdata

For +more details about the individual classes, see API +test suites for Multimedia.

Some of the test scripts use an +audio file as a seed. As each test case runs the test data is converted into +the test data format required for input to the subsequent test case. The approach +is taken to help reduce the size on disk that the test data occupies and speed-up +transfer of data to device under test.

Testing with corrupt files +is difficult, as they are reported by most virus checker applications as viruses, +when saved on the PC. To get around this the files are stored in the source +tree as complete files, modified during the test cases on the device under +test and cleaned up afterwards. Such files can still potentially cause problems +while executing tests on the emulator if the test run is terminated before +the test data has been cleaned up.

Building and Execution

The following structure illustrates +the Multimedia test suite hierarchical structure after the build process. +This structure can be seen in the following location after building the test +suite:

%EPOCROOT%\epoc32\testdriver\testproduct

For +more information on for general information about testdriver test suite structure, +see Symbian Verification Suite » TestDriver V2 » TestDriver Output » Viewing +the test results.

<Component> - Multimedia + <Sub-component> - ICL + <suite> - MM-ICL-DECDE-PublicApi-suite + <suite> - MM-ICL-ENCDE-PublicApi-suite + <Sub-component> - MMF + <suite> - MM-MMF-ACLNT-CNVRT-PublicAPI-suite + <suite> - MM-MMF-ACLNT-INPT-PublicAPI-suite + <suite> - MM-MMF-ACLNT-OUTPT-PublicAPI-suite + <suite> - MM-MMF-ACLNT-PLYR-PublicAPI-suite + <suite> - MM-MMF-ACLNT-RCRDR-PublicAPI-suite + <suite> - MM-MMF-ACLNT-TONE-PublicAPI-suite + <suite> - MM-MMF-MIDI-PublicAPI-suite + <suite> - MM-MMF-VCLNT-PLYR-PublicAPI-suite + <suite> - MM-MMF-VCLNT-RCRDR-PublicAPI-suite + <Sub-component> - ECAM + <suite> - MM-ECM-PublicApi-suite + <suite> - MM-MMF-VCLNT-PLYR-PublicAPI-suite + <suite> - MM-MMF-VCLNT-RCRDR-PublicAPI-suite +

The hierarchical test suite structure facilitates easy build +and execution of the complete test suite or part of it. For example, if the +path specified is:

<multimedia> - +The entire suite is built/executed.
<multimedia.audio> - +All the six suites under audio are built/executed.
<multimedia.audio.MM-MMF-ACLNT-CNVRT-PublicAPI-suite> - +MM-MMF-ACLNT-CNVRT-PublicAPI-suite is built/executed.

The following list show the sequence of commands to be executed on +emulator and hardware:

General environment setup for WINSCW and ARMV5
Steps to build and execute tests using test driver
Steps to build and execute tests manually (tests built into ROM)

General environment setup +for WINSCW and ARMV5

Ensure that all the +parameters in the files t_multimedia.ini and t_multimedia_location.ini are +set to the correct values. For more details, see Test data.
Build additional components. +For details regarding downloading and building additional components before +building the test suite, see the specific sub-component test suite in API test suites for Multimedia.
The entire Multimedia +test component can be built from …\os\mm\mmapitest\mmsvs\suite\group\….

Individual +test components can be built from the respective group directory:
- …\os\mm\mmapitest\mmsvs\suite\ecam\t_<sub-component>\group +
- …\os\mm\mmapitest\mmsvs\suite\icl\t_<sub-component>\group
- …\os\mm\mmapitest\mmsvs\suite\mmf\t_<sub-component>\group +
Execute the following commands:
1. bldmake bldfiles
2. abld test build +[winscw|armv5] [urel|udeb]

Steps to build and execute +tests using test driver

On +WINSCW platform

Communication between +the PC and the device under test (Symbian platform device) is enabled using +STAT. Hence, the device under test should have STAT installed.
TestDriver can be configured +using the automated scripts located at, …\os\mm\mmapitest\mmsvs\suite\group\… using +the following command:

testdriversetup

testdriversetup.bat calls testdriversetup.pl, which applies appropriate set of configuration parameters.
Build test code using +the following command:

testdriver build [–p [winscw|armv5]] +[–b [udeb|urel]] –s multimedia[.audio|image|video[.<test suite name >]]

Note: You +can execute TestDriver commands from any location.

For the multimedia +test suite hierarchical structure, refer to Building and Execution section.
Execute the tests on +the emulator using the following command:

testdriver run

Note: All scripts and test data are automatically removed +from the system drive of the device under test after the execution.

On ARMV5 platform

Communication between +the PC and the device under test (Symbian platform device) is enabled using +STAT. Hence, the device under test should have STAT installed.
Communication between +PC and hardware. For more details, refer to Symbian Verification Suite +» TestDriver » Using TestDriver » Communicating with the device. The ROM +image is built on the PC and transferred to the hardware board via MMC card.
TestDriver can be configured +using the automated scripts located at, …\os\mm\mmapitest\mmsvs\suite\group\… using +the following command:

testdriversetup

testdriversetup.bat calls testdriversetup.pl, which applies appropriate set of configuration parameters.
Build test code using +the following command:

testdriver build [–p [winscw|armv5]] +[–b [udeb|urel]] –s multimedia[.mmf|icl|ecam[.<test suite name >]]

Note: You +can execute TestDriver commands from any location.

For the multimedia +test suite hierarchical structure, refer to Building and Execution section.
Build the ROM image +using the following command:

buildrom -D_STARTUPMODE2 -D_EABI=ARMV5 +-DRVCT -DUSE_STRONG_CRYPTOGRAPHY -D_CUSTOM_COMMSDAT <h2/h4hrp> techview_statapi +td_multimedia
Execute the tests on +the device using the following command:

testdriver run +-t <transport service used>

The transport service used +can be one of the following:
- serial<COM port number +that is connected to the board>
- bt<COM port number +for BT dongle>
- usb<mapped port number>
For more details, refer to Symbian Verification Suite » TestDriver +» Using TestDriver » Communicating with the device.

Note: All scripts and test data are automatically removed +from the system drive of the device under test after the execution.

Steps to build and execute +tests manually (tests built into ROM)

On +WINSCW platform

The following setup +scripts are used to execute tests:
- Execute the entire multimedia +test suite using the setup script, setup_t_multimedia.script
- Execute the test suite +for a specific class using one of the following setup scripts:
  - setup-MM-ECM-PublicApi.script
  - setup-MM-ICL-DECDE-PublicApi.script
  - setup-MM-ICL-ENCDE-PublicApi.script
  - setup-MM-MMF-ACLNT-CNVRT-PublicAPI.script
  - setup-MM-MMF-ACLNT-INPT-PublicAPI.script
  - setup-MM-MMF-ACLNT-OUTPT-PublicAPI.script
  - setup-MM-MMF-ACLNT-PLYR-PublicAPI.script
  - setup-MM-MMF-ACLNT-RCRDR-PublicAPI.script
  - setup-MM-MMF-ACLNT-TONE-PublicAPI.script
  - setup-MM-MMF-MIDI-PublicAPI.script
  - setup-MM-MMF-VCLNT-PLYR-PublicAPI.script
  - setup-MM-MMF-VCLNT-RCRDR-PublicAPI.script
Run scripts on the emulator +by changing the directory to, %EPOCROOT%\epoc32\release\winscw\<urel/udeb>.
Execute tests using +the following commands:

testexecute z:\multimedia\setup_<test +suite name>.script

setup_t_multimedia.script is +used to execute the entire multimedia setup script.

testexecute +c:\multimedia\<test suite name>.script

t_multimedia.script can +be used to execute the entire multimedia test suite. The scripts mentioned +in the t_multimedia.script can be run individually.

On +ARMV5 platform

Build the ROM image +using the following command:

buildrom -D_EABI=ARMV5 –DRVCT +-DUSE_STRONG_CRYPTOGRAPHY -D_CUSTOM_COMMSDAT <h2/h4hrp> techview t_multimedia
The following setup +scripts are used to execute tests:
- Execute the entire multimedia +test suite using the setup script, setup_t_multimedia.script
- Execute the test suite +for a specific class using one of the following setup scripts:
  - setup-MM-ECM-PublicApi.script
  - setup-MM-ICL-DECDE-PublicApi.script
  - setup-MM-ICL-ENCDE-PublicApi.script
  - setup-MM-MMF-ACLNT-CNVRT-PublicAPI.script
  - setup-MM-MMF-ACLNT-INPT-PublicAPI.script
  - setup-MM-MMF-ACLNT-OUTPT-PublicAPI.script
  - setup-MM-MMF-ACLNT-PLYR-PublicAPI.script
  - setup-MM-MMF-ACLNT-RCRDR-PublicAPI.script
  - setup-MM-MMF-ACLNT-TONE-PublicAPI.script
  - setup-MM-MMF-MIDI-PublicAPI.script
  - setup-MM-MMF-VCLNT-PLYR-PublicAPI.script
  - setup-MM-MMF-VCLNT-RCRDR-PublicAPI.script
Run scripts on the device +using eshell.
Execute tests using +the following commands:

testexecute z:\multimedia\setup_<test +suite name>.script

setup_t_multimedia.script is +used to execute the entire multimedia setup script.

testexecute +c:\multimedia\<test suite name>.script

t_multimedia.script can +be used to execute the entire multimedia test suite. The scripts mentioned +in the t_multimedia.script can be run individually.

Steps +to build and execute tests on phone directly using .pkg file

For ARMV5 platform:

EShell and TestExecute:

The +phone must have eshell and testexecute installed.
Certificate files:

Ensure .cer and .key files are placed into /epoc32/pkg/ directory on the +PC and renamed aszeus.cer and SymbianACS.key correspondingly.
Create and install +SIS-file:

Execute makesisfiles.bat from /epoc32/pkg/ + directory.

makesisfiles.bat

This +creates files.
- t_camera.sis
- t_imagedecoder.sis
- t_imageencoder.sis
- t_mdaaudioconvertutility.sis
- t_mdaaudioinputstream.sis
- t_mdaaudiooutputstream.sis
- t_mdaaudioplayerutility.sis
- t_mdaaudiorecorderutility.sis
- t_mdaaudiotoneutility.sis
- t_midiclientutility.sis
- t_videoplayerutility.sis
- t_videorecorderutility.sis
The files must be copied to an MMC card, transferred to the phone +and executed.

When the above .sis files are executed, all multimedia +test suites are installed on the device.
Running Scripts:

Run +the test scripts. For more information refer to Steps +to build and execute tests manually (tests built into ROM) .

Excluding +Test Cases from Test Run using TCS File

Test cases can be executed +selectively using a .tcs file. The .tcs file +contains the IDs of test cases:

to be excluded from +a test run
to be executed on their +own. Therefore, excluding all other test cases in the test script from a test +run.

The test case IDs can be specified as a range, a list, or a combination +of both, where each range or individual test case ID is delimited by a new-line +character.

For example, range of test case IDs to be excluded can +be specified as:

MM-ICL-DECDE-PublicApi-0003: MM-ICL-DECDE-PublicApi-0012

Test +cases from 3 to 12 are (3 and 12 included) either excluded or are the only +ones to be executed during a test run.

A list of test cases IDs to +be excluded can be:

MM-ICL-ENCDE-PublicApi-0007

MM-ICL-ENCDE-PublicApi-0010

MM-ICL-ENCDE-PublicApi-0012

A combination of range and +list can be:

MM-MMF-ACLNT-CNVRT-PublicAPI-0005

MM-MMF-ACLNT-CNVRT-PublicAPI-0008

MM-MMF-ACLNT-CNVRT-PublicAPI-0010: MM-MMF-ACLNT-CNVRT-PublicAPI-0015

There +is only one .tcs file, which is located at the following +location:

…\os\mm\mmapitest\mmsvs\suite\config\t_multimedia.tcs

For +more information on location of .tcs files, see t_camera +TCS files source location, , image +conversion TCS files source location, and multimedia framework TCS files source location.

Only the .tcs files +has to be modified when using the test driver for test execution. When executing +the tests manually, the following testexecute command illustrates the usage +of the .tcs file:

testexecute %EPOCROOT%\epoc32\release\winscw\<urel/udeb>\<test +suite name>.script –[tcx|tci] %EPOCROOT%\epoc32\release\winscw\<urel/udeb>\t_multimedia.tcs

The -tcx switch +is used to execute all the tests in the specified script excluding the test +cases specified in the .tcs file. While the –tci switch +is used to execute only the tests specified in the .tcs file +from the specified script.

Test +Results

Test +Results log file location

After tests complete execution, result +logs can be found on the system drive of the device in the \logs\testexecute\directory. +On the PC, if tests are executed using test driver, results can be in any +of the following locations:

%EPOCROOT%\epoc32\TestDriver\results\
%EPOCROOT%\epoc32\winscw\c\logs\testexecute

If the tests are manually run, when using test driver, a new test +result folder is created for each test run. That is, results are not overwritten +each time a test run is executed. In its own subfolder, an .html log +file can be found for each script contained in the suite.

Each test +case in the script contains one test block, which performs a number of actions +on an API. The log will display the percentage of test cases that have passed +successfully. The script should ideally have a 100% pass rate.

How to interpret test results

Each test suite produces its +own HTML log file containing the results of the test run. When examining the +log file, the results at the bottom of the log should be checked first. This +gives a summary of the number of tests in the suite that have passed, failed +or panicked. In the body of the log, passing tests are highlighted in green, +and failing tests in red, while tests which cause a panic are shown in blue.

If +a failing or panicking test has been identified, its reason for failing or +panicking can be seen by examining each step in the test, listed between the +“START_TESTCASE” and “END_TESTCASE” tags. The steps which passes have the +word “INFO” followed by some information regarding the step. A failing step +will instead have the word “ERROR”, followed by an explanation of the error +(usually with an error code). If a panic occurs there may be no textual explanation +of the problem, but a panic code will be given (within the blue highlighted +text).

When this test suite is run on a real device, some of the tests +may fail due to limitations of the device. This happens because of the device +not having the correct hardware (for example, no modem exists for a test which +needs one), or lacking the right software plugins to run the test. This can +be spotted if the log displays that a test has failed with a -5 error (which +means KErrNotSupported). The -5 error basically means that the test has failed +because the functionality in this test is not supported on the device used. +Therefore, any tests failing with -5 are not due to defects, but to the limitations +of the device used.

For more information on test driver results, see +R4Error! Reference source not found in Symbian Verification Suite » TestDriver +V2 » TestDriver Output » Viewing the test results.

OS Focussed Components

The following table illustrates +the sub-components and classes available on each Symbian platform.

Table +1: Component Multimedia sub-components and classes tested on each Symbian +platform baseline

+ + + +Symbian platform baseline +9.1 +9.2 +9.3 +9.4 +9.5 + + + + +

Sub-component1 - ECam

+ + +

Class1 – CCamera

+ + +

Sub-component2 - ICL

+ + +

Class1 - CImageEncoder

+ + +

Class2 – CImageDecoder

+ + +

Sub-component3 – MMF

+ + +

Class1 – CMdaAudioConvertUtility

+ + +

Class2 - CMdaAudioInputStream

+ + +

Class3 – CMdaAudioOutputStream

+ + +

Class4 – CMdaAudioPlayerUtility

+ + +

Class5 – CMdaAudioRecorderUtility

+ + +

Class6 - CMdaAudioToneUtility

+ + +

Class7 – CMidiClientUtility

+ + +

Class8 - CVideoPlayerUtility

+ + +

Class9 – CVideoRecorderUtility

+ + + +

\ No newline at end of file