Reference

This topic provides a summary of documents related to Network Performance Testing that you can refer.

The following table explains the special tags and its equivalent parameter in master script configuration.

Tag name Parameter equivalent in master script configuration

====IPADDR====

[DeviceUnderTest].TestBearerIP

====IPGATEWAY====

[DeviceUnderTest].TestBearerGateway

====IPNETMASK====

Subnet mask is assumed from: [DeviceUnderTest].TestBearerIP

This can be specified in CIDR format. For example, 10.16.83.2/24. If omitted the default is 24 bits.

====ETHERPDDFILENAME==== ====ETHERPDDNAME====

Replaced by the appropriate driver names depending on the chosen target.

Note: This is called Ethernet because it only supports the Ethernet bearer for emulator and dev boards. Other bearers would have to be hard coded into the Communications Database configuration file.

TestControllerPC

This section specifies information about the TestControllerPC that sets up and drive the performance test.

Parameter

Default

Description

Method

none- must be specified

For fully-automatic mode (driven from Testdriver on the PC) set this parameter to "Testdriver ". For semi-automatic mode (driven from device by test scripts generated by NetPerf) set this parameter to "CopyTestScripts "

TestBearerIP

none- must be specified

The TestControllerPC IP address. This is the address that the Device Under Test uses to send and receive data. If the TestControllerPC has a separate network interface used for control, make sure that this IP address is on the network interface used for the test bearer, not the control interface.

ControlIP

none- must be specified

This parameter is the TestControllerPC IP address on the network interface used for control. If it is acceptable to share the Bearer used for testing then the value of this parameter should match the TestBearerIP above.

ControlPort

1683

The TCP port of the UCC service running on the TestControllerPC. This should only be changed to a different value if another device or application is using the default 1683 port.

DestinationForSetupData

netperf_[target]_setup

[target] is the emulator/devboard/device

This parameter is the path where NetPerf should write the setup data.

The setup data comprises:

  • The Communications database, if specified below

  • The STAT .ini file, if the tests are running in fully-automatic mode

  • test configurations and scripts, if tests are running in semi-automatic mode

  • a script to install/run the above.

For example, to run performance tests on a dev board this is likely to be a card reader, for example F:\. Ensure a card is inserted in the card reader before running the script.

When running performance tests on an emulator, this is set to the C: drive of the emulator on the local or remote PC. For example \epoc32\winscw\c or \\some_host\shared_epoc32\winscw\c).

If the default netperf_[target]_setup is used, you must copy the files from the location specified on the Test Controller PC to the Device Under Test before running the tests.

RunServices

yes

This parameter tells the script to start the Test Monitor PC services UCC and exeservice, and to shut them down at the end. Set this to "no " if theUCC and exeservice are started before this script starts.

Cert

n/a

n/a

Key

n/a

n/a

PassOpts

n/a

n/a

Device Under Test

This section explains the DUT parameters and its description.

Parameter

Default

Description

TestBearerIP

none- must be specified

This parameter is the address on the test bearer of the DeviceUnderTest. This is the address that the PC-side performance tool sends data to and receives data from. iperf is an example of a PC-side performance tool.

TestBearerIAP

TestBearerSNAP

0

The connection of the test bearer on the DeviceUnderTest. This is the connection that the DeviceUnderTest performance library attaches to before sending data to and receiving data from the [Test Controller PC].TestBearerIP

This parameter contains either an IAP number or a SNAP number. This parameter defaults to 0 which means use the "default connection" specified in the Communications database.

TestBearerGateway

<network address>.1

The address of the gateway on the test bearer. This value is placed in the generated DeviceUnderTest Communications database to specify a static network interface.

ControlIP

none- must be specified

The IP address of the DeviceUnderTest, which the Test Controller PC contacts to run tests using testdriver /STAT. If control is sent over the same connection as data, use the same value for this parameter as for TestBearerIP.

ControlIAP

ControlSNAP

0

The connection of the control link on the DeviceUnderTest. UCC attaches to this connection before sending control calls to the [Test Controller PC].ControlIP. In TestDriver mode, STAT also runs on this connection.

This parameter is contains either an IAP number or a SNAP number. This parameter defaults to 0 which means use the "default connection" from the Communications database.

ControlPort

3000

The TCP/IP port on the DeviceUnderTest where STAT is listening. This should only be changed to a different value if another device or application is using the default 3000 port.

CpuMeterThreadPriority

100

This parameter sets the thread priority of the idle loop which is used to measure available CPU cycles during performance tests. It does this by counting the number of cycles used by the idle loop over time.

If the CPU measurements are not recorded, this parameter is set too low. If the test system runs slowly or comes to a halt, it is a sign that this parameter is set too high. Depending on the system scheduler or the exact test environment it may be necessary to adjust this value.

See TThreadPriority in %EPOCROOT%epoc32/include/e32const.h for possible values. Defaults to 100 (EPriorityAbsoluteVeryLow)

RunSamplingProfiler

false

This parameter determines whether to run the sampling profiler, and retrieve its results for each test. The profiler must be present on the device for this to work, for example, by including profiler.iby when building the ROM.

The default value is false because running the sampling profiler applies a small processing overhead.

Platform

BuildVariant

armv5

urel

These parameters are the Platform and Variant of the software sent to the DeviceUnderTest.

For accurate performance measurement on target devices, omit these fields. The default values armv5 and urel will be used.

Note: udeb can be specified here, but it is a lot slower due to the logging calls. Therefore the use udeb is not recommended for reliable performance measurements. udeb should only be used for debugging scripts.

winscw with TestDriver is not a valid platform and variant because TestDriver tries to start the emulator locally, which is on the Test Controller PC. TestDriver also runs the test software on the TestControllerPC and this software, in particular iperf, conflicts with the software that needs to run on the DeviceUnderTest.

It is not possible to use TestDriver with a remote emulator because TestDriver will try to start the emulator on the TestControllerPC, and communicate over a TAP device. Therefore winscw can only be used if the [TestControllerPC].Method is "CopyTestScriptsTo ... "

CommsdatTemplate

n/a

If required. See Configuring the Communications Database.

LocationForSetupData

C:\

The location of the setup data on the DeviceUnderTest. For example the drive of a card reader for a dev board, or the C: drive for either an emulator or a device with no card reader. If the data is being copied manually (either from a card or via FTP) then this parameter must specify where that data can be found.

ROM Image

This is a template for building a ROM image where the values shown in [square brackets] must be set as follows:

[macros]

Macro

Note

-D_SERIAL_DOWNLOAD

Allows loading ROM image onto a development board across a serial link or from a CF card.

-D_STARTUPMODE1

Text shell only

-D_CUSTOM_COMMSDAT

Use customized Communications Database

-D_DEBUG

Select debug versions of some files

FULL DEBUG

Select debug versions of all files

-DEXCLUDE_JAVA

Exclude Java from ROM image

-DUSE_STRONG_CRYPTOGRAPHY

Includes strong cryptography.dll library into ROM

[feature manager options]

Feature manager is available for the Symbian platform.

[target device]

To build a development board ROM, specify the type of development board. For an H2 development board, use h2, and for an H4 development board, use h4hrp.

[components list]

Components to be included into ROM image:

Oby/iby files

Required for test type

Automated

Required for test type

Semi-automated

Note

techview.oby

yes

yes

TechView user interface component

PlatSec.oby

yes

yes

Enables platform security on a board

NoWatcher.iby

yes

yes

Disables running TechView watchers

statauto.iby

yes

no

STAT component

netperfdelayplugin.iby

yes

no

NetPerf delaymeter plugin and is available for Symbian platform

nerperf.iby

no

yes

NetPerf test suite (includes netperfdelayplugin.iby file)

testexecute.iby

optional

yes

TestExecute Framework (TEF)

ucc.iby

no

yes

User case controller component

ether802.iby

yes

yes

Ethernet driver for devboard

profiler.iby

optional

optional

Only include to measure on-target profiling data

ced.iby

yes

yes

Communications Database editor

ceddump.iby

optional

optional

Exports Communications Database configuration to a file.

ping.iby

optional

optional

Ping utility

ipconfig.iby

optional

optional

ipconfig utility

Note: On the command line specify the .oby component names first and then the .iby.

[card drive]

Specifies the drive where a flash card is mounted, for example d: