1 <#-- |
|
2 ============================================================================ |
|
3 Name : stage_ats.rst.inc.ftl |
|
4 Part of : Helium |
|
5 |
|
6 Copyright (c) 2009 Nokia Corporation and/or its subsidiary(-ies). |
|
7 All rights reserved. |
|
8 This component and the accompanying materials are made available |
|
9 under the terms of the License "Eclipse Public License v1.0" |
|
10 which accompanies this distribution, and is available |
|
11 at the URL "http://www.eclipse.org/legal/epl-v10.html". |
|
12 |
|
13 Initial Contributors: |
|
14 Nokia Corporation - initial contribution. |
|
15 |
|
16 Contributors: |
|
17 |
|
18 Description: |
|
19 |
|
20 ============================================================================ |
|
21 --> |
|
22 |
|
23 .. index:: |
|
24 single: ATS - STIF, TEF, RTEST, MTF and EUnit |
|
25 |
|
26 .. _`Stage-ATS-label`: |
|
27 |
|
28 Stage: ATS - STIF, TEF, RTEST, MTF and EUnit (also Qt) |
|
29 ======================================================= |
|
30 |
|
31 ATS testing is the automatic testing of the phone code once it has been compiled and linked to create a ROM image. |
|
32 |
|
33 Explanation of the process for getting ATS (`STIF`_ and `EUnit`_) tests compiled and executed by Helium, through the use of the :hlm-t:`ats-test` target. |
|
34 |
|
35 http://developer.symbian.org/wiki/index.php/Symbian_Test_Tools |
|
36 |
|
37 <#if !(ant?keys?seq_contains("sf"))> |
|
38 .. _`STIF`: http://s60wiki.nokia.com/S60Wiki/STIF |
|
39 .. _`EUnit`: http://s60wiki.nokia.com/S60Wiki/EUnit |
|
40 </#if> |
|
41 |
|
42 .. image:: ats.dot.png |
|
43 |
|
44 Prerequisites |
|
45 ---------------- |
|
46 |
|
47 * `Harmonized Test Interface (HTI)`_ needs to be compiled and into the image. |
|
48 * The reader is expected to already have a working ATS setup in which test cases can be executed. ATS server names, |
|
49 access rights and authentication etc. is supposed to be already taken care of. |
|
50 |
|
51 <#if !(ant?keys?seq_contains("sf"))> |
|
52 .. _`Harmonized Test Interface (HTI)`: http://s60wiki.nokia.com/S60Wiki/HTI |
|
53 <#else> |
|
54 .. _`Harmonized Test Interface (HTI)`: http://developer.symbian.org/wiki/index.php/HTI_Tool |
|
55 </#if> |
|
56 |
|
57 Test source components |
|
58 ------------------------- |
|
59 |
|
60 Test source usually lives in a component's ``tsrc`` directory. Test source components are created like any other Symbian SW component; |
|
61 there is a ``group`` directory with a ``bld.inf`` file for building, ``.mmp`` files for defining the targets, and so on. |
|
62 |
|
63 The test generation code expects ``.pkg`` file in the ``group`` directory of test component to be compiled, to get the paths of the files |
|
64 (can be data, configuration, initialization, etc files) to be installed and where to install on the phone. |
|
65 |
|
66 |
|
67 Three STEPS to setup ATS with Helium |
|
68 -------------------------------------- |
|
69 |
|
70 **Step 1: Configure System Definition Files** |
|
71 If the tsrc directory structure meets the criteria defined in the `new API test automation guidelines`_, then test components |
|
72 should be included in the System Definition files. |
|
73 |
|
74 **System Definition Files supporting layers.sysdef.xml** |
|
75 **layers** in ``layers.sysdef.xml`` file and **configuration** in ``build.sysdef.xml`` file (`Structure of System Definition files version 1.4`_). |
|
76 |
|
77 <#if !(ant?keys?seq_contains("sf"))> |
|
78 .. _`new API test automation guidelines`: http://s60wiki.nokia.com/S60Wiki/Test_Asset_Guidelines |
|
79 .. _`Structure of System Definition files version 1.4`: http://delivery.nmp.nokia.com/trac/helium/wiki/SystemDefinitionFiles |
|
80 </#if> |
|
81 |
|
82 A template of layer in layers.sysdef.xml for system definition files |
|
83 |
|
84 .. code-block:: xml |
|
85 |
|
86 <layer name="name_test_layer"> |
|
87 <module name="module_name_one"> |
|
88 <unit unitID="unit_id1" name="unit_name1" bldFile="path_of_tsrc_folder_to_be_built" mrp="" /> |
|
89 </module> |
|
90 |
|
91 <module name="module_name_two"> |
|
92 <unit unitID="unit_id2" name="unit_name2" bldFile="path_of_tsrc_folder_to_be_built" mrp="" /> |
|
93 </module> |
|
94 </layer> |
|
95 |
|
96 * Layer name should end with **_test_layer** |
|
97 * Two standard names for ATS test layers are being used; ``unit_test_layer`` and ``api_test_layer``. Test components (the``unit`` tags) |
|
98 should be specified under these layers and grouped by ``module`` tag(s). |
|
99 * In the above, two modules means two drop files will be created; ``module`` may have one or more ``unit`` |
|
100 * By using property ``exclude.test.layers``, complete layers can be excluded and the components inside that layer will not be included in the AtsDrop. This property is a comma (,) separated list |
|
101 |
|
102 **System Definition Files version 3.0 (SysDefs3)** (new Helium v.10.79) |
|
103 The `structure of System Definition files version 3.0`_ is different than previous versions of system definition files. In SysDefs3, package definition files are used for components specification. Instead of layers naming conventions, filters are used to identify test components and test types, for example: "test, unit_test, !api_test" etc. |
|
104 |
|
105 <#if !(ant?keys?seq_contains("sf"))> |
|
106 .. _`structure of System Definition files version 3.0`: http://wikis.in.nokia.com/view/SWManageabilityTeamWiki/PkgdefUse |
|
107 <#else> |
|
108 .. _`structure of System Definition files version 3.0`: sysdef3.rst |
|
109 </#if> |
|
110 |
|
111 An example template for defining test components in a package definition file. |
|
112 |
|
113 .. code-block:: xml |
|
114 |
|
115 <package id="dummytest" name="dummytest" levels="demo"> |
|
116 <collection id="test_nested" name="test_nested" level="demo"> |
|
117 |
|
118 <component id="tc1" name="tc1" purpose="development" filter="test, unit_test"> |
|
119 <unit bldFile="test_nested/tc1/group" mrp="" /> |
|
120 </component> |
|
121 |
|
122 <component id="tc2" name="tc2" purpose="development" filter="test"> |
|
123 <meta rel="testbuild"> |
|
124 <group name="drop_tc2_and_tc3" /> |
|
125 </meta> |
|
126 <unit bldFile="test_nested/tc2/group" mrp="" /> |
|
127 </component> |
|
128 |
|
129 <component id="tc3" name="tc3" purpose="development" filter="test"> |
|
130 <meta rel="testbuild"> |
|
131 <group name="drop_tc2_and_tc3" /> |
|
132 </meta> |
|
133 <unit bldFile="test_nested/tc3/group" mrp="" /> |
|
134 </component> |
|
135 |
|
136 </collection> |
|
137 </package> |
|
138 |
|
139 * Filter "test" must be specified for every test component. If it is not specified, the component will not be considered as a test component. |
|
140 * <meta>/<group> are now used to group test components, it work in the same way as <module>...<module> in sysdef v1.4 works. The components having same group name are grouped together. |
|
141 Separate drop files are created for different groups. In the above example, if only 'test' is selected, then two drop files will be created, one with tc1 and the other one with tc2 and tc3. |
|
142 |
|
143 |
|
144 **Step 2: Configure ATS properties in build.xml** |
|
145 |
|
146 **(A)** Username and Password for the ATS should be set in the `.netrc file`_:: |
|
147 |
|
148 machine ats login ats_user_name password ats_password |
|
149 |
|
150 Add the above line in the ``.netrc`` file and replace ``ats_user_name`` with your real ATS username and ``ats_password`` with ATS password. |
|
151 |
|
152 **(B)** The following properties are ATS dependent with their edit status |
|
153 |
|
154 * [must] - must be set by user |
|
155 * [recommended] - should be set by user but not mandatory |
|
156 * [allowed] - should **not** be set by user however, it is possible. |
|
157 |
|
158 .. csv-table:: ATS Ant properties |
|
159 :header: "Property name", "Edit status", "Description" |
|
160 |
|
161 ":hlm-p:`ats.server`", "[must]", "For example: ``4fix012345`` or ``catstresrv001.company.net:80``. Default server port is ``8080``, but it is not allowed between intra and Noklab. Because of this we need to define server port as 80. The host can be different depending on site and/or product." |
|
162 ":hlm-p:`ats.drop.location`", "[allowed]", "Server location (UNC path) to save the ATSDrop file, before sending to the ATS Server. For example: ``\\\\trwsem00\\some_folder\\``. In case, :hlm-p:`ats.script.type` is set to ``import``, ATS doesn't need to have access to :hlm-p:`ats.drop.location`, its value can be any local folder on build machine, for example ``c:/temp`` (no network share needed)." |
|
163 ":hlm-p:`ats.product.name`", "[must]", "Name of the product to be tested." |
|
164 ":hlm-p:`eunit.test.package`", "[allowed]", "The EUnit package name to be unzipped on the environment, for executing EUnit tests." |
|
165 ":hlm-p:`eunitexerunner.flags`", "[allowed]", "Flags for EUnit exerunner can be set by setting the value of this variable. The default flags are set to ``/E S60AppEnv /R Off``." |
|
166 ":hlm-p:`ats.email.list`", "[allowed]", "The property is needed if you want to get an email from ATS server after the tests are executed. There can be one to many semicolon-separated email addresses." |
|
167 ":hlm-p:`ats.report.type`", "[allowed]", "Value of the ats email report, for ATS4 set to 'no_attachment' so email size is reduced" |
|
168 ":hlm-p:`ats.flashfiles.minlimit`", "[allowed]", "Limit of minimum number of flash files to execute :hlm-t:`ats-test` target, otherwise ``ATSDrop.zip`` will not be generated. Default value is 2 files." |
|
169 ":hlm-p:`ats.plan.name`", "[allowed]", "Modify the plan name if you have understanding of ``test.xml`` file or leave it as it is. Default value is ``plan``." |
|
170 ":hlm-p:`ats.product.hwid`", "[allowed]", "Product HardWare ID (HWID) attached to ATS. By default the value of HWID is not set." |
|
171 ":hlm-p:`ats.script.type`", "[allowed]", "There are two types of ats script files to send drop to ATS server, ``runx`` and ``import``; only difference is that with ``import`` ATS doesn't have to have access rights to ``testdrop.zip`` file, as it is sent to the system over http and import doesn't need network shares. If that is not needed ``import`` should not be used. Default value is ``runx`` as ``import`` involves heavy processing on ATS server." |
|
172 ":hlm-p:`ats.target.platform`", "[allowed]", "Sets target platform for compiling test components. Default value is ``armv5 urel``." |
|
173 ":hlm-p:`ats.test.timeout`", "[allowed]", "To set test commands execution time limit on ATS server, in seconds. Default value is ``60``." |
|
174 ":hlm-p:`ats.testrun.name`", "[allowed]", "Modify the test-run name if you have understanding of ``test.xml`` file or leave it as it is. Default value is a string consist of build id, product name, major and minor versions." |
|
175 ":hlm-p:`ats.trace.enabled`", "[allowed]", "Should be ``true`` if tracing is needed during the tests running on ATS. Default value is ``false``, the values are case-sensitive. See http://s60wiki.nokia.com/S60Wiki/CATS/TraceTools." |
|
176 ":hlm-p:`ats.ctc.enabled`", "[allowed]", "Should be ``true`` if coverage measurement and dynamic analysis (CTC) tool support is to be used by ATS. Default value is ``false``. The values are case-sensitive." |
|
177 ":hlm-p:`ats.ctc.host`", "[allowed]", "CTC host, provided by CATS used to create coverage measurement reports. MON.sym files are copied to this location, for example ``10.0.0.1``. If not given, code coverage reports are not created" |
|
178 ":hlm-p:`ats.obey.pkgfiles.rule`", "[allowed]", "If the property is set to ``true``, then the only test components which will have PKG files, will be included into the ``test.xml`` as a test-set. Which means, even if there's a test component (executable) but there's no PKG file, it should not be considered as a test component and hence not included into the test.xml as a separate test. By default the property value is ``false``." |
|
179 "``reference.ats.flash.images``", "[allowed]", "Fileset for list of flash images (can be .fpsx, .C00, .V01 etc) It is recommended to set the fileset, default filset is given below which can be overwritten. set *dir=""* attribute of the filset to ``${r'$'}{build.output.dir}/variant_images`` if hlm-t:`variant-image-creation` target is being used." |
|
180 ":hlm-p:`tsrc.data.dir`", "[allowed]", "The default value is ``data`` and refers to the 'data' directory under 'tsrc' directory." |
|
181 ":hlm-p:`tsrc.path.list`", "[allowed]", "Contains list of the tsrc directories. Gets the list from system definition layer files. Assuming that the test components are defined already in te ``layers.sysdef.xml`` files to get compiled. Not recommended, but the property value can be set if there are no System Definition file(s), and tsrc directories paths to set manually." |
|
182 ":hlm-p:`ats.report.location`", "[allowed]", "Sets ATS reports store location. Default location is ``${r'$'}{publish.dir}/${r'$'}{publish.subdir}``." |
|
183 ":hlm-p:`ats.multiset.enabled`", "[allowed]", "Should be ``true`` so a set is used for each pkg file in a component, this allows tests to run in parallel on several devices." |
|
184 ":hlm-p:`ats.diamonds.signal`", "[allowed]", "Should be ``true`` so at end of the build diamonds is checked for test results and Helium fails if tests failed." |
|
185 ":hlm-p:`ats.delta.enabled`", "[allowed]", "Should be ``true`` so only ADOs changed during :hlm-t:`do-prep-work-area` are tested by ATS." |
|
186 ":hlm-p:`ats4.enabled`", "[allowed]", "Should be ``true`` if ATS4 is to be used." |
|
187 ":hlm-p:`ats.emulator.enable`", "[allowed]", "Should be ``true`` if ``WINSCW`` emulator is to be used." |
|
188 ":hlm-p:`ats.specific.pkg`", "[allowed]", "Text in name of PKG files to use eg. 'sanity' would only use xxxsanity.pkg files from components." |
|
189 ":hlm-p:`ats.singledrop.enabled`", "[allowed]", "If present and set to 'true', it will create one drop file, if set to any other value or not present it will create multiple drop files (defined by the sysdef file). This is to save traffic to the server." |
|
190 ":hlm-p:`ats.java.importer.enabled`", "[allowed]", "If set to 'true', for older uploader is used for ats3 which shows improved error message." |
|
191 ":hlm-p:`ats.test.filterset`", "[allowed]", "(new Helium v.10.79)Contains a name of test filterset (see example below). A filterset is used to select/unselect test components. The filter(s) is/are effective when the same filters are defined in the package definition file for component(s)." |
|
192 |
|
193 An example of setting up properties: |
|
194 |
|
195 .. code-block:: xml |
|
196 |
|
197 <property name="ats.server" value="4fio00105" /> |
|
198 <property name="ats.drop.location" location="\\trwsimXX\ATS_TEST_SHARE\" /> |
|
199 <property name="ats.email.list" value="temp.user@company.com; another.email@company.com" /> |
|
200 <property name="ats.flashfiles.minlimit" value="2" /> |
|
201 <property name="ats.product.name" value="PRODUCT" /> |
|
202 <property name="ats.plan.name" value="plan" /> |
|
203 <property name="ats.product.hwid" value="" /> |
|
204 <property name="ats.script.type" value="runx" /> |
|
205 <property name="ats.target.platform" value="armv5 urel" /> |
|
206 <property name="ats.test.timeout" value="60" /> |
|
207 <property name="ats.testrun.name" value="${r'$'}{build.id}_${r'$'}{ats.product.name}_${r'$'}{major.version}.${r'$'}{minor.version}" /> |
|
208 <property name="ats.trace.enabled" value="false" /> |
|
209 <property name="ats.ctc.enabled" value="false" /> |
|
210 <property name="ats.obey.pkgfiles.rule" value="false" /> |
|
211 <property name="ats.report.location" value="${r'$'}{publish.dir}/${r'$'}{publish.subdir}" /> |
|
212 <property name="eunit.test.package" value="" /> |
|
213 <property name="eunitexerunner.flags" value="/E S60AppEnv /R Off" /> |
|
214 <property name="ats.test.filterset" value="sysdef.filters.tests" /> |
|
215 |
|
216 <hlm:sysdefFilterSet id="sysdef.filters.tests"> |
|
217 <filter filter="test, " type="has" /> |
|
218 <config file="bldvariant.hrh" includes="" /> |
|
219 </hlm:sysdefFilterSet> |
|
220 |
|
221 |
|
222 ... |
|
223 <import file="${r'$'}{helium.dir}/helium.ant.xml" /> |
|
224 ... |
|
225 |
|
226 <fileset id="reference.ats.flash.images" dir="${r'$'}{release.images.dir}"> |
|
227 <include name="**/${r'$'}{build.id}*.core.fpsx"/> |
|
228 <include name="**/${r'$'}{build.id}*.rofs2.fpsx"/> |
|
229 <include name="**/${r'$'}{build.id}*.rofs3.fpsx"/> |
|
230 </fileset> |
|
231 |
|
232 |
|
233 .. Note:: |
|
234 |
|
235 Always declare *Properties* before and *filesets* after importing helium.ant.xml. |
|
236 |
|
237 **STEP 3: Call target ats-test** |
|
238 |
|
239 To execute the target, a property should be set(``<property name="ats.enabled" value="true" />``). |
|
240 |
|
241 Then call :hlm-t:`ats-test`, which will create the ATSDrop.zip (test package). |
|
242 |
|
243 If property *ats.email.list* is set, an email (test report) will be sent when the tests are ready on ATS. |
|
244 |
|
245 CTC: |
|
246 ---- |
|
247 |
|
248 CTC code coverage measurements reports can be created as part of Test Automation process. |
|
249 |
|
250 1. Build the src using ``build_ctc`` configuration, which is in ``build.sysdef.xml`` file, to create ``MON.sym`` files. It means that a property ``sysdef.configurations.list`` should be modified either add or replace current build configuration with ``build_ctc`` |
|
251 |
|
252 2. Set the property, ``ats.ctc.host``, as described above, for sending the ``MON.sym`` files to the network drive. *(Please contact ATS server administrator and ask for the value to set this property)* |
|
253 |
|
254 3. Enable CTC process by setting up property ``ats.ctc.enabled`` to "true" |
|
255 |
|
256 4. Test drops are sent to the ATS server, where, after executing tests ``ctcdata.txt`` files are created. ``ctcdata.txt`` and ``MON.sym`` files are then further processed to create code coverage reports. |
|
257 |
|
258 5. View or download the Code coverage reports by following the link provided in the ATS report email (sent after the tests are executed on ATS) |
|
259 |
|
260 *NOTE: After receiving the email notification, it may take a few minutes before the code coverage reports are available.* |
|
261 |
|
262 |
|
263 Qt Tests: |
|
264 --------- |
|
265 |
|
266 QtTest.lib is supported and the default harness is set to EUnit. If ``QtTest.lib`` is there in ``.mmp`` file, Helium sets the Harness to Eunit and ATS supported Qt steps are added to test.xml file |
|
267 |
|
268 In ``layers.sysdef.xml`` file, the layer name should end with "_test_layer" e.g. "qt_unit_test_layer". |
|
269 |
|
270 There are several ``.PKG`` files created after executing ``qmake``, but only one is selected based on which target platform is set. Please read the property (``ats.target.platform``) description above. |
|
271 |
|
272 .. _`Skip-Sending-AtsDrop-label`: |
|
273 |
|
274 Skip Sending AtsDrop to ATS |
|
275 ---------------------------- |
|
276 |
|
277 By setting property of ``ats.upload.enabled`` to ``false``, ``ats-test`` target only creates a drop file, and does not send the drop (or package) to ATS server. |
|
278 |
|
279 Customizing the test.xml in ATS |
|
280 -------------------------------- |
|
281 |
|
282 The user can customize the generated test.xml with files: |
|
283 |
|
284 * **preset_custom.xml** goes before first set |
|
285 * **postset_custom.xml** goes after last set |
|
286 * **precase_custom.xml** goes before first case |
|
287 * **postcase_custom.xml** goes after last case |
|
288 * **prestep_custom.xml** goes before first step |
|
289 * **poststep_custom.xml** goes after last step |
|
290 * **prerun_custom.xml** goes before first run or execute step |
|
291 * **postrun_custom.xml** goes after last run or execute step |
|
292 * **prepostaction.xml** goes before first postaction |
|
293 * **postpostaction.xml** goes after last postaction |
|
294 |
|
295 The files must be in the directory 'custom' under the 'tsrc' or 'group' folder to be processed. |
|
296 |
|
297 The files need to be proper XML snippets that fit to their place. In case of an error an error is logged and a comment inserted to the generated XML file. |
|
298 |
|
299 A postaction section customization file (prepostaction.xml or postpostaction.xml) could look like this |
|
300 |
|
301 .. code-block:: xml |
|
302 |
|
303 <postAction> |
|
304 <type>Pre PostAction from custom file</type> |
|
305 <params> |
|
306 <param name="foo2" value="bar2" /> |
|
307 </params> |
|
308 </postAction> |
|
309 |
|
310 |
|
311 |
|
312 The ``prestep_custom.xml`` can be used to flash and unstall something custom. |
|
313 |
|
314 .. code-block:: xml |
|
315 |
|
316 <step name="Install measurement tools" harness="STIF" significant="false"> |
|
317 <!-- Copy SIS-packages to DUT --> |
|
318 <command>install</command> |
|
319 <params> |
|
320 <param src="Nokia_Energy_Profiler_1_1.sisx"/> |
|
321 <param dst="c:\data\Nokia_Energy_Profiler_1_1.sisx"/> |
|
322 </params> |
|
323 ... |
|
324 </step> |
|
325 |
|
326 |
|
327 And then the ``prerun_custom.xml`` can be used to start measuring. |
|
328 |
|
329 .. code-block:: xml |
|
330 |
|
331 <step name="Start measurement" harness="STIF" significant="false"> |
|
332 <!-- Start measurement --> |
|
333 <command>execute</command> |
|
334 <params> |
|
335 <param file="neplauncher.exe"/> |
|
336 <param parameters="start c:\data\nep.csv"/> |
|
337 <param timeout="30"/> |
|
338 </params> |
|
339 </step> |
|
340 |
|
341 |
|
342 |
|
343 **Note:** The users is expected to check the generated test.xml manually, as there is no validation. Invalid XML input files will be disregarded and a comment will be inserted to the generated XML file. |
|
344 |
|
345 Overriding Test xml values |
|
346 -------------------------- |
|
347 |
|
348 Set the property ``ats.config.file`` to the location of the config file. |
|
349 |
|
350 Example configuration: |
|
351 |
|
352 .. code-block:: xml |
|
353 |
|
354 <ATSConfigData> |
|
355 <config name="common" abstract="true"> |
|
356 |
|
357 <!-- Properties to add/ modify --> |
|
358 <config type="properties"> |
|
359 <set name="HARNESS" value="STIF" /> |
|
360 <set name="2" value="3" /> |
|
361 </config> |
|
362 |
|
363 <!-- Settings to add/ modify --> |
|
364 <config type="settings"> |
|
365 <set name="HARNESS" value="STIF" /> |
|
366 <set name="2" value="3" /> |
|
367 </config> |
|
368 |
|
369 <!-- Attributes to modify --> |
|
370 <config type="attributes"> |
|
371 <set name="xyz" value="2" /> |
|
372 <set name="significant" value="true" /> |
|
373 </config> |
|
374 </config> |
|
375 </ATSConfigData> |
|
376 |
|
377 |
|
378 .. index:: |
|
379 single: ATS - ASTE |
|
380 |
|
381 Stage: ATS - ASTE |
|
382 =================== |
|
383 |
|
384 Explanation of the process for getting ATS `ASTE`_ tests compiled and executed by Helium, through the use of the :hlm-t:`ats-aste` target. |
|
385 |
|
386 <#if !(ant?keys?seq_contains("sf"))> |
|
387 .. _`ASTE`: http://s60wiki.nokia.com/S60Wiki/ASTE |
|
388 </#if> |
|
389 |
|
390 Prerequisites |
|
391 -------------- |
|
392 |
|
393 * `Harmonized Test Interface (HTI)`_ needs to be compiled and into the image. |
|
394 * The reader is expected to already have a working ATS setup in which test cases can be executed. ATS server names, access rights and authentication etc. is supposed to be already taken care of. |
|
395 * `SW Test Asset`_ location and type of test should be known. |
|
396 |
|
397 <#if !(ant?keys?seq_contains("sf"))> |
|
398 .. _`Harmonized Test Interface (HTI)`: http://s60wiki.nokia.com/S60Wiki/HTI |
|
399 .. _`SW Test Asset`: http://s60wiki.nokia.com/S60Wiki/MC_SW_Test_Asset_documentation |
|
400 </#if> |
|
401 |
|
402 Test source components |
|
403 -------------------------- |
|
404 |
|
405 Unlike STIF, EUnit etc tests, test source components (or ``tsrc`` structure) is not needed for `ASTE`_ tests. |
|
406 |
|
407 Two STEPS to setup ASTE with Helium |
|
408 ------------------------------------ |
|
409 |
|
410 **STEP 1: Configure ASTE properties in build.xml** |
|
411 |
|
412 **(A)** Username and Password for the ATS should be set in the `.netrc file`_ |
|
413 |
|
414 .. code-block:: text |
|
415 |
|
416 machine ats login ats_user_name password ats_password |
|
417 |
|
418 Add the above line in the .netrc file and replace *ats_user_name* with your real ats username and "ats_password" with ats password. |
|
419 |
|
420 .. _`.netrc file`: configuring.html?highlight=netrc#passwords |
|
421 |
|
422 |
|
423 **(B)** The following properties are ASTE dependent with their edit status |
|
424 |
|
425 * [must] - must be set by user |
|
426 * [recommended] - should be set by user but not mandatory |
|
427 * [allowed] - should **not** be set by user however, it is possible. |
|
428 |
|
429 .. csv-table:: ATS Ant properties |
|
430 :header: "Property name", "Edit status", "Description" |
|
431 |
|
432 ":hlm-p:`ats.server`", "[must]", "For example: ``4fio00105`` or ``catstresrv001.company.net:80``. Default server port is ``8080``, but it is not allowed between intra and Noklab. Because of this we need to define server port as ``80``. The host can be different depending on site and/or product." |
|
433 ":hlm-p:`ats.drop.location`", "[must]", "Server location (UNC path) to save the ATSDrop file, before sending to the ATS. For example: ``\\\\trwsem00\\some_folder\\``. In case, ``ats.script.type`` is set to ``import``, ATS doesn't need to have access to :hlm-p:`ats.drop.location`, its value can be any local folder on build machine, for example ``c:/temp`` (no network share needed)." |
|
434 ":hlm-p:`ats.product.name`", "[must]", "Name of the product to be tested." |
|
435 ":hlm-p:`ats.aste.testasset.location`", "[must]", "Location of SW Test Assets, if the TestAsset is not packaged then it is first compressed to a ``.zip`` file. It should be a UNC path." |
|
436 ":hlm-p:`ats.aste.software.release`", "[must]", "Flash images releases, for example 'SPP 51.32'." |
|
437 ":hlm-p:`ats.aste.software.version`", "[must]", "Version of the software to be tested. For example: 'W810'" |
|
438 ":hlm-p:`ats.aste.email.list`", "[recommended]", "The property is needed if you want to get an email from ATS server after the tests are executed. There can be one to many semicolon(s) ";" separated email addresses." |
|
439 ":hlm-p:`ats.flashfiles.minlimit`", "[recommended]", "Limit of minimum number of flash files to execute ats-test target, otherwise ATSDrop.zip will not be generated. Default value is "2" files." |
|
440 ":hlm-p:`ats.aste.plan.name`", "[recommended]", "Modify the plan name if you have understanding of test.xml file or leave it as it is. Default value is "plan"." |
|
441 ":hlm-p:`ats.product.hwid`", "[recommended]", "Product HardWare ID (HWID) attached to ATS. By default the value of HWID is not set." |
|
442 ":hlm-p:`ats.test.timeout`", "[recommended]", "To set test commands execution time limit on ATS server, in seconds. Default value is '60'." |
|
443 ":hlm-p:`ats.aste.testrun.name`", "[recommended]", "Modify the test-run name if you have understanding of ``test.xml`` file or leave it as it is. Default value is a string consists of build id, product name, major and minor versions." |
|
444 ":hlm-p:`ats.aste.test.type`", "[recommended]", "Type of test to run. Default is 'smoke'." |
|
445 ":hlm-p:`ats.aste.testasset.caseids`", "[recommended]", "These are the cases that which tests should be run from the TestAsset. For example, value can be set as ``100,101,102,103,105,106,``. A comma is needed to separate case IDs" |
|
446 ":hlm-p:`ats.aste.language`", "[recommended]", "Variant Language to be tested. Default is 'English'" |
|
447 "``reference.ats.flash.images``", "[recommended]", "Fileset for list of flash images (can be .fpsx, .C00, .V01 etc) It is recommended to set the fileset, default filset is given below which can be overwritten. set *dir=\"\"* attribute of the filset to ``${r'$'}{build.output.dir}/variant_images`` if :hlm-t:`variant-image-creation` target is being used." |
|
448 |
|
449 |
|
450 An example of setting up properties: |
|
451 |
|
452 .. code-block:: xml |
|
453 |
|
454 <property name="ats.server" value="4fio00105" /> |
|
455 <property name="ats.drop.location" value="\\trwsimXX\ATS_TEST_SHARE\" /> |
|
456 <property name="ats.aste.email.list" value="temp.user@company.com; another.email@company.com" /> |
|
457 <property name="ats.flashfiles.minlimit" value="2" /> |
|
458 <property name="ats.product.name" value="PRODUCT" /> |
|
459 <property name="ats.aste.plan.name" value="plan" /> |
|
460 <property name="ats.product.hwid" value="" /> |
|
461 <property name="ats.test.timeout" value="60" /> |
|
462 <property name="ats.aste.testrun.name" value="${r'$'}{build.id}_${r'$'}{ats.product.name}_${r'$'}{major.version}.${r'$'}{minor.version}" /> |
|
463 <property name="ats.aste.testasset.location" value="" /> |
|
464 <property name="ats.aste.software.release" value="SPP 51.32" /> |
|
465 <property name="ats.aste.test.type" value="smoke" /> |
|
466 <property name="ats.aste.testasset.caseids" value="100,101,102,104,106," /> |
|
467 <property name="ats.aste.software.version" value="W810" /> |
|
468 <property name="ats.aste.language" value="English" /> |
|
469 |
|
470 ... |
|
471 <import file="${r'$'}{helium.dir}/helium.ant.xml" /> |
|
472 ... |
|
473 |
|
474 <fileset id="reference.ats.flash.images" dir="${r'$'}{release.images.dir}"> |
|
475 <include name="**/${r'$'}{build.id}*.core.fpsx"/> |
|
476 <include name="**/${r'$'}{build.id}*.rofs2.fpsx"/> |
|
477 <include name="**/${r'$'}{build.id}*.rofs3.fpsx"/> |
|
478 </fileset> |
|
479 |
|
480 |
|
481 *PLEASE NOTE:* Always declare *Properties* before and *filesets* after importing helium.ant.xml. |
|
482 |
|
483 **STEP 2: Call target ats-aste** |
|
484 |
|
485 To execute the target, a property should be set(``<property name="enabled.aste" value="true" />``). |
|
486 |
|
487 Then call :hlm-t:`ats-aste`, which will create the ATSDrop.zip (test package). |
|
488 |
|
489 If property ``ats.aste.email.list`` is set, an email (test report) will be sent when the tests are ready on ATS/ASTE. |
|
490 |
|
491 |
|
492 Skip Sending AtsDrop to ATS |
|
493 ------------------------------ |
|
494 |
|
495 click :ref:`Skip-Sending-AtsDrop-label`: |
|
496 |
|
497 Stage: ATS - Custom Drop |
|
498 ======================== |
|
499 |
|
500 If you need to send a static drop to ATS then you can call the target :hlm-t:`ats-custom-drop` and set a property to your own template. |
|
501 |
|
502 A example template is in helium/tools/testing/ats/templates/ats4_naviengine_template.xml |
|
503 |
|
504 .. code-block:: xml |
|
505 |
|
506 <property name="ats.custom.template" value="path/to/mytemplate.xml" /> |
|