|
1 .. ============================================================================ |
|
2 Name : metrics.rst |
|
3 Part of : Helium |
|
4 |
|
5 Copyright (c) 2009 Nokia Corporation and/or its subsidiary(-ies). |
|
6 All rights reserved. |
|
7 This component and the accompanying materials are made available |
|
8 under the terms of the License "Eclipse Public License v1.0" |
|
9 which accompanies this distribution, and is available |
|
10 at the URL "http://www.eclipse.org/legal/epl-v10.html". |
|
11 |
|
12 Initial Contributors: |
|
13 Nokia Corporation - initial contribution. |
|
14 |
|
15 Contributors: |
|
16 |
|
17 Description: |
|
18 |
|
19 ============================================================================ |
|
20 |
|
21 #################### |
|
22 Helium Build Metrics |
|
23 #################### |
|
24 |
|
25 .. index:: |
|
26 module: Helium Build Metrics |
|
27 |
|
28 |
|
29 .. contents:: |
|
30 |
|
31 Introduction |
|
32 ============ |
|
33 |
|
34 This describes the types of build and release metrics that can be collected using Helium and a Diamonds installation and how they can be collected. |
|
35 |
|
36 .. index:: |
|
37 single: Diamonds |
|
38 |
|
39 Diamonds Link for builds: |
|
40 ========================= |
|
41 Diamonds homepage: <http://diamonds.nmp.nokia.com/diamonds> |
|
42 |
|
43 |
|
44 Helium configuration |
|
45 ==================== |
|
46 To enable logging to diamonds from Helium one needs to ensure that: |
|
47 |
|
48 * The properties :hlm-p:`diamonds.host` and :hlm-p:`diamonds.port` are set correctly. |
|
49 * By default they are taken from ``helium/tools/common/companyproperties.ant.xml``, but can be overridden by using: |
|
50 |
|
51 * **Command line** |
|
52 |
|
53 For example, if the Diamonds server IP address changed to ``new.diamonds.server:newport`` then you would use: |
|
54 |
|
55 * *hlm.bat -Ddiamonds.host=new.diamonds.server -Ddiamonds.port=newport* |
|
56 |
|
57 * **ANT team file** |
|
58 |
|
59 For example, if the Diamonds server IP address changed to ``new.diamonds.server:newport`` then you would put the following lines in your ``<team>.ant.xml``:: |
|
60 |
|
61 <property name="diamonds.host" value="new.diamonds.server"/>* |
|
62 <property name="diamonds.port" value="newport"/>* |
|
63 |
|
64 * If you define the property :hlm-p:`skip.diamonds` to ``true`` Diamonds is disabled. |
|
65 |
|
66 |
|
67 .. index:: |
|
68 single: Diamonds server configuration |
|
69 |
|
70 Diamonds server configuration |
|
71 ============================= |
|
72 |
|
73 Config file: ``helium/config/diamonds_config.xml.ftl``. |
|
74 |
|
75 |
|
76 Properties need to be defined for successful logging: |
|
77 ----------------------------------------------------- |
|
78 |
|
79 |
|
80 .. csv-table:: Diamonds logging related properties |
|
81 :header: "Property name", "Description" |
|
82 |
|
83 "``diamonds.host``", "Diamonds server address" |
|
84 "``diamonds.port``", "Server port number" |
|
85 "``diamonds.path``", "Builds path in Diamonds server" |
|
86 ":hlm-p:`build.family`", "Category of product" |
|
87 ":hlm-p:`stages`", "Start and end target of a stages with logical stage name" |
|
88 ":hlm-p:`sysdef.configurations.list`", "System definition name list to log component faults" |
|
89 ":hlm-p:`build.name`", "Name of product" |
|
90 ":hlm-p:`release.label`", "Name of release" |
|
91 ":hlm-p:`publish`", "Set this property to publish to network" |
|
92 ":hlm-p:`publish.dir`", "Published build environment location" |
|
93 ":hlm-p:`diamonds.build.tags`", "Set this property to send custom build tag(s) to Diamonds" |
|
94 |
|
95 |
|
96 .. index:: |
|
97 single: Metrics |
|
98 |
|
99 Metrics |
|
100 ======= |
|
101 |
|
102 |
|
103 Metrics name: Build duration |
|
104 ---------------------------- |
|
105 |
|
106 Description |
|
107 ~~~~~~~~~~~~ |
|
108 Build duration in hours as a function of time. |
|
109 |
|
110 Collection method |
|
111 ~~~~~~~~~~~~~~~~~~~ |
|
112 The started time and finished time are uploaded to diamonds automatically from Helium. |
|
113 |
|
114 Location in Diamonds |
|
115 ~~~~~~~~~~~~~~~~~~~~~~ |
|
116 In Diamonds, Builds->Summary. |
|
117 |
|
118 For categorization by product programs, Build->Click "category" hyperlink. For |
|
119 categorization by build accelerators, Build->Other->Click "Build system" hyperlink. |
|
120 |
|
121 |
|
122 RVCT compiler warnings |
|
123 ---------------------- |
|
124 |
|
125 Description |
|
126 ~~~~~~~~~~~ |
|
127 Number of build warnings in SW build - RVCT compiler warnings to tell about the quality of the software. |
|
128 |
|
129 Collection Method |
|
130 ~~~~~~~~~~~~~~~~~ |
|
131 Number of RVCT bad warnings, warnings and errors are send to diamond aumatically from Helium after each build. |
|
132 |
|
133 Location in Diamonds |
|
134 ~~~~~~~~~~~~~~~~~~~~~~ |
|
135 In Diamonds, Builds->Summary->Compilation error summary. |
|
136 |
|
137 |
|
138 Metrics name: "number of object files" and "number of generated files" |
|
139 ---------------------------------------------------------------------- |
|
140 |
|
141 Description |
|
142 ~~~~~~~~~~~~ |
|
143 Number of object files and generated files for a build |
|
144 |
|
145 Collection method |
|
146 ~~~~~~~~~~~~~~~~~~~ |
|
147 Necessary data are collected from build information automatically |
|
148 |
|
149 Based on helium/config/diamonds_config.xml.ftl cofiguration, Helium automatically sends the start and end time of a stage to diamonds. |
|
150 |
|
151 |
|
152 Location in Diamonds |
|
153 ~~~~~~~~~~~~~~~~~~~~~~ |
|
154 In Diamonds, Builds->Others->Object files & Generated files. |
|
155 |
|
156 |
|
157 Metrics name: Build stage duration |
|
158 ---------------------------------- |
|
159 |
|
160 Description |
|
161 ~~~~~~~~~~~~ |
|
162 Date and time of start and finish. A=Date and time of start B= Date and time of finish. |
|
163 Metric = B-A calculated for each build stages. In the graph only the 4 main stages are shown.\ |
|
164 |
|
165 * *1. pre-build (Synergy check outs and snapshots, build area preparation)* |
|
166 * *2. build (main build)* |
|
167 * *3. post build (Post build, China, Japan, EE images, EE zip, Localization, Localized roms)* |
|
168 * *4. release to channels (db, ftp, network disk)* |
|
169 |
|
170 Collection method |
|
171 ~~~~~~~~~~~~~~~~~~~ |
|
172 The started time and finished time are uploaded to diamonds automatically from Helium. |
|
173 |
|
174 Location in Diamonds |
|
175 ~~~~~~~~~~~~~~~~~~~~~~ |
|
176 In Diamonds, Builds->Summary. Click "Stages>>" |
|
177 |
|
178 |
|
179 Metrics name: API Usage |
|
180 ---------------------------------- |
|
181 |
|
182 Description |
|
183 ~~~~~~~~~~~~ |
|
184 Types of api are private, internal, domain and sdk. Illegal API is (internal+private), if any illegal api exists it will show the Illegal API's name with path. |
|
185 |
|
186 Collection method |
|
187 ~~~~~~~~~~~~~~~~~~~ |
|
188 If disable.analysis.tool is not set, data will be uploaded to diamonds automatically from Helium. |
|
189 |
|
190 Location in Diamonds |
|
191 ~~~~~~~~~~~~~~~~~~~~~~ |
|
192 In Diamonds, Builds->Other->API usage |
|
193 |
|
194 |
|
195 Metrics name: Build tags |
|
196 ------------------------ |
|
197 |
|
198 Description |
|
199 ~~~~~~~~~~~~ |
|
200 Build tags are used to group builds for metric collection purposes. |
|
201 |
|
202 Collection method |
|
203 ~~~~~~~~~~~~~~~~~~~ |
|
204 To send custom build tags to Diamonds the property diamonds.build.tags should be set as follows: |
|
205 * For a single build tag (e.g. "build_tag1") -> hlm -Ddiamonds.build.tags="build_tag1" |
|
206 * For multiple build tags (e.g. "build_tag1" and "build_tagN") -> hlm -Ddiamonds.build.tags="build_tag1,build_tagN" |
|
207 |
|
208 Note: |
|
209 * Build tags should not exceed 50 characters. |
|
210 * Duplicate build tags will be ignored. |
|
211 * If an "Available Tag" is set, then in Diamonds it gets removed from that list and transferred to "Build's Tags" list. |
|
212 |
|
213 Location in Diamonds |
|
214 ~~~~~~~~~~~~~~~~~~~~~~ |
|
215 In Diamonds, Builds->Tags->Build's Tags. |
|
216 |
|
217 For categorization by tags, Click Builds->"Navigation" pane->"Build Archives"->by tags |
|
218 |
|
219 |
|
220 Metrics name: Information about "base environment" |
|
221 -------------------------------------------------- |
|
222 |
|
223 Description |
|
224 ~~~~~~~~~~~~ |
|
225 Information about what "base environment" is unzipped. |
|
226 |
|
227 Collection method |
|
228 ~~~~~~~~~~~~~~~~~~~ |
|
229 Necessary data are collected from build information automatically if currentRelease.xml exists in the environment. |
|
230 |
|
231 Location in Diamonds |
|
232 ~~~~~~~~~~~~~~~~~~~~~~ |
|
233 In Diamonds, Builds->Content. See "Input" for s60. |