sbsv2/raptor/bin/recipestats.py
author timothy.murphy@nokia.com
Fri, 30 Apr 2010 16:07:17 +0100
branchfix
changeset 511 7581d432643a
parent 488 bae97f326378
child 528 f6aa47076e0f
permissions -rwxr-xr-x
fix: support new trace compiler features for preventing clashes. Automatically turn on OST_TRACE_COMPILER_IN_USE macro. Look for trace header in systemincludes. Make directories in makefile parse to prevent clashes during build. Correct path for autogen headers. Correct case issue with autogen headers on Linux.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
88
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
     1
#!/usr/bin/env python
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
     2
#
488
timothy.murphy@nokia.com
parents: 485
diff changeset
     3
# Copyright (c) 2007-2010 Nokia Corporation and/or its subsidiary(-ies).
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
     4
# All rights reserved.
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
     5
# This component and the accompanying materials are made available
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
     6
# under the terms of the License "Eclipse Public License v1.0"
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
     7
# which accompanies this distribution, and is available
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
     8
# at the URL "http://www.eclipse.org/legal/epl-v10.html".
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
     9
#
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    10
# Initial Contributors:
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    11
# Nokia Corporation - initial contribution.
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    12
#
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    13
# Contributors:
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    14
#
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    15
# Description: 
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    16
# 
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    17
# display summary information about recipes from raptor logs
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    18
# e.g. total times and so on.
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    19
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    20
import time
485
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    21
import __future__
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    22
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    23
class RecipeStats(object):
485
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    24
	def __init__(self, name, count, time):
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    25
		self.name=name
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    26
		self.count=count
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    27
		self.time=time
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    28
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    29
	def add(self, duration):
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    30
		self.time += duration
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    31
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    32
class BuildStats(object):
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    33
	STAT_OK = 0
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    34
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    35
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    36
	def __init__(self):
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    37
		self.stats = {}
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    38
		self.failcount = 0
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    39
		self.failtime = 0.0
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    40
		self.failtypes = {}
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    41
		self.retryfails = 0
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    42
		
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    43
	def add(self, starttime, duration, name, status):
485
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    44
		if status != BuildStats.STAT_OK:
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    45
			self.failcount += 1
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    46
			if name in self.failtypes:
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    47
				self.failtypes[name] += 1
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    48
			else:
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    49
				self.failtypes[name] = 1
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    50
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    51
			if status == 128:
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    52
				self.retryfails += 1
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    53
			return
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    54
			
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    55
		if name in self.stats:
485
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    56
			r = self.stats[name]
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    57
			r.add(duration)
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    58
		else:
485
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    59
			self.stats[name] = RecipeStats(name,1,duration)
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    60
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    61
	def recipe_csv(self):
485
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    62
		s = '"name", "time", "count"\n'
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    63
		l = sorted(self.stats.values(), key= lambda r: r.time, reverse=True)
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    64
		for r in l:
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    65
			s += '"%s",%s,%d\n' % (r.name, str(r.time), r.count)
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    66
		return s
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    67
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    68
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    69
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    70
import sys
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    71
import re
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    72
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    73
def main():
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    74
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    75
	f = sys.stdin
485
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
    76
	st = BuildStats()
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    77
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    78
	recipe_re = re.compile(".*<recipe name='([^']+)'.*")
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    79
	time_re = re.compile(".*<time start='([0-9]+\.[0-9]+)' *elapsed='([0-9]+\.[0-9]+)'.*")
88
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
    80
	status_re = re.compile(".*<status exit='(?P<exit>(ok|failed))'( *code='(?P<code>[0-9]+)')?.*")
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    81
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    82
	alternating = 0
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    83
	start_time = 0.0
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    84
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    85
	
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    86
	for l in f.xreadlines():
88
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
    87
		l2 = l.rstrip("\n\r")
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    88
		rm = recipe_re.match(l2)
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    89
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    90
		if rm is not None:
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    91
			rname = rm.groups()[0]
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    92
			continue
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    93
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    94
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    95
		tm = time_re.match(l2)
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
    96
		if tm is not None:
88
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
    97
			try:
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
    98
				s = float(tm.groups()[0])
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
    99
				elapsed = float(tm.groups()[1])
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
   100
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
   101
				if start_time == 0.0:
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
   102
					start_time = s
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   103
88
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
   104
				s -= start_time
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   105
88
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
   106
				continue
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
   107
			except ValueError, e:
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
   108
				raise Exception("Parse problem: float conversion on these groups: %s\n%s" %(str(tm.groups()), str(e)))
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
   109
		else:
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
   110
			if l2.find("<time") is not -1:
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
   111
				raise Exception("unparsed timing status: %s\n"%l2)
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   112
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   113
		sm = status_re.match(l2)
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   114
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   115
		if sm is None:
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   116
			continue
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   117
88
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
   118
		if sm.groupdict()['exit'] == 'ok':
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   119
			status = 0
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   120
		else:
88
b5820d0f3a1c Fix errors with parsing exit codes in status tags
timothy.murphy@nokia.com
parents: 87
diff changeset
   121
			status = int(sm.groupdict()['code'])
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   122
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   123
		st.add(s, elapsed, rname, status)
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   124
485
926e968477c6 fix: recipestats.py produces "correct" header line, also sorts by time to show recipe using the largest time first.
timothy.murphy@nokia.com
parents: 100
diff changeset
   125
	print(st.recipe_csv())
87
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   126
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   127
01cb4707f979 Added a script for finding out how time each class of recipes took in the build
timothy.murphy@nokia.com
parents:
diff changeset
   128
if __name__ == '__main__': main()