releaseAutomation/mercurialComparison.pl
author teknolog
Thu, 27 May 2010 14:19:36 +0100
changeset 271 b20b2eae00e9
parent 143 e4ad889f1aa5
permissions -rw-r--r--
Added a very crude tool to check for existence of various tools in the system PATH.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
50
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     1
#!perl -w
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     2
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     3
# Copyright (c) 2009 Symbian Foundation Ltd
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     4
# This component and the accompanying materials are made available
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     5
# under the terms of the License "Eclipse Public License v1.0"
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     6
# which accompanies this distribution, and is available
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     7
# at the URL "http://www.eclipse.org/legal/epl-v10.html".
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     8
#
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     9
# Initial Contributors:
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    10
# Symbian Foundation Ltd - initial contribution.
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    11
# 
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    12
# Contributors:
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    13
#
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    14
# Description:
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    15
# Automates the creation of part of the PDK Release Notes: "Mercurial Comparison with PDK XXXXX"
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    16
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    17
use strict;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    18
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    19
use FindBin;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    20
$FindBin::Bin =~ s{/}{\\}g;;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    21
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    22
my $bomInfoFile = shift or die "First argument must be BOM file for build being built/released\n";
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    23
my $previousPdkLabel = shift or die  "Second argument must be hg label to compare against\n";
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    24
my $detailsTsvFilename = shift or die "Third argument must be filename to write detailed TSV data into\n";
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    25
defined shift and die "No more than three arguments please\n";
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    26
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    27
# Use external scripts to get the raw data and produce the CSV summary (to go into Excel, etc)
58
18edc8e9ec9e Added detection of packages that are new, or have changed from SFL -> EPL.
Simon Howkins <simonh@symbian.org>
parents: 57
diff changeset
    28
my @pkgErrors = `perl $FindBin::Bin\\..\\clone_packages\\clone_all_packages.pl -packagelist $bomInfoFile -exec -- hg status -A --rev $previousPdkLabel 2>&1 | perl $FindBin::Bin\\..\\williamr\\summarise_hg_status.pl 2>&1 > $detailsTsvFilename`;
50
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    29
58
18edc8e9ec9e Added detection of packages that are new, or have changed from SFL -> EPL.
Simon Howkins <simonh@symbian.org>
parents: 57
diff changeset
    30
# The redirection above means that we capture STDERR from summarise_hg_status,
50
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    31
# which lists packages for which it was unable to generate any data
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    32
# 
58
18edc8e9ec9e Added detection of packages that are new, or have changed from SFL -> EPL.
Simon Howkins <simonh@symbian.org>
parents: 57
diff changeset
    33
# It's captured because that happens either because it's a new package or has
50
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    34
# moved from SFL -> EPL or we've reverted to using the MCL instead of the FCL
58
18edc8e9ec9e Added detection of packages that are new, or have changed from SFL -> EPL.
Simon Howkins <simonh@symbian.org>
parents: 57
diff changeset
    35
# (in which case it's dealt with in another part of the release notes) or it
50
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    36
# just hasn't had any changes since the last release
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    37
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    38
# Input from TSV file
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    39
my @rawData;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    40
open my $fh, "<", $detailsTsvFilename;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    41
my @columns;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    42
foreach my $line (<$fh>)
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    43
{
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    44
	chomp $line;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    45
	my @values = split "\t", $line;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    46
	if (!@columns)
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    47
	{
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    48
		@columns = @values;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    49
	}
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    50
	else
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    51
	{
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    52
		my %lineData;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    53
		@lineData{@columns} = @values;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    54
		push @rawData, \%lineData;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    55
	}
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    56
}
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    57
close $fh;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    58
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    59
# Pivot
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    60
my %cookedData;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    61
foreach my $datum (@rawData)
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    62
{
56
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    63
	# Accumulate the total number of files in the old revision of the pkg
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    64
	$cookedData{$datum->{Package}}->{totalFiles} += $datum->{Count} unless $datum->{Change} eq "A";
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    65
	$cookedData{$datum->{Package}}->{same} += $datum->{Count} if $datum->{Change} eq "same";
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    66
	$cookedData{$datum->{Package}}->{addRemove} += $datum->{Count} if $datum->{Change} =~ m{^[AR]$};
50
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    67
}
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    68
# Cut-off for "interesting" packages
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    69
foreach my $package (keys %cookedData)
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    70
{
56
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    71
	# Ensure items are defined
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    72
	$cookedData{$package}->{totalFiles} |= 1;
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    73
	$cookedData{$package}->{same} |= 0;
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    74
	$cookedData{$package}->{addRemove} |= 0;
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    75
	$cookedData{$package}->{percentChurn} = 100 * (1 - ($cookedData{$package}->{same} / $cookedData{$package}->{totalFiles}));
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    76
	
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    77
	# More than N files added + removed
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    78
	next if $cookedData{$package}->{addRemove} >= 400;
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    79
	# More than M% churn
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    80
	next if $cookedData{$package}->{percentChurn} > 30;
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    81
	# Nothing interesting about this package
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 50
diff changeset
    82
	delete $cookedData{$package};
50
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    83
}
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    84
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    85
# Output
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    86
foreach my $package (sort keys %cookedData)
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    87
{
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    88
	print <<"EOT";
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    89
=== $package ===
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    90
83
bbe9783cf993 Updated wiki markup output so that lines don't run together (now a list).
Simon Howkins <simonh@symbian.org>
parents: 58
diff changeset
    91
* $cookedData{$package}->{addRemove} files added/removed
143
e4ad889f1aa5 Tweaked text used when reporting files churned, so it's not likely to confuse people who know of churn in the QWG sense.
Simon Howkins <simonh@symbian.org>
parents: 107
diff changeset
    92
* $cookedData{$package}->{percentChurn}% of files churned
107
b51d77004ea8 Removed attempts to identify new and relicensed packages, as this is already covered by another script and another part of the release notes.
Simon Howkins <simonh@symbian.org>
parents: 87
diff changeset
    93
83
bbe9783cf993 Updated wiki markup output so that lines don't run together (now a list).
Simon Howkins <simonh@symbian.org>
parents: 58
diff changeset
    94
# Cause1
bbe9783cf993 Updated wiki markup output so that lines don't run together (now a list).
Simon Howkins <simonh@symbian.org>
parents: 58
diff changeset
    95
# etc
50
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    96
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    97
EOT
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    98
}
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    99
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   100
if (!keys %cookedData)
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   101
{
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   102
	print "'''No packages were identified with large changes.'''";
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   103
}
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   104