releaseAutomation/mercurialComparison.pl
author Simon Howkins <simonh@symbian.org>
Thu, 17 Sep 2009 14:05:23 +0100
changeset 55 67168a24b405
parent 49 a5ee079f00dd
child 56 9b6317ec9e32
permissions -rw-r--r--
Made test for interesting packages consider % files churned. >30% is considered interesting. Adjusted the number of files added/removed that make a package interesting to offset the items added by the churn check.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
49
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     1
#!perl -w
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     2
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     3
# Copyright (c) 2009 Symbian Foundation Ltd
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     4
# This component and the accompanying materials are made available
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     5
# under the terms of the License "Eclipse Public License v1.0"
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     6
# which accompanies this distribution, and is available
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     7
# at the URL "http://www.eclipse.org/legal/epl-v10.html".
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     8
#
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     9
# Initial Contributors:
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    10
# Symbian Foundation Ltd - initial contribution.
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    11
# 
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    12
# Contributors:
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    13
#
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    14
# Description:
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    15
# Automates the creation of part of the PDK Release Notes: "Mercurial Comparison with PDK XXXXX"
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    16
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    17
use strict;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    18
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    19
use FindBin;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    20
$FindBin::Bin =~ s{/}{\\}g;;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    21
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    22
my $bomInfoFile = shift or die "First argument must be BOM file for build being built/released\n";
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    23
my $previousPdkLabel = shift or die  "Second argument must be hg label to compare against\n";
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    24
my $detailsTsvFilename = shift or die "Third argument must be filename to write detailed TSV data into\n";
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    25
defined shift and die "No more than three arguments please\n";
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    26
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    27
# Use external scripts to get the raw data and produce the CSV summary (to go into Excel, etc)
55
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    28
#system("perl $FindBin::Bin\\..\\clone_packages\\clone_all_packages.pl -packagelist $bomInfoFile -exec -- hg status -A --rev $previousPdkLabel 2>&1 | perl $FindBin::Bin\\..\\williamr\\summarise_hg_status.pl 2> nul: > $detailsTsvFilename");
49
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    29
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    30
# The redirection above means that we discard STDERR from summarise_hg_status,
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    31
# which lists packages for which it was unable to generate any data
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    32
# 
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    33
# It's discarded because that happens either because it's a new package or has
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    34
# moved from SFL -> EPL or we've reverted to using the MCL instead of the FCL
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    35
# (in which cases it's dealt with in another part of the release notes) or it
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    36
# just hasn't had any changes since the last release
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    37
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    38
# Input from TSV file
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    39
my @rawData;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    40
open my $fh, "<", $detailsTsvFilename;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    41
my @columns;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    42
foreach my $line (<$fh>)
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    43
{
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    44
	chomp $line;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    45
	my @values = split "\t", $line;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    46
	if (!@columns)
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    47
	{
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    48
		@columns = @values;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    49
	}
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    50
	else
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    51
	{
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    52
		my %lineData;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    53
		@lineData{@columns} = @values;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    54
		push @rawData, \%lineData;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    55
	}
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    56
}
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    57
close $fh;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    58
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    59
# Pivot
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    60
my %cookedData;
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    61
foreach my $datum (@rawData)
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    62
{
55
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    63
	# Accumulate the total number of files in the old revision of the pkg
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    64
	$cookedData{$datum->{Package}}->{totalFiles} += $datum->{Count} unless $datum->{Change} eq "A";
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    65
	$cookedData{$datum->{Package}}->{same} += $datum->{Count} if $datum->{Change} eq "same";
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    66
	$cookedData{$datum->{Package}}->{addRemove} += $datum->{Count} if $datum->{Change} =~ m{^[AR]$};
49
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    67
}
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    68
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    69
# Cut-off for "interesting" packages
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    70
foreach my $package (keys %cookedData)
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    71
{
55
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    72
	# Ensure items are defined
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    73
	$cookedData{$package}->{totalFiles} |= 1;
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    74
	$cookedData{$package}->{same} |= 0;
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    75
	$cookedData{$package}->{addRemove} |= 0;
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    76
	$cookedData{$package}->{percentChurn} = 100 * (1 - ($cookedData{$package}->{same} / $cookedData{$package}->{totalFiles}));
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    77
	
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    78
	# More than N files added + removed
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    79
	next if $cookedData{$package}->{addRemove} >= 400;
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    80
	# More than M% churn
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    81
	next if $cookedData{$package}->{percentChurn} > 30;
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    82
	# Nothing interesting about this package
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
    83
	delete $cookedData{$package};
49
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    84
}
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    85
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    86
# Output
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    87
print <<"EOT";
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    88
== Mercurial Comparison with $previousPdkLabel ==
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    89
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    90
The Mercurial changes from Nokia were delivered as a bulk update based on '''XXXXXXXXXXXXXXXXXXXXXX'''.
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    91
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    92
List of the Mercurial changes (files added/removed/modified) between $previousPdkLabel and PDK '''XXXXX''' - [[Media:XXXX.txt]].
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    93
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    94
A short study of the results which concentrated on the added and removed files has identified these significant package changes: 
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    95
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    96
EOT
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    97
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    98
foreach my $package (sort keys %cookedData)
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    99
{
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   100
	print <<"EOT";
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   101
=== $package ===
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   102
55
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
   103
$cookedData{$package}->{addRemove} files added/removed
67168a24b405 Made test for interesting packages consider % files churned. >30% is
Simon Howkins <simonh@symbian.org>
parents: 49
diff changeset
   104
$cookedData{$package}->{percentChurn}% churn
49
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   105
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   106
* Cause1
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   107
* etc
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   108
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   109
EOT
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   110
}
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   111
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   112
if (!keys %cookedData)
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   113
{
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   114
	print "'''No packages were identified with large changes.'''";
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   115
}
a5ee079f00dd Tool to help automate the "Mercurial Comparison" part of the Release Notes
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
   116