clone_packages/clone_all_packages.pl
author dave.jones <dave.jones@accenture.com>
Wed, 10 Feb 2010 13:30:10 +0000
changeset 162 fb679efeb2dd
parent 116 671e371caeef
child 174 4c0108e0b6dd
permissions -rw-r--r--
Improve the parsing of the script location to determine the location of .txt files using File::Basename. Should be platform safe as $program_path will include trailing path separator.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
6
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     1
#! perl
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     2
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     3
# Copyright (c) 2009 Symbian Foundation Ltd
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     4
# This component and the accompanying materials are made available
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     5
# under the terms of the License "Eclipse Public License v1.0"
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     6
# which accompanies this distribution, and is available
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     7
# at the URL "http://www.eclipse.org/legal/epl-v10.html".
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     8
#
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     9
# Initial Contributors:
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    10
# Symbian Foundation Ltd - initial contribution.
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    11
# 
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    12
# Contributors:
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    13
#
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    14
# Description:
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    15
# Perl script to clone or update all of the Foundation MCL repositories
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    16
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    17
use strict;
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    18
use Getopt::Long;
162
fb679efeb2dd Improve the parsing of the script location to determine the location of .txt files using File::Basename.
dave.jones <dave.jones@accenture.com>
parents: 116
diff changeset
    19
use File::Basename;
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    20
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    21
sub Usage($)
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    22
  {
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    23
  my ($msg) = @_;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    24
  
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    25
  print "$msg\n\n" if ($msg ne "");
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    26
  
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    27
	print <<'EOF';
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    28
clone_all_repositories - simple script for cloning Symbian repository tree
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    29
	
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    30
This script will clone repositories, or pull changes into a previously
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    31
cloned repository. The script will prompt for your username and
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    32
password, which will be needed to access the SFL repositories, or you can
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    33
supply them with command line arguments.
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    34
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
    35
The list of packages can be supplied in a text file using the -packagelist
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
    36
option, which is capable of reading the build-info.xml files supplied with 
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
    37
Symbian PDKs. Supplying a build-info.xml file will cause the clone or update
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
    38
operation to use the exact revision for each of the relevant repositories.
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
    39
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    40
Important: 
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    41
  This script uses https access to the repositories, so the username and
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    42
  password will be stored as cleartext in the .hg/hgrc file for each repository.
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    43
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    44
Used with the "-mirror" option, the script will copy both MCL and FCL
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    45
repositories into the same directory layout as the Symbian website, and will
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    46
use the Mercurial "--noupdate" option when cloning.
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    47
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    48
Options:
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    49
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    50
-username      username at the Symbian website
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    51
-password      password to go with username
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    52
-mirror        create a "mirror" of the Symbian repository tree
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
    53
-packagelist   file containing the URLs for the packages to be processed
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    54
-retries       number of times to retry a failed operation (default 1)
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    55
-verbose       print the underlying "hg" commands before executing them
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    56
-n             do nothing - don't actually execute the commands
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    57
-help          print this help information
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    58
-exec          execute command on each repository
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    59
-filter <RE>   only process repository paths matching regular expression <RE>
51
88f86cd8d99b clone_all_packages.pl - Enabled 'dummyrun' to be set from the command line using '-d' (or -'dummyrun') to help with trying out --exec style commands.
MattD <mattd@symbian.org>
parents: 48
diff changeset
    60
-dummyrun      Dummy Run, don't execute any Mercurial commands.
97
bf4111f05106 clone_all_packages.pl - Added optional '-webhost' parameter.
MattD <mattd@symbian.org>
parents: 95
diff changeset
    61
-webhost       Web Mercurial host (defaults to developer.symbian.org)
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    62
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    63
The -exec option processes the rest of the command line, treating it as
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    64
a command to apply to each repository in turn. Some keywords are expanded
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    65
to repository-specific values, and "hg" is always expanded to "hg -R %REPO%"
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    66
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    67
%REPO%         relative path to the repository
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    68
%WREPO%        relative path to repository, with Windows path separators
44
ddf62a543fb6 Added %HREPO% and %WHREPO%, to assist in cloning from network drives
William Roberts <williamr@symbian.org>
parents: 41
diff changeset
    69
%HREPO%        path to the repository on the server
ddf62a543fb6 Added %HREPO% and %WHREPO%, to assist in cloning from network drives
William Roberts <williamr@symbian.org>
parents: 41
diff changeset
    70
%WHREPO%       path to the repository on the server, with Windows separators
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    71
%URL%          URL of the master repository
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    72
%PUSHURL%      URL suitable for pushing (always includes username & password)
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
    73
%REV%          revision associated with the repository (defaults to "tip")
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    74
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    75
It's often useful to use "--" to separate the exec command from the options
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    76
to this script, e.g. "-exec -- hg update -C tip"
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    77
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    78
EOF
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    79
  exit (1);  
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    80
  }
6
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    81
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    82
my @clone_options = (); # use ("--noupdate") to clone without extracting the source
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
    83
my @pull_options  = (); # use ("--rebase") to rebase your changes when pulling
8
2184cc44590a Clone from developer.symbian.org (Bug 79) and use https for SFL packages
William Roberts <williamr@symbian.org>
parents: 6
diff changeset
    84
my $hostname = "developer.symbian.org";
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    85
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    86
my $username = "";
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    87
my $password = "";
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
    88
my $mirror = 0; # set to 1 if you want to mirror the repository structure
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
    89
my $retries = 1;  # number of times to retry problem repos
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    90
my $verbose = 0;  # turn on more tracing
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    91
my $do_nothing = 0; # print the hg commands, don't actually do them
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    92
my $help = 0;
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    93
my $exec = 0;
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
    94
my $filter = "";
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
    95
my @packagelist_files = ();
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    96
39
759027e1c6b3 Improved robustness of package list location searching.
thomase
parents: 38
diff changeset
    97
# Analyse the rest of command-line parameters
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    98
if (!GetOptions(
13
dda0176e838b Fix mistake in -password and -username options, tweak the (doomed) sfl repository screen scraping
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    99
    "u|username=s" => \$username,
dda0176e838b Fix mistake in -password and -username options, tweak the (doomed) sfl repository screen scraping
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   100
    "p|password=s" => \$password,
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   101
    "m|mirror" => \$mirror, 
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   102
    "r|retries=i" => \$retries,
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   103
    "v|verbose" => \$verbose,
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   104
    "n" => \$do_nothing,
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   105
    "h|help" => \$help,
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   106
    "e|exec" => \$exec,
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   107
    "f|filter=s" => \$filter,
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   108
    "l|packagelist=s" => \@packagelist_files,
51
88f86cd8d99b clone_all_packages.pl - Enabled 'dummyrun' to be set from the command line using '-d' (or -'dummyrun') to help with trying out --exec style commands.
MattD <mattd@symbian.org>
parents: 48
diff changeset
   109
    "d|dummyrun" => \$do_nothing,
97
bf4111f05106 clone_all_packages.pl - Added optional '-webhost' parameter.
MattD <mattd@symbian.org>
parents: 95
diff changeset
   110
    "w|webhost=s" => \$hostname,
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   111
    ))
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   112
  {
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   113
  Usage("Invalid argument");
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   114
  }
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   115
  
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   116
Usage("Too many arguments") if (scalar @ARGV > 0 && !$exec);
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   117
Usage("Too few arguments for -exec") if (scalar @ARGV == 0 && $exec);
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   118
Usage("") if ($help);
6
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   119
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   120
# Important: This script uses http access to the repositories, so
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   121
# the username and password will be stored as cleartext in the
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   122
# .hg/hgrc file in each repository.
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   123
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   124
my $needs_id = 1; # assumed necessary for clone/pull
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   125
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   126
my @exec_cmd = @ARGV;
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   127
if ($exec)
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   128
  {
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   129
  if ($exec_cmd[0] eq "hg")
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   130
    {
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   131
    shift @exec_cmd;
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   132
    unshift @exec_cmd, "hg", "-R", "%REPO%";
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   133
    }
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   134
  if ($verbose)
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   135
    {
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   136
    print "* Exec template = >", join("<,>", @exec_cmd), "<\n";
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   137
    }
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   138
  $needs_id = grep /URL%/,@exec_cmd; # only need id if using %URL% or %PUSHURL%
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   139
  }
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   140
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   141
if ($needs_id && $username eq "" )
6
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   142
  {
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   143
  print "Username: ";
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   144
  $username = <STDIN>;
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   145
  chomp $username;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   146
  }
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   147
if ($needs_id && $password eq "" )
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   148
  {
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   149
  print "Password: ";
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   150
  $password = <STDIN>;
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   151
  chomp $password;
6
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   152
  }
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   153
31
cf4bd27a8280 Add special case handling for export controlled OSS repositories (currently sf/os/security)
William Roberts <williamr@symbian.org>
parents: 30
diff changeset
   154
my %export_control_special_case = (
cf4bd27a8280 Add special case handling for export controlled OSS repositories (currently sf/os/security)
William Roberts <williamr@symbian.org>
parents: 30
diff changeset
   155
  "oss/MCL/sf/os/security" => 1,
cf4bd27a8280 Add special case handling for export controlled OSS repositories (currently sf/os/security)
William Roberts <williamr@symbian.org>
parents: 30
diff changeset
   156
  "oss/FCL/sf/os/security" => 1,
cf4bd27a8280 Add special case handling for export controlled OSS repositories (currently sf/os/security)
William Roberts <williamr@symbian.org>
parents: 30
diff changeset
   157
  );
cf4bd27a8280 Add special case handling for export controlled OSS repositories (currently sf/os/security)
William Roberts <williamr@symbian.org>
parents: 30
diff changeset
   158
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   159
sub do_system(@)
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   160
  {
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   161
  my (@cmd) = @_;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   162
  
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   163
  if ($verbose)
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   164
    {
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   165
    print "* ", join(" ", @cmd), "\n";
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   166
    }
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   167
  return 0 if ($do_nothing);
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   168
  
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   169
  return system(@cmd);
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   170
  }
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   171
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   172
my %revisions;
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   173
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   174
sub process_one_repo($)
6
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   175
  {
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   176
  my ($package) = @_;
6
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   177
  my @dirs = split /\//, $package;
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   178
  my $license = shift @dirs;
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   179
  my $repotree = shift @dirs; # remove the MCL or FCL repo tree information
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   180
  my $destdir = pop @dirs;  # ignore the package name, because Mercurial will create that
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   181
  
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   182
  if ($mirror)
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   183
    {
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   184
    # Mirror the full directory structure, so put back the license & repotree dirs
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   185
    unshift @dirs, $repotree;
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   186
    unshift @dirs, $license;
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   187
    }
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   188
6
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   189
  # Ensure the directories already exist as far as the parent of the repository
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   190
  my $path = "";
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   191
  foreach my $dir (@dirs)
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   192
    {
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   193
    $path = ($path eq "") ? $dir : "$path/$dir";
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   194
    if (!-d $path)
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   195
      {
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   196
      mkdir $path;
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   197
      }
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   198
    }
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   199
  
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   200
  $path .= "/$destdir";   # this is where the repository will go
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   201
8
2184cc44590a Clone from developer.symbian.org (Bug 79) and use https for SFL packages
William Roberts <williamr@symbian.org>
parents: 6
diff changeset
   202
  my $repo_url = "https://$username:$password\@$hostname/$package/";
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   203
  my $repo_push_url =$repo_url;
31
cf4bd27a8280 Add special case handling for export controlled OSS repositories (currently sf/os/security)
William Roberts <williamr@symbian.org>
parents: 30
diff changeset
   204
  if ($license ne "sfl" && !$export_control_special_case{$package})
6
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   205
    {
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   206
    # user registration is not required for reading public package repositories
97
bf4111f05106 clone_all_packages.pl - Added optional '-webhost' parameter.
MattD <mattd@symbian.org>
parents: 95
diff changeset
   207
    $repo_url = "http://$hostname/$package/";
6
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   208
    }
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   209
  
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   210
  my @rev_options = ();
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   211
  my $revision = $revisions{$package};
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   212
  if (defined($revision))
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   213
    {
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   214
    @rev_options = ("--rev", $revision);
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   215
    }
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   216
  else
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   217
    {
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   218
    $revision = "tip";
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   219
    # and leave the rev_options list empty
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   220
    }
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   221
  
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   222
  my $ret;
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   223
  if ($exec)
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   224
    {
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   225
    # iteration functionality - process the keywords
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   226
    my $wpath = $path;
44
ddf62a543fb6 Added %HREPO% and %WHREPO%, to assist in cloning from network drives
William Roberts <williamr@symbian.org>
parents: 41
diff changeset
   227
    my $wpackage = $package;
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   228
    $wpath =~ s/\//\\/g;  # win32 path separator
44
ddf62a543fb6 Added %HREPO% and %WHREPO%, to assist in cloning from network drives
William Roberts <williamr@symbian.org>
parents: 41
diff changeset
   229
    $wpackage =~ s/\//\\/g;  # win32 path separator
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   230
    my @repo_cmd = ();
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   231
    foreach my $origcmd (@exec_cmd)
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   232
      {
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   233
      my $cmd = $origcmd; # avoid altering the original
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   234
      $cmd =~ s/%REPO%/$path/;
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   235
      $cmd =~ s/%WREPO%/$wpath/;
44
ddf62a543fb6 Added %HREPO% and %WHREPO%, to assist in cloning from network drives
William Roberts <williamr@symbian.org>
parents: 41
diff changeset
   236
      $cmd =~ s/%HREPO%/$package/;
ddf62a543fb6 Added %HREPO% and %WHREPO%, to assist in cloning from network drives
William Roberts <williamr@symbian.org>
parents: 41
diff changeset
   237
      $cmd =~ s/%WHREPO%/$wpackage/;
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   238
      $cmd =~ s/%URL%/$repo_url/;
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   239
      $cmd =~ s/%PUSHURL%/$repo_push_url/;
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   240
      $cmd =~ s/%REV%/$revision/;
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   241
      push @repo_cmd, $cmd;
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   242
      }
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   243
    print "Processing $path...\n";
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   244
    $ret = do_system(@repo_cmd);
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   245
    }
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   246
  elsif (-d "$path/.hg")
6
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   247
    {
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   248
    # The repository already exists, so just do an update
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   249
    
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   250
    print "Updating $destdir from $package...\n";
95
8b3aad972b66 Fix Bug 489 by moving the "--rev xxx" options to the correct place in the command line
William Roberts <williamr@symbian.org>
parents: 63
diff changeset
   251
    $ret = do_system("hg", "pull", @pull_options, @rev_options, "-R", $path, $repo_url);
41
0c8b7325c98e Add extra login to do "hg update" after "hg pull" if not mirroring - to match clone_BOM.pl
William Roberts <williamr@symbian.org>
parents: 40
diff changeset
   252
    if ($ret == 0 && ! $mirror)
0c8b7325c98e Add extra login to do "hg update" after "hg pull" if not mirroring - to match clone_BOM.pl
William Roberts <williamr@symbian.org>
parents: 40
diff changeset
   253
      {
0c8b7325c98e Add extra login to do "hg update" after "hg pull" if not mirroring - to match clone_BOM.pl
William Roberts <williamr@symbian.org>
parents: 40
diff changeset
   254
      $ret = do_system("hg", "update", "-R", $path, @rev_options)
0c8b7325c98e Add extra login to do "hg update" after "hg pull" if not mirroring - to match clone_BOM.pl
William Roberts <williamr@symbian.org>
parents: 40
diff changeset
   255
      }
6
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   256
    }
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   257
  else
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   258
    {
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   259
    # Clone the repository
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   260
    
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   261
    print "Cloning $destdir from $package...\n";
95
8b3aad972b66 Fix Bug 489 by moving the "--rev xxx" options to the correct place in the command line
William Roberts <williamr@symbian.org>
parents: 63
diff changeset
   262
    $ret = do_system("hg", "clone", @clone_options, @rev_options, $repo_url, $path);
6
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   263
    }
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   264
  
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   265
  $ret = $ret >> 8;   # extract the exit status
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   266
  print "* Exit status $ret for $path\n\n" if ($verbose);
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   267
  return $ret;
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   268
  }
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   269
46
00e1ecc15345 Change -mirror so that explicit package lists don't get the "implied FCLs" added.
William Roberts <williamr@symbian.org>
parents: 44
diff changeset
   270
my $add_implied_FCL_repos = 0; 
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   271
if (scalar @packagelist_files == 0)
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   272
  {
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   273
  # Read the package list files alongside the script itself
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   274
  
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   275
  # Extract the path location of the program and locate package list files
162
fb679efeb2dd Improve the parsing of the script location to determine the location of .txt files using File::Basename.
dave.jones <dave.jones@accenture.com>
parents: 116
diff changeset
   276
  my ($program_name,$program_path) = &File::Basename::fileparse($0);
fb679efeb2dd Improve the parsing of the script location to determine the location of .txt files using File::Basename.
dave.jones <dave.jones@accenture.com>
parents: 116
diff changeset
   277
  
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   278
  foreach my $file ("sf_mcl_packages.txt", "sftools_mcl_packages.txt", "other_packages.txt")
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   279
    {
162
fb679efeb2dd Improve the parsing of the script location to determine the location of .txt files using File::Basename.
dave.jones <dave.jones@accenture.com>
parents: 116
diff changeset
   280
    if (! -e $program_path.$file)
fb679efeb2dd Improve the parsing of the script location to determine the location of .txt files using File::Basename.
dave.jones <dave.jones@accenture.com>
parents: 116
diff changeset
   281
    	{
fb679efeb2dd Improve the parsing of the script location to determine the location of .txt files using File::Basename.
dave.jones <dave.jones@accenture.com>
parents: 116
diff changeset
   282
    	print "Cannot find implied packagelist $program_path$file\n";
fb679efeb2dd Improve the parsing of the script location to determine the location of .txt files using File::Basename.
dave.jones <dave.jones@accenture.com>
parents: 116
diff changeset
   283
    	next;
fb679efeb2dd Improve the parsing of the script location to determine the location of .txt files using File::Basename.
dave.jones <dave.jones@accenture.com>
parents: 116
diff changeset
   284
			}
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   285
    push @packagelist_files, $program_path.$file;
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   286
    }
46
00e1ecc15345 Change -mirror so that explicit package lists don't get the "implied FCLs" added.
William Roberts <williamr@symbian.org>
parents: 44
diff changeset
   287
  $add_implied_FCL_repos = 1;   # lists only contain the MCL repo locations
6
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   288
  }
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   289
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   290
my @all_packages = ();
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   291
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   292
foreach my $file (@packagelist_files)
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   293
  {
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   294
  print "* reading package information from $file...\n" if ($verbose);
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   295
  open PKG_LIST, "<$file" or die "Can't open $file: $!\n";
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   296
  foreach my $line (<PKG_LIST>)
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   297
    {
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   298
    chomp($line);
63
9e7536b80766 removed duplicate lines due to faulty merge
tahirm@symbian.org
parents: 60
diff changeset
   299
60
e86c659b78a0 Fix Bug 387 - full newline processing as per "perlport" guidance
William Roberts <williamr@symbian.org>
parents: 51
diff changeset
   300
    $line =~ s/\015//g; # remove CR, in case we are processing Windows text files on Linux
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   301
    
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   302
    my $revision; # set when processing build-info listings
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   303
    
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   304
    # build-info.xml format
48
8e73266ba54f Cope with PDK 2.0.c variation on build-info.xml format
William Roberts <williamr@symbian.org>
parents: 47
diff changeset
   305
    # <baseline>//v800008/Builds01/mercurial_master_prod/sfl/MCL/sf/adaptation/stubs/#7:e086c7f635d5</baseline>
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   306
    # <baseline>//v800008/Builds01/mercurial_master_prod/sfl/MCL/sf/adaptation/stubs/#:e086c7f635d5</baseline>
48
8e73266ba54f Cope with PDK 2.0.c variation on build-info.xml format
William Roberts <williamr@symbian.org>
parents: 47
diff changeset
   307
    # <baseline>//v800008/Builds01/mercurial_master_prod/sfl/MCL/sf/adaptation/stubs/#e086c7f635d5</baseline>
8e73266ba54f Cope with PDK 2.0.c variation on build-info.xml format
William Roberts <williamr@symbian.org>
parents: 47
diff changeset
   308
    if ($line =~ /<baseline>(.*)#(\d*:)?([0-9a-fA-F]+)<\/baseline>/i)
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   309
      {
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   310
      $line = $1;   # discard the wrapping
48
8e73266ba54f Cope with PDK 2.0.c variation on build-info.xml format
William Roberts <williamr@symbian.org>
parents: 47
diff changeset
   311
      $revision = $3;
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   312
      }
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   313
 
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   314
    # Look for the oss/MCL/ prefix to a path e.g.
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   315
    # https://developer.symbian.org/oss/FCL/interim/contrib/WidgetExamples
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   316
    if ($line =~ /((oss|sfl)\/(FCL|MCL)\/.*)\s*$/)
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   317
      {
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   318
      my $repo_path = $1;
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   319
      $repo_path =~ s/\/$//;  # remove trailing slash, if any
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   320
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   321
      push @all_packages, $repo_path;
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   322
      $revisions{$repo_path} = $revision if (defined $revision);
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   323
      next;
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   324
      }
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   325
    }
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   326
  close PKG_LIST;
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   327
  }
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   328
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   329
if ($mirror)
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   330
  {
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   331
  push @clone_options, "--noupdate";
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   332
  
46
00e1ecc15345 Change -mirror so that explicit package lists don't get the "implied FCLs" added.
William Roberts <williamr@symbian.org>
parents: 44
diff changeset
   333
  if ($add_implied_FCL_repos)
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   334
    {
46
00e1ecc15345 Change -mirror so that explicit package lists don't get the "implied FCLs" added.
William Roberts <williamr@symbian.org>
parents: 44
diff changeset
   335
    # Assume that every MCL has a matching FCL. As we are mirroring,
00e1ecc15345 Change -mirror so that explicit package lists don't get the "implied FCLs" added.
William Roberts <williamr@symbian.org>
parents: 44
diff changeset
   336
    # we can process both without them overlapping in the local filesystem
00e1ecc15345 Change -mirror so that explicit package lists don't get the "implied FCLs" added.
William Roberts <williamr@symbian.org>
parents: 44
diff changeset
   337
    my @list_with_fcls = ();
00e1ecc15345 Change -mirror so that explicit package lists don't get the "implied FCLs" added.
William Roberts <williamr@symbian.org>
parents: 44
diff changeset
   338
    foreach my $package (@all_packages)
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   339
      {
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   340
      push @list_with_fcls, $package;
46
00e1ecc15345 Change -mirror so that explicit package lists don't get the "implied FCLs" added.
William Roberts <williamr@symbian.org>
parents: 44
diff changeset
   341
      if ($package =~ /MCL/)
00e1ecc15345 Change -mirror so that explicit package lists don't get the "implied FCLs" added.
William Roberts <williamr@symbian.org>
parents: 44
diff changeset
   342
        {
00e1ecc15345 Change -mirror so that explicit package lists don't get the "implied FCLs" added.
William Roberts <williamr@symbian.org>
parents: 44
diff changeset
   343
        $package =~ s/MCL/FCL/;
00e1ecc15345 Change -mirror so that explicit package lists don't get the "implied FCLs" added.
William Roberts <williamr@symbian.org>
parents: 44
diff changeset
   344
        push @list_with_fcls, $package;
00e1ecc15345 Change -mirror so that explicit package lists don't get the "implied FCLs" added.
William Roberts <williamr@symbian.org>
parents: 44
diff changeset
   345
        }
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   346
      }
46
00e1ecc15345 Change -mirror so that explicit package lists don't get the "implied FCLs" added.
William Roberts <williamr@symbian.org>
parents: 44
diff changeset
   347
    @all_packages = @list_with_fcls;
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   348
    }
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   349
  }
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   350
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   351
my @problem_packages = ();
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   352
my $total_packages = 0;
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   353
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   354
foreach my $package (@all_packages)
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   355
  {
16
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   356
  if ($filter && $package !~ /$filter/)
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   357
    {
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   358
    next; # skip repos which don't match the filter
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 13
diff changeset
   359
    }
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   360
  my $err = process_one_repo($package);
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   361
  $total_packages++;
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   362
  push @problem_packages, $package if ($err < 0 || $err > 127); 
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   363
  }
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   364
  
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   365
# retry problem packages
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   366
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   367
my $attempt = 0;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   368
while ($attempt < $retries && scalar @problem_packages) 
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   369
  {
12
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   370
  $attempt++;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   371
  printf "\n\n------------\nRetry attempt %d on %d packages\n",
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   372
    $attempt, scalar @problem_packages;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   373
  print join("\n", @problem_packages, ""), "\n";
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   374
    
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   375
  my @list = @problem_packages;
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   376
  @problem_packages = ();
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   377
  foreach my $package (@list)
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   378
    {
40
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   379
    my $err = process_one_repo($package);
674034f6180c Add -packagelist option, which also supports build-info.xml to provide the equivalent of the clone_BOM.pl script
William Roberts <williamr@symbian.org>
parents: 39
diff changeset
   380
    push @problem_packages, $package if ($err < 0 || $err > 127); 
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   381
   }
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   382
  }
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   383
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   384
printf "\n------------\nProcessed %d packages, of which %d reported errors\n", 
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   385
  $total_packages, scalar @problem_packages;
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   386
if (scalar @problem_packages)
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   387
  {
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   388
  print join("\n", @problem_packages, "");
116
671e371caeef add exit code for clone_all_packages.pl to make easier to integrate to automated jobs
andy simpson <andrews@symbian.org>
parents: 97
diff changeset
   389
  exit(1);
11
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 8
diff changeset
   390
  }
116
671e371caeef add exit code for clone_all_packages.pl to make easier to integrate to automated jobs
andy simpson <andrews@symbian.org>
parents: 97
diff changeset
   391
  else
671e371caeef add exit code for clone_all_packages.pl to make easier to integrate to automated jobs
andy simpson <andrews@symbian.org>
parents: 97
diff changeset
   392
  {
671e371caeef add exit code for clone_all_packages.pl to make easier to integrate to automated jobs
andy simpson <andrews@symbian.org>
parents: 97
diff changeset
   393
  exit(0);
671e371caeef add exit code for clone_all_packages.pl to make easier to integrate to automated jobs
andy simpson <andrews@symbian.org>
parents: 97
diff changeset
   394
  }
671e371caeef add exit code for clone_all_packages.pl to make easier to integrate to automated jobs
andy simpson <andrews@symbian.org>
parents: 97
diff changeset
   395