clone_all_packages.pl
author William Roberts <williamr@symbian.org>
Mon, 10 Aug 2009 14:28:06 +0100
changeset 28 7896313fcd8f
parent 16 35c6c3f66bc4
child 29 09f253c20209
permissions -rw-r--r--
Added new repositories sfl/MCL/sf/mw/helix and oss/FCL/interim/QEMU
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
5
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     1
#! perl
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     2
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     3
# Copyright (c) 2009 Symbian Foundation Ltd
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     4
# This component and the accompanying materials are made available
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     5
# under the terms of the License "Eclipse Public License v1.0"
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     6
# which accompanies this distribution, and is available
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     7
# at the URL "http://www.eclipse.org/legal/epl-v10.html".
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     8
#
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
     9
# Initial Contributors:
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    10
# Symbian Foundation Ltd - initial contribution.
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    11
# 
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    12
# Contributors:
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    13
#
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    14
# Description:
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    15
# Perl script to clone or update all of the Foundation MCL repositories
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    16
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    17
use strict;
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    18
use Getopt::Long;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    19
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    20
sub Usage($)
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    21
  {
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    22
  my ($msg) = @_;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    23
  
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    24
  print "$msg\n\n" if ($msg ne "");
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    25
  
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    26
	print <<'EOF';
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    27
clone_all_repositories - simple script for cloning Symbian repository tree
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    28
	
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    29
This script will clone repositories, or pull changes into a previously
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    30
cloned repository. The script will prompt for your username and
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    31
password, which will be needed to access the SFL repositories, or you can
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    32
supply them with command line arguments.
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    33
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    34
Important: 
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    35
  This script uses https access to the repositories, so the username and
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    36
  password will be stored as cleartext in the .hg/hgrc file for each repository.
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    37
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    38
Used with the "-mirror" option, the script will copy both MCL and FCL
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    39
repositories into the same directory layout as the Symbian website, and will
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    40
use the Mercurial "--noupdate" option when cloning.
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    41
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    42
Options:
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    43
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    44
-username      username at the Symbian website
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    45
-password      password to go with username
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    46
-mirror        create a "mirror" of the Symbian repository tree
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    47
-retries       number of times to retry a failed operation (default 1)
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    48
-verbose       print the underlying "hg" commands before executing them
15
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    49
-n             do nothing - don't actually execute the commands
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    50
-help          print this help information
15
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    51
-exec          execute command on each repository
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    52
-filter <RE>   only process repository paths matching regular expression <RE>
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    53
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    54
The -exec option processes the rest of the command line, treating it as
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    55
a command to apply to each repository in turn. Some keywords are expanded
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    56
to repository-specific values, and "hg" is always expanded to "hg -R %REPO%"
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    57
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    58
%REPO%         relative path to the repository
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    59
%WREPO%        relative path to repository, with Windows path separators
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    60
%URL%          URL of the master repository
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    61
%PUSHURL%      URL suitable for pushing (always includes username & password)
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    62
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    63
It's often useful to use "--" to separate the exec command from the options
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    64
to this script, e.g. "-exec -- hg update -C tip"
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    65
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    66
EOF
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    67
  exit (1);  
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    68
  }
5
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    69
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
    70
my @clone_options = (); # use ("--noupdate") to clone without extracting the source
10
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
    71
my @pull_options  = (); # use ("--rebase") to rebase your changes when pulling
7
2184cc44590a Clone from developer.symbian.org (Bug 79) and use https for SFL packages
William Roberts <williamr@symbian.org>
parents: 5
diff changeset
    72
my $hostname = "developer.symbian.org";
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    73
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    74
my $username = "";
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    75
my $password = "";
10
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
    76
my $mirror = 0; # set to 1 if you want to mirror the repository structure
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
    77
my $retries = 1;  # number of times to retry problem repos
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    78
my $verbose = 0;  # turn on more tracing
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    79
my $do_nothing = 0; # print the hg commands, don't actually do them
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    80
my $help = 0;
15
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    81
my $exec = 0;
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    82
my $filter = "";
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    83
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    84
if (!GetOptions(
12
dda0176e838b Fix mistake in -password and -username options, tweak the (doomed) sfl repository screen scraping
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    85
    "u|username=s" => \$username,
dda0176e838b Fix mistake in -password and -username options, tweak the (doomed) sfl repository screen scraping
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
    86
    "p|password=s" => \$password,
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    87
    "m|mirror" => \$mirror, 
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    88
    "r|retries=i" => \$retries,
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    89
    "v|verbose" => \$verbose,
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    90
    "n" => \$do_nothing,
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    91
    "h|help" => \$help,
15
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    92
    "e|exec" => \$exec,
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    93
    "f|filter=s" => \$filter,
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    94
    ))
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    95
  {
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    96
  Usage("Invalid argument");
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    97
  }
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
    98
  
15
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
    99
Usage("Too many arguments") if (scalar @ARGV > 0 && !$exec);
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   100
Usage("Too few arguments for -exec") if (scalar @ARGV == 0 && $exec);
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   101
Usage("") if ($help);
5
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   102
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   103
# Important: This script uses http access to the repositories, so
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   104
# the username and password will be stored as cleartext in the
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   105
# .hg/hgrc file in each repository.
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   106
15
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   107
my $needs_id = 1; # assumed necessary for clone/pull
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   108
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   109
my @exec_cmd = @ARGV;
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   110
if ($exec)
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   111
  {
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   112
  if ($exec_cmd[0] eq "hg")
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   113
    {
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   114
    shift @exec_cmd;
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   115
    unshift @exec_cmd, "hg", "-R", "%REPO%";
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   116
    }
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   117
  if ($verbose)
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   118
    {
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   119
    print "* Exec template = >", join("<,>", @exec_cmd), "<\n";
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   120
    }
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   121
  $needs_id = grep /URL%/,@exec_cmd; # only need id if using %URL% or %PUSHURL%
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   122
  }
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   123
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   124
if ($needs_id && $username eq "" )
5
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   125
  {
10
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   126
  print "Username: ";
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   127
  $username = <STDIN>;
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   128
  chomp $username;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   129
  }
15
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   130
if ($needs_id && $password eq "" )
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   131
  {
10
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   132
  print "Password: ";
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   133
  $password = <STDIN>;
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   134
  chomp $password;
5
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   135
  }
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   136
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   137
my @sf_packages = (
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   138
"sfl/MCL/sf/adaptation/stubs",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   139
"sfl/MCL/sf/app/camera",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   140
"sfl/MCL/sf/app/commonemail",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   141
"sfl/MCL/sf/app/conntools",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   142
"sfl/MCL/sf/app/contacts",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   143
"sfl/MCL/sf/app/contentcontrol",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   144
"sfl/MCL/sf/app/conversations",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   145
"sfl/MCL/sf/app/devicecontrol",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   146
"sfl/MCL/sf/app/dictionary",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   147
"sfl/MCL/sf/app/files",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   148
"sfl/MCL/sf/app/gallery",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   149
"sfl/MCL/sf/app/graphicsuis",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   150
"sfl/MCL/sf/app/helps",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   151
"sfl/MCL/sf/app/homescreen",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   152
"sfl/MCL/sf/app/im",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   153
"sfl/MCL/sf/app/imgeditor",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   154
"sfl/MCL/sf/app/imgvieweruis",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   155
"sfl/MCL/sf/app/iptelephony",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   156
"sfl/MCL/sf/app/java",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   157
"sfl/MCL/sf/app/location",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   158
"sfl/MCL/sf/app/messaging",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   159
"sfl/MCL/sf/app/mmsharinguis",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   160
"sfl/MCL/sf/app/musicplayer",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   161
"sfl/MCL/sf/app/organizer",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   162
"sfl/MCL/sf/app/phone",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   163
"sfl/MCL/sf/app/photos",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   164
"sfl/MCL/sf/app/poc",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   165
"sfl/MCL/sf/app/printing",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   166
"sfl/MCL/sf/app/profile",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   167
"sfl/MCL/sf/app/radio",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   168
"sfl/MCL/sf/app/screensaver",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   169
"sfl/MCL/sf/app/settingsuis",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   170
"sfl/MCL/sf/app/speechsrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   171
"sfl/MCL/sf/app/techview",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   172
# "sfl/MCL/sf/app/test",  - removed in 7 May 09 delivery
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   173
"sfl/MCL/sf/app/utils",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   174
"sfl/MCL/sf/app/videocenter",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   175
"sfl/MCL/sf/app/videoeditor",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   176
"sfl/MCL/sf/app/videoplayer",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   177
"sfl/MCL/sf/app/videotelephony",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   178
"sfl/MCL/sf/app/voicerec",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   179
  "oss/MCL/sf/app/webuis",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   180
"sfl/MCL/sf/mw/accesssec",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   181
"sfl/MCL/sf/mw/appinstall",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   182
"sfl/MCL/sf/mw/appsupport",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   183
"sfl/MCL/sf/mw/camerasrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   184
"sfl/MCL/sf/mw/classicui",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   185
"sfl/MCL/sf/mw/dlnasrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   186
"sfl/MCL/sf/mw/drm",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   187
"sfl/MCL/sf/mw/hapticsservices",
28
7896313fcd8f Added new repositories sfl/MCL/sf/mw/helix and oss/FCL/interim/QEMU
William Roberts <williamr@symbian.org>
parents: 16
diff changeset
   188
"sfl/MCL/sf/mw/helix",
5
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   189
"sfl/MCL/sf/mw/homescreensrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   190
"sfl/MCL/sf/mw/imghandling",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   191
"sfl/MCL/sf/mw/imsrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   192
"sfl/MCL/sf/mw/inputmethods",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   193
"sfl/MCL/sf/mw/ipappprotocols",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   194
"sfl/MCL/sf/mw/ipappsrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   195
"sfl/MCL/sf/mw/ipconnmgmt",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   196
"sfl/MCL/sf/mw/legacypresence",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   197
"sfl/MCL/sf/mw/locationsrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   198
"sfl/MCL/sf/mw/mds",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   199
"sfl/MCL/sf/mw/messagingmw",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   200
"sfl/MCL/sf/mw/metadatasrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   201
"sfl/MCL/sf/mw/mmappfw",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   202
"sfl/MCL/sf/mw/mmmw",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   203
"sfl/MCL/sf/mw/mmuifw",
10
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   204
# "sfl/MCL/sf/mw/mobiletv", - empty package abandoned
5
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   205
"sfl/MCL/sf/mw/netprotocols",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   206
"sfl/MCL/sf/mw/networkingdm",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   207
"sfl/MCL/sf/mw/opensrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   208
"sfl/MCL/sf/mw/phonesrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   209
"sfl/MCL/sf/mw/remoteconn",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   210
"sfl/MCL/sf/mw/remotemgmt",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   211
"sfl/MCL/sf/mw/remotestorage",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   212
"sfl/MCL/sf/mw/securitysrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   213
  "oss/MCL/sf/mw/serviceapi",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   214
  "oss/MCL/sf/mw/serviceapifw",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   215
"sfl/MCL/sf/mw/shortlinkconn",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   216
"sfl/MCL/sf/mw/svgt",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   217
"sfl/MCL/sf/mw/uiaccelerator",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   218
"sfl/MCL/sf/mw/uiresources",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   219
"sfl/MCL/sf/mw/uitools",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   220
"sfl/MCL/sf/mw/videoutils",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   221
"sfl/MCL/sf/mw/vpnclient",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   222
  "oss/MCL/sf/mw/web",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   223
"sfl/MCL/sf/mw/websrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   224
"sfl/MCL/sf/mw/wirelessacc",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   225
"sfl/MCL/sf/os/boardsupport",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   226
"sfl/MCL/sf/os/buildtools",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   227
"sfl/MCL/sf/os/cellularsrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   228
"sfl/MCL/sf/os/commsfw",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   229
"sfl/MCL/sf/os/deviceplatformrelease",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   230
"sfl/MCL/sf/os/devicesrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   231
"sfl/MCL/sf/os/graphics",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   232
"sfl/MCL/sf/os/imagingext",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   233
"sfl/MCL/sf/os/kernelhwsrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   234
"sfl/MCL/sf/os/lbs",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   235
# "sfl/MCL/sf/os/misc",  - removed in 7 May 09 delivery
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   236
"sfl/MCL/sf/os/mm",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   237
"sfl/MCL/sf/os/networkingsrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   238
"sfl/MCL/sf/os/ossrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   239
"sfl/MCL/sf/os/persistentdata",
16
35c6c3f66bc4 Updated now that sf/os/security package has moved from sfl to oss
William Roberts <williamr@symbian.org>
parents: 15
diff changeset
   240
  "oss/MCL/sf/os/security",  # moved from SFL to EPL, 8th July 09
5
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   241
"sfl/MCL/sf/os/shortlinksrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   242
"sfl/MCL/sf/os/textandloc",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   243
"sfl/MCL/sf/os/unref",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   244
"sfl/MCL/sf/os/wlan",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   245
"sfl/MCL/sf/os/xmlsrv",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   246
"sfl/MCL/sf/ostools/osrndtools",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   247
"sfl/MCL/sf/tools/build_s60",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   248
"sfl/MCL/sf/tools/buildplatforms",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   249
"sfl/MCL/sf/tools/homescreentools",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   250
"sfl/MCL/sf/tools/makefile_templates",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   251
"sfl/MCL/sf/tools/platformtools",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   252
"sfl/MCL/sf/tools/rndtools",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   253
"sfl/MCL/sf/tools/swconfigtools",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   254
);
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   255
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   256
my @sftools_packages = (
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   257
"sfl/MCL/sftools/ana/compatanaapps",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   258
"sfl/MCL/sftools/ana/compatanamdw",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   259
"sfl/MCL/sftools/ana/dynaanaapps",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   260
"sfl/MCL/sftools/ana/dynaanactrlandcptr",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   261
"sfl/MCL/sftools/ana/dynaanamdw/analysistools",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   262
"sfl/MCL/sftools/ana/dynaanamdw/crashmdw",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   263
"sfl/MCL/sftools/ana/staticanaapps",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   264
"sfl/MCL/sftools/ana/staticanamdw",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   265
"sfl/MCL/sftools/ana/testcreationandmgmt",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   266
"sfl/MCL/sftools/ana/testexec",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   267
"sfl/MCL/sftools/ana/testfw",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   268
# "sfl/MCL/sftools/depl/sdkcreationapps",  - removed in 7 May 09 delivery
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   269
"sfl/MCL/sftools/depl/sdkcreationmdw/packaging",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   270
# "sfl/MCL/sftools/depl/sdkcreationmdw/sdkbuild",  - removed in 7 May 09 delivery
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   271
# "sfl/MCL/sftools/depl/sdkcreationmdw/sdkdelivery",  - removed in 7 May 09 delivery
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   272
# "sfl/MCL/sftools/depl/sdkcreationmdw/sdktest",  - removed in 7 May 09 delivery
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   273
"sfl/MCL/sftools/depl/swconfigapps/configtools",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   274
"sfl/MCL/sftools/depl/swconfigapps/swmgnttoolsguides",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   275
"sfl/MCL/sftools/depl/swconfigapps/sysmodeltools",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   276
"sfl/MCL/sftools/depl/swconfigmdw",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   277
# "sfl/MCL/sftools/depl/sysdocapps",  - removed in 7 May 09 delivery
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   278
# "sfl/MCL/sftools/depl/sysdocmdw",  - removed in 7 May 09 delivery
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   279
# "sfl/MCL/sftools/depl/toolsplatrelease",  - removed in 7 May 09 delivery
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   280
"sfl/MCL/sftools/dev/build",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   281
"sfl/MCL/sftools/dev/dbgsrvsmdw",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   282
"sfl/MCL/sftools/dev/devicedbgsrvs",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   283
  "oss/MCL/sftools/dev/eclipseenv/buildlayout34",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   284
  "oss/MCL/sftools/dev/eclipseenv/eclipse",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   285
  "oss/MCL/sftools/dev/hostenv/compilationtoolchains",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   286
  "oss/MCL/sftools/dev/hostenv/cpptoolsplat",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   287
  "oss/MCL/sftools/dev/hostenv/dist",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   288
  "oss/MCL/sftools/dev/hostenv/javatoolsplat",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   289
  "oss/MCL/sftools/dev/hostenv/makeng",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   290
  "oss/MCL/sftools/dev/hostenv/pythontoolsplat",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   291
  "oss/MCL/sftools/dev/ide/carbidecpp",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   292
"sfl/MCL/sftools/dev/ide/carbidecppplugins",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   293
"sfl/MCL/sftools/dev/iss",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   294
"sfl/MCL/sftools/dev/ui",
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   295
);
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   296
10
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   297
my @other_repos = (
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   298
# Foundation build framework
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   299
"oss/FCL/interim/fbf/bootstrap",
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   300
"oss/FCL/interim/fbf/configs/default",
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   301
"oss/FCL/interim/fbf/configs/pkgbuild",
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   302
"oss/FCL/interim/fbf/projects/packages/serviceapi",
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   303
"oss/FCL/interim/fbf/projects/packages/serviceapifw",
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   304
"oss/FCL/interim/fbf/projects/packages/web",
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   305
"oss/FCL/interim/fbf/projects/packages/webuis",
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   306
"oss/FCL/interim/fbf/projects/platforms",
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   307
# Utilities
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   308
"oss/MCL/utilities",
28
7896313fcd8f Added new repositories sfl/MCL/sf/mw/helix and oss/FCL/interim/QEMU
William Roberts <williamr@symbian.org>
parents: 16
diff changeset
   309
# QEMU
7896313fcd8f Added new repositories sfl/MCL/sf/mw/helix and oss/FCL/interim/QEMU
William Roberts <williamr@symbian.org>
parents: 16
diff changeset
   310
"oss/MCL/interim/QEMU",
10
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   311
);
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   312
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   313
sub do_system(@)
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   314
  {
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   315
  my (@cmd) = @_;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   316
  
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   317
  if ($verbose)
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   318
    {
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   319
    print "* ", join(" ", @cmd), "\n";
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   320
    }
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   321
  return 0 if ($do_nothing);
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   322
  
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   323
  return system(@cmd);
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   324
  }
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   325
10
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   326
sub get_repo($)
5
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   327
  {
10
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   328
  my ($package) = @_;
5
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   329
  my @dirs = split /\//, $package;
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   330
  my $license = shift @dirs;
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   331
  my $repotree = shift @dirs; # remove the MCL or FCL repo tree information
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   332
  my $destdir = pop @dirs;  # ignore the package name, because Mercurial will create that
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   333
  
10
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   334
  if ($mirror)
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   335
    {
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   336
    # Mirror the full directory structure, so put back the license & repotree dirs
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   337
    unshift @dirs, $repotree;
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   338
    unshift @dirs, $license;
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   339
    }
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   340
5
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   341
  # Ensure the directories already exist as far as the parent of the repository
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   342
  my $path = "";
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   343
  foreach my $dir (@dirs)
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   344
    {
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   345
    $path = ($path eq "") ? $dir : "$path/$dir";
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   346
    if (!-d $path)
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   347
      {
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   348
      mkdir $path;
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   349
      }
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   350
    }
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   351
  
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   352
  $path .= "/$destdir";   # this is where the repository will go
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   353
7
2184cc44590a Clone from developer.symbian.org (Bug 79) and use https for SFL packages
William Roberts <williamr@symbian.org>
parents: 5
diff changeset
   354
  my $repo_url = "https://$username:$password\@$hostname/$package/";
15
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   355
  my $repo_push_url =$repo_url;
5
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   356
  if ($license ne "sfl")
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   357
    {
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   358
    # user registration is not required for reading public package repositories
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   359
    $repo_url = "http://developer.symbian.org/$package/";
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   360
    }
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   361
  
15
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   362
  if ($exec)
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   363
    {
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   364
    # iteration functionality - process the keywords
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   365
    my $wpath = $path;
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   366
    $wpath =~ s/\//\\/g;  # win32 path separator
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   367
    my @repo_cmd = ();
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   368
    foreach my $origcmd (@exec_cmd)
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   369
      {
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   370
      my $cmd = $origcmd; # avoid altering the original
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   371
      $cmd =~ s/%REPO%/$path/;
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   372
      $cmd =~ s/%WREPO%/$wpath/;
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   373
      $cmd =~ s/%URL%/$repo_url/;
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   374
      $cmd =~ s/%PUSHURL%/$repo_push_url/;
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   375
      push @repo_cmd, $cmd;
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   376
      }
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   377
    print "Processing $path...\n";
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   378
    return do_system(@repo_cmd);
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   379
    }
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   380
  elsif (-d "$path/.hg")
5
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   381
    {
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   382
    # The repository already exists, so just do an update
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   383
    
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   384
    print "Updating $destdir from $package...\n";
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   385
    return do_system("hg", "pull", @pull_options, "-R", $path, $repo_url);
5
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   386
    }
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   387
  else
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   388
    {
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   389
    # Clone the repository
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   390
    
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   391
    print "Cloning $destdir from $package...\n";
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   392
    return do_system("hg", "clone", @clone_options, $repo_url, $path);
5
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   393
    }
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   394
  
c34a018f3291 Re-introduce the clone_all_packages.pl script
William Roberts <williamr@symbian.org>
parents:
diff changeset
   395
  }
10
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   396
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   397
my @all_packages;
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   398
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   399
@all_packages = (@sf_packages, @sftools_packages, @other_repos);
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   400
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   401
if ($mirror)
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   402
  {
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   403
  push @clone_options, "--noupdate";
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   404
  
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   405
  if (0)
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   406
    {
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   407
    # Prototype code to scrape the repository list from the website
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   408
    # Needs to have extra modules and executables installed to support https
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   409
    # so this would only work for the oss packages at present...
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   410
    
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   411
    # Create a user agent object
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   412
    use LWP::UserAgent;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   413
    use HTTP::Request::Common;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   414
    my $ua = LWP::UserAgent->new;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   415
    $ua->agent("clone_all_packages.pl ");
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   416
  
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   417
    # Request the oss package list
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   418
    my $res = $ua->request(GET "http://$hostname/oss");
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   419
  
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   420
    # Check the outcome of the response
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   421
    if (! $res->is_success) 
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   422
      {
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   423
      print "Failed to read oss package list:\n\t", $res->status_line, "\n";
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   424
      }
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   425
    
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   426
    my @oss_packages = ($res->content =~ m/<td><a href="\/(oss\/[^"]+)\/?">/g);  # umatched "
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   427
    print join ("\n\t",@oss_packages), "\n";
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   428
12
dda0176e838b Fix mistake in -password and -username options, tweak the (doomed) sfl repository screen scraping
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   429
    # Request the sfl package list
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   430
    $res = $ua->request(GET "https://$username:$password\@$hostname/sfl");
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   431
  
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   432
    # Check the outcome of the response
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   433
    if (! $res->is_success) 
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   434
      {
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   435
      print "Failed to read sfl package list:\n\t", $res->status_line, "\n";
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   436
      }
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   437
    
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   438
    my @sfl_packages = ($res->content =~ m/<td><a href="\/(sfl\/[^"]+)\/?">/g);  # umatched "
12
dda0176e838b Fix mistake in -password and -username options, tweak the (doomed) sfl repository screen scraping
William Roberts <williamr@symbian.org>
parents: 11
diff changeset
   439
    print join ("\n\t",@sfl_packages), "\n";
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   440
    
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   441
    @all_packages = (@sfl_packages, @oss_packages);
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   442
    }
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   443
  else
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   444
    {
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   445
    # Assume that every MCL has a matching FCL
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   446
    my @list_with_fcls = ();
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   447
    foreach my $package (@all_packages)
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   448
      {
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   449
      push @list_with_fcls, $package;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   450
      if ($package =~ /MCL/)
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   451
        {
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   452
        # If mirroring, get the matching FCLs as well as MCLs
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   453
        $package =~ s/MCL/FCL/;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   454
        push @list_with_fcls, $package;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   455
        }
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   456
      }
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   457
    @all_packages = @list_with_fcls;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   458
    }
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   459
10
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   460
  }
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   461
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   462
my @problem_packages = ();
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   463
my $total_packages = 0;
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   464
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   465
foreach my $package (@all_packages)
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   466
  {
15
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   467
  if ($filter && $package !~ /$filter/)
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   468
    {
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   469
    next; # skip repos which don't match the filter
880bb4a0a72d Add the -exec and -filter options to clone_all_repositories, giving simple "iterator" functionality
William Roberts <williamr@symbian.org>
parents: 12
diff changeset
   470
    }
10
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   471
  my $err = get_repo($package);
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   472
  $total_packages++;
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   473
  push @problem_packages, $package if ($err); 
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   474
  }
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   475
  
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   476
# retry problem packages
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   477
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   478
my $attempt = 0;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   479
while ($attempt < $retries && scalar @problem_packages) 
10
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   480
  {
11
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   481
  $attempt++;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   482
  printf "\n\n------------\nRetry attempt %d on %d packages\n",
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   483
    $attempt, scalar @problem_packages;
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   484
  print join("\n", @problem_packages, ""), "\n";
319764718a57 Add proper usage and more options (Bug 80). Add prototype screen-scraping to read repository list.
William Roberts <williamr@symbian.org>
parents: 10
diff changeset
   485
    
10
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   486
  my @list = @problem_packages;
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   487
  @problem_packages = ();
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   488
  foreach my $package (@list)
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   489
    {
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   490
    my $err = get_repo($package);
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   491
    push @problem_packages, $package if ($err); 
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   492
   }
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   493
  }
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   494
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   495
printf "\n------------\nProcessed %d packages, of which %d reported errors\n", 
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   496
  $total_packages, scalar @problem_packages;
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   497
if (scalar @problem_packages)
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   498
  {
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   499
  print join("\n", @problem_packages, "");
ccca32510405 Prompt for username and password (Bug 80), support retries and repo mirroring
William Roberts <williamr@symbian.org>
parents: 7
diff changeset
   500
  }