common/tools/sortUnique.pl
author Simon Howkins <simonh@symbian.org>
Tue, 05 Oct 2010 13:15:25 +0100
changeset 1279 3329abb5855b
parent 246 4f2482f1dd48
child 1316 0b4a09013baf
permissions -rw-r--r--
Added default value for content of RnD sources.csv, so it doesn't cause problems if there's no file there
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
246
4f2482f1dd48 Added sorting and removal of duplicates from TSV files.
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     1
#!perl -w
4f2482f1dd48 Added sorting and removal of duplicates from TSV files.
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     2
use strict;
4f2482f1dd48 Added sorting and removal of duplicates from TSV files.
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     3
# Sorts the input, removes duplicates, and outputs it
4f2482f1dd48 Added sorting and removal of duplicates from TSV files.
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     4
4f2482f1dd48 Added sorting and removal of duplicates from TSV files.
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     5
# Read it
4f2482f1dd48 Added sorting and removal of duplicates from TSV files.
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     6
my @content = <>;
4f2482f1dd48 Added sorting and removal of duplicates from TSV files.
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     7
4f2482f1dd48 Added sorting and removal of duplicates from TSV files.
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     8
# Sort it, and grep to remove duplicates
4f2482f1dd48 Added sorting and removal of duplicates from TSV files.
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
     9
my $previous = "\n\n";
4f2482f1dd48 Added sorting and removal of duplicates from TSV files.
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    10
@content = grep {$_ ne $previous && ($previous = $_, 1) } sort @content;
4f2482f1dd48 Added sorting and removal of duplicates from TSV files.
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    11
4f2482f1dd48 Added sorting and removal of duplicates from TSV files.
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    12
# Write it
4f2482f1dd48 Added sorting and removal of duplicates from TSV files.
Simon Howkins <simonh@symbian.org>
parents:
diff changeset
    13
print @content;