Case Insensitive Unique Array Elements in Perl - perl

I am using the uniq function exported by the module, List::MoreUtils to find the uniq elements in an array. However, I want it to find the uniq elements in a case insensitive way. How can I do that?
I have dumped the output of the Array using Data::Dumper:
#! /usr/bin/perl
use strict;
use warnings;
use Data::Dumper qw(Dumper);
use List::MoreUtils qw(uniq);
use feature "say";
my #elements=<array is formed here>;
my #words=uniq #elements;
say Dumper \#words;
Output:
$VAR1 = [
'John',
'john',
'JohN',
'JOHN',
'JoHn',
'john john'
];
Expected output should be: john, john john
Only 2 elements, rest all should be filtered since they are the same word, only the difference is in case.
How can I remove the duplicate elements ignoring the case?

Use lowercase, lc with a map statement:
my #uniq_no_case = uniq map lc, #elements;
The reason List::MoreUtils' uniq is case sensitive is that it relies on the deduping characteristics of hashes, which also is case sensitive. The code for uniq looks like so:
sub uniq {
my %seen = ();
grep { not $seen{$_}++ } #_;
}
If you want to use this sub directly in your own code, you could incorporate lc in there:
sub uniq_no_case {
my %seen = ();
grep { not $seen{$_}++ } map lc, #_;
}
Explanation of how this works:
#_ contains the args to the subroutine, and they are fed to a grep statement. Any elements that return true when passed through the code block are returned by the grep statement. The code block consist of a few finer points:
$seen{$_}++ returns 0 the first time an element is seen. The value is still incremented to 1, but after it is returned (as opposed to ++$seen{$_} who would inc first, then return).
By negating the result of the incrementation, we get true for the first key, and false for every following such key. Hence, the list is deduped.
grep as the last statement in the sub will return a list, which in turn is returned by the sub.
map lc, #_ simply applies the lc function to all elements in #_.

Use a hash to keep track of the words you have already seen, but also normalize them for upper/lower case:
my %seen;
my #unique;
for my $w (#words) {
next if $seen{lc($w)}++;
push(#unique, $w);
}
# #unique has the unique words
Note that this will preserve the case of the original words.
UPDATE: As noted in the comments, it's not clear exactly what the OP needs, but I wrote the solution this way to illustrate a general technique for selecting unique representatives from a list under some "equivalence relation." In this case the equivalence relationship is word $a is equivalent to word $b if and only if lc($a) eq lc($b).
Most equivalence relationships can be expressed in this way, that is, the relationship is defined by a classifier function f() such that $a is equivalent to $b if and only if f($a) eq f($b). For instance, if we want to say that two words are equivalent if they have the same length, then f() would be length().
So now you might see why I wrote the algorithm this way - the classifier function may not produce values that are part of the original list. In the case of f = length, we want to select words, but f of a word is a number.

Related

Is it possible to push a key-value pair directly to hash in perl?

I know pushing is only passible to array, not hash. But it would be much more convenient to allow pushing key-value pair directly to hash (and I am still surprise it is not possible in perl). I have an example:
#!/usr/bin/perl -w
#superior words begin first, example of that word follow
my #ar = qw[Animals,dog Money,pound Jobs,doctor Food];
my %hash;
my $bool = 1;
sub marine{
my $ar = shift if $bool;
for(#$ar){
my #ar2 = split /,/, $_;
push %hash, ($ar2[0] => $ar2[1]);
}
}
marine(\#ar);
print "$_\n" for keys %hash;
Here I have an array, which has 2 words separately by , comma. I would like to make a hash from it, making the first a key, and the second a value (and if it lacks the value, as does the last Food word, then no value at all -> simply undef. How to make it in perl?
Output:
Possible attempt to separate words with commas at ./a line 4.
Experimental push on scalar is now forbidden at ./a line 12, near ");"
Execution of ./a aborted due to compilation errors.
I might be oversimplyfing things here, but why not simply assign to the hash rather than trying to push into it?
That is, replace this unsupported expression:
push %hash, ($ar2[0] => $ar2[1]);
With:
$hash{$ar2[0]} = $ar2[1];
If I incoporate this in your code, and then dump the resulting hash at the end, I get:
$VAR1 = {
'Food' => undef,
'Money' => 'pound',
'Animals' => 'dog',
'Jobs' => 'doctor'
};
Split inside map and assign directly to a hash like so:
my #ar = qw[Animals,dog Money,pound Jobs,doctor Food];
my %hash_new = map {
my #a = split /,/, $_, 2;
#a == 2 ? #a : (#a, undef)
} #ar;
Note that this can also handle the case with more than one comma delimiter (hence splitting into a max of 2 elements). This can also handle the case with no commas, such as Food - in this case, the list with the single element plus the undef is returned.
If you need to push multiple key/value pairs to (another) hash, or merge hashes, you can assign a list of hashes like so:
%hash = (%hash_old, %hash_new);
Note that the same keys in the old hash will be overwritten by the new hash.
We can assign this array to a hash and perl will automatically look at the values in the array as if they were key-value pairs. The odd elements (first, third, fifth) will become the keys and the even elements (second, fourth, sixth) will become the corresponding values. check url https://perlmaven.com/creating-hash-from-an-array
use strict;
use warnings;
use Data::Dumper qw(Dumper);
my #ar;
my %hash;
#The code in the enclosing block has warnings enabled,
#but the inner block has disabled (misc and qw) related warnings.
{
#You specified an odd number of elements to initialize a hash, which is odd,
#because hashes come in key/value pairs.
no warnings 'misc';
#If your code has use warnings turned on, as it should, then you'll get a warning about
#Possible attempt to separate words with commas
no warnings 'qw';
#ar = qw[Animals,dog Money,pound Jobs,doctor Food];
# join the content of array with comma => Animals,dog,Money,pound,Jobs,doctor,Food
# split the content using comma and assign to hash
# split function returns the list in list context, or the size of the list in scalar context.
%hash = split(",", (join(",", #ar)));
}
print Dumper(\%hash);
Output
$VAR1 = {
'Animals' => 'dog',
'Money' => 'pound',
'Jobs' => 'doctor',
'Food' => undef
};

Perl: simple foreach on hash hands mixed results? [duplicate]

activePerl 5.8 based
#!C:\Perl\bin\perl.exe
use strict;
use warnings;
# declare a new hash
my %some_hash;
%some_hash = ("foo", 35, "bar", 12.4, 2.5, "hello",
"wilma", 1.72e30, "betty", "bye\n");
my #any_array;
#any_array = %some_hash;
print %some_hash;
print "\n";
print #any_array;
print "\n";
print $any_array[0];
print "\n";
print $any_array[1];
print "\n";
print $any_array[2];
print "\n";
print $any_array[3];
print "\n";
print $any_array[4];
print "\n";
print $any_array[5];
print "\n";
print $any_array[6];
print "\n";
print $any_array[7];
print "\n";
print $any_array[8];
print "\n";
print $any_array[9];
Output as this
D:\learning\perl>test.pl
bettybye
bar12.4wilma1.72e+030foo352.5hello
bettybye
bar12.4wilma1.72e+030foo352.5hello
betty
bye
bar
12.4
wilma
1.72e+030
foo
35
2.5
hello
D:\learning\perl>
What decided the elements print order in my sample code?
Any rule to follow when print a mixed(strings, numbers) hash in Perl? Thank you.
bar12.4wilma1.72e+030foo352.5hello
[Updated]
With you guys help, i updated the code as below.
#!C:\Perl\bin\perl.exe
use strict;
use warnings;
# declare a new hash
my %some_hash;
%some_hash = ("foo", 35, "bar", 12.4, 2.5, "hello",
"wilma", 1.72e30, "betty", "bye");
my #any_array;
#any_array = %some_hash;
print %some_hash;
print "\n";
print "\n";
print #any_array;
print "\n";
print "\n";
my #keys;
#keys = keys %some_hash;
for my $k (sort #keys)
{
print $k, $some_hash{$k};
}
output
D:\learning\perl>test.pl
bettybyebar12.4wilma1.72e+030foo352.5hello
bettybyebar12.4wilma1.72e+030foo352.5hello
2.5hellobar12.4bettybyefoo35wilma1.72e+030
D:\learning\perl>
Finially, after called keys and sort functions. The hash keys print followed the rule below
2.5hellobar12.4bettybyefoo35wilma1.72e+030
Elements of a hash are printed out in their internal order, which can not be relied upon and will change as elements are added and removed. If you need all of the elements of a hash in some sort of order, sort the keys, and use that list to index the hash.
If you are looking for a structure that holds its elements in order, either use an array, or use one of the ordered hash's on CPAN.
the only ordering you can rely upon from a list context hash expansion is that key => value pairs will be together.
From perldoc -f keys:
The keys of a hash are returned in an apparently random order. The actual random order is subject to change in future versions of Perl, but it is guaranteed to be the same order as either the values or each function produces (given that the hash has not been modified). Since Perl 5.8.1 the ordering is different even between different runs of Perl for security reasons (see Algorithmic Complexity Attacks in perlsec).
...
Perl has never guaranteed any ordering of the hash keys, and the ordering has already changed several times during the lifetime of Perl 5. Also, the ordering of hash keys has always been, and continues to be, affected by the insertion order.
Also note that while the order of the hash elements might be randomised, this "pseudoordering" should not be used for applications like shuffling a list randomly (use List::Util::shuffle() for that, see List::Util, a standard core module since Perl 5.8.0; or the CPAN module Algorithm::Numerical::Shuffle), or for generating permutations (use e.g. the CPAN modules Algorithm::Permute or Algorithm::FastPermute), or for any cryptographic applications.
Note: since you are evaluating a hash in list context, you are at least guaranteed that each key is followed by its corresponding value; e.g. you will never see an output of a 4 b 3 c 2 d 1.
I went over your code and made some notes that I think you will find helpful.
use strict;
use warnings;
# declare a new hash and initialize it at the same time
my %some_hash = (
foo => 35, # use the fat-comma or '=>' operator, it quotes the left side
bar => 12.4,
2.5 => "hello",
wilma => 1.72e30,
betty => "bye", # perl ignores trailing commas,
# the final comma makes adding items to the end of the list less bug prone.
);
my #any_array = %some_hash; # Hash is expanded into a list of key/value pairs.
print "$_ => $some_hash{$_}\n"
for keys %some_hash;
print "\n\n", # You can print multiple newlines in one string.
"#any_array\n\n"; # print takes a list of things to print.
# In print #foo; #foo is expanded into a list of items to print.
# There is no separator between the members of #foo in the output.
# However print "#foo"; interpolates #foo into a string.
# It inserts spaces between the members of the arrays.
# This is the block form of 'for'
for my $k (sort keys %some_hash)
{
# Interpolating the variables into a string makes it easier to read the output.
print "$k => $some_hash{$k}\n";
}
Hashes provide unordered, access to data by a string key.
Arrays provide access to ordered data. Random access is available by using a numerical index.
If you need to preserve the order of a group of values, use an array. If you need to look up members of the group by an associated name, use a hash.
If you need to do both, you can use both structures together:
# Keep an array of sorted hash keys.
my #sorted_items = qw( first second third fourth );
# Store the actual data in the hash.
my %item;
#item{ #sorted_items } = 1..4; # This is called a hash slice.
# It allows you to access a list of hash elements.
# This can be a very powerful way to work with hashes.
# random access
print "third => $item{third}\n";
# When you need to access the data in order, iterate over
# the array of sorted hash keys. Use the keys to access the
# data in the hash.
# ordered access
for my $name ( #sorted_items ) {
print "$name => $item{$name}\n";
}
Looking at your code samples, I see a couple of things you might want to work on.
how looping structures like for and while can be used to reduce repeated code.
how to use variable interpolation
BTW, I am glad to see you working on basics and improving your code quality. This investment of time will pay off. Keep up the good work.
The elements are (almost certainly) printed out in the order they appear (internally) in the hash table itself -- i.e. based on the hash values of their keys.
The general rule to follow is to use something other than a hash table if you care much about the order.
Hashes are not (necessarily) retrieved in a sorted manner. If you want them sorted, you have to do it yourself:
use strict;
use warnings;
my %hash = ("a" => 1, "b" => 2, "c" => 3, "d" => 4);
for my $i (sort keys %hash) {
print "$i -> $hash{$i}\n";
}
You retrieve all the keys from a hash by using keys and you then sort them using sort. Yeah, I know, that crazy Larry Wall guy, who would've ever thought of calling them that? :-)
This outputs:
a -> 1
b -> 2
c -> 3
d -> 4
For most practical purposes, the order in which a hash table (not just Perl hash variables, but hash tables in general) can be considered random.
In reality, depending on the hashing implementation, the order may actually be deterministic. (i.e., If you run the program multiple times putting the same items into the hash table in the same order each time, they'll be stored in the same order each time.) I know that Perl hashes used to have this characteristic, but I'm not sure about current versions. In any case, hash key order is not a reliable source of randomness to use in cases where randomness is desirable.
Short version, then:
Don't use a hash if you care about the order (or lack of order). If you want a fixed order, it will be effectively random and if you want a random order, it will be effectively fixed.
A hash defines no ordering properties. The order in which things come out will be unpredictable.
And if you are crazy and have no duplicate values in your hash, and you need the values sorted, you can call reverse on it.
my %hash = ("a" => 1, "b" => 2, "c" => 3, "d" => 4);
my %reverse_hash = reverse %hash;
print $_ for sort keys %reverse_hash;
Caveat is the unique values part, duplicates will be overwritten and only one value will get in.

Sorting an Array Reference to Hashes

After executing these lines in Perl:
my $data = `curl '$url'`;
my $pets = XMLin($data)->(pets);
I have an array reference that contains references to hashes:
$VAR1 = [
{
'title' => 'cat',
'count' => '210'
},
{
'title' => 'dog',
'count' => '210'
}
]
In Perl, how do I sort the hashes first by count and secondarily by title. Then print to STDOUT the count followed by the title on each newline.
Assuming you want counts in descending order and titles ascending:
print map join(" ", #$_{qw/ count title /}) . "\n",
sort { $b->{count} <=> $a->{count}
||
$a->{title} cmp $b->{title} }
#$pets;
That's compact code written in a functional style. To help understand it, let's look at equivalent code in a more familiar, imperative style.
Perl's sort operator takes an optional SUBNAME parameter that allows you to factor out your comparison and give it a name that describes what it does. When I do this, I like to begin the sub's name with by_ to make sort by_... ready more naturally.
To start, you might have written
sub by_count_then_title {
$b->{count} <=> $a->{count}
||
$a->{title} cmp $b->{title}
}
my #sorted = sort by_count_then_title #$pets;
Note that no comma follows the SUBNAME in this form!
To address another commenter's question, you could use or rather than || in by_count_then_title if you find it more readable. Both <=> and cmp have higher precedence (which you might think of as binding more tightly) than || and or, so it's strictly a matter of style.
To print the sorted array, a more familiar choice might be
foreach my $p (#sorted) {
print "$p->{count} $p->{title}\n";
}
Perl uses $_ if you don't specify the variable that gets each value, so the following has the same meaning:
for (#sorted) {
print "$_->{count} $_->{title}\n";
}
The for and foreach keywords are synonyms, but I find that the uses above, i.e., foreach if I'm going to name a variable or for otherwise, read most naturally.
Using map, a close cousin of foreach, instead isn't much different:
map print("$_->{count} $_->{title}\n"), #sorted;
You could also promote print through the map:
print map "$_->{count} $_->{title}\n",
#sorted;
Finally, to avoid repetition of $_->{...}, the hash slice #$_{"count", "title"} gives us the values associated with count and title in the loop's current record. Having the values, we need to join them with a single space and append a newline to the result, so
print map join(" ", #$_{qw/ count title /}) . "\n",
#sorted;
Remember that qw// is shorthand for writing a list of strings. As this example shows, read a map expression back-to-front (or bottom-to-top the way I indented it): first sort the records, then format them, then print them.
You could eliminate the temporary #sorted but call the named comparison:
print map join(" ", #$_{qw/ count title /}) . "\n",
sort by_count_then_title
#$pets;
If the application of join is just too verbose for your taste, then
print map "#$_{qw/ count title /}\n",
sort by_count_then_title
#$pets;

What decides the order of keys when I print a Perl hash?

activePerl 5.8 based
#!C:\Perl\bin\perl.exe
use strict;
use warnings;
# declare a new hash
my %some_hash;
%some_hash = ("foo", 35, "bar", 12.4, 2.5, "hello",
"wilma", 1.72e30, "betty", "bye\n");
my #any_array;
#any_array = %some_hash;
print %some_hash;
print "\n";
print #any_array;
print "\n";
print $any_array[0];
print "\n";
print $any_array[1];
print "\n";
print $any_array[2];
print "\n";
print $any_array[3];
print "\n";
print $any_array[4];
print "\n";
print $any_array[5];
print "\n";
print $any_array[6];
print "\n";
print $any_array[7];
print "\n";
print $any_array[8];
print "\n";
print $any_array[9];
Output as this
D:\learning\perl>test.pl
bettybye
bar12.4wilma1.72e+030foo352.5hello
bettybye
bar12.4wilma1.72e+030foo352.5hello
betty
bye
bar
12.4
wilma
1.72e+030
foo
35
2.5
hello
D:\learning\perl>
What decided the elements print order in my sample code?
Any rule to follow when print a mixed(strings, numbers) hash in Perl? Thank you.
bar12.4wilma1.72e+030foo352.5hello
[Updated]
With you guys help, i updated the code as below.
#!C:\Perl\bin\perl.exe
use strict;
use warnings;
# declare a new hash
my %some_hash;
%some_hash = ("foo", 35, "bar", 12.4, 2.5, "hello",
"wilma", 1.72e30, "betty", "bye");
my #any_array;
#any_array = %some_hash;
print %some_hash;
print "\n";
print "\n";
print #any_array;
print "\n";
print "\n";
my #keys;
#keys = keys %some_hash;
for my $k (sort #keys)
{
print $k, $some_hash{$k};
}
output
D:\learning\perl>test.pl
bettybyebar12.4wilma1.72e+030foo352.5hello
bettybyebar12.4wilma1.72e+030foo352.5hello
2.5hellobar12.4bettybyefoo35wilma1.72e+030
D:\learning\perl>
Finially, after called keys and sort functions. The hash keys print followed the rule below
2.5hellobar12.4bettybyefoo35wilma1.72e+030
Elements of a hash are printed out in their internal order, which can not be relied upon and will change as elements are added and removed. If you need all of the elements of a hash in some sort of order, sort the keys, and use that list to index the hash.
If you are looking for a structure that holds its elements in order, either use an array, or use one of the ordered hash's on CPAN.
the only ordering you can rely upon from a list context hash expansion is that key => value pairs will be together.
From perldoc -f keys:
The keys of a hash are returned in an apparently random order. The actual random order is subject to change in future versions of Perl, but it is guaranteed to be the same order as either the values or each function produces (given that the hash has not been modified). Since Perl 5.8.1 the ordering is different even between different runs of Perl for security reasons (see Algorithmic Complexity Attacks in perlsec).
...
Perl has never guaranteed any ordering of the hash keys, and the ordering has already changed several times during the lifetime of Perl 5. Also, the ordering of hash keys has always been, and continues to be, affected by the insertion order.
Also note that while the order of the hash elements might be randomised, this "pseudoordering" should not be used for applications like shuffling a list randomly (use List::Util::shuffle() for that, see List::Util, a standard core module since Perl 5.8.0; or the CPAN module Algorithm::Numerical::Shuffle), or for generating permutations (use e.g. the CPAN modules Algorithm::Permute or Algorithm::FastPermute), or for any cryptographic applications.
Note: since you are evaluating a hash in list context, you are at least guaranteed that each key is followed by its corresponding value; e.g. you will never see an output of a 4 b 3 c 2 d 1.
I went over your code and made some notes that I think you will find helpful.
use strict;
use warnings;
# declare a new hash and initialize it at the same time
my %some_hash = (
foo => 35, # use the fat-comma or '=>' operator, it quotes the left side
bar => 12.4,
2.5 => "hello",
wilma => 1.72e30,
betty => "bye", # perl ignores trailing commas,
# the final comma makes adding items to the end of the list less bug prone.
);
my #any_array = %some_hash; # Hash is expanded into a list of key/value pairs.
print "$_ => $some_hash{$_}\n"
for keys %some_hash;
print "\n\n", # You can print multiple newlines in one string.
"#any_array\n\n"; # print takes a list of things to print.
# In print #foo; #foo is expanded into a list of items to print.
# There is no separator between the members of #foo in the output.
# However print "#foo"; interpolates #foo into a string.
# It inserts spaces between the members of the arrays.
# This is the block form of 'for'
for my $k (sort keys %some_hash)
{
# Interpolating the variables into a string makes it easier to read the output.
print "$k => $some_hash{$k}\n";
}
Hashes provide unordered, access to data by a string key.
Arrays provide access to ordered data. Random access is available by using a numerical index.
If you need to preserve the order of a group of values, use an array. If you need to look up members of the group by an associated name, use a hash.
If you need to do both, you can use both structures together:
# Keep an array of sorted hash keys.
my #sorted_items = qw( first second third fourth );
# Store the actual data in the hash.
my %item;
#item{ #sorted_items } = 1..4; # This is called a hash slice.
# It allows you to access a list of hash elements.
# This can be a very powerful way to work with hashes.
# random access
print "third => $item{third}\n";
# When you need to access the data in order, iterate over
# the array of sorted hash keys. Use the keys to access the
# data in the hash.
# ordered access
for my $name ( #sorted_items ) {
print "$name => $item{$name}\n";
}
Looking at your code samples, I see a couple of things you might want to work on.
how looping structures like for and while can be used to reduce repeated code.
how to use variable interpolation
BTW, I am glad to see you working on basics and improving your code quality. This investment of time will pay off. Keep up the good work.
The elements are (almost certainly) printed out in the order they appear (internally) in the hash table itself -- i.e. based on the hash values of their keys.
The general rule to follow is to use something other than a hash table if you care much about the order.
Hashes are not (necessarily) retrieved in a sorted manner. If you want them sorted, you have to do it yourself:
use strict;
use warnings;
my %hash = ("a" => 1, "b" => 2, "c" => 3, "d" => 4);
for my $i (sort keys %hash) {
print "$i -> $hash{$i}\n";
}
You retrieve all the keys from a hash by using keys and you then sort them using sort. Yeah, I know, that crazy Larry Wall guy, who would've ever thought of calling them that? :-)
This outputs:
a -> 1
b -> 2
c -> 3
d -> 4
For most practical purposes, the order in which a hash table (not just Perl hash variables, but hash tables in general) can be considered random.
In reality, depending on the hashing implementation, the order may actually be deterministic. (i.e., If you run the program multiple times putting the same items into the hash table in the same order each time, they'll be stored in the same order each time.) I know that Perl hashes used to have this characteristic, but I'm not sure about current versions. In any case, hash key order is not a reliable source of randomness to use in cases where randomness is desirable.
Short version, then:
Don't use a hash if you care about the order (or lack of order). If you want a fixed order, it will be effectively random and if you want a random order, it will be effectively fixed.
A hash defines no ordering properties. The order in which things come out will be unpredictable.
And if you are crazy and have no duplicate values in your hash, and you need the values sorted, you can call reverse on it.
my %hash = ("a" => 1, "b" => 2, "c" => 3, "d" => 4);
my %reverse_hash = reverse %hash;
print $_ for sort keys %reverse_hash;
Caveat is the unique values part, duplicates will be overwritten and only one value will get in.

How do I print unique elements in Perl array?

I'm pushing elements into an array during a while statement. Each element is a teacher's name. There ends up being duplicate teacher names in the array when the loop finishes. Sometimes they are not right next to each other in the array, sometimes they are.
How can I print only the unique values in that array after its finished getting values pushed into it? Without having to parse the entire array each time I want to print an element.
Heres the code after everything has been pushed into the array:
$faculty_len = #faculty;
$i=0;
while ($i != $faculty_len)
{
printf $fh '"'.$faculty[$i].'"';
$i++;
}
use List::MoreUtils qw/ uniq /;
my #unique = uniq #faculty;
foreach ( #unique ) {
print $_, "\n";
}
Your best bet would be to use a (basically) built-in tool, like uniq (as described by innaM).
If you don't have the ability to use uniq and want to preserve order, you can use grep to simulate that.
my %seen;
my #unique = grep { ! $seen{$_}++ } #faculty;
# printing, etc.
This first gives you a hash where each key is each entry. Then, you iterate over each element, counting how many of them there are, and adding the first one. (Updated with comments by brian d foy)
I suggest pushing it into a hash.
like this:
my %faculty_hash = ();
foreach my $facs (#faculty) {
$faculty_hash{$facs} = 1;
}
my #faculty_unique = keys(%faculty_hash);
#array1 = ("abc", "def", "abc", "def", "abc", "def", "abc", "def", "xyz");
#array1 = grep { ! $seen{ $_ }++ } #array1;
print "#array1\n";
This question is answered with multiple solutions in perldoc. Just type at command line:
perldoc -q duplicate
Please note: Some of the answers containing a hash will change the ordering of the array. Hashes dont have any kind of order, so getting the keys or values will make a list with an undefined ordering.
This doen't apply to grep { ! $seen{$_}++ } #faculty
This is a one liner command to print unique lines in order it appears.
perl -ne '$seen{$_}++ || print $_' fileWithDuplicateValues
I just found hackneyed 3 liner, enjoy
my %uniq;
undef #uniq(#non_uniq_array);
my #uniq_array = keys %uniq;
Just another way to do it, useful only if you don't care about order:
my %hash;
#hash{#faculty}=1;
my #unique=keys %hash;
If you want to avoid declaring a new variable, you can use the somehow underdocumented global variable %_
#_{#faculty}=1;
my #unique=keys %_;
If you need to process the faculty list in any way, a map over the array converted to a hash for key coalescing and then sorting keys is another good way:
my #deduped = sort keys %{{ map { /.*/? ($_,1):() } #faculty }};
print join("\n", #deduped)."\n";
You process the list by changing the /.*/ regex for selecting or parsing and capturing accordingly, and you can output one or more mutated, non-unique keys per pass by making ($_,1):() arbitrarily complex.
If you need to modify the data in-flight with a substitution regex, say to remove dots from the names (s/\.//g), then a substitution according to the above pattern will mutate the original #faculty array due to $_ aliasing. You can get around $_ aliasing by making an anonymous copy of the #faculty array (see the so-called "baby cart" operator):
my #deduped = sort keys %{{ map {/.*/? do{s/\.//g; ($_,1)}:()} #{[ #faculty ]} }};
print join("\n", #deduped)."\n";
print "Unmolested array:\n".join("\n", #faculty)."\n";
In more recent versions of Perl, you can pass keys a hashref, and you can use the non-destructive substitution:
my #deduped = sort keys { map { /.*/? (s/\.//gr,1):() } #faculty };
Otherwise, the grep or $seen[$_]++ solutions elsewhere may be preferable.