Remove duplicate numbers within parentheses using sed - sed

Question

I am trying to remove duplicate numbers within parentheses using sed.
So I have the following string:
Abdc 1234 1234 (5678) (5678) (9012) (9012) (3456)
I want to use sed to remove any 4-digit numbers within the parentheses, including the parentheses. So my string should look like this:
Abdc 1234 1234 (5678) (9012) (3456)
In this case the "(5678)" and "(9012)" were removed because they were 4-digit numbers within parentheses that repeated. The "1234" numbers were not removed because they were not within parenthesis. The "(3456)" was not removed because it was not repeating.

Answer 1

I do not know how to do this with sed but you could try the following with awk:
$ echo "Abdc 1234 1234 (5678) (5678) (9012) (9012) (3456)" | awk '
{
for(i=1;i<=NF;i++) {
if(substr($i,0,1) != "(" || (seen[$i] != 1)) {
seen[$i]=1;
printf "%s ",$i
}
};
print ""
}'
Output:
Abdc 1234 1234 (5678) (9012) (3456)
This loops through the line fields then prints each field only if it has never been seen before or if it is not starting with (.

Answer 2

This works for your input:
echo 'Abdc 1234 1234 (5678) (5678) (9012) (9012) (3456)' |
sed 's/\(([0-9][0-9]*)\) \1/\1/g'
It assumes duplicates follow each other, if that is not the case, use this version:
echo 'Abdc 1234 1234 (5678) (5678) (9012) (9012) (3456)' |
sed 's/\(([0-9][0-9]*)\) \(.*\)\1/\1\2/g'
Or a bit shorter with GNU sed extended expressions:
echo 'Abdc 1234 1234 (5678) (5678) (9012) (9012) (3456)' |
sed -r 's/(\([0-9]+\)) (.*)\1/\1\2/g'
Output in all cases:
Abdc 1234 1234 (5678) (9012) (3456)
Edit - handle situation where more than two identical items exist
This can be done by looping over the pattern until it no longer matches:
echo 'Abdc 1234 1234 (5678) (5678) (9012) (9012) (3456) (5678) (5678)' |
sed -r ':a; s/(\([0-9]+\))(.*)\1 ?/\1\2/g; ta'

Answer 3

Using Perl :
$ echo "Abdc 1234 1234 (5678) (5678) (9012) (9012) (3456)" |
perl -ne '
my (#arr, %hash);
for (split) {
if (/^\(.*\)/) {
$hash{$_}++;
push #arr, $_ if $hash{$_} == 1;
}
else {
push #arr, $_;
}
}
print join " ", #arr, "\n";
'
That will works with multi line as input and N occurrences of repeated stuff with parenthesis.

Answer 4

This might work for you (GNU sed):
sed ':a;s/\(\(([0-9]\+) *\).*\)\2/\1/g;ta' file

Answer 5

awk -F"(" '{for(i in a)delete a[i];for(i=2;i<=NF;i++){if($i in a){$i="";}else{a[$i];$i="("$i}}print $0}' your_file
Tested below:
input:
> cat temp
Abdc 1234 1234 (5678) (5678) (9012) (9012) (3456)
1234 1234 (1234) (5678) (9012) (1234) (3456)
(5678) (6467) (6467) (9012) (5678)
Now the execution:
> awk -F"(" '{for(i in a)delete a[i];for(i=2;i<=NF;i++){if($i in a){$i="";}else{a[$i];$i="("$i}}print $0}' temp
Abdc 1234 1234 (5678) (9012) (3456)
1234 1234 (1234) (5678) (9012) (3456)
(5678) (6467) (9012) (5678)
>