Greping an array obtained through NET:TELNET - perl

I'm writing a Munin-Pluging and I like to capture the screen output from a telnet session.
The output of such a session looks as follows:
...
0x00017 0x41b3f340 BPING 0 0 0 0 198 132 330
0x00018 0x41b47340 CHKFAILED 0 0 0 0 198 132 330
0x00026 0x41b4f340 CIP 0 0 0 0 370 264 634
0x0001e 0x41b57340 CONTROL 0 1 0 0 3876 2178 6054
0x01014 0x41b5f340 UNETSRVR 0 0 0 1 296 198 494
0x00037 0x41b67340 ---- 0 0 0 0 198 132 330
0x00000 0x43b67450 ---- 0 0 0 0 0 0 0
0x00000 0x4bb67450 ---- 0 0 0 0 5084 4224 9308
0x00000 0x49367450 ---- 0 0 0 0 14742 4158 18900
-------------------------------------------------------------------------------------------
SUMMARY : 2 40 5 7 4898229 2728176 7626405
This script extract the screen content into an array (#lines).
#!/usr/bin/perl
use Net::Telnet ();
use strict;
use warnings;
my $t = new Net::Telnet (Timeout => 10);
$t->port(777);
$t->open("192.168.0.1");
$t->buffer_empty;
my #lines = $t->waitfor(match =>"m/.* SUMMARY : .* \n/");
my #gagu = grep { "$_" =~ /^.*BPING.*\n/ } #lines;
print #gagu;
Of what type is the array #lines?
Why do I always get the whole
content from grep and not a filtered line?
Is the array i got from net:telnet different from other arrays?
Yes, I'm new to Perl.

I am not familiar with this module and what it does, but I assume it gives you some sort of return value similar to what you have stated.
If you are getting all the lines in your #gagu array, that can be either that your data in the #lines array consists of just one line, or that the grep fails.
For example, #lines may contain the string:
"foo bar baz\nfoo1 bar1 baz1\n";
and not, as you expect
"foo bar baz\n";
"foo1 bar1 baz1\n";
Your grep statement probably works as expected, though you might want to consider:
Not quoting $_, since that serves no purpose.
Not using $_ at all, since that is the default variable it is not needed (except for clarity) to use it.
Not using anchors ^ and \n, because they are redundant.
For example, ^.* matches any string, anywhere. Using it to simply match a string is redundant. Ending the regex with .*\n is redundant, because all it says is "match any character except newline until we find a newline". Assuming you have newlines, it does nothing. Assuming you don't, it gives you a false negative. All you need for this match is /BPING/. So here's what your code might look like:
use Data::Dumper;
my #lines = $t->waitfor(match =>"m/ SUMMARY :/");
my #gagu = grep /BPING/, #lines;
print Dumper \#gagu;
If you want to see whitespace printed out visibly, you can use the $Data::Dumper::Useqq variable:
$Data::Dumper::Useqq = 1;
print Dumper \#gagu;
Printing variables is a very good debugging tool.

Related

Combining two different multi-row strings into one using Perl

I'm having some issues trying to combine two multi-row strings into one after performing regular expression manipulations on those strings. As an example, I start with data in this form:
TMS: xxxxxxx11110000
TDI: xxxxxxx00001111
TMS: xxxx00001111
TDI: xxxx11110000
To get it in the form I need, I search the file for the key word "TMS: ", extract just the data, use regular expressions to remove the "x's", reverse the data, and then place each bit on its own line and store it in a string. Resultant string would look like this:
0
0
0
0
1
1
1
1
I then search the file for "TDI: " and repeat that same process. The last step would be to concatenate the first string with the second string to get the following output (given the example above):
01
01
01
01
10
10
10
10
10
10
10
10
01
01
01
01
However, when I concatenate the two strings, what I'm getting as an output right now is
0
0
0
0
1
1
1
1
1
1
1
1
0
0
0
0
1
1
1
1
0
0
0
0
0
0
0
0
1
1
1
1
Is there a way to get the result I'm looking for without changing too much about my process? I've tried using the split command, chomp command, etc. without any luck.
Would be good to have a minimal example to see how you're approaching this problem. Additionally, there are a lot of things about your input file that are not clear. For example, are TMS and TDI always paired in the file, or do you have to check for that? Will you always take the next TDI instance to pair with the preceeding TMS event, or can they be more disjointed? Does TMS always preceed TDI or can they be reversed?
One simple way to do this assuming that the data look just like you've indicated in your example, might be to read each line and store the data in one array for the TMS string and one array for the TDI string. If both arrays are full, then we have a pair to output, so output the pair and clear the arrays for the next events. Otherwise, read the next line to get the TDI data:
#!/usr/bin/env perl
use strict;
use warnings;
my (#first, #second);
while (my $elem = <DATA>) {
($elem =~ /^TMS/)
? (#first = read_string($elem))
: (#second = read_string($elem));
if (#second) {
for my $index (0..$#first) {
print "$first[$index]$second[$index]\n";
}
print "\n";
#first = #second = ();
}
}
sub read_string {
my $string = shift;
my #bits = grep {/\d/} split('', $string);
return reverse(#bits);
}
__DATA__
TMS: xxxxxxx11110000
TDI: xxxxxxx00001111
TMS: xxxx00001111
TDI: xxxx11110000
Output from this would be:
01
01
01
01
10
10
10
10
10
10
10
10
01
01
01
01
What you want is a zip operation. Conveniently List::MoreUtils provides one for you.
#x = qw/a b c d/;
#y = qw/1 2 3 4/;
# returns [a, 1], [b, 2], [c, 3], [d, 4]
#z = zip6 #x, #y;
To get the input for zip either put your resultants into an array in the first place, or split your input string.
hobbs answer from Code Golf: Lasers was solving quite a different problem, but part of the solution was about how to "rotate" a multi-line string, and it could be useful here.
First, don't put each bit on its own line, just separate bits from different rows of input on different lines. Put the multi-line string into $_.
$_ = '0000111111110000
1111000000001111';
Now execute the following code:
$_ = do {
my $o;
$o .= "\n" while s/^./$o.=$&,""/meg;
$o };
(the substitution in hobbs's algorithm started with s/.$/.../. By using s/^./.../, it becomes an algorithm for transposition rather than for rotation)
Input:
$_ = '0000111111110000
1111000000001111';
Output:
01
01
01
01
10
10
10
10
10
10
10
10
01
01
01
01
This algorithm easily generalizes to any number of rows and columns in the input.
Input:
$_='ABCDE
12345
FGHIJ
67890';
Output:
A1F6
B2G7
C3H8
D4I9
E5J0

Extract reads from a BAM/SAM file of a designated length

I am a bit of new to Perl and wish to use it in order to extract reads of a specific length from my BAM (alignment) file.
The BAM file contains reads, whose length is from 19 to 29 nt.
Here is an example of first 2 reads:
YT:Z:UUA00182:193:HG2NLDMXX:1:1101:29884:1078 0 3R 6234066 42 22M * 0 0 TCACTGGGCTTTGTTTATCTCA FF:FFFF,FFFFFFFF:FFFFF AS:i:0 XN:i:0 XM:i:0 XO:i:0 XG:i:0 NM:i:0 MD:Z:22
YT:Z:UUA00182:193:HG2NLDMXX:1:1101:1777:1094 16 4 1313373 1 24M * 0 0 TCGCATTCTTATTGATTTTCCTTT FFFFFFF,FFFFFFFFFFFFFFFF AS:i:0 XS:i:0 XN:i:0 XM:i:0 XO:i:0 XG:i:0 NM:i:0 MD:Z:24
I want to extract only those, which are, let's say, 21 nt in length.
I try to do this with the following code:
my $string = <STDIN>;
$length = samtools view ./file.bam | head | perl -F'\t' -lane'length #F[10]';
if ($length == 21){
print($string)
}
However, the program does not give any result...
Could anyone please suggest the right way of doing this?
Your question is a bit confusing. Is the code snippet supposed to be a Perl script or a shell script that calls a Perl one-liner?
Assuming that you meant to write a Perl script into which you pipe the output of samtools view to:
#!/usr/bin/perl
use strict;
use warnings;
while (<STDIN>) {
my #fields = split("\t", $_);
# debugging, just to see what field is extracted...
print "'$fields[10]' ", length($fields[10]), "\n";
if (length($fields[10]) eq 21) {
print $_;
}
}
exit 0;
With your test data in dummy.txt I get:
# this would be "samtools view ./file.bam | head | perl dummy.pl" in your case?
$ cat dummy.txt | perl dummy.pl
'FF:FFFF,FFFFFFFF:FFFFF' 22
'FFFFFFF,FFFFFFFFFFFFFFFF' 24
Your test data doesn't contain a sample with length 21 though, so the if clause is never executed.
Note that the 10th field in your sample input is having either 22 or 24 in length. Also, the syntax that you use is wrong. Here is the Perl one-liner to match the field with length=22.
$ cat pkom.txt
YT:Z:UUA00182:193:HG2NLDMXX:1:1101:29884:1078 0 3R 6234066 42 22M * 0 0 TCACTGGGCTTTGTTTATCTCA FF:FFFF,FFFFFFFF:FFFFF AS:i:0 XN:i:0 XM:i:0 XO:i:0 XG:i:0 NM:i:0 MD:Z:22
YT:Z:UUA00182:193:HG2NLDMXX:1:1101:1777:1094 16 4 1313373 1 24M * 0 0 TCGCATTCTTATTGATTTTCCTTT FFFFFFF,FFFFFFFFFFFFFFFF AS:i:0 XS:i:0 XN:i:0 XM:i:0 XO:i:0 XG:i:0 NM:i:0 MD:Z:24
$ perl -lane ' print if length($F[9])==22 ' pkom.txt
YT:Z:UUA00182:193:HG2NLDMXX:1:1101:29884:1078 0 3R 6234066 42 22M * 0 0 TCACTGGGCTTTGTTTATCTCA FF:FFFF,FFFFFFFF:FFFFF AS:i:0 XN:i:0 XM:i:0 XO:i:0 XG:i:0 NM:i:0 MD:Z:22
$

k/Column number in Perl

So I'm sure this is somewhere on the site, but as always, I have looked high and low before asking a question.
In Bash, you can use certain flags on some commands (such as k[number] on sort) to grab a certain column from a text file. What is the method for doing this in Perl? For an example from my input file:
Jess 6 8 25000
Say that I want to run a statement
if (k2 =< 6)
{
print "foo";
}
Of course, k2 doesn't work in Perl. May someone show me (or link me to) how this is done?
You can try with the command line Perl as well
with the inputs
$ cat reubens.txt
0 5 0
0 10 0
0 15 0
0 20 0
0 1 0
0 10 0
$ perl -lane ' print "The second column is ", $F[1] < 10 ? "less than 10": $F[1]==10 ? "equal to 10" : "more than 10" ' reubens.txt
The second column is less than 10
The second column is equal to 10
The second column is more than 10
The second column is more than 10
The second column is less than 10
The second column is equal to 10
$
In case you want to do it for exactly one column and want to avoid maintaining the result of "split" in array shape. (Otherwise use split as mentioned in the comments.)
perl -ne"/(?:\w+\s+){1}(\w+\b)/;print $1.\"\n\""
Will print the column of word-like characters between space-like characters, identified by the number inside "{}", in this case "1"; counting columns starting with 0.
E.g. it prints "6" for the input example, using "1".
How:
Make a regular expression for a column followed by space,
(?:\w+\s+)
require it a number of times,
{1}
then grab a regular expression for a colum followed by anything not word-like (including end of line)
(\w+\b)
The desired column is found in the grabbed string
$1
I did this in a command line one liner which expects standard input, to be able to test it.
Please just adapt it into your script.
This will check the second column:
(split)[1]
Sample script:
use strict;
use warnings;
my $filename = 'input';
open(FILE, $filename) or die "Can not open $filename.";
print "\n";
while(<FILE>)
{
#The test
if ((split)[1] < 10)
{
print "The second column is less than ten\n";
}
elsif ((split)[1] > 10)
{
print "The second column is more than ten\n";
}
else
{
print "The second column is equal to ten\n";
}
}
Input:
#Input file
0 5 0
0 10 0
0 15 0
0 20 0
0 1 0
0 10 0
Output:
The second column is less than ten
The second column is equal to ten
The second column is more than ten
The second column is more than ten
The second column is less than ten
The second column is equal to ten

Perl: perl regex for extracting values from complex lines

Input log file:
Nservdrx_cycle 4 servdrx4_cycle
HCS_cellinfo_st[10] (type = (LTE { 2}),cell_param_id = (28)
freq_info = (10560),band_ind = (rsrp_rsrq{ -1}),Qoffset1 = (0)
Pcompensation = (0),Qrxlevmin = (-20),cell_id = (7),
agcreserved{3} = ({ 0, 0, 0 }))
channelisation_code1 16/5 { 4} channelisation_code1
sync_ul_info_st_ (availiable_sync_ul_code = (15),uppch_desired_power =
(20),power_ramping_step = (3),max_sync_ul_trans = (8),uppch_position_info =
(0))
trch_type PCH { 7} trch_type8
last_report 0 zeroth bit
I was trying to extract only integer for my above inputs but I am facing some
issue with if the string contain integer at the beginning and at the end
For ( e.g agcreserved{3},HCS_cellinfo_st[10],Qoffset1)
here I don't want to ignore {3},[10] and 1 but in my code it does.
since I was extracting only integer.
Here I have written simple regex for extracting only integer.
MY SIMPLE CODE:
use strict;
use warnings;
my $Ipfile = 'data.txt';
open my $FILE, "<", $Ipfile or die "Couldn't open input file: $!";
my #array;
while(<$FILE>)
{
while ($_ =~ m/( [+-]?\d+ )/xg)
{
push #array, ($1);
}
}
print "#array \n";
output what I am getting for above inputs:
4 4 10 2 28 10560 -1 1 0 0 -20 7 3 0 0 0 1 16 5 4 1 15 20 3 8 0 7 8 0
expected output:
4 2 28 10560 -1 0 0 -20 7 0 0 0 4 15 20 3 8 0 7 0
If some body can help me with explanation ?
You are catching every integer because your regex has no restrictions on which characters can (or can not) come before/after the integer. Remember that the /x modifier only serves to allow whitespace/comments inside your pattern for readability.
Without knowing a bit more about the possible structure of your output data, this modification achieves the desired output:
while ( $_ =~ m! [^[{/\w] ( [+-]?\d+ ) [^/\w]!xg ) {
push #array, ($1);
}
I have added rules before and after the integer to exclude certain characters. So now, we will only capture if:
There is no [, {, /, or word character immediately before the number
There is no / or word character immediately after the number
If your data could have 2-digit numbers in the { N} blocks (e.g. PCH {12}) then this will not capture those and the pattern will need to become much more complex. This solution is therefore quite brittle, without knowing more of the rules about your target data.

How do I get the correct rgb value for a pixel with GD?

I have a 4x1 pixel png where the left most pixel is white (#ffffff), the second pixel is red (#ff0000), the third pixel is green (#00ff00) and the right most pixel is blue (#0000ff), almost invisible here: .
With the following perl script, I tried to read the rgb values for each pixel:
use warnings;
use strict;
use GD;
my $image = new GD::Image('white-red-green-blue.png') or die;
for (my $x=0; $x<4; $x++) {
my $index = $image->getPixel($x, 0);
my ($r,$g,$b) = $image->rgb($index);
printf "%3d %3d %3d\n", $r, $g, $b;
}
Much to my surprise, it printed
252 254 252
252 2 4
4 254 4
4 2 252
whereas I expected
255 255 255
255 0 0
0 255 0
0 0 255
Why does the script report wrong rgb values and how can I teach it to report the correct ones?
Edit As per mpapec's question, the output of base64 white-red-green-blue.png is
iVBORw0KGgoAAAANSUhEUgAAAAQAAAABCAIAAAB2XpiaAAAAAXNSR0IArs4c6QAAAARnQU1B
AACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAASSURBVBhXY/gPBAwMDEDM8B8AL90F
+8V5iZQAAAAASUVORK5CYII=
Edit II As per dgw's suggestion, I also tried my $image = newFromPng GD::Image('white-red-green-blue.png') or die; but with the same result.
Update: I have tried the same thing with Image::Magick instead of GD:
use warnings;
use strict;
use Image::Magick;
my $image = Image::Magick->new or die;
my $read = $image -> Read('white-red-green-blue.png');
for (my $x=0; $x<4; $x++) {
my #pixels = $image->GetPixels(
width => 1,
height => 1,
x => $x,
y => 0,
map =>'RGB',
#normalize => 1
);
printf "%3d %3d %3d\n", $pixels[0] / 256, $pixels[1] / 256, $pixels[2] / 256;
}
and, somewhat unsurpringly, it prints the expected
255 255 255
255 0 0
0 255 0
0 0 255
Updated
Ok, it works fine with your image if you do this:
GD::Image->trueColor(1);
BEFORE starting anything with GD. I think it is because one image is palettized and the other is not. See here:
Original Answer
It works fine on my iMac. I generated the image with ImageMagick like this:
convert -size 1x1! xc:rgb\(255,255,255\) xc:rgb\(255,0,0\) xc:rgb\(0,255,0\) xc:rgb\(0,0,255\) +append wrgb.png
./go.pl
255 255 255
255 0 0
0 255 0
0 0 255
I suspect your image is not being generated correctly.
Your source image is not pure RGB as you've expected, but you can generate one if you want that,
use warnings;
use strict;
use GD;
my $image = new GD::Image('white-red-green-blue.png') or die;
my #c = (
[255,255,255],
[255,0,0],
[0,255,0],
[0,0,255],
);
for my $x (0 .. 3) {
##
$image->setPixel(
$x,0, $image->colorAllocate(#{ $c[$x] })
);
##
my $index = $image->getPixel($x, 0);
my ($r,$g,$b) = $image->rgb($index);
printf "%3d %3d %3d\n", $r, $g, $b;
}
output
255 255 255
255 0 0
0 255 0
0 0 255
echo 'iVBORw0KGgoAAAANSUhEUgAAAAQAAAABBAMAAAALEhL+AAAAGFBMVEUEAvwE/gT8AgT8/vz/////
AAAA/wAAAP/i4quSAAAAC0lEQVQImWNwTQcAAPQArQCZcI0AAAAASUVORK5CYII=' |
base64 -d > new.png