Print Values In Array - perl

I can't seem to figure out how to print just some of the return values such as: title, url, content
#!/usr/bin/perl
print "Content-type: text/html\n\n";
use REST::Google;
# set service to use
REST::Google->service('http://ajax.googleapis.com/ajax/services/search/web');
# provide a valid http referer
REST::Google->http_referer('http://www.example.com');
my $res = REST::Google->new(
q => 'ice cream',
);
die "response status failure" if $res->responseStatus != 200;
my $data = $res->responseData;
use Data::Dumper;
print Dumper( $data );
my #results = $data->results;
# CANT MAKE THIS WORK
foreach my $r (#result) {
print "\n";
print $r->title;
print $r->url;
print $r->content;
}

Try:
foreach my $r (#results) {
note the "s" -- if you put at the top of your script:
use strict;
use warnings;
you will catch these things

#!/usr/bin/perl
use strict;
print "content-type: text/html\n\n";
use REST::Google;
# set service to use
REST::Google->service(
'http://ajax.googleapis.com/ajax/services/search/web' );
# provide a valid http referer
REST::Google->http_referer( 'http://www.example.com' );
my $res = REST::Google->new( q => 'ice cream', );
die "response status failure" if $res->responseStatus != 200;
my $data = $res->responseData;
my #results = #{ $data->{results} };
foreach my $r ( #results ) {
print "\n";
print $r->{title};
print $r->{url};
print $r->{content};
}
A couple of problems here:
1) $data is not an object, so you can't treat it like an object.
$data->results would be the correct syntax if you're calling a method on an object. In this case $data is just a regular HASHREF, so the syntax is:
$data->{results}
2) $data->{results} is an ARRAYREF and not an ARRAY. So, you need to de-reference it in order to get at the values.
Now, my #results = $data->{results} becomes:
my #results = #{ $data->{results} };
#{ ARRAYREF } is how you dereference the array.
3) As you're iterating over #results, you're once again using the object syntax. However, the values of #results are also just plain HASHREFs. So, $r->title becomes:
$r->{title}
Using a tool like Data::Dumper to inspect return values can be key in sorting this kind of thing out. You may also want to look at Data::Printer, which is even sexier than Data::Dumper

Related

I get error 'Can't use string as a HASH ref while "strict refs"....' when i use my perl code

im using this perl code to transform JSON into other form with some regular expressions:
use strict;
use warnings;
use feature 'say';
use JSON;
use utf8;
my %IDs = ( 'User awx01 logged in.' => 1001 );
my %levels = ( INFO => 4 );
my $data = <DATA>;
my $json = data2json($data);
my $record = decode_json($json);
say rec2msg($record);
sub data2json {
my $json = shift;
$json =~ s/[""]/"/g;
$json =~ s/\\//g;
$json =~ s/"(\{.*?\})"/$1/;
return $json;
}
sub rec2msg {
my $r = shift;
$r->{Message}{message} =~ /(\w+) (\w+) (.+)/;
my($user,$msg) = ($2,"$1 $3");
my $ID = $IDs{$r->{Message}{message}};
my $level = $levels{$r->{Message}{level}};
my $out = "$r->{Message}{'#timestamp'} host CEF:0|OpenSource|AWX|7.0.0|$ID|$msg|$level|src=127.0.0.1 dst=$r->{MessageSourceAddress} duser=$user";
return $out;
}
__DATA__
{"MessageSourceAddress":"192.168.81.20","EventReceivedTime":"2020-02-06 11:55:14","SourceModuleName":"udp","SourceModuleType":"im_udp","SyslogFacilityValue":1,"SyslogFacility":"USER","SyslogSeverityValue":5,"SyslogSeverity":"NOTICE","SeverityValue":2,"Severity":"INFO","EventTime":"2020-02-06 11:55:14","Hostname":"192.168.81.20","Message":"{\"#timestamp\": \"2020-02-06T08:55:52.907Z\", \"message\": \"User awx01 logged in.\", \"host\": \"awxweb\", \"level\": \"INFO\", \"logger_name\": \"awx.api.generics\", \"stack_info\": null, \"type\": \"other\", \"cluster_host_id\": \"awx-contr-01\", \"tower_uuid\": \"333b4131-495f-4460-8e4b-890241a9d73d\"}"}
But im getting this error:
2020-03-31 20:48:50 ERROR perl subroutine rec2msg failed with an error: 'Can't use string ("140511667030448") as a HASH ref while "strict refs" in use at /usr/libexec/nxlog/modules/extension/perl/event1.pl line 21.;'
What im doing wrong? How could i solve it?
You have JSON embedded in JSON, so you need to decode it twice. This often happens when you have one service passing through the response for another service.
Your data2json wasn't decoding that second level, so the value for the Message name was still a string. Since that value wasn't a hash reference, you get the error you reported.
You don't want to use a bunch of substitutions on the entire thing because you can inadvertently change things you shouldn't be messing with. Decode the top level just as you did, but then do the same thing for the Message value:
# read in all the data, even though it looks like a single line. Maybe it won't be later.
my $data = do { local $/; <DATA> };
# decode the first layer
my $decoded = decode_json( $data );
# decode the Message value:
$decoded->{Message} = decode_json( $decoded->{Message} );
Now, when you call rec2msg it should work out.
Note that this has the opposite problem to reverse it. You can't merely encode the entire thing to JSON again. The value for Message still needs to be a string, so you have to encode that first if you want to send it somewhere else. If you are doing that, you probably want to work on a copy. I use dclone to make a deep copy so whatever I do to $encoded does not show up in $decoded:
# make a deep copy so nested references aren't shared
use Storable qw(dclone);
my $encoded = dclone( $decoded );
$encoded->{Message} = encode_json( $encoded->{Message} );
my $new_data = encode_json( $encoded );
Then $new_data will have the same escaping as the original input.
Here it is altogether:
use strict;
use warnings;
use feature 'say';
use JSON;
use utf8;
my %IDs = ( 'User awx01 logged in.' => 1001 );
my %levels = ( INFO => 4 );
# read in all the data, even though it looks
my $data = do { local $/; <DATA> };
my $decoded = decode_json( $data );
$decoded->{Message} = decode_json( $decoded->{Message} );
say rec2msg($decoded);
sub rec2msg {
my $r = shift;
$r->{Message}{message} =~ /(\w+) (\w+) (.+)/;
my($user,$msg) = ($2,"$1 $3");
my $ID = $IDs{$r->{Message}{message}};
my $level = $levels{$r->{Message}{level}};
my $out = "$r->{Message}{'#timestamp'} host CEF:0|OpenSource|AWX|7.0.0|$ID|$msg|$level|src=127.0.0.1 dst=$r->{MessageSourceAddress} duser=$user";
return $out;
}

"Use of uninitialized value $_" warning with a Mojo::UserAgent non-blocking request

I am trying to make a non-blocking request with
Mojo::UserAgent
but when I run the code below I get
Use of uninitialized value $_ in concatenation (.) or string
on the print line.
How can I access $_ inside the callback?
my $ua = Mojo::UserAgent->new();
my #ids = qw( id1 id2 id3 );
foreach ( #ids ) {
my $res = $ua->get('http://my_site/rest/id/'.$_.'.json' => sub {
my ($ua, $res) = #_;
print "$_ => " . $res->result->json('/net/id/desc'), "\n";
});
}
Mojo::IOLoop->start unless Mojo::IOLoop->is_running;
$_ is a special kind of variable where the value depends on the context. Inside the foreach (#ip) context it is set as an alias of specific item in the #ip array. But, the callback for $ua->get(...) gets not executed within the foreach (#ip) context and thus $_ no longer is an alias into the #ip array.
Instead of using this special variable you need to use a normal variable scoped inside the foreach (#ip) loop, so that it can be bound to the subroutine (see also What's a closure in perlfaq7):
foreach (#ip) {
my $THIS_IS_A_NORMAL_VARIABLE = $_;
my $res= $ua->get( ... => sub {
my ($ua, $res) = #_;
print "$THIS_IS_A_NORMAL_VARIABLE =>" . $res->result->json('/net/id/desc'),"\n";
});
}

Getting Absolute URLs with module creating object outside loop

I have a doubt I've been trying to solve myself using CPAN modules documentation, but I'm a bit new and I'm confused with some terminology and sections within the different modules.
I'm trying to create the object in the code below, and get the absolute URL for relative links extracted from a website.
#!/usr/bin/perl
use strict;
use warnings;
use LWP::UserAgent;
use Digest::MD5 qw(md5_hex);
use URI;
my $url = $ARGV[0];
if ($url !~ m{^https?://[^\W]+-?\.com/?}i) {
exit(0);
}
my $ua = LWP::UserAgent->new;
$ua->timeout( 10 );
my $response = $ua->get( $url );
my $content = $response->decoded_content();
my $links = URI->new($content);
my $abs = $links->abs('http:', $content);
my $abs_links = $links->abs($abs);
while ($content =~ m{<a[^>]\s*href\s*=\s*"?([^"\s>]+)}gis) {
$abs_links = $1;
print "$abs_links\n";
print "Digest for the above URL is " . md5_hex($abs_links) . "\n";
}
The problem is when I try to add that part outside the While loop (the 3-line block preceding the loop), it does not work, whereas if I add the same part in the While loop, it will work fine. This one just gets the relative URLs from a given website, but instead of printing "Http://..." it prints "//...".
The script that works fine for me is the following:
#!/usr/bin/perl
use strict;
use warnings;
use LWP::UserAgent;
use Digest::MD5 qw(md5_hex);
use URI::URL;
my $url = $ARGV[0]; ## Url passed in command
if ($url !~ m{^https?://[\w]+-?[\w]+\.com/?}i) {
exit(0); ## Program stops if not valid URL
}
my $ua = LWP::UserAgent->new;
$ua->timeout( 10 );
my $response = $ua->get( $url ); ## Get response, not content
my $content = $response->decoded_content(); ## Now let's get the content
while ($content =~ m{<a[^>]\s*href\s*=\s*"?([^"\s>]+)}gis) { ## All links
my $links = $1;
my $abs = new URI::URL "$links";
my $abs_url = $abs->abs('http:', $links);
print "$abs_url\n";
print "Digest for the above URL is " . md5_hex($abs_url) . "\n";
}
Any ideas? Much appreciated.
I don't understand your code. There are a few weird bits:
[^\W] is the same as \w
The regex allows an optional - before and an optional / after .com, i.e. http://bitwise.complement.biz matches but http://cool-beans.com doesn't.
URI->new($content) makes no sense: $content is random HTML, not a URI.
$links->abs('http:', $content) makes no sense: $content is simply ignored, and $links->abs('http:') tries to make $links an absolute URL relative to 'http:', but 'http:' is not a valid URL.
Here's what I think you're trying to do:
#!/usr/bin/perl
use strict;
use warnings;
use LWP::UserAgent;
use HTML::LinkExtor;
use Digest::MD5 qw(md5_hex);
#ARGV == 1 or die "Usage: $0 URL\n";
my $url = $ARGV[0];
my $ua = LWP::UserAgent->new(timeout => 10);
my $response = $ua->get($url);
$response->is_success or die "$0: " . $response->request->uri . ": " . $response->status_line . "\n";
my $content = $response->decoded_content;
my $base = $response->base;
my #links;
my $p = HTML::LinkExtor->new(
sub {
my ($tag, %attrs) = #_;
if ($tag eq 'a' && $attrs{href}) {
push #links, "$attrs{href}"; # stringify
}
},
$base,
);
$p->parse($content);
$p->eof;
for my $link (#links) {
print "$link\n";
print "Digest for the above URL is " . md5_hex($link) . "\n";
}
I don't try to validate the URL passed in $ARGV[0]. Leave it to LWP::UserAgent. (If you don't like this, just add the check back in.)
I make sure $ua->get($url) was successful before proceeding.
I get the base URL for absolutifying relative links from $response->base.
I use HTML::LinkExtor for parsing the content, extracting links, and making them absolute.
I think your biggest mistake is trying to parse links out of HTML using a regular expression. You would be far better advised to use a CPAN module for this. I'd recommend WWW::Mechanize, which would make your code look something like this:
#!/usr/bin/perl
use strict;
use warnings;
use feature 'say';
use WWW::Mechanize;
use Digest::MD5 qw(md5_hex);
use URI;
my $url = $ARGV[0];
if ($url !~ m{^https?://[^\W]+-?\.com/?}i) {
exit(0);
}
my $ua = WWW::Mechanize->new;
$ua->timeout( 10 );
$ua->get( $url );
foreach ($ua->links) {
say $_->url;
say "Digest for the above URL is " . md5_hex($_->url) . "\n";
}
That looks a lot simpler to me.

How to use refernce concept and access element of subroutine argument using Perl?

I am writing a code for calling a subroutine which has 4 argument(3 hashes and one file handler).i want to know how to access them in subroutine.My code is as below.
#print OUTFILE "Content of TPC file:.\n";
my $DATA_INFO = $ARGV[0];
my $OUT_DIR = $ARGV[1];
my $log= "$OUT_DIR/log1";
open(LOG1,">$log");
require "$DATA_INFO";
my $SCRIPT_DIR = $ENV{"SCRIPT_DIR"} ;
require "$SCRIPT_DIR/cmp_fault.pl";
require "$SCRIPT_DIR/pattern_mismatch.pl";
require "$SCRIPT_DIR/scan_count.pl";
print "\nComparing data:\n\n" ;
pattern_mismatch("\%data","\%VAR1","\%status",*LOG1);
cmp_fault("\%data","\%VAR1","\%status",*LOG1);
scan_count("\%data","\%status",*LOG1);
print "\n Comparison done:\n";
foreach $pattern (keys %status) {
print "pattern";
foreach $attr (keys %{$status{$pattern}}) {
print ",$attr";
}
print "\n";
last;
}
#Print Data
foreach $pattern (keys %status) {
print "$pattern";
foreach $attr (keys %{$status{$pattern}}) {
print ",$status{$pattern}{$attr}";
}
print "\n";
Sub routine cmp_fault is here:
sub cmp_fault {
use strict;
use warning;
$data_ref= $_[0];;
$VAR1_ref= $_[1];
$status_ref = $_[2];
$log1_ref=$_[3];
# print LOG1"For TPC : First find the pattern and then its fault type\n";
for $pat ( keys %$data_ref ) {
print "fgh:\n$pat,";
for $key (keys %{$data_ref{$pat}}) {
if($key=~/fault/){
print LOG1 "$key:$data_ref{$pat}{$key},\n";
}
}
}
# print LOG1 "\nFor XLS : First find the pattern and then its pattern type\n";
for $sheet (keys %$VAR1_ref){
if ("$sheet" eq "ATPG") {
for $row (1 .. $#{$VAR1_ref->{$sheet}}) {
$patname = $VAR1_ref->{'ATPG'}[$row]{'Pattern'} ;
next if ("$patname" eq "") ;
$faultXls = $VAR1_ref->{'ATPG'}[$row]{'FaultType'} ;
# print LOG1 " $patname==>$faultXls \n";
if (defined $data{$patname}{'fault'}) {
$faultTpc = $data{$patname}{'fault'} ;
# print LOG1 "\n $patname :XLS: $faultXls :TPC: $faultTpc\n";
if("$faultXls" eq "$faultTpc") {
print LOG1 "PASS: FaultType Matched $patname :XLS: $faultXls :TPC: $faultTpc\n\n\n";
print "PASS: FaultType Matched $patname :XLS: $faultXls :TPC: $faultTpc\n\n";
$status_ref->{$patname}{'FaultType'} = PASS;
}
else {
print LOG1 "FAIL: FaultType Doesn't Match\n\n";
$status_ref->{$patname}{'FaultType'} = Fail;
}
}
}
}
}
}
return 1;
When passing parameters into an array, you can only ever pass a single list of parameters.
For scalars, this isn't a problem. If all you're acting on is a single array, this also isn't a problem.
If you need to send scalars and an array or hash, then the easy way is to 'extract' the scalar parameters first, and then treat 'everything else' as the list.
use strict;
use warnings;
sub scalars_and_array {
my ( $first, $second, #rest ) = #_;
print "$first, $second, ", join( ":", #rest ), "\n";
}
scalars_and_array( "1", "2", "3", 4, 5, 6 );
But it should be noted that by doing so - you're passing values. You can do this with hashes too.
To pass data structure references, it's as you note - pass by reference, then dereference. It's useful to be aware though, that -> becomes useful, because it's accessing a hash and dereferencing it.
use strict;
use warnings;
use Data::Dumper;
sub pass_hash {
my ( $hashref ) = #_;
print $hashref,"\n";
print $hashref -> {"one"},"\n";
print $hashref -> {"fish"} -> {"haddock"};
}
my %test_hash = ( "one" => 2,
"three" => 4,
"fish" => { "haddock" => "plaice" }, );
pass_hash ( \%test_hash );
print "\n";
print Dumper \%test_hash;
The core of your problem here though, is that you haven't turned on strict and warnings which would tell you that:
for $pat ( keys %data_ref ) {
is wrong - there is no hash called data_ref there's only a scalar (which holds a hash reference) called $data_ref.
You need %$data_ref here.
And here:
for $key ( keys %{ $data{$pat} } ) {
You also have no $data - your code says $data_ref. (You might have %data in scope, but that's a really bad idea to mess around with within a sub).
There's a bunch of other errors - which would also be revealed by strict and warnings. That's a very basic debugging step, and you will generally get a much better response from Stack Overflow if you do this before asking for assistance. So please - do that, tidy up your code and remove errors/warnings. If you are still having problems after that, then by all means make a post outlining where and what problem you're having.

Unable to pass a hash and a string to a function, together in perl!

I am basically trying to pass a string and a hash to a subroutine in perl.
sub coru_excel {
my(%pushed_hash, $filename) = #_;
print Dumper(%pushed_hash);
}
But it seems data is getting mixed up. The dumped data also includes the $filename. here is the output.
...................
$VAR7 = 'Address';
$VAR8 = [
'223 VIA DE
................
];
$VAR9 = 'data__a.xls' <----- $filename
$VAR10 = undef;
$VAR11 = 'DBA';
$VAR12 = [
'J & L iNC
..................
];
Here is how I called the subroutine.
coru_excel(%hash, "data_".$first."_".$last.".xls");
Arguments are passed to subroutines as one undifferentiated list.
One solution is to reverse the order of the arguments so that the scalar is first.
sub coru_excel {
my($filename, %pushed_hash) = #_;
}
coru_excel("FILE_NAME", %hash);
Another approach is to pass the hash by reference:
sub coru_excel {
my($pushed_hash_ref, $filename) = #_;
}
coru_excel(\%hash, "FILE_NAME");
You could pass the hash as a reference:
sub coru_excel {
my($pushed_hashref, $filename) = #_;
print Dumper(%$pushed_hashref);
}
coru_excel(\%my_hash, $file);
Or you could give special treatment to the final argument before you initialize the hash:
sub coru_excel {
my $filename = pop #_;
my(%pushed_hash) = #_;
print Dumper(%pushed_hash);
}
You have to pass the hash as a reference:
coru_excel(\%hash, "data_".$first."_".$last.".xls");
You use it like this:
sub coru_excel {
my($pushed_hash_ref, $filename) = #_;
my %pushed_hash = %{$pushed_hash_ref};
print Dumper(%pushed_hash); # better: \%pushed_hash or $pushed_hash_ref
}
See perlreftut for a tutorial on references and perlref for further information.
Dumper also produces better usable information when you pass a hash (or array) reference.
See also the related Perl FAQ. From the command line:
perldoc -q pass
or
perldoc -q hash
Refer to perlfaq7: How can I pass/return a {Function, FileHandle, Array, Hash, Method, Regex}?
A small program demonstrating how to do this using reference notation when passing the hash and shift in the subroutine to pull out the parameters.
#!/usr/bin/perl -w
use strict;
sub coru_excel(%$);
my %main_hash = ('key1' => 'val1', 'key2' => 'val2');
my $first = "ABC";
my $last = "xyz";
coru_excel(\%main_hash, "data_" . $first . "_" . $last . ".xls");
exit;
sub coru_excel(%$)
{
my %passed_hash = %{(shift)};
my $passed_string = shift;
print "%passed_hash:\n";
for my $k (keys %passed_hash) {
print " $k => $passed_hash{$k}\n";
}
print "\$passed_string = $passed_string\n";
return;
}