I'm trying to decode a json then only print specific values from it. I want to get the front and back from the json and save the file. Code is not getting past the decode_base64 and says "Not a HASH reference"
use strict;
use warnings;
use JSON;
use Data::Dumper
use MIME:Base64
my $imageDir = "/usr/documents/images/";
my $testJson = '[{ "front":"", "back":"", "shortID":"", "longID":"" } ]';
my $Json = decode_json($testJson);
print Dumper $Json;
my $fImage = decode_base64($Json->{front});
my $fImagefile = saveFile('Front.jpg',$fImage);
my $bImage = decode_base64($Json->{back});
my $bImagefile = saveFile('Back.jpg',$bImage);
return ($fImagefile, $bImagefile);
sub saveFile{
my ($file $image) = #_;
my imageFile = $imageDir.$file;
open (IMGFILE, '>'.$imageFile) or die;
print IMGFILE $image;
close IMGFILE;
return $imageFile;
}
#ikegami already said this in a comment but I'll expand on it.
You have a JSON fragment that's an array, and the element in the array is a hash. T. You have one more level to drill down into to get what you want. The perldsc (Perl Data Structures Cookbook) has lots of hints and tricks for multi-level data structures:
my $testJson = '[{ "front":"", "back":"", "shortID":"", "longID":"" } ]';
my $Json = decode_json($testJson);
print Dumper $Json;
In your Dumper output you should see something like this. It's an anonymous array with a hashref as its first and only element:
$VAR1 = [
{
'longID' => '',
'shortID' => '',
'front' => '',
'back' => ''
}
];
If you want to process only that element, you can get the first element of the array reference. This is going to be the hash reference that you want to process:
my $element = $Json->[0];
my $fImage = decode_base64($element->{front});
...
But, you might want to process all the elements. Simply wrap a foreach around it:
foreach my $element ( #$Json ) {
my $fImage = decode_base64($element->{front});
...
}
Even though that array dereference is simple, the postfix dereferencing introduced in v5.26 is quite nice and what I use most often:
foreach my $element ( $Json->#* ) {
my $fImage = decode_base64($element->{front});
...
}
Related
I am new to perl and i had written below snippet of code to get the JSON objects from the data. But the input data has both hashes and arrays, So how do i get those values for "id" which is an array ?
use strict;
use warnings;
use Data::Dumper;
use JSON
my $data = '{"total":325,"id": [78,234,737,1253,1459,1733,2166,2653,2855,3133,3414,3538,3729,3905,3991,4110,4160,4536,4692,4701]}';
print Dumper($data);
my #hash_ref = from_json($data);
foreach my $hash_scalar (#hash_ref) {
foreach (keys %{$hash_scalar}) {
print "$_ => ${$hash_scalar}{$_}\n";
}
}
Output Getting
#
$VAR1 = '{"total":325,"id":
[78,234,737,1253,1459,1733,2166,2653,2855,3133,3414,3538,3729,3905,3991,4110,4160,4536,4692,4701]}';
id => ARRAY(0x2afee4c)
total => 325
The id is a key in the hash reference you get back, and you want to grab its value, which is an array reference. You can either keep that as a reference or grab its elements right away:
#!perl
use v5.24;
use JSON qw(from_json);
my $data = '{"total":325,"id": [78,234,737]}';
my $json = from_json($data);
# get the hash key to get the hash value, which is an array reference
my $ids_array_ref = $json->{id};
# OR, use the postfix dereference to get the ids as a normal list
my #ids_array = $json->{id}->#*;
# OR, use the older circumfix notation (before Perl v5.24). These are all
# the same:
my #ids_array = #{ $json->{id} };
my #ids_array = #{ $ids_array_ref };
my #ids_array = #$ids_array_ref;
For example:
#!perl
use v5.24;
use JSON qw(from_json);
my $data = '{"total":325,"id": [78,234,737]}';
my $json = from_json($data);
foreach my $id ( $json->{id}->#* ) {
say "Got id $id";
}
This outputs:
Got id 78
Got id 234
Got id 737
But, you have to handle the other hash value types too. To decide how to print something, you have to check if it's a reference and what sort of reference it is (I ignore the other sorts here):
#!perl
use v5.24;
use JSON qw(from_json);
my $data = '{"total":325,"id": [78,234,737]}';
my $json = from_json($data);
foreach my $key ( keys $json->%* ) {
print "$key: ";
if( ref $json->{$key} eq 'ARRAY' ) {
say join ' ', $json->{$key}->#*;
}
elsif( ref $json->{$key} ) { # all other reference types
warn "I don't handle this yet"
}
else {
say $json->{$key};
}
}
But, you might have deeper levels of nesting, so you'd need to think about that too if you want to output everything. If I know what keys I want, I don't try to go through everything in a general way. I go directly to want I want:
#!perl
use v5.24;
use JSON qw(from_json);
my $data = '{"total":325,"id": [78,234,737]}';
my $json = from_json($data);
say "id: ", join ' ', $json->{id}->#*;
say "total: ", $json->{total};
Your original code was doubly complicated. The result of from_json is the reference that represents the JSON data structure. That's a JSON object, which is what Perl calls a hash, so you get a hash reference back.
my $hash_ref = from_json( $data );
Your example almost worked because you got an array of one element, which is that hash reference. The other foreach then iterates over that one element. Get rid of that outer array and you get rid of the outer loop.
We cover references and data structures in Intermediate Perl, although the perldsc is good too. We have a long examples of deeply-nested and recursive data structures.
If you're new, you may want to start with Learning Perl, though.
My Perl is getting rusty, but I think something like this should do the trick. You need to de-reference your array reference with another #{...} like #{${$hash_scalar}{$_}}.
use strict;
use warnings;
use Data::Dumper;
use JSON
my $data = '{"total":325,"id": [78,234,737,1253,1459,1733,2166,2653,2855,3133,3414,3538,3729,3905,3991,4110,4160,4536,4692,4701]}';
print Dumper($data);
my #hash_ref = from_json($data);
for my $hash_scalar (#hash_ref) {
for my $key (keys(%{$hash_scalar})) {
print("$key => #{${$hash_scalar}{$_}}\n");
}
}
Perhaps following code will cover your question in more complete extent.
NOTE: I've added a hash in data block as bonus
use strict;
use warnings;
use feature 'say';
use JSON;
use Data::Dumper;
my $debug = 0;
my $data = '{ "total":325,
"id": [78,234,737,1253,1459,1733,2166,2653,2855,3133,3414,3538,3729,3905,3991,4110,4160,4536,4692,4701],
"person": {
"first": "John",
"last": "Smith",
"age": 27,
"occupation": "accountant",
"wife": "Maria Smith",
"son": "Alex Smith",
"daughter": "Samantha Smith",
"dog": "Sparky",
"hobby": "photography"
}
}';
say Dumper($data) if $debug;
my $json = from_json($data);
say Dumper($json) if $debug;
while( my($k,$v) = each %{$json} ) {
say "Key: $k";
if( ref $v eq 'ARRAY' ) {
say "\tValue is ARRAY";
for ( #{$v} ) { say "\t\t[$_]" }
} elsif( ref $v eq 'HASH' ) {
say "\tValue is HASH";
while( my($hk,$hv) = each %{$v} ) {
say "\t\t$hk => $hv";
}
} else {
say "\tValue: $v";
}
}
say '-' x 40;
say 'Element with index 5 is ' . $json->{id}[5];
say 'Name of the son is ' . $json->{person}{son};
say 'Name of the daughter is ' . $json->{person}{daughter};
Output
Key: person
Value is HASH
son => Alex Smith
occupation => accountant
daughter => Samantha Smith
age => 27
wife => Maria Smith
dog => Sparky
last => Smith
first => John
hobby => photography
Key: id
Value is ARRAY
[78]
[234]
[737]
[1253]
[1459]
[1733]
[2166]
[2653]
[2855]
[3133]
[3414]
[3538]
[3729]
[3905]
[3991]
[4110]
[4160]
[4536]
[4692]
[4701]
Key: total
Value: 325
----------------------------------------
Element with index 5 is 1733
Name of the son is Alex Smith
Name of the daughter is Samantha Smith
Consider this snippet:
use Data::Dumper;
#targetDirsToScan = ("./");
use IO::All;
$io = io(#targetDirsToScan); # Create new directory object
#contents = $io->all(0); # Get all contents of dir
for my $contentry ( #contents ) {
print Dumper($contentry) ."\n";
}
This prints something like:
$VAR1 = bless( \*Symbol::GEN298, 'IO::All::File' );
$VAR1 = bless( \*Symbol::GEN307, 'IO::All::Dir' );
$VAR1 = bless( \*Symbol::GEN20, 'IO::All::File' );
...
I expected I would get the all the fields of the respective objects dumped, instead; at first, I thought this was a reference, so I thought the fields would be printed if I dereference the variable - but I realized I don't really know how to dereference it.
So - how can I print out all the fields and contents of the #contents, using the same kind of for my ... loop?
You can do this:
use Data::Dumper;
use IO::All;
$io = io('/tmp');
for my $file ( $io->all(0) ) {
print Dumper \%{*$file};
}
But you should seriously consider whether doing this is a good idea. One of the core tenets of object-oriented programming is encapsulation. You should not care about the guts of a blessed object - you should interact with it only via the methods it provides.
use 5.010;
use strict;
use warnings;
use JSON::XS;
use YAML::XS;
my %data = ();
my $content = <<HERE;
{
"name":"BLAHBLAH","contact":{"phone":"12345","twitter":"BLAHBLAH"},
"location": {"address":"NOTTELLING","lat":10,"lng":10,"postalCode":"1234",
"city":"BLAH","state":"BLAH","country":"BLAH"},
"categories":[{"id":"BLAH","name":"BLAH"}]
}
HERE
my $id = "name1";
sub function {
my ( $id, $data, $content ) = #_;
my %data = %$data;
my $out = decode_json($content);
say "out:", Dump $out;
$data{$id} = $out;
}
function( $id, \%data, $content );
say "data:", Dump %data;
This doesn't work as the way I expected. Can you please tell me why and how it will work?
"This doesn't work as the way i expected."
What were you expecting? Let's step through the errors:
1) date != data
2) $content=~m!(,*)! will leave $1 empty, since $content doesn't contain any commas.
3) decode_json($1) will throw a runtime error, since $1 is empty and decode_json() can only be applied to a properly formatted JSON string.
4) $id is not defined.
"Can you please tell me why and how it will work?"
It won't work, if that isn't clear yet. There are more errors than code there.
"how do I assign a hash ref into hash?"
Use the \ unary reference operator, eg:
my %h = ();
my %h2 = (
a => 10
);
$h{h2} = \%h2;
print $h{h2}->{a};
You can also declare a scalar ($) as a reference to an anonymous (unnamed) hash; here $hr is a reference, the hash itself has no symbol or name associated with it:
my $hr = {
n => 42
};
# as an existing hash member:
$h{h3} = {
x => 666,
# some other examples:
hr => \%h2,
hr2 => {
x => 1024
}
};
Notice curly braces {} used in the declaration instead of (). When you are nesting (anonymous) hashes as with hr2, always use that form.
If you search for perl hash tutorial you'll find more in-depth things.
The reason that you're not finding anything in the package-scoped %data (the one defined just after use YAML::XS) is because you're creating a brand-new and completely independent %data inside of function with the line
my %data = %$data;
This creates a new hash and copies the contents of the hash referenced by $data into it.
Try this instead:
sub function {
my ($id, $data, $content) = #_;
my $out = decode_json($content);
say "out:", Dump $out;
$data->{$id} = $out;
}
I think you have a typo:
function($id,/%data,$content);
must be
function($id,\%data,$content);
and $content is not a reference to %data hash, so in your function you should do:
my %data=%$data; # in place of "my %content=%$content;"
All right, I'll try to explain what I've done so far. I'm using a Parellel::ForkManager to grab data from an array of URLs, which is then stored in variables (value1, value2, value3).
I then collect the data from all of those processes, and display the data with $pm->run_on_finish.
#...
my $pm = new Parallel::ForkManager(10);
$pm->run_on_finish (
sub {
my #info = #{$data_structure_reference};
print $info[0];
print $info[1];
print $info[2];
}
);
for my $var (#urls) {
$pm->start and next;
#...
#returned = &something($var);
#...
$pm->finish(0, \#returned);
}
sub something {
#... getting data from each URL and storing it in variables
my #array = (
$value1,
$value2,
$value3
);
return #array;
}
Now, what I want to do, is to pass an array, #value4, as well, and then only display that data if there is something in the array. So, I want it to look like this:
sub something {
#... getting data from each URL and storing it in variables
my #array = (
$value1,
$value2,
$value3,
#value4
);
return #array;
}
And then I want it to print that array, only if there is something in it.
Unfortunately, I'm not entirely sure how to go about doing that.
I assume that what you are asking is how to return an array along with the three scalars returned from the something() sub, and print it?
I also assume that those three scalars are what's referred to as being in #info.
The simplest way seems to me to be to simply tack them to the end of the array you return, use the three first values, and if there's anything left, print that too.
$pm->run_on_finish (
sub {
my #info = #{$data_structure_reference};
print splice #info, 0, 3;
print #info if (#info);
}
);
sub something {
return (
$value1,
$value2,
$value3,
#value4
);
}
As you'll notice, you do not need to fill a dummy array for the return value, simply return values inside the parens. You do not need to dereference the array, since you can use the #info array straight up if you splice off the first three values.
I like it simple. If it works.
I've previously provided a simple solution to this problem. It can use threads (use threads;) or processes (use forks;).
use threads; # or: use forks;
use Thread::Queue qw( );
use constant NUM_WORKERS => 10;
my $request_q = Thread::Queue->new();
my $response_q = Thread::Queue->new();
# Create the workers.
for (1..NUM_WORKERS) {
(async {
while (defined(my $request = $request_q->dequeue())) {
$response_q->enqueue(process_request($request));
}
})->detach();
}
# Submit work to workers.
$request_q->enqueue(#requests);
# Signal the workers they are done.
$request_q->enqueue(undef) for 1..NUM_WORKERS;
# Collect the results.
my $working = NUM_WORKERS;
while ($working) {
my $result = $response_q->dequeue();
if (!defined($result)) {
--$working;
next;
}
process_response($result);
}
The work to be done in the children is done by process_request.
sub process_request {
# ...
return [
$value1,
$value2,
$value3,
\#value4,
];
}
The results are passed to process_response in the parent.
sub process_response {
my ($value1, $value2, $value3, $value4) = #{ $_[0] };
...
}
I am not totally sure what you are asking, but in order to pass multiple arrays to a function in Perl one must pass by reference.
my #array1 = (1, 2, 3);
my #array2 = ('a', 'b', 'c');
&fn(\#array1, \#array2);
In order to print an array only in the case when it has value, one must simply check that it has value and print it:
print "#array" if #array;
Though, the nice function of "#array" is that if #array has no value then "#array" evaluates to "". This allows you to reduce the previous statement to a simple:
print "#array"
I was wondering if you could shed some lights regarding the code I've been doing for a couple of days.
I've been trying to convert a Perl-parsed hash back to XML using the XMLout() and XMLin() method and it has been quite successful with this format.
#!/usr/bin/perl -w
use strict;
# use module
use IO::File;
use XML::Simple;
use XML::Dumper;
use Data::Dumper;
my $dump = new XML::Dumper;
my ( $data, $VAR1 );
Topology:$VAR1 = {
'device' => {
'FOC1047Z2SZ' => {
'ChassisID' => '2009-09',
'Error' => undef,
'Group' => {
'ID' => 'A1',
'Type' => 'Base'
},
'Model' => 'CATALYST',
'Name' => 'CISCO-SW1',
'Neighbor' => {},
'ProbedIP' => 'TEST',
'isDerived' => 0
}
},
'issues' => [
'TEST'
]
};
# create object
my $xml = new XML::Simple (NoAttr=>1,
RootName=>'data',
SuppressEmpty => 'true');
# convert Perl array ref into XML document
$data = $xml->XMLout($VAR1);
#reads an XML file
my $X_out = $xml->XMLin($data);
# access XML data
print Dumper($data);
print "STATUS: $X_out->{issues}\n";
print "CHASSIS ID: $X_out->{device}{ChassisID}\n";
print "GROUP ID: $X_out->{device}{Group}{ID}\n";
print "DEVICE NAME: $X_out->{device}{Name}\n";
print "DEVICE NAME: $X_out->{device}{name}\n";
print "ERROR: $X_out->{device}{error}\n";
I can access all the element in the XML with no problem.
But when I try to create a file that will house the parsed hash, problem arises because I can't seem to access all the XML elements. I guess, I wasn't able to unparse the file with the following code.
#!/usr/bin/perl -w
use strict;
#!/usr/bin/perl
# use module
use IO::File;
use XML::Simple;
use XML::Dumper;
use Data::Dumper;
my $dump = new XML::Dumper;
my ( $data, $VAR1, $line_Holder );
#this is the file that contains the parsed hash
my $saveOut = "C:/parsed_hash.txt";
my $result_Holder = IO::File->new($saveOut, 'r');
while ($line_Holder = $result_Holder->getline){
print $line_Holder;
}
# create object
my $xml = new XML::Simple (NoAttr=>1, RootName=>'data', SuppressEmpty => 'true');
# convert Perl array ref into XML document
$data = $xml->XMLout($line_Holder);
#reads an XML file
my $X_out = $xml->XMLin($data);
# access XML data
print Dumper($data);
print "STATUS: $X_out->{issues}\n";
print "CHASSIS ID: $X_out->{device}{ChassisID}\n";
print "GROUP ID: $X_out->{device}{Group}{ID}\n";
print "DEVICE NAME: $X_out->{device}{Name}\n";
print "DEVICE NAME: $X_out->{device}{name}\n";
print "ERROR: $X_out->{device}{error}\n";
Do you have any idea how I could access the $VAR1 inside the text file?
Regards,
newbee_me
$data = $xml->XMLout($line_Holder);
$line_Holder has only the last line of your file, not the whole file, and not the perl hashref that would result from evaling the file. Try something like this:
my $ref = do $saveOut;
The do function loads and evals a file for you. You may want to do it in separate steps, like:
use File::Slurp "read_file";
my $fileContents = read_file( $saveOut );
my $ref = eval( $fileContents );
You might want to look at the Data::Dump module as a replacement for Data::Dumper; its output is already ready to re-eval back.
Basically to load Dumper data you eval() it:
use strict;
use Data::Dumper;
my $x = {"a" => "b", "c"=>[1,2,3],};
my $q = Dumper($x);
$q =~ s{\A\$VAR\d+\s*=\s*}{};
my $w = eval $q;
print $w->{"a"}, "\n";
The regexp (s{\A\$VAR\d+\s*=\s*}{}) is used to remove $VAR1= from the beginning of string.
On the other hand - if you need a way to store complex data structure, and load it again, it's much better to use Storable module, and it's store() and retrieve() functions.
This has worked for me, for hashes of hashes. Perhaps won't work so well with structures which contain references other structures. But works well enough for simple structures, like arrays, hashes, or hashes of hashes.
open(DATA,">",$file);
print DATA Dumper(\%g_write_hash);
close(DATA);
my %g_read_hash = %{ do $file };
Please use dump module as a replacement for Data::Dumper
You can configure the variable name used in Data::Dumper's output with $Data::Dumper::Varname.
Example
use Data::Dumper
$Data::Dumper::Varname = "foo";
my $string = Dumper($object);
eval($string);
...will create the variable $foo, and should contain the same data as $object.
If your data structure is complicated and you have strange results, you may want to consider Storable's freeze() and thaw() methods.