I wrote this script but I'm not sure if it is correct.
What I want to do is process a JSON file by reading its content, decoding it, and looping through each item as $item. The contents from a certain URL with the ID defined as $items[$i]['paper_item_id'] are saved with that ID into the defined destination.
But the code doesn't seem to function. I'm not sure on where I went wrong but any help or tips to improve the code and make it work would be good.
I'm not asking you to do the job, just need help seeing on where I went wrong and correct it for me.
The script should basically decode the JSON and then download the swf files from a certain directory URL to a directory on my PC using the IDs.
This is the code
use LWP::Simple;
$items = 'paper_items.json';
my $s = $items or die;
$dcode = decode_json($items);
for ($i = 0 ; $i < $count ($items) ; $i++) {
use File::Copy;
$destination = "paper/";
copy(
"http://media1.clubpenguin.com/play/v2/content/global/clothing/paper/"
. $items[$i]['paper_item_id'] . ".swf",
$destination . $items[$i]['paper_item_id'] . ".swf"
);
The program can be broken down into three steps:
Fetch the JSON source.
Parse the JSON.
Iterate over decoded data structure. We expect an array of hashes. Mirror files denoted by the paper_item_id to the working directory.
We will use LWP::Simple functions here.
Our script has the following header:
#!/usr/bin/perl
use strict; # disallow bad constructs
use warnings; # warn about possible bugs
use LWP::Simple;
use JSON;
Fetching the JSON
my $json_source = get "http://media1.clubpenguin.com/play/en/web_service/game_configs/paper_items.json";
die "Can't access the JSON source" unless defined $json_source;
That was easy: we dispatch a get request on that URL. If the output is undefined, we throw a fatal exception.
Parsing the JSON
my $json = decode_json $json_source;
That was easy; we expect the $json_source to be an UTF-8 encoded binary string.
If we want to inspect what is inside that data structure, we can print it out like
use Data::Dumper; print Dumper $json;
or
use Data::Dump; dd $json;
If everything works as expected, this should give a screenfull of an array of hashes.
Iterating
The $json is an array reference, so we'll loop over all items:
my $local_path = "paper";
my $server_path = "http://media1.clubpenguin.com/play/v2/content/global/clothing/paper";
for my $item (#$json) {
my $filename = "$item->{paper_item_id}.swf";
my $response = mirror "$server_path/$filename" => "$local_path/$filename";
warn "mirror failed for $filename with $response" unless $response == 200;
}
Perl has a concept of references, which is similar to pointers. Because data structures like hashes or arrays can only contain scalars, other arrays or hashes are only referenced. Given an array reference, we can access the array like #$reference or #{ $reference }.
To access an entry, the subscript operator [...] for arrays or {...} for hashes is seperated by the dereference operator ->.
Thus, given %hash and $hashref to the same hash,
my %hash = (key => "a", otherkey => "b");
my $hashref = \%hash;
then $hashref->{key} eq $hash{key} holds.
Therefore, we loop over the items in #$json. All of these items are hash references, therefore we use $item->{$key}, not $hash{key} syntax.
What you are trying to do is to download the Shockwave Flash resources from Disney's Club Penguin game site.
I cannot imagine Disney would be too happy about this, and the site's terms of use say this under "Use of Content" ("DIMG" is Disney Interactive Media Group)
Except as we specifically agree in writing, no Content from any DIMG Site may be used, reproduced, transmitted, distributed or otherwise exploited in any way other than as part of the DIMG Site ...
Code is untested.
use File::Slurp qw(read_file);
use JSON qw(decode_json);
use LWP::Simple qw(mirror);
for my $item (#{ decode_json read_file 'paper_items.json' }) {
my $id = $item->{paper_item_id};
mirror "http://media1.clubpenguin.com/play/v2/content/global/clothing/paper/$id.swf", "paper/$id.swf";
}
Related
It has been awhile since I have used perl and I am attempting to print out a list of files on a SFTP server.
This is my Perl script -
#! /usr/bin/env perl
use strict;
use warnings;
use feature qw(say);
use autodie;
use Net::SFTP::Foreign;
# US Server Configuration
use constant {
HOST => "Server_Name",
REMOTE_DIR => "\\",
LOCAL_DIR => "sample/local",
PORT => "3235",
USER_NAME => "name",
PASSWORD => "password",
BACKEND => "Net_SSH2",
DEBUG => "0",
};
my $stfp = Net::SFTP::Foreign->new (
HOST,
backend => BACKEND,
timeout => 240,
user => USER_NAME,
password => PASSWORD,
port => PORT,
autodie => 1,
);
#
# List remote directory contents
#
my $remotefiles;
$remotefiles = $stfp->ls(REMOTE_DIR);
#
# Loop through remote files and print each filename
#
foreach ($remotefiles){
my $file = $_;
my $filename = $file->{filename};
if($filename ne "." && $filename ne ".."){
print"the filename is $filename";
}
}
$stfp->disconnect;
I am getting the following error - Not a HASH reference at this line -> my $filename = $file->{filename};
Not sure what the problem is or how to fix it.
Here's the documentation for Net::SFTP::Foreign. This is what it says about the ls() method:
Returns a reference to a list of entries. Every entry is a
reference to a hash with three keys: filename, the name of the
entry; longname, an entry in a "long" listing like ls -l; and
a, a Net::SFTP::Foreign::Attributes object containing file
atime, mtime,permissions and size
So when you call this method:
$remotefiles = $sftp->ls(REMOTE_DIR);
What you get back is a reference to an array. When you try to iterate across that array:
foreach ($remotefiles)
You're not doing what you think you're doing. You're iterating over a list with a single element - which is an array reference. So $file gets an array reference, not the hash reference that you're looking for.
So how to fix it? Well, it's simple enough. You need to dereference $remotefiles which will turn it into an array (rather than an array reference).
# Just add a '#' to dereference the reference and
# get to the underlying array
foreach (#$remotefiles)
Now, each time round the loop, $_ (and, therefore, $file) will contain one of the hash references described in my quote above, and everything will work as you expect.
The important lesson to learn here is that you need to read and understand the documentation for the code that you are using. Or, at least, you need to be better at copying code from the examples. The ls() example clearly uses the dereference syntax.
my $ls = $sftp->ls('/home/foo')
or die "unable to retrieve directory: ".$sftp->error;
print "$_->{filename}\n" for (#$ls);
$filenane is an array reference. Dereference the variable to get at the referant. See perlreftut for the details. It's easy when you get the hang of it, but doing it the first time may be a little painful.
So I parsed the following XML code using Perl and i'm trying to call the spectrum results but i'm having difficulty since it is a hash. I keep getting the error message reference found where even sized list expected.
<message>
<cmd id="result_data">
<result-file-header>
<path>String</path>
<duration>Float</duration>
<spectra-count>Integer</spectra-count>
</result-file-header>
<scan-results count="Integer">
<scan-result>
<spectrum-index>Integer</spectrum-index>
<time-stamp>Integer</time-stamp>
<tic>Float</tic>
<start-mass>Float</start-mass>
<stop-mass>Float</stop-mass>
<spectrum count="Integer">mass,abundance;mass1,abundance1;
mass2,abundance2</spectrum>
</scan-result>
<scan-result>
<spectrum-index>Integer</spectrum-index>
<time-stamp>Integer</time-stamp>
<tic>Float</tic>
<start-mass>Float</start-mass>
<stop-mass>Float</stop-mass>
<spectrum count="Integer">mass3,abundance3;mass4,abundance4;
mass5,abundance5</spectrum>
</scan-result>
</scan-results>
</cmd>
</message>
Here is the Perl code i'm using:
my $file = "gapiparseddataexample1.txt";
unless(open FILE, '>'.$file) {
die "\nUnable to create $file\n";
}
use warnings;
use XML::Simple;
use Data::Dumper;
my $values= XMLin('samplegapi.xml', ForceArray => [ 'scan-result' ,'result-file-header']);
print Dumper($values);
my $results = $values->{'cmd'}->{'scan-results'}->{'scan-result'};
my $results1=$values->{'cmd'}->{'result-file-header'};
for my $data (#$results) {
print FILE "Spectrum Index",":",$data->{"spectrum-index"},"\n";
print FILE "Total Ion Count",":",$data->{tic},"\n";
%spectrum=$data->{spectrum};
print FILE "Spectrum",":",%spectrum, "\n";
for my $data1 (#$results1) {
print FILE "Duration",":",$data1->{duration},"\n";
}
}
I want to be able to print out the spectrum value pairs.
This:
$spectrum=$data->{spectrum};
print FILE "Spectrum",":", $spectrum->{'content'}, "\n";
for my $data1 (#$results1) {
print FILE "Duration",":",$data1->{duration},"\n";
}
Should give you this (which I assume is what you want):
Spectrum:mass,abundance;mass1,abundance1;
mass2,abundance2
You'll want to remove the newline value from 'content' I imagine (so it doesn't split over two lines).
Explanation for anyone that's curious
The element contents have been shoved into "->content" because element also has an attribute. In this case, one called "count":
<spectrum count="Integer">mass3,abundance3;mass4,abundance4;
mass5,abundance5</spectrum>
This sort of behaviour is common in other languages and other XML parsing libraries too (e.g. sometimes they shove it into an element with the key 0). Sometimes it happens even when elements don't have regular attributes but are of specific types.
If you were to var dump $data->{$spectrum} you'd see the structure (again that usually applies in other languages and with other XML parsing libraries too).
I need to be able to send a file stream and a byte array as a response to an HTTP POST for the testing of a product. I am using CGI perl for the back end, but I am not very familiar with Perl yet, and I am not a developer, I am a Linux Admin. Sending a string based on query strings was very easy, but I am stuck on these two requirements. Below is the script that will return a page with Correct or Incorrect depending on the query string. How can I add logic to return a filestream and byte array as well?
#!/usr/bin/perl
use CGI ':standard';
print header();
print start_html();
my $query = new CGI;
my $value = $ENV{'QUERY_STRING'};
my $number = '12345';
if ( $value == $number ) {
print "<h1>Correct Value</h1>\n";
} else {
print "<h1>Incorrect value, you said: $value</h1>\n";
}
print end_html();
Glad to see new people dabbling in Perl from the sysadmin field. This is precisely how I started.
First off, if you're going to use the CGI.pm module I would suggest you use it to your advantage throughout the script. Where you've previously inputted <h1> you can use your CGI object to do this for you. In the end, you'll end up with much cleaner, more manageable code:
#!/usr/bin/perl
use CGI ':standard';
print header();
print start_html();
my $value = $ENV{'QUERY_STRING'};
my $number = '12345';
if ( $value == $number ) {
h1("Correct Value");
} else {
h1("Incorrect value, you said: $value");
}
print end_html();
Note that your comparison operator (==) will only work if this is a number. To make it work with strings as well, use the eq operator.
A little clarification regarding what you mean regarding filestreams and byte arrays ... by file stream, do you mean that you want to print out a file to the client? If so, this would be as easy as:
open(F,"/location/of/file");
while (<F>) {
print $_;
}
close(F);
This opens a file handle linked to the specified file, read-only, prints the content line by line, then closes it. Keep in mind that this will print out the file as-is, and will not look pretty in an HTML page. If you change the Content-type header to "text/plain" this would probably be more within the lines of what you're looking for. To do this, modify the call which prints the HTTP headers to:
print header(-type => 'text/plain');
If you go this route, you'll want to remove your start_html() and end_html() calls as well.
As for the byte array, I guess I'll need a little bit more information about what is being printed, and how you want it formatted.
I am getting
$VAR1 = bless( \*{'Fh::fh00001Screenshot.png'}, 'Fh' );
in a variable. But I need to retrieve fh00001Screenshot.png from it. How can I get it?
The Fh package is used internally by the CGI module to handle temporary files used for building multipart data. You shouldn't be using it directly.
Check carefully to make sure there is no better way before using this code which comes from the CGI code for Fh::asString
(my $name = $$VAR1) =~ s/^\*(\w+::fh\d{5})+//;
print $name;
output
Screenshot.png
Update
Rather than picking bits out of the CGI code, it looks like this package - which should really be a private one - is accessible from calling code. Use just $var->asString instead, like this
use strict;
use warnings;
use CGI;
my $var = do {
no strict 'refs';
my $var = bless( \*{'Fh::fh00001Screenshot.png'}, 'Fh' );
};
print $var->asString;
I'm using Perl 5.12 on Mac 10.5.7. I have a JAR file, that I wish to unzip, and then process a file matching a file pattern. I'm having trouble figuring out how to iterate over the results of unzipping. I have ...
### build an Archive::Extract object ###
my $ae = Archive::Extract->new( archive => $dbJar );
### what if something went wrong?
my $ok = $ae->extract or die $ae->error;
### files from the archive ###
my #files = $ae->files;
for (#files) {
print "file : $_\n";
But there is only one thing returned from the loop -- "ARRAY(0x7fd5c22f7cd8)". I know there are lots of files in the archive, so I'm curious what I"m doing wrong here. - Dave
$ae->files returns an array reference, not an array. Try this:
my $files = $ae->files;
for(#$files) {
print "file : $_\n";
}
From the Perldoc of Archive::Extract:
$ae->files
This is an array ref holding all the paths from the archive. See extract() for details.
It's quite common for methods to return not arrays and hashes, but references to them. It's also quite common for methods to take references to arrays and hashes as arguments. This is because less data has to be passed back and forth between the method and your call.
You can do this:
for my $file ( #{ $ae->files } ) {
print "$file\n";
}
The #{...} dereferences the reference and makes it into a simple array. And yes, you can put method calls that return an array reference in the #{...} like I did.
As already mentioned, a very helpful package is Data::Dumper. This can take a reference and show you the data structure contained therein. It also will tell you if that data structure represents a blessed object which might be a clue that there is a method you can use to pull out the data.
For example, imagine an object called Company. One of the methods might be $company->Employees which returns an array reference to Employee objects. You might not realize this and discover that you get something like ARRAY{0x7fd5c22f7cd8) returned, pushing this through Data::Dumper might help you see the structure:
use Data::Dumper;
use Feature qw(say);
use Company;
[...]
#employee_list = $company->Employees;
# say join "\n", #employee_list; #Doesn't work.
say Dumper #employee_list;
This might print:
$VAR = [
{
FIRST => 'Marion',
LAST => 'Librarian',
POSITION => 'Yes Man',
} Employee,
{
FIRST => 'Charlie',
LAST => 'australopithecus`,
POSITION => 'Cheese Cube Eater'
} Employee,
]
Not only do you see this is an array of hash references, but these are also objects Employee too. Thus, you should use some methods from the Employee class to parse the information you want.
use Feature qw(say);
use Company;
use Employee;
[...]
for my $employee ( #{ $company->Employees } ) {
say "First Name: " . $employee->First;
say "Last Name: " . $employee->Last;
say "Position: " . $employee->Position;
print "\n";
}
Try doing this :
use Data::Dumper;
print Dumper #files;
That way, you will see the content of #files.
If you don't know how to process your data structure, paste the output of Dumper here