Extract specific XML element in CDATA taken from SOAP::Lite Response Hash - perl

My code below is connecting to a .asmx web service to get data.
The code extractes the CDATA into %keyHash below.
Rather than parsing the entire CDATA, is it possible to grab a specific data element in the SOAP CDATA by calling out it's path?
I read that I could use $soap->valueof() to get the data, is that correct? and that this would require use of XPATH?
I ask as I am unfamiliar with this and I do not know if I am on the right path, are there other ways to do this?
My $soap->valueof('//Images/Front') attempts have failed, saying that, my first time using XPATH, could be getting it wrong but at this point I am guessing if this is the right way to go.
Any direction on whether I am on the right or wrong path in using valueof() would be appreciated!
Here is the code, it works. I have also included the obscurated CDATA data extracted from %keyHash.
use SOAP::Lite +trace => 'all';
$soap = SOAP::Lite
-> uri('..../')
-> on_action( sub { join '/', '.....', $_[1] } )
-> proxy('......asmx');
$method = SOAP::Data->name('methodName')
->attr({xmlns => ...../'});
#params = (
SOAP::Data->name(tran=> 765) ->type(''),
SOAP::Data->name(token => 0)->type(''),
SOAP::Data->name(type=> 1)->type('')
);
%keyHash = %{ $soap->call($method => #params)->body->{'GetmethodNameResponse'}->{'GetmethodNameResult'} };
# iterate through all fields and print them
foreach my $k (keys %keyHash) {
print "$k=$keyHash{$k}\n";
}
Example of the data output, I want the data in the string "THIS_IS_THE_DATA_I_WANT" (unable to put the path here for some reason)
RequestResult=0
Xml=<?xml version="1.0" encoding="utf-8"?>
<Images>
<Front>THIS_IS_THE_DATA_I_WANT</Front>
</Images>
Thank You,
A

I solved this by using the following, hope it helps someone...
use XML::Simple;
%keyhash = %{ $soap->call($method => #params)->body->{'GetCheckXmlResponse'}->{'GetCheckXmlResult'}};
$getxml= %keyhash->{Xml};
$parsexml = XMLin($getxml);
print Dumper($parsexml); # Use this to point to your data and then grab it as per the line below
$frontside = $parsexml->{Images}->{Front};

Related

Using variable for HTTP request headers with Perl

I am trying to write a function to create HTTP requests (POST and GET mostly) in Perl. I am keeping everything generic by using variables so that I don't have to worry about the type of request, the payload, headers, etc, however HTTP::Request->header() doesn't seem to like my variable:
my($req_type, $headers, $endpoint, $args, $request, $jsonfile) = #_;
my $ua = LWP::UserAgent->new;
my $req = HTTP::Request->new($req_type => $endpoint);
$req->content_type('application/json');
foreach (#$headers) {
$req->push_header($_);
}
$req->content($args);
$req->content($request);
print "request : ".$req->as_string;
I tried a few different approches, and using push_header got me the closest, but I realize it may not be the best solution. I think it might have something to do with single quotes getting passed in:
#headers = "'x-auth-token' => '$_token'";
I can post more of the code if it is helpful. I'm hoping some Perl guru will know exactly what I'm doing wrong. I'm sure it's something to do with the format of the string I'm passing in.
#headers = "'x-auth-token' => '$_token'";
The header function expects to be passed two arguments. The header name and the header value.
You are passing it one argument: a string containing a fragment of Perl code.
You need to format your data more sensibly.
my %headers = (
"x-auth-token" => $_token;
);
and
foreach my $header_name (keys %headers) {
$req->push_header($header_name => $headers{$header_name});
}

Method to parse request_uri header from decoded JSON in Perl

Ok, so here is what we are doing. We are viewing a json request/response string.
Code snippet (assuming relevant modules been used):
if( open( my $json_file, $filename ))
{
my $json = JSON->new;
my $data = $json->decode(<$json_file>);
close( json_file );
$request_uri = $data->{'input'}{'Headers'}{'REQUEST_URI'};
}
So $request_uri looks something like
/user/12345?param1=4&param2=9956
Whilst I could use regex or whatever to extract data out of there, I am sure this is a common situation and there should be a method to parse this particular REST into its parts and then extract them out. I do not see this in the REST manual which seems to be more about constructing requests.
Use the URI module.
my $request_uri = URI->new( $data->{'input'}{'Headers'}{'REQUEST_URI'} );
my $path = $request_uri->path;
my $query = $request_uri->query;
# etc

Removing top-directory-only URLs from a list of URLs?

I have a question that I'm having trouble researching, as I don't know how to ask it correctly on a search engine.
I have a list of URLs. I would like to have some automated way (Perl for preference) to go through the list and remove all URLs that are top directory only.
So for example I might have this list:
http://www.example.com/hello.html
http://www.foo.com/this/thingrighthere.html
In this case I would want to remove example.com from my list, as it is either top-directory only or they reference files in a top directory.
I'm trying to figure out how to do that. My first thought was, count forward slashes and if there's more than two, eliminate the URL from the list. But then you have trailing forward slashes, so that wouldn't work.
Any ideas or thoughts would be much appreciated.
Something like this:
use URI::Split qw( uri_split );
my $url = "http://www.foo.com/this/thingrighthere.html";
my ($scheme, $auth, $path, $query, $frag) = uri_split( $url );
if (($path =~ tr/\///) > 1 ) {
print "I care about this $url";
}
http://metacpan.org/pod/URI::Split
You could do this with regexes, but its much less work to let the URI library do it for you. You won't get caught out by funny schemes, escapes, and extra stuff before and after the path (query, anchor, authorization...). There's some trickiness around how paths are represented by path_segments(). See the comments below and the URI docs for details.
I have assumed that http://www.example.com/foo/ is considered a top directory. Adjust as necessary, but its something you have to think about.
#!/usr/bin/env perl
use URI;
use File::Spec;
use strict;
use warnings;
use Test::More 'no_plan';
sub is_top_level_uri {
my $uri = shift;
# turn it into a URI object if it isn't already
$uri = URI->new($uri) unless eval { $uri->isa("URI") };
# normalize it
$uri = $uri->canonical;
# split the path part into pieces
my #path_segments = $uri->path_segments;
# for an absolute path, which most are, the absoluteness will be
# represented by an empty string. Also /foo/ will come out as two elements.
# Strip that all out, it gets in our way for this purpose.
#path_segments = grep { $_ ne '' } #path_segments;
return #path_segments <= 1;
}
my #filtered_uris = (
"http://www.example.com/hello.html",
"http://www.example.com/",
"http://www.example.com",
"https://www.example.com/",
"https://www.example.com/foo/#extra",
"ftp://www.example.com/foo",
"ftp://www.example.com/foo/",
"https://www.example.com/foo/#extra",
"https://www.example.com/foo/?extra",
"http://www.example.com/hello.html#extra",
"http://www.example.com/hello.html?extra",
"file:///foo",
"file:///foo/",
"file:///foo.txt",
);
my #unfiltered_uris = (
"http://www.foo.com/this/thingrighthere.html",
"https://www.example.com/foo/bar",
"ftp://www.example.com/foo/bar/",
"file:///foo/bar",
"file:///foo/bar.txt",
);
for my $uri (#filtered_uris) {
ok is_top_level_uri($uri), $uri;
}
for my $uri (#unfiltered_uris) {
ok !is_top_level_uri($uri), $uri;
}
Use the URI module from CPAN. http://search.cpan.org/dist/URI
This is a solved problem. People have already written, tested and debugged code that handles this already. Whenever you have a programming problem that others have probably had to deal with, then look for existing code that does it for you.

Perl: what kind of data should i feed to delcampe API?

I write soap-client based on Delcampe API. Simple methods work fine, but functions with need on complex data give me an error message like "You must send item's data!". Based on PHP example here i thought, that data should be either hash or hashref, but both give me error mentioned before.
Sample script i use:
use 5.010;
use SOAP::Lite;
use SOAP::WSDL;
use strict;
use warnings;
use Data::Dumper;
my $API_key = 'xyz';
my $service = SOAP::Lite->service('http://api.delcampe.net/soap.php?wsdl');
my $return = $service->authenticateUser($API_key);
if ($return->{status}) {
my $key = $return->{data};
my %data = (description => 'updated description');
my $response = $service->updateItem($key, 123456, \%data);
if ($response->{status}) {
say Dumper $response->{data};
} else {
say $response->{errorMsg};
}
} else {
say "no: " . $return->{status};
}
So, what kind of data structure should i use instead of %data or how could i debug the SOAP-envelope, which is produced as request? (PHP code based on example works fine)
ADDITION
with use SOAP::Lite qw(trace); igot SOAP envelope too:
<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/" xmlns:soap-enc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:tns="http://api.delcampe.net/soap.php">
<soap:Body>
<tns:updateItem>
<token xsi:type="xsd:string">secret_one</token>
<id_item xsi:type="xsd:int">123456</id_item>
<arrData xsi:nil="true" xsi:type="soap-enc:Array" />
</tns:updateItem>
</soap:Body>
</soap:Envelope>
As seen above, there is no bit of data sent. I tried data also as string, array and arrayref. Maybe it is bug of SOAP::Lite?
May be you'd try to replace
my %data = (description => 'updated description');
with
my $data = SOAP::Data->name(description => 'updated description');
We have similar issues when working on our SOAP API, and it was solved by something like that, wrapping complex data into SOAP::Data. So I hope this'll help. )
UPDATE:
The previous advice didn't help: looks like it's indeed the SOAP::Lite bug, which ignores the 'soap-enc:Array' definition in WSDL file whatsoever.
Have finally found a workaround, though. It's not pretty, but as a final resort it may work.
First, I've manually downloaded the WSDL file from Delcampe site, saved it into local directory, and referred to it as ...
my $service = SOAP::Lite->service('file://...delcampe.wsdl')
... as absolute path is required.
Then I've commented out the 'arrData line' within WSDL updateItem definition.
And, finally, I've made this:
my $little_monster = SOAP::Data->name(arrData =>
\SOAP::Data->value((
SOAP::Data->name(item =>
\SOAP::Data->value(
SOAP::Data->name(key => 'personal_reference'),
SOAP::Data->name(value => 'Some Personal Reference')->type('string'),
)
),
SOAP::Data->name(item =>
\SOAP::Data->value(
SOAP::Data->name(key => 'title'),
SOAP::Data->name(value => 'Some Amazing Title')->type('string'),
)
),
# ...
))
)->type('ns1:Map');
... and, I confess, successfully released it into the wilderness by ...
$service->updateItem($key, 123456, $little_monster);
... which, at least, generated more-o-less likable Envelope.
I sincerely hope that'll save at least some poor soul from banging head against the wall as much as I did working on all that. )

How do I use Perl's Google::Search to look for specific URLs?

The Google::Search module, which is based on the AJAX Search API, doesn't seems to work very well, or it is just me?
For example, I use firefox to search google for: http://bloggingheads.tv/forum/member.php?u=12129
It brings results.
But when I use the module this way:
$google_search = Google::Search->Web ( q => "http://bloggingheads.tv/forum/member.php?u=12129" );
#result = $google_search->all;
I get nothing in the array.
Any idea?
Seems like this API doesn't bring the same results like when searching manually, am I missing something?
I had a similar problem with cyrillic queries. Both Google::Search and REST::Google from CPAN didn't work for me - they were giving back fewer or no results compared to manual test.
Eventually I wrote a scraping module using WWW::Mechanize and HTML::TreeBuilder.
Here's a sample to get result stats:
my $tree = HTML::TreeBuilder->new_from_content($content);
if (my $div = $tree->look_down(_tag => 'div', id => 'resultStats')) {
my $stats = $div->as_text();
}
else { warn "no stats" }
Looking at the POD for Google::Search, it looks like it expects you to pass search terms to Web, instead of a URL. I downloaded a test script from CPAN, ran it, and it seems to produce expected results:
use strict;
use warnings;
use Google::Search;
my $search = Google::Search->Web(q => "rock");
my $result = $search->first;
while ($result) {
print $result->number, " ", $result->uri, "\n";
$result = $result->next;
}
print $search->error->reason, "\n" if $search->error;
__END__
0 http://www.rock.com/
1 http://en.wikipedia.org/wiki/Rock_music
2 http://en.wikipedia.org/wiki/Rock_(geology)
3 http://rockyourphone.com/
4 http://rockhall.com/
5 http://www.co.rock.mn.us/
6 http://www.co.rock.wi.us/
7 http://www.rockride.org/
etc...
I realize this does not specifically answer your question, but perhaps it steers you in the right direction.