Unable to supply complex form values using WWW::Mechanize - perl

I have an HTML form that I want to process with the WWW::Mechanize module. It seems that I can't POST an input value that contains a dollar sign $. I get this run time error message:
Can't call method "value" on an undefined value at script.pl line 599
My script looks like this
my $agent = WWW::Mechanize->new( autocheck => 1, ssl_opts => { verify_hostname => 0 }, );
my $formfiller = WWW::Mechanize::FormFiller->new();
$agent->env_proxy();
$agent->get('https://site.com/form.php');
$agent->form_number(1) if $agent->forms and scalar #{$agent->forms};
$agent->form_number(1);
{
local $^W;
$agent->current_form->value('page_contentleft_0$form_D69D9E5239FC412ABB92A81D73F690AD$field_F40A1785D7B3490CBD5E72EDBE6B966D', 'John Doe');
}; #// First NAme
{
local $^W;
$agent->current_form->value('page_contentleft_0$form_D69D9E5239FC412ABB92A81D73F690AD$field_E3316C46A4404C73ACBAE107DF6206D2', 'Bridgeport');
}; #// City
{
local $^W;
$agent->current_form->value('page_contentleft_0$form_D69D9E5239FC412ABB92A81D73F690AD$field_E5747EB507E74DC1937557F9285CB57C', 'CT');
}; #// state
{
local $^W;
$agent->current_form->value('page_contentleft_0$form_D69D9E5239FC412ABB92A81D73F690AD$field_A612569BE44C4BA3AD0AB3FBF8FB0553', '06604');
}; #// Zipcode
$agent->submit(Submit);
As far as I can see the problem is from the parameters to the value method, for instance
page_contentleft_0$form_D69D9E5239FC412ABB92A81D73F690AD$field_A612569BE44C4BA3AD0AB3FBF8FB0553
How can I make Perl pass through the $, or to fix it in another way?

Are you sure you have?
$agent->submit(Submit);
Try this code instead:
$agent->get('https://site.com/form.php');
$agent->submit_form(
with_fields => {
'page_contentleft_0$form_D69D9E5239FC412ABB92A81D73F690AD$field_F40A1785D7B3490CBD5E72EDBE6B966D' => 'John Doe',
'page_contentleft_0$form_D69D9E5239FC412ABB92A81D73F690AD$field_E3316C46A4404C73ACBAE107DF6206D2' => 'Bridgeport',
'page_contentleft_0$form_D69D9E5239FC412ABB92A81D73F690AD$field_E5747EB507E74DC1937557F9285CB57C' => 'CT',
'page_contentleft_0$form_D69D9E5239FC412ABB92A81D73F690AD$field_A612569BE44C4BA3AD0AB3FBF8FB0553' => '06604'},
button => "Submit",);

Related

Debugging HTTP::Tiny

I am using HTTP::Tiny to interact with rest api below an example :
my $req = $ua->get('https://myapi.com/user?xyz', {headers =>
{token => 'xyzzz',
data => '5343'.
}});
i am using Data::Dumper to see the response header and body
print Dumper($req); -#i can see the response
is there an option to see also the HTTP request i am sending using HTTP::Tiny ,without the need of other tool like wireshark etc..
something like what mojo::useragent achieve with MOJO_CLIENT_DEBUG=1 any idea?
I don't see such a feature mentioned in the HTTP::Tiny documentation, nor do I see it facilitated in the source code for the module. However, you can use a module such as Test::MockModule to get a closer look at what is going on. Here is an example:
#!/usr/bin/env perl
use strict;
use warnings;
use HTTP::Tiny;
use Test::MockModule;
use Data::Dumper;
my $t = HTTP::Tiny->new();
my $mock = Test::MockModule->new('HTTP::Tiny');
$mock->redefine('_request' => sub {
warn "In _request: ", Dumper {self => $_[0], method => $_[1], url => $_[2], args => $_[3]};
return $mock->original('_request')->(#_);
});
print "Response: ", Dumper $t->get('http://localhost:3000');
So in this snippet we are mocking _request, but the mock method we install in behalf of _request calls out to the original _request method, so our mocked method becomes a wrapper around the original method, and we can dump the underlying object and args passed into the method. I'm doing the dump prior to the request being made, but I could have done the initial dump, captured the return value of the real request, and then dump the object again afterward if I thought it might contain additional useful information. So long as we return the actual response, the calling code is none the wiser.
Here's an example of the output. The URL I'm hitting is just a default Mojolicious::Lite app.
In _request: $VAR1 = {
'method' => 'GET',
'self' => bless( {
'agent' => 'HTTP-Tiny/0.076',
'no_proxy' => [],
'max_redirect' => 5,
'keep_alive' => 1,
'verify_SSL' => 0,
'timeout' => 60
}, 'HTTP::Tiny' ),
'args' => {},
'url' => 'http://localhost:3000'
};
Response: $VAR1 = {
'headers' => {
'content-length' => '146',
'content-type' => 'text/html;charset=UTF-8',
'server' => 'Mojolicious (Perl)',
'date' => 'Sun, 26 May 2019 03:19:57 GMT'
},
'protocol' => 'HTTP/1.1',
'reason' => 'OK',
'url' => 'http://localhost:3000',
'success' => 1,
'status' => '200',
'content' => '<!DOCTYPE html>
<html>
<head><title>Welcome</title></head>
<body><h1>Welcome to the Mojolicious real-time web framework!</h1>
</body>
</html>
'
};
Before I was able to know that _request was where I wanted to target my mock, I had to look at the source code for HTTP::Tiny. Fortunately the ::Tiny part of the module means there's not a whole lot to look at. It's really a relatively simple module.
You may decide that for your own purposes it makes more sense to introduce your wrapper at some other point in HTTP::Tiny, but wrapping _request seems like a pretty good choice for most cases.

Config::IniFiles hash behaves different than manually written hash

I am loading a config file, which ends up as an embedded hash, with Config::IniFiles. After that, I want to modify the resulting hash by, for some keys, bringing its values one level up. In the example below, I am aiming for this as a result:
$VAR1 = {
'max_childrensubtree' => '7',
'port' => '1984',
'user' => 'someuser',
'password' => 'somepw',
'max_width' => '20',
'host' => 'localhost',
'attrs' => {
'subattr2' => 'cat',
'topattr1' => 'cat',
'subattr2_1' => 'pt',
'subattr1' => 'rel'
},
'max_descendants' => '1000'
};
So for the keys params and basex at the highest level, I want to move its contents (key-value pairs) to the highest level - and remove the items themselves. In short:
(
a => {
'key1' => 'ok',
'key2' => 'hello'
}
)
turns into
(
'key1' => 'ok',
'key2' => 'hello'
)
The strange thing is that what I am trying to do does not work on a hash built from a read INI file, but it does work with a manually inserted hash. In other words, this works:
#!/usr/bin/perl
use utf8;
use strict;
use warnings;
use Data::Dumper;
my %ini = (
'params' => {
'max_width' => '20',
'max_childrensubtree' => '7',
'max_descendants' => '1000'
},
'attrs' => {
'topattr1' => 'cat',
'subattr1' => 'rel',
'subattr2' => 'cat',
'subattr2_1' => 'pt',
},
'basex' => {
'host' => 'localhost',
'port' => '1984',
'user' => 'someuser',
'password' => 'somepw'
}
);
&_parse_ini(\%ini);
sub _parse_ini {
my $ref = shift;
foreach (('params', 'basex')) {
foreach my $k (keys %{$ref->{$_}}) {
$ref->{$k} = $ref->{$_}->{$k};
}
delete $ref->{$_};
}
print Dumper($ref);
}
But this does not:
#!/usr/bin/perl
use utf8;
use strict;
use warnings;
use Data::Dumper;
use Config::IniFiles;
# Load config file
tie my %ini, 'Config::IniFiles', (-file => $ARGV[0]);
&_parse_ini(\%ini);
sub _parse_ini {
my $ref = shift;
foreach (('params', 'basex')) {
foreach my $k (keys %{$ref->{$_}}) {
$ref->{$k} = $ref->{$_}->{$k};
}
delete $ref->{$_};
}
print Dumper($ref);
}
The input ini file for this example would be:
[params]
max_width = 20
max_childrensubtree = 7
max_descendants = 1000
[attrs]
topattr1 = cat
subattr1 = rel
subattr2 = cat
subattr2_1 = pt
[basex]
host = localhost
port = 1984
user = admin
password = admin
I have been looking in the documentation and on SO for similar issues but have found none. It appears that the hashes are identical (Config::IniFiles doesn't seem to add something specific), so I have no idea why it works for 'manual' hashes, and not for read-in ones.
The two hashes are not identical at all, although they may appear to be from the point of view of the data they contain.
The first one is a regular hash. You can do whatever you like with it.
The second one is a tied hash. It becomes an object of Config::IniFiles, but with a hash like interface. So whilst it appears to be a hash, the package can override the methods for storing or fetching information in the hash however it likes.
In this particular case, it looks like Config::IniFiles will only store a new key value in the hash if the value is hash ref. So you can't flatten out the tied hash as you want. Instead you'll have to create a new hash and copy the data in to it to do what you want.

Getting "000004DC: LdapErr: DSID-0C090752." when performing bind in perl using Net::LDAP

Objective is to get the "dn" attribute of all the computers in my Active Directory server.
When the code executes I get: "000004DC: LdapErr: DSID-0C090752, comment: In order to perform this operation a successful bind must be completed on the connection."
Here is my code:
#!/usr/bin/perl
use strict;
use Net::LDAP;
use Data::Dumper;
my $ldap = Net::LDAP->new( 'my.domain.com' ) or die $#;
my $user = 'CN=username,OU=orgname,DC=my,DC=domain,DC=com';
my $pass = 'my_password';
$ldap->bind($user, password => $pass);
#$ldap->bind;
my $mesg = $ldap->search(
base => "DC=my,DC=domain,DC=com",
filter => "ObjectClass=Computers",
attrs => "dn"
);
I've tested the user / password to log into the domain directly with success.
Additional information if I add this to the end of the script: print Dumper($mesg);
$VAR1 = bless( {
'parent' => bless( {
'net_ldap_version' => 3,
'net_ldap_scheme' => 'ldap',
'net_ldap_debug' => 0,
'net_ldap_socket' => bless( \*Symbol::GEN0, 'IO::Socket::INET' ),
'net_ldap_host' => 'my.domain.com',
'net_ldap_uri' => 'my.domain.com',
'net_ldap_resp' => {},
'net_ldap_mesg' => {},
'net_ldap_async' => 0,
'net_ldap_port' => 389,
'net_ldap_refcnt' => 1
}, 'Net::LDAP' ),
'errorMessage' => '000004DC: LdapErr: DSID-0C090752, comment: In order to perform this operation a successful bind must be completed on the connection., data 0, v2580',
'ctrl_hash' => undef,
'resultCode' => 1,
'callback' => undef,
'mesgid' => 2,
'matchedDN' => '',
'controls' => undef,
'raw' => undef
}, 'Net::LDAP::Search' );
Any suggestions on how to get this script working is what I'm looking for.
Thanks!
With regard to the error message you posted I would say the bind attempt failed.
It might help you to improve bind result:
$mesg = $_ldap->bind("***", password => "***");
$mesg->is_error && die join ';' $mesg->code, $mesg->error
See Net::LDAP:
The return value from these methods is an object derived from the
Net::LDAP::Message class. The methods of this class allow you to
examine the status of the request.

Perl vCard.pm how do I compute inputs?

The documentation for vCard shows it wants:
addresses()
Accepts/returns an arrayref that looks like:
[
{ type => ['work'], street => 'Main St', preferred => 0 },
{ type => ['home'],
pobox => 1234,
extended => 'asdf',
street => 'Army St',
city => 'Desert Base',
region => '',
post_code => '',
country => 'USA',
preferred => 1,
},
]
I must generate this call from a CSV file. I can extract the data, but cannot figure out how to pass it to $vcard->addresses.
This fails:
if (keys %addrwork > 0 )
{
$addrwork{'type'} = ['work'];
push #arraytmp, %addrwork;
}
# did we get any part of a home or work address?
if ( $#arraytmp >= 0 ) {
$vcard->addresses( #arraytmp );
Fails with:
Can't use string ("type") as an ARRAY ref while "strict refs" in use at /usr/local/share/perl/5.22.1/vCard.pm line 254, <FILE> line 2.
Accepts/returns an arrayref that looks like:
You're passing an array, not an arrayref. Fix it:
$vcard->addresses( \#arraytmp );

How can I do a scrolled search on MetaCPAN?

I'm trying to convert this script to use the new Elasticsearch official client instead of the older (now deprecated) ElasticSearch.pm, but I can't get the scrolled search to work. Here's what I've got:
#! /usr/bin/perl
use strict;
use warnings;
use 5.010;
use Elasticsearch ();
use Elasticsearch::Scroll ();
my $es = Elasticsearch->new(
nodes => 'http://api.metacpan.org:80',
cxn => 'NetCurl',
cxn_pool => 'Static::NoPing',
#log_to => 'Stderr',
#trace_to => 'Stderr',
);
say 'Getting all results at once works:';
my $results = $es->search(
index => 'v0',
type => 'release',
body => {
filter => { range => { date => { gte => '2013-11-28T00:00:00.000Z' } } },
fields => [qw(author archive date)],
},
);
foreach my $hit (#{ $results->{hits}{hits} }) {
my $field = $hit->{fields};
say "#$field{qw(date author archive)}";
}
say "\nUsing a scrolled search does not work:";
my $scroller = Elasticsearch::Scroll->new(
es => $es,
index => 'v0',
search_type => 'scan',
size => 100,
type => 'release',
body => {
filter => { range => { date => { gte => '2013-11-28T00:00:00.000Z' } } },
fields => [qw(author archive date)],
},
);
while (my $hit = $scroller->next) {
my $field = $hit->{fields};
say "#$field{qw(date author archive)}";
} # end while $hit
The first search, where I'm just getting all the results in 1 chunk, works fine. But the second search, where I'm trying to scroll through the results, produces:
Using a scrolled search does not work:
[Request] ** [http://api.metacpan.org:80]-[500]
ActionRequestValidationException[Validation Failed: 1: scrollId is missing;],
called from sub Elasticsearch::Transport::try {...}
at .../Try/Tiny.pm line 83. With vars: {'body' =>
'ActionRequestValidationException[Validation Failed: 1: scrollId is missing;]',
'request' => {'path' => '/_search/scroll','serialize' => 'std',
'body' => 'c2Nhbjs1OzE3MjU0NjM2MjowakFELUU3VFFibTJIZW1ibUo0SUdROzE3MjU0NjM2NDowakFELUU3VFFibTJIZW1ibUo0SUdROzE3MjU0NjM2MTowakFELUU3VFFibTJIZW1ibUo0SUdROzE3MjU0NjM2MDowakFELUU3VFFibTJIZW1ibUo0SUdROzE3MjU0NjM2MzowakFELUU3VFFibTJIZW1ibUo0SUdROzE7dG90YWxfaGl0czoxNDQ7',
'method' => 'GET','qs' => {'scroll' => '1m'},'ignore' => [],
'mime_type' => 'application/json'},'status_code' => 500}
What am I doing wrong? I'm using Elasticsearch 0.75 and Elasticsearch-Cxn-NetCurl 0.02, and Perl 5.18.1.
I finally got it working with the newer Search::Elasticsearch official client. Here's the short version:
#! /usr/bin/perl
use strict;
use warnings;
use 5.010;
use Search::Elasticsearch ();
my $es = Search::Elasticsearch->new(
cxn_pool => 'Static::NoPing',
nodes => 'api.metacpan.org:80',
);
my $scroller = $es->scroll_helper(
index => 'v0',
type => 'release',
search_type => 'scan',
scroll => '2m',
size => 100,
body => {
fields => [qw(author archive date)],
query => { range => { date => { gte => '2015-02-01T00:00:00.000Z' } } },
},
);
while (my $hit = $scroller->next) {
my $field = $hit->{fields};
say "#$field{qw(date author archive)}";
} # end while $hit
Note that the records are not sorted when you do a scrolled search. I wound up dumping the records into a temporary database and sorting them locally. The updated script is on GitHub.
I don't have a direct answer, but I might have an approach to trouble shooting:
I followed your link to the Elasticsearch::Client and found a scroll() method:
https://metacpan.org/pod/Elasticsearch::Client::Direct#scroll
This method takes scroll and scroll_id as parameters. scroll is the number of minutes that you can keep calling the scroll method before the search expires. scroll_id is a marker to the place where the last call to scroll() ended.
$results = $e->scroll(
scroll => '1m',
scroll_id => $id
);
Elasticsearch::Scroll is an object oriented wrapper around scroll() which hides scroll and scroll_id.
I would run perl -d on your script, and step in to $scroller->next and follow that as far down the rabbit hole as you can. Something in there is trying a search which should be populating scroll_id or scrollId and is failing.
My description here is admittedly pretty rough... I ran across an accurate description of what the scroll id is and does during my googling, but I can't seem to find it again.