Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I'm trying to replicate the following command line curl:
curl -F file=#myfile.csv 'https://myserver/api/import/data_save.html'
Does anyone have an example in either www::curl or lwp perhaps? I have been trying all day and it's not even worth me posting my tries at this point, it would only confuse things. Thank you!!!
I think you are asking how to submit a form with a file field named file populated with the contents of file myfile.csv.
use LWP::UserAgent qw( );
my $ua = LWP::UserAgent->new();
my $response = $ua->post('https://myserver/api/import/data_save.html',
Content_Type => 'form-data',
Content => [
file => [ 'myfile.csv' ],
],
);
die($response->status_line)
if !$response->is_success;
The arguments to $ua->post are documented in HTTP::Request::Common.
Well, let's unpack what that curl command actually does, because -F implies a lot.
It sets the HTTP method of the request to POST.
It sets the Content-Type of the request to multipart/form-data.
It composes a MIME multipart request body with the form metadata indicating that the "file" input is being provided from a file named "myfile.csv", as well as a part containing the actual file contents.
Here's how to replicate that using WWW::Curl:
use WWW::Curl::Easy;
use WWW::Curl::Form;
my $curl = WWW::Curl::Easy->new;
my $form = WWW::Curl::Form->new;
$form->formaddfile('myfile.csv', 'file', "multipart/form-data");
$curl->setopt(CURLOPT_URL, 'https://myserver/api/import/data_save.html');
$curl->setopt(CURLOPT_HTTPPOST, $form);
my $response = $curl->perform;
But LWP is probably easier; see ikegami's answer.
Related
Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 3 years ago.
Improve this question
I get a guzzle error as above , when i am sending the following code from a drupal 8 site:
$url="https://mywebsite.com/index.php";
$client = \Drupal::httpClient();
$client->setDefaultOption( array('verify' ,false));
$post_data = $form_state->cleanValues()->getValues();
$request = $client->post($url, $post_data);
This line was causing it to think this was the URL. When I took it out it works okay
$client->setDefaultOption(array('verify', false));
Basically, it should work (because the $url obviously is a string).
If it doesn't, check the stack trace of the exception (is it this call, or maybe another?).
And provide the Guzzle version please.
POSTDATA is not the correct answer. I have read the docs and still don't see how I can get the data.
I want to receive this request:
POST /cgi-bin/myscript.cgi HTTP/1.1
Host: myhost.com
Content-Length: 3
Content-Type: application/x-www-form-urlencoded
255
and have the server respond
You sent the string "255"
Please assist, I am a Perl beginner and have gotten a bunch of seemingly wrong and useless answers to this seemingly simple request.
CGI will automatically parse form data, so you need to hide that what you got is form data (or at least claims to be).
use CGI qw( );
$ENV{CONTENT_TYPE} = 'application/octet-stream';
my $cgi = CGI->new();
my $post_data = $cgi->param('POSTDATA');
Better solution: Have the requester use a correct content type (e.g. application/octet-stream), or have the requester actually send form data (e.g. data=255).
Unique solution for me, was change of ContentType on client's petition to 'application/octet-stream'
Module CGI CPAN says:
If POSTed data is not of type application/x-www-form-urlencoded or
multipart/form-data, then the POSTed data will not be processed, but
instead be returned as-is in a parameter named POSTDATA.
So if you can't change on clients petition to other ContentType, it won't be processed.
CGI (in recent versions at least) will stuff incorrectly encoded x-www-form-urlencoded params into a parameter named keywords. Better to send a proper content type though, then the POSTDATA works exactly as the docs say:
If POSTed data is not of type application/x-www-form-urlencoded or
multipart/form-data, then the POSTed data will not be processed...
use strictures;
use CGI::Emulate::PSGI;
use Plack::Test;
use HTTP::Request::Common;
use Test::More;
my $post = POST "/non-e-importa",
"Content-Length" => 5,
"Content-Type" => "application/x-www-form-urlencoded",
Content => "ohai\n";
my $cgis = CGI::Emulate::PSGI->handler( sub {
use CGI "param", "header";
my $incorrectly_encoded_body = param("keywords");
print header("text/plain"), $incorrectly_encoded_body;
});
test_psgi $cgis, sub {
my $cb = shift;
my $res = $cb->($post);
is $res->content, "ohai", "Soopersek437 param: keywords";
};
done_testing();
__END__
prove so-16846138 -v
ok 1 - Soopersek437 param: keywords
1..1
ok
All tests successful.
Result: PASS
I'm trying to get a html source of a webpage using the Perl "get" function. I have written the code 5 months back and it was working fine, but yesterday I made a small edit, but it failed to work after that, no matter how much I tried.
Here is the code I tried.
#!usr/bin/perl
use strict;
use warnings;
use LWP::Simple;
my $link = 'www.google.com';
my $sou = get($link) or die "cannot retrieve code\n";
print $sou;
The code works fine , but its not able to retrieve the source, instead it displays
cannot retrieve code
my $link = 'http://www.google.com';
This might be a bit late,
I have been struggling with the same problem and I think I have figured why this occurs. I usually web-scrape websites with python and I have figured out that It is ideal to include some extra header info to the get requests.This fools the website into thinking the bot is a person and gives the bot access to the website and does not invoke a 400 bad request status code.
So I applied this thinking to my Perl script, which was similar to yours, and just added some extra header info. The result gave me the source code for the website with no strugle.
Here is the code:
#!/usr/bin/perl
# This line specifies the LWP version and if not put in place the code will fail.
use LWP 5.64;
# This line defines the virtual browser.
$browser = LWP::UserAgent->new;
# This line defines the header infomation that will be given to the website (eg. google) incase the website invokes a 400 bad request status code.
#ns_headers = (
'User-Agent' => 'Mozilla/4.76 [en] (Win98; U)',
'Accept' => 'image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */*',
'Accept-Charset' => 'iso-8859-1,*,utf-8',
'Accept-Language' => 'en-US',
);
# This line defines the url that the user agent will browse.
$url = 'https://www.google.com/';
# This line is used to request data from the specified url above.
$response = $browser->get($url, #ns_headers) or die "cannot retrieve code\n";;
# Decodes responce so the HTML source code is visable.
$HTML = $response->decoded_content;
print($HTML);
I have LWP::Useragent as this has the ability for you to add extra header infomation.
I hope this helped,
ME.
PS. Sorry if you already have the answer for this, just wanted to help.
My colleagues and I are maintaining and developing a Perl web-project that works via mod_perl.
Now we are going through a major legacy code refactoring in which we have implemented some sort of an MVC pattern.
Among other things, my task is to make sure that all HTTP response headers are processed and sent back to the browser inside the main controller. For example, if a redirect is required, a page handler throws an exception, then the main controller catches it and generates the corresponding headers.
It all looked well until I started to implement cookie handling. Before that our code just printed cookie headers to output when it was required, like so:
# $response is an instance of the CGI class
print $response->redirect(
-uri => "/some_uri/",
-cookie => $response->cookie(
-name => 'user_id',
-value => $user->{'id'},
-path => '/', -expires => '+1M'));
And now I want the $response object to store that information, so I can later send all headers together. I thought that it would go something like that:
sub page_handler {
# ...
$response->cookie(-name => 'user_id',
-value => $user->{'id'},
-path => '/', -expires => '+1M');
return;
}
# And then, inside the controller
sub controller {
# ...
# the same $response instance
print $response->header();
print $output;
# ....
exit();
}
But it seems that the CGI class object doesn't store all headers that it creates with the header method. Some headers seem to persist, while others do not, here is what I get in re.pl:
$ use CGI;
$ my $response = CGI->new();
$CGI1 = CGI=HASH(0xa6efba0);
$ $response->header();
Content-Type: text/html; charset=ISO-8859-1
$ $response->header(-type => 'text/plain', -charset => 'UTF-8', -status => '200 OK');
Status: 200 OK
Content-Type: text/plain; charset=UTF-8
$ $response->header();
Content-Type: text/html; charset=UTF-8
I expected the last output to be either the same as the previous one, or the same as the first one, where I have not yet set any headers. I did not expect it to change partially.
That is why I ask my question: Why does some header information in a CGI.pm object persist while another does not?
Am I using the object incorrectly? Is there a way I could use it the way I intended to?
PS: Sorry for the long question, I wanted to make sure you understand what I want to do.
PPS: Also, I know that many people around here recommend going away from CGI and using Catalyst. This is, I am afraid, not an option right now, because we have too much legacy code, and we are hoping to get away from mod_perl altogether. This is required only for a certain feature.
To answer your question, the header method doesn't store any information, nothing is persistent.
With your example of the header 'object' persisting, reading TFM helps:
The -charset parameter can be used to control the character set sent to the browser. If not provided, defaults to ISO-8859-1. As a side effect, this sets the charset() method as well. [emphasis mine]
After you call header with some parameters, then call it as default, the only thing to 'persist' is the character set.
For your cookie problem, i think you'd have to store $response->cookie(); somewhere. TFM doesn't say that the cookie() sub stores the data anywhere, it just says that it creates a cookie.
I agree with Sinan though - throwing exceptions is crazy talk, especially to cover CGI.pm's redirect sub. I'd rethink that one. Or go completely the other way and write the whole webapp only using exception handling - there's be some good laughs along the way :o)
I'm trying to post from one of my subroutines in Perl a request to a Java based controller.
But I'm not getting any kind of response back. I know the Java code works file because I can get a response if I post to it from a HTML form.
This is my Perl code:
use HTTP::Request::Common;
my $ua = LWP::UserAgent->new;
my $response = $ua->request(POST 'http://testserver/testing.nc',
Content_Type => 'form-data',
Content => [
method => 'submit',
ftp_server => 'ftp.localhost',
ftp_user => 'testuser',
ftp_password => 'testpass',
remote_path => '/home/files',
port => 22,
file_to_upload => ["$file"]
]);
Is there something wrong with this code?
Posted data must be of type multipart/form-data.
Edit: OK, so it turns out, specifying form-data is enough as mentioned in the HTTP::Request::Common docs:
The POST method also supports the multipart/form-data content used for Form-based File Upload as specified in RFC 1867. You trigger this content format by specifying a content type of form-data as one of the request headers.
However, to use HTTP::Request::Common::POST the way you are using, you will need to import POST:
use HTTP::Request::Common qw(POST);
or use $ua->post:
The post(...) method of LWP::UserAgent exists as a shortcut for $ua->request(POST ...).
You can make your life easier by using WWW::Mechanize. See also this upload example.