I have the following script at �://192.168.1.3/homeworks/hw10/testcookie.cgi:
#!/usr/bin/perl -wT
use CGI qw(:standard);
use CGI::Carp qw(warningsToBrowser fatalsToBrowser);
use CGI::Cookie qw(cookie);
use strict;
use warnings;
sub makecookie {
return my $cookie = cookie(-name=>'hw10',
-value=>shift,
-expires=>shift,
-path=>'/hw10/testcookie.cgi',
-domain=>'192.168.1.3',
-secure=>1);
}
my $cgi = CGI->new();
my $cookie = makecookie("192.168.1.3",'+3d');
print STDOUT $cgi->header(-cookie=>$cookie);
print STDOUT $cgi->start_html("Test Cookie");
print STDOUT "<h1>TEST</h1>";
print STDOUT $cgi->end_html();
I checked with Live headers that something was sent:
GET /homeworks/hw10/testcookie.cgi HTTP/1.1
Host: 192.168.1.3
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:34.0) Gecko/20100101 Firefox/34.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
HTTP/1.1 200 OK
Date: Wed, 24 Dec 2014 16:11:24 GMT
Server: Apache/2.2.22 (Debian)
Set-Cookie: hw10=192.168.1.3; domain=192.168.1.3; path=/hw10/testcookie.cgi; expires=Sat, 27-Dec-2014 16:11:24 GMT; secure
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 258
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=ISO-8859-1
----------------------------------------------------------
but firefox says there is no cookie associated with the site. Why does firefox reject the cookie?
The path component of the URL is
/homeworks/hw10/testcookie.cgi
yet you're trying to set a cookie for
/hw10/testcookie.cgi
The latter is not contained by the former, so that's an error. You might want to use
-path => $cgi->url( -absolute => 1 )
instead of
-path => '/hw10/testcookie.cgi'
This may not be the only error. You didn't provide the URL you requested, so I don't know if you're using HTTP or HTTPS. You'll have problems if you're using HTTP because you specified the cookie should only be provided over secure connections.
Related
Using Powershell 5.0.10586.117 I want to request a specific web-site (on IIS 8.5) containing a form to log-in. Here is the specific code:
$Url = Invoke-Webrequest -Uri "https://not-work/hello.aspx" -SessionVariable SesVar
$UrlForm = $Url.Forms[0]
When doing so Powershell hangs and never returns anything as long as I do not interrupt manually.
Using
$Url = Invoke-Webrequest -Verbose -Uri "https://not-work/hello.aspx" -SessionVariable SesVar
I can see the following error:
VERBOSE: Get https://not-work/hello.aspx with 0-byte payload.
Running the same website in Google Chrome 55.0.2883.87 it works. Using an older version of the application code on a different (likely older IIS) it works too in Powershell.
So I am wondering, if this issue has to do with this bug in Powershell?
Note: Even when using
$Url = Invoke-Webrequest -UseBasicParsing -Verbose -Uri "https://not-work/hello.aspx" -SessionVariable SesVar
it fails.
For the sake of completeness, curl also gives an HTTP status code 200 (like in Chrome), but there are some differences in comparison to the working version:
Non-working website
C:\Tools\curl>curl -k https://not-work/hello.aspx -I HTTP/1.1 200 OK
Cache-Control: private
Content-Length: 12671
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/8.5 X-AspNet-Version: 4.0.30319
Set-Cookie: __AntiXsrfToken=d438e79fe7004ad09d950a3b8b4e507d; path=/; HttpOnly
Set-Cookie: .ASPXAUTH=; expires=Mon, 11-Oct-1999 22:00:00 GMT; path=/; HttpOnly
X-Powered-By: ASP.NET
Date: Thu, 12 Jan 2017 13:57:33 GMT
Working website
C:\Tools\curl>curl -k https://works-well/hello.aspx -I HTTP/1.1 200 OK
Cache-Control: private Content-Length: 12306 Content-Type: text/html; charset=utf-8
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
Server: WWW Server/1.1 X-XSS-Protection: 1; mode=block
Set-Cookie: __AntiXsrfToken=6142de1c5d474da780530da39fea033f; path=/;HttpOnly; Secure
Date: Thu, 12 Jan 2017 15:56:47 GMT
By the way, in Internet Explorer (IE) 11.0.9600.18537 objects like images are not loaded (as I believe Powershell works "under the hood" using IE DLLs) and the "hourglass" is running permanently too next to the tab of the requested URI.
So again my question: Has this issue to do with this bug in Powershell or could it be something else?
I'm troubleshooting my own situation of the same nature. My website seems to have changed a few things around. One of which is the FORMS.
Can you confirm that your website has a FORMS option?
EX.
#GET WEBSESSION
$url = "https://myaccount.nytimes.com/"
$r = Invoke-WebRequest -Uri $url -SessionVariable ws -UseBasicParsing
return $r
Notice that I use -UseBasicParsing. For my nytimes example I have to. This is because nytimes is using JS and that doesn't seem to return well for PS. Removing -UseBasicParsing will result in my WebRequest hanging.
You will notice that with the above return the 'FORMS : ' is null.
Right now I do not have a solution. Other than, my website clearly requires credentials and I clearly need to provide. So I'm now writing a muttery of different attempts using -Body $bodyParam #{} and -Headers $headerParams #{}. So far it's to no avail.
Simply put, PS doesn't seem to behave well with JS type websites and FORMS seems to be used for only a specific set of website, not for all type sites.
Hopefully someone more skilled can assist cause this is racking my brain.
I currently expand my test suite to increase the test coverage. I want to test my controller and the html output that it renders, but I found a problem in using delete methods. Let me explain it in an example.
I have a route:
$r->delete('/backups/:id')
->to('backup#delete_backup')
->name('backup_delete');
that points to the following function in the backup controller:
sub delete_backup {
my $self = shift;
my $id = $self->param('id');
if ( something ) {
$self->flash( msg => "Backup id $id deleted!" );
}
else{
$self->flash( msg => "Cannot delete, backup id $id not found!" );
}
$self->redirect_to($self->url_for('backup_index'));
}
where the method that handles the route backup_index just displays the $msg and shows few other irrelevant data.
Next, I want to test this method, so I write a test:
$t_logged_in->ua->max_redirects(3);
my $page = $t_logged_in->app->url_for( 'backup_delete', id => $backup_id );
$t_logged_in->delete_ok($page)
->status_isnt( 404, "Checking: 404 $page" )
->status_isnt( 500, "Checking: 500 $page" );
The test is passed. But now, I want to check if the text is correct on the web page that is shown after redirecting. So I do the following:
$t_logged_in->ua->max_redirects(3);
my $page = $t_logged_in->app->url_for( 'backup_delete', id => $backup_id );
$t_logged_in->delete_ok($page)
->status_isnt( 404, "Checking: 404 $page" )
->status_isnt( 500, "Checking: 500 $page" )
->content_unlike(qr/Cannot delete,/i)
->content_like(qr/deleted/i);
The test fails. It fails because the content is empty, so the matching is done as there were:
'' =~ /deleted/i;
'' !~ /Cannot delete,/i;
and this is of course false in both cases. Of course, in the browser, the redirects work perfectly and I see everything as designed in the test. I can change the method to POST or GET but I wanted to make the routing properly in the way an API would be designed.
Question: how to design the test such that the content can be matched after the redirect?
For those who want to dig deeper, I give links to Github.
Routing definition line 303
Controller Backup function delete_backup line 135
Functional Test of Backup Controller line 53
Sorry no one answered this yet. I used to try to keep a closer eye on stack overflow but I got lazy :-P. This is a very interesting question.
Standard redirects (301/302) use the same verb as the original request UNLESS the original request was a POST Interestingly, this behavior was not really intended. A redirect is supposed to use the same request method as the original, but the POST->GET redirect is so commonly intended that most browsers added this behavior even though it was against the original definition. However some people really wanted a POST to remain a POST and now it depended on the implementation. In fact this got so bad that HTTP added response statuses 307/308 which always redirects as the same http verb (ie POST -> POST), even thought that was already the official behavior of the old ones but that ship had sailed. This means that POST->GET, but DELETE->DELETE is the defacto behavior of the 301/302.
This can be demonstrated in the following one liner ($_ in these one-liners is the controller):
perl -Mojo -E 'del "/delete" => sub { $_->redirect_to("/") }; any "/" => { inline => q[I got a <%= $c->req->method %>] }; app->start' get -r -v -M DELETE /delete
DELETE /delete HTTP/1.1
Host: 127.0.0.1:62690
Accept-Encoding: gzip
User-Agent: Mojolicious (Perl)
Content-Length: 0
HTTP/1.1 302 Found
Date: Sun, 18 Sep 2016 03:25:58 GMT
Server: Mojolicious (Perl)
Location: /
Content-Length: 0
DELETE / HTTP/1.1
Content-Length: 0
User-Agent: Mojolicious (Perl)
Host: 127.0.0.1:62690
Accept-Encoding: gzip
HTTP/1.1 200 OK
Content-Type: text/html;charset=UTF-8
Content-Length: 5
Date: Sun, 18 Sep 2016 03:25:58 GMT
Server: Mojolicious (Perl)
I got a DELETE
You can see that since my / route handles all methods we get a response, however the request was a DELETE. I see that in your app, you redirect to another named route whose url also handles DELETE. This is a source of your confusion as you are actually ending up there in your tests. Your redirect is effectively DELETE /backups/<<id>> -> DELETE /backups, if I'm reading your code correctly.
Now as opposed to the people who wanted to keep POST as POST, what you want here is the opposite, you want to redirect from DELETE to GET because you are trying to show a response that is not the original resource. This is the defined behavior of the little-known 303 response in which any request method is redirected to a GET. Indeed this is what early browsers should have insisted on rather than breaking the 302.
In the following example I modify the response code to 303 explicitly.
$ perl -Mojo -E 'del "/delete" => sub { $_->res->code(303); $_->redirect_to("/") }; any "/" => { inline => q[I got a <%= $c->req->method %>] }; app->start' get -r -v -M DELETE /delete
DELETE /delete HTTP/1.1
User-Agent: Mojolicious (Perl)
Host: 127.0.0.1:62716
Accept-Encoding: gzip
Content-Length: 0
HTTP/1.1 303 See Other
Location: /
Content-Length: 0
Server: Mojolicious (Perl)
Date: Sun, 18 Sep 2016 03:27:19 GMT
DELETE / HTTP/1.1
User-Agent: Mojolicious (Perl)
Accept-Encoding: gzip
Content-Length: 0
Host: 127.0.0.1:62716
HTTP/1.1 200 OK
Content-Type: text/html;charset=UTF-8
Server: Mojolicious (Perl)
Content-Length: 5
Date: Sun, 18 Sep 2016 03:27:19 GMT
I got a DELETE
But whoops, that's still a DELETE! That's because there was a bug in Mojo::UserAgent so that it didn't handle 303s correctly. I'd suggest you still make the change to your code since browsers seem to handle 303s correctly. However since Mojo::UserAgent powers Test::Mojo, your tests can't test that behavior yet. Once fixed you will see it work correctly, as my example does in my local branch:
$ perl -Ilib -Mojo -E 'del "/delete" => sub { $_->res->code(303); $_->redirect_to("/") }; any "/" => { inline => q[I got a <%= $c->req->method %>] }; app->start' get -r -v -M DELETE /delete
DELETE /delete HTTP/1.1
Content-Length: 0
User-Agent: Mojolicious (Perl)
Host: 127.0.0.1:64924
Accept-Encoding: gzip
HTTP/1.1 303 See Other
Date: Sun, 18 Sep 2016 04:59:37 GMT
Location: /
Server: Mojolicious (Perl)
Content-Length: 0
GET / HTTP/1.1
Content-Length: 0
User-Agent: Mojolicious (Perl)
Host: 127.0.0.1:64924
Accept-Encoding: gzip
HTTP/1.1 200 OK
Date: Sun, 18 Sep 2016 04:59:37 GMT
Content-Type: text/html;charset=UTF-8
Server: Mojolicious (Perl)
Content-Length: 12
I got a GET
To see more about redirect response types see https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_Redirection
I Can't set headers in Perl.
print "Expires: Thu, 08 May 2003 08:37:25 GMT\n\n";
print "Content-Type: text/html; charset=windows-1251\n\n";
print "Vary: Accept-Encoding\n\n";
First one works only. Then I have Content-Type: text/x-perl. What is wrong?
I'll assume you're using CGI to connect your web server to Perl. CGI uses a blank line to separate the headers from the response body. Since
print "Expires: Thu, 08 May 2003 08:37:25 GMT\n\n";
prints a blank line after the Expires: header, the remaining print statements are considered part of the body, not headers. You wanted:
print "Expires: Thu, 08 May 2003 08:37:25 GMT\n";
print "Content-Type: text/html; charset=windows-1251\n";
print "Vary: Accept-Encoding\n\n";
I would like to download PDF file from online form. I used the fiddler to find out the header and post fields. However, I cannot get the header from Firefox to match with Powershell using Invoke-webrequest.
Original request
POST http://somesight/SPLeam/EAMDisplayReportWithParamValues.asp HTTP/1.1
Host: someserver
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:27.0) Gecko/20100101 Firefox/27.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://someserver/SPLeam/EAMPickParamValues.asp?plant=01&reportuser=reports&reportuserpw=pw&ReportName=SPLEAM%5CMNT+%2D+MM+Tracking+Rpt%2Erpt
Cookie: ASPSESDFSDIDSSQAQBQC=APHPHMNCHIECSFSSPOIKBDCAM`
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 132
Text0=12%2F1%2F2013&Text1=12%2F31%2F2013&Text2=MESGI&Drop2=MESCI&Text3=01&Text4=&Drop4=&Text5=R&Text6=N&cmdSaveParameters=Run+Report
POWERSHELL
POST http://someserver/SPLeam/EAMPickParamValues.asp?plant=01&reportuser=reports&reportuserpw=pw&ReportName=SPLEAM%5CMNT+%2D+MM+Tracking+Rpt%2Erpt HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT; Windows NT 6.1; en-US) WindowsPowerShell/3.0
Content-Type: application/x-www-form-urlencoded
Host: someserver
Content-Length: 129
Connection: Keep-Alive
cmdSaveParameters=Run%2bReport&Text3=01&Text5=R&Text1=12%252F31%252F2013&Drop2=MI&Text2=MI&Text4=&Text6=N&Text0=12%252F1%252F2013
I run it as
$f = invoke-webrequest -uri $url -UserAgent $useragent -method Post -Body $param
How can I make Powershell's request match what Firefox sent?
.NET's HTTPWebRequest object has separate properties for the Accept and Referer headers; you must manually set these properties instead of trying to add the headers directly.
(These header restrictions exist partly for security purposes; partial trust code cannot set the properties).
use WWW::Curl::Easy;
$curl->setopt(CURLOPT_HEADER,1);
$curl->setopt(CURLOPT_RETURNTRANSFER,1);
$curl->setopt(CURLOPT_URL,"http://example.com/login.php");
$curl->setopt(CURLOPT_POSTFIELDS,"user=usertest&pass=passwdtest");
$curl->perform();
It will printout like this.
How do I get the output into a variable from perform function?
HTTP/1.1 302 Found Cache-Control:
no-cache, must-revalidate Expires:
Sat, 11 Jan 200 05:00:00 GMT
Location: ?cookiecheck=1
Content-type: text/html Date: Thu, 28
Apr 2011 09:15:57 GMT Server:
xxxx/0.1 Content-Length: 0
Connection: Keep-Alive Set-Cookie:
auth=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx;
expires=Sat, 27-Apr-2013 09:15:57 GMT;
path=/; domain=.example.com
I agree with PacoRG that you most likely should look into using a module from the LWP:: space. Since you have more specific needs I would recommend the LWP::UserAgent.
That said if you really need to get something which is being printed to instead be stored in a variable, we can play some games with deeper Perl magic.
# setopt method calls here
## the variable you want to store your data in
my $variable;
{
## open a "filehandle" to that variable
open my $output, '>', \$variable;
## then redirect STDOUT (where stuff goes when it is printed) to the filehandle $output
local *STDOUT = $output;
## when you do the perform action, the results should be stored in your variable
$curl->perform();
}
## since you redirected with a 'local' command, STDOUT is restored outside the block
## since $output was opened lexically (with my), its filehandle is closed when the block ends
# do stuff with $variable here
Perhaps WWW::Curl::Easy has a better way of doing this, since I don't know that module's commands I have provided you with a hack that will do what you need.
What are you trying to do? May be LWP::Simple is what you need...
Over 11 years later, I hope this helps:
my $response;
sub write_data($$$$) {
$response = shift;
return length($response);
}
# ... curl setup
$curl->setopt(CURLOPT_WRITEFUNCTION,\&write_data);
# ... curl perform
print "$response\n"