Odd cgi behaviour - perl

I am getting some very annoying behaviour from my perl cgi scripts running under apache.
I get referer information added on the end of simple print statements, and it's driving me nuts.
[Sun Feb 20 21:34:47 2011] [error] [client xx] ruid: 48, referer: http://www.x.com/
[Sun Feb 20 21:34:47 2011] [error] [client xx] euid: 48, referer: http://www.x.com/
[Sun Feb 20 21:34:47 2011] [error] [client xx] test, referer: http://www.x.com/
[Sun Feb 20 21:34:47 2011] [error] [client xx] Premature end of script headers: test.cgi, referer: http://www.x.com/
This only seems to happen when the url is reached by navigating from another page (hence having a referer yes). The above apache log output was produced with the below incredibly simple depo script:
#!/usr/bin/perl -w
use strict;
use warnings;
use CGI;
my $q = CGI->new;
print STDERR "ruid: $<\n";
print STDERR "euid: $>\n";
print STDERR "test\n";
Anyone seen this before? It feels like an apache setting i need to turn off.
Thanks
Matt

Take a look at your apache config files (httpd.conf and friends) and find the CustomLog directive which is used by your error log to see which LogFormat it uses, then modify that LogFormat (or create a new one) to remove %{Referer} from the list of fields to include in the log messages. (And don't forget to reload the apache config after changing it, of course.)

Related

Writing a filter for a regex that works in fail2ban-regex on the command line

I have entries like these in apache2 error.log
[Thu Jan 12 09:18:51.078445 2023] [core:error] [pid 47992] [client 152.89.196.211:53158] AH10244: invalid URI path (/cgi-bin/.%2e/.%2e/.%2e/.%2e/bin/sh)
[Wed Jan 11 06:01:09.820582 2023] [core:error] [pid 30833] [client 185.225.74.55:39856] AH10244: invalid URI path (/cgi-bin/.%%%%32%%65/.%%%%32%%65/.%%%%32%%65/.%%%%32%%65/.%%%%32%%65/bin/sh)
[Wed Jan 11 17:16:49.643509 2023] [core:error] [pid 41882] [client 152.89.196.211:52746] AH10244: invalid URI path (/cgi-bin/.%2e/.%2e/.%2e/.%2e/bin/sh)
I got this to work on the command line:
fail2ban-regex test.log '.*\[client <HOST>:\d+\] AH10244.*$'
Every time I try to stick the regex into a .conf file like so:
[Definition]
failregex = .*\[client <HOST>:\d+\] AH10244.*$
ignoreregex =
fail2ban complains:
Running tests
=============
Use failregex line : filter.conf
ERROR: No failure-id group in 'filter.conf'
I've looked in the man pages and online but I can't find an explanation of what this message is trying to say, or how to fix it.
The Questions
How do I wrap a .conf file around this regex?
What does that error mean?
Could I (how would I) use the pre-defined stuff in apache-common.conf to make this regex more robust?
This fixed it:
fail2ban-regex test.log ./filter.conf
I had my test files (test.log and filter.conf) in my home dir. When I ( from the home dir ) issued the command:
fail2ban-regex test.log filter.conf
I assumed that I was referencing ./test.log and ./filter.conf but I think that fail2ban was looking in the filter.d/ folder to try to find filter.conf.
I found that if filter.conf was in the /etc/fail2ban/filter.d/ folder, then fail2ban-regex test.log filter.conf succeeded.

Getting error in perl Can't find string terminator "EOF" anywhere before EOF at /var/www/sandeep/testeof.cgi line 2

iam using perl to print some data but it is giving me error as Can't find string terminator "EOF" anywhere before EOF at
The code is:
#!/usr/local/bin/perl -w
print <<EOF;
hello
EOF
ERROR:
[Mon May 23 11:32:12 2016] [error] [client 192.168.10.117] Directory index forbidden by Options directive: /var/www/, referer: http://192.168.10.100/
[Mon May 23 11:32:12 2016] [error] [client 192.168.10.117] malformed header from script. Bad header=hello: testeof.cgi
[Mon May 23 11:32:18 2016] [error] [client 192.168.10.117] Directory index forbidden by Options directive: /var/www/, referer: http://192.168.10.100/
[Mon May 23 11:32:18 2016] [error] [client 192.168.10.117] Can't find string terminator "EOF" anywhere before EOF at /var/www/sandeep/testeof.cgi line 2.
[Mon May 23 11:32:18 2016] [error] [client 192.168.10.117] Premature end of script headers: testeof.cgi
I tried putting EOF in single quotes (print <<'EOF';) as shown in this answer, but the error is same. Printing by this method is working in otherr files in the same directory.
I also referred this question (Why am I getting β€œCan't find string terminator ”'β€œ anywhere before EOF at -e line 1” when I try to run a Perl one-liner on Windows?) but in that question OP is using different method to print and iam using linux(UBUNTU).
Please guide where I am doing wrong?
Your code is correct.
Ensure that your EOF is on a single line with no spaces around it.
Use quotes to designate your terminator:
print <<_EOF_;
hi $$
_EOF_
print <<"_EOF_";
hi $$
_EOF_
Both are exactly the same (print hi 12345 where 12345 is the process id of the current process), but the second one is more clear compared to single quotes:
print <<'_EOF_';
hi $$
_EOF_
This one will print hi $$, because no variable replacing is done with singles quotes.
Web-script always require a header (at least for Apache). Add this line as the first output line to get rid of the malformed header from script error:
print "Content-type: text/html\r\n\r\n";

WWW::Mechanize::PhantomJS code in Mojolicious Lite script doesn't work when running in background mode

I have this very simple Mojolicious Lite script:
#!/usr/bin/env perl
use v5.10;
use WWW::Mechanize::PhantomJS;
use Mojolicious::Lite;
my $mech = WWW::Mechanize::PhantomJS->new();
$mech->get('http://stackoverflow.com/');
get '/test' => sub {
my $c = shift;
$mech->get("https://stackoverflow.com/questions");
$c->render(template => 'activity');
};
app->secrets(['test secret']);
app->start;
__DATA__
## activity.html.ep
<!DOCTYPE html>
<html>
<head><title>Test</title></head>
<body><h2>Test</h2></body>
</html>
When I start it using hypnotoad in foreground mode (hypnotoad -f ./script.pl), and access /test url -- I get my test page, and clean logs:
[Fri Dec 11 18:00:23 2015] [info] Listening at "http://*:8080"
[Fri Dec 11 18:00:23 2015] [info] Manager 3011 started
[Fri Dec 11 18:00:23 2015] [info] Creating process id file "/home/username/pc_activity/demo_site/hypnotoad.pid"
When I start it using background mode (hypnotoad ./script.pl), and access /test url -- I get "something went very wrong" error page with throwing up raptor.
[Fri Dec 11 17:58:07 2015] [info] Listening at "http://*:8080"
[Fri Dec 11 17:58:07 2015] [info] Manager 2964 started
[Fri Dec 11 17:58:07 2015] [info] Creating process id file "/home/username/pc_activity/demo_site/hypnotoad.pid"
[Fri Dec 11 17:58:14 2015] [error] Error while executing command: get: Server returned error message Can't connect to localhost:8910
Connection refused at /usr/local/share/perl/5.18.2/LWP/Protocol/http.pm line 47, <DATA> line 49.
instead of data at /usr/local/share/perl/5.18.2/Selenium/Remote/Driver.pm line 310.
It turns out the localhost:8910 is default settings for PhanomJS to run at in webdriver mode. Which it isn't on my machine. But even if I start it, and then access the URL, I still get an error:
[Fri Dec 11 18:41:01 2015] [error] Error while executing command: get: Server returned error message Variable Resource Not Found - {"headers":{"Accept":"application/json","Connection":"TE, close","Content-Length":"45","Content-Type":"application/json; charset=utf-8","Host":"localhost:8910","TE":"deflate,gzip;q=0.3","User-Agent":"libwww-perl/6.15"},"httpVersion":"1.1","method":"POST","post":"{\"url\":\"https://stackoverflow.com/questions\"}","url":"/session/98799e80-a060-11e5-8907-0b365878087d/url","urlParsed":{"anchor":"","query":"","file":"url","directory":"/session/98799e80-a060-11e5-8907-0b365878087d/","path":"/session/98799e80-a060-11e5-8907-0b365878087d/url","relative":"/session/98799e80-a060-11e5-8907-0b365878087d/url","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/session/98799e80-a060-11e5-8907-0b365878087d/url","queryKey":{},"chunks":["session","98799e80-a060-11e5-8907-0b365878087d","url"]}} instead of data at /usr/local/share/perl/5.18.2/Selenium/Remote/Driver.pm line 310.
I guess, I don't understand, why does it run in foreground mode, and not in background mode. And then, what can I do to make it run in background mode?
The reason it will not run in background mode is because of threading. After you start hypnotoad in background it spawns some processes.
664161507 46028 1 0 10:34AM ?? 0:00.01 script.pl
664161507 46029 46028 0 10:34AM ?? 0:00.08 script.pl
664161507 46030 46028 0 10:34AM ?? 0:00.01 script.pl
664161507 46031 46028 0 10:34AM ?? 0:00.01 script.pl
664161507 46032 46028 0 10:34AM ?? 0:00.01 script.pl
These will not have access to the PhantomJS you created in the parent process. I didn't look into how this communication is done, but if you want to share PhantomJS for your workers you need to make it a separate service.
If you however want to have a PhantomJS for each request you can initialize it in the request, but I don't necessarily encourage this approach:
get '/test' => sub {
my $c = shift;
my $mech = WWW::Mechanize::PhantomJS->new();
$mech->get('http://stackoverflow.com/');
$mech->get("https://stackoverflow.com/questions");
$c->render(template => 'activity');
};

CGI script not running, Internal Server Error (500) Error

I am trying to run a cgi script, its a long script so i wont paste here but it works on my other servers but for some reason when I try to run it on my new server I get Internal server error (500), and when I check the apache log here is what I get :
[Fri Mar 30 08:38:29 2012] [error] [client 199.212.76.124] (2)No such file or directory: exec of '/var/www/cgi-bin/test.cgi' failed
[Fri Mar 30 08:38:29 2012] [error] [client 199.212.76.124] Premature end of script headers: test.cgi
[Fri Mar 30 08:38:29 2012] [error] [client 199.212.76.124] File does not exist: /var/www/htdocs/favicon.ico
[Fri Mar 30 08:41:11 2012] [error] [client 199.212.76.124] (2)No such file or directory: exec of '/var/www/cgi-bin/test.cgi' failed
[Fri Mar 30 08:41:11 2012] [error] [client 199.212.76.124] Premature end of script headers: test.cgi
[Fri Mar 30 08:41:11 2012] [error] [client 199.212.76.124] File does not exist: /var/www/htdocs/favicon.ico
[Fri Mar 30 08:41:12 2012] [error] [client 199.212.76.124] (2)No such file or directory: exec of '/var/www/cgi-bin/test.cgi' failed
[Fri Mar 30 08:41:12 2012] [error] [client 199.212.76.124] Premature end of script headers: test.cgi
[Fri Mar 30 08:41:12 2012] [error] [client 199.212.76.124] File does not exist: /var/www/htdocs/favicon.ico
[Fri Mar 30 08:41:15 2012] [error] [client 199.212.76.124] (2)No such file or directory: exec of '/var/www/cgi-bin/test.cgi' failed
[Fri Mar 30 08:41:15 2012] [error] [client 199.212.76.124] Premature end of script headers: test.cgi
[Fri Mar 30 08:41:15 2012] [error] [client 199.212.76.124] File does not exist: /var/www/htdocs/favicon.ico
[Fri Mar 30 08:41:20 2012] [error] [client 199.212.76.124] (2)No such file or directory: exec of '/var/www/cgi-bin/upload.cgi' failed
[Fri Mar 30 08:41:20 2012] [error] [client 199.212.76.124] Premature end of script headers: upload.cgi
[Fri Mar 30 08:41:20 2012] [error] [client 199.212.76.124] File does not exist: /var/www/htdocs/favicon.ico
[Fri Mar 30 08:41:21 2012] [error] [client 199.212.76.124] (2)No such file or directory: exec of '/var/www/cgi-bin/upload.cgi' failed
[Fri Mar 30 08:41:21 2012] [error] [client 199.212.76.124] Premature end of script headers: upload.cgi
[Fri Mar 30 08:41:21 2012] [error] [client 199.212.76.124] File does not exist: /var/www/htdocs/favicon.ico
[Fri Mar 30 08:41:29 2012] [error] [client 199.212.76.124] (2)No such file or directory: exec of '/var/www/cgi-bin/upload.cgi' failed, referer: http://bla.com
[Fri Mar 30 08:41:29 2012] [error] [client 199.212.76.124] Premature end of script headers: upload.cgi, referer: http://bla.com
I have tried everything, chmod the files, reinstall perl modules, rechecking the configuration etc!!!! Can not figure out what is wrong!!! but I can guarantee that the script is not broken!
Regards
Try running:
perl -c /var/www/cgi-bin/upload.cgi
Quote from perldoc perlrun:
-c causes Perl to check the syntax of the program and then exit
without executing it. Actually, it will execute "BEGIN",
"UNITCHECK", "CHECK", and "use" blocks, because these are
considered as occurring outside the execution of your program.
"INIT" and "END" blocks, however, will be skipped.
Run your script with perl -w /var/www/cgi-bin/upload.cgi.
I should write out an HTTP header
Content Type: text/html
[blank line]
If not, your script not working well. Maybe the other server pass you a variable and your script won't create a proper header any more.
Paste this code in the very to of your script. If your script looks bad, but working, you found out the bug.print("Content Type: text/html\n\n");
print("Content Type: text/html\n\n");
regards,
Other things to check would be
the location of perl in the first like (#!/usr/..)
dos2unix the file, no asci characters
other cgi scripts in different locations to eliminate cgi setup
directory perms? Not sure if that effects it

How can I create and maintain sessions in series of Perl CGI scripts?

I am new to CGI-Perl and sessions. I have a problem running the following code.
I have four files
1. Session.html or Session.pm
2. name.pl
3. hobbies.pl
4. job.pl
The Session.pm is place in /var/www/html folder and the rest of the files are placed in /var/www/cgi-bin/session folder.
I have a basic html file(Session.html) which has a link to perl script it is giving the following error
[Mon Jan 04 14:21:24 2010] [error] [client ::1] Options ExecCGI is off in this directory: /var/www/html/name.pl, referer: http://localhost/Session.html
[Mon Jan 04 14:21:29 2010] [error] [client ::1] Options ExecCGI is off in this directory: /var/www/html/name.pl, referer: http://localhost/Session.html
[Mon Jan 04 14:22:42 2010] [error] [client ::1] (13)Permission denied: exec of '/var/www/cgi-bin/session/name.pl' failed, referer: http://localhost/Session.pm
[Mon Jan 04 14:22:42 2010] [error] [client ::1] Premature end of script headers: name.pl, referer: http://localhost/Session.pm
Please help me out in executing the above example. Please give me the steps to be followed from the start.
... Options ExecCGI is off in this directory: /var/www/html/name.pl, ...
This error points to trying to run name.pl in /var/www/html/ instead of /var/www/cgi-bin/session/
Is the Session.html you posted correct because it doesn't seem to tally with this error?
... (13)Permission denied: exec of '/var/www/cgi-bin/session/name.pl' failed, ...
The name.pl cannot be run because it doesn't have execution rights set (on Linux/Unix you need to chmod a+x name.pl).
You may need to show what Session.pm does (it doesn't look like you using the CPAN module Session here).
On a general note I would consider simplifying what you doing down to a simple HTML & CGI script first and get that working. Then start looking into sessions etc.
Check out Ovid's CGI Tutorial for some sage advice on the Perl/CGI subject.
/I3az/