LWP Won't Run in CGI Script - perl

Question

I have a CGI script to load publications from BibBase:
#!/usr/bin/perl
use LWP::UserAgent;
my $url = 'https://bibbase.org/show?bib=http://www.example.com/pubs.bib';
my $ua = LWP::UserAgent->new;
my $can_accept = HTTP::Message::decodable;
my $response = $ua->get($url, 'Accept-Encoding' => $can_accept);
print "Content-type: text/html\n\n";
print $response->decoded_content;
(This is copied from BibBase with the exception that the URL is hard-coded.)
I have three webservers running RHEL7 and Apache 2.4 that are configured the same way by Puppet. On all three I can run the script on the command line and get the expected results:
[root#server1 cgi-bin]# ./bibbase_proxy2.cgi | head
Content-type: text/html
<img src="//bibbase.org/img/ajax-loader.gif" id="spinner" style="display: none;" alt="Loading.." />
<div id="bibbase">
<script type="text/javascript">
var bibbase = {
params: {"bib":"http://www.example.com/pubs.bib","host":"bibbase.org"},
When I try to run the script with CGI, I get three different results:
Server1
Unrecognised protocol tcp at /usr/share/perl5/LWP/Protocol/http.pm line 31.
Server2
Can't connect to bibbase.org:443 System error at /usr/share/perl5/LWP/Protocol/http.pm line 51.
Server3
No http output and the error log says AH01215: Out of memory!.
I can't find anything different between the three servers and I can't figure out why the script works fine on the command line and doesn't work when run as a CGI.
I have selinux in permissive mode and it is logging the outgoing request, so I know the script gets that far:
type=AVC msg=audit(1532465859.921:331235): avc: denied { name_connect } for pid=161178 comm="perl" dest=80 scontext=system_u:system_r:httpd_sys_script_t:s0 tcontext=system_u:object_r:http_port_t:s0 tclass=tcp_socket
For testing, I have set selinux to disabled and restarted the server.

Answer 1

SE-Linux denied the TCP connection.
avc: denied { name_connect }
The default access controls for networking by SELinux are based on the labels assigned to TCP and UDP ports and sockets. For instance, the TCP port 80 is labeled with http_port_t (and class tcp_socket). Access towards this port is then governed through SELinux access controls, such as name_connect and name_bind.
When an application is connecting to a port, the name_connect permission is checked. However, when an application binds to the port, the name_bind permission is checked.
Permissive mode or not, Perl is acting like it was denied a TCP connection. Unrecognised protocol tcp means getprotobyname("tcp") failed inside IO::Socket::IP. That's very, very unusual. One of the ways that can happen is via exactly that SELinux denial.
I'm no SELinux expert, but according to RedHat and Gentoo some SELinux aware applications will ignore the global permissive setting and go it alone. RHEL 7 Apache appears to be one of them. It appears to have its own domain which must be set permissive.
On all three I can run the script on the command line and get the expected results:
There's two reasons for that, and they both have to do with users.
When you run the program you're running as your own user with your own configuration, permissions, and environment variables. In fact, you ran it as root which usually bypasses restrictions. When it runs on the server it runs as a different user, probably the web server user with severe restrictions.
In order to do a realistic test, you need to run it as the same user the web server will. You can use sudo -u for this. For example, if the user is apache...
sudo -u apache ./bibbase_proxy2.cgi
BTW Do not test software as root! Not only is it not going to give you sensible results, but if there's a bug in the software there are no safeguards preventing it from wrecking your system.
The second problem is #!/usr/bin/env perl. That means to run whatever perl is in your PATH. PATH will be different for different users. Running ./bibbase_proxy2.cgi may run with one Perl on the command line and a different one via the web server.
In a server environment, use a hard coded path to Perl like #!/usr/bin/perl.

Answer 2

We tested by rewriting the same script in Python and PHP. Both of them showed error which pointed us in the right direction.
Python urllib2 produced the error
<class 'urllib2.URLError'>: <urlopen error [Errno 16] Device or resource busy>
args = (error(16, 'Device or resource busy'),)
errno = None
filename = None
message = ''
reason = error(16, 'Device or resource busy')
strerror = None
PHP (run as CGI) wouldn't even start:
[Wed Jul 25 15:24:52.988582 2018] [cgi:error] [pid 10369] [client 172.28.6.200:44387] AH01215: PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib64/php/modules/curl.so' - libssh2.so.1: failed to map segment from shared object: Cannot allocate memory in Unknown on line 0
[Wed Jul 25 15:24:52.988980 2018] [cgi:error] [pid 10369] [client 172.28.6.200:44387] AH01215: PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib64/php/modules/dba.so' - libtokyocabinet.so.9: failed to map segment from shared object: Cannot allocate memory in Unknown on line 0
---- Similar lines for all extensions. ----
It appears that RLimitMEM blocks access to shared memory and that is required for opening sockets. I can't find any documentation, but removing that line makes it work.