Using MongoDB 2.4.5 with version 0.702 of the Perl MongoDB driver, I frequently run into exceptions like these:
recv timed out (30000 ms) at ...MongoDB/Cursor.pm line 161.
couldn't get response to throw out at ...MongoDB/Cursor.pm line 161.
missed the response we wanted, please try again at ...MongoDB/Cursor.pm line 161
invalid header received at ...MongoDB/Cursor.pm line 161.
can't get db response, not connected at ...MongoDB/Cursor.pm line 161.
The exceptions are intermittent, and often vanish on the next request (this is a web app). Occasionally, the exceptions will persist over several consecutive requests.
This is a tiny database running the default configuration (no sharding or anything fancy). I've tried using some of the tools listed here and here, but I'm unclear on how to apply them to this situation.
This is all running on Debian 7.1 64-bit. The web server is Mojolicious' hypnotoad 4.07 on perl 5.16.3 running behind apache2.
Can you kindly suggest some tools & strategies for diagnosing the problem? Thanks for your time.
Related
Yesterday we had crash of PostgreSQL 9.5.14 running on Debian 8 (Linux xxxxxx 3.16.0-7-amd64 #1 SMP Debian 3.16.59-1 (2018-10-03) x86_64 GNU/Linux) - Segmentation fault. Database closed all connections and reinitialized itself staying ~1 minute in recovery mode.
PostgreSQL log:
2018-10-xx xx:xx:xx UTC [580-2] LOG: server process (PID 16461) was
terminated by signal 11: Segmentation fault
kern.log:
Oct xx xx:xx:xx xxxxxxxx kernel: [117977.301353] postgres[16461]:
segfault at 7efd3237db90 ip 00007efd3237db90 sp 00007ffd26826678 error
15 in libc-2.19.so[7efd322a2000+1a1000]
According to libc documentation (https://support.novell.com/docs/Tids/Solutions/10100304.html) error code 15 means:
NX_EDEADLK 15 resource deadlock would occur - which does not tell me much.
Could you tell me please if we can do something to avoid this problem in the future? Because this server is of course production one.
All packages are up to date currently. Upgrade of PG is unfortunately not the option. Server runs on Google Compute Engine.
error code 15 means: NX_EDEADLK 15
No, it doesn't mean that. This answer explains how to interpret 15 here.
It's bits 0, 1, 2, 3 set => protection fault, write access, user mode, use of reserved bit. Most likely your postgress process attempted to write to some wild pointer.
if we can do something to avoid this problem in the future?
The only thing you can do is find the bug and fix it, or upgrade to a release of postgress where that bug is already fixed (and hope that no new ones were introduced).
To understand where the bug might be, you should check whether a core dump was produced (if not, do enable them). If you have the core, use gdb /path/to/postgress /path/to/core, and then where GDB command. That will give you crash stack trace, which may allow you to find similar bug reports.
I have a script that uses Net::SSH:Perl module. It is able to ssh to RHEL6.9 hosts but not RHEL7.4 hosts. I get the following error:
Protocol error: expected packet type 91, got 80 at /Net/SSH/Perl/Packet.pm line 221
How do I remedy this?
A simple search for the error message gives this post from 2016 and this bug report from 2016 which both indicate that the issue was fixed in version 2.01 of the module, released in 10/2016. Thus, you are probably using an older version and need to upgrade.
I got a connection test script that is run on my client system and is calling a Perl DBI Proxy server via
my $dbh = DBI->connect("dbi:Proxy:$dbiproxy;dsn=$dsn", "$user","$pass",{ PrintError => 1}) || ($errmsg = "ERROR on connect");
When running the script it will fail stating
connect('hostname=***;port=2001;dsn=dbi:Oracle:***','***',...) failed: Cannot log in to DBI::ProxyServer: Unexpected EOF from server at /usr/local/share/perl/5.10.1/RPC/PlClient.pm line 79.
The log on the DBI Proxy server reads
Wed Dec 15 14:47:47 2015 err, Child died: Storable binary image v2.7 more recent than I am (v2.6) at blib/lib/Storable.pm (autosplit into blib/lib/auto/Storable/thaw.al) line 363, at /usr/lib/perl5/site_perl/5.8.0/RPC/PlServer/Comm.pm line 179
So I checked the Perl and Storable module version numbers of the two systems and the DBI proxy server seems to be rather outdated:
source: perl 5.10.1, storable 2.20, storable 2.7 (binary), Debian 6.0
target: perl 5.8.0 , storable 2.13, storable 2.6 (binary), RHEL 3
The problem is that I can not update the module/perl version of the target system, so I thought I could simply lower the utilized client version by modifying the /usr/local/share/perl/5.10.1/RPC/PlServer/Comm.pm file and change the line
require Storable;
to
use Storable 2.13;
but this will result in the same error as above. I also tried to use Storable 2.13 directly in my Perl test-script, but that does not change anything, either.
When googling this I could not find any solution, only confirmation that this version discrepancy can be a problem. Increasing the DBI_TRACE level on the client side also did not reveal anything new. Frankly I don't know what to try next and any help would be greatly appreciated.
I am getting this error in error_log one of my Perl CGI application. I am pretty sure I haven't changed my script at all and all of a sudden i have started getting this error.
This is what I see in error_log:
[Wed Jul 8 15:18:20 2009] [warn] FastCGI: server "/local/web/test/cgi-bin/test.pl" (pid 17033)
terminated by calling exit with status '255'
[Wed Jul 8 15:18:20 2009] [warn] FastCGI: server "/local/web/test/cgi-bin/test.pl"
has failed to remain running for 30 seconds given 3 attempts, its restart interval has been backed off to 600 seconds
(The snippet was edited for clarity)
Also, the AddHandler line for FastCGI exists in the config file.
Can any tell me the possible reasons for this error?
There is nothing recorded in Apache logs.
Here are two tips that might help (assuming your app is respecting the fastcgi protocol):
1. try to run the app in command line, this prove that you have execute bit and no compile errors in code.
2. check suexec.log of your apache server, this might show user/group or other errors related to your script.
You could try redirecting the STDERR from your Perl script, something like:
BEGIN { open STDERR, '>stderr.log' }
If your stderr.log file doesn't get created at all, it means that the script didn't even get executed, likely a suexec/permissions problem. Otherwise, you should have whatever went wrong with the Perl script in that file.
Sorry, not really programming question, but I am not sure where else I could find some help.
After a recent update (Xorg was updated among other things), GTK apps stopped running in my kde4. I have a Debian unstable, updated around 22 April. When I try to run them I get the following error:
ga#grzes:~$ iceweasel
The program 'firefox-bin' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadName (named color or font does not exist)'.
(Details: serial 888 error_code 15 request_code 45 minor_code 0)
(Note to programmers: normally, X errors are reported asynchronously;
that is, you will receive the error a while after causing it.
To debug your program, run it with the --sync command line
option to change this behavior. You can then get a meaningful
backtrace from your debugger if you break on the gdk_x_error() function.)
ga#grzes:~$ gimp The program 'gimp' received an X Window
System error.
This probably reflects a bug in the program.
The error was 'BadName (named color or font does not exist)'.
(Details: serial 6955 error_code 15 request_code 45 minor_code 0)
(Note to programmers: normally, X errors are reported asynchronously;
that is, you will receive the error a while after causing it.
To debug your program, run it with the --sync command line
option to change this behavior. You can then get a meaningful
backtrace from your debugger if you break on the gdk_x_error() function.)
(script-fu:4643): LibGimpBase-WARNING **: script-fu: gimp_wire_read():
error
I have to restart the font server manually to have it fixed:
ga#grzes:~$ su
Password:
grzes:/home/ga# /etc/init.d/xfs restart
Stopping X font server: xfs.
Setting up X font server socket directory /tmp/.font-unix...done.
Starting X font server: xfs.
Any ideas what could be wrong? Is it a configuration issue? My system has been updated for the last 7 years, so I can have some old settings.
Debian unstable is very... unstable now, since a release was made a short time ago. Major changes and packages migrations are happening. Xorg (and all X related stuff) being one of the critical packages in that process. My advice is to perform a new update/upgrade in order to catch a new version that may resolve this problem.
It's very frequent that after an update some thing will get broken in inexplicable ways, simply because the developers are uploading new, and not much tested, version of the applications
I finally figured this out: seems like xfs is not compatible with the other components currently and luckily removing it form the system completely solves the problem.