I am using a third-party C++ app, which owing to some bug occasionally spews out a double free or corruption error. The error does not prevent the app from functioning normally and I do not have desire to debug third-party apps. Therefore, I just lived with the occasional messages. However, after updating Ubuntu from 14.04 to 16.04 the one-line error messages became pages of annoying text in my terminal, like:
*** Error in `/usr/bin/~': double free or corruption (!prev): 0x000055eebccb43d0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f78579047e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x7fe0a)[0x7f785790ce0a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f785791098c]
/usr/lib/x86_64-linux-gnu/lib~.so.6(Nlm_MemFree+0xe)[0x7f7857e9381e]
/usr/lib/x86_64-linux-gnu/lib~.so.6(ReleaseAppErrInfo+0xb3)[0x7f7857e8e4b3]
/usr/lib/x86_64-linux-gnu/lib~.so.6(Nlm_ReleaseAppContext+0x56)[0x7f7857e91ae6]
/usr/bin/~(NlmThreadExit+0x73)[0x55eebb7d1513]
/usr/bin/~(+0x85d7)[0x55eebb7d15d7]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7f7857c5d6ba]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f785799382d]
======= Memory map: ========
55eebb7c9000-55eebb7d4000 r-xp 00000000 08:01 4334530 /usr/bin/~
~
~
My question: is there a way to disable backtrace and limit the message to just the first line under Ubuntu 16.04?
Related
I am trying to hit an external web service using Progress OE 11.5. When I execute the code from GUI Procedure Editor or the CHUI Procedure Editor, it crashes when calling the API
RUN ProcessTrack IN hTrackPortType(INPUT lcRequest, OUTPUT lcResponse) no-error.
I don't get any errors. The progress GUI window just crashes.
When I traced the logs it has "C0000005 ACCESS_VIOLATION" exception. Any idea why this is caused? But when I access the same web services from SoapUI or from a Python program it works fine. I am not sure if Progress OpenEdge has any access restrictions to contact the apis.
I have the full stack trace here.
=====================================================
PROGRESS stack trace as of Fri Aug 07 12:26:40 2020
=====================================================
Progress OpenEdge Release 11.5 build 1114 on WINNT
Startup parameters:
-pf C:\Progressx86\OpenEdge\startup.pf,-cpinternal ISO8859-1,-cpstream ISO8859-1,-cpcoll Basic,-cpcase Basic,-d mdy,-numsep 44,-numdec 46,(end .pf),-param C:\.....\api_request.p
Exception code: C0000005 ACCESS_VIOLATION
Fault address: 025C21CC 1C3:0034002D
Registers:
EAX:086496B8
EBX:00000002
ECX:03100000
EDX:03100000
ESI:59DF2175
EDI:085AA1E0
CS:EIP:0023:025C21CC
SS:ESP:002B:00F4BFA0 EBP:00F4BFD0
DS:002B ES:002B FS:0053 GS:002B
Flags:00210206
Debugging dll: C:\Progressx86\OpenEdge\bin\DBGHELP.DLL
Symbol Path:
C:\Progressx86\OpenEdge\bin;C:\Progressx86\OpenEdge\pdbfiles
Call Stack:
Address Frame
025C21CC 00F4BF9C 0000:00000000
085AA1E0 00F4BFD0 0000:00000000
59DF27DB 00F4BFDC WSDLAttribute::getHandle+3F52B
59DA415C 00F4F130 WSDLArray_Empty+23ABC
59DCD9A0 00F4F144 WSDLAttribute::getHandle+1A6F0
59E502A8 00F4F198 WSDLAttribute::getHandle+9CFF8
59E5032C 00F4F1CC WSDLAttribute::getHandle+9D07C
59DCF04E 00F4F204 WSDLAttribute::getHandle+1BD9E
59D9B724 00F4F240 WSDLArray_Empty+1B084
59E50403 00F4F260 WSDLAttribute::getHandle+9D153
59D55D6F 00F4F2A0 csp_tweakFileURL+312F
** ABL Stack Trace **
--> C:\....\p56215_api_request.ped at line 54 (C:\.....\p56215_api_request.ped)
adecomm/_runcode.p at line 665 (adecomm/_runcode.r)
ExecuteRun adeedit/_proedit.p at line 3613 (adeedit/_proedit.r)
RunFile adeedit/_proedit.p at line 10625 (adeedit/_proedit.r)
USER-INTERFACE-TRIGGER adeedit/_proedit.p at line 1985 (adeedit/_proedit.r)
adeedit/_proedit.p at line 12280 (adeedit/_proedit.r)
_edit.p at line 408 (C:\Progressx86\OpenEdge\gui\_edit.r)
** Persistent procedures/Classes **
** PROPATH **
.,C:\Progressx86\OpenEdge\gui,C:\Progressx86\OpenEdge\gui\ablunit.pl,C:\Progressx86\OpenEdge\gui\adecomm.pl,C:\Progressx86\OpenEdge\gui\adecomp.pl,C:\Progressx86\OpenEdge\gui\adedesk.pl,C:\Progressx86\OpenEdge\gui\adedict.pl,C:\Progressx86\OpenEdge\gui\adeedit.pl,C:\Progressx86\OpenEdge\gui\adeicon.pl,C:\Progressx86\OpenEdge\gui\aderes.pl,C:\Progressx86\OpenEdge\gui\adeshar.pl,C:\Progressx86\OpenEdge\gui\adeuib.pl,C:\Progressx86\OpenEdge\gui\adeweb.pl,C:\Progressx86\OpenEdge\gui\adexml.pl,C:\Progressx86\OpenEdge\gui\dataadmin.pl,C:\Progressx86\OpenEdge\gui\OpenEdge.BusinessLogic.pl,C:\Progressx86\OpenEdge\gui\OpenEdge.Core.pl,C:\Progressx86\OpenEdge\gui\OpenEdge.ServerAdmin.pl,C:\Progressx86\OpenEdge\gui\prodict.pl,C:\Progressx86\OpenEdge\gui\protools.pl,C:\Progressx86\OpenEdge,C:\Progressx86\OpenEdge\bin
** Databases (logical/type/physical) **
** End of Protrace **
This KnowledgeBase post indicates that this is a known error. If you run a version below 11.7.1 you should consider upgrading to the latest version of 11.7 (currently 11.7.6). If you run a version later than 11.7.1 that's mentioned in the article you should consider contacting Progress support.
EDIT: since running 11.5 upgrading should be a priority!
Yesterday we had crash of PostgreSQL 9.5.14 running on Debian 8 (Linux xxxxxx 3.16.0-7-amd64 #1 SMP Debian 3.16.59-1 (2018-10-03) x86_64 GNU/Linux) - Segmentation fault. Database closed all connections and reinitialized itself staying ~1 minute in recovery mode.
PostgreSQL log:
2018-10-xx xx:xx:xx UTC [580-2] LOG: server process (PID 16461) was
terminated by signal 11: Segmentation fault
kern.log:
Oct xx xx:xx:xx xxxxxxxx kernel: [117977.301353] postgres[16461]:
segfault at 7efd3237db90 ip 00007efd3237db90 sp 00007ffd26826678 error
15 in libc-2.19.so[7efd322a2000+1a1000]
According to libc documentation (https://support.novell.com/docs/Tids/Solutions/10100304.html) error code 15 means:
NX_EDEADLK 15 resource deadlock would occur - which does not tell me much.
Could you tell me please if we can do something to avoid this problem in the future? Because this server is of course production one.
All packages are up to date currently. Upgrade of PG is unfortunately not the option. Server runs on Google Compute Engine.
error code 15 means: NX_EDEADLK 15
No, it doesn't mean that. This answer explains how to interpret 15 here.
It's bits 0, 1, 2, 3 set => protection fault, write access, user mode, use of reserved bit. Most likely your postgress process attempted to write to some wild pointer.
if we can do something to avoid this problem in the future?
The only thing you can do is find the bug and fix it, or upgrade to a release of postgress where that bug is already fixed (and hope that no new ones were introduced).
To understand where the bug might be, you should check whether a core dump was produced (if not, do enable them). If you have the core, use gdb /path/to/postgress /path/to/core, and then where GDB command. That will give you crash stack trace, which may allow you to find similar bug reports.
We have an Orion instance which crashes about once each day or two.
In /var/log/contextBroker/contextBroker-service.out I have found:
log directory: '/var/log/contextBroker'
*** glibc detected *** /usr/bin/contextBroker: corrupted double-linked list: 0x00007f0ed92e3f70 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x75f4e)[0x7f0eecdeaf4e]
/lib64/libc.so.6(+0x763d3)[0x7f0eecdeb3d3]
/lib64/libc.so.6(+0x78c88)[0x7f0eecdedc88]
/usr/lib64/libstdc++.so.6(_ZNSsD1Ev+0x39)[0x7f0eed6404c9]
/usr/bin/contextBroker(_Z9jsonParseP14ConnectionInfoPKcRKSsP8JsonNodeP9ParseData+0x539)[0x56fb99]
/usr/bin/contextBroker(_Z9jsonTreatPKcP14ConnectionInfoP9ParseData11RequestTypeRKSsPP11JsonRequest+0x17d)[0x56cf0d]
/usr/bin/contextBroker(_Z12payloadParseP14ConnectionInfoP9ParseDataP11RestServicePP10XmlRequestPP11JsonRequestP18JsonDelayedReleaseRSt6vectorISsSaISsEE+0x3f2)[0x564012]
/usr/bin/contextBroker(_Z11restServiceP14ConnectionInfoP11RestService+0x126c)[0x5654bc]
/usr/bin/contextBroker[0x55cbb6]
/usr/bin/contextBroker[0x55f987]
/usr/lib64/libmicrohttpd.so.10(+0x5599)[0x7f0eee1cf599]
/usr/lib64/libmicrohttpd.so.10(MHD_connection_handle_idle+0x518)[0x7f0eee1d0078]
/usr/lib64/libmicrohttpd.so.10(+0xc3c8)[0x7f0eee1d63c8]
/lib64/libpthread.so.0(+0x7a51)[0x7f0eec957a51]
/lib64/libc.so.6(clone+0x6d)[0x7f0eece5d93d]
And in /var/log/contextBroker/contextBroker-service.out.old the following:
log directory: '/var/log/contextBroker'
*** glibc detected *** /usr/bin/contextBroker: free(): invalid next size (fast): 0x00007fe6d4262110 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x75f4e)[0x7fe6e8e9cf4e]
/lib64/libc.so.6(+0x78cf0)[0x7fe6e8e9fcf0]
/usr/bin/contextBroker(_ZN20ContextElementVector7releaseEv+0x2fa)[0x5f4a4a]
/usr/bin/contextBroker(_Z17postUpdateContextP14ConnectionInfoiRSt6vectorISsSaISsEEP9ParseDatab+0x1472)[0x4d6692]
/usr/bin/contextBroker(_Z11restServiceP14ConnectionInfoP11RestService+0x6d6)[0x564926]
/usr/bin/contextBroker[0x55cbb6]
/usr/bin/contextBroker[0x55f987]
/usr/lib64/libmicrohttpd.so.10(+0x5599)[0x7fe6ea281599]
/usr/lib64/libmicrohttpd.so.10(MHD_connection_handle_idle+0x518)[0x7fe6ea282078]
/usr/lib64/libmicrohttpd.so.10(+0xc3c8)[0x7fe6ea2883c8]
/lib64/libpthread.so.0(+0x7a51)[0x7fe6e8a09a51]
/lib64/libc.so.6(clone+0x6d)[0x7fe6e8f0f93d]
Data is sent to the Orion in batches each 5 minutes:
a request with around 500 contextElements
a request with around 10 contextElements
a request with a single contextElements
Orion has only 2 subscriptions (AFAIK) which send the data to Proton-CEP.
The Orion version is:
[centos#orion ~]$ /usr/bin/contextBroker --version
0.25.0 (git version: a8cf800d4e9fdd7b4293a886490c40309a5bb58c)
Copyright 2013-2015 Telefonica Investigacion y Desarrollo, S.A.U
Is there anything we can do to debug the issue?
Taking into account user inputs, Orion seems to be running below the recommended CPU and RAM thresholds (see recomendations). Thus, probably with more resources (e.g. 2 vCPU and 4GM RAM) it run better, specially if MongoDB runs in the same machine that Orion (MongoDB is known to be a memory-intensive process).
I have been getting a stackoverflow exception in my program which may be originating from a thirdparty libary, microsoft.sharepoint.client.runtime.dll.
Using adplus to create the crash dump, I'm facing the problem that I'm struggling to get any information from it when i open it in windbg. This is what I get as a response:
> 0:000> .restart /f
Loading Dump File [C:\symbols\FULLDUMP_FirstChance_epr_Process_Shut_Down_DocumentumMigrator.exe__0234_2011-11-17_15-19-59-426_0d80.dmp]
User Mini Dump File with Full Memory: Only application data is available
Comment: 'FirstChance_epr_Process_Shut_Down'
Symbol search path is: C:\symbols
Executable search path is:
Windows 7 Version 7601 (Service Pack 1) MP (8 procs) Free x64
Product: Server, suite: Enterprise TerminalServer SingleUserTS
Machine Name:
Debug session time: Thu Nov 17 15:19:59.000 2011 (UTC + 2:00)
System Uptime: 2 days 2:44:48.177
Process Uptime: 0 days 0:13:05.000
.........................................WARNING: rsaenh overlaps cryptsp
.................WARNING: rasman overlaps apphelp
......
..WARNING: webio overlaps winhttp
.WARNING: credssp overlaps mswsock
.WARNING: IPHLPAPI overlaps mswsock
.WARNING: winnsi overlaps mswsock
............
wow64cpu!CpupSyscallStub+0x9:
00000000`74e42e09 c3 ret
Any ideas as to how i can get more information from the dump, or how to use it to find where my stackoverflow error is occuring?
The problem you are facing is that the process is 32-bit, but you are running on 64-bit, therefore your dump is a 64-bit dump. To make use of the dump you have to run the following commands:
.load wow64exts
.effmach x86
!analyze -v
The last command should give you a meaningful stack trace.
This page provides lots of useful information and method to analyze the problem.
http://www.dumpanalysis.org/blog/index.php/2007/09/11/crash-dump-analysis-patterns-part-26/
You didn't mention if your code is managed or unmanaged. Assuming it is unmanaged. In debugger:
.symfix
.reload
~*kb
Look through the call stack for all threads and identify thread that caused SO. It is easy to identify the thread with SO, because the call stack will be extra long. Switch to that thread using command ~<N>s, where is thread number, dump more of the call stack using command k 200 to dump up to 200 lines of call stack. At the very bottom of the call stack you should be able to see the code that originated the nested loop.
If your code is managed, use SOS extension to dump call stacks.
Sorry, not really programming question, but I am not sure where else I could find some help.
After a recent update (Xorg was updated among other things), GTK apps stopped running in my kde4. I have a Debian unstable, updated around 22 April. When I try to run them I get the following error:
ga#grzes:~$ iceweasel
The program 'firefox-bin' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadName (named color or font does not exist)'.
(Details: serial 888 error_code 15 request_code 45 minor_code 0)
(Note to programmers: normally, X errors are reported asynchronously;
that is, you will receive the error a while after causing it.
To debug your program, run it with the --sync command line
option to change this behavior. You can then get a meaningful
backtrace from your debugger if you break on the gdk_x_error() function.)
ga#grzes:~$ gimp The program 'gimp' received an X Window
System error.
This probably reflects a bug in the program.
The error was 'BadName (named color or font does not exist)'.
(Details: serial 6955 error_code 15 request_code 45 minor_code 0)
(Note to programmers: normally, X errors are reported asynchronously;
that is, you will receive the error a while after causing it.
To debug your program, run it with the --sync command line
option to change this behavior. You can then get a meaningful
backtrace from your debugger if you break on the gdk_x_error() function.)
(script-fu:4643): LibGimpBase-WARNING **: script-fu: gimp_wire_read():
error
I have to restart the font server manually to have it fixed:
ga#grzes:~$ su
Password:
grzes:/home/ga# /etc/init.d/xfs restart
Stopping X font server: xfs.
Setting up X font server socket directory /tmp/.font-unix...done.
Starting X font server: xfs.
Any ideas what could be wrong? Is it a configuration issue? My system has been updated for the last 7 years, so I can have some old settings.
Debian unstable is very... unstable now, since a release was made a short time ago. Major changes and packages migrations are happening. Xorg (and all X related stuff) being one of the critical packages in that process. My advice is to perform a new update/upgrade in order to catch a new version that may resolve this problem.
It's very frequent that after an update some thing will get broken in inexplicable ways, simply because the developers are uploading new, and not much tested, version of the applications
I finally figured this out: seems like xfs is not compatible with the other components currently and luckily removing it form the system completely solves the problem.