We are having exception thrown on Production that causes the w3wp process to crash. To figure out the faulted code, we configured the Debug Diag that is creating dump file when exception occur. Then we are trying to run the dump file with WinDbg to obtain the Stack Trace to figure out the faulted code but this is what we are experiencing after opening the dump file and running the required commands.
As you can see in the image above, it's not giving the stack trace after running the commands, I'm not sure what I'm missing
UPDATE
After running a command twice as suggested in the comments, I'm able to get the stack trace. But seems like there is no faulted code pointed out in the stack instead there is a long list of underlying framework in the stack. Below is the snapshot for the start of stack. Not sure how to identify the error. Any suggestion or I may need to open separate Question for this?
Related
I am trying to run the scalapb command from a threadpool with each thread running the scalapbc command;
When I do that, I get an error of the form :
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGBUS (0x7) at pc=0x00007f32dd71fd70, pid=8346, tid=0x00007f32e0018700
As per my google search, this issue occurs when the /tmp folder is full or it is trying to be accessed by multiple code simultaneously.
My question is, is there a way to issue scalapbc commands using threading without getting above error? how to make sure that the temp folders being used by the individual threads don't interfere with each other?
This issue occurs most of the times when I run the code but sometimes the build passes as well.
My dogfood machine hit a deadlock earlier this morning in a debug build. It's at a breakpoint in gdb now. What is my best next step before I kill it?
Turn off pagination: set pagination off
Turn on logging: set logging on
Get a stack trace: thread apply all bt
Get a core dump in case we want more info: generate-core-file
Confirm the core file and log file are reasonable (the core file should be binary, the log file should be text, both should be non-empty) before exiting from gdb.
Upload the log file to a Gist so we can all see it.
I am trying to start a jBoss 5.1.0.GA instance and the output console is hanging on the [ProfileServiceBootstrap] Loading profile: ProfileKey#3f5f852e[domain=default, server=default, name=default] line.
The jBoss instance is copied from a remote server on which it works well.
There is not so much work logged (no more than 50 rows) and No error is displayed in console while starting up.
I understand that there may be some dependencies/connections/etc that it needs and are not satisfied, but I would expect an error to be thrown. Instead, it only hangs, without any other issue being reported.
I hope that this message will sound familiar to others that have worked more with older versions of jBoss and may direct me to investigate potential root causes.
Not proud of my findings... :) the issue was in fact that the logs were set not to write into the console. So, after looking for other log files, I found that the server started with the very well known statement Started in 1m:21s:836ms...
It is not really an issue nor an answer, but I leave this here in case others will find themselves in the same "I do not see the logs" situation (which should be also the title).
Note: in order logs to be shown, I have modified /server/default/conf/jboss-log4j.xml
The magic incantation
LD_PRELOAD=/lib/libSegFault.so someapp
runs someapp with libSegFault.so providing backtrace information on a SIGSEGV as described in many different places.
Other than using signal(7)-like approaches to cause SIGABRT to invoke the SIGSEGV handler, is there some way to get libSegFault to provide backtrace information for assert(3) failures?
env SEGFAULT_SIGNALS="abrt segv" LD_PRELOAD=/lib/libSegFault.so someapp
Note that the actual path to the preload library may differ. On my machine, I'd use
env SEGFAULT_SIGNALS="abrt segv" LD_PRELOAD=/lib/x86_64-linux-gnu/libSegFault.so some-64bit-app
or
env SEGFAULT_SIGNALS="abrt segv" LD_PRELOAD=/lib/i386-linux-gnu/libSegFault.so some-32bit-app
depending whether the application I was running was compiled 64-bit or 32-bit. (You can use file to check.)
The source tells us there are three environment variables that define how libSegFault.so behaves:
SEGFAULT_SIGNALS: The list of signals that cause a stack trace.
The default is SIGSEGV. A defined but empty SEGFAULT_SIGNALS means no signals cause a stack trace.
The supported values are segv, ill, abrt, fpe, bus on systems that have the SIGBUS signal, stkflt on systems that have the SIGSTKFLT signal, and all for all of these.
SEGFAULT_USE_ALTSTACK: If defined in the environment, libSegFault.so uses an altenate stack for the stack trace signals.
This may come in handy if you are debugging stack corruption.
SEGFAULT_OUTPUT_NAME: If defined in the environment, the stack trace is written to this file instead of standard error.
To be honest, I found these initially by examining the library with strings /lib/libSegFault.so | sed -e '/[^0-9A-Z_]/ d'. All standard libraries (libSegFault.so having become a part of GNU C library) are tunable via environment variables, so using something like that command to dump any strings that look like environment variable names is a quick way to find stuff to search for. Doing a web search on "SEGFAULT_SIGNALS" "SEGFAULT_OUTPUT_NAME" produces a number of useful links; seeing that it was part of the GNU C library nowadays, I went to the source git archives, found the actual source file for the library, and posted my answer.
In a similar vein, the glibc exception handler writes a stack dump to /dev/console on heap corruption errors.
If you are running your executable in a non tty (i.e. a systemd process or other detached process), the crash output goes to /dev/null, which is not so useful.
There is an undocumented feature to redirect the output to /dev/stderr. Set the following environment variable:
export LIBC_FATAL_STDERR_=1
This can be used in conjunction with libSegFault.so for maximal forensics.
It is also worth mentioning that this might give you two stack traces if you also enable backtraces for SIGABRT, as the glibc first does a stack trace, then signals SIGABRT ... and then libSegFault gives a second stack trace.
I am running a java program on my machine. I print the stack trace on console whenever an exception occurs. Where can i find the logs of those stack trace?
By default prints to console, this won't be written to file. If you want it in file, you need to do Run Configurations--->Common--->Select 'File' and file location. Then you will see your stacktrace in that file.