windbg exception in sos.threads on first run - windbg

When I load a crash dump in windbg (x64), version 6.3.9600.16384, and load the sos extension for .net, the first time I run the !threads command I get this error:
c0000005 Exception in C:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos.threads debugger extension.
PC: 00007ffa`8fe6c7e3 VA: 00000000`00000000 R/W: 0 Parameter: 00000000`00000000
Subsequent times the command runs fine. Full transcript:
Loading Dump File [C:\Users\celdredge\AppData\Local\Temp\w3wp (2).DMP]
User Mini Dump File with Full Memory: Only application data is available
************* Symbol Path validation summary **************
Response Time (ms) Location
Deferred srv*
************* Symbol Path validation summary **************
Response Time (ms) Location
Deferred srv*
OK c:\projects\dumps\symbols
Symbol search path is: srv*;c:\projects\dumps\symbols
Executable search path is: srv*
Windows 8 Version 9600 MP (4 procs) Free x64
Product: WinNt, suite: SingleUserTS
Built by: 6.3.9600.16384 (winblue_rtm.130821-1623)
Machine Name:
Debug session time: Tue Dec 17 23:03:00.000 2013 (UTC - 5:00)
System Uptime: 0 days 9:56:04.777
Process Uptime: 0 days 0:01:41.000
................................................................
................................................................
......................................................
ntdll!NtWaitForSingleObject+0xa:
00007ffa`a1d265ba c3 ret
0:000> .loadby sos clr
0:000> !threads
c0000005 Exception in C:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos.threads debugger extension.
PC: 00007ffa`8fe6c7e3 VA: 00000000`00000000 R/W: 0 Parameter: 00000000`00000000
CLR version:
0:000> lm v mclr
start end module name
00007ffa`84450000 00007ffa`84de8000 clr (pdb symbols) C:\ProgramData\dbg\sym\clr.pdb\252574218A084BE3AFEFF8921ADADB6F2\clr.pdb
Loaded symbol image file: clr.dll
Image path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Image name: clr.dll
Browse all global symbols functions data
Timestamp: Tue Sep 10 02:54:48 2013 (522EC238)
CheckSum: 00994334
ImageSize: 00998000
File version: 4.0.30319.34003
Product version: 4.0.30319.34003
SOS version:
0:000> .chain
Extension DLL search Path:
<snip/>
Extension DLL chain:
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\SOS.dll: image 4.0.30319.34003, API 1.0.0, built Tue Sep 10 02:44:16 2013
[path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos.dll]
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos: image 4.0.30319.34003, API 1.0.0, built Tue Sep 10 02:44:16 2013
[path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos.dll]

This seems to be a weird issue caused by saving an explicit workspace which remembers which extensions are loaded. If I .loadby sos clr and save the workspace, next time I open the workspace it will have sos loaded twice. However if I do .load c:\path\to\sos.dll and save the workspace, it only gets loaded once when I reopen it.
In summary, workspaces in windbg are confusing.

Related

Analizing crash dump

I'm having an issue trying to find out to which problem is crash dump pointing. If someone could help me it would be nice.
This is what I get in windbg.
Microsoft (R) Windows Debugger Version 6.12.0002.633 X86
Copyright (c) Microsoft Corporation. All rights reserved.
Loading Dump File [C:\Users\MAJSTOR\Documents\Sports Interactive\Football Manager 2015\crash dumps\FM 2015 v15.3.2.627042 (2015.06.26 17.55.38).dmp]
User Mini Dump File: Only registers, stack and portions of memory are available
Symbol search path is: *** Invalid ***
****************************************************************************
* Symbol loading may be unreliable without a symbol search path. *
* Use .symfix to have the debugger choose a symbol path. *
* After setting your symbol path, use .reload to refresh symbol locations. *
****************************************************************************
Executable search path is:
Windows 7 Version 7601 (Service Pack 1) MP (2 procs) Free x86 compatible
Product: WinNt, suite: SingleUserTS
Machine Name:
Debug session time: Fri Jun 26 17:55:38.000 2015 (UTC + 2:00)
System Uptime: not available
Process Uptime: 0 days 0:00:32.000
................................................................
.......................
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(1bac.ea8): Access violation - code c0000005 (first/second chance not available)
eax=76a80781 ebx=00000000 ecx=0a7ff803 edx=777970f4 esi=000002c4 edi=00000000
eip=777970f4 esp=0a7ff794 ebp=0a7ff800 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
*** ERROR: Symbol file could not be found. Defaulted to export symbols for ntdll.dll -
ntdll!KiFastSystemCallRet:
777970f4 c3 ret

Which version of sos and clr/mscorwks.dll to load?

I have a 32-bit application (targeting .NET 3.5) hosted on a 64-bit machine. I want to analyze the memory dump of this 32-bit application. I captured the memory dump using 32-bit adplus and cdb. I am loading the memory dump into 32-bit windbg. When I load .net 2.0 sos.dll and .net 2.0 mscorwks.dll into windbg and execute !clrstack, I get the following error: "Failed to find runtime DLL (mscorwks.dll), 0x80004005
Extension commands need mscorwks.dll in order to have something to do." What am I doing wrong?
Info as requested in the comments
ADPlus command line:
adplus -hang -quiet -p 2440 -o C:\temp
WinDbg commands:
0:000> .load <fullpathto>\sos.dll
0:000> lmvm mscorwks
start end module name
0:000> .exr -1
ExceptionAddress: 00000000
ExceptionCode: 80000007 (Wake debugger)
ExceptionFlags: 00000000
NumberParameters: 0
The dump indicates that no .NET 2 was loaded. Otherwise the output of lmvm mscorwks should show the details of the .NET runtime, like this:
0:003> lmvm mscorwks
start end module name
61bc0000 6216e000 mscorwks (deferred)
Image path: C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll
...
File version: 2.0.50727.5485
...
You mentioned that you loaded SOS by full path. If the dump was taken on your machine, you would typically load it using
0:003> .loadby sos mscorwks
In your case, this should already give you the hint that .NET was not loaded:
Unable to find module 'mscorwks'
If you're not so sure about the .NET version, try
.loadby sos clr; *** .NET 4
.loadby sos coreclr; *** Silverlight / Universal Apps
Maybe you had a typo in your AdPlus command line and specified the wrong process ID. If that PID accidentally exists, you got a wrong dump. Use | to check the process name
0:003> |
. 0 id: 1e78 attach name: E:\...\NET2x32.exe
BTW: The -quiet parameter of ADPlus is obsolete, you can omit it.

UDK error, how to examine crash dump

I am working on a game in UDK, and sometimes the game crashes when restarting the level or trying to open a new one.
I cannot find the problem trough the log files, they are just displaying a critical error.
Now I am trying to fix it by examining the crash dump, but I do not have a clue how to do this. Does anyone have an idea how I can further investigate the problem? I tried some things that I found online and this is what I have so far.
*** ERROR: Symbol file could not be found. Defaulted to export symbols for UDK.exe -
eax=00000000 ebx=39280070 ecx=0cdc0f10 edx=ffffffff esi=2a193f40 edi=296f96a0
eip=01c2caf3 esp=007cdf84 ebp=2c0132b0 iopl=0 nv up ei pl zr na pe nc
cs=0023 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010246
UDK!GetStackOwnerClass+0x10a73:
01c2caf3 8b10 mov edx,dword ptr [eax] ds:002b:00000000=????????
0:000> !sym noisy
noisy mode - symbol prompts on
0:000> lmvm ntdll
start end module name
77890000 77a10000 ntdll (export symbols) ntdll.dll
Loaded symbol image file: ntdll.dll
Mapped memory image file: C:\Windows\SysWOW64\ntdll.dll
Image path: C:\Windows\SysWOW64\ntdll.dll
Image name: ntdll.dll
Timestamp: Thu Aug 29 03:50:31 2013 (521EA8E7)
CheckSum: 00140982
ImageSize: 00180000
File version: 6.1.7601.18247
Product version: 6.1.7601.18247
File flags: 0 (Mask 3F)
File OS: 40004 NT Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0409.04b0
CompanyName: Microsoft Corporation
ProductName: Microsoft® Windows® Operating System
InternalName: ntdll.dll
OriginalFilename: ntdll.dll
ProductVersion: 6.1.7601.18247
FileVersion: 6.1.7601.18247 (win7sp1_gdr.130828-1532)
FileDescription: NT Layer DLL
LegalCopyright: © Microsoft Corporation. All rights reserved.
The most basic analysis in WInDbg is done with
.symfix
.reload
!analyze -v
Since it crashes, there should be an exception somewhere. Try
.ecxr
.exr -1
Also the call stack is potentially interesting. Since this is about UDK (Unreal Engine Development Kit), I assume it's written in C++ and thus we need a native call stack
k

How do I make windbg load clr.dll from a custom location?

I am starting windbg using the following command line:
C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64>windbg -i c:\tmp\Psscor4\amd64;c:\tmp\Psscor4\x86;c:\tmp;srv*E:\symbols*http://msdl.microsoft.com/download/symbols
C:\Program Files (x86)\Windows Kits\8.0\Debuggers\x64>
Then I load a memory crash dump and inspect where did it load the clr.dll from:
Microsoft (R) Windows Debugger Version 6.2.9200.20512 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.
Loading Dump File [C:\tmp\Memory.dmp]
User Mini Dump File with Full Memory: Only application data is available
Comment: 'Dump created by DbgHost. First chance exception 0XE0434352'
Symbol search path is: c:\tmp\Psscor4\amd64;c:\tmp\Psscor4\x86;c:\tmp;srv*E:\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is: c:\tmp\Psscor4\amd64;c:\tmp\Psscor4\x86;c:\tmp;srv*E:\symbols*http://msdl.microsoft.com/download/symbols
Windows 7 Version 7601 (Service Pack 1) MP (16 procs) Free x64
Product: Server, suite: Enterprise TerminalServer SingleUserTS
Built by: 6.1.7601.17965 (win7sp1_gdr.121004-0333)
Machine Name:
Debug session time: Mon Oct 14 13:45:55.000 2013 (UTC - 4:00)
System Uptime: not available
Process Uptime: 0 days 2:49:12.000
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
............................................................
Loading unloaded module list
..
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(5768.5db4): CLR exception - code e0434352 (first/second chance not available)
KERNELBASE!RaiseException+0x39:
000007fe`fd33bccd 0000 add byte ptr [rax],al ds:00000000`3af07bb2=00
0:122> lm vm clr
start end module name
000007fe`f9a70000 000007fe`fa3ce000 clr (deferred)
Image path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Image name: clr.dll
Timestamp: Mon Jul 09 00:10:25 2012 (4FFA59B1)
CheckSum: 00959DDE
ImageSize: 0095E000
File version: 4.0.30319.17929
Product version: 4.0.30319.17929
File flags: 8 (Mask 3F) Private
File OS: 4 Unknown Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4
0:122> ld clr
Symbols loaded for clr
0:122> lm vm clr
start end module name
000007fe`f9a70000 000007fe`fa3ce000 clr (pdb symbols) e:\symbols\clr.pdb\D3D86782AEDD446F917F5D81FDFD3D252\clr.pdb
Loaded symbol image file: clr.dll
Mapped memory image file: c:\tmp\clr.dll
Image path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Image name: clr.dll
Timestamp: Mon Jul 09 00:10:25 2012 (4FFA59B1)
CheckSum: 00959DDE
ImageSize: 0095E000
File version: 4.0.30319.17929
Product version: 4.0.30319.17929
File flags: 8 (Mask 3F) Private
File OS: 4 Unknown Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4
0:122> .exepath
Executable image search path is: c:\tmp\Psscor4\amd64;c:\tmp\Psscor4\x86;c:\tmp;srv*E:\symbols*http://msdl.microsoft.com/download/symbols
Expanded Executable image search path is: c:\tmp\psscor4\amd64;c:\tmp\psscor4\x86;c:\tmp;srv*e:\symbols*http://msdl.microsoft.com/download/symbols
So, my question is why does windbg insist on loading clr.dll from C:\Windows\Microsoft.NET\Framework64\v4.0.30319 when both the image path and the symbol path direct to another location where sits the clr.dll that I truly need - c:\tmp?
Now, when I force loading of the symbols, then we can see this:
Loaded symbol image file: clr.dll
Mapped memory image file: c:\tmp\clr.dll
Image path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Image name: clr.dll
I do not like it. I want the image path to come from c:\tmp as well.
How do I do it?
The Image path shows where debugee (the process which were dumped) found the clr.dll.
Like it or not, it's noting you can do about it :-)

Why does WinDBG show mismatched FileVersion in dump?

TL;DR
Why does WinDBG lmvdisplay two version info fields (when no other tool I know of does this) and under which circumstances can these fields differ?
Background: I have a live dump (from a deadlock) of our application. Symbols are loaded correctly and I was able to trace back the deadlock to Microsoft's pdm.dll ("Process Debug Manager" used for our vbscript engine).
I then wanted to check which version of this DLL was loaded in the session at the production site:
0:000> lmv m pdm
start end module name
51860000 518b8000 pdm # (pdb symbols) d:\symcache\pdm.pdb\7BE601EDE9234816B72B49DA4A25DF042\pdm.pdb
Loaded symbol image file: pdm.dll
Image path: C:\Program Files (x86)\Common Files\Microsoft Shared\VS7DEBUG\pdm.dll
Image name: pdm.dll
Timestamp: Tue Jul 29 16:46:11 2008 (488F2D33)
CheckSum: 000663E0
ImageSize: 00058000
?? File version: 9.0.30729.1
?? Product version: 9.0.30729.1
File flags: 0 (Mask 3F)
File OS: 4 Unknown Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0409.04b0
CompanyName: Microsoft Corporation
ProductName: Microsoft® Visual Studio .NET
InternalName: pdm.dll
OriginalFilename: pdm.dll
?? ProductVersion: 7.10.3077
?? FileVersion: 7.10.3077
FileDescription: Process Debug Manager
LegalCopyright: Copyright© Microsoft Corporation. All rights reserved.
As you can see, the file and product versions are displayed twice, but they are mismatched in the dump!
When I cross-check on my machine for (apparently, see the timestamp and checksum!) the same file for a running iexplore process:
0:043> lmv m pdm
start end module name
3efa0000 3eff8000 pdm (pdb symbols) c:\windows\symbols\martin-cache\pdm.pdb\415D0A165EB24613BC01CE516512062C2\pdm.pdb
Loaded symbol image file: C:\Program Files (x86)\Internet Explorer\pdm.dll
Image path: C:\Program Files (x86)\Internet Explorer\pdm.dll
Image name: pdm.dll
Timestamp: Tue Jul 29 16:46:11 2008 (488F2D33)
CheckSum: 000663E0
ImageSize: 00058000
File version: 9.0.30729.1
Product version: 9.0.30729.1
File flags: 0 (Mask 3F)
File OS: 4 Unknown Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0409.04b0
CompanyName: Microsoft Corporation
ProductName: Microsoft® Visual Studio® 2008
InternalName: pdm.dll
OriginalFilename: pdm.dll
ProductVersion: 9.0.30729.1
FileVersion: 9.0.30729.1 built by: SP
FileDescription: Process Debug Manager
LegalCopyright: © Microsoft Corporation. All rights reserved.
the version infos match up.
lmv displays the strings which are defined in the resource file,
I don’t know why there is common to two set of File/Product versions names the same except some spaces.
0:041> lmv m kernel32
start end module name
753e0000 754f0000 kernel32 (deferred)
Image path: C:\Windows\SysWOW64\kernel32.dll
Image name: kernel32.dll
Timestamp: Fri Aug 02 03:53:25 2013 (51FB1115)
CheckSum: 00111A9F
ImageSize: 00110000
File version: 6.1.7601.18229
Product version: 6.1.7601.18229
File flags: 0 (Mask 3F)
File OS: 40004 NT Win32
File type: 2.0 Dll
File date: 00000000.00000000
Translations: 0409.04b0
CompanyName: Microsoft Corporation
ProductName: Microsoft® Windows® Operating System
InternalName: kernel32
OriginalFilename: kernel32
ProductVersion: 6.1.7601.18229
FileVersion: 6.1.7601.18229 (win7sp1_gdr.130801-1533)
In your case you have two different dll’s, look at the image file path.
Image path: C:\Program Files (x86)\Common Files\Microsoft Shared\VS7DEBUG\
and
Image path: C:\Program Files (x86)\Internet Explorer
They must have different strings in the resource section, winDbg can’t do anything but display it.
Since the time stamps are identical, one may have been tampered.