Sometimes Windows Crash while LabVIEW operator Advantech PCI-1716 card - windbg

The system include 2 Advantech PCI-1716 card:http://www.advantech.com/products/PCI-1716/mod_86EC4C4D-F497-45C5-81DA-B8600C0EB36F.aspx
I write a program with NI LabVIEW 7.1.
The LabVIEW program can control the two PCI 1716 card.
It work very good.
But about a year later, The computer crashed and sometimes auto restart.
Nobody change the software and hardware.
I used WinDbg to analysis the Windows crash dump file and I think I find something abnormal.
For the WinDbg result, I think maybe the PCI-1716 driver was broken so I re-installed it.
But the problem also happen.
And I also re-installed my windows xp.
The also problem happen again.
I don't know how.
Maybe the PCI 1716 hardware broken.
But how to find which one, there are two PCI 1716 card.
The WinDbg result is:
Loading Dump File [F:\WinDBG\Mini012313-01.dmp] Mini Kernel Dump File: Only registers and stack trace are available
Symbol search path is: srv*C:\Documents and Settings\cky\symbols*http://msdl.microsoft.com/download/symbols Executable search path is: Windows XP Kernel Version 2600 (Service Pack 2) MP (2 procs) Free x86 compatible Product: WinNt, suite: TerminalServer SingleUserTS Built by: 2600.xpsp_sp2_rtm.040803-2158 Kernel base = 0x804d8000 PsLoadedModuleList = 0x8055d700 Debug session time: Wed Jan 23 23:09:23.250 2013 (GMT+8) System Uptime: 0 days 23:47:48.802 Loading Kernel Symbols ....................................................................................................... Loading User Symbols Loading unloaded module list ....
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
Use !analyze -v to get detailed debugging information.
BugCheck 1000000A, {ffdf, 2, 1, 806e5a8e}
Unable to load image ADS1716S.sys, Win32 error 0n2
*** WARNING: Unable to verify timestamp for ADS1716S.sys
*** ERROR: Module load completed but symbols could not be loaded for ADS1716S.sys Probably caused by : ADS1716S.sys ( ADS1716S+f08 )
Followup: MachineOwner
---------
0: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
IRQL_NOT_LESS_OR_EQUAL (a) An attempt was made to access a pageable (or completely invalid) address at an interrupt request level (IRQL) that is too high. This is usually caused by drivers using improper addresses. If a kernel debugger is available get the stack backtrace. Arguments: Arg1: 0000ffdf, memory referenced Arg2: 00000002, IRQL Arg3: 00000001, bitfield : bit 0 : value 0 = read operation, 1 = write operation bit 3 : value 0 = not an execute operation, 1 = execute operation (only on chips which support this level of status) Arg4: 806e5a8e, address which referenced memory
Debugging Details:
------------------
WRITE_ADDRESS: 0000ffdf
CURRENT_IRQL: 2
FAULTING_IP: hal!KeAcquireQueuedSpinLock+42 806e5a8e 8902 mov dword ptr [edx],eax
CUSTOMER_CRASH_COUNT: 1
DEFAULT_BUCKET_ID: DRIVER_FAULT
BUGCHECK_STR: 0xA
PROCESS_NAME: LabVIEW.exe
LAST_CONTROL_TRANSFER: from 804efbb0 to 806e5a8e
STACK_TEXT: aa286c00 804efbb0 aa286c20 804f0de4 aa286c1c hal!KeAcquireQueuedSpinLock+0x42 aa286c08 804f0de4 aa286c1c 8602dc38 806e53b8 nt!IoAcquireCancelSpinLock+0xe aa286c20 804f104a 8602dc38 00000001 aa286c58 nt!IopStartNextPacket+0x18 aa286c30 f788ef08 8602dc38 00000001 85eb0230 nt!IoStartNextPacket+0x38 WARNING: Stack unwind information not available. Following frames may be wrong. aa286c58 80575529 85eb0230 86090da0 85f32448 ADS1716S+0xf08 aa286c80 805d1cb9 85f32448 85f32448 85f32690 nt!IoCancelThreadIo+0x33 aa286d08 805d209a 00000001 85f32448 00000000 nt!PspExitThread+0x403 aa286d28 805d2275 85f32448 00000001 aa286d64 nt!PspTerminateThreadByPointer+0x52 aa286d54 8054160c 00000000 00000001 0012e810 nt!NtTerminateProcess+0x105 aa286d54 7c92eb94 00000000 00000001 0012e810 nt!KiFastCallEntry+0xfc 0012e810 00000000 00000000 00000000 00000000 0x7c92eb94
STACK_COMMAND: kb
FOLLOWUP_IP: ADS1716S+f08 f788ef08 ?? ???
SYMBOL_STACK_INDEX: 4
SYMBOL_NAME: ADS1716S+f08
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: ADS1716S
IMAGE_NAME: ADS1716S.sys
DEBUG_FLR_IMAGE_TIMESTAMP: 45b96de9
FAILURE_BUCKET_ID: 0xA_W_ADS1716S+f08
BUCKET_ID: 0xA_W_ADS1716S+f08
Followup: MachineOwner
---------
0: kd> lmvm ADS1716S start end module name f788e000 f7893ac0 ADS1716S T (no symbols)
Loaded symbol image file: ADS1716S.sys
Image path: ADS1716S.sys
Image name: ADS1716S.sys
Timestamp: Fri Jan 26 10:56:41 2007 (45B96DE9)
CheckSum: 00010C75
ImageSize: 00005AC0
Translations: 0000.04b0 0000.04e0 0409.04b0 0409.04e0

I'd run Memtest86+ just to see if you have any bad RAM.
Following that maybe you can pull 1 card out and write a simple script to do some I/O (similar to what you're doing normally maybe). If it takes a while you could put it into a loop and run it for a while and see if you can force a crash. You could repeat the test on each card to see if you have any issues with just one of the cards.
Does the crash happen often or rarely?

Related

The ba command (Break on Access) in WinDbg is not working as advertised in the book Advanced Windows Debugging by Daniel Pravat and Mario Hewardt

I am reading the book Advanced Windows Debugging: Developing and Administering Reliable, Robust, and Secure Software by Daniel Pravat and Mario Hewardt. I have questions about Chapter 2.
I am using WinDBG 10.0.19041.1 X86 on Windows 10 Pro Version 2004 build 19041.572. I built 02sample.exe with Microsoft Visual Studio Community 2019 Version 16.7.6 with the GenerateDebugInformation property set to DebugFull. I am debugging the Debug|x86 configuration of 02sample.exe.
I am reading the section in chapter 2 named Setting a Breakpoint on Access and I am seeing differences between what is in the book and what I am experiencing.
The first difference in behavior is with the following command.
0:000> dt gGlobal
This command fails with the following error.
Symbol gGlobal not found.
The following command does work.
0:000> dt 02sample!gGlobal
*** WARNING: Unable to verify checksum for 02sample.exe
gGlobal
+0x000 m_ref : 0n0
The next difference in behavior is with the following command.
0:000> ba w4 gGlobal+0
Based on the following output, this appears to work.
0:000> bl
0 e Disable Clear 00969130 w 4 0001 (0001) 0:**** 02sample!gGlobal
However, the breakpoint is not being hit. I cannot figure out why.
Symbols Are probably Not Yet Loaded
try either .reload /f or .reload /f foo.exe
before attempting unqualified dt gGlobal
a qualified foo!gGlobal will always work because it loads the symbols
check with !sym noisy
0:000> !sym noisy
noisy mode - symbol prompts off
0:000> dt gGlobal
Symbol gGlobal not found.
0:000> dt gGlobal
Symbol gGlobal not found.
0:000> dt gGlobal
Symbol gGlobal not found.
0:000> dt awd!gGlobal
SYMSRV: BYINDEX: 0x2
snipxxxxxxxxxxxxxxx
DBGHELP: awd - public symbols & lines
C:\Users\XXX\Desktop\awd\Chapter2\Debug\awd.pdb
+0x000 m_ref : 0n0
0:000>
how is the Book Telling You to set the ba breakpoint ?
You Cannot Set a ba breakpoint on a module when you are Stopped At System Breakpoint
Because the system will reset the thread context
you have to go the Entry Point and then Set the ba breakpoint as windbg suggests do you do that ?
:\>cdb -c "g #$exentry;ba w4 awd!gGlobal;g;u .;kb;q" awd.exe |awk "/Reading/,/quit/"
0:000> cdb: Reading initial command 'g #$exentry;ba w4 awd!gGlobal;g;u .;kb;q'
*** WARNING: Unable to verify checksum for awd.exe
Breakpoint 0 hit
awd!Global::Global+0x21:
00a23461 8b45fc mov eax,dword ptr [ebp-4]
00a23464 83c404 add esp,4
00a23467 3bec cmp ebp,esp
00a23469 e895ddffff call awd!ILT+510(__RTC_CheckEsp) (00a21203)
00a2346e 8be5 mov esp,ebp
00a23470 5d pop ebp
00a23471 c3 ret
00a23472 cc int 3
ChildEBP RetAddr Args to Child
0026f9d4 00a21687 0026f9ec 522c59df 00a21670 awd!Global::Global+0x21
0026f9dc 522c59df 00a21670 00a2b208 0026fa50 awd!`dynamic initializer for 'gGlobal''+0x17
0026f9ec 00a24a5e 00a2b000 00a2b30c 91b61bad ucrtbased!_initterm+0x3f
0026fa50 00a2498d 0026fa60 00a24d08 0026fa6c awd!__scrt_common_main_seh+0xbe
0026fa58 00a24d08 0026fa6c 7683ed6c 7ffde000 awd!__scrt_common_main+0xd
0026fa60 7683ed6c 7ffde000 0026faac 773337eb awd!wmainCRTStartup+0x8
0026fa6c 773337eb 7ffde000 761f96f6 00000000 kernel32!BaseThreadInitThunk+0xe
0026faac 773337be 00a21145 7ffde000 00000000 ntdll!__RtlUserThreadStart+0x70
0026fac4 00000000 00a21145 7ffde000 00000000 ntdll!_RtlUserThreadStart+0x1b
quit:
if by chance you misunderstood entrypoint to wmain() and set a ba break on reaching wmain() it probably will never be hit because the code in question has already been executed

How to do hybrid user-mode/kernel-mode debugging?

Basically, I have a user mode program that calls kernel32.CreateProcessA() which internally calls kernel32.CreateProcessInternalW(). Within this function, I'm interested in what is happening inside ntdll.NtCreateSection() which attempts to map the executable in virtual memory. Once in this function, the program quickly sets up the kernel call as EAX=0x32 and executes the SYSENTER instruction.
Obviously I can't see beyond the call gate in a user mode debugger. I have a little experience debugging kernel-mode drivers, so I loaded a copy of XP SP3 in a VMWare window and used VirtualKD to conect the pipe to the WinDbg (which I happen to be running inside IDA). After connecting the kernel debugger, I copied my user-mode EXE program and PDB onto the virtual machine, but I'm kind of at a loss on how to set the initial breakpoint in my user-mode program properly. I don't want to intercept all calls to the equivalent ntdll.ZwCreateSection() which I believe to be on the other side of the call gate. Ideally, I'd like to break into the user-mode code and step through that call gate now that I'm using a Kernel debugger, but I don't know what the first steps are.
I've done some googling and I've come close by setting a "ntsd -d" value in
HKLM\Software\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\myprocess.exe
This causes a break in the kernel debugger when I start my process, but I can't seem to set any breakpoints following the .breakin command I need to issue to IDA to get to the WinDbg prompt. I've been following this guide where I locate my process with !process then switch to the context, and reload the symbols but I'm having problems setting the breakpoint in my process or advancing past the initial breakpoint set by "ntsd -d". After getting the message that the breakpoint could not be resolved and a deferred breakpoint is added, I cannot seem to advance "into" to the process without clearing the breakpoints if that makes any sense. Here's the stack of where I seem to be at when I hit that initial break:
ChildEBP RetAddr
b2b55ccc 8060e302 nt!RtlpBreakWithStatusInstruction
b2b55d44 8053d638 nt!NtSystemDebugControl+0x128
b2b55d44 7c90e4f4 nt!KiFastCallEntry+0xf8
0007b270 7c90de3c ntdll!KiFastSystemCallRet
0007b274 6d5f5ca6 ntdll!ZwSystemDebugControl+0xc
0007bd48 6d5f6102 dbgeng!DotCommand+0xd0d
0007de8c 6d5f7077 dbgeng!ProcessCommands+0x318
0007dec4 6d5bec6c dbgeng!ProcessCommandsAndCatch+0x1a
0007eedc 6d5bed4d dbgeng!Execute+0x113
0007ef0c 010052ce dbgeng!DebugClient::Execute+0x63
0007ff3c 010069fb ntsd!MainLoop+0x1ec
0007ff44 01006b31 ntsd!main+0x10e
0007ffc0 7c817067 ntsd!mainCRTStartup+0x125
0007fff0 00000000 kernel32!BaseProcessStart+0x23
To be honest, I'm not sure my PDB is being loaded but I suspect its probably not my immediate problem; my modules pane is only showing kernel driver modules, not user mode modules. When I had been doing driver debugging in the past, I could see my driver image in this pane and whether or not the symbols had loaded, so I'm not sure what to expect for a user-mode image.
Without the image, I can't really expect the debugger to resolve any breakpoints.
I realize I may be going about this completely wrong but I'm not having any luck searching for how to do user-mode/kernel-mode hybrid debugging. Is there anyone out there that could point me in the right direction so I can step into this kernel mode function from a specific user-mode process? Or, at least set a proper kernel-mode breakpoint so it is only triggered as a result of my particular user-mode process?
UPDATE:
I loaded my module (happens to be named runlist.exe) in a user-mode debugger on the debugged OS (I happened to use OllyDbg). Once I was paused at a user-mode breakpoint only a couple instructions from SYSENTER, I suspended the OS using the kernel debugger. I then set the process context. The WinDbg command window contents were as follows:
WINDBG>!process 0 0 runlist.exe
PROCESS 820645a8 SessionId: 0 Cid: 01b4 Peb: 7ffd7000 ParentCid: 02b0
DirBase: 089c02e0 ObjectTable: e1671bb0 HandleCount: 8.
Image: runlist.exe
WINDBG>.process /i /r /p 820645a8
You need to continue execution (press 'g' <enter>) for the context
to be switched. When the debugger breaks in again, you will be in
the new process context.
WINDBG>g
This command cannot be passed to the WinDbg plugin directly, please use IDA Debugger menu to achieve the same result.
Break instruction exception - code 80000003 (first chance)
WINDBG>.reload /user
Loading User Symbols
....
Caching 'Modules'... ok
WINDBG>lmu
start end module name
00400000 00405000 runlist C (no symbols)
7c340000 7c396000 MSVCR71 (private pdb symbols) g:\symcache\msvcr71.pdb\630C79175C1942C099C9BC4ED019C6092\msvcr71.pdb
7c800000 7c8f6000 kernel32 (pdb symbols) e:\windows\symbols\dll\kernel32.pdb
7c900000 7c9af000 ntdll (pdb symbols) e:\windows\symbols\dll\ntdll.pdb
WINDBG>bp 0x7c90d16a
WINDBG>bl
0 e 7c90d16a 0001 (0001) ntdll!ZwCreateSection+0xa
Although I couldn't get my process' symbols to load with ".reload" (PDB is in the same directory - might need to copy it to my symbols dir), the breakpoint I care about is in ntdll anyway so I set it on the address 0x7C90D16A which the debugger recognized as being within ntdll.ZwCreateSection(). Oddly to me, in user-mode code this address resolves to ntdll.NtCreateSection(), but either way that breakpoint was only 2 instructions from where I had my user-mode break. When I resumed the machine, my intention was to "run" the user-mode debugged-process and this would trigger the kernel-mode breakpoint 2 instructions away. The kernel breakpoint was never hit and the app resumed past this point. I can however set a breakpoint on ntdll!ZwCreateSection() but then when resuming the OS, the breakpoint is repeatedly hit by other processes preventing me from getting back to the user-mode debugger so I can "run" it to that location only within my own process.
UPDATE
Merging the tips provided by #conio, the following steps worked for me:
1> after attaching kernel debugger and booting target OS, suspend the OS and apply some configuration options:
!gflag +ksl //allow sxe to report user-mode module load events under kernel debugger
sxe ld myproc.exe //cause kernel debugger break upon process load
.sympath+ <path> //path to HOST machine's user-mode app's symbols
2> run debugger to resume target OS
3> on the target, run the EXE we want to debug
4> kernel debugger should break; now enter the following commands to switch to the usermode context:
!process 0 0 myproc.exe //get address of EProcess structure (first number on 1st line after "PROCESS")
.process /i /r /p <eprocess*> //set kernel debugger to process context
g //continue execution to allow the context switch; debugger will break after switch complete
.reload /user //reload user symbols
lmu //ensure you have symbols although not really necessary in my particular case
5> now since I already know what happens in the user-mode side of ntdll.NtCreateSection(), I just went ahead and set a breakpoint for the kernel mode side of that function, but specifying that I want the breakpoint to occur only within the context of my process. This way, the breakpoint is not triggered OS wide:
bu /p <eprocess*> nt!NtCreateSection //set breakpoint in kernel side of function
g //run to break
6> if all goes as planned, the breakpoint will wake up the debugger on the kernel mode side of NtCreateSection(). I appreciate all the responses and tips!
There are two ways to combine user-mode debugging with kernel-mode debugging and you're confusing and mixing them up.
The way you tried is to use the kernel-mode debugger to debug kernel-mode code, use the user-mode debugger (ntsd) to debug user-mode code, and control the user-mode debugger running on the target machine from the kernel debugger. That's what the -d flag to ntsd does. This method is described in the Controlling the User-Mode Debugger from the Kernel Debugger page and its subpages on MSDN.
What this does (more or less) is to redirect ntsd input and output to the kernel debugger. The modules pane - as the rest of the windows in WinDbg - belong to the kernel debugger. Your only interaction with the user-mode debugger is through the tunnel the kernel debugger creates, and you can access it only through the command window. This is documented in the documentation for the -d flag:
-d
        Passes control of this debugger to the kernel debugger. If you are debugging CSRSS, this control redirection always is active, even if -d is not specified. (This option cannot be used during remote debugging -- use -ddefer instead.) See Controlling the User-Mode Debugger from the Kernel Debugger for details. This option cannot be used in conjunction with either the -ddefer option or the -noio option.
        Note  If you use WinDbg as the kernel debugger, many of the familiar features of WinDbg are not available in this scenario. For example, you cannot use the Locals window, the Disassembly window, or the Call Stack window, and you cannot step through source code. This is because WinDbg is only acting as a viewer for the debugger (NTSD or CDB) running on the target computer.
The second way, which is the one used in the link you put, is to use the kernel debugger to debug both kernel-mode code and user-mode code. No user-mode debugger. No ntsd. You said you've followed the guide, but in fact you didn't. If you had, there wouldn't be any ntsd.
I suggest you use this method for start, and after you use the user-mode debugger only if you find out you need to ( because you want to use a user-mode extension, for example).
In order for the kernel debugger to work well with user-mode modules you have to enable the Enable loading of kernel debugger symbols GlobalFlag. Use !gflag +ksl to do that.
Once you do that, break on the loading of your process using sxe ld:runlist, set the breakpoint (possibly with the /p option) and debug whatever it is that you want.
Just do that instead of all the ntsd mess.
Use ntsd -d and start debugging the executabke from target with a kd connection you can use the kd as an usermode debugger as well as kernel debugger read the docs several times it is not easy doing it furst time but over several trials you should get the hang of it read about .breakin etc
How to break on the entry point of a program when debug in kernel mode with windbg?
edited to add a demo for using ntsd -d
setup
1 ) a vm running winxp sp3 and windbg version 6.12 installed in it
2 ) _NT_SYMBOL_PATH in vm is set to z:\
3 ) z:\ is a mapped network drive that points to e:\symbols in host
4 ) host running win 7 sp2
5 ) host windbg 10.0010586
starting an application in vm under ntsd and redirecting it to kd
opened a command prompt in vm navigated to windbg installation directory and issued ntsd -s -d calc -s is to disable lazy symbol loading
0:000> version
version
Windows XP Version 2600 (Service Pack 3) UP Free x86 compatible
Product: WinNt, suite: SingleUserTS
kernel32.dll version: 5.1.2600.5512 (xpsp.080413-2111)
Machine Name:
Debug session time: Thu Mar 16 16:44:29.222 2017
System Uptime: 0 days 0:10:12.941
Process Uptime: 0 days 0:01:40.980
Kernel time: 0 days 0:00:01.632
User time: 0 days 0:00:00.040
Live user mode: <Local>
Microsoft (R) Windows Debugger Version 6.12.0002.633 X86
Copyright (c) Microsoft Corporation. All rights reserved.
command line: 'ntsd -s -d calc' Debugger Process 0xA8
dbgeng: image 6.12.0002.633, built Tue Feb 02 01:38:31 2010
[path C:\Documents and Settings\admin\Desktop\Debugging Tools for Windows (x86)\dbgeng.dll]
windbg breaks on SystemBreakPoint and the Debug prompt is Input:\>
lm shows the symbol was loaded from z:\
CommandLine: calc
Symbol search path is: z:\
Executable search path is:
ModLoad: 01000000 0101f000 calc.exe
xxxxx
ntdll!DbgBreakPoint:
7c90120e cc int 3
0:000> lm
lm
start end module name
01000000 0101f000 calc (pdb symbols) z:\calc.pdb\3B7D84101\calc.pdb
77c10000 77c68000 msvcrt (export symbols) C:\WINDOWS\system32\msvcrt.dll
Executing till AddressOfEntryPoint
0:000> g #$exentry
g #$exentry
calc!WinMainCRTStartup:
01012475 6a70 push 70h
Setting a breakpoint in user mode and its counterpart in kernel mode at once
0:000> bp ntdll!ZwCreateSection <--- user mode bp notice prompt 0:000
bp ntdll!ZwCreateSection
0:000> .breakin <<---- transferring to kd mode
.breakin
Break instruction exception - code 80000003 (first chance)
nt!RtlpBreakWithStatusInstruction:
804e3592 cc int 3
kd> !process 0 0 calc.exe <<----- looking for our process of interest
Failed to get VAD root
PROCESS ffae2020 SessionId: 0 Cid: 0410 Peb: 7ffde000 ParentCid: 00a8
DirBase: 04d87000 ObjectTable: e1bd5238 HandleCount: 26.
Image: calc.exe
kd> bp /p ffae2020 nt!NtCreateSection << setting a kernel mode bp
on counterpart that matches with our process of interest notice prompt kd>
kd> g <<<---- return to user mode after setting a breakpoint
0:000> g <<<<<--------- executing in user mode
g
now calc process is running in usermode in the vm
click help about (this will trigger a Loadlib and that needs a Section so we will break on our user mode bp in the kernel debugger )
Breakpoint 0 hit
eax=00000000 ebx=00000000 ecx=00000001 edx=ffffffff esi=0007f368 edi=00000000
eip=7c90d160 esp=0007f22c ebp=0007f2a8 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
ntdll!NtCreateSection:
7c90d160 b832000000 mov eax,32h
now we can merrily trace around use t trace not p or g or any other execution commands
0:000> t
t
eax=00000032 ebx=00000000 ecx=00000001 edx=ffffffff esi=0007f368 edi=00000000
eip=7c90d165 esp=0007f22c ebp=0007f2a8 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
ntdll!ZwCreateSection+0x5:
7c90d165 ba0003fe7f mov edx,offset SharedUserData!SystemCallStub (7ffe0300)
0:000>
eax=00000032 ebx=00000000 ecx=00000001 edx=7ffe0300 esi=0007f368 edi=00000000
eip=7c90d16a esp=0007f22c ebp=0007f2a8 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
ntdll!ZwCreateSection+0xa:
7c90d16a ff12 call dword ptr [edx] ds:0023:7ffe0300={ntdll!KiFastSystemCall (7c90e4f0)}
0:000>
eax=00000032 ebx=00000000 ecx=00000001 edx=7ffe0300 esi=0007f368 edi=00000000
eip=7c90e4f0 esp=0007f228 ebp=0007f2a8 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
ntdll!KiFastSystemCall:
7c90e4f0 8bd4 mov edx,esp
0:000>
eax=00000032 ebx=00000000 ecx=00000001 edx=0007f228 esi=0007f368 edi=00000000
eip=7c90e4f2 esp=0007f228 ebp=0007f2a8 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
ntdll!KiFastSystemCall+0x2:
7c90e4f2 0f34 sysenter
0:000>
Breakpoint 1 hit
nt!NtCreateSection:
805652b3 6a2c push 2Ch
when in bp at kernel .reload and see the stack trace
the 2nd stack trace is same as first but with corrected symbols for
Shell32.dll (vm doesnt have internet access so it fails for first time
so i drag dropped that specific dll from vm and fetched its sumbols from host using windbg -z shell32.dll and .reload (since the downstore in host is network mappped in vm the second trace properly loads the pdb and gives a correct stack trace without warnings
kd> kb
# ChildEBP RetAddr Args to Child
00 f8bb1d40 804de7ec 0007f368 0000000f 00000000 nt!NtCreateSection
01 f8bb1d40 7c90e4f4 0007f368 0000000f 00000000 nt!KiFastCallEntry+0xf8
02 0007f224 7c90d16c 7c91c993 0007f368 0000000f ntdll!KiFastSystemCallRet
03 0007f228 7c91c993 0007f368 0000000f 00000000 ntdll!NtCreateSection+0xc
04 0007f2a8 7c91c64a 0007f340 00000790 0007f300 ntdll!LdrpCreateDllSection+0x92
05 0007f388 7c91624a 000add00 0007f414 0007f93c ntdll!LdrpMapDll+0x28f
06 0007f648 7c9164b3 00000000 000add00 0007f93c ntdll!LdrpLoadDll+0x1e9
07 0007f8f0 7c801bbd 000add00 0007f93c 0007f91c ntdll!LdrLoadDll+0x230
08 0007f958 7c801d72 7ffddc00 00000000 00000000 kernel32!LoadLibraryExW+0x18e
09 0007f96c 7ca625a3 7ca625ac 00000000 00000000 kernel32!LoadLibraryExA+0x1f
WARNING: Stack unwind information not available. Following frames may be wrong.
0a 0007f990 010057b8 000700ac 000a7c84 00000000 SHELL32!SHCreateQueryCancelAutoPlayMoniker+0x2062d
0b 0007fbc4 010041ac 0000012e 00000111 01006118 calc!MenuFunctions+0x15d
0c 0007fcb4 01004329 0000012e 00000111 01006118 calc!RealProcessCommands+0x1b61
0d 0007fcdc 01006521 0000012e 0007fd6c 01006118 calc!ProcessCommands+0x2d
0e 0007fd04 7e418734 000700ac 00000111 0000012e calc!CalcWndProc+0x409
0f 0007fd30 7e418816 01006118 000700ac 00000111 USER32!InternalCallWinProc+0x28
10 0007fd98 7e4189cd 00000000 01006118 000700ac USER32!UserCallWinProcCheckWow+0x150
11 0007fdf8 7e418a10 0007fee8 00000000 0007ff1c USER32!DispatchMessageWorker+0x306
12 0007fe08 010021a7 0007fee8 7c80b731 000a1ee4 USER32!DispatchMessageW+0xf
13 0007ff1c 010125e9 000a7738 00000055 000a7738 calc!WinMain+0x256
14 0007ffc0 7c817067 00000000 00000000 7ffde000 calc!WinMainCRTStartup+0x174
15 0007fff0 00000000 01012475 00000000 78746341 kernel32!BaseProcessStart+0x23
STACKtrace without warnings
Breakpoint 0 hit
nt!NtCreateSection:
805652b3 6a2c push 2Ch
kd> kb
# ChildEBP RetAddr Args to Child
00 f8aa0d40 804de7ec 0007f368 0000000f 00000000 nt!NtCreateSection
01 f8aa0d40 7c90e4f4 0007f368 0000000f 00000000 nt!KiFastCallEntry+0xf8
02 0007f224 7c90d16c 7c91c993 0007f368 0000000f ntdll!KiFastSystemCallRet
03 0007f228 7c91c993 0007f368 0000000f 00000000 ntdll!NtCreateSection+0xc
04 0007f2a8 7c91c64a 0007f340 00000790 0007f300 ntdll!LdrpCreateDllSection+0x92
05 0007f388 7c91624a 000add00 0007f414 0007f93c ntdll!LdrpMapDll+0x28f
06 0007f648 7c9164b3 00000000 000add00 0007f93c ntdll!LdrpLoadDll+0x1e9
07 0007f8f0 7c801bbd 000add00 0007f93c 0007f91c ntdll!LdrLoadDll+0x230
08 0007f958 7c801d72 7ffdfc00 00000000 00000000 kernel32!LoadLibraryExW+0x18e
09 0007f96c 7ca625a3 7ca625ac 00000000 00000000 kernel32!LoadLibraryExA+0x1f
0a 0007f97c 7ca62e8e 003800dd 000a7c84 00000000 SHELL32!GetXPSP1ResModuleHandle+0x16
0b 0007f990 010057b8 000900ac 000a7c84 00000000 SHELL32!ShellAboutW+0x1f
0c 0007fbc4 010041ac 0000012e 00000111 01006118 calc!MenuFunctions+0x15d
0d 0007fcb4 01004329 0000012e 00000111 01006118 calc!RealProcessCommands+0x1b61
0e 0007fcdc 01006521 0000012e 0007fd6c 01006118 calc!ProcessCommands+0x2d
0f 0007fd04 7e418734 000900ac 00000111 0000012e calc!CalcWndProc+0x409
10 0007fd30 7e418816 01006118 000900ac 00000111 USER32!InternalCallWinProc+0x28
11 0007fd98 7e4189cd 00000000 01006118 000900ac USER32!UserCallWinProcCheckWow+0x150
12 0007fdf8 7e418a10 0007fee8 00000000 0007ff1c USER32!DispatchMessageWorker+0x306
13 0007fe08 010021a7 0007fee8 7c80b731 000a1ee4 USER32!DispatchMessageW+0xf
14 0007ff1c 010125e9 000a7738 00000055 000a7738 calc!WinMain+0x256
15 0007ffc0 7c817067 00000000 00000000 7ffda000 calc!WinMainCRTStartup+0x174
16 0007fff0 00000000 01012475 00000000 78746341 kernel32!BaseProcessStart+0x23
dump the arguments to NtCreateSection
kd> dds #esp l8
f8bb1d44 804de7ec nt!KiFastCallEntry+0xf8
f8bb1d48 0007f368
f8bb1d4c 0000000f
f8bb1d50 00000000
f8bb1d54 00000000
f8bb1d58 00000010
f8bb1d5c 01000000 calc!_imp__RegOpenKeyExA <PERF> (calc+0x0)
f8bb1d60 00000790
we know the seventh argument is HANDLE according to prototype of DDI
NTSTATUS ZwCreateSection(
_Out_ PHANDLE SectionHandle,
_In_ ACCESS_MASK DesiredAccess,
_In_opt_ POBJECT_ATTRIBUTES ObjectAttributes,
_In_opt_ PLARGE_INTEGER MaximumSize,
_In_ ULONG SectionPageProtection,
_In_ ULONG AllocationAttributes,
_In_opt_ HANDLE FileHandle
);
kd> !handle 790
Failed to get VAD root
PROCESS ffae2020 SessionId: 0 Cid: 0410 Peb: 7ffde000 ParentCid: 00a8
DirBase: 04d87000 ObjectTable: e1bd5238 HandleCount: 29.
Image: calc.exe
Handle table at e1bd5238 with 29 entries in use
0790: Object: 8124b028 GrantedAccess: 00100020 Entry: e1032f20
Object: 8124b028 Type: (8127b900) File
ObjectHeader: 8124b010 (old version)
HandleCount: 1 PointerCount: 1
Directory Object: 00000000 Name: \WINDOWS\system32\xpsp1res.dll {HarddiskVolume1}
return back to user mode from kernel mode and inspect the new Section Handle
kd> g
eax=00000000 ebx=00000000 ecx=00000001 edx=ffffffff esi=0007f368 edi=00000000
eip=7c90d16c esp=0007f22c ebp=0007f2a8 iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
ntdll!ZwCreateSection+0xc:
7c90d16c c21c00 ret 1Ch
checking the return value of HANDLE in user mode
0:000> dd 7f368 l1
dd 7f368 l1
0007f368 0000078c
0:000> !handle 78c
!handle 78c
Handle 78c
Type Section
0:000> !handle 78c f
!handle 78c f
Handle 78c
Type Section
Attributes 0
GrantedAccess 0xf:
None
Query,MapWrite,MapRead,MapExecute
HandleCount 2
PointerCount 3
Name <none>
Object Specific Information
Section base address 0
Section attributes 0x1800000
Section max size 0x2f000
0:000>
If not satisifed we can revert to kd set the process context and check the returned handle in kernel mode
kd> !handle 78c f
Failed to get VAD root
PROCESS ffae2020 SessionId: 0 Cid: 0410 Peb: 7ffde000 ParentCid: 00a8
DirBase: 04d87000 ObjectTable: e1bd5238 HandleCount: 30.
Image: calc.exe
Handle table at e1bd5238 with 30 entries in use
078c: Object: e1088f30 GrantedAccess: 0000000f Entry: e1032f18
Object: e1088f30 Type: (8128b900) Section
ObjectHeader: e1088f18 (old version)
HandleCount: 1 PointerCount: 1
now if you continue execution you can see the loaded library dbgprint in windbg and the about dialog in vm :)
kd> g
0:000> g
g
ModLoad: 10000000 1002f000 C:\WINDOWS\system32\xpsp1res.dll

Which version of sos and clr/mscorwks.dll to load?

I have a 32-bit application (targeting .NET 3.5) hosted on a 64-bit machine. I want to analyze the memory dump of this 32-bit application. I captured the memory dump using 32-bit adplus and cdb. I am loading the memory dump into 32-bit windbg. When I load .net 2.0 sos.dll and .net 2.0 mscorwks.dll into windbg and execute !clrstack, I get the following error: "Failed to find runtime DLL (mscorwks.dll), 0x80004005
Extension commands need mscorwks.dll in order to have something to do." What am I doing wrong?
Info as requested in the comments
ADPlus command line:
adplus -hang -quiet -p 2440 -o C:\temp
WinDbg commands:
0:000> .load <fullpathto>\sos.dll
0:000> lmvm mscorwks
start end module name
0:000> .exr -1
ExceptionAddress: 00000000
ExceptionCode: 80000007 (Wake debugger)
ExceptionFlags: 00000000
NumberParameters: 0
The dump indicates that no .NET 2 was loaded. Otherwise the output of lmvm mscorwks should show the details of the .NET runtime, like this:
0:003> lmvm mscorwks
start end module name
61bc0000 6216e000 mscorwks (deferred)
Image path: C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll
...
File version: 2.0.50727.5485
...
You mentioned that you loaded SOS by full path. If the dump was taken on your machine, you would typically load it using
0:003> .loadby sos mscorwks
In your case, this should already give you the hint that .NET was not loaded:
Unable to find module 'mscorwks'
If you're not so sure about the .NET version, try
.loadby sos clr; *** .NET 4
.loadby sos coreclr; *** Silverlight / Universal Apps
Maybe you had a typo in your AdPlus command line and specified the wrong process ID. If that PID accidentally exists, you got a wrong dump. Use | to check the process name
0:003> |
. 0 id: 1e78 attach name: E:\...\NET2x32.exe
BTW: The -quiet parameter of ADPlus is obsolete, you can omit it.

Bad symbols for NTDLL (error 3). Aborting

What could cause the error:
Bad symbols for NTDLL (error 3). Aborting.
While using !cs commands?
I'm analyzing a keyboard-forced complete dump and have set the symbol path to the MS symbol server.
lml does not list ntdll as loaded
.sym noisy and a reload shows:
3: kd> .reload ntdll
"ntdll" was not found in the image list.
Debugger will attempt to load "ntdll" at given base 00000000`00000000.
Please provide the full image name, including the extension (i.e. kernel32.dll)
for more reliable results.Base address and size overrides can be given as
.reload <image.ext>=<base>,<size>.
DBGENG: ntdll - Partial symbol image load missing image info
DBGHELP: No header for ntdll. Searching for dbg file
DBGHELP: .\ntdll.dbg - file not found
DBGHELP: ntdll missing debug info. Searching for pdb anyway
DBGHELP: Can't use symbol server for ntdll.pdb - no header information available
DBGHELP: ntdll.pdb - file not found
DBGHELP: ntdll - no symbols loaded
Unable to add module at 00000000`00000000
3: kd> .reload nt
DBGHELP: nt - public symbols
c:\symbols\ntkrnlmp.pdb\BF9E190359784C2D8796CF5537B238B42\ntkrnlmp.pdb
Should ntdll be listed in the lm commands?
Is there something special needed to load the symbols or run these commands?
Additionally I am also curious why I cannot dump all threads in forced/complete system dumps:
3: kd> ~* k
^ Syntax error in '~* k'
Have I simply failed to set up my windbg correctly or are these commands meant for other types of dumps?
It seems you have done user mode debugging before. Now you're in kernel mode, which you can see from the x: kd> prompt.
Kernel mode debugging is somewhat different to user mode debugging. Most important IMHO: not all application memory (virtual memory) is available, just the part that was in RAM at the time of the dump (physical memory) aka. Working Set (physical memory of a particular process).
You can search for your executable using !process 0 0 <exename>
0: kd> !process 0 0 NotMyFault.exe
PROCESS ff3b58f0 SessionId: 0 Cid: 05ac Peb: 7ffde000 ParentCid: 039c
DirBase: 018c02e0 ObjectTable: e165d728 HandleCount: 35.
Image: NotMyfault.exe
Unfortunately there is no wildcard search possible at this point. You can then switch to that process using .process <process>:
0: kd> .process ff3b58f0
Implicit process is now ff3b58f0
To see the threads use !process <process> 2:
0: kd> !process ff3b58f0 2
PROCESS ff3b58f0 SessionId: 0 Cid: 05ac Peb: 7ffde000 ParentCid: 039c
DirBase: 018c02e0 ObjectTable: e165d728 HandleCount: 35.
Image: NotMyfault.exe
THREAD ff1d4020 Cid 05ac.05b0 Teb: 7ffdd000 Win32Thread: e1c6e2b0 RUNNING on processor 0
Next, use !thread <thread> to get output similar to ~:
0: kd> !thread ff1d4020
THREAD ff1d4020 Cid 05ac.05b0 Teb: 7ffdd000 Win32Thread: e1c6e2b0 RUNNING on processor 0
IRP List:
81942f68: (0006,0094) Flags: 40000000 Mdl: 00000000
Not impersonating
DeviceMap e169ffd0
Owning Process 0 Image: <Unknown>
Attached Process ff3b58f0 Image: NotMyfault.exe
Wait Start TickCount 13575 Ticks: 0
Context Switch Count 653 IdealProcessor: 0 LargeStack
UserTime 00:00:00.000
KernelTime 00:00:00.078
Win32 Start Address NotMyfault (0x01002945)
Start Address kernel32!BaseProcessStartThunk (0x7c8106f5)
Stack Init f36f1000 Current f36f030c Base f36f1000 Limit f36ec000 Call 0
Priority 9 BasePriority 8 PriorityDecrement 0 DecrementCount 16
ChildEBP RetAddr Args to Child
f36f0ad0 8052036a 00000050 81617000 00000001 nt!KeBugCheckEx+0x1b (FPO: [Non-Fpo])
f36f0b38 80544578 00000001 81617000 00000000 nt!MmAccessFault+0x9a8 (FPO: [Non-Fpo])
f36f0b38 fca6161d 00000001 81617000 00000000 nt!KiTrap0E+0xd0 (FPO: [0,0] TrapFrame # f36f0b50)
f36f0bd8 fca61a24 81942f68 f36f0c1c fca61b26 myfault+0x61d
f36f0be4 fca61b26 80fa7350 00000001 00000000 myfault+0xa24
f36f0c1c 804ef18f 80f7b600 81942f68 806e6428 myfault+0xb26
...
If you have trouble loading symbols, also try .reload /user after using the .process <process>.

windbg conflicting information

I have WinDBG 6.12.0002.633 x86 and I'm using it to view a post-mortem kdmp from a Windows Mobile 6 ARMV4I application.
When I go to analyze the callstack I get a lot of unknowns. In the analysis, I can see in the *FAULTING_IP* section that the fault is in the tcpstk module. (for which I also have symbols. But, in the *STACK_TEXT* section, the tcpstk addresses appear as just addresses, no symbols.
Also, in the *MODULE_NAME* section, I get another unknown even though it just said the faulting module was in tcpstk.
The result of the !analyze -v command is:
1:128:armce> !analyze -v
***snip!***
FAULTING_IP:
tcpstk!_DerefIF+38 [\private\winceos\comm\tcpipw\ip\iproute.c # 1032]
01b0d6f0 ???????? ???
***snip!***
IP_ON_HEAP: 07b00090
The fault address in not in any loaded module, please check your build's rebase
log at <releasedir>\bin\build_logs\timebuild\ntrebase.log for module which may
contain the address if it were loaded.
FRAME_ONE_INVALID: 1
STACK_TEXT:
761efa6c 07b00090 : 7b858453 00000003 00000000 00000000 : 0x7b0d6f0
761efa7c 07b0020c : 7b858453 506f010a 00000000 00000000 : 0x7b00090
761efacc 78012d38 : 7b858453 506f010a 00000000 00000000 : 0x7b0020c
761efaf4 78013cdc module_78010000!AdapterBindingManager::NetUp+0xb4 [bar.cpp # 268]
761efb34 78014b78 module_78010000!AdapterBindingManager::EnterState+0x5e4 [bar.cpp # 1327]
761efda4 78015c08 module_78010000!AdapterBindingManager::ProcessEvent+0x8e4 [bar.cpp # 1298]
761efdd8 03f668dc module_78010000!MediaSense+0x25c [foo.cpp # 673]
761efe94 00000000 coredll_3f49000!ThreadBaseFunc+0x98 [\private\winceos\coreos\core\dll\apis.c # 633]
MODULE_NAME: Unknown_Module
IMAGE_NAME: Unknown_Image
DEBUG_FLR_IMAGE_TIMESTAMP: 0
STACK_COMMAND: ~128s ; kb
FAILURE_BUCKET_ID: INVALID_POINTER_WRITE_c0000005_Unknown_Image!Unknown
If I switch to the kp command, I suddenly can see that part of the callstack
1:128:armce> kp
Child-SP RetAddr Call Site
761efa6c 01b0d6e0 tcpstk!_DerefIF(struct Interface * IF = 0x7b858453)+0x38 [\private\winceos\comm\tcpipw\ip\iproute.c # 1032]
761efa6c 00000000 tcpstk!_DerefIF(struct Interface * IF = 0x7b858453)+0x28 [\private\winceos\comm\tcpipw\ip\iproute.c # 1026]
Why isn't the !analyze -v command able to show the fully decoded callstack? Why does it show so many unknowns?
I think that WinDBG cannot debug ARM I have not seen any documentation that states it is capable of debugging ARM, only x86 and x64 applications.
There is a Windbg provided in the ARM toolkit that is the windowed version of armsd which is not related to the microsoft WindDbg.