Links for FCMDA-materials

http://www.dumpanalysis.com/FCMDA-materials
http://www.dumpanalysis.com/FCMDA-materials-Rev2

Visual Studio: How to: Debug in Mixed Mode

http://msdn.microsoft.com/en-us/library/kbaht4dh(v=vs.80).aspx

!htrace: The difference between the two snapshot is too large

Click on Debug > Break (or just CTRL+Break)

Type “!htrace –diff” without the quotation marks and then press Enter.

Note: By default, Windows Vista, Windows Server 2008, Windows 7 and Windows Server 2008 R2 keep a history of 4000 handles open and close operations.
With !htrace you can enable to keep a much higher history by doing the following:

Type “!htrace –enable 0x20000” without the quotation marks and then press Enter.
In this example, we are increasing the handle history to 128000 (decimal, 0x20000 hexadecimal).


Link:- http://blogs.technet.com/b/yongrhee/archive/2011/12/19/how-to-troubleshoot-a-handle-leak.aspx

MSIL tutorials

The following is some useful links to MSIL tutorials.

• Introduction to MSIL by Kenny Kerr
• ILDASM is Your New Best Friend by John Robbins
• MSIL Tutorial by Alex F

Windows 7 No Longer Supports Full Memory Dumps??

A new registry value has been added that will override this behavior and always cause the crash dump file to be generated. By setting the \HKLM\System\CCS\Control\CrashControl\AlwaysKeepMemoryDump DWORD value to 1 you will guarantee that you will always have a crash dump file after crashing the system.

For details fo though this:- http://www.osronline.com/article.cfm?article=545

Debugging Focus Bugs

1. Jeffrey Richter wrote an article in 1997 which contains a tool for finding the focus among other things: http://www.microsoft.com/msj/0397/Win32/Win320397.aspx
2. We can use SPY++ and analyze messages
3. We can do Remote Debugging and Multi-Monitor.
4. The Inspect tool, part of the Windows SDK download, can be useful here. It's designed to test out the two Accessibility-related APIs - MSAA and UI Automation - and one of the things that both those APIs to is allow accessibility and test tools to track the focus.
   The simplest way to use it for tracking focus is to put it into MSAA mode, check the options to follow only focus changes (ie. turn off following the mouse pointer), then turn on the yellow highlight rectangle. Now as focus changes, you can see the rectangle move. As a bonus, if focus goes to something that's hidden or offscreen, you won't see a rectangle, but the name and Win32 class of the control will be displayed in the window.
   Note that Inspect shows a superset of focus events: you get not just HWND focus changes, but also notifications when focus moves within certain controls - such as between items in a list box. Automated test and accessibility need these, but for your purposes you should be ok to just ignore these; it's extra information, but shouldn't get in the way too much.
5. Win32 Activation and Focus :- http://blogs.msdn.com/b/jfoscoding/archive/2006/08/02/686141.aspx

DllMain and RPC Call

This post is calling RPC function from DllMain()

ntdll!ZwWaitForSingleObject+0x15 (FPO: [3,0,0])
ntdll!RtlpWaitOnCriticalSection+0x1a3 (FPO: [Non-Fpo])
ntdll!RtlEnterCriticalSection+0xa8 (FPO: [Non-Fpo])
RPCRT4!PerformRpcInitialization+0x22 (FPO: [Non-Fpo])
RPCRT4!RpcStringBindingComposeW+0x14 (FPO: [Non-Fpo])
somedll!InternalDllMain+0x41 (FPO: [Non-Fpo]) (CONV: stdcall) [f:\sp\vctools\vc7libs\ship\atlmfc\src\mfc\dllmodul.cpp @ 119]
somedll!__DllMainCRTStartup+0x6c (FPO: [Non-Fpo]) (CONV: cdecl) [f:\sp\vctools\crt_bld\self_x86\crt\src\dllcrt0.c @ 365]
somedll!_DllMainCRTStartup+0x1d (FPO: [3,0,0]) (CONV: stdcall) [f:\sp\vctools\crt_bld\self_x86\crt\src\dllcrt0.c @ 328]
ntdll!LdrpCallInitRoutine+0x14
ntdll!LdrpRunInitializeRoutines+0x367 (FPO: [Non-Fpo])
ntdll!LdrpLoadDll+0x44b (FPO: [Non-Fpo])
ntdll!LdrLoadDll+0x198 (FPO: [Non-Fpo])
kernel32!LoadLibraryExW+0x1b2 (FPO: [Non-Fpo])
kernel32!LoadLibraryW+0x11 (FPO: [Non-Fpo])
someexe!somefunction

Assembly for PerformRpcInitialization(This is not full assembly but only which is required for this post)

RPCRT4!PerformRpcInitialization:
7da3933f 8bff mov edi,edi
7da39341 55 push ebp
7da39342 8bec mov ebp,esp
7da39344 83ec28 sub esp,28h
7da39347 56 push esi
7da39348 33f6 xor esi,esi
7da3934a 3935dc00ad7d cmp dword ptr [RPCRT4!RpcHasBeenInitialized (7dad00dc)],esi
7da39350 0f842eec0000 je RPCRT4!PerformRpcInitialization+0x17 (7da47f84)

RPCRT4!PerformRpcInitialization+0x236:
7da39356 33c0 xor eax,eax

RPCRT4!PerformRpcInitialization+0x238:
7da39358 5e pop esi
7da39359 c9 leave
7da3935a c3 ret

RPCRT4!PerformRpcInitialization+0x17:
7da47f84 687800ad7d push offset RPCRT4!GlobalMutex (7dad0078)
7da47f89 ff15b402a37d call dword ptr [RPCRT4!_imp__RtlEnterCriticalSection (7da302b4)]
7da47f8f 3935dc00ad7d cmp dword ptr [RPCRT4!RpcHasBeenInitialized (7dad00dc)],esi7da47f95 0f85ba010000 jne RPCRT4!PerformRpcInitialization+0x227 (7da48155)

RPCRT4!PerformRpcInitialization+0x2e:
7da47f9b 6888c1a37d push offset RPCRT4!`string' (7da3c188)
7da47fa0 ff155c01a37d call dword ptr [RPCRT4!_imp__LoadLibraryW (7da3015c)]7da47fa6 85c0 a test eax,eax

If you see lines marked in bold, it could lead into another loadlibrary call, hence deadlock could happen.

If rpc is not initialized then it will endup into loadlibrary(). Hence it will try to acquire loaderlock which is already hold by our process hence deadlock!!!