memory leak detection notes (continuously updating)

Memory leak problem is very common, below is some my previous working notes:

• BoundsCheck

BoundsCheck does not work well with mixed codes. Since my latest projects are almost all mixed, so, seldom used in my daily work.

• Visual studio
• Enable memory leak reporting

There are two ways to enable memory leak check in VC, one is to define DEBUG_NEW macro, another is as below:

   1: #define _CRTDBG_MAP_ALLOC 

   2: #include<stdlib.h> 

   3: #include<crtdbg.h>

Then calling _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF ); when initializing your app.

With the above changes, visual studio can dump memory block leak when quitting or between two snapshot via Windows API. For instance:

c:\program files(x86)\microsoft visual studio\vc98\include\crtdbg.h(552) : {43} normal
block at 0x00441C20, 40 bytes long.
Data: < C > 08 02 43 00 16 00 00 00 00 00 00 00 00 00 00 00

• Output mode

The above output is one sample of block leak, in which, {43} is the heap block allocation id, which can be used to set breakpoints. “normal” is the classification of heap blocks, some other types include: client/crt/free (free will no appear here). Then double click the output item, the allocation location can be watched.

By default, visual studio generate the report to the output window, which can also be changed by CrtSetReportMode calling, for instance: to std-err console or some files (needing to call _CrtSetReportFile first)

• Conditional breakpoint by allocation #

Input {,,msvcrtd.dll}_crtBreakAlloc in your watch window, by default value would be –1, which is the condition when allocation would be broken, for instance, 43, and when the allocation # equals to 43, DbgBreak is triggered. The value can be changed manually in visual studio.

You can also call _CrtSetBreakAlloc(43) to do the same thing.

• create memory snapshot and compare difference

   1: _CrtMemState s1, s2, s3;

   2: _CrtMemCheckpoint( &s1 );

   3: _CrtMemCheckpoint( &s2 );

   4: if ( _CrtMemDifference( &s3, &s1, &s2) )

   5:     _CrtMemDumpStatistics( &s3 );

The above codes show how to do this. In the middle, CrtMemDumpStatistics can be used to generate statistics info for each _CrtMemState snapshot.

• AppVerif and Windbg

We can also get stacktrace for each leaking block above by enable it in gflags.exe. Then, attaching your target with WinDbg, check the stacktrace info for each heap by typing:

!heap –p –a <blockAddr>

Then, the block info like type, stacktrace will be displayed.

• LeakDiag vs UMPH

Both these tools can help us generate memory leak reports. UMPH can work only with standard heap allocation, while LeakDiag can work with all 6 types of allocations:

standard windows heap allocation

virtual memory allocator

MPHeap allocator

COM Allocator

COM Internal Allocator

C Runtime Allocator

With LeakDiag, you can simply create two snapshots, then compare them as you wish, similar to the above steps in visual studio. Then, a final xml file can be generated.

one sample can be found:

http://mcfunley.com/277/using-leakdiag-to-debug-unmanaged-memory-leaks

http://thetweaker.wordpress.com/2009/04/09/native-memory-leaks-part-1-leakdiag/

• Visualizing the report

For LeakDiag, there are some tools to view the report, like: “Leak XML Logfile Analyzer”, “LDGrapher”,

c++/CLR/SEH/MFC/VEH exception notes (continuously updating)

• SEH/VEH

A new interesting topic is VEH (vector exception handler), which is introduced since windows XP. The difference is as below:

• Not like SEH, which is saved in the TEB, is popped as stack unwinds and only valid in the current thread; VEH is saved in the global heap, which is effective for the whole process;
• All VEH will be executed before SEH
• VEH only exists in the user-mode codes, SEH can be both user/kernel modes
• SEH is registered/revoked by the codes generated by compiler, VEH is registered/revoked by system API:

AddVectoredExceptionhandler

RemoveVectoredExceptionHandler

• The signature for VEH is:

LONG CALLBACK VectorHandler(PEXCEPTION_POINTERS ExceptionInfo)

• C++/CLR/SEH exception
• relationship

SEH exception means structured exception handler, CLR exception and C++ exception are two special SEH one, each has its own SEH code. C++ exception code is: 0xe06d7363 and CLR is 0xe0434f4d. C++ exception keywords are try/catch, SEH is __try/__except, MFC is the TRY/CATCH macros, built on the top of SEH keywords. Microsoft strongly suggests converting usage of MFC exceptions to C++ ones, and also provides some samples about how to convert correctly.

http://msdn.microsoft.com/en-us/library/19z28s5c.aspx

• 64-bit windows no longer saves exception handler on the stack

In 64-bit windows, the exception handler is no longer saved on the stack, instead an exception handler table is defined to hold them.

• translate SEH to c++ exception

SEH, also can be called as c exception, not like c++ one, only deal with integer, and c++ exception is based on the types. We can also use _set_se_translator to enable the translation from SEH exception to c++ one. For instance:

   1: // from msdn http://msdn.microsoft.com/en-us/library/5z4bw5h5%28v=VS.90%29.aspx

   2: // crt_settrans.cpp

   3: // compile with: /EHa

   4: #include <stdio.h>

   5: #include <windows.h>

   6: #include <eh.h>

   7:  

   8: void SEFunc();

   9: void trans_func( unsigned int, EXCEPTION_POINTERS* );

  10: class SE_Exception

  11: {

  12: private:

  13:     unsigned int nSE;

  14: public:

  15:     SE_Exception() {}

  16:     SE_Exception( unsigned int n ) : nSE( n ) {}

  17:     ~SE_Exception() {}

  18:     unsigned int getSeNumber() { return nSE; }

  19: };

  20: int main( void )

  21: {

  22:     try

  23:     {

  24:         _set_se_translator( trans_func );

  25:         SEFunc();

  26:     }

  27:     catch( SE_Exception e )

  28:     {

  29:         printf( "Caught a __try exception with SE_Exception.\n" );

  30:     }

  31: }

  32:  

  33: void SEFunc()

  34: {

  35:     __try

  36:     {

  37:         int x, y=0;

  38:         x = 5 / y;

  39:     }

  40:     __finally

  41:     {

  42:         printf( "In finally\n" );

  43:     }

  44: }

  45: void trans_func( unsigned int u, EXCEPTION_POINTERS* pExp )

  46: {

  47:     printf( "In trans_func.\n" );

  48:     throw SE_Exception();

  49: }

• Async and Sync exception

We often mention async and sync exception models, since for the async case, there is no explicit statement throwing the exception, for the sync, we can always find kernel32!RaiseException on the callstack. The basic SEH is the async model, for instance, processor thrown when finding access violation.

Visual studio can specify two exception models, /Eha and /Ehsc, for async and sync mode respectively. CLR always assumes /Eha, native C++ program can use either of them. In the mixed programming, when exception thrown from native, caught in managed, different exception models can cause trouble (refer to my another post). When specifying async mode, c++ program can also deal with SEH ones, not necessary use __try statements.

• MFC exception

As for MFC exception, it does not belong to c++ exception, since when Microsoft introduces exception handling to MFC, the c++ exception standard may not be mature. In your new application, C++ exception is suggested, but the old MFC one is still available. But additional attention should be paid when mixing them in your application. Refer to msdn: http://msdn.microsoft.com/en-us/library/sas5wzs9%28v=vs.80%29.aspx. Since MFC macros automatically deleted caught exceptions when they go out of blocks, while c++ does not. So, don’t mix them in one block.

Furthermore, MFC exception is still a special SEH exception, whose code is: 0XE04D5343 and the ASCII is “.msc”, kind of microsoft standard.

• DebugOutputString and exception

Another interesting thing is that DebugOutputString API is also implemented based on exception, so, in windbg we can also set breakpoints or execute commands based on the target’s debug output. For instance:

sxe out:Open?Database*: stop when finding matched trace output like: “Open Database 1”

.ocommand “MyWindbgCmd:” will execute command “!mk;g;” when finding “MyWindbgCmd: !mk;g;”

• exception handler list in TEB

In the thread block associated with each thread, there is a pointer to stack of the exception handlers in the current thread. The top is the inner-most one.

The exception handler chain can be watched in Windbg via “!exchain”

• some good article links:

Differences in Exception Handling Behavior Under /CLR

http://msdn.microsoft.com/en-us/library/2ww6y7y2.aspx

Exceptions: Converting from MFC Exception Macros

http://msdn.microsoft.com/en-us/library/19z28s5c.aspx

Assembly loading strategy misc notes (continuously updated)

 

• Load from GAC
• Load from private path

   1: <runtime>

   2:   <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">

   3:     <probing privatePath="myPrivatePath"/>

   4:   </assemblyBinding>

   5: </runtime>

• Change machine.config to load from anywhere globally on your machine
  
• Overwrite private path setting

   1: <runtime>

   2:   <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">

   3:     <probing privatePath="myPrivate"/>

   4:     <dependentAssembly>

   5:       <assemblyIdentity name="MyAssemblyPath"

   6:                         publicKeyToken="abcd"

   7:                         culture="neutral" />

   8:       <codebase href = "file://localhost/D:/Temp/DLL/MyAssembly.dll">

   9:       <!--Redirection and codeBase policy for myAssembly.-->

  10:     </dependentAssembly>

  11:   </assemblyBinding>

CLR Exception Misc Notes(continuously updated)

 

• Different unhandled exception for WinForm and Console (from debugging .net app 2.0)

for winform, if unhandled exception thrown from pool thread/background thread/ finalize thread, AppDomain.UnhandledException event will be called.

For Console, main thread still runs, just as if you have handled them

• Unhandled exception dialogs shows while your app still running

so, you app’s state may be changed, you cannot get the image when the exception is thrown, until you set external debugger to auto attach when it happens.

 

 

Rebase your dlls to get more compact address space

 

Generally, each dll loaded has a preferable starting address, which is determined by the compiler, by default, assigned by name. During the loading, if conflict occurs, loader will relocate them. So, if too many conflicts happens, it may cause starting performance problem.

 

In one of our product, we are using a bunch of 3rd binary packages from one company, the names follow the pattern “FO***”, for most of client machines, they seem to be the only libraries to be loaded into address space starting with that name. After watching the run-time address space of our distributed program, it seems that the compiler is not allocating compact enough address policy, there are several mega free spaces between two adjacent libraries. Since our product is memory extensive one, I manually rebased those dlls and it seems that dozens of 10M memory can be grouped into one 150M memory finally. So, some operations which caused out-of-memory exceptions now can work well.

The basic step is rather straightforward.

• Dump the run-time address space for you program first by using vmmap.exe from sysinternal or using !address from windbg, the first is much more friendly.
• reassign the proper base address for dlls
• Use rebase.exe from visual studio 2008 command, the syntax is:

rebase.exe –b newAddress dllName.dll

 

Just found another interesting method:

rebase.exe /b Ginger.dll GooseBerries.dll

can make those two dlls auto rebased back-to-back.

 

WinDbg debugging scenario (2) find clr gchandle leak

 

!gchandleleak can search heap to get referenced handles, may have false positive. But if the number keeps increasing after each specific operation, then it may cause handle leaks.

!do poi(handleValue) to see what is contained by the handle.

 

For most cases, PowerDbg can be used to automatically check the leakages.

Windbg Debugging Scenario (1): Finding un-disposed objects

 

Use !finalizequeue to find all disposible objects, then use !gcroot to check whether they are orphan, then dump the orphaned objects, in most case, developers will define some private flags like _IsDisposed or _IsClosed, if already orphaned, should be true.

 

generally, the steps can be completed by PowerDbg scripts.

Windbg Debugging Misc Tips (Continuously Updating)

 

• Native
• .prefer_dml 1:

a hidden command, enable marking language, showing hyperlinks in some commands, like lm, etc

• .exr -1

show last exception, for both native or managed

• “.symopt+0x40”:force debugger to load unmatched symbol files, like time-stamp mis-match. To disable, –0x40
• .ocommand: execute command from the target trace,

.command “Sa Windbg Cmd:”

then, in the target program, debug output can be written:

Trace.WriteLine(“sa Windbg Cmd: !mk;g;”);

• Managed
• Threads/Callstack/Frame
• !mk

show both managed/native callstack

• !mframe/!mdv

switch to managed frame or show variables for some frame, get frame index from !mk

• Exception/Breakpoints
• debug CLR start-up code:

bu mscorwks!EEStartup “gu;.loadby sos mscorwks”

• !pe

display exception object or most recent exception if no argument

• !soe –derived | –create | –create2 <typeName> [pseudo-register number for results]

e.g, !soe –create System.OutOfMemoryException 1

!soe System.OutOfMemoryException 1; if (@$t1 == 0) {g};

• good sxe sample
• check if it is the expected exception whenever a clr exception thrown:

sxe –c “!soe System.ArgumentNullException 1; .if (@#t1 == 1) {g} .else{.echo “ArgumentNullException hit”}” clr

sxe –c “!soe System.ArgumentNullException 5; !soe System.NotImplementedException 6; .if @@(@#t5 == 0 && @$t6 == 0) {g} .else{‘my exceptions hit’}” clr

"sxe -c "!mk;g" clr"

stop when finding matched trace output(sxe out:Open?Database*)

• Symbols
• !mln

display symbol info for given address

• !mx

display matching symbols

• Heap/Handle Dump/Inspect
• !dumpgen 3

display LOH objects, kind of similar to !dumpheap –min 85000, since LOH also includes quite a few smaller heap blocks.

• !psscor2.gchandleleaks

check potential gchandle leaks, may have false positive info, but generally, if it keeps increasing for each operation, then the operation may cause some handle leaks.

• Object Dump/Analyze
• !gcroot

display references for given object 

 

 

 

 

VS Tips (Continuously updated)

• Special commands in Immediate/Watch windows
  • c# temp variables defined in Immediate Window

input “int tempVar = 0;” in the immediate window, then type $temp or @temp in Watch window, see what happens. Can be helpful combined with the usage of Conditional Breakpoints.

VS can even create an object in immediate window, like Something s = new Something();

• Time-measurement feature by conditional breakpoint

set a trace point, and change the statement as below:

{@clk}, {@clk=0}, $FUNCTION

on my 64-bit win7, time resolution is 10-6 second

see what happens

• control var’s display suffix
  • var,h: in watch window shows hex value
  • var,nq: show string without escaped quotes, helpful when showing “\””
  • var,ac: force to evaluate the expression even if “implicit expression evaluation” is disabled in debug-general setting. May be helpful to watch some variables which has nasty side effect when auto evaluate expression, like properties
• watch objects out of scope: “Make Object Id”, then watch by “1#”, “2#”, etc. Even if the object is being GCed.
• pseudo-vars in watch windows:
  • $exception
  • $user
• Visualizer in watch window
  • for c#:
    • MyDocument\Visual Studio 2008\Visualizer\autoexp.cs and autoexp.dll
    • write your own visualizer plug-in
    • customize debugdisplay attribute in c# classes
  • for native c++: c:\program files(x86)\visual studio 2008\common7\packages\debugger\autoexp.dat
• Execute SOS from immediate window

.load sos

note: only 32bit mixed debugging mode supports this

• Debugger Behavior
  • Set Breakpoint at any system function

type “{,,user32.dll}_SendMessageA@16” in the New breakpoint window

some function has different symbol name, like: LockWindowUpdate

{,,user32.dll}_NtUserLockWindowUpdate@4 should be used.

• Why Edit&Continue not working for my project?

refer to below link:

http://msdn.microsoft.com/en-us/library/ms164927%28v=VS.80%29.aspx

some common cases when E&C not available are:

mixed debugging

64-bit CLR

attached by debugger and not started from debugger

• Why my VS cannot detach from the target

when attaching to the target, remember to choose Managed only mode, not mixed mode

• Misc
  • Dump
    • VS-generated minidump has no handle info, so not good for multiple thread deadlock troubleshooting

PowerDbg: a great module to write windbg script

Last week during the trip in Las Vegas, I learned something about PowerDbg, a PowerShell module which can automate windbg debugging. It is really a exciting tool and just what I have been searching for a long time.
During the debugging, we may wish to analyze elements in a big array one by one, but manually doing so is too troublesome, for instance, one of the element is not disposed or one of the element's flag is not set correctly.
In this case, if a script can help us do it automatically, it would be great.
PowerShell is microsoft's new shell replacing the old WSH. It is strong typed and based on .net framework, can access all .net framework.
It can also be integrated with windbg: Windbg works as debugging server, PowerDbg as remoting host, sends debugging commands to windbg,
retrieves output, parses them and continues analysis.
One sample is like:
http://www.leeholmes.com/blog/2009/01/21/scripting-windbg-with-powershell/
PowerDbg can be downloaded from:
http://powerdbg.codeplex.com/
The latest staple build is 5.3, I did not try the 6.0(alpha). I strongly suggest the pscx and powertab modules. The first is an extension for powershell and the latter can provide intelligence in shell.
I will later post some of my sample scripts.

"Server Busy" and AfxOleInit

Recently, we had one problem of showing "Server Busy" dialog during the long time WaitHandle.WaitOne waiting, kind of annoying. After research work, found the reason. Maybe it is by design of Microsoft, to provide opportunities for calls to a com thread which is already blocked by any wait or sleep functions to deal the situation correctly, like retry or cancel. While, this may not be expected as most users do.

The work-around is simple, since AfxOleInit() register a COleMessageFilter internally, which calls CoRegisterMessageFilter. So, we can just register it again and calling EnableBusy and EnableBusyDialog(FALSE).

Another solution is to call OleInitialize or CoInitializeEx with apartment thread model instead of AfxOleInit().

Program disappears when accessing 0 element in a empty vector

For some reason, some codes like below exist in our product:

   1: try

   2: {

   3:   std::vector<int> v;

   4:   int i = v[0];

   5:   int* pTemp = NULL;

   6:   *pTemp = 0;

   7: }

   8: catch(...)

   9: {

  10:   AfxMessageBox(L"haha");

  11: }

  12:  

Under VS2005, the above code can show message, but under 2008, it just disappears silently. Checking system event log can find some exception with code 0xc0000417. But the exception could not be caught, since it is the crt's invalid_parameter_info handler, internally, _invoke_watson is called and it unregister all existing unhandled exception handlers, then it terminates the process. (can be found under crt's code: "invarg.c). So, we have no change to catch this exception.


If wishing to catch such error, we can set:

   1: void myInvalidParameterHandler(const wchar_t* expression,

   2:    const wchar_t* function,

   3:    const wchar_t* file,

   4:    unsigned int line,

   5:    uintptr_t pReserved)

   6: {

   7:    throw 0;

   8: }

   9:  

  10:    _invalid_parameter_handler oldHandler, newHandler;

  11:    newHandler = myInvalidParameterHandler;

  12:    oldHandler = _set_invalid_parameter_handler(newHandler);

  13:  

some reference link can be found as below:

https://connect.microsoft.com/feedback/ViewFeedback.aspx?FeedbackID=101337&SiteID=210

Can reference guarantee non-empty input?

Previously, when discussing the difference between pointer and reference, one may say that the argument passed as reference can guarantee non-empty input. We may just simply take it for granted, but it does not hold for all cases.

For instance,

   1: class TempClass

   2: {

   3: public:

   4:     int member;

   5: };

   6:  

   7: void TestFunction(TempClass& input)

   8: {

   9:     TempClass* pInput = &input;

  10:     if(pInput == NULL)

  11:         throw;

  12:     int value = input.member;

  13: }

  14:  

  15: TempClass* p = NULL;

  16: TestFunction(*p);

  17:  

What happens for the above code? The "input" is a null-reference.

After reading c++ standard, it can be found that dereferencing a null pointer has undefined behavior, so, the above code has broken c++ standard already and is not suggested to use.

"[Note: in particular, a null reference cannot exist in a well-defined program, because the only way to create such a reference would be to bind it to the "object" obtained by dereferencing a null pointer, which causes undefined behavior]"

BoundsCheck not working with mixed codes

Previously, we have tried to use BoundsChecker to test our mixed programs. But it does not support managed code and also cause huge memory usage during running,

Recently, I tried "Application Verifier", a great tool distributed by Microsoft, can be used alone or together with Visual studio team version,
and it seems to be acceptable. The below code can be found immediately:

   1: int* p = new int[10];

   2: p[11]=1;       // this cannot be detected, since the heap block has additional 8-bytes suffix, it seems that AppVerif does not check with this

   3: p[100] = 1;   // this can be detected immediately, since AppVerif generates memory fence after the heap block.

After running with Application Verifier, the memory usage is also acceptable.

Another thing related to this is that AppVerif can only work with native codes. As for the mixed code, msdn does not say clearly.
Two potential results can be:
1: AppVerif can only check native memory, ignore managed objects.
2: AppVerif cannot deal with mixed code correctly, may corrupt program's running by mistake.

After research work, I found the below link: saying that AppVerif does not actually validate the managed objects and codes,
instead it validates the CLR itself. If so, it seems that it can work for checking native memory overwriting.

http://social.msdn.microsoft.com/forums/en-US/vstsappverifier/thread/7f5e85c4-c715-43cf-af6f-e4917cfd7108/

unexpected WM_PAINT message reentry when waiting for COM request under Vista/7

Recently, we had a weird problem: when CLR is waiting for the COM request result, especially a remoting one, the WM_PAINT message can be processed before it returns.

We are waiting the COM request in rendering thread, and assumes that the code should be blocked and does not consider reenter problem. This really cause lots of trouble.

When the problem happened, the callstack trace looks like below:

 

   1: UIFramework.dll!UIFramework.MainForm.viewport_Paint

   2: System.Windows.Forms.dll!System.Windows.Forms.Control.ControlNativeWindow.WndProc(ref System.Windows.Forms.Message m) + 0x31 bytes    

   3: System.Windows.Forms.dll!System.Windows.Forms.NativeWindow.DebuggableCallback(System.IntPtr hWnd, int msg = 15, System.IntPtr wparam, System.IntPtr lparam) + 0x57 bytes    

   4: ...

   5: user32.dll!_DispatchMessageW@4()  + 0xf bytes    

   6: ole32.dll!CCliModalLoop::HandleWakeForMsg()  + 0x48c30 bytes    

   7: ole32.dll!CCliModalLoop::BlockFn()  - 0x5df7 bytes    

   8: ole32.dll!ModalLoop()  + 0x52 bytes    

...
>   

Initially, I thought it may be related to COM's IMessageFilter interface, which prevents COM deadlock problem when client/server calls each other, just like SendMessage can be interrupted by the non-queue messages when being blocked. But after registering the interface, the problem is still there. After checking manual, finding the interface can deal with COM requests (also message based) and windows activation related messages, but not including WM_PAINT.

Then, jumping to ole32.dll!CCliModalLoop::HandleWakeForMsg(), we can find below codes:

   1: 75DA612A  call        CCliModalLoop::MyPeekMessage (75D5D3D3h) 

   2: 75DA612F  test        eax,eax 

   3: 75DA6131  je          CCliModalLoop::HandleWakeForMsg+95h (75DA613Dh) 

   4: 75DA6133  lea         eax,[ebp-44h] 

   5: 75DA6136  push        eax  

   6: 75DA6137  call        dword ptr [__imp__DispatchMessageW@4 (75D417ACh)] 

   7: 75DA613D  or          dword ptr [edi+0Ch],80000h 

   8:  

And below the function we can see "_DisableNewWM_PAINTDispatch@0:" symbol, meaning this feature can be disabled.

Finally, I found this is a new feature after windows vista. When being blocked in any COM request, the thread can still process WM_PAINT messages, and this new feature has brought a lot of troubles for software developers. Fortunately, it can be disabled for specific program via Windows Application Compatibility Toolkit. For instance, for my sample, I can create a specific setting file for my program disabling this feature, then use "sdbinst.exe" to install it, then the program can work as before.