Friday, December 9, 2011

EMET causes troubles for visual studio’s debugging

Since last month, one of my projects fails to be started by visual studio when trying to debug it, while still can run standalone and also can be attached after having started, and can also be started from windbg.

Sometimes showing the message: STATUS_STACK_BUFFER_OVERRUN, sometimes studio hangs, sometimes debugger keeps interrupted at ntdll.dll!_LdrpSnapThunk@36 when trying to load cryptsp.dll. Due to the tight dev schedule, I chose to move on with other projects.

Today, I tried to fix it. First, since STATUS_STACK_BUFFER_OVERRUN is VC’s runtime security checking, after disable this compiling option, still not work. Then, check appverif and gflags setting options, clearing them, still not work. Then, tried to debug the _LdrpSnapThunk codes, no infinite loop and seems just unexpected interrupted, anyway, could not figure out why it failsSad smile

Then, I tried to rename my executable, it works! So, it should not be my debugger problem, must be kind of specific setting like Application Compatibility Toolkits. I reviewed such kind of settings on my machine, and found last month, I installed EMET(Enhanced Mitigation Experience Toolkit) and played with my project without restoring it, then trouble comesSmile

I also found other guys got similar problems, causing lot of troubles to developers.

http://forums.silverlight.net/t/221268.aspx

Sunday, December 12, 2010

memory leak detection notes (continuously updating)

Memory leak problem is very common, below is some my previous working notes:

  • BoundsCheck

BoundsCheck does not work well with mixed codes. Since my latest projects are almost all mixed, so, seldom used in my daily work.

  • Visual studio
    • Enable memory leak reporting

There are two ways to enable memory leak check in VC, one is to define DEBUG_NEW macro, another is as below:

   1: #define _CRTDBG_MAP_ALLOC 
   2: #include<stdlib.h> 
   3: #include<crtdbg.h>

Then calling _CrtSetDbgFlag ( _CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF ); when initializing your app.


With the above changes, visual studio can dump memory block leak when quitting or between two snapshot via Windows API. For instance:


c:\program files(x86)\microsoft visual studio\vc98\include\crtdbg.h(552) : {43} normal
block at 0x00441C20, 40 bytes long.
Data: < C > 08 02 43 00 16 00 00 00 00 00 00 00 00 00 00 00



    • Output mode

The above output is one sample of block leak, in which, {43} is the heap block allocation id, which can be used to set breakpoints. “normal” is the classification of heap blocks, some other types include: client/crt/free (free will no appear here). Then double click the output item, the allocation location can be watched.

By default, visual studio generate the report to the output window, which can also be changed by CrtSetReportMode calling, for instance: to std-err console or some files (needing to call _CrtSetReportFile first)



    • Conditional breakpoint by allocation #

Input {,,msvcrtd.dll}_crtBreakAlloc in your watch window, by default value would be –1, which is the condition when allocation would be broken, for instance, 43, and when the allocation # equals to 43, DbgBreak is triggered. The value can be changed manually in visual studio.

You can also call _CrtSetBreakAlloc(43) to do the same thing.



    • create memory snapshot and compare difference


   1: _CrtMemState s1, s2, s3;
   2: _CrtMemCheckpoint( &s1 );
   3: _CrtMemCheckpoint( &s2 );
   4: if ( _CrtMemDifference( &s3, &s1, &s2) )
   5:     _CrtMemDumpStatistics( &s3 );

The above codes show how to do this. In the middle, CrtMemDumpStatistics can be used to generate statistics info for each _CrtMemState snapshot.


  • AppVerif and Windbg

We can also get stacktrace for each leaking block above by enable it in gflags.exe. Then, attaching your target with WinDbg, check the stacktrace info for each heap by typing:

!heap –p –a <blockAddr>

Then, the block info like type, stacktrace will be displayed.


  • LeakDiag vs UMPH

Both these tools can help us generate memory leak reports. UMPH can work only with standard heap allocation, while LeakDiag can work with all 6 types of allocations:

standard windows heap allocation

virtual memory allocator

MPHeap allocator

COM Allocator

COM Internal Allocator

C Runtime Allocator

With LeakDiag, you can simply create two snapshots, then compare them as you wish, similar to the above steps in visual studio. Then, a final xml file can be generated.

one sample can be found:

http://mcfunley.com/277/using-leakdiag-to-debug-unmanaged-memory-leaks

http://thetweaker.wordpress.com/2009/04/09/native-memory-leaks-part-1-leakdiag/



    • Visualizing the report

For LeakDiag, there are some tools to view the report, like: “Leak XML Logfile Analyzer”, “LDGrapher”,

c++/CLR/SEH/MFC/VEH exception notes (continuously updating)

  • SEH/VEH

A new interesting topic is VEH (vector exception handler), which is introduced since windows XP. The difference is as below:

    • Not like SEH, which is saved in the TEB, is popped as stack unwinds and only valid in the current thread; VEH is saved in the global heap, which is effective for the whole process;
    • All VEH will be executed before SEH
    • VEH only exists in the user-mode codes, SEH can be both user/kernel modes
    • SEH is registered/revoked by the codes generated by compiler, VEH is registered/revoked by system API:

AddVectoredExceptionhandler

RemoveVectoredExceptionHandler

    • The signature for VEH is:

LONG CALLBACK VectorHandler(PEXCEPTION_POINTERS ExceptionInfo)

  • C++/CLR/SEH exception
    • relationship

SEH exception means structured exception handler, CLR exception and C++ exception are two special SEH one, each has its own SEH code. C++ exception code is: 0xe06d7363 and CLR is 0xe0434f4d. C++ exception keywords are try/catch, SEH is __try/__except, MFC is the TRY/CATCH macros, built on the top of SEH keywords. Microsoft strongly suggests converting usage of MFC exceptions to C++ ones, and also provides some samples about how to convert correctly.

http://msdn.microsoft.com/en-us/library/19z28s5c.aspx

    • 64-bit windows no longer saves exception handler on the stack

In 64-bit windows, the exception handler is no longer saved on the stack, instead an exception handler table is defined to hold them.

    • translate SEH to c++ exception

SEH, also can be called as c exception, not like c++ one, only deal with integer, and c++ exception is based on the types. We can also use _set_se_translator to enable the translation from SEH exception to c++ one. For instance:

   1: // from msdn http://msdn.microsoft.com/en-us/library/5z4bw5h5%28v=VS.90%29.aspx
   2: // crt_settrans.cpp
   3: // compile with: /EHa
   4: #include <stdio.h>
   5: #include <windows.h>
   6: #include <eh.h>
   7:  
   8: void SEFunc();
   9: void trans_func( unsigned int, EXCEPTION_POINTERS* );
  10: class SE_Exception
  11: {
  12: private:
  13:     unsigned int nSE;
  14: public:
  15:     SE_Exception() {}
  16:     SE_Exception( unsigned int n ) : nSE( n ) {}
  17:     ~SE_Exception() {}
  18:     unsigned int getSeNumber() { return nSE; }
  19: };
  20: int main( void )
  21: {
  22:     try
  23:     {
  24:         _set_se_translator( trans_func );
  25:         SEFunc();
  26:     }
  27:     catch( SE_Exception e )
  28:     {
  29:         printf( "Caught a __try exception with SE_Exception.\n" );
  30:     }
  31: }
  32:  
  33: void SEFunc()
  34: {
  35:     __try
  36:     {
  37:         int x, y=0;
  38:         x = 5 / y;
  39:     }
  40:     __finally
  41:     {
  42:         printf( "In finally\n" );
  43:     }
  44: }
  45: void trans_func( unsigned int u, EXCEPTION_POINTERS* pExp )
  46: {
  47:     printf( "In trans_func.\n" );
  48:     throw SE_Exception();
  49: }


  • Async and Sync exception

We often mention async and sync exception models, since for the async case, there is no explicit statement throwing the exception, for the sync, we can always find kernel32!RaiseException on the callstack. The basic SEH is the async model, for instance, processor thrown when finding access violation.


Visual studio can specify two exception models, /Eha and /Ehsc, for async and sync mode respectively. CLR always assumes /Eha, native C++ program can use either of them. In the mixed programming, when exception thrown from native, caught in managed, different exception models can cause trouble (refer to my another post). When specifying async mode, c++ program can also deal with SEH ones, not necessary use __try statements.



  • MFC exception

As for MFC exception, it does not belong to c++ exception, since when Microsoft introduces exception handling to MFC, the c++ exception standard may not be mature. In your new application, C++ exception is suggested, but the old MFC one is still available. But additional attention should be paid when mixing them in your application. Refer to msdn: http://msdn.microsoft.com/en-us/library/sas5wzs9%28v=vs.80%29.aspx. Since MFC macros automatically deleted caught exceptions when they go out of blocks, while c++ does not. So, don’t mix them in one block.


Furthermore, MFC exception is still a special SEH exception, whose code is: 0XE04D5343 and the ASCII is “.msc”, kind of microsoft standard.



  • DebugOutputString and exception

Another interesting thing is that DebugOutputString API is also implemented based on exception, so, in windbg we can also set breakpoints or execute commands based on the target’s debug output. For instance:


sxe out:Open?Database*: stop when finding matched trace output like: “Open Database 1”


.ocommand “MyWindbgCmd:” will execute command “!mk;g;” when finding “MyWindbgCmd: !mk;g;”



  • exception handler list in TEB

In the thread block associated with each thread, there is a pointer to stack of the exception handlers in the current thread. The top is the inner-most one.


The exception handler chain can be watched in Windbg via “!exchain”



  • some good article links:

Differences in Exception Handling Behavior Under /CLR


http://msdn.microsoft.com/en-us/library/2ww6y7y2.aspx


Exceptions: Converting from MFC Exception Macros


http://msdn.microsoft.com/en-us/library/19z28s5c.aspx

Friday, December 10, 2010

Assembly loading strategy misc notes (continuously updated)

 

  • Load from GAC
  • Load from private path
   1: <runtime>
   2:   <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
   3:     <probing privatePath="myPrivatePath"/>
   4:   </assemblyBinding>
   5: </runtime>


  • Change machine.config to load from anywhere globally on your machine
  • Overwrite private path setting


   1: <runtime>
   2:   <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
   3:     <probing privatePath="myPrivate"/>
   4:     <dependentAssembly>
   5:       <assemblyIdentity name="MyAssemblyPath"
   6:                         publicKeyToken="abcd"
   7:                         culture="neutral" />
   8:       <codebase href = "file://localhost/D:/Temp/DLL/MyAssembly.dll">
   9:       <!--Redirection and codeBase policy for myAssembly.-->
  10:     </dependentAssembly>
  11:   </assemblyBinding>

CLR Exception Misc Notes(continuously updated)

 

  • Different unhandled exception for WinForm and Console (from debugging .net app 2.0)

for winform, if unhandled exception thrown from pool thread/background thread/ finalize thread, AppDomain.UnhandledException event will be called.

For Console, main thread still runs, just as if you have handled them

  • Unhandled exception dialogs shows while your app still running

so, you app’s state may be changed, you cannot get the image when the exception is thrown, until you set external debugger to auto attach when it happens.

 

 

Rebase your dlls to get more compact address space

 

Generally, each dll loaded has a preferable starting address, which is determined by the compiler, by default, assigned by name. During the loading, if conflict occurs, loader will relocate them. So, if too many conflicts happens, it may cause starting performance problem.

 

In one of our product, we are using a bunch of 3rd binary packages from one company, the names follow the pattern “FO***”, for most of client machines, they seem to be the only libraries to be loaded into address space starting with that name. After watching the run-time address space of our distributed program, it seems that the compiler is not allocating compact enough address policy, there are several mega free spaces between two adjacent libraries. Since our product is memory extensive one, I manually rebased those dlls and it seems that dozens of 10M memory can be grouped into one 150M memory finally. So, some operations which caused out-of-memory exceptions now can work well.

The basic step is rather straightforward.

  • Dump the run-time address space for you program first by using vmmap.exe from sysinternal or using !address from windbg, the first is much more friendly.
  • reassign the proper base address for dlls
  • Use rebase.exe from visual studio 2008 command, the syntax is:

rebase.exe –b newAddress dllName.dll

 

Just found another interesting method:

rebase.exe /b Ginger.dll GooseBerries.dll

can make those two dlls auto rebased back-to-back.

 

WinDbg debugging scenario (2) find clr gchandle leak

 

!gchandleleak can search heap to get referenced handles, may have false positive. But if the number keeps increasing after each specific operation, then it may cause handle leaks.

!do poi(handleValue) to see what is contained by the handle.

 

For most cases, PowerDbg can be used to automatically check the leakages.