Trigraph in C literal string

Last week, one of my colleague asked me one question, the c++ codes like:

char *p = "??--AB";

will be translated to “~-AB” automatically by the compiler.

After checking the standard reference document, found

Really an interesting point for old C codes

The wiki says:

History

The basic character set of the C programming language is a subset of the ASCII character set that includes nine characters which lie outside the ISO 646 invariant character set. This can pose a problem for writing source code when the keyboard being used does not support any of these nine characters. The ANSI C committee invented trigraphs as a way of entering source code using keyboards that support any version of the ISO 646 character set.

[edit]Implementations

Trigraphs are not commonly encountered outside compiler test suites.^[1] Some compilers support an option to turn recognition of trigraphs off, or disable trigraphs by default and require an option to turn them on. Some can issue warnings when they encounter trigraphs in source files. Borland supplied a separate program, the trigraph preprocessor, to be used only when trigraph processing is desired (the rationale was to maximise speed of compilation).

unittest problems for c++/cli via nunit in VS2008

Recently, my colleagues have some problems of running/debugging c++/cli nunit unittest in VS2008 via TestDriven and Resharper.

For TestDriven, dependent assemblies could not be loaded. For Resharper, VS2010 is fine, under VS2008, running testcases is ok, debugging them gets the error of 89710016 and the hosting process is not even started. After investigation, now find reasons and solutions.

Solutions:
1: Install testdriven.net, whose latest stable version is 3.0, then copy all dlls under
C:\Program Files (x86)\TestDriven.NET 3\NUnit\2.5\lib
to the upper folder. It seems to be testdriven.net’s privatePath setting problem.

2: Install resharper, open regeditor, goes to
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\9.0\AD7Metrics\Engine\{449EC4CC-30D2-4032-9256-EE18EB41B62B}
add a string entry:
CLRVersionForDebugging
value:
v2.0.50727

Now, the debugger can start, while the breakpoint is not active. Then, goes to
C:\Program Files (x86)\JetBrains\ReSharper\v6.1\Bin
open "JetBrains.ReSharper.TaskRunner.CLR4.exe.config"
change the <startup> section to below:

  <startup useLegacyV2RuntimeActivationPolicy="true">
    <requiredRuntime version="v2.0.50727" />
  </startup>

note: please make a copy of the .config file first.

Then, debugger and breakpoint work as expected.

It seems that VS2008 has some registry checking for the debugging extensions, not compatible with .net 4.0, and the external hosting process should also run with CLR 2.0 to successfully communicate with VS to support breakpoints.

don’t call GdiShutdown when unloading dlls, an interesting matlab hang problem when closing.

I just fixed an interesting matlab hang problem for my colleague, who tested his image components with matlab, and always got hang problem when closing matlab, really a headache.

After analysing the dump, finding that matlab’s main thread is blocked at

   1: 00c2e1d8 4ecbcd7d 00000a38 ffffffff 00c2e2f0 kernel32!WaitForSingleObject+0x12

   2: 00c2e1f8 4ecbcd2c 4edd72bc 4ec7683a 00c2e4a8 GdiPlus!BackgroundThreadShutdown+0x47

   3: 00c2e200 4ec7683a 00c2e4a8 00c2e2f0 0dfd00de GdiPlus!InternalGdiplusShutdown+0x12

   4: 00c2e20c 0dfd00de 0c1355b6 00c2e3c4 00c2e4a8 GdiPlus!GdiplusShutdown+0x2c

   5: ...

   6: 00c2e4b4 7c91d0f4 0dfbcee2 0df50000 00000000 ntdll!LdrpCallInitRoutine+0x14

   7: 00c2e5ac 7c80ac97 0cdb0000 00000000 0c784a02 ntdll!LdrUnloadDll+0x41c

   8: 00c2e5c0 7bc2aa71 0cdb0000 00000000 0c78027f kernel32!FreeLibrary+0x3f

and the handle 0xa38 is a thread object, which is the background working thread of gdiplus.

   1: Handle 00000a38

   2:   Type             Thread

   3:   Attributes       0

   4:   GrantedAccess    0x1f03ff:

   5:          Delete,ReadControl,WriteDac,WriteOwner,Synch

   6:          Terminate,Suspend,Alert,GetContext,SetContext,SetInfo,QueryInfo,SetToken,Impersonate,DirectImpersonate

   7:   HandleCount      6

   8:   PointerCount     10

   9:   Name             <;none>

  10:   Object specific information

  11:     Thread Id   1f84.1e78

  12:     Priority    10

  13:     Base Priority 0

then further check the thread 1e78,

   1: 0e92fe7c 7c90df5a 7c919b23 000015c8 00000000 ntdll!KiFastSystemCallRet

   2: 0e92fe80 7c919b23 000015c8 00000000 00000000 ntdll!ZwWaitForSingleObject+0xc

   3: 0e92ff08 7c901046 0197e174 7c9138b0 7c97e174 ntdll!RtlpWaitForCriticalSection+0x132

   4: 0e92ff10 7c9138b0 7c97e174 00000000 7ff92000 ntdll!RtlEnterCriticalSection+0x46

   5: 0e92ff7c 7c80c136 00000000 00000011 00000000 ntdll!LdrShutdownThread+0x22

   6: 0e92ffb4 7c80b72e 00000001 00000000 00000011 kernel32!ExitThread+0x3e

   7: 0e92ffec 00000000 4ec67456 00000000 00000000 kernel32!BaseThreadStart+0x3c

The thread is trying to acquire the loader CS when calling ExitThread, which belongs to main thread since it is in the process of unloading dll. And the below shows the CS info, and the owner is the main thread.

   1: -----------------------------------------

   2: Critical section   = 0x7c97e174 (ntdll!LdrpLoaderLock+0x0)

   3: DebugInfo          = 0x7c97e1a0

   4: LOCKED

   5: LockCount          = 0x7

   6: OwningThread       = 0x00001ff4

   7: RecursionCount     = 0x1

   8: LockSemaphore      = 0x15C8

   9: SpinCount          = 0x00000000

Finally, the simple fix is to expose a function to call GdiShutDown in my colleague’s component, and matlab calls it explicitly before closing.

Make full use of parallel build in visual studio

I just got to know recently that visual studio supports parallel build of both multiple projects in one solution and multiple source files in each project via /MP(n) option. The former can be set in Tool-Option, and the latter can be found in project setting of VS2010, as show

http://blogs.msdn.com/b/visualstudio/archive/2010/03/08/tuning-c-build-parallelism-in-vs2010.aspx

As for VS2008, I have to manually specify it in the Project-Property-Compiling-Advanced page, “/MP8”.

Due to historic reason, our major product, consists some big projects which are bottlenecks, and fully rebuild is time consuming, generally you can hang out and enjoy your coffee with enough time. Since I joined the team, it is really a headache experience whenever I thought of rebuilding it. With the new option, the build process has been much faster.

While this option may conflict with existing codes, for our cases, like improper usage of precompiled header files, #import statements, and etc.

A bug of JIT

 

Long time no update my blog

Recently, one of our 32bit product experiences a weird crashing problem, the callstack shows crash happens during JITting a c# function. While if starting from debugger or disable JIT optimization from the config file, it works well.

After experiments, we isolated the specific statements causing the crashing problem:

   1: fixed(Point* pPoint = ...)

   2: {

   3:     ...

   4:     pPoint[index] = GetPoint();

   5:     ...

   6: }

The GetPoint is another c# function, returning a Point instance, which is a value type.

And just JITting the above codes with optimization enabled will cause troubles. And one work-around is to rewrite with a temp variable first, then assign with the temp var.

   1: Point thePoint = GetPoint();

   2: pPoints[index] = thePoint;

Though seems to be strange, it works well.

Sorry for not providing the callstack and debugging logs, since this happened one month ago, and I could not find detailed logs.

Some other things related to this interesting bug:

First, Obfuscator will remove the [MethodImplOptions.NoOptimization] attribute after obfuscating our assemblies, maybe there are some other settings, while finding the root reason is always the best solution than work-arounds

Second, the above exception thrown from JITting will be translated to first-chance ManagedException if attached with Visual Studio 2010, while show no CLR exception from windbg, maybe the CLR exception notification which is translated from JIT will not be sent to windbg.

VS2005 can also access .Net 3.5 libraries

 

Last week, one of SDKs built with VS2005 threw an exception from client’s hosting apps on their machines. Finally we found the reason – we call GC.WaitForFullGCComplete in our app, though the function was introduced in .Net 3.5, while since 2.0/3.5 share the same mscorlib.dll in the Framework 2.0’s dir, which is also updated by .Net 3.5 after VS2008 is installed. Then VS2005 can use the new functions without problem. Developers did not notice it. Then, if the client’s apps do not install .Net 3.5, the problem happens. So, we need to tell clients to install the 3.5 framework with latest service pack in the future.

This is kind of interesting, and the reason why we distributed the SDK built with VS2005 is that some of our clients have not upgraded to VS2008, and can not open our SDK’s samples if we only distribute VS2008 solutions and samples.

Why debugging in my visual studio 2008 is too slow?

 

Recently, I just realized that my visual studio 2008 is extremely slow during debugging one of our mixed-code products, a single step over will take a few seconds! What happens? The same product works well on my colleagues’ machine.

And today, one of my colleagues told me debugging on his machine also got slow down recently for another c# project. I checked his debugging setting, disable some options, and his problem disappeared.

Then, I tried them on my machine, a little improved, but still quite slow. After profiling the visual studio during debugging, finally found it tries to search some missing source code for each debugging command. Since I copied symbols for some 3rd party components sometime ago for troubleshooting one of the component bug, and leave them there. Then, when visual studio starts debugging, it will try to locate those missing source files for each debugging command, Why? Once fails, it should stop searching during the session until I manually start the search.

Then, I made a copy for those symbols of 3rd components until later they are needed, now, my studio works quite well.