On Mon, Dec 12, 2011 at 6:04 PM, Robert O'Callahan <
rob...@ocallahan.org>wrote:
> On Tue, Dec 13, 2011 at 8:58 AM, Ehsan Akhgari <
ehsan....@gmail.com>wrote:
>
>> I have an idea which might enable us to use VS2010 to build binaries that
>> will run with Win2k, XP and XP SP1. We were going to switch to VS2010 for
>> Gecko 12 anyways, so if we can get this to work, we can switch to VS2010
>> today and we wouldn't need to rip out anything either.
>>
>> I will have useful results in a few hours.
>>
>
> What's the idea? I gather from #developers it didn't work, but I'd still
> like to know :-).
>
This might sound a bit crazy, but please bear with me.
If we had a way to provide msvcr100.dll with our own
EncodePointer/DecodePointer implementation (those functions are easy to
implement), everything would be fine on those platforms. I originally had
a few ideas, but none of them worked because of different reasons (we don't
link to crt statically, we can't ship modified versions of the binary,
etc). But I now have a solution which I think will work (I've just got the
hard part working):
I looked at the CRT source code, and it calls EncodePointer very early
during the startup (in its DllMain function, called _CRT_INIT). So firstly
we need a way to run our code before _CRT_INIT. I have got this part
working by generating a DLL which does not link against the CRT and only
imports functions from kernel32.dll (which itself does not link against the
CRT), and adding my code to the DllMain function inside that DLL. For this
to work correctly, the library for that DLL needs to come before msvcrt.lib
in the import table entries, something which can be achieved by passing
/nodefaultlib to the linker reordering the libraries on the command line
accordingly.
Once we have code which runs before the CRT initialization, we can look to
see if kernel32.dll includes the two functions we're interested in. If it
does not, we can parse the PE header of kernel32.dll, get to its
EXPORT_IMAGE_DIRECTORY table, copy the table while interleaving our two
entries to point to our own versions of those functions, and overwrite the
kernel32.dll PE header to point to our new EXPORT_IMAGE_DIRECTORY.
This way, when the loader attempts to load msvcr100.dll, it can
successfully find entries for EncodePointer/DecodePointer. This is sort of
similar to how we currently intercept LdrLoadDll on Windows for DLL
blacklisting, so while it's sort of hackish, it relies on documented stuff
from Windows. Therefore, I suspect that it will work just fine.
I'm currently working on the second half of the implementation.