whence mozcrt19.lib ?
Nelson Bolyard
mozcrt19.dll, without building the full browser ?
For years, the NSS developers test their NSS builds with Firefox (or
SeaMonkey) by just building their NSS shared libraries as stand-alone
builds (not part of a browser build), and then substituting their NSS
libraries for the ones that come with a nightly build. This has always
worked very well ... until recently. Now, various misbehaviors are
seen when using standalone NSS DLLs with FF or SM trunk.
I think the problem is that NSS built with the browser is linked to
use mozcrt19.dll instead of the vanilla msvcr80 file used in the
stand-alone builds. So, I'd like to try building NSS stand-alone, but
linking it with the "import library" for mozcrt19 instead of the import
library for msvcr80. Obviously, I get a copy of mozcrt19.dll with the
nightly builds. What I'm missing is the import library for it.
Any ideas?
Benjamin Smedberg
We cannot legally redistribute the import library, because it contains MS
CRT code. I suggest either using your stock NSS libraries and installing the
ms crt globally, or statically linking the CRT.
--BDS
Nelson Bolyard
> Nelson Bolyard wrote:
>> Where can I get a copy of mozcrt19.lib, the "import library" for
>> mozcrt19.dll, without building the full browser ?
> We cannot legally redistribute the import library, because it contains MS
> CRT code.
I hope it's clear that, by "import library", I'm not referring to the .lib
that contains all the code that ends up in the DLL itself. I'm referring
the tiny .lib that contains none of the code in the DLL, but contains only
code that knows how to call (jump) into the right places in the DLL. It's
an output of the linker step that creates the DLL, not an input to it.
Your nightly builds are distributing mozcrt19, which (as I understand it)
is built from the slightly modified sources of MS CRT. How is it that you
can redistribute the CRT code in the DLL, and not also the import library?
Is that specifically disallowed in the source license?
Without that import library, how do users of MSVC 2005 express (the free
version, which does NOT come with source code to the CRT) build FF3 to
run with mozcrt19? Or don't they? Do they build FF3 to never use mozcrt19?
> I suggest either using your stock NSS libraries and installing the
> ms crt globally,
How does that help me? My NSS libraries are already using MS CRT, so
I don't need to take extra steps to make that happen.
I hypothesize that the problem is that the MS CRT and MOZCRT19 have
separate heaps, and that trying to use both in FF3 is a problem.
> or statically linking the CRT.
The MS CRT? How does that help me to use the mozcrt19 in NSS?
Is your strategy to try to eliminate mozcrt19, so that all of FF3 is
using the stock MS CRT?
Benjamin Smedberg
> I hope it's clear that, by "import library", I'm not referring to the .lib
> that contains all the code that ends up in the DLL itself. I'm referring
> the tiny .lib that contains none of the code in the DLL, but contains only
Yes
> Your nightly builds are distributing mozcrt19, which (as I understand it)
> is built from the slightly modified sources of MS CRT. How is it that you
> can redistribute the CRT code in the DLL, and not also the import library?
> Is that specifically disallowed in the source license?
Yes. The license says that you may redistribute the CRT with an application
that provides "significant additional functionality", but you may not
redistribute it by itself, nor may you redistribute other files (such as
import libraries) which are not necessary for your application to run.
> Without that import library, how do users of MSVC 2005 express (the free
> version, which does NOT come with source code to the CRT) build FF3 to
> run with mozcrt19? Or don't they? Do they build FF3 to never use mozcrt19?
They do not use mozcrt19.
Building mozcrt19 is triggered by the --enable-jemalloc build flag, which on
Windows is only avaialble when you have the professional version of MSVC.
>
>> I suggest either using your stock NSS libraries and installing the
>> ms crt globally,
>
> How does that help me? My NSS libraries are already using MS CRT, so
> I don't need to take extra steps to make that happen.
>
> I hypothesize that the problem is that the MS CRT and MOZCRT19 have
> separate heaps, and that trying to use both in FF3 is a problem.
They do have separate heaps. But unless we're passing allocated objects
around, allocating them in one shared library and freeing them in another,
that shouldn't cause any problems. And I would consider that a code bug (in
PSM most likely).
Does Mozilla crash if you use the separate NSS libs with a stock Firefox?
For example, we recommend that extension authors link against the CRT
statically. They have their own heap, but it doesn't matter because memory
that is passed across the XPCOM boundary is always allocated/freed with
NS_Alloc/NS_Free
> The MS CRT? How does that help me to use the mozcrt19 in NSS?
> Is your strategy to try to eliminate mozcrt19, so that all of FF3 is
> using the stock MS CRT?
It doesn't help you to use mozcrt19... it only helps you to avoid the MS CRT
dlls in NSS, which makes deployment easier, especially on Windows Vista
which requires that the CRT be enclosed in an assembly.
--BDS
Nelson Bolyard
Benjamin, Thanks for the above info. Please consider putting it into a
README file in http://mxr.mozilla.org/firefox/source/memory/jemalloc/
You might even just put the above text into that file.
> Does Mozilla crash if you use the separate NSS libs with a stock Firefox?
No. But I believe it's not using the debug CRT, which has the assertions
about such things. AFAIK, VC2005 Express does not have that debug CRT.
> For example, we recommend that extension authors link against the CRT
> statically. They have their own heap, but it doesn't matter because memory
> that is passed across the XPCOM boundary is always allocated/freed with
> NS_Alloc/NS_Free
In theory, all allocation done in NSS is done with PR_Malloc and PR_Free.
Hmm. I wonder if my tests were using my stand-alone NSPR build, too, or
if they were using the NSPR from the FF3 nightly. Maybe using the FF3
nightly NSPR would solve this. I need to repeat the test to check that.
>> The MS CRT? How does that help me to use the mozcrt19 in NSS?
>> Is your strategy to try to eliminate mozcrt19, so that all of FF3 is
>> using the stock MS CRT?
>
> It doesn't help you to use mozcrt19... it only helps you to avoid the MS CRT
> dlls in NSS, which makes deployment easier, especially on Windows Vista
> which requires that the CRT be enclosed in an assembly.
Where can I read about that requirement? Some MSDN page?
--
12345678901234567890123456789012345678901234567890123456789012345678901234567890
00000000011111111112222222222333333333344444444445555555555666666666677777777778
Benjamin Smedberg
> Where can I read about that requirement? Some MSDN page?
>
http://msdn.microsoft.com/en-us/library/ms235624(VS.80).aspx
--BDS
Ted Mielczarek
wrote:
> > It doesn't help you to use mozcrt19... it only helps you to avoid the MS CRT
> > dlls in NSS, which makes deployment easier, especially on Windows Vista
> > which requires that the CRT be enclosed in an assembly.
>
> Where can I read about that requirement? Some MSDN page?
I don't think there's a clear reference anywhere, but this page talks
about it some:
http://msdn.microsoft.com/en-us/library/aa375193(VS.85).aspx
Basically, the Visual C++ CRT since VC 2005 is a "side-by-side
assembly", which means that it's not just a set of DLLs, but it also
ships with a manifest file. Applications linking to the DLLs need to
also embed a manifest in their EXE or DLL file that references the
CRT's manifest. If this isn't done properly, the application won't
work on Windows XP SP2 or newer. When we started building our own CRT
from source in order to enable jemalloc on Windows, we no longer
needed to ship the standard CRT, so we don't embed these manifests and
link to mozcrt19.dll instead. As Benjamin said, people providing
binary components in extensions or plugins need to provide their own
CRT, either by statically linking to it, or by linking to the DLL
version and shipping that along with their component. Either way, if
memory management is being handled properly, we should not be
allocating memory with one allocator and freeing it with another.
Does that clear anything up for you?
-Ted
Nelson Bolyard
> Nelson Bolyard wrote:
>> The MS CRT? How does that help me to use the mozcrt19 in NSS?
>> Is your strategy to try to eliminate mozcrt19, so that all of FF3 is
>> using the stock MS CRT?
>
> It doesn't help you to use mozcrt19... it only helps you to avoid the MS CRT
> dlls in NSS, which makes deployment easier, especially on Windows Vista
> which requires that the CRT be enclosed in an assembly.
NSS is not a single shared library. It is multiple shared libraries.
We definitely do NOT want to link each one with its own static copy of the
MS CRT.
Now, here's the immediate issue. FF 302 is blocked by an NSS bug that can
only be reproduced with a certain third party smart card library that none
of the NSS team members has, or is eligible to receive. This means that
none of the NSS team members can adequately test the proposed fix for that
bug. We have a patch for this bug, but we want the user who reported it to
test the fix before we commit it into FF 302. We want to build a test build
of NSS DLLs which we can send to the user and ask him to test is FF302pre
build with them, but we can't do that for the reasons given in this thread.
The bottom line is that the NSS team needs to be able to build NSS DLLs that
are fully interoperable with FF 302 as built and distributed by MoCo, and
those NSS DLLs cannot each have a statically linked copy of any MS CRT.
Nelson Bolyard
> Basically, the Visual C++ CRT since VC 2005 is a "side-by-side
> assembly", which means that it's not just a set of DLLs, but it also
> ships with a manifest file. Applications linking to the DLLs need to
> also embed a manifest in their EXE or DLL file that references the
> CRT's manifest.
Yeah, we've been building NSS that way since we started using VC 2005,
long ago. Our link step, that builds the dll, also produces a manifest,
and then we run mt.exe to combine the manifest into the dll itself, and
then discard the manifest.
We install the SP1 or SP2 (I forget) version of the CRT for VC 2005,
which fixes some nasty bugs in the earlier versions.
> If this isn't done properly, the application won't
> work on Windows XP SP2 or newer.
Our NSS shared libs run on WinXP SP2 ... on systems where that same
SP1/2 version of the CRT has been installed in the WinSxS directory.
Now I'm trying to figure out how to get that to work with msvcr80
installed in the same directory as the NSS DLLs, and not in the WinSxS
directory. It doesn't seem that we explicitly tell any build step
where to find the WinSxS CRT. It just does. We don't provide any path
to the import library for that CRT.
Now I'm wondering how we tell the linker to use the dll that's in the
directory where the exe/dll lives.
Is there a tool that will tell us the path name for the CRT that's in a
manifest or in the DLL (after the manifest is embedded into it by mt) ?
dumpbin /dependents doesn't seem to provide path info or version info.
How do we get the manifest to point to the MSVCR80 in the .exe directory
instead of in the WinSxS directory.
> When we started building our own CRT
> from source in order to enable jemalloc on Windows, we no longer
> needed to ship the standard CRT, so we don't embed these manifests and
> link to mozcrt19.dll instead.
Or it's import library, yes?
How do you get it to stop linking with the CRT?
> As Benjamin said, people providing
> binary components in extensions or plugins need to provide their own
> CRT, either by statically linking to it, or by linking to the DLL
> version and shipping that along with their component.
Got it. I'm trying to link with the DLL version that's not in the
WinSxS directory. But it always takes the WinSxS version. :(
I'm thinking I need an import library for the CRT. I don't know where
to find that. The WinSxS directory contains only .dlls, not the import
libraries.
Ted Mielczarek
hacks to avoid linking with the default CRT (although bug 429745 aims
to fix most of those). When you embed the manifest into the NSS DLLs,
what resource ID do you use? If you're embedding them at ID 2, then
this should just work. When your DLL is loaded by Windows, it will
read the manifest, and look for the dependent assemblies (the CRT) and
load them. You don't have to do anything special to load them from the
same directory as your application, you just put the 3 DLLs +
the .manifest file there, and the DLL loader checks that directory
first. We were shipping Firefox betas this way before we switched to
jemalloc + the custom compiled CRT.
This is a complicated and not-well-documented subject, so I'm not
surprised that you're hitting some pain here. I struggled with this
for quite a while before hitting a workable solution for Firefox.
-Ted
Ted Mielczarek
wrote:
> Now I'm trying to figure out how to get that to work with msvcr80
> installed in the same directory as the NSS DLLs, and not in the WinSxS
> directory. It doesn't seem that we explicitly tell any build step
> where to find the WinSxS CRT. It just does. We don't provide any path
> to the import library for that CRT.
>
> Now I'm wondering how we tell the linker to use the dll that's in the
> directory where the exe/dll lives.
>
> Is there a tool that will tell us the path name for the CRT that's in a
> manifest or in the DLL (after the manifest is embedded into it by mt) ?
> dumpbin /dependents doesn't seem to provide path info or version info.
>
> How do we get the manifest to point to the MSVCR80 in the .exe directory
> instead of in the WinSxS directory.
Oh, one thing I realized right after posting. The DLL loader, IIRC,
can choose to use the copy of the assembly from the Windows\SxS
directory if it's newer. This is Microsoft's way of trying to work
around "DLL Hell" by letting you install a private copy, but allowing
the system copy to be used if it's newer. If you read the link I
posted earlier, it talks about "isolated applications." Microsoft
provides some facilities for you to state that your application should
only use a particular version of an assembly. I don't know more about
that as I've never actually used it, but you can read the docs.
-Ted
Nelson Bolyard
> Ok, your reply has cleared some things up for me. We do some awful
> hacks to avoid linking with the default CRT (although bug 429745 aims
> to fix most of those).
Our builds "just work" when built and run on a system where the CRT has
been installed in the WinSxS directory. Now, I'm trying to figure out
how to get them to work on systems where that file is not installed in
the WinSxS directory, but rather is in the directory with the FF .exe file.
I tried just copying the msvcr80.dll file from the WinSxS dir on one
machine to the FF .exe directory on another machine (on which it is not
in the WinSxS directory), and the result didn't work. FF wouldn't start.
I'm trying to figure out how to get that to work.
To try to make it more clear, I built NSS on my machine, where the CRT is
in the WinSxS directory. Then I installed FF302, and then copied my
nss3.dll over the one from FF302. It worked very well. So then I tried
copying my nss3.dll and the msvcr80.dll (from the SxS dir) to the FF302
directory on another machine where no compiler or alternate CRT was
installed, and repeated the test there. No joy. I thought it would find
the expected version of the CRT in the .exe directory and be happy, but
alas.
So, I really don't know what the problem is. Wrong path? Wrong version?
I wish I could look inside the embedded manifest and see what it says.
> When you embed the manifest into the NSS DLLs,
> what resource ID do you use?
The mt.exe command looks like this:
mt.exe -NOLOGO -MANIFEST WIN954.0_DBG.OBJ/nss3.dll.manifest \
-OUTPUTRESOURCE:WIN954.0_DBG.OBJ/nss3.dll\;2
So, I think the answer to your question is "2", but I'm not sure.
> If you're embedding them at ID 2, then this should just work.
That's what I thought, too.
> When your DLL is loaded by Windows, it will read the manifest, and look
> for the dependent assemblies (the CRT) and load them. You don't have to
> do anything special to load them from the same directory as your
> application, you just put the 3 DLLs + the .manifest file there, and the
> DLL loader checks that directory first.
I wonder what information about the particular CRT DLL is recorded that
makes the loader accept only it, and reject others (apparently).
I thought it might be path name. Maybe it's version number.
None of my hypotheses has proved correct so far.
If I can't do any better, I'll have people install the VC2005-SP1 CRT
on their systems to test my patch DLL. But I'd really rather not.
Ted Mielczarek
wrote:
> I wonder what information about the particular CRT DLL is recorded that
> makes the loader accept only it, and reject others (apparently).
> I thought it might be path name. Maybe it's version number.
> None of my hypotheses has proved correct so far.
You can look at the manifest by either opening the DLL in Visual
Studio's resource editor, or using the mt.exe commandline tool like
so:
mt.exe -inputresource:foo.dll;#2 -out:foo.manifest
(you may need to escape things for your shell)
If you'd like to send me a copy of the NSS DLLs that you've built, I'd
be happy to test them on a clean VM and see if I can help you figure
this out. You can mail them to me at this email address, or send me a
HTTP link, whatever works for you.
-Ted