Windows installer bug when same custom action DLL used by 2 merge modules in same msi

[dcaldwell]

Hi,

Since VisualStudio .NET 2003 makes a custom action's uninstall conditional upon the uninstall of the custom action DLL itself, it appears that I must install a copy of the DLL in 2 different locations if I want to use the same custom action in 2 different merge modules in the same install. However when I do this, both copies of the DLL are installed, but only one of the DLLs is accompanied by a InstallState file. When the custom action's Uninstall method is called for the merge module with the missing .InstallState, savedState is null.

The custom action DLL is implemented in C#. Let's call it CA.dll.
Merge module A installs CA.dll in "module retargetable folder\A"
Merge module B installs CA.dll in "module retargetable folder\B"
An InstallShield developer 9 ism has feature A that uses merge module A and feature B that uses merge module B. Their retargetable folders are both mapped to the same directory D.
In the msi file, each instance of CA.dll has its own component id, and there are 2 sets of distinct custom actions scheduled.
Upon install, "program files\product\D\A\CA.dll" and "program files\product\D\B\CA.dll" are both installed properly, and CA.Install is properly called for both.

However, although the installation created a file "program files\product\D\B\CA.dll.InstallState", the file "program files\product\D\A\CA.dll.InstallState" was *not* created.
Then during uninstall, when CA.Uninstall is called for merge module A, savedState is null.

If I do a custom install and just install feature A, then "program files\product\D\A\CA.dll.InstallState" is created and uninstall works. Ditto for feature B alone. But when I install both features at the same time, the problem occurs.

A wild guess is that the installer is confusing the 2 CA.dll components in some internal data structure, so when it comes time to write the installstate files, it only knows about the last one.

Am I missing something, or is this a windows installer bug?

Thanks,
David

[Rich [Microsoft Windows Installer MVP]]

[Please do not mail me a copy of your followup]

=?Utf-8?B?ZGNhbGR3ZWxs?= <anon...@discussions.microsoft.com> spake the secret code
<226D8E22-BF8C-454A...@microsoft.com> thusly:

>Since VisualStudio .NET 2003 makes a custom action's uninstall
>conditional upon the uninstall of the custom action DLL itself, it
>appears that I must install a copy of the DLL in 2 different locations
>if I want to use the same custom action in 2 different merge modules in
>the same install.

Only if you install the DLL as part of your product. If you create
the custom action by adding a file in the "browse to DLL" dialog, then
VS.NET will add the DLL to the binary table and omit the component
portion of the condition.

>The custom action DLL is implemented in C#.

Windows Installer doesn't natively support .NET assemblies as custom
actions. VS.NET inserts a thunk to call your CA and invoke any
Installer classes in the assembly.

Windows Installer doesn't care about "InstallState" files. (I don't
even know what those are, frankly.)
--
"The Direct3D Graphics Pipeline"-- code samples, sample chapter, FAQ:
<http://www.xmission.com/~legalize/book/>
Pilgrimage: Utah's annual demoparty
<http://pilgrimage.scene.org>

[Phil Wilson]

I seem to recall an issue with custom action DLLs with the same name, the
problem being that if the CA DLL is loaded into an install process then another
DLL with the same name won't be loaded again. I think this might still be an MSI
issue, and it might be the underlying cause of your problem.
What are you doing in your custom action DLL with VS Installer classes? Pretty
much anything you want to do can be done with native MSI functionality, and the
architectural issue is that VS custom actions add that InstallUtilLib.dll layer
between MSI and your code. AFAIK, it's only VS developers who believe that you
need installer classes to install Services, for example. :=)
--
Phil Wilson [MVP Windows Installer]
----
"dcaldwell" <anon...@discussions.microsoft.com> wrote in message
news:C21B0095-A77E-43A2...@microsoft.com...
> Hi Rich,
>
> Sorry to be dense, but I can't figure out how to do what you suggest in
VS.NET. The only way I know to add a custom action is in the custom action
editor for the merge module. In that view, when I right-click Install, I get a
"Select item in project" dialog. Not only does the name of the dialog imply the
DLL must be on the target machine, but the directory tree in the combo box at
the top of the dialog starts at "File system on target machine." Furthermore,
if I click one of the buttons to the right (add file, add output or add
assembly), I can browse to a DLL anyplace on my dev machine, but when I select
it and press ok, the DLL is added to the file system of the target machine.
>
> So I'm totally missing how I can create a custom action with VS.NET without
installing the DLL as part of my product.
>
> Can you please give me or point me at more detailed instructions on how to do
this?
>
> Thanks,
> David

[Rich [Microsoft Windows Installer MVP]]

[Please do not mail me a copy of your followup]

=?Utf-8?B?ZGNhbGR3ZWxs?= <anon...@discussions.microsoft.com> spake the secret code

<C21B0095-A77E-43A2...@microsoft.com> thusly:

>Can you please give me or point me at more detailed instructions on how
>to do this?

Sure. Here's a walkthrough.

1. New Win32 DLL Project to contain your custom action code.
2. Add the following code to the DLL project:

#include <msi.h>
#include <tchar.h>

extern "C" __declspec(dllexport)
UINT __stdcall DumpAction(MSIHANDLE)
{
::OutputDebugString(_T("Inside DumpAction.\n"));
return ERROR_SUCCESS;
}

3. Add a new setup project to the solution
4. View the custom actions for the new setup project
5. Add a new install custom action
6. Browse to Application Folder
7. Click Add File...
8. Select the DLL from the custom action project
9. Click OK, OK
10. Change the EntryPoint property for the new custom action to _DumpAction@4
11. Build the setup project

The resulting MSI has no entries in the File table and an entry
identified by a GUID in the Binary table. This entry is your custom
action DLL.

Now for some reason this works *only* if you do Add File; if you do
Add Output then it installs the file no matter what.

Source code implementing the above walkthrough is here:
<http://www.xmission.com/~legalize/msi/DLLCA.zip>

[dcaldwell]

The custom action uses osql to execute a script that creates a database, constraints, stored procedures, etc. Then it uses bcp to load the tables.

I didn't make any attempt to monitor DLL load/unload, but I did put a call to debug break in the Install method. It appeared to be correctly called twice, once with the proper CustomActionData parameters for DB A and again with parameters for DB B. Both DBs are created and loaded properly. It seems like if something severe happened like a LoadLibrary failure, that the Install method for the 2nd merge module would not be called at all.

The only thing I see wrong with the install is that 1 of the .InstallState files is missing, and I wouldn't care about that if I didn't think it was what is causing the 2nd Uninstall's savedState to be null. After Rich's reply, I realized that these files are probably created by the VS Installer class from which my class inherits, rather than Windows Installer itself.

I haven't tried to get any visibility into the Installer base class functionality, so this is still a WAG, but I've got this sneaking suspicion that it's a relatively simple bug where some code is creating a data structure (like a hash table) that includes entries for both DLLs, but it's using the CA DLL name as a key rather than the component ID, so the 2nd entry overwrites the 1st.

Is there someone on the VS.NET side of things that could easily confirm or deny this theory?

BTW, we realize that we can work directly with Windows Installer, but we're intentionally trying to migrate as much installer functionality as is feasible from the installer developers (who use InstallShield) into the hands of product developers (who use VS.NET). This should help us with both manpower and licensing issues.

But part of that process is figuring out the limitations of VS.NET's merge module capabilities. I got pretty excited when Rich's reply seemed to say that there was an easy way to add a custom action via VS.NET that would put the CA DLL in the binary table. That makes a lot more sense than installing 2 copies of the same file on the target machine and then using their removal as a trigger for the uninstall CA.

So is that really possible? If so, can you give me more details or point me at a documented procedure for doing so?

Thanks,
David

[Phil Wilson]

Well of course the custom action will be called twice. The problem is that
the first DLL gets loaded and called for the first custom action. When the
second custom action is called, it doesn't load the other DLL because the
name is the same, so it calls the same method in the first DLL again.

--
Phil Wilson
[MVP Windows Installer]

"dcaldwell" <anon...@discussions.microsoft.com> wrote in message

news:A7255EF3-18ED-4AF8...@microsoft.com...

[Phil Wilson]

>>>>>>BTW, we realize that we can work directly with Windows Installer, but
we're intentionally trying to migrate as much installer functionality as is
feasible from the installer developers (who use InstallShield) into the hands of
product developers (who use VS.NET). This should help us with both manpower and
licensing issues.>>>>>>

I'm not promoting InstallShield here, but VS has much less functionality than IS
in building MSI files. This problem is exactly the kind of complication you'll
get more of. The VS developers will want to run installer class custom actions
to (for example) install Services, instead of using the ServiceControl/Install
tables that you use in IS's IDE. The minute they want to suspend a Service
during the install they'll be adding more code etc. BTW if you search
InstallShield support you'll find issues with consuming VS merge modules, such
as:

http://support.installshield.com/kb/view.asp?articleid=q108690

--
Phil Wilson [MVP Windows Installer]
----
"dcaldwell" <anon...@discussions.microsoft.com> wrote in message

news:A7255EF3-18ED-4AF8...@microsoft.com...

[Rich [Microsoft Windows Installer MVP]]

[Please do not mail me a copy of your followup]

"Phil Wilson" <phil....@unisys.spamcom> spake the secret code
<OXtT$eGEEH...@TK2MSFTNGP11.phx.gbl> thusly:

>I'm not promoting InstallShield here, but VS has much less functionality than
>IS in building MSI files.

True. If you want to get away from the licensing costs, you might
want to look into a tool like makemsi, which you could integrate into
VS.NET as a project with a custom build step.

>[...] BTW if you search

>InstallShield support you'll find issues with consuming VS merge modules, such
>as:
>
>http://support.installshield.com/kb/view.asp?articleid=q108690

Note that this article describes the problem as a bug in IS, not a bug
in VS.NET.

Are there other issues with VS.NET MSMs and consuming them in
InstallSUCK? We've been doing this for quite some time and haven't
had any problems.

[Rich [Microsoft Windows Installer MVP]]

[Please do not mail me a copy of your followup]

=?Utf-8?B?ZGNhbGR3ZWxs?= <anon...@discussions.microsoft.com> spake the secret code

<7D578A21-A5C0-4807...@microsoft.com> thusly:

>However, following the same steps with a .NET assembly does not work.

You can't run assemblies from the binary table, only Win32 DLLs.