Application won’t launch from document

[Steve Achelis]

I’m porting an MFC VC 6.0 app to VS 2008. The application won’t launch
by opening one of its documents (i.e., the app’s file type isn’t
getting registered correctly).

My document string is defined as follows (no, my app isn’t really
named AppName <g>):

IDR_MAINFRAME "AppName\nSystems\nSystem\n AppName Files (*.rig)\n.rig
\n AppName System\n AppName "

In InitInstance() I have:

…
pDocTemplate = new CSingleDocTemplate( … );
…
EnableShellOpen();
RegisterShellFileTypes(true);
…

If I step into RegisterShellFileTypes(true), I can see that a call to
_AfxSetRegKey() fails. (The failure happens in a call to
AfxRegSetValue () which returns 5.) This call posts the following
message to the Output window:

Warning: registration database update failed for key 'AppNameSystem'.

I’m running Vista with the latest updates.

Help? This one has beaten me up far too long.

Thanks!

[David Ching]

"Steve Achelis" <in...@RescueRigger.com> wrote in message
news:d6ecf273-246c-44f3...@i37g2000yqn.googlegroups.com...

> If I step into RegisterShellFileTypes(true), I can see that a call to
> _AfxSetRegKey() fails. (The failure happens in a call to
> AfxRegSetValue () which returns 5.) This call posts the following
> message to the Output window:
>
> Warning: registration database update failed for key 'AppNameSystem'.
>
> I’m running Vista with the latest updates.
>

If in the IDE you run Tools | Error Lookup, you can easily see what '5'
means. It means "Access denied."

Since you are running Vista, the problem may be UAC is enabled and when your
app tries to associate the shell types, it is being denied since this
affects HKCR which requires Admin rights. You could run your program As
Administrator or change the manifest to require Admin rights in order to
execute to workaround this. I don't recommend the latter, since Admin
rights would not be needed after the first time the app is run and the shell
types are correctly set up.

The true way to fix this is to modify your installer (which always runs with
Admin rights) to set the shell types for you, and remove the call to
RegisterFileTypes() in your app.

-- David

[Joseph M. Newcomer]

This is not surprising; in fact, it is expected. Vista enforces various security
mechanisms, one of which is that VS (which does not require privileges to run VS2008) runs
in a "real" environment, the kind your end users will see, which means HKEY_LOCAL_MACHINE
(HKLM) is protected, and you can't write to it. So a return code of 5 is not surprising.
Have you never gotten an error code before? Do you know how to look them up? 5 means
"access denied" which means "you weren't allowed to do that!" which is the CORRECT
behavior for ANY program that tries to write HKLM. This should have been universally true
in the past, always, for everyone, at all times, but Microsoft was sloppy about security
back then. This is because There Can Never Be Malware On The Nice Friendly Internet.

You will need to make this association when you install the program, You will typically
want to have an option on the command line, say, /REGISTER, which you look for, and only
if it is present will you call RegisterShellFileTypes (you would think that this would
have been already done for you in the MFC framework, but since it is sensible, it probably
won't happen). Then, you set up your install script (which is running with extended
privileges) to launch your app with the /REGISTER option. What I do is return FALSE from
InitInstance after this, so the app shuts down.
joe

Joseph M. Newcomer [MVP]
email: newc...@flounder.com
Web: http://www.flounder.com
MVP Tips: http://www.flounder.com/mvp_tips.htm

[Hector Santos]

SWAG: Did you compile for Uni-CRAP (Unicode)? Try turning it off to
see if that resolves it. Then fix it up if you must use unicode.

I just went thru a similar process with CParseCommandLine in VC6
recompiled for VS2005. I had to change the ParamParam() prototype in
my CCommandLineInfo subclass.

IMV, UNI-GARBAGE introduces BUGS so this is just SWAG if you are
experiencing VC6 porting woes.

---

Steve Achelis wrote:

> I�m porting an MFC VC 6.0 app to VS 2008. The application won�t launch
> by opening one of its documents (i.e., the app�s file type isn�t
> getting registered correctly).
>
> My document string is defined as follows (no, my app isn�t really

> named AppName <g>):
>
> IDR_MAINFRAME "AppName\nSystems\nSystem\n AppName Files (*.rig)\n.rig
> \n AppName System\n AppName "
>
> In InitInstance() I have:
>

> �
> pDocTemplate = new CSingleDocTemplate( � );
> �
> EnableShellOpen();
> RegisterShellFileTypes(true);
> �

>
> If I step into RegisterShellFileTypes(true), I can see that a call to
> _AfxSetRegKey() fails. (The failure happens in a call to
> AfxRegSetValue () which returns 5.) This call posts the following
> message to the Output window:
>
> Warning: registration database update failed for key 'AppNameSystem'.
>

> I�m running Vista with the latest updates.

>
> Help? This one has beaten me up far too long.
>
> Thanks!

--
HLS

[Steve Achelis]

Thanks guys. I hadn't run into this before, probably because during
install they're running higher privileges so it gets set. I have been
calling this every time on startup (which has probably been failing
all along under Vista). I'll shift it to my setup program... And no, I
didn’t know about the nifty error lookup in VS2008. Sweet.

[Joseph M. Newcomer]

One of the important development strategies is that you never test your code at elevated
privileges. I run with an account that is an "ordinary user" set of privileges. This
means I'm testing my app in the same environment that the end user will see. It avoids
getting into situations where you are able to do something that your end users can't.

Back when I was using VS2005, which required elevated privileges to run on Vista (bad
mistake on Microsoft's part) it would run the code being tested at elevated privilege (an
even worse mistake). I had to run code that looked at the "integrity level" of the
current process, and if it was elevated, re-launch the process at a lower integrity level.
Then I had to re-attach the debugger to it, which was a real pain, but I was able to do
legitimate testing of the app.
joe

[Tom Serface]

You may find this code to be useful. I add this to all of my programs now
to get the associations written to HKCU instead:

This goes in InitInstance()

OverrideHKCR();
RegisterShellFileTypes(true);
RestoreHKCR();

This goes wherever:

bool OverrideHKCR()
{
// If we can't write to HKEY_CLASSES_ROOT, we should try
// HKEY_CURRENT_USER\Software\Classes instead.

HKEY hkcr;
long ret = RegOpenKeyEx(HKEY_LOCAL_MACHINE, _T("Software\\Classes"), 0,
KEY_ALL_ACCESS, &hkcr);
if(ret != ERROR_SUCCESS) { // Need to override
HKEY hkcu;
ret = RegOpenKey(HKEY_CURRENT_USER, _T("Software\\Classes"), &hkcu);
if(ret == ERROR_SUCCESS) {
ret = RegOverridePredefKey(HKEY_CLASSES_ROOT, hkcu);
RegCloseKey(hkcu);
return ret == ERROR_SUCCESS;
}
}
else {
RegCloseKey(hkcr);
}
return false; // Didn't need to do this
}

void RestoreHKCU()
{
RegOverridePredefKey(HKEY_CLASSES_ROOT, NULL);
}

"Steve Achelis" <in...@RescueRigger.com> wrote in message
news:d6ecf273-246c-44f3...@i37g2000yqn.googlegroups.com...