Python DLL in Windows Folder

105 views
Skip to first unread message

Markus Gritsch

unread,
Dec 23, 2007, 9:52:50 AM12/23/07
to pytho...@python.org
Hi,

why does the Python installer on Windows put the Python DLL into the
Windows system32 folder? Wouldn't it be more clean to place it into
the Python installation folder beside the python.exe file?

Kind regards,
Markus

Thorsten Kampe

unread,
Dec 23, 2007, 11:54:41 AM12/23/07
to
* Markus Gritsch (Sun, 23 Dec 2007 15:52:50 +0100)

> why does the Python installer on Windows put the Python DLL into the
> Windows system32 folder?

Are you sure it does?!

Thorsten

Markus Gritsch

unread,
Dec 23, 2007, 12:28:41 PM12/23/07
to Thorsten Kampe, pytho...@python.org

Yes.

Markus

Christian Heimes

unread,
Dec 23, 2007, 12:33:43 PM12/23/07
to pytho...@python.org
Markus Gritsch wrote:
> why does the Python installer on Windows put the Python DLL into the
> Windows system32 folder? Wouldn't it be more clean to place it into
> the Python installation folder beside the python.exe file?

It's the easiest and best way to expose Python for 3rd party
applications and COM. The DLL is removed by the Windows Installer when
its usage counter drops to 0. There is no need to worry ;)

Christian

Markus Gritsch

unread,
Dec 23, 2007, 1:27:41 PM12/23/07
to Christian Heimes, pytho...@python.org

I am not worrying about an orphaned DLL. The Problem is that this way
the Python DLL is being made available for 3rd party applications,
which possibly need a version of Python which is compiled using
another compiler. We are embedding Python into our application which
gets compiled using MSVC 8.0. We like to link dynamically, so the
Python interpreter is not statically linked into the program. The
Python DLL from the Python installer in the Windows system32 folder is
compiled using MSVC 7.1.

Our current solution to the problem is modifying the Python build
process to produce our Python DLL with a filename containing also the
patchlevel of the version number i.e. python251.dll instead of
python25.dll. This way we can be sure that *our* Python DLL gets
loaded. This works fine.

I was just curious, if there is a special reason for placing the
Python DLL into the system32 folder. So COM seems to be the answer.

Thank you,
Markus

Ross Ridge

unread,
Dec 23, 2007, 6:35:57 PM12/23/07
to
Markus Gritsch <m.gr...@gmail.com> wrote:
>... We are embedding Python into our application which

>gets compiled using MSVC 8.0. We like to link dynamically, so the
>Python interpreter is not statically linked into the program. The
>Python DLL from the Python installer in the Windows system32 folder is
>compiled using MSVC 7.1.

If you're targetting Windows XP or newer you can use manifests to force
Windows to link your application with a particular version of the DLL.
On Windows 98SE or newer you can use .local files to force Windows to
link with whatever DLLs are in the same directory as your executable.

>Our current solution to the problem is modifying the Python build
>process to produce our Python DLL with a filename containing also the
>patchlevel of the version number i.e. python251.dll instead of
>python25.dll. This way we can be sure that *our* Python DLL gets
>loaded. This works fine.

I'd give it a more unique name, like "companyname_python25_1_msvc80.dll".
That gives you more certainty that it is actually your DLL. You also
won't have to worry if your users copied python251.dll to the system
directory when you upgrade to Visual Studio 2008.

Ross Ridge

--
l/ // Ross Ridge -- The Great HTMU
[oo][oo] rri...@csclub.uwaterloo.ca
-()-/()/ http://www.csclub.uwaterloo.ca/~rridge/
db //

Thorsten Kampe

unread,
Dec 24, 2007, 3:31:22 AM12/24/07
to
* Markus Gritsch (Sun, 23 Dec 2007 18:28:41 +0100)

If you choose "install just for me" it doesn't.

Chris Mellon

unread,
Dec 24, 2007, 1:07:42 PM12/24/07
to pytho...@python.org
On Dec 23, 2007 12:27 PM, Markus Gritsch <m.gr...@gmail.com> wrote:
> On 23/12/2007, Christian Heimes <li...@cheimes.de> wrote:
> > Markus Gritsch wrote:
> > > why does the Python installer on Windows put the Python DLL into the
> > > Windows system32 folder? Wouldn't it be more clean to place it into
> > > the Python installation folder beside the python.exe file?
> >
> > It's the easiest and best way to expose Python for 3rd party
> > applications and COM. The DLL is removed by the Windows Installer when
> > its usage counter drops to 0. There is no need to worry ;)
>
> I am not worrying about an orphaned DLL. The Problem is that this way
> the Python DLL is being made available for 3rd party applications,
> which possibly need a version of Python which is compiled using
> another compiler. We are embedding Python into our application which

> gets compiled using MSVC 8.0. We like to link dynamically, so the
> Python interpreter is not statically linked into the program. The
> Python DLL from the Python installer in the Windows system32 folder is
> compiled using MSVC 7.1.
>


What the python installer is doing is the Right Thing for making the
standard python dll available to third party applications.
Applications that want a specific version of a specific DLL should use
the mechanisms available for doing so, instead of relying on there
being a specific version of the python dll in the windows folder. This
is just general best practice on Windows.

Markus Gritsch

unread,
Dec 24, 2007, 2:48:18 PM12/24/07
to Thorsten Kampe, pytho...@python.org

Interesting. Therefore you asked me if I am sure :) Does this mean,
that COM stuff does not work when installing Python "just for me"?

Markus

Markus Gritsch

unread,
Dec 24, 2007, 3:04:33 PM12/24/07
to Chris Mellon, pytho...@python.org
On 24/12/2007, Chris Mellon <ark...@gmail.com> wrote:
> On Dec 23, 2007 12:27 PM, Markus Gritsch <m.gr...@gmail.com> wrote:
> > On 23/12/2007, Christian Heimes <li...@cheimes.de> wrote:
> > > Markus Gritsch wrote:
> > > > why does the Python installer on Windows put the Python DLL into the
> > > > Windows system32 folder? Wouldn't it be more clean to place it into
> > > > the Python installation folder beside the python.exe file?
> > >
> > > It's the easiest and best way to expose Python for 3rd party
> > > applications and COM. The DLL is removed by the Windows Installer when
> > > its usage counter drops to 0. There is no need to worry ;)
> >
> > I am not worrying about an orphaned DLL. The Problem is that this way
> > the Python DLL is being made available for 3rd party applications,
> > which possibly need a version of Python which is compiled using
> > another compiler. We are embedding Python into our application which
> > gets compiled using MSVC 8.0. We like to link dynamically, so the
> > Python interpreter is not statically linked into the program. The
> > Python DLL from the Python installer in the Windows system32 folder is
> > compiled using MSVC 7.1.
> >
>
>
> What the python installer is doing is the Right Thing for making the
> standard python dll available to third party applications.

But this forces the 3rd party application being compiled with the same
compiler as the Python DLL which is MSVC 7.1 for Python 2.5 which is a
quite old compiler nowadays.

> Applications that want a specific version of a specific DLL should use
> the mechanisms available for doing so, instead of relying on there
> being a specific version of the python dll in the windows folder.

We do not rely on having some specific version of the Python DLL in
the Windows system 32 folder. However, the MSVC 7.1 compiled version
gets in the way when using a different compiler.

Kind regards,
Markus

Lie

unread,
Dec 24, 2007, 5:53:24 PM12/24/07
to
On Dec 25, 3:04 am, "Markus Gritsch" <m.grit...@gmail.com> wrote:
> On 24/12/2007, Chris Mellon <arka...@gmail.com> wrote:

>
>
>
> > On Dec 23, 2007 12:27 PM, Markus Gritsch <m.grit...@gmail.com> wrote:
> > > On 23/12/2007, Christian Heimes <li...@cheimes.de> wrote:
> > > > Markus Gritsch wrote:
> > > > > why does the Python installer on Windows put the Python DLL into the
> > > > > Windows system32 folder?  Wouldn't it be more clean to place it into
> > > > > the Python installation folder beside the python.exe file?
>
> > > > It's the easiest and best way to expose Python for 3rd party
> > > > applications and COM. The DLL is removed by the Windows Installer when
> > > > its usage counter drops to 0. There is no need to worry ;)
>
> > > I am not worrying about an orphaned DLL.  The Problem is that this way
> > > the Python DLL is being made available for 3rd party applications,
> > > which possibly need a version of Python which is compiled using
> > > another compiler.  We are embedding Python into our application which
> > > gets compiled using MSVC 8.0.  We like to link dynamically, so the
> > > Python interpreter is not statically linked into the program.  The
> > > Python DLL from the Python installer in the Windows system32 folder is
> > > compiled using MSVC 7.1.
>
> > What the python installer is doing is the Right Thing for making the
> > standard python dll available to third party applications.
>
> But this forces the 3rd party application being compiled with the same
> compiler as the Python DLL which is MSVC 7.1 for Python 2.5 which is a
> quite old compiler nowadays.

Unfortunately, Windows doesn't give a lot of support around preventing
DLL duplication. It is regular to found multiple duplicate DLLs on a
Windows system (especially for different version), a common example is
msvcrtxx.dll, which is C's runtime, many program pack their own msvcrt
into their installation package. Therefore it is important for a
windows program to include all the required DLLs into the installer
package and use their own, unless they're sure that they can use other
version (good programs are not DLL implementation specific and good
DLLs maintain a stable interface).

> > Applications that want a specific version of a specific DLL should use
> > the mechanisms available for doing so, instead of relying on there
> > being a specific version of the python dll in the windows folder.
>
> We do not rely on having some specific version of the Python DLL in
> the Windows system 32 folder.  However, the MSVC 7.1 compiled version
> gets in the way when using a different compiler.

If that's the case, just use the MSVC 7.1's compiled code, there won't
be much difference. If you insist to use your own version (or don't
have a choice), you could instead pack your own DLL, use the local
DLL, and ignore the system's DLL. That's the Best Practice in Windows.

> Kind regards,
> Markus

Ross Ridge

unread,
Dec 24, 2007, 6:23:38 PM12/24/07
to
Chris Mellon <ark...@gmail.com> wrote:
>What the python installer is doing is the Right Thing for making the
>standard python dll available to third party applications.
>Applications that want a specific version of a specific DLL should use
>the mechanisms available for doing so, instead of relying on there
>being a specific version of the python dll in the windows folder. This
>is just general best practice on Windows.

No, it's not best practice for an application install any of its files
in the Windows system directory. The system directory is ment only
for drivers and system files. Installing application DLLs in the
system directory is something that should only be done for backwards
compatiblity.

Tim Roberts

unread,
Dec 25, 2007, 1:20:36 AM12/25/07
to
Ross Ridge <rri...@caffeine.csclub.uwaterloo.ca> wrote:

>Chris Mellon <ark...@gmail.com> wrote:
>>What the python installer is doing is the Right Thing for making the
>>standard python dll available to third party applications.
>>Applications that want a specific version of a specific DLL should use
>>the mechanisms available for doing so, instead of relying on there
>>being a specific version of the python dll in the windows folder. This
>>is just general best practice on Windows.
>
>No, it's not best practice for an application install any of its files
>in the Windows system directory.

Python is more than an application. It's a development tool, and its DLLs
are needed by any of the Python applications I create. I disagree with
your assertion.

>The system directory is ment only for drivers and system files.

In your view, what disqualifies python24.dll as a "system file"? The fact
that it wasn't produced by Microsoft?



>Installing application DLLs in the system directory is something that
>should only be done for backwards compatiblity.

Deployment of Python applications is much easier when python24.dll is
located there. That has to weigh in the equation.
--
Tim Roberts, ti...@probo.com
Providenza & Boekelheide, Inc.

Markus Gritsch

unread,
Dec 25, 2007, 4:04:45 AM12/25/07
to Tim Roberts, pytho...@python.org
On 25/12/2007, Tim Roberts <ti...@probo.com> wrote:
> Ross Ridge <rri...@caffeine.csclub.uwaterloo.ca> wrote:
>
> >Chris Mellon <ark...@gmail.com> wrote:
> >>What the python installer is doing is the Right Thing for making the
> >>standard python dll available to third party applications.
> >>Applications that want a specific version of a specific DLL should use
> >>the mechanisms available for doing so, instead of relying on there
> >>being a specific version of the python dll in the windows folder. This
> >>is just general best practice on Windows.
> >
> >No, it's not best practice for an application install any of its files
> >in the Windows system directory.
>
> Python is more than an application. It's a development tool, and its DLLs
> are needed by any of the Python applications I create. I disagree with
> your assertion.

I assume that your Python applications are .py files. To be able to
run them it would be perfectly ok if the Python DLL would be located
beside the python.exe in the Python installation folder.

> Deployment of Python applications is much easier when python24.dll is
> located there. That has to weigh in the equation.

Same doubts as above: For .py files to be executed it is not necessary
that the Python DLL is located in the Windows system32 folder.

Markus

Ross Ridge

unread,
Dec 25, 2007, 4:31:42 AM12/25/07
to
Chris Mellon <ark...@gmail.com> wrote:
>What the python installer is doing is the Right Thing for making the
>standard python dll available to third party applications.
>Applications that want a specific version of a specific DLL should use
>the mechanisms available for doing so, instead of relying on there
>being a specific version of the python dll in the windows folder. This
>is just general best practice on Windows.

Ross Ridge <rri...@caffeine.csclub.uwaterloo.ca> wrote:
>No, it's not best practice for an application install any of its files
>in the Windows system directory.

Tim Roberts <ti...@probo.com> wrote:
>Python is more than an application. It's a development tool, and its DLLs
>are needed by any of the Python applications I create. I disagree with
>your assertion.

A development tool is an application, and is covered by the same best
practice guidelines as any other Windows application.

>>The system directory is ment only for drivers and system files.
>
>In your view, what disqualifies python24.dll as a "system file"? The fact
>that it wasn't produced by Microsoft?

By definition, all system files on Windows are supplied by Microsoft.
They're sometimes produced by other companies, and they can be installed
by third-party applications, but only Microsoft gets to say what is a
supported part of the Windows operating system and thus a system file.

>>Installing application DLLs in the system directory is something that
>>should only be done for backwards compatiblity.
>
>Deployment of Python applications is much easier when python24.dll is
>located there. That has to weigh in the equation.

*shrug* I'm sure it wasn't done malicously. I'm not arguing that Python
should change it behaviour, I know how utterly pointless that would be.
However, like it or not it doesn't follow the best practices for Windows
applications. I'm just hoping that no one reading this thread will
think that Python is doing the "Right Thing" and copy its behaviour.

Markus Gritsch

unread,
Dec 25, 2007, 4:43:01 AM12/25/07
to Lie, pytho...@python.org
On 24/12/2007, Lie <Lie....@gmail.com> wrote:
>
> (good programs are not DLL implementation specific

Assume an application embedding Python being compiled with MSVC 8.0.
It uses the runtime msvcr80.dll. If you now pass objects created in
your application to the Python interpreter which is compiled with MSVC
7.1 which uses msvcr71.dll, you are asking for serious trouble. It is
not a matter of being a "good program".

> If that's the case, just use the MSVC 7.1's compiled code, there won't
> be much difference.

See above.

> If you insist to use your own version (or don't
> have a choice), you could instead pack your own DLL, use the local
> DLL, and ignore the system's DLL.

Yes, that works fine when deploying the application. However, during
development it is inconveninent to put our own Python DLL beside our
executable every time. Our Python DLL (compiled with MSVC 8.0) is in
the PATH. If the original Python DLL would be located in the Python
installation folder, there would be no problem. But since it is
placed in the system32 folder which is searched before the PATH
(http://msdn2.microsoft.com/en-us/library/ms682586(VS.85).aspx), the
original Python DLL (compiled with MSVC 7.1) is used instead of your
own one.

Kind regards,
Markus

Markus Gritsch

unread,
Dec 25, 2007, 5:07:14 AM12/25/07
to Thorsten Kampe, pytho...@python.org
On 24/12/2007, Markus Gritsch <m.gr...@gmail.com> wrote:
> Interesting. Therefore you asked me if I am sure :) Does this mean,
> that COM stuff does not work when installing Python "just for me"?

Indeed, pywin32 stops working :( I installed Python "just for me",
then I installed pywin32-210, and now Pythonwin.exe is not able to
start. So for this scenario it seems to be necessary, that the Python
DLL is installed into the system32 folder.

Maybe the Python installer should issue a warning that pywin32 will
not be able to work when using the option "just for me"?

Kind regards,
Markus

Stef Mientki

unread,
Dec 25, 2007, 5:39:47 AM12/25/07
to pytho...@python.org
Markus Gritsch wrote:
> On 25/12/2007, Tim Roberts <ti...@probo.com> wrote:
>
>> Ross Ridge <rri...@caffeine.csclub.uwaterloo.ca> wrote:
>>
>>
>>> Chris Mellon <ark...@gmail.com> wrote:
>>>
>>>> What the python installer is doing is the Right Thing for making the
>>>> standard python dll available to third party applications.
>>>> Applications that want a specific version of a specific DLL should use
>>>> the mechanisms available for doing so, instead of relying on there
>>>> being a specific version of the python dll in the windows folder. This
>>>> is just general best practice on Windows.
>>>>
>>> No, it's not best practice for an application install any of its files
>>> in the Windows system directory.
>>>
>> Python is more than an application. It's a development tool, and its DLLs
>> are needed by any of the Python applications I create. I disagree with
>> your assertion.
>>
>
> I assume that your Python applications are .py files. To be able to
> run them it would be perfectly ok if the Python DLL would be located
> beside the python.exe in the Python installation folder.
>
>
>> Deployment of Python applications is much easier when python24.dll is
>> located there. That has to weigh in the equation.
>>
>
> Same doubts as above: For .py files to be executed it is not necessary
> that the Python DLL is located in the Windows system32 folder.
>
> Markus
>
Another reason "not to put the DLL in win/wys32":
on a lot of computers writing to win/sys32 or even the whole C-drive is
prohibited !

cheers,
Stef


Neil Hodgson

unread,
Dec 25, 2007, 6:13:51 AM12/25/07
to
Markus Gritsch:

> I assume that your Python applications are .py files. To be able to
> run them it would be perfectly ok if the Python DLL would be located
> beside the python.exe in the Python installation folder.

Your Python application may not be being run by python.exe but as an
in-process COM object by, for example, Internet Explorer. COM loads the
Python COM DLL pythoncomXY.dll which loads pythonXY.dll through an
implicit link step. The implicit link will only find pythonXY.dll in a
particular set of places (path, system directories, host executable),
with the Windows system directory being the easiest to find and use.

Neil

Markus Gritsch

unread,
Dec 25, 2007, 6:33:47 AM12/25/07
to Neil Hodgson, pytho...@python.org

Ok, thank you for this use case and its explanation. It seems this
justifies Pythons DLL being placed in the system32 folder.

Kind regards,
Markus

Thorsten Kampe

unread,
Dec 25, 2007, 1:53:16 PM12/25/07
to
* Stef Mientki (Tue, 25 Dec 2007 11:39:47 +0100)

> Another reason "not to put the DLL in win/wys32":
> on a lot of computers writing to win/sys32 or even the whole C-drive
> is prohibited !

No, definitely not. It's just not allowed for everyone.

Thorsten

Chris Mellon

unread,
Dec 26, 2007, 11:27:18 AM12/26/07
to pytho...@python.org
On Dec 24, 2007 5:23 PM, Ross Ridge <rri...@caffeine.csclub.uwaterloo.ca> wrote:
> Chris Mellon <ark...@gmail.com> wrote:
> >What the python installer is doing is the Right Thing for making the
> >standard python dll available to third party applications.
> >Applications that want a specific version of a specific DLL should use
> >the mechanisms available for doing so, instead of relying on there
> >being a specific version of the python dll in the windows folder. This
> >is just general best practice on Windows.
>
> No, it's not best practice for an application install any of its files
> in the Windows system directory. The system directory is ment only
> for drivers and system files. Installing application DLLs in the
> system directory is something that should only be done for backwards
> compatiblity.


Python (when installed with "for everyone") is not (just) an
application, it's a runtime. I know that Microsoft has totally dropped
the ball on handling shared libraries and has recently thrown up its
arms and just said that all apps should ship with everything they need
("Don't use shared libraries any more, versioning is hard") but that's
not applicable to Python *the runtime*. system32 is precisely the
correct spot for it, when installed by an administrator, as a system
runtime.

Note that "for me only" Python, doesn't install to there.

Markus Gritsch

unread,
Dec 26, 2007, 12:30:00 PM12/26/07
to Lie Ryan, pytho...@python.org
On 26/12/2007, Lie Ryan <lie....@gmail.com> wrote:
>
>
> On Dec 25, 2007 4:43 PM, Markus Gritsch <m.gr...@gmail.com> wrote:

> > On 24/12/2007, Lie <Lie....@gmail.com> wrote:
> > >
> > > (good programs are not DLL implementation specific
> >
> > Assume an application embedding Python being compiled with MSVC 8.0.
> > It uses the runtime msvcr80.dll. If you now pass objects created in
> > your application to the Python interpreter which is compiled with MSVC
> > 7.1 which uses msvcr71.dll, you are asking for serious trouble. It is
> > not a matter of being a "good program".
>
> don't cut my word, good programs aren't DLL implementation specific and good
> DLL have consistent interface (consistent interface implies consistent
> behavior). Of course this is a bit "utopia", as most DLL would sadly have to
> change its interfaces once or twice and it is sometimes impossible for
> programs to be completely free from its DLLs implementation

Instead of being upset about cutting your word (which was not my
intention, sorry about that), it would be nice if you could make a
statement concerning the problem I mentioned: Having an object being
created by one MSVC runtime, msvcr80.dll and passing it to another
one, msvcr71.dll.

> > > If that's the case, just use the MSVC 7.1's compiled code, there won't
> > > be much difference.
> >

> > See above.
> >
>
> I think you don't get my point... if you don't modify the python interpreter
> (from what I understand, you just recompile your Python DLL without
> modification), there is no need to compile your own interpreter, it just
> makes problem and the performance benefits is outweighed by the problems of
> having to use 3rd party Python DLL (even if you don't modify the code and
> just recompile it, it is not the official release, thus 3rd party).

I do get your point, but your point is maintaining compatibility
between two different *versions* of a DLL/Program. I am talking about
having a DLL/Program combination which uses different MSVC runtimes.

> > > If you insist to use your own version (or don't
> > > have a choice), you could instead pack your own DLL, use the local
> > > DLL, and ignore the system's DLL.
> >

> > Yes, that works fine when deploying the application. However, during
> > development it is inconveninent to put our own Python DLL beside our
> > executable every time. Our Python DLL (compiled with MSVC 8.0) is in
> > the PATH. If the original Python DLL would be located in the Python
> > installation folder, there would be no problem. But since it is
> > placed in the system32 folder which is searched before the PATH
> >
> (http://msdn2.microsoft.com/en-us/library/ms682586(VS.85).aspx),
> the
> > original Python DLL (compiled with MSVC 7.1) is used instead of your
> > own one.
>

> If that is the case, you could just simply substitute the system32's dll
> with your in the test machine. If you don't want to dirty the system
> (perhaps because you use the system for other things too), then you could
> benefit from dual booting or virtualization technology.
>
> Dual Booting is putting two (almost) completely seperate OSes in a single
> machine.
> Virtualization is a technology that allows an OS to be run inside another
> OS, an example would be VMWare. This result in degraded performance in the
> client OS, but it's easier to manage than Dual Booting.
>
> You don't need any special program to set up a dual boot system, although
> you might require special knowledge on how to do it. For virtualization
> technology, VMWare and Microsoft have a free version of their virtual
> machines. So basically, these two set ups costs nothing.

Thank you for explaining that.

Lie

unread,
Dec 26, 2007, 4:49:45 PM12/26/07
to
On Dec 27, 12:30 am, "Markus Gritsch" <m.grit...@gmail.com> wrote:
> On 26/12/2007, Lie Ryan <lie.1...@gmail.com> wrote:
> > > On Dec 25, 2007 4:43 PM, Markus Gritsch <m.grit...@gmail.com> wrote:

> > > On 24/12/2007, Lie <Lie.1...@gmail.com> wrote:
>
> > > > (good programs are not DLL implementation specific
>
> > > Assume an application embedding Python being compiled with MSVC 8.0.
> > > It uses the runtime msvcr80.dll.  If you now pass objects created in
> > > your application to the Python interpreter which is compiled with MSVC
> > > 7.1 which uses msvcr71.dll, you are asking for serious trouble.  It is
> > > not a matter of being a "good program".
>
> > don't cut my word, good programs aren't DLL implementation specific and good
> > DLL have consistent interface (consistent interface implies consistent
> > behavior). Of course this is a bit "utopia", as most DLL would sadly have to
> > change its interfaces once or twice and it is sometimes impossible for
> > programs to be completely free from its DLLs implementation
>
> Instead of being upset about cutting your word (which was not my
> intention, sorry about that), it would be nice if you could make a
> statement concerning the problem I mentioned: Having an object being
> created by one MSVC runtime, msvcr80.dll and passing it to another
> one, msvcr71.dll.

I'm not upset, but if my wording seems to be like an upset person I'm
sorry, English is not my first language, and I'm very bad at choosing
words.

A simple solution (but possibly inefficient, if the object is complex
and need to be exchanged thousands of times) for the problem mentioned
is to convert the object to a common agreed format that both sides can
understand when transferring the data such as a simple string (not a
string object, just simple string with no strings attached: pun (not)
intended).

In your case, which is:


> Assume an application embedding Python being compiled with MSVC 8.0.
> It uses the runtime msvcr80.dll. If you now pass objects created in
> your application to the Python interpreter which is compiled with MSVC
> 7.1 which uses msvcr71.dll, you are asking for serious trouble. It is
> not a matter of being a "good program".

No, you're not asking for trouble, you're passing the object to the
Python DLL, not to the msvcr71 behind the Python so it shouldn't be a
problem in a good designed program and DLL. This isn't a case about
msvcr71 talking with msvcr8, nor it is msvcr71 with your program, nor
it is msvcr8 with Python DLL, this is a case of Your Program talking
to Python DLL.

One way to solve this is:
Assuming you don't have access to Python internals (although since
Python is open source, you do have access) you could build a wrapper
around Python Interpreter. The wrapper has two functions. One: accepts
a string input (string as simple string, not string object) that
describes the original object and returns a file that Python can pick
with pickle. Two: accepts a pickled object and returns a string that
describes the pickled object that Your Application could understand.

This functions could be built in into Your Application, but the
easiest way is to build the wrapper with Python, as you won't need to
know how to read and write pickled file.

For a simple case, where the object that needs to be passed is
predetermined your function could just pass any data that is required
to reconstruct the original object to the wrapper (for example, in a
vector class, just the values of x, y, z), a versatile way to pass
them is by command-line arguments, like "wrapper 1 2 3 > output". The
wrapper would add any other required infos (methods, operator
overloads, etc) before pickling the object and call the Python DLL to
unpickle the object. For the vice versa case, the wrapper unpickle the
file returned from Python DLL, and convert it to a format Your Program
could understand, then notifying your program.

Of course there may be other easier, more efficient ways to solve this
problem, but this is a proof-of-concept that it is possible to compile
your application in VS8 while still using the official VS<unknown>
Python.

But I can't think of a need why Python DLL needs to be used like this
though

> > > > If that's the case, just use the MSVC 7.1's compiled code, there won't
> > > > be much difference.
>
> > > See above.
>
> > I think you don't get my point... if you don't modify the python interpreter
> > (from what I understand, you just recompile your Python DLL without
> > modification), there is no need to compile your own interpreter, it just
> > makes problem and the performance benefits is outweighed by the problems of
> > having to use 3rd party Python DLL (even if you don't modify the code and
> > just recompile it, it is not the official release, thus 3rd party).
>
> I do get your point, but your point is maintaining compatibility
> between two different *versions* of a DLL/Program.  I am talking about
> having a DLL/Program combination which uses different MSVC runtimes.

It doesn't matter, your program is talking to the Python DLL not to
the MSVC Runtimes behind it. If you pass an object to the DLL, a good
program would use an intermediary format that is understood by both
sides (Python DLL and Application), and not rely on the implementation
behind the Pythond DLL to understand the object being traded.

"Martin v. Löwis"

unread,
Dec 26, 2007, 6:23:02 PM12/26/07
to
> Indeed, pywin32 stops working :( I installed Python "just for me",
> then I installed pywin32-210, and now Pythonwin.exe is not able to
> start. So for this scenario it seems to be necessary, that the Python
> DLL is installed into the system32 folder.
>
> Maybe the Python installer should issue a warning that pywin32 will
> not be able to work when using the option "just for me"?

That is not supposed to happen; pywin32 should work. Just COM servers
written in Python may not.

Regards,
Martin

Ross Ridge

unread,
Dec 26, 2007, 8:35:24 PM12/26/07
to
Chris Mellon <ark...@gmail.com> wrote:
>Python (when installed with "for everyone") is not (just) an
>application, it's a runtime.

Whatever you want to call it, the Python DLL is not part of the operating
system, it's not a driver and doesn't belong in the system directory.

Tim Roberts

unread,
Dec 26, 2007, 11:49:22 PM12/26/07
to
"Markus Gritsch" <m.gr...@gmail.com> wrote:
>
>Instead of being upset about cutting your word (which was not my
>intention, sorry about that), it would be nice if you could make a
>statement concerning the problem I mentioned: Having an object being
>created by one MSVC runtime, msvcr80.dll and passing it to another
>one, msvcr71.dll.

The C runtime doesn't create objects. Python does use it to allocate
memory, but in the Microsoft run-times, malloc and free call the Win32 heap
APIs, no matter what the version.

It's true that mallocing in one CRT DLL and freeing in another can cause
problems, but in Python, I don't think that can happen. Proper Python
add-ins call Python APIs to create and destroy objects, so only the Python
runtime will manage the memory.

Markus Gritsch

unread,
Dec 27, 2007, 5:12:04 AM12/27/07
to Martin v. Löwis, pytho...@python.org

Ok, my mistake. I was launching Pythonwin.exe by double clicking on
the executable, which works when Python is installed "for all users"
and does not work when installed "just for me". When Pythonwin is
launched by clicking on the shortcut in the start menu, which sets the
CWD to the Python installation folder, it works also when Python is
installed "just for me".

Kind regards,
Markus

"Martin v. Löwis"

unread,
Dec 27, 2007, 9:45:06 AM12/27/07
to
> Whatever you want to call it, the Python DLL is not part of the operating
> system, it's not a driver and doesn't belong in the system directory.

Is that just your personal opinion, or a guideline from the operating
system vendor?

Regards,
Martin

"Martin v. Löwis"

unread,
Dec 27, 2007, 9:47:37 AM12/27/07
to Tim Roberts
>> Instead of being upset about cutting your word (which was not my
>> intention, sorry about that), it would be nice if you could make a
>> statement concerning the problem I mentioned: Having an object being
>> created by one MSVC runtime, msvcr80.dll and passing it to another
>> one, msvcr71.dll.
>
> The C runtime doesn't create objects.

Depends on your definition of "object". fopen will create objects,
and passing those to a different CRT will cause crashes.

Regards,
Martin

Ross Ridge

unread,
Dec 27, 2007, 6:13:31 PM12/27/07
to
Ross Ridge writes:
> Whatever you want to call it, the Python DLL is not part of the operating
> system, it's not a driver and doesn't belong in the system directory.

<mar...@v.loewis.de> wrote:
>Is that just your personal opinion, or a guideline from the operating
>system vendor?

It's one of Microsoft guidelines. The "'Designed for Microsoft Windows
XP' Application Specification" only allows DLLs to be installed in the
system directory if it's required for backwards compatiblity.

"Martin v. Löwis"

unread,
Dec 27, 2007, 8:02:49 PM12/27/07
to
> It's one of Microsoft guidelines. The "'Designed for Microsoft Windows
> XP' Application Specification" only allows DLLs to be installed in the
> system directory if it's required for backwards compatiblity.

I guess you are referring to 2.6 here. I think Python perfectly meets
the specification (i.e. is designed for Windows XP) in this respect (*):
- python25.dll is a shared component
- it is not private to a single software vendor
- it is not a control panel item
- it is not a service or device driver
- it does not support side-by-side installation
- hence, it can be placed in the System directory.

The backwards compatibility with "those applications" is necessary, in
particular with PythonWin (but also with any other software that embeds
Python).

Regards,
Martin

(*) Python fails to meet other requirements; in addition, it has not
formally been certified, so it is not "Designed for Microsoft Windows
XP".

Ross Ridge

unread,
Dec 27, 2007, 9:15:35 PM12/27/07
to
<mar...@v.loewis.de> wrote:
> I think Python perfectly meets the specification (i.e. is designed
>for Windows XP) in this respect (*):
>- python25.dll is a shared component
>- it is not private to a single software vendor
>- it is not a control panel item
>- it is not a service or device driver
>- it does not support side-by-side installation
>- hence, it can be placed in the System directory.

No, that doesn't follow from the requirements given. The only exception
for something that isn't a service or device driver is for backwards
compatibility. Also, pretty much any DLL supports side-by-side
installation.

>The backwards compatibility with "those applications" is necessary, in
>particular with PythonWin (but also with any other software that embeds
>Python).

The backwards compatibility exception doesn't apply when the name of
the DLL changes. If the name of the DLL changes, the shared location
its installed to can also change.

"Martin v. Löwis"

unread,
Dec 27, 2007, 10:05:28 PM12/27/07
to
> No, that doesn't follow from the requirements given. The only exception
> for something that isn't a service or device driver is for backwards
> compatibility. Also, pretty much any DLL supports side-by-side
> installation.

Hmm. How do I have to build and install python25.dll exactly to make
that work? How do applications have to link against it?

> The backwards compatibility exception doesn't apply when the name of
> the DLL changes.

Chapter and verse of the spec?

> If the name of the DLL changes, the shared location
> its installed to can also change.

No. The existing applications still require Python to be located on
the path, even if they are rebuilt against the new DLL. This is
source backwards compatibility, not binary backwards compatibility.

Regards,
Martin

Ross Ridge

unread,
Dec 28, 2007, 12:25:00 AM12/28/07
to
Ross Ridge writes:
> No, that doesn't follow from the requirements given. The only exception
> for something that isn't a service or device driver is for backwards
> compatibility. Also, pretty much any DLL supports side-by-side
> installation.

<mar...@v.loewis.de> wrote:
>Hmm. How do I have to build and install python25.dll exactly to make
>that work? How do applications have to link against it?

As I said before, I know how futile it is to argue that Python should
change it's behaviour. I'm not going to waste my time telling you what
to do. If you really want to know how side-by-side installation works,
you can try reading the Windows SDK documentation.

>> The backwards compatibility exception doesn't apply when the name of
>> the DLL changes.
>
>Chapter and verse of the spec?

It's a logical conclusion that results from the fact an application
can't have a dependecy on a DLL that doesn't exist yet. Putting FOO.DLL
in a different directory than BAR.DLL was installed in won't break any
existing application that uses BAR.DLL.

>> If the name of the DLL changes, the shared location
>> its installed to can also change.
>
>No. The existing applications still require Python to be located on
>the path, even if they are rebuilt against the new DLL. This is
>source backwards compatibility, not binary backwards compatibility.

No, simply by changing the name you've prevented backwards compatiblity
with the old DLL. You can't argue you're trying to maintain backwards
compatibilty with an old DLL when you've already broken compatibility
with it. Since the existing applications have to be rebuilt to use the
new DLL they also can be changed to use it from a new shared location.

"Martin v. Löwis"

unread,
Dec 28, 2007, 4:58:37 AM12/28/07
to
> As I said before, I know how futile it is to argue that Python should
> change it's behaviour. I'm not going to waste my time telling you what
> to do. If you really want to know how side-by-side installation works,
> you can try reading the Windows SDK documentation.

I did, and determined that it's not possible. We would have to use
assembly manifests, and can't, as the tool chain doesn't support them.

> No, simply by changing the name you've prevented backwards compatiblity
> with the old DLL.

First, the requirement for backwards compatibility is not normative.
Section 2.6 is entitled "Install any shared files that are not
side-by-side to the correct locations". python25.dll, as a matter of
fact, *is* a shared file that is not side-by-side; the *only* location
I can install in according to 2.6 is the System folder.

> You can't argue you're trying to maintain backwards
> compatibilty with an old DLL when you've already broken compatibility
> with it.

Furthermore, it says "to ensure backward compatibility with those
applications", not "to ensure backward compatibility with the old DLL".
I'm not trying to maintain compatibility with the binaries of those
applications, but with their source.

> Since the existing applications have to be rebuilt to use the
> new DLL they also can be changed to use it from a new shared location.

No, they can't, because there is not enough manpower available to change
them.

Regards,
Martin

Ross Ridge

unread,
Dec 28, 2007, 7:41:48 AM12/28/07
to
Ross Ridge writes:
> As I said before, I know how futile it is to argue that Python should
> change it's behaviour. I'm not going to waste my time telling you what
> to do. If you really want to know how side-by-side installation works,
> you can try reading the Windows SDK documentation.

<mar...@v.loewis.de> wrote:
> I did, and determined that it's not possible. We would have to use
> assembly manifests, and can't, as the tool chain doesn't support them.

Your choice to use a tool chain that doesn't support side-by-side
assemblies does not make it impossible.

>> No, simply by changing the name you've prevented backwards compatiblity
>> with the old DLL.
>
>First, the requirement for backwards compatibility is not normative.
>Section 2.6 is entitled "Install any shared files that are not
>side-by-side to the correct locations". python25.dll, as a matter of
>fact, *is* a shared file that is not side-by-side; the *only* location
>I can install in according to 2.6 is the System folder.

No, the only thing that the guidelines require to be installed in the
system directory are services and drivers. The only exception for other
kinds of applications is when necessary for backwards compatibility.

>> You can't argue you're trying to maintain backwards
>> compatibilty with an old DLL when you've already broken compatibility
>> with it.
>
>Furthermore, it says "to ensure backward compatibility with those
>applications", not "to ensure backward compatibility with the old DLL".

Now you're just trying play pointless semantic games. If you change
the DLL's name, the DLL is no longer compatibile with those applications.

>> Since the existing applications have to be rebuilt to use the
>> new DLL they also can be changed to use it from a new shared location.
>
>No, they can't, because there is not enough manpower available to change
>them.

And that's what this all really comes down to. Because neither you nor
anyone else is willing to do the work necessary to do the "Right Thing",
you have chosen not to follow Microsoft's recommended best practices
for installing Windows applications. It's not because it's impossible,
it's because you're unwilling.

Is that really all that hard to admit?

Christian Heimes

unread,
Dec 28, 2007, 8:28:45 AM12/28/07
to Ross Ridge, pytho...@python.org
Ross Ridge wrote:
>> I did, and determined that it's not possible. We would have to use
>> assembly manifests, and can't, as the tool chain doesn't support them.
>
> Your choice to use a tool chain that doesn't support side-by-side
> assemblies does not make it impossible.

I'm looking forward to your contribution to solve the problem. Our new
tool chain for Python 2.6 and 3.0 on Windows is VS 2008.

Christian

Ross Ridge

unread,
Dec 28, 2007, 10:11:36 AM12/28/07
to
Ross Ridge wrote:
> Your choice to use a tool chain that doesn't support side-by-side
> assemblies does not make it impossible.

Christian Heimes <li...@cheimes.de> wrote:
> I'm looking forward to your contribution to solve the problem.

I find that highly unlikely. Regardless, as I said twice before I'm not
interested in changing Python's behaviour. I don't have a problem with
the fact that Python isn't following Microsoft's guidelines. My problem
is only with the false claim that it is following them.

Chris Mellon

unread,
Dec 28, 2007, 10:29:02 AM12/28/07
to pytho...@python.org
On Dec 28, 2007 6:41 AM, Ross Ridge <rri...@caffeine.csclub.uwaterloo.ca> wrote:
> Ross Ridge writes:
> > As I said before, I know how futile it is to argue that Python should
> > change it's behaviour. I'm not going to waste my time telling you what
> > to do. If you really want to know how side-by-side installation works,
> > you can try reading the Windows SDK documentation.
>
> <mar...@v.loewis.de> wrote:
> > I did, and determined that it's not possible. We would have to use
> > assembly manifests, and can't, as the tool chain doesn't support them.
>
> Your choice to use a tool chain that doesn't support side-by-side
> assemblies does not make it impossible.
>

It's worth pointing out that when the choice was made,Windows didn't
support SxS at all and there was no toolchain for it.

> >> No, simply by changing the name you've prevented backwards compatiblity
> >> with the old DLL.
> >
> >First, the requirement for backwards compatibility is not normative.
> >Section 2.6 is entitled "Install any shared files that are not
> >side-by-side to the correct locations". python25.dll, as a matter of
> >fact, *is* a shared file that is not side-by-side; the *only* location
> >I can install in according to 2.6 is the System folder.
>
> No, the only thing that the guidelines require to be installed in the
> system directory are services and drivers. The only exception for other
> kinds of applications is when necessary for backwards compatibility.
>

You're not reading the document correctly. The "backwards
compatability" is supporting non SxS libraries.

> >> You can't argue you're trying to maintain backwards
> >> compatibilty with an old DLL when you've already broken compatibility
> >> with it.
> >
> >Furthermore, it says "to ensure backward compatibility with those
> >applications", not "to ensure backward compatibility with the old DLL".
>
> Now you're just trying play pointless semantic games. If you change
> the DLL's name, the DLL is no longer compatibile with those applications.
>

Python 2.5 already exists and has shipped.

> >> Since the existing applications have to be rebuilt to use the
> >> new DLL they also can be changed to use it from a new shared location.
> >
> >No, they can't, because there is not enough manpower available to change
> >them.
>
> And that's what this all really comes down to. Because neither you nor
> anyone else is willing to do the work necessary to do the "Right Thing",
> you have chosen not to follow Microsoft's recommended best practices
> for installing Windows applications. It's not because it's impossible,
> it's because you're unwilling.
>

You're persisting in calling Python an "application", which it clearly
isn't. How relevant these guidelines are at all, since they're
targeted toward end-user boxed software and not toward developers of
third party libraries, is questionable, but lets see what it says.

Section 2.6 *clearly* states that non side by side dlls (which Python
is) can (and, in fact *must*) be placed in the System directory. Note
that making your libraries SxS is *not* a requirement. So if we're
going to follow this (questionable) document at all, Python is
absolutely compliant.

Ross Ridge

unread,
Dec 28, 2007, 7:49:04 PM12/28/07
to
<mar...@v.loewis.de> wrote:
> I did, and determined that it's not possible. We would have to use
> assembly manifests, and can't, as the tool chain doesn't support them.

Ross Ridge writes:
> Your choice to use a tool chain that doesn't support side-by-side
> assemblies does not make it impossible.

Chris Mellon <ark...@gmail.com> wrote:
>It's worth pointing out that when the choice was made,Windows didn't
>support SxS at all and there was no toolchain for it.

Python 2.5 was released on September 16, 2006, Python 2.4 on November
30, 2004, and Python 2.3 on July 29, 2003. Windows first supported
assembly manifests in Windows XP, which was released on October 25, 2001.
There have been plenty of opportunities to add assembly manifests to
the Python installer, if the developers were willing.

Note also that private side-by-side DLL installation using .local files
has been supported since Windows 98SE.

Michael Doppler

unread,
Dec 29, 2007, 6:05:39 AM12/29/07
to
On 27 Dez., 05:49, Tim Roberts <t...@probo.com> wrote:
> It's true that mallocing in one CRT DLL and freeing in another can cause
> problems, but in Python, I don't think that can happen.  Proper Python
> add-ins call Python APIs to create and destroy objects, so only the Python
> runtime will manage the memory.

There is at least one actual problem I came across in an application
linked with MSVCR 8.0, loading the Python DLL linked against MSVCR
7.1. When running a Python file using one of the PyRun_File[1]
functions, the passed file handle has to be created from the embedding
application. This handle is incompatible with the C file handling
functions used by the C runtime of the Python DLL, which causes it to
crash when calling PyRun_File.

regards,
Michael Doppler

[1] http://docs.python.org/api/veryhigh.html

"Martin v. Löwis"

unread,
Dec 29, 2007, 6:19:50 AM12/29/07
to Michael Doppler
> There is at least one actual problem I came across in an application
> linked with MSVCR 8.0, loading the Python DLL linked against MSVCR
> 7.1. When running a Python file using one of the PyRun_File[1]
> functions, the passed file handle has to be created from the embedding
> application. This handle is incompatible with the C file handling
> functions used by the C runtime of the Python DLL, which causes it to
> crash when calling PyRun_File.

Indeed. That is one of the main reasons to remove that API in Py3k.

Regards,
Martin

Reply all
Reply to author
Forward
0 new messages