Which compiler will Python 2.5 / Windows (Intel) be built with?
me...@mesw.de
which compiler will Python 2.5 on Windows (Intel) be built with? I
notice that Python 2.4 apparently has been built with the VS2003
toolkit compiler, and I read a post from Scott David Daniels [1] where
he said that probably the VS2003 toolkit will be used for Python 2.5
again. However, even before the release of Python 2.5, I cannot seem to
find many retailers around here that still carry Visual Studio 2003,
and some were a bit surprised about my request since Visual Studio 2005
has been available for some months now. Even more importantly, there
does not seem to be an official way to still get the 2003 toolkit from
Microsoft. The site where it used to be available [2] now redirects you
to the 2005 toolkit. The 2003 toolkit also seems to have disappeared
from the browseable downloads on the Microsoft page. Searching for
VCToolkitSetup.exe on the net, I found a few pages that still appear to
carry the file, but most are down or just redirect to a broken link on
the Microsoft site. That means that if Python 2.5 will be based on the
2003 toolkit compiler, it will be increasingly difficult to get a
compiler for extensions.
Since we have some Python extensions here that use MFC internally (MFC
is only available in the retail version of Visual Studio .NET), we need
to know in which compiler we have to invest to keep our extensions
compatible with Python 2.5. Furthermore, since we also have legacy C++
applications that link to the same libraries that are also used in the
Python extensions, we would be disappointed when we now had to switch
to Visual Studio 2003 just to be compatible with Python 2.5, loosing
official support from Microsoft in near future, and having to use an
outdated compiler throughout the lifetime of Python 2.5.
Thanks in advance for your comments!
Markus Meyer
[1] Google Groups: Python 2.5 Schedule (18 messages)
http://groups.google.ca/group/comp.lang.python/browse_thread/thread/2f8be89236999a37/f6f95174484c24cc?hl=en
[2] Microsoft Visual C++ Toolkit 2003
http://msdn.microsoft.com/visualc/vctoolkit2003/
Scott David Daniels
> Hi everyone,
>
> which compiler will Python 2.5 on Windows (Intel) be built with?
Same as for Python 2.4 (the decision was taken a while ago).
Intel sells a compatible compiler, I believe.
--
--Scott David Daniels
scott....@acm.org
Jarek Zgoda
>> which compiler will Python 2.5 on Windows (Intel) be built with?
>
> Same as for Python 2.4 (the decision was taken a while ago).
> Intel sells a compatible compiler, I believe.
Sounds rather bad.
Anyway, there should be some kits available from second-hand at auction
websites. Besides no free (as in "free beer") toolkit will be available,
as noone except Microsoft can distribute it.
--
Jarek Zgoda
http://jpa.berlios.de/
nikie
> Hi everyone,
>
> which compiler will Python 2.5 on Windows (Intel) be built with? I
> notice that Python 2.4 apparently has been built with the VS2003
> toolkit compiler, and I read a post from Scott David Daniels [1] where
> he said that probably the VS2003 toolkit will be used for Python 2.5
> again. However, even before the release of Python 2.5, I cannot seem to
> find many retailers around here that still carry Visual Studio 2003,
> and some were a bit surprised about my request since Visual Studio 2005
> has been available for some months now. ...
If you want to *buy* VS 2003, you could still purchase a 1-year MSDN
Pro Subscription. The price difference isn't *that* big compared to a
single-user license of VS, and it automatically includes past VS
versions (everything from VC++ 6.0 and upwards, IIRC).
Jarek Zgoda
> If you want to *buy* VS 2003, you could still purchase a 1-year MSDN
> Pro Subscription. The price difference isn't *that* big compared to a
> single-user license of VS, and it automatically includes past VS
> versions (everything from VC++ 6.0 and upwards, IIRC).
This doesn't make building Python exension libraries any easier.
In some cases, you can still build Python extension with MinGW. I didn't
try this with anything more complicated than linking to libxml2, but
still, it's some workaround. Not sure about the performace of such
build, though.
Steve Holden
> Hi everyone,
>
> which compiler will Python 2.5 on Windows (Intel) be built with?
The default build will be the VC 2003 compiler as for 2.4, but there
will also be VC 2005 project support files in a PCBuild8 subdirectory,
if current efforts by CCP succeed. I am certainly compiling the 2.5
trunk with VC 2005 (VC++ 8.0, IIRC).
It would be great if Tim Peters got more support for his efforts in that
area, so it's good to see people besides Tim, Mark Hammond and Martin
von Loewis doing significant work on that platform.
Eventually someone may contribute (say) mingw support, but I'm not aware
that anyone has yet stood up for the challenge of getting that into the
distributions (and anyway I suspect that VC 200X will remain the default).
regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Love me, love my blog http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden
me...@mesw.de
Scott David Daniels wrote:
> me...@mesw.de wrote:
> > Hi everyone,
> >
> > which compiler will Python 2.5 on Windows (Intel) be built with?
>
> Same as for Python 2.4 (the decision was taken a while ago).
> Intel sells a compatible compiler, I believe.
the problem is not the ABI, but the runtime libraries. From what you're
saying, it looks like we will have to standardize on VS2003. As I said,
we need to buy VS anyway because of the MFC support. On the other hand,
I really worry about all those people that want to build open source
extensions for Python 2.5. It is really possible that there will be no
legal, free way to do that soon if you don't have an old installation
of the 2003 toolkit lying around somewhere... So I'd like to ask you:
why was the decision taken a while ago (and is not subject to
reconsideration) and what are the reasons for using VS2003? I mean
there must be a real good reason why you're doing this, as I only see
disadvantages in it.
Markus
Mr Roboto
> nikie napisa³(a):
I haven't personally tried a Python compile w/ this, but I'll
share it in hopes that it'll help: one can download a free copy
of Visual C++ 2K5 *Express* from microsoft itself. If you're
interested, try:
http://msdn.microsoft.com/vstudio/express/visualc/default.aspx
It's legal, free (no registration, no BS.) HTH....MR
me...@mesw.de
> I haven't personally tried a Python compile w/ this, but I'll
> share it in hopes that it'll help: one can download a free copy
> of Visual C++ 2K5 *Express* from microsoft itself. If you're
> interested, try:
The problem is, when you compile an extension module with VS (Express)
2005 and try to load it in a VS2003-compiled Python (which apparently
2.5 will be), there will be errors. So you have to recompile Python
itself with VS2005. This in turn will make it incompatible with any
binary open-source extension out there. E.g., if you use wxPython, you
will then have to recompile that also etc. pp. Also, since there is no
"official" support for the build with VS 2005, it is not clear, if the
differences in the compiler will introduce subtile bugs, say in memory
handling and the like. All in all, this situation feels not good to
me...
Markus
Fredrik Lundh
> the problem is not the ABI, but the runtime libraries. From what you're
> saying, it looks like we will have to standardize on VS2003. As I said,
> we need to buy VS anyway because of the MFC support. On the other hand,
> I really worry about all those people that want to build open source
> extensions for Python 2.5. It is really possible that there will be no
> legal, free way to do that soon if you don't have an old installation
> of the 2003 toolkit lying around somewhere... So I'd like to ask you:
> why was the decision taken a while ago (and is not subject to
> reconsideration) and what are the reasons for using VS2003? I mean
> there must be a real good reason why you're doing this, as I only see
> disadvantages in it.
hint: most people who provide third-party extensions to Python support
more than just the latest Python version...
</F>
Jarek Zgoda
> hint: most people who provide third-party extensions to Python support
> more than just the latest Python version...
We're happy with your support for us, Windows users, but you are an
exception to the general rule of providing only sources.
That's the reason we are fragile on compiler. If the core will be buid
with "non-generally-available" compiler, we would end up with searching
for person willing to compile a library for us, if the MinGW way won't
succeed. I'd like to see core compiled with latest available "free"
toolkit compiler available, as previous versions can not be downloaded
from provider's home.
Scott David Daniels
> ... So I'd like to ask you:
> why was the decision taken a while ago (and is not subject to
> reconsideration) and what are the reasons for using VS2003? I mean
> there must be a real good reason why you're doing this, as I only see
> disadvantages in it.
The disruption in Python 2.4 in switching from one compiler (VC6) to
another VS2003 was not insubstantial. By sticking with VS2003, sometime
users can at least use the same tool for Python 2.4 and Python 2.5. It
does seem inevitable we will have to switch for 2.6. We are very far
along in the process of releasing Python 2.5 (beta1 is due out soon),
and rebuilding and testing with a new translation system is too big a
change at this point.
Note there was strong resistance to leaving VC6 for Python 2.4. That
resistance was overcome only by the fact that it was no longer possible
to purchase suitable versions of VC6.
--Scott David Daniels
scott....@acm.org
Chris Lambacher
> Fredrik Lundh napisa?(a):
MingW. That one was the win32all extensions. The free (beer) compiler that
Microsoft provides did not work for that either since you MFC.
There should be no problem building extensions with MinGW unless you are
writing an extension to an extension that was written in C++ and distributed
only in object form(ie no source), or the extension makes use of the c
runtime (fopen, printf etc).
That said it would be nice if the barrier to entry on compiling Python windows
was reduced by making the effort to moving to gcc as the compiler. Given that
this argument has come up several times on both c.l.p and the python-dev list
with no one biting, I would say that it is not likely to happen any time soon.
I also seem to remember a discussion about it being possible to compile python
with VS2005, but you would then be responsible for your own build of python,
plus building any extension modules you need.
-Chris
Jarek Zgoda
> The disruption in Python 2.4 in switching from one compiler (VC6) to
> another VS2003 was not insubstantial. By sticking with VS2003, sometime
> users can at least use the same tool for Python 2.4 and Python 2.5. It
> does seem inevitable we will have to switch for 2.6. We are very far
> along in the process of releasing Python 2.5 (beta1 is due out soon),
> and rebuilding and testing with a new translation system is too big a
> change at this point.
>
> Note there was strong resistance to leaving VC6 for Python 2.4. That
> resistance was overcome only by the fact that it was no longer possible
> to purchase suitable versions of VC6.
Fcuk, now it's nearly impossible to buy VC2003 toolkit. What now?
Shouldn't Python dev team switch to current MS compiler?
Sorry, gals and guys, but if you force us to buy something irrelevant
like VC2003, you will not get our sympathy. How should I explain my boss
that we need to buy one Pro license more, just to be able to build our
Python app? Please, don't left us with pants down.
Fredrik Lundh
> Sorry, gals and guys, but if you force us to buy something irrelevant
> like VC2003, you will not get our sympathy. How should I explain my boss
> that we need to buy one Pro license more, just to be able to build our
> Python app? Please, don't left us with pants down.
huh? 2.5 isn't released yet. if you *have* a Python app, you can
continue to use the same compiler when you upgrade from 2.4 and 2.5.
it's not like anyone is forcing you to uninstall the compiler just
because you upgrade Python...
</F>
me...@mesw.de
thanks for keeping up the friendly discussion. Comments below.
Scott David Daniels wrote:
> The disruption in Python 2.4 in switching from one compiler (VC6) to
> another VS2003 was not insubstantial. By sticking with VS2003, sometime
> users can at least use the same tool for Python 2.4 and Python 2.5. It
> does seem inevitable we will have to switch for 2.6. We are very far
> along in the process of releasing Python 2.5 (beta1 is due out soon),
> and rebuilding and testing with a new translation system is too big a
> change at this point.
I understand that you are far in the release cycle and that this change
would maybe even delay the whole release process. Those are good
points. OTOH I think that sometimes it's better to change decisions in
light of new facts. Of course I don't know exactly when this decision
was fixed, but I guess since then Microsoft has created two new facts
that cannot be ignored:
* It wasn't clear that Microsoft would stop distributing the free 2003
toolkit in favor of the 2005 toolkit. I cannot remember that they did
something like this in the past, so this is something that came as a
surprise.
* At least to me it wasn't clear that Microsoft would release a new
version of Visual Studio so early, and that it would link to a new,
incompatible C runtime.
One can like or not like Microsoft politics, but I think in case of
those new and surprising facts a re-evaluation of the decision for
compiling Python with VS2003 might very well be justified.
> Note there was strong resistance to leaving VC6 for Python 2.4. That
> resistance was overcome only by the fact that it was no longer possible
> to purchase suitable versions of VC6.
I'm not sure how that backs the point you made. Infact, you're saying
that people accepted that Python 2.4 was compiled with VS2003 because
VC6 could not longer be bought. How is that different from the current
situation where the VS2003 toolkit cannot longer be downloaded and it
is at least becoming increasingly difficult to buy versions of VS2003?
You also seem to imply that there is a large group of people that want
you to stay with VS2003 for compiling Python 2.5. Of course I have no
actual figures, but at least in this thread it seems to me that every
single person who wrote in this thread until now was pro-2005 and
against-2003.
Markus
Scott David Daniels
> Sorry, gals and guys, but if you force us to buy something irrelevant
> like VC2003, you will not get our sympathy.
appreciation of our efforts.
--Scott David Daniels
scott....@acm.org
me...@mesw.de
first, thanks for PIL, I use it extensively in my daily work ;)
Fredrik Lundh wrote:
> huh? 2.5 isn't released yet. if you *have* a Python app, you can
> continue to use the same compiler when you upgrade from 2.4 and 2.5.
> it's not like anyone is forcing you to uninstall the compiler just
> because you upgrade Python...
No, but the compiler that used to be available for compiling 2.4
extensions is no longer available and/or supported. So there is/was
hope that 2.5 might improve the situation because then compiling
extensions would be possible again with the currently available
compiler from Microsoft. This would require that Python 2.5 be built
with this compiler of course. So the situation is actually worse than
before (when the 2003 toolkit was available), and the decision for 2003
means that it won't improve with the release of 2.5. I believe this is
what the gp was trying to say...
Markus
Jarek Zgoda
We don't have VC2003. We still use 2.3, as 2.3.3 for a long time was the
latest available version for iSeries. Consider Python to be a part of
"dll-hell".
Why wouldn't Python dev team standarize on one compiler for all
officially supported platforms?
Fredrik Lundh
> I'm not sure how that backs the point you made. Infact, you're saying
> that people accepted that Python 2.4 was compiled with VS2003 because
> VC6 could not longer be bought. How is that different from the current
> situation where the VS2003 toolkit cannot longer be downloaded and it
> is at least becoming increasingly difficult to buy versions of VS2003?
> You also seem to imply that there is a large group of people that want
> you to stay with VS2003 for compiling Python 2.5.
what part of "Python 2.4 is built with VC2003 and everyone who's ever
built Windows stuff for Python 2.4 already has it" do you have trouble
understanding ?
</F>
Jarek Zgoda
>> Sorry, gals and guys, but if you force us to buy something irrelevant
>> like VC2003, you will not get our sympathy.
>
> Oh, no. And just when our bank account was getting full from your
> appreciation of our efforts.
Prepare to be happy, as you owe us $9999.98 for legal & financial
assistance. We got a public record of your misbehaviour.
--
Jarek Zgoda
http://jpa.berlios.de
me...@mesw.de
I'm actually a bit disappointed that this thread is slowly turning into
a flame-war (and you would not be uninvolved when it does so). I
originally only wanted to create a discussion on a simple topic. I
understand that the Python community is exactly that -- a community
where people voice their opinion and the result of a discussion will
ultimately be a consensus that is followed. I do understand that a
decision on the issue at stake has already been made, but as I
explained in the GP I think that this is a decision that one might
re-evaluate in light of new facts created by a third party (Microsoft).
In any case, there's no reason to get hot about the topic.
But responding to your original question, let me give an example so
maybe you can see my point a bit clearer. I have a customer where I
recently had to add a feature to a Python 2.4 extension. About a year
ago, I wrote the extension, downloaded the 2003 toolkit from Microsoft,
compiled the extension and installed it on the customer's site. Since
then, I got a new laptop, and I never installed the 2003 toolkit again,
because all my other software is in either pure Python or pure Visual
C++ 6 w/ MFC. So I was at the customer's site and discovered that the
2003 toolkit is no longer available at Microsoft. Bummer! Since it is
an important customer, I decided to phone around in the customer's city
if any shop had a version of Visual Studio 2003 lying around so I would
be able to compile the extension. They all would have readily sold me
Visual Studio 2005, but had no 2003 in stock. Second bummer! What I'm
trying to make here are two points:
* Using outdated development tools for new releases is just begging for
difficulties. Python 2.5 will be out there some time, and it will be
increasingly difficult to get VS 2003. The support for VS 2003
(including support for security holes) will probably expire during the
lifetime of Python 2.5. It's like releasing a Python 2.5 that does not
work on Windows XP, only on Windows 2000.
* There is currently a very good optimizing free (as in beer) compiler
available for the Windows platform: The VS Toolkit 2005. But you won't
be able to write extensions for Python 2.5 with this compiler. That's a
pity for the Open Source community as whole, as many Open Source
developers cannot afford to buy a commercial compiler (and many won't
do it even if they could). Note that this is different from the
situation we had a year ago. A year ago, we could download the 2003
toolkit and build extensions for Python 2.4. Now, when Python 2.5 comes
out, we cannot download the 2003 toolkit anymore. Therefore, we cannot
build extensions for 2.4, but not for 2.5 either. Of course those that
still have an install of the 2003 toolkit lying around are happy -- but
only until the next reinstall of their OS.
Markus
Scott David Daniels
> I understand that you are far in the release cycle and that this change
> would maybe even delay the whole release process. Those are good
> points. OTOH I think that sometimes it's better to change decisions in
> light of new facts.
of testing and debugging has gone on, and there is a single person
trying to keep the source VS2005 compatible.
> Of course I don't know exactly when this decision was fixed, but I
> guess since then Microsoft has created two new facts that cannot be
ignored:
>
> * It wasn't clear that Microsoft would stop distributing the free 2003
> toolkit in favor of the 2005 toolkit. I cannot remember that they did
> something like this in the past, so this is something that came as a
> surprise.
>
> * At least to me it wasn't clear that Microsoft would release a new
> version of Visual Studio so early, and that it would link to a new,
> incompatible C runtime.
Nor was it clear to the PyDev community. Microsoft offered free
development systems to those among the PyDev group who were core
developers, and we took that offer. At the time we had no idea
it was on such a short-windowed product. VC6 lasted a _long_ time.
> One can like or not like Microsoft politics, but I think in case of
> those new and surprising facts a re-evaluation of the decision for
> compiling Python with VS2003 might very well be justified.
If we were pre-alpha, or possibly just after alpha1, I might agree with you.
>> Note there was strong resistance to leaving VC6 for Python 2.4. That
>> resistance was overcome only by the fact that it was no longer possible
>> to purchase suitable versions of VC6.
> I'm not sure how that backs the point you made.
I pointed that out to explain that we are reluctant to force developers
to continuously upgrade their VC toolkits and build procedures. Just
because you want to hop from 2.3 to 2.5, doesn't mean that there aren't
those already on 2.4 (and they have already suffered through new build
info pain).
> In fact, you're saying that people accepted that Python 2.4 was
> compiled with VS2003 because VC6 could not longer be bought. How is
> that different from the current situation where the VS2003 toolkit
> cannot longer be downloaded and it is at least becoming increasingly
> difficult to buy versions of VS2003?
As nikie pointed out, you can buy a 1-year MSDN Pro Subscription that
includes the VS2003 system. All that stopped is the free toolkit.
> You also seem to imply that there is a large group of people that want
> you to stay with VS2003 for compiling Python 2.5. Of course I have no
> actual figures, but at least in this thread it seems to me that every
> single person who wrote in this thread until now was pro-2005 and
> against-2003.
Taking percentages of people who complain is not reflect necessarily
reflective of percentages of the general population. If I recall
correctly, there was some angst about using VC6, because it was only
available for a hefty-for-hobbyists price. That was quelled with the
explanation that essentially all Windows developers used the Visual
Studio toolkit.
As to MinGW, nobody has signed up to commit long-term to doing the PyDev
work that is required to get (and keep) it working. Such a developer
would be welcome. There _are_ notes out there on the web to help you
get such things going; I have done my own little bit to share what I
know about how to do that.
--Scott David Daniels
scott....@acm.org
Fredrik Lundh
> Well, it's simply not true. I switched to Cygwin Python because then I
> could use gcc to compile my extensions.
and you're shipping extensions that works with a stock Python distribution ?
or are we playing the "but if we ignore the context, a literal
interpretation of your last sentence" game, again ? boring.
</F>
Scott David Daniels
> Jarek Zgoda wrote:
>> Sorry, gals and guys, but if you force us to buy something irrelevant
>> like VC2003, you will not get our sympathy.
> Oh, no. And just when our bank account was getting full from your
> appreciation of our efforts.
Sorry guys, I'm getting a bit too snippy from being sniped at by people
who don't do the work but insist that they know the best direction for
the project. I'll try to stop answering (at least in that vein) for a
day or two.
--Scott David Daniels
scott....@acm.org
me...@mesw.de
thanks for your clear words.
Scott David Daniels wrote:
> Nor was it clear to the PyDev community. Microsoft offered free
> development systems to those among the PyDev group who were core
> developers, and we took that offer. At the time we had no idea
> it was on such a short-windowed product. VC6 lasted a _long_ time.
Hmm, I see....
> As nikie pointed out, you can buy a 1-year MSDN Pro Subscription that
> includes the VS2003 system. All that stopped is the free toolkit.
The MSDN Pro Subscription is not really an option because we have no
use for the 2005 compiler at all if it doesn't work with Python. I
think we'll just try to get some boxed version of Visual Studio 2003
Professional from somewhere.
> Taking percentages of people who complain is not reflect necessarily
> reflective of percentages of the general population.
Of course you're right, but still I think you will get lots of
complaints once 2.5 is out and people realize that they (again) have to
buy a compiler to compile extensions. Don't get me wrong, it's not a
problem for me, but it will be a problem for other people.
> If I recall
> correctly, there was some angst about using VC6, because it was only
> available for a hefty-for-hobbyists price. That was quelled with the
> explanation that essentially all Windows developers used the Visual
> Studio toolkit.
The difference here is that back then one had no other option than to
buy the compiler. Today, there _is_ a free Microsoft compiler
available, but it unfortunately is the wrong one.
> As to MinGW, nobody has signed up to commit long-term to doing the PyDev
> work that is required to get (and keep) it working. Such a developer
> would be welcome. There _are_ notes out there on the web to help you
> get such things going; I have done my own little bit to share what I
> know about how to do that.
As for what compiler vendor to use, I agree that it is the right
decision to use the Microsoft compiler on Microsoft platforms even if
it means buying it. GCC is not really an alternative on Windows.
Markus
Rob Williscroft
comp.lang.python:
>> As nikie pointed out, you can buy a 1-year MSDN Pro Subscription that
>> includes the VS2003 system. All that stopped is the free toolkit.
>
> The MSDN Pro Subscription is not really an option because we have no
> use for the 2005 compiler at all if it doesn't work with Python. I
> think we'll just try to get some boxed version of Visual Studio 2003
> Professional from somewhere.
This is the .NET 11 SDK, I belive it includes the 2003 compiler (*):
http://www.microsoft.com/downloads/thankyou.aspx?familyId=9b3a2ca6-3647-
4070-9f41-a333c6b9181d&displayLang=en&oRef=
I found the link with google: net 1.1 sdk download
its the first hit.
*) I'm not going to check as I have VS2003 installed already.
me...@mesw.de
Last time I checked the .NET SDK they had the C# compiler in there, but
not the C++ optimizing 2003 compiler. Might be wrong though....
"Martin v. Löwis"
> the problem is not the ABI, but the runtime libraries. From what you're
> saying, it looks like we will have to standardize on VS2003. As I said,
> we need to buy VS anyway because of the MFC support. On the other hand,
> I really worry about all those people that want to build open source
> extensions for Python 2.5. It is really possible that there will be no
> legal, free way to do that soon if you don't have an old installation
> of the 2003 toolkit lying around somewhere...
As others have pointed out already: This is not true. You can build
Python extensions with GCC just fine; gcc provides an import library
for msvcr71.dll.
It might be possible to integrate an msvcr71.dll import library
with VS 2005, in which case you could use VS 2005 to create Python
extensions as well.
Regards,
Martin
me...@mesw.de
thanks for the tip, I wasn't fully aware of that. OTOH, though GCC
might be a theoretical alternative, it isn't a practical one for many
situations:
* In a professional environment, it opens up another can of potential
problems, where one would rather like to stay with one single
compiler/build system.
* I suppose Python's distutils have to be tweaked to work with GCC
* The Makefiles/build system will need to be changed to work with the
GCC toolchain
* The different semantics of GCC and Microsoft C often need rewriting
some of the code
* There is no support for MFC/ATL in GCC
* The code created by the Windows GCC is not as good as the one created
by the Microsoft compiler
Markus
Antoon Pardon
What about new people who would be interested in building Window stuff
for Python? If I understand the situation correctly, people who
want to take up, writing C-extentions on windows now, may not be able to
do so, because they can't find a compatible compilor for the moment.
--
Antoon Pardon
robert
> Hi everyone,
>
> which compiler will Python 2.5 on Windows (Intel) be built with? I
> notice that Python 2.4 apparently has been built with the VS2003
> toolkit compiler, and I read a post from Scott David Daniels [1] where
> he said that probably the VS2003 toolkit will be used for Python 2.5
> again. However, even before the release of Python 2.5, I cannot seem to
> find many retailers around here that still carry Visual Studio 2003 ...
For me the great great problem with Python2.4's lib geometry was that
the size of distributable app installers swelled suddenly by many megs
with msvcr71.dll and mfc71 and codecs in core and all. Typically
installer sizes went from 1.5MB to over 4MB for basic non-trivial apps.
Thats a show stopper still in many situations today - at least for my
requirements.
See e.g. for an example of magnitudes:
http://groups.google.de/group/comp.lang.python/msg/edf469a1b3dc3802
Updating to a new expensive compiler problem for extensions was secondary.
Thus I decided so far to stay at Python2.3 for longer time for most
projects, while I'm using Python2.4+ only for local/web scripts and
single-installation projects. The little improvements in Python2.4/2.5
mostly don't justify their monster footprint in memory and installers.
Some questions:
* Is there a fundamental reason that the C-RTL of VC6 (which is
pre-installed on on all Windows today) is not sufficient for current
Python and extensions? instable?
In case not: As the short living VS 2003 compiler is now more rare than
the good old VC6, wouldn't it be better to switch back to VC6 for Py2.5
or at least to VC6 libs (which are maybe "free" of dev-license as they
sit on each Windows).
Maybe a suitable policy: the default crtl for Python should better be
the default library of the OS and not that of a random compiler which is
currently hip?
* can't the Mingw/gcc be used together with Windows default crt/mfc libs
for Python2.5 ? - Python getting away the from this MS studio (lib)
harassment?
( Personally I'd give no cent for that little runtime speed advantage by
the VS2005 compiler when comparing to a slimness + stable standard +
freedom )
Together with a clear decision to clean the Python core libs from recent
habits to "statically" preload OS-kind-of-packages (e.g. codecs, the
licentious pre-imports in urllib and friends ), I'd have hope to get
out of my deadlock on Python2.3.
* how many (serious) python users require to build distributable
installers (which have carry the python-rtls and non-default crtl's)?
I guess, almost all GUI apps have this requirement? And GUI apps
probably count more (also in line numbers) than web apps today as more
and more Delphi, BCPPB, Java, C++/MFC developers switch to Python?
-robert
sam
dll for Python 2.3, 2.4, and 2.5 without any problems. I just have to
make shure that I link with the correct Python.lib
Sam Schulenburg
Philip Austin
I just downloaded and installed this, and see a directory called
c:\program files\microsoft visual studio .net 2003\vc7
with bin\cl.exe and lib and include directories. So presumably
I'm good to go?
I'm following this thread because I'll need to
compile and install some extensions I've written for linux/gcc/python2.4
in our Windows computer lab. Presuming I succeed in setting up
vc7 correctly, is it as simple as 'python setup.py install' from here?
Thanks, Phil
Fredrik Lundh
> Presuming I succeed in setting up vc7 correctly, is it as simple
> as 'python setup.py install' from here?
yup (for a suitable definition of "correctly"; afaik, all you need is a
couple of registry settings; for a normal VC install, the easiest way to
get them in place is to start the visual studio application once).
</F>
"Martin v. Löwis"
> thanks for the tip, I wasn't fully aware of that. OTOH, though GCC
> might be a theoretical alternative, it isn't a practical one for many
> situations:
>
> * In a professional environment, it opens up another can of potential
> problems, where one would rather like to stay with one single
> compiler/build system.
That's a theoretic argument to me: Can you name four or five problems
out of that can?
> * I suppose Python's distutils have to be tweaked to work with GCC
No, it should work out of the box. If it doesn't: contributions are
welcome.
> * The Makefiles/build system will need to be changed to work with the
> GCC toolchain
If you are using distutils, you don't need a Makefile, and the
setup.py won't have to be tweaked. If you are not using distutils,
but, say, nmake already, then you will already have an earlier version
of the compiler - how else did you create the nmake files in the first
place?
> * The different semantics of GCC and Microsoft C often need rewriting
> some of the code
For C code, the semantic differences are really minor. You almost
never need any rewriting when moving from MS compilable code to
gcc; if you do, it is fairly straight forward to formulate the
code so that it compiles with both compilers
> * There is no support for MFC/ATL in GCC
That's true.
> * The code created by the Windows GCC is not as good as the one created
> by the Microsoft compiler
I never found that relevant for practical purposes. While it is possible
to create hand-crafted programs that show a difference, real-world
programs are often more limited by the performance of the OS (disk
IO, database IO, GUI) than raw processor speed.
Still, if you really cannot use gcc, then go ahead and compile with
VS 2005. Just make sure you link with mscvr71.dll. If you absolutely
need that to work, you will find a way to make it work.
Regards,
Martin
"Martin v. Löwis"
> For me the great great problem with Python2.4's lib geometry was that
> the size of distributable app installers swelled suddenly by many megs
> with msvcr71.dll and mfc71 and codecs in core and all.
codecs are in python24.dll, mscvr71, mfc71 and all are not.
However, they are not in core - the operating system demand-pages code,
loading into core memory only what is being used. So if you don't use
the codecs, they are not loaded into core.
While mscvr71 is likely loaded into core (even though it is not in
python24.dll), mfc71.dll does not play a role at all in Python 2.4.
Python is written entire in C, not in C++.
> * Is there a fundamental reason that the C-RTL of VC6 (which is
> pre-installed on on all Windows today) is not sufficient for current
> Python and extensions? instable?
The compiler that is used (VS 2003) won't ship it, and links with
msvcr71.dll. That's why that library version is used.
> In case not: As the short living VS 2003 compiler is now more rare than
> the good old VC6, wouldn't it be better to switch back to VC6 for Py2.5
> or at least to VC6 libs (which are maybe "free" of dev-license as they
> sit on each Windows).
Better in what respect? Predictability of software development?
Certainly not: the development will be more predictable if a decision
that was once taken is implemented as promised.
Making more users happy? Certainly not, either. Some users request that
VS2005 is being used, not that VC6 is being used. Other users request
that Python 2.5 continues to be built with VS 2003. You can't please
everybody.
> Maybe a suitable policy: the default crtl for Python should better be
> the default library of the OS and not that of a random compiler which is
> currently hip?
That would make a good policy if the OS had a system C library. Windows
doesn't. Windows NT does have a CRT, namely crtdll.dll - but you weren't
suggesting to use that one, were you?
> * can't the Mingw/gcc be used together with Windows default crt/mfc libs
> for Python2.5 ? - Python getting away the from this MS studio (lib)
> harassment?
Sure, you can use gcc/mingw to build extensions for Python 2.4 and
Python 2.5.
> * how many (serious) python users require to build distributable
> installers (which have carry the python-rtls and non-default crtl's)?
Apparently not many. I repeatedly asked for contribution of a
specification how pythonxy.dll should be modularized, and never
received one.
Regards,
Martin
robert
> robert wrote:
>
> codecs are in python24.dll, mscvr71, mfc71 and all are not.
> However, they are not in core - the operating system demand-pages code,
> loading into core memory only what is being used. So if you don't use
> the codecs, they are not loaded into core.
>
> While mscvr71 is likely loaded into core (even though it is not in
> python24.dll), mfc71.dll does not play a role at all in Python 2.4.
> Python is written entire in C, not in C++.
yes, yet as a consequence win32xx is forced to be linked against
MFC71.dll (no other functional reason): another MegaByte which I'd need
to package most times - while mfc42 from VC6 is pre-installed on each
Windows.
>
>>* Is there a fundamental reason that the C-RTL of VC6 (which is
>>pre-installed on on all Windows today) is not sufficient for current
>>Python and extensions? instable?
>
>
> The compiler that is used (VS 2003) won't ship it, and links with
> msvcr71.dll. That's why that library version is used.
hmm, yet msvcrt4 is obviously preinstalled on each Windows - and its in
Windows Update Process. Its tagged: "4.20 - OS use only. DO NOT
DISTRIBUTE")
Think, in principle its possible to compile against that with
VS2003/2005... ?
( think msvcrt4 is not delivered extra even in the python2.3 installer )
>>In case not: As the short living VS 2003 compiler is now more rare than
>>the good old VC6, wouldn't it be better to switch back to VC6 for Py2.5
>>or at least to VC6 libs (which are maybe "free" of dev-license as they
>>sit on each Windows).
>
> Better in what respect? Predictability of software development?
> Certainly not: the development will be more predictable if a decision
> that was once taken is implemented as promised.
2.4 and 2.5 compiled stuff is anyway not interchangeable - thus at
alpha-stage a (down) step to a common system lib would probably have no
countable negative effects at all.
> Making more users happy? Certainly not, either. Some users request that
> VS2005 is being used, not that VC6 is being used. Other users request
> that Python 2.5 continues to be built with VS 2003. You can't please
> everybody.
Yet if the C runtime lib (distribution) problem is solved, the question
of the compiler becomes irrelevant. Everybody could use his compiler
without worry - a soft recommendation could be in the Python2.5 doc's to
link extensions also against the same common basic ctrl in order to keep
the numer of libs small.
>>Maybe a suitable policy: the default crtl for Python should better be
>>the default library of the OS and not that of a random compiler which is
>>currently hip?
>
> That would make a good policy if the OS had a system C library. Windows
> doesn't. Windows NT does have a CRT, namely crtdll.dll - but you weren't
> suggesting to use that one, were you?
msvcrt4 is probably a reliable system C lib and still used by most apps.
Think it doesn't lack Pythons needs.
there is also a "msvcrt.dll" - also updated by the Windows system. maybe
its on each Windows ? On XP it has version 7. What is this?
>>* can't the Mingw/gcc be used together with Windows default crt/mfc libs
>>for Python2.5 ? - Python getting away the from this MS studio (lib)
>>harassment?
>
> Sure, you can use gcc/mingw to build extensions for Python 2.4 and
> Python 2.5.
yes, the lib is 90% the object of concern.
>
>>* how many (serious) python users require to build distributable
>>installers (which have carry the python-rtls and non-default crtl's)?
>
>
> Apparently not many. I repeatedly asked for contribution of a
That would be very intersting for me - number about where (project type,
LOC, #-of-users) and how (system, distribution, ..) python is used.
Is there already something. Maybe a research/poll in this NG would be
feasible.
> specification how pythonxy.dll should be modularized, and never
> received one.
Maybe most a most simple cutoff criteria with "costs" does it magically.
Basic proposal:
cost = (C1 * module-size - C2 * frequency-of-module-usage)
I'd suggest:
cutoff cost = 0
C1 = 1/200kB
C2 = 1/80%
Example CJK codecs:
cost = 500k/200k - 30%/80% = 2.13 => too expensive by far
Example zlib:
cost = 70k/200k - 95%/80% = -0.84 => go in
-robert
Roel Schroeven
> * The code created by the Windows GCC is not as good as the one created
> by the Microsoft compiler
Isn't Python for other platforms built with GCC? Seems to me that if it
GCC is good enough for other platforms, it's good enough for Windows.
--
If I have been able to see further, it was only because I stood
on the shoulders of giants. -- Isaac Newton
Roel Schroeven
"Martin v. Löwis"
> hmm, yet msvcrt4 is obviously preinstalled on each Windows - and its in
> Windows Update Process. Its tagged: "4.20 - OS use only. DO NOT
> DISTRIBUTE")
> Think, in principle its possible to compile against that with
> VS2003/2005... ?
> ( think msvcrt4 is not delivered extra even in the python2.3 installer )
Unfortunately, you also need the header files and an import library
to link against this library. Likewise, for applications that bundle
MFC, you need the corresponding set of header files and libraries.
Microsoft does not provide import libraries for msvcrt4.dll anymore.
>> Making more users happy? Certainly not, either. Some users request that
>> VS2005 is being used, not that VC6 is being used. Other users request
>> that Python 2.5 continues to be built with VS 2003. You can't please
>> everybody.
>
> Yet if the C runtime lib (distribution) problem is solved, the question
> of the compiler becomes irrelevant. Everybody could use his compiler
> without worry - a soft recommendation could be in the Python2.5 doc's to
> link extensions also against the same common basic ctrl in order to keep
> the numer of libs small.
Unfortunately, that is a real technological challenge. Where do you get
the header files and import libraries, and how do you explain to your
compiler to use those instead of the ones it comes with?
> there is also a "msvcrt.dll" - also updated by the Windows system. maybe
> its on each Windows ? On XP it has version 7. What is this?
It used to be possible to link with it. See
http://msdn2.microsoft.com/en-us/library/abx4dbyh(VS.80).aspx
This is now a "known DLL", and meant for use by system-level components
only.
> Maybe most a most simple cutoff criteria with "costs" does it magically.
> Basic proposal:
>
> cost = (C1 * module-size - C2 * frequency-of-module-usage)
Unfortunately, that criterion cannot be determined in an objective
manner. It's not possible to determine frequency-of-module-usage
in any meaningful way for existing modules, let alone for modules
that are contributed and to be included only in the upcoming
release.
Regards,
Martin
me...@mesw.de
Martin v. Löwis wrote:
> > * In a professional environment, it opens up another can of potential
> > problems, where one would rather like to stay with one single
> > compiler/build system.
> That's a theoretic argument to me: Can you name four or five problems
> out of that can?
In bigger (i.e. real-world) projects, it can be a lot of hassle
switching to another compiler, even from the same vendor. Of course, in
theory C and C++ compilers should be compatible, but in practice
there's lots of small things that are implemented differently. Among
them are #pragmas, the support for precompiled headers, template
instantiation, different semantics for "half-legal" constructs etc.
Also, some syntactic constructs which compile on one compiler are
rejected by another compiler. Further, often code implicitely depends
on the behaviour of one single compiler, although the developer doesn't
really know that. Of course you can cry fool on the developer that
wrote the code and blame him for not following the C-whatever spec, or
not using the "correct" way to do X, but fact is, there's always stuff
that will not work when switching to another compiler. And there's a
bunch of other stuff that will appear to work but "activate" bugs in
the control structure that were there previously but luckily did not
manifest with the other compiler.
Btw, the same goes for different versions of libraries, say different
versions of wxWidgets, MFC, Visual Basic (anyone tried to switch from
VB6 to VB.NET?) and of course also for different versions of Java and
its libraries.
> > * The Makefiles/build system will need to be changed to work with the
> > GCC toolchain
>
> If you are using distutils, you don't need a Makefile, and the
> setup.py won't have to be tweaked. If you are not using distutils,
> but, say, nmake already, then you will already have an earlier version
> of the compiler - how else did you create the nmake files in the first
> place?
We have lots of stuff written in VC++6 which is not
distutils-compatible, and we can't really switch that code over to GCC
anytime soon, see above. All in all, we would end up compiling
something with VC, linking it with GCC to another VC app (Python). No,
thanks.
> Still, if you really cannot use gcc, then go ahead and compile with
> VS 2005. Just make sure you link with mscvr71.dll. If you absolutely
> need that to work, you will find a way to make it work.
The question is, is it cheaper and more hassle-free to spend the time
to "find a way to make it work" and hope everything goes smoothly
(remember, the fact, that it says "Linker results: 0 errors / 0
warnings" does not mean that the app will work as expected and as
tested before) or to buy VC (which costs a mere few hundred dollars).
Bottom Line: As I said before, I don't have a problem using VC2003 or
anything. It's by far the cheapest and easiest way just to buy VC2003
and be done with it, than to fiddle around with GCC or anything. I just
think that Python should use the best technology available at the time
of release, which is VC2005 on Windows. But as I indicated before, of
course I do understand the argument that the release cycle has already
been planned and should not be changed, so we'll just live with it as
it is now.
Markus
Scott David Daniels
> Isn't Python for other platforms built with GCC? Seems to me that if it
> GCC is good enough for other platforms, it's good enough for Windows.
You clearly misunderstand the interface to the Windows OS & GUI system.
Microsoft provides that interface through its language systems, and has
used that edge to knock off other compilers (such as WatCom) by
providing header files that are not standard C compatible.
--Scott David Daniels
scott....@acm.org
Roel Schroeven
> Roel Schroeven wrote:
>> Isn't Python for other platforms built with GCC? Seems to me that if it
>> GCC is good enough for other platforms, it's good enough for Windows.
>
> You clearly misunderstand the interface to the Windows OS & GUI system.
Very well possible.
> Microsoft provides that interface through its language systems, and has
> used that edge to knock off other compilers (such as WatCom) by
> providing header files that are not standard C compatible.
It seems that many people successfully use mingw. At work I use a
Borland compiler all the time, without compatibility problems with
Windows' APIs.
But what I meant was not related to interface compatibilities, but to
the performance of the generated code: many people say that the code
generated by gcc is not as well optimized as code generated by
Microsoft's compilers. Yet I never hear complaints about that from
people who use Python on e.g. Linux, where Python is compiled with gcc.
So, while there might be valid reasons not to use mingw, as far as I can
see the optimization capabilities of the compiler are not a compelling
reason.
"Martin v. Löwis"
> Bottom Line: As I said before, I don't have a problem using VC2003 or
> anything. It's by far the cheapest and easiest way just to buy VC2003
> and be done with it, than to fiddle around with GCC or anything. I just
> think that Python should use the best technology available at the time
> of release, which is VC2005 on Windows. But as I indicated before, of
> course I do understand the argument that the release cycle has already
> been planned and should not be changed, so we'll just live with it as
> it is now.
Thanks for the confirmation; that's good enough for me.
Regards,
Martin
Scott David Daniels
> ... But what I meant was not related to interface compatibilities, but to
> generated by gcc is not as well optimized as code generated by
> Microsoft's compilers. Yet I never hear complaints about that from
> people who use Python on e.g. Linux, where Python is compiled with gcc.
> So, while there might be valid reasons not to use mingw, as far as I can
> see the optimization capabilities of the compiler are not a compelling
> reason.
I musunderstood you. I thought you were advocating that Python itself
be built on gcc, obviating many compiler access issues. That wouldn't
work because gcc cannot, by itself (as I understand it) get to all the
nooks and crannies a windows developer may need to traverse. I know I
just repeated my argument here against a strawman, but that was really
for other readers, not for you.
--Scott David Daniels
scott....@acm.org
"Martin v. Löwis"
> I musunderstood you. I thought you were advocating that Python itself
> be built on gcc, obviating many compiler access issues. That wouldn't
> work because gcc cannot, by itself (as I understand it) get to all the
> nooks and crannies a windows developer may need to traverse. I know I
> just repeated my argument here against a strawman, but that was really
> for other readers, not for you.
Even that is incorrect. Python 2.x can be built fully with gcc. It's
PythonWin that includes a lot of C++ and MFC code that won't compile
with gcc. That doesn't preclude to use gcc for Python 2.x compilation,
as long as a msvcrt is selected that can work together with some
version of MFC and the MS C++ RTL.
Regards,
Martin
Roel Schroeven
> I musunderstood you. I thought you were advocating that Python itself
> be built on gcc, obviating many compiler access issues. That wouldn't
> work because gcc cannot, by itself (as I understand it) get to all the
> nooks and crannies a windows developer may need to traverse. I know I
> just repeated my argument here against a strawman, but that was really
> for other readers, not for you.
I'm not actively advocating it since I realize I don't know enough about
all the pros and cons, but yes, I would like for Python and other open
source projects to use gcc even on Windows.
It gives me an uneasy feeling when you can't use the source (apart from
just reading it) of open source projects without depending on the whims
of a third party. As an example, look what happened to the Linux kernel
and Bitkeeper. One might argue that Microsoft is not really a third
party since the whole Windows platform is made by them, but the problems
are the same: as far as I understand, Visual Studio Express 2003 is no
longer available, and the 2005 version is not binary compatible.
In my case, I'm even unable to uninstall any modern Microsoft compiler
since AFAICT they all require XP SP2 and I'm stuck with SP1 on my laptop
since SP2 conflicts with the touchpad driver resulting in Windows
blocking on booting, but that only effects me (I hope) and is something
for a whole other rant.
me...@mesw.de
> me...@mesw.de schreef:
> > * The code created by the Windows GCC is not as good as the one created
> > by the Microsoft compiler
>
> Isn't Python for other platforms built with GCC? Seems to me that if it
> GCC is good enough for other platforms, it's good enough for Windows.
Actually, GCC on Linux does provide worse machine code than say, the
Intel compiler for Linux. This is not a "bug" of GCC, but a feature,
because GCC was created for maximum portability and flexibility, not
for maximum performance on one single platform like the Intel compiler.
But GCC is truly free and can be distributed easily (because it's GPL),
so it is the default compiler for most (if not all) distros at the
moment. Also, there are many free software programs that use special
features of GCC and cannot be compiled using another compiler.
Of course I do acknowledge that often in the real world the IO forms
the bottleneck of a system and the efficiency of the machine code is
somewhat neglegible (other posters pointed that out already).
Markus
Scott David Daniels
> .... Also, there are many free software programs that use special
> features of GCC and cannot be compiled using another compiler.
lock-in is lock-in whether you must pay cash or not.
--Scott David Daniels
scott....@acm.org