[Python-Dev] externals?

[Kristján Valur Jónsson]

This may be a silly question, but haven‘t the python externals been moved to HG yet?

I usually work on cpython without bothering with the externals, but I found today that I needed them.  On Windows this is a bit of a bother.  And I‘ve thrown away all my SVN stuff...

K

[Benjamin Peterson]

Their still in svn as far I know.

2012/11/13 Kristján Valur Jónsson <kris...@ccpgames.com>:

> _______________________________________________
> Python-Dev mailing list
> Pytho...@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/benjamin%40python.org
>

--
Regards,
Benjamin
_______________________________________________
Python-Dev mailing list
Pytho...@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/dev-python%2Bgarchive-30976%40googlegroups.com

[Benjamin Peterson]

2012/11/13 Benjamin Peterson <benj...@python.org>:

> Their still in svn as far I know.

s/Their/They're

[Kristján Valur Jónsson]

Okay, that means I need to re-install svn, cool.
But I should mention that this needs to be mentioned in the core development FAQs, setting up and so forth.
There is no mention of it there. Perhaps the unix makefiles get the proper version, but a windows developer has to fetch those externals manually.

Also, is there any reason to keep this in svn? Why not check this in to HG, we need not worry about history, etc.

K

[Trent Nelson]

On Thu, Nov 15, 2012 at 01:20:09AM -0800, Kristj?n Valur J?nsson wrote:
> Perhaps the unix makefiles get the proper version, but a windows developer has to fetch those externals manually.

Pro-tip: if you're developing on Windows, you're mad if you don't
prime your dev env with Tools\buildbot\externals.bat. It takes
care of *everything*. I wish every proprietary UNIX system we
support had something similar.

> Also, is there any reason to keep this in svn?

I think it's more a case of there being no tangible benefit (and
numerous drawbacks) to switching it to hg. I personally have no
need for a local hg repo with 30 different Tcl/Tk histories in
it.

Subversion's good for this sort of use case. The externals repo
gets committed to maybe, what, 3-4 times a year?

> Why not check this in to HG, we need not worry about history, etc.

Who are these mystical people worried about history? ;-)

Trent.

[Kristján Valur Jónsson]

Thanks for your pro-tip. Might I suggest that it ought to go into the dev FAQ? Along with an explanation that a windows dev has to have SVN installed too, just for the laughs?
I think there might be a benefit to moving at least the current externals to a separate HG repository. We could easily have multiple branches in that repo reflecting the required externals for each version under active HG development).
There is an inherent drawback in having to rely on two different RCS to fetch the necessary stuff, imho.
K

-----Original Message-----
From: Trent Nelson [mailto:tr...@snakebite.org]
Sent: 16. nóvember 2012 12:13
To: Kristján Valur Jónsson
Cc: Benjamin Peterson; Python-Dev (pytho...@python.org)
Subject: Re: [Python-Dev] externals?

[mar...@v.loewis.de]

Zitat von Kristján Valur Jónsson <kris...@ccpgames.com>:

> Thanks for your pro-tip. Might I suggest that it ought to go into
> the dev FAQ? Along with an explanation that a windows dev has to
> have SVN installed too, just for the laughs?
> I think there might be a benefit to moving at least the current
> externals to a separate HG repository. We could easily have
> multiple branches in that repo reflecting the required externals for
> each version under active HG development).

Feel free to work on this. Consider that using hg may significantly
increase the amount of network traffic, since the repo(s) will contain
multiple versions, when only one specific version is needed.

When working on this, try to come up with other automated download
procedures, e.g. ones that do not involve revision control and have
support built into Windows.

Regards,
Martin

[Georg Brandl]

Am 17.11.2012 23:47, schrieb mar...@v.loewis.de:
>
> Zitat von Kristján Valur Jónsson <kris...@ccpgames.com>:
>
>> Thanks for your pro-tip. Might I suggest that it ought to go into
>> the dev FAQ? Along with an explanation that a windows dev has to
>> have SVN installed too, just for the laughs?
>> I think there might be a benefit to moving at least the current
>> externals to a separate HG repository. We could easily have
>> multiple branches in that repo reflecting the required externals for
>> each version under active HG development).
>
> Feel free to work on this. Consider that using hg may significantly
> increase the amount of network traffic, since the repo(s) will contain
> multiple versions, when only one specific version is needed.
>
> When working on this, try to come up with other automated download
> procedures, e.g. ones that do not involve revision control and have
> support built into Windows.

One way would be to use one hg repo per version, and (maybe, if needed)
a master repo that has them as subrepos. Whoever needs externals can
download the repo as a zipfile and unpack it (both of which is possible
with batteries only).

Note that in this scenario, hg is used mostly in order to avoid another
service (such as rsync), and for developer convenience.

Georg

[Armin Rigo]

Hi,

On Sun, Nov 18, 2012 at 8:22 AM, Georg Brandl <g.br...@gmx.net> wrote:
> One way would be to use one hg repo per version, and (maybe, if needed)
> a master repo that has them as subrepos.

Or have all versions in the same repo as usual (with branches), but
have hg subrepos point to different repos: ones extracted from the
main repo by containing only the correct branch. But it might be a
bit delicate to pull this off. (hg clone takes a "-r" option and
copies only things needed for the given revision or branch, but
apparently we can't pass this option automatically to the cloning of
subrepos. (Maybe it points out that subrepos are a hack best done
without altogether, which is what we did in pypy.))


A bientôt,

Armin.

[Georg Brandl]

Am 18.11.2012 10:00, schrieb Armin Rigo:
> Hi,
>
> On Sun, Nov 18, 2012 at 8:22 AM, Georg Brandl <g.br...@gmx.net> wrote:
>> One way would be to use one hg repo per version, and (maybe, if needed)
>> a master repo that has them as subrepos.
>
> Or have all versions in the same repo as usual (with branches), but
> have hg subrepos point to different repos: ones extracted from the
> main repo by containing only the correct branch. But it might be a
> bit delicate to pull this off. (hg clone takes a "-r" option and
> copies only things needed for the given revision or branch, but
> apparently we can't pass this option automatically to the cloning of
> subrepos. (Maybe it points out that subrepos are a hack best done
> without altogether, which is what we did in pypy.))

Yep. Anyway, if every external version goes into a branch, then we don't
need subrepos anyway. That is a better idea than mine.

Since you can use (e.g.) "hg clone -r tk-8.5" or download a tarball
specific to a branch, nobody should need to get the whole externals
history on clone.

Georg

[mar...@v.loewis.de]

Zitat von Armin Rigo <ar...@tunes.org>:

> Hi,
>
> On Sun, Nov 18, 2012 at 8:22 AM, Georg Brandl <g.br...@gmx.net> wrote:
>> One way would be to use one hg repo per version, and (maybe, if needed)
>> a master repo that has them as subrepos.
>
> Or have all versions in the same repo as usual (with branches), but
> have hg subrepos point to different repos: ones extracted from the
> main repo by containing only the correct branch. But it might be a
> bit delicate to pull this off. (hg clone takes a "-r" option and
> copies only things needed for the given revision or branch, but
> apparently we can't pass this option automatically to the cloning of
> subrepos. (Maybe it points out that subrepos are a hack best done
> without altogether, which is what we did in pypy.))

I'd like to stress that we don't need any versioning here. wget and
tar would be sufficient, except that it's Windows, so we have neither
wget nor tar. However, including a PowerShell script may be an option;
most developers will have PowerShell already on their system. AFAICT,
PowerShell can do HTTP downloads and extract zip files.

Regards,
Martin

[Brian Curtin]

On Sun, Nov 18, 2012 at 6:18 AM, <mar...@v.loewis.de> wrote:
>
> Zitat von Armin Rigo <ar...@tunes.org>:
>
>
>> Hi,
>>
>> On Sun, Nov 18, 2012 at 8:22 AM, Georg Brandl <g.br...@gmx.net> wrote:
>>>
>>> One way would be to use one hg repo per version, and (maybe, if needed)
>>> a master repo that has them as subrepos.
>>
>>
>> Or have all versions in the same repo as usual (with branches), but
>> have hg subrepos point to different repos: ones extracted from the
>> main repo by containing only the correct branch. But it might be a
>> bit delicate to pull this off. (hg clone takes a "-r" option and
>> copies only things needed for the given revision or branch, but
>> apparently we can't pass this option automatically to the cloning of
>> subrepos. (Maybe it points out that subrepos are a hack best done
>> without altogether, which is what we did in pypy.))
>
>
> I'd like to stress that we don't need any versioning here. wget and
> tar would be sufficient, except that it's Windows, so we have neither
> wget nor tar. However, including a PowerShell script may be an option;
> most developers will have PowerShell already on their system. AFAICT,
> PowerShell can do HTTP downloads and extract zip files.

I would hope we can just write a simple Python script to do this,
rather than require PowerShell.

I'm 99.99999% certain anyone building Python on Windows will already
have a version of Python installed. Plus, they're going to need it
anyway to build OpenSSL (see PCbuild/build_ssl.py and the references
to it in VS projects).

[Terry Reedy]

On 11/18/2012 12:05 PM, Brian Curtin wrote:
> On Sun, Nov 18, 2012 at 6:18 AM, <mar...@v.loewis.de> wrote:
>>
>> Zitat von Armin Rigo <ar...@tunes.org>:

>>> Or have all versions in the same repo as usual (with branches), but
>>> have hg subrepos point to different repos: ones extracted from the
>>> main repo by containing only the correct branch. But it might be a
>>> bit delicate to pull this off. (hg clone takes a "-r" option and
>>> copies only things needed for the given revision or branch, but
>>> apparently we can't pass this option automatically to the cloning of
>>> subrepos. (Maybe it points out that subrepos are a hack best done
>>> without altogether, which is what we did in pypy.))
>>
>>
>> I'd like to stress that we don't need any versioning here. wget and
>> tar would be sufficient, except that it's Windows, so we have neither
>> wget nor tar. However, including a PowerShell script may be an option;
>> most developers will have PowerShell already on their system. AFAICT,
>> PowerShell can do HTTP downloads and extract zip files.
>
> I would hope we can just write a simple Python script to do this,
> rather than require PowerShell.
>
> I'm 99.99999% certain anyone building Python on Windows will already
> have a version of Python installed. Plus, they're going to need it
> anyway to build OpenSSL (see PCbuild/build_ssl.py and the references
> to it in VS projects).

After reading the thread, I realize that I do not actually want externam
dependency files moved to hg. I and others are not going to push changes
back, so we do not need hg clones.

What would be good would to be able to access the files and use them to
build python without svn installed. I don't know the best way to do
that, but if tarred or zipped releases were made for each version that
should be downloaded, our urllib, tarfile/ziplib, and any other modules
needed should be sufficient to transfer and unpack.

--
Terry Jan Reedy

[Kristján Valur Jónsson]

But that's what hg clone does.
You have a lorry for your work at the mine. You don't need a Mini to go to the fishmongers. You can use your lorry even if you are not going to dump a tonne of ore on the pavement.
K

> -----Original Message-----
>
> What would be good would to be able to access the files and use them to
> build python without svn installed. I don't know the best way to do that, but
> if tarred or zipped releases were made for each version that should be
> downloaded, our urllib, tarfile/ziplib, and any other modules needed should
> be sufficient to transfer and unpack.

[Trent Nelson]

On Sun, Nov 18, 2012 at 04:18:43AM -0800, mar...@v.loewis.de wrote:
> I'd like to stress that we don't need any versioning here. wget and
> tar would be sufficient, except that it's Windows, so we have neither
> wget nor tar. However, including a PowerShell script may be an option;
> most developers will have PowerShell already on their system. AFAICT,
> PowerShell can do HTTP downloads and extract zip files.

I'm disturbed that I subconsciously interpreted this as a challenge
to do it via a standalone batch/.bat ;-)

Trent.