Compatibility tags for pure-python wheels

73 views
Skip to first unread message

Paul Moore

unread,
Mar 19, 2014, 11:32:18 AM3/19/14
to pypa-dev
I'd like to make some changes to how wheels handle pure-python
compatibility tags. I'm not intending to change the meaning of the
tags, just clarify things, although I do want to suggest adding the
ability for the user to specify the tag that gets used. I'm *not*
suggesting any changes to architecture-specific tags here, this is
solely about pyXY-none-any tags.

PEP 425 defines the tag by saying "The Python tag indicates the
implementation and version required by a distribution". In fact, the
tag indicates the *minimum* minor version of Python that the
distribution (wheel) supports for the "generic" tag (pyXY). This is
because the standard compatibility checks accept (for example) py32 as
compatible with Python 3.3. I think the PEP wording should make this
explicit - but having said that, I don't know if doing so would need a
new PEP version - if so, it's probably not a big enough deal to worry
about for this revision of the PEP.

However, the fact that pyXY is actually a minimum supported version
*is* important, in terms of how projects should tag their wheels. At
the moment, bdist_wheel uses the version of Python that was used to
build the wheel as the tag. While that's a sensible guess, it isn't
necessarily right for projects that support older versions (I
typically develop pip using Python 3.3, or 3.4 now, and yet pip
supports 3.2).

What I'd like to do is to add the ability for projects to specify the
minimum version explicitly, either on the command line, or in
setup.cfg. Originally, I was also going to propose that we make the
default tag be either py2 or py3 (depending on which major version was
used to build the wheel) but this may be going too far - a project
built solely using Python 3.3 has probably never even been tested
under 3.2, so a default py3 tag is optimistic at best.

I have a pull request for wheel
(https://bitbucket.org/pypa/wheel/pull-request/39) that implements
this, changing the default behaviour.

So, questions:

1. Do people agree in principle with characterising pyXY as "the
minimum supported minor version"?
2. Is it worth changing the PEP wording to match this?
3. Is my PR a reasonable approach for this?
4. What do people think about the current default value for the tag?
Is it OK, or would py2 or py3 (depending on the interpreter used to
build) be better? Or maybe we should warn if the user doesn't specify
a tag?
5. Is anybody actually still reading this? :-)

Note, by the way, that --universal covers the py2.py3 case, that stays the same.

Thanks,
Paul

Donald Stufft

unread,
Mar 19, 2014, 11:46:20 AM3/19/14
to Paul Moore, pypa-dev
How would you specify that a Wheel is only viable on Python 3.2 and not
Python 3.3? Although maybe for a pure Python Wheel that’s not very
likely and it doesn’t actually matter. Although it does make the universal
flag somewhat misleading/wrong? I doubt there’s *any* project
that works in every Python 2.x version and every 3.x version, so more likely
if we’re interpreting these as minimum versions and not exact versions
then probably no project should ever be a universal Wheel and should
instead be something like py26.py32 (for example, pip 1.6).

> 2. Is it worth changing the PEP wording to match this?

Probably, we need to make a new PEP anyways to add compatibility
tags for Linux stuff so we could roll it into that.

> 3. Is my PR a reasonable approach for this?

I didn’t look at the PR yet.

> 4. What do people think about the current default value for the tag?
> Is it OK, or would py2 or py3 (depending on the interpreter used to
> build) be better? Or maybe we should warn if the user doesn't specify
> a tag?

This is hard, because as I said above if we’re defining it to be minimum
version than py2 or py3 is probably always wrong for most Wheels and
in general isn’t very useful and would be more explicitly py30/py20.

> 5. Is anybody actually still reading this? :-)

Yes!

>
> Note, by the way, that --universal covers the py2.py3 case, that stays the same.
>
> Thanks,
> Paul


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

signature.asc

Paul Moore

unread,
Mar 19, 2014, 12:03:59 PM3/19/14
to Donald Stufft, pypa-dev
On 19 March 2014 15:46, Donald Stufft <don...@stufft.io> wrote:
> How would you specify that a Wheel is only viable on Python 3.2 and not
> Python 3.3?

You can't. The way the tags are checked simply doesn't allow that -
Python 3.3 accepts a py32 tag. (This is why I was careful to say I
didn't want to change how tags were checked - this is arguably the
wrong way to do it, Daniel and I had a bit of a debate over it when he
first implemented it, but changing it would be backward incompatible
and I didn't want to go there at this point).

The relevant code is at
https://github.com/pypa/pip/blob/develop/pip/pep425tags.py#L51

> Although maybe for a pure Python Wheel that's not very
> likely and it doesn't actually matter. Although it does make the universal
> flag somewhat misleading/wrong? I doubt there's *any* project
> that works in every Python 2.x version and every 3.x version, so more likely
> if we're interpreting these as minimum versions and not exact versions
> then probably no project should ever be a universal Wheel and should
> instead be something like py26.py32 (for example, pip 1.6).

The principle is not so much that the code will work, but rather that
this wheel is the one you should choose because you'll get the same
result as you get with a sdist install. Whether the code uses features
that aren't available in a particular version is more about the
metadata (the requires-python metadata might have something to say
here, as and when checking that gets implemented).

So, the py2 vs py3 distinction is important, because of 2to3. But
pretty much any minor-version tag is irrelevant, because if there's no
tag for that minor version, we'll just fall back to the sdist, which
will work the same, just a bit slower. By this line of argument, I
could easily argue that we only really need the two tags py2 and py3.
But I won't go that far :-)

It does mean that I'm more in favour of defaulting to the appropriate
one of py2 or py3, on the basis that it's never going to be any worse
than using the sdist.

>> 2. Is it worth changing the PEP wording to match this?
>
> Probably, we need to make a new PEP anyways to add compatibility
> tags for Linux stuff so we could roll it into that.

That was what I meant by a new version of the PEP. I'm not sure if
clarifying the wording without a version bump to the spec is allowed,
and I don't think this is important enough to try to make an
exception.

>> 5. Is anybody actually still reading this? :-)
>
> Yes!

Phew!

Paul

Donald Stufft

unread,
Mar 19, 2014, 12:53:33 PM3/19/14
to Paul Moore, pypa-dev

On Mar 19, 2014, at 12:03 PM, Paul Moore <p.f....@gmail.com> wrote:

> On 19 March 2014 15:46, Donald Stufft <don...@stufft.io> wrote:
>> How would you specify that a Wheel is only viable on Python 3.2 and not
>> Python 3.3?
>
> You can't. The way the tags are checked simply doesn't allow that -
> Python 3.3 accepts a py32 tag. (This is why I was careful to say I
> didn't want to change how tags were checked - this is arguably the
> wrong way to do it, Daniel and I had a bit of a debate over it when he
> first implemented it, but changing it would be backward incompatible
> and I didn't want to go there at this point).
>
> The relevant code is at
> https://github.com/pypa/pip/blob/develop/pip/pep425tags.py#L51

It would be backwards incompatible to the implementation, but not to the
Spec. I do think either the spec needs to be updated or the code needs
to be updated. Either way I don’t think there is going to be many (any?)
one relying not this currently. To me it feels like smell to have both
py30 and py3 mean the same thing *and* it feels surprising to me (having
read the spec multiple times but not the implementation).

>
>> Although maybe for a pure Python Wheel that's not very
>> likely and it doesn't actually matter. Although it does make the universal
>> flag somewhat misleading/wrong? I doubt there's *any* project
>> that works in every Python 2.x version and every 3.x version, so more likely
>> if we're interpreting these as minimum versions and not exact versions
>> then probably no project should ever be a universal Wheel and should
>> instead be something like py26.py32 (for example, pip 1.6).
>
> The principle is not so much that the code will work, but rather that
> this wheel is the one you should choose because you'll get the same
> result as you get with a sdist install. Whether the code uses features
> that aren't available in a particular version is more about the
> metadata (the requires-python metadata might have something to say
> here, as and when checking that gets implemented).
>
> So, the py2 vs py3 distinction is important, because of 2to3. But
> pretty much any minor-version tag is irrelevant, because if there's no
> tag for that minor version, we'll just fall back to the sdist, which
> will work the same, just a bit slower. By this line of argument, I
> could easily argue that we only really need the two tags py2 and py3.
> But I won't go that far :-)

That’s not exactly the case, I’ve put errors in setup.py’s that raised an
error rather than installed on Pythons that I knew didn’t work.

I think if we take the stance that the code is correct and the spec is wrong
then py2 and py3 should be deprecated in favor of py30 and py20. Those
are more accurately describing what those tags *actually* mean and are
going to be less confusing than having py2, py26, py27, py3, py32, py33.

I think most reasonable people, if you give them a set of tags that includes
py3, py30, py31, py32, py33, py34 are going to conclude that py3 spans
all of Python3 while py3N are for that specific version.

So IMO we need to either:

Decide the Spec is accurate and the way pip acts is currently a bug and fix
it. This means most Wheels will want to use py2 or py3 and not pyXY because
otherwise you need to upload a pyXY for each supported version. The downside
to py2/py3 is that you claim support for all of Python 2 or all of Python 3 when
that is most often not the case.

OR

Decide pip’s implementation is accurate and the way the spec acts currently
is a bug and fix it. This should also include deprecating py2 and py3 and
making them aliases for py20 and py30 and making the —universal flag
use py20 and py30. This makes it impossible to specify that this wheel works
*only* with a particular version of Python but it makes Wheels more convenient
because you can upload one Wheel to cover multiple Python versions without
also covering versions you explicitly don’t support[1]


[1] In this case it would fall back to sdist which often times will work, but also
might contain a check to prevent installation on older Pythons. Also maybe
in the future sdists could support these same semantics!

Another tricky thing to decide here is what this means for cpXY and ppXY
and the other implementation specific tags. If we go with the more convenient
reading of pyXY (#2 above) and we apply that to that to the implementation
defined tags, then we’re going to have a holy mess on our hands because
currently that’s the only thing that locks binary Wheels down to a specific python
version on 2.x (lxml-3.3.3-cp27-none-macosx_10_4_x86_64.whl). If we don’t
apply that logic to the implementation specific tags then we have different
behavior which I feel is very confusing.

More confusing PEP425 explicitly says that py2 and py3 are not shorthand
for py20/py30 but with how pip is implemented that is exactly what they are.

So i don’t know the *right* answer is, but it’s apparently a mess because PEP425
defines one thing, pip implements another, and because of ~behaviors~ we
can’t just pick either the PEP or the implementation in pip as the “right” answer.

>
> It does mean that I'm more in favour of defaulting to the appropriate
> one of py2 or py3, on the basis that it's never going to be any worse
> than using the sdist.
>
>>> 2. Is it worth changing the PEP wording to match this?
>>
>> Probably, we need to make a new PEP anyways to add compatibility
>> tags for Linux stuff so we could roll it into that.
>
> That was what I meant by a new version of the PEP. I'm not sure if
> clarifying the wording without a version bump to the spec is allowed,
> and I don't think this is important enough to try to make an
> exception.
>
>>> 5. Is anybody actually still reading this? :-)
>>
>> Yes!
>
> Phew!
>
> Paul


signature.asc

Paul Moore

unread,
Mar 19, 2014, 1:38:18 PM3/19/14
to Donald Stufft, pypa-dev
On 19 March 2014 16:53, Donald Stufft <don...@stufft.io> wrote:
> Another tricky thing to decide here is what this means for cpXY and ppXY
> and the other implementation specific tags. If we go with the more convenient
> reading of pyXY (#2 above) and we apply that to that to the implementation
> defined tags, then we're going to have a holy mess on our hands because
> currently that's the only thing that locks binary Wheels down to a specific python
> version on 2.x (lxml-3.3.3-cp27-none-macosx_10_4_x86_64.whl). If we don't
> apply that logic to the implementation specific tags then we have different
> behavior which I feel is very confusing.

The current XY-means-X-and-anything-up-to-Y behaviour is only for
"py". All other cases are explicit match only, IIRC. It is confusing,
agreed.

> More confusing PEP425 explicitly says that py2 and py3 are not shorthand
> for py20/py30 but with how pip is implemented that is exactly what they are.

The trouble with the current implementation, IMO, is that it tries too
hard to be general in ways that nobody actually has any practical
experience of, or any real use case for. Look at get_supported in the
pip sources, it has a "versions" argument which allows the caller to
override the default X0, X1, ... XY list of accepted pure-python
versions.

When I debated this with Daniel, my view was that we needed a function
which took a tag set and answered the question "do I support this tag
set?". Daniel preferred a function to return a list of tags that are
supported, in order of preference. My way had problems with doing
"best match" checks, even though I thought it was conceptually
cleaner, so I didn't pursue it.

> So i don't know the *right* answer is, but it's apparently a mess because PEP425
> defines one thing, pip implements another, and because of ~behaviors~ we
> can't just pick either the PEP or the implementation in pip as the "right" answer.

It's worth remembering that the compatibility tags are *not* about
supported-version metadata. That's what requires-python in Metadata
2.0 is for. The tags are purely intended as a selection mechanism for
choosing the correct wheel to install from a list of wheels (or
determining that there is no suitable wheel so you need to fall back
on the sdist or fail). So to that extent, the *only* important
function of a tag is to say whether or not the content of a wheel is
loadable in a given version of Python. For binary extensions, this is
a key feature, and that's why the tags are strict for binaries.

For pure Python code, given that wheels do not contain pyc files by
design, there is no equivalent concept of "binary incompatibility". So
the selection question becomes simply a question of whether the
project ships different source code for different versions. To that
extent, the only practical use case is projects that ship separate
source for Python 2 and 3 (usually as a result of using 2to3). So the
only case where selecting a different wheel will matter is where a
major version change is involved.

OK, I think I've just convinced myself that the only significant
python-version compatibility tags in today's packaging environment
are:

py2 and py3 (possibly combined as py2.py3)
cpXY (for binary extensions, where only the specific minor version is
compatible).

All others are either not used, or are being used incorrectly and
should be replaced by one of the above (go on - prove me wrong with a
counterexample :-))

To my knowledge, nobody ships binaries specifically for PyPy, or
Jython, or IronPython (at least not on PyPI). Also, I don't believe
the stable ABI is in common enough use yet that we can say what works
in terms of a tag for it.

Serious question - should we strip back tag support to this bare
minimum until we get actual use cases for anything more complex? We
could still accept existing wheels with pyXY tags, and just treat pyXY
as equivalent to pyX until we get a report of a case where that
doesn't do the right thing.

Paul

PS I'm not saying that going forwards projects couldn't use the
existing tags to ship (for example) one wheel containing code for 3.4+
that uses asyncio and another that supports 3.3 and earlier using
threads - but they don't do that yet and it'd be easier to add such
support correctly if we had someone with an actual requirement asking
for it.

Donald Stufft

unread,
Mar 19, 2014, 1:49:24 PM3/19/14
to Paul Moore, pypa-dev

On Mar 19, 2014, at 1:38 PM, Paul Moore <p.f....@gmail.com> wrote:

> On 19 March 2014 16:53, Donald Stufft <don...@stufft.io> wrote:
>> Another tricky thing to decide here is what this means for cpXY and ppXY
>> and the other implementation specific tags. If we go with the more convenient
>> reading of pyXY (#2 above) and we apply that to that to the implementation
>> defined tags, then we're going to have a holy mess on our hands because
>> currently that's the only thing that locks binary Wheels down to a specific python
>> version on 2.x (lxml-3.3.3-cp27-none-macosx_10_4_x86_64.whl). If we don't
>> apply that logic to the implementation specific tags then we have different
>> behavior which I feel is very confusing.
>
> The current XY-means-X-and-anything-up-to-Y behaviour is only for
> "py". All other cases are explicit match only, IIRC. It is confusing,
> agreed.

Right, and we can’t change that (although it would be useful in the same way
that “py” is useful, but it’s too late to change because it’s how selection works
at all for binary Wheels on 2.x).
I think if we do any stripping it would only be with the pyXY tags and
not any of the implementation specific ones or any of the others.

>
> Paul
>
> PS I'm not saying that going forwards projects couldn't use the
> existing tags to ship (for example) one wheel containing code for 3.4+
> that uses asyncio and another that supports 3.3 and earlier using
> threads - but they don't do that yet and it'd be easier to add such
> support correctly if we had someone with an actual requirement asking
> for it.

After reading this, and looking at distlib (which implements it similarly to pip)
and pip, and thinking about it some more. I think we should just update
the current PEP to specify that pyXY (and only pyXY) is defined to be
“or later” and we should suggest that in most situations py2 or py3 is probably
more reasonable. I don’t think there’s any harm in leaving the spec more capable
incase someone does need that ability. This could be handled as that spec
was just underspecified.

I think that Wheel should change to defaulting to pyX instead of pyXY but that
there should be an easy way to specify exactly which compatibility tags you
want to use as well as —universal which will do py2.py3.
signature.asc

Paul Moore

unread,
Mar 19, 2014, 1:59:10 PM3/19/14
to Donald Stufft, pypa-dev
On 19 March 2014 17:49, Donald Stufft <don...@stufft.io> wrote:
> After reading this, and looking at distlib (which implements it similarly to pip)
> and pip, and thinking about it some more. I think we should just update
> the current PEP to specify that pyXY (and only pyXY) is defined to be
> "or later" and we should suggest that in most situations py2 or py3 is probably
> more reasonable. I don't think there's any harm in leaving the spec more capable
> incase someone does need that ability. This could be handled as that spec
> was just underspecified.

That's cool. I'm happy to go with that. Do you know what the process
is for updating the PEP? Presumably I'll need to create a PR against
the PEP repository. I'll have a word with Nick once the discussion
settles down.

> I think that Wheel should change to defaulting to pyX instead of pyXY but that
> there should be an easy way to specify exactly which compatibility tags you
> want to use as well as --universal which will do py2.py3.

Yeah, that's easy enough to do. Matt Iversen had a PR implementing an
option to specify the exact tag you want (although he was using the
adhoc "wheel" section of setup.cfg). I can see about creating a new PR
taking the best ideas from the current two.

Let's see what anyone else has to say, but it sounds like we may have
a plan [1].

Paul

[1] which is worrying, my motto is that if God had wanted us to plan,
why did he make it a four-letter word?

Daniel Holth

unread,
Mar 19, 2014, 1:59:15 PM3/19/14
to Paul Moore, Donald Stufft, pypa-dev
I agree that it makes sense for py2 and py3 to be the default tags for
pure Python wheels. You would not have to change the current
installers at all.

"How do you publish a wheel that is only for py32?": You could publish
one wheel for py32, and another for py33. The py32 wheel would be
preferred on py32, and the py33 wheel would be preferred for py34 and
up.

That is less obviously useful than publishing different py2 and py3
wheels because while we have 2to3 and 3to2 it is less common to have
compilers that convert Python 3.2 to Python 3.3 or vice-versa. Perhaps
if you were desperate you could implement the pip-x.y binaries this
way.

The ABI tag *should* be set on Python 2 instead of using none but no
one has implemented it, it requires a (simple) re-implementation of
the Python 3 idea.

The PEP is trying to force you to do == and not <= >= or startswith()
matching. This has always been confusing to some and seems harmless
for the py26 tag, but it is better than trying to have the
specification parse for example the Mac OS X version numbers out of
the third tag so it can decide whether it's running under a newer OS
X.

Matthew Iversen

unread,
Mar 19, 2014, 6:00:48 PM3/19/14
to pypa...@googlegroups.com

Sorry, the below looks like a forwarded message but the top most level writing is mine (yay managing multiple email accounts):

> On 20/03/14 03:53, Donald Stufft wrote:
>>
>> On Mar 19, 2014, at 12:03 PM, Paul Moore <p.f....@gmail.com> wrote:
>>
>>> On 19 March 2014 15:46, Donald Stufft <don...@stufft.io> wrote:
>>>>
>>>> How would you specify that a Wheel is only viable on Python 3.2 and not
>>>> Python 3.3?
>>>
>>> You can't. The way the tags are checked simply doesn't allow that -
>>> Python 3.3 accepts a py32 tag. (This is why I was careful to say I
>>> didn't want to change how tags were checked - this is arguably the
>>> wrong way to do it, Daniel and I had a bit of a debate over it when he
>>> first implemented it, but changing it would be backward incompatible
>>> and I didn't want to go there at this point).
>>>
>>> The relevant code is at
>>> https://github.com/pypa/pip/blob/develop/pip/pep425tags.py#L51
>>
>> It would be backwards incompatible to the implementation, but not to the
>> Spec. I do think either the spec needs to be updated or the code needs
>> to be updated. Either way I don’t think there is going to be many (any?)
>> one relying not this currently. To me it feels like smell to have both
>> py30 and py3 mean the same thing *and* it feels surprising to me (having
>> read the spec multiple times but not the implementation).
>

> To me, the spec is fine - excerpt from 425:
>
> > It is recommended that installers try to choose the most feature complete built distribution available (the one most specific to the installation environment) by default before falling back to pure Python versions published for older Python releases.
>
> So a pure py32 tag should work fine for python 3.3, which is what pip should hopefully be doing. Binary packages obviously aren't pure so a python 3.3 shouldn't try to install a py3.2 tagged binary distribution.


>
>>
>> That’s not exactly the case, I’ve put errors in setup.py’s that raised an
>> error rather than installed on Pythons that I knew didn’t work.
>>
>> I think if we take the stance that the code is correct and the spec is wrong
>> then py2 and py3 should be deprecated in favor of py30 and py20. Those
>> are more accurately describing what those tags *actually* mean and are
>> going to be less confusing than having py2, py26, py27, py3, py32, py33.
>

> Naively to me, "py20" is confusing at first glance (really I imagine people will be asking "why is there a 0 there? I don't support python 2.0..."), whereas "py2" is not confusing. It just says to me that this distribution hopes to be compatible with python 2 in general. Note that "py2.py3" is really an optimistic tag at the moment, e.g there are probably many of them that are not compatible with python 3.1 or 2.4. I think it's important to ask in general whether you can be certain about python compatibility, or just optimistic. I think it can be realistic to be more certain about compatibility with binary tags any, but only really optimistic with a pure package - because you could be running on *anyone's* implementation.

> For pure packages, I can't imagine why pyMN meaning compatible with >= M.N wouldn't make sense, and it's not a problem with binary packages because they're always ==. I think the wording for the py2 != py20 is meant to take binary packages into account. E.g, if somehow you had a py4 binary package that wouldn't be a py40 binary package.
>
> --
> Matt Iversen
> PGP: 0xc046e8a874522973 // 2F04 3DCC D6E6 D5AC D262  2E0B C046 E8A8 7452 2973

signature.asc

Marcus Smith

unread,
Mar 19, 2014, 11:53:12 PM3/19/14
to Paul Moore, Donald Stufft, pypa-dev

> After reading this, and looking at distlib (which implements it similarly to pip)
> and pip, and thinking about it some more. I think we should just update
> the current PEP to specify that pyXY (and only pyXY) is defined to be
> "or later" and we should suggest that in most situations py2 or py3 is probably
> more reasonable. I don't think there's any harm in leaving the spec more capable
> incase someone does need that ability. This could be handled as that spec
> was just underspecified.

That's cool. I'm happy to go with that. Do you know what the process
is for updating the PEP? Presumably I'll need to create a PR against
the PEP repository. I'll have a word with Nick once the discussion
settles down.

> I think that Wheel should change to defaulting to pyX instead of pyXY but that
> there should be an easy way to specify exactly which compatibility tags you
> want to use as well as --universal which will do py2.py3.

Yeah, that's easy enough to do. Matt Iversen had a PR implementing an
option to specify the exact tag you want (although he was using the
adhoc "wheel" section of setup.cfg). I can see about creating a new PR
taking the best ideas from the current two.

from my skim, this all sounds fine to me.
I like the idea of overriding with the exact tag you want.

Paul Moore

unread,
Mar 20, 2014, 5:47:39 AM3/20/14
to Marcus Smith, Donald Stufft, pypa-dev
On 20 March 2014 03:53, Marcus Smith <qwc...@gmail.com> wrote:
> I like the idea of overriding with the exact tag you want.

One question, though. Assuming (for the sake of argument) that we go
with a syntax of --impl=py32, then what should happen if the user
specifies --impl=py32 and the project includes a C extension (for
added fun, if there's an optional speedup extension)?

The advantage of specifying the Python version is that it can be
justifiably ignored for non-pure builds.

Paul.

Marcus Smith

unread,
Mar 20, 2014, 12:41:37 PM3/20/14
to pypa...@googlegroups.com, Paul Moore

One question, though. Assuming (for the sake of argument) that we go
with a syntax of --impl=py32, then what should happen if the user
specifies --impl=py32 and the project includes a C extension (for
added fun, if there's an optional speedup extension)?

in bdist_wheel, I was NOT assuming we'd add error/warnings. possibly later.
i.e. nothing that says "you should really use 'cp32'  (or whatever impl they are on)

same on the pip side, I was also assuming no additional checking because of this.
 
btw, I'd prefer "python" over "impl".  
PEP425 does call it the "Python Tag"
the python tag is implementation + python version.

Paul Moore

unread,
Mar 20, 2014, 1:00:08 PM3/20/14
to Marcus Smith, pypa-dev
On 20 March 2014 16:41, Marcus Smith <qwc...@gmail.com> wrote:
>>
>> One question, though. Assuming (for the sake of argument) that we go
>> with a syntax of --impl=py32, then what should happen if the user
>> specifies --impl=py32 and the project includes a C extension (for
>> added fun, if there's an optional speedup extension)?
>
> in bdist_wheel, I was NOT assuming we'd add error/warnings. possibly later.
> i.e. nothing that says "you should really use 'cp32' (or whatever impl they
> are on)
>
> same on the pip side, I was also assuming no additional checking because of
> this.

I'm not quite sure if we're thinking of the same things here - I
wasn't thinking about warnings. I was thinking about users making
mistakes and ending up with wrongly-tagged wheels. This could be
particularly annoying if they put the settings in setup.cfg.

What I was suggesting is that we only respect the --python tag for
pure-python builds, as the non-pure tagging algorithm is fine as it
stands. I was thinking that made --python=cp32 badly wrong as it would
never match, but actually I checked the code and it's just unusual -
cpXY tags are allowed along with pyXY. (Meh, I cannot bring myself to
like that get_supported approach :-()

> btw, I'd prefer "python" over "impl".
> PEP425 does call it the "Python Tag"
> the python tag is implementation + python version.

Yes, that's probably better - can it just be --python=py32 or does it
need to be --python-tag?

Paul

Donald Stufft

unread,
Mar 20, 2014, 2:16:52 PM3/20/14
to Paul Moore, Marcus Smith, pypa-dev
It should be at least possible to force it to py2 even if there is a compiled library. Someone could have an optional extension module that they bundle with their Wheel which they do runtime detection if they should load it or not.
signature.asc

Marcus Smith

unread,
Mar 20, 2014, 2:31:41 PM3/20/14
to Paul Moore, pypa-dev
 
I'm not quite sure if we're thinking of the same things here - I
wasn't thinking about warnings. I was thinking about users making
mistakes and ending up with wrongly-tagged wheels.

I was thinking to let them make mistakes.
I was thinking of this as just a step above the 'mv' command.

Paul Moore

unread,
Mar 20, 2014, 2:49:50 PM3/20/14
to Marcus Smith, pypa-dev
OK, I can go with that. Let's see if people raise issues asking for
anything more complicated. (My instincts say they might if they start
putting the flag into setup.cfg, but I'm happy to call YAGNI for now).

Paul

Paul Moore

unread,
Mar 22, 2014, 11:26:24 AM3/22/14
to Marcus Smith, pypa-dev
On 20 March 2014 18:49, Paul Moore <p.f....@gmail.com> wrote:
> OK, I can go with that. Let's see if people raise issues asking for
> anything more complicated. (My instincts say they might if they start
> putting the flag into setup.cfg, but I'm happy to call YAGNI for now).

I've put up a PR, https://bitbucket.org/pypa/wheel/pull-request/42
implementing this (including tests for the various tagging options,
yay!)

Please take a look and comment.
Paul

PS Editing PRs and dealing with rejected PRs is a real nightmare in
Mercurial, as far as I can see. If anyone can tell me a good workflow
that doesn't involve juggling specific revision numbers and always
adding --force to my pushes to tell it I'm OK with crating new
branches, please let me know. I'm officially a convert to git now...

Donald Stufft

unread,
Mar 22, 2014, 11:34:56 AM3/22/14
to Paul Moore, Marcus Smith, pypa-dev
One of Us. One of Us. One of Us. One of Us.
signature.asc

Marcus Smith

unread,
Mar 22, 2014, 1:23:08 PM3/22/14
to Paul Moore, pypa-dev
specific revision numbers?  I'm certainly no whiz with hg, but use a non-default branch to push PRs

hg branch mybranch
edit....
hg push -b mybranch --new-branch


Paul Moore

unread,
Mar 22, 2014, 1:49:40 PM3/22/14
to Marcus Smith, pypa-dev
On 22 March 2014 17:23, Marcus Smith <qwc...@gmail.com> wrote:
> specific revision numbers? I'm certainly no whiz with hg, but use a
> non-default branch to push PRs
>
> hg branch mybranch
> edit....
> hg push -b mybranch --new-branch

Hmm, you may be right. I was always told that branches in Mercurial
are heavyweight things, not intended for throwaway use like git
branches (hg bookmarks are more equivalent to git branches,
apparently). I might give that a try.

Paul

Marcus Smith

unread,
Mar 22, 2014, 2:41:18 PM3/22/14
to Paul Moore, pypa-dev
btw, we could convert it to git (but still leave it in bitbucket due to all the issue/PR history)
bitbucket supports git.

Donald Stufft

unread,
Mar 22, 2014, 3:58:26 PM3/22/14
to Marcus Smith, Paul Moore, pypa-dev
We could also move it to github too! Resistance is futile. 

Daniel Holth

unread,
Mar 22, 2014, 7:26:53 PM3/22/14
to Donald Stufft, pypa...@googlegroups.com, Marcus Smith, Paul Moore

I would prefer it remain in mercurial.

Paul Moore

unread,
Mar 22, 2014, 7:41:31 PM3/22/14
to Daniel Holth, Donald Stufft, pypa-dev, Marcus Smith
On 22 March 2014 23:26, Daniel Holth <dho...@gmail.com> wrote:
> I would prefer it remain in mercurial.

Borg-style assimilation jokes aside, I'm happy that it be left in
Mercurial. Personally, I consider Daniel's preference to be final on
this.

(Although Daniel, if you have any advice on how to manage pull
requests in Mercurial as per my earlier comment, I'd be grateful).
Paul

Daniel Holth

unread,
Mar 22, 2014, 7:46:56 PM3/22/14
to Paul Moore, Marcus Smith, pypa...@googlegroups.com, Donald Stufft

The bookmarks extension seems the most promising and there are advanced history edit and changeset states features in the latest version. Unfortunately I have not used them that much.

Paul Moore

unread,
Mar 22, 2014, 7:51:09 PM3/22/14
to Daniel Holth, Marcus Smith, pypa-dev, Donald Stufft
On 22 March 2014 23:46, Daniel Holth <dho...@gmail.com> wrote:
> The bookmarks extension seems the most promising and there are advanced
> history edit and changeset states features in the latest version.
> Unfortunately I have not used them that much.

Yeah, those are the features I think would be most useful. But while
Mercurial has become a lot better about locally editing history (it
was never a big issue, mq is awesome) I'm not sure what its
capabilities are like when pushing to somewhere like bitbucket. With
git, I tend to see my personal clone of pip as "local" and I'm happy
to mess with history on there (this helps as I typically work between
2 PCs, and use github as the bridge) but I'm not sure how well that
will work with hg.

I'll do some digging and maybe ask on the hg mailing lists.

Thanks,
Paul
Reply all
Reply to author
Forward
0 new messages