Pypi release policy

75 views
Skip to first unread message

M3nt0r3

unread,
Feb 14, 2011, 4:57:22 AM2/14/11
to sqlalchemy
Yesterday my life become an hell. :)
On saturday ( :O ) SA0.7.b1 is released and easy_install start to use
it. The problem is that it broke some installer and doesn't work with
my app.
I think that i'd better if easy_install sqlalchemy continue di install
the stable branches of the lib and maybe a sqlalchemy==dev can use de
0.7.b ...cicle.

thanks

Wichert Akkerman

unread,
Feb 14, 2011, 5:18:47 AM2/14/11
to sqlal...@googlegroups.com
On 2/14/11 10:57 , M3nt0r3 wrote:
> Yesterday my life become an hell. :)
> On saturday ( :O ) SA0.7.b1 is released and easy_install start to use
> it. The problem is that it broke some installer and doesn't work with
> my app.

That sounds like a bug in your installer, not in SQLAlchemy.

Wichert.

Tarek Ziadé

unread,
Feb 14, 2011, 5:22:28 AM2/14/11
to sqlal...@googlegroups.com

I had a similar issue this week-end, all my buildbots turned red this
week-end, so I had to pin to 0.6.6 my trunk

It seems that 0.6.7b1 introduced a backward incompatible change
without a deprecation step, from 0.6.6 to 0.7.1b

(Unless I missed a previous deprecation warning, but I don't recall seeing any)


ERROR: Failure: TypeError (__init__() takes exactly 1 argument (2 given))
----------------------------------------------------------------------
Traceback (most recent call last):
File "/srv/hudson/jobs/Sync-Server-Unit-tests/workspace/lib/python2.6/site-packages/nose-1.0.0-py2.6.egg/nose/loader.py",
line 390, in loadTestsFromName
addr.filename, addr.module)
File "/srv/hudson/jobs/Sync-Server-Unit-tests/workspace/lib/python2.6/site-packages/nose-1.0.0-py2.6.egg/nose/importer.py",
line 39, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/srv/hudson/jobs/Sync-Server-Unit-tests/workspace/lib/python2.6/site-packages/nose-1.0.0-py2.6.egg/nose/importer.py",
line 86, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/srv/hudson/jobs/Sync-Server-Unit-tests/workspace/deps/server-storage/syncstorage/tests/test_sql.py",
line 41, in <module>
from syncstorage.storage.sqlmappers import get_wbo_table_name
File "/srv/hudson/jobs/Sync-Server-Unit-tests/workspace/deps/server-storage/syncstorage/storage/sqlmappers.py",
line 52, in <module>
class Collections(_Base):
File "/srv/hudson/jobs/Sync-Server-Unit-tests/workspace/deps/server-storage/syncstorage/storage/sqlmappers.py",
line 55, in Collections
userid = Column(Integer(11), primary_key=True, nullable=False)
TypeError: __init__() takes exactly 1 argument (2 given)


The code: http://hg.mozilla.org/services/server-storage/file/78762deede5d/syncstorage/storage/sqlmappers.py#l55


Cheers
Tarek

--
Tarek Ziadé | http://ziade.org

M3nt0r3

unread,
Feb 14, 2011, 5:25:50 AM2/14/11
to sqlal...@googlegroups.com
My installer for windows download during  the process some libs. I use easy_install sqlalchemy and it works. Now too works but 0.7.b1 is not working with the app itself now so it create a lot of problem.  I thought that if 0.7.b1 is a beta and brokes API maybe it should be better to use sqlalchemy==0.7 or sqlalchemy == dev.

Wichert Akkerman

unread,
Feb 14, 2011, 5:31:40 AM2/14/11
to sqlal...@googlegroups.com

It sounds like the dependencies in your package are too liberal. I would
change your dependency to "SQLAlchemy >=0.6, <0.7dev". That way you can
never accidentally install an incompatible SQLAlchemy version.

Wichert.

Tarek Ziadé

unread,
Feb 14, 2011, 5:39:20 AM2/14/11
to sqlal...@googlegroups.com

If the changes that break the code did not have a deprecation step in
0.6, that's still an issue to fix imo

you don't release at pypi a version that breaks the latest stable. or
if you do, you check the "hidden" attribute on that release, to avoid
this problem with installers

>
> Wichert.
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To post to this group, send email to sqlal...@googlegroups.com.
> To unsubscribe from this group, send email to
> sqlalchemy+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/sqlalchemy?hl=en.

Wichert Akkerman

unread,
Feb 14, 2011, 5:44:48 AM2/14/11
to sqlal...@googlegroups.com
On 2/14/11 11:39 , Tarek Ziad� wrote:
> On Mon, Feb 14, 2011 at 11:31 AM, Wichert Akkerman<wic...@wiggy.net> wrote:
>> It sounds like the dependencies in your package are too liberal. I would
>> change your dependency to "SQLAlchemy>=0.6,<0.7dev". That way you can
>> never accidentally install an incompatible SQLAlchemy version.
>
> If the changes that break the code did not have a deprecation step in
> 0.6, that's still an issue to fix imo

Sure.

> you don't release at pypi a version that breaks the latest stable. or
> if you do, you check the "hidden" attribute on that release, to avoid
> this problem with installers

SQLAlchemy 0.7b1 is hidden. The hidden flag only hides it from humans
though, not from setuptools.

Wichert.

Tarek Ziadé

unread,
Feb 14, 2011, 5:53:47 AM2/14/11
to sqlal...@googlegroups.com
On Mon, Feb 14, 2011 at 11:44 AM, Wichert Akkerman <wic...@wiggy.net> wrote:
..

>
>> you don't release at pypi a version that breaks the latest stable.  or
>> if you do, you check the "hidden" attribute on that release, to avoid
>> this problem with installers
>
> SQLAlchemy 0.7b1 is hidden. The hidden flag only hides it from humans
> though, not from setuptools.

oh true...I forgot it's not hidden in the simple index :/

Michael Bayer

unread,
Feb 14, 2011, 12:00:11 PM2/14/11
to sqlal...@googlegroups.com

On Feb 14, 2011, at 5:39 AM, Tarek Ziadé wrote:

> On Mon, Feb 14, 2011 at 11:31 AM, Wichert Akkerman <wic...@wiggy.net> wrote:
>> On 2/14/11 11:25 , M3nt0r3 wrote:
>>>
>>> My installer for windows download during the process some libs. I use
>>> easy_install sqlalchemy and it works. Now too works but 0.7.b1 is not
>>> working with the app itself now so it create a lot of problem. I thought
>>> that if 0.7.b1 is a beta and brokes API maybe it should be better to use
>>> sqlalchemy==0.7 or sqlalchemy == dev.
>>
>> It sounds like the dependencies in your package are too liberal. I would
>> change your dependency to "SQLAlchemy >=0.6, <0.7dev". That way you can
>> never accidentally install an incompatible SQLAlchemy version.
>
> If the changes that break the code did not have a deprecation step in
> 0.6, that's still an issue to fix imo
>
> you don't release at pypi a version that breaks the latest stable. or
> if you do, you check the "hidden" attribute on that release, to avoid
> this problem with installers

Tarek -

We're talking about 0.7b1 and "hidden" is checked, I always check it immediately and un-hide the previous released version as soon as I put up a beta. It appears that easy_install and pip still go out and install the highest version regardless of the "hidden" flag.

http://pypi.python.org/pypi?%3Aaction=search&term=sqlalchemy&submit=search -> shows 0.6.6

http://pypi.python.org/pypi/SQLAlchemy/ -> shows 0.6.6


what else am I supposed to be clicking so that I can have beta releases of newer versions on pypi ?

Michael Bayer

unread,
Feb 14, 2011, 12:00:52 PM2/14/11
to sqlal...@googlegroups.com

OK so you're the expert - how does one release a beta on pypi without crashing everyone's "stable" install ? should i just stick to sourceforge until final release ?

Tarek Ziadé

unread,
Feb 14, 2011, 12:11:16 PM2/14/11
to sqlal...@googlegroups.com
On Mon, Feb 14, 2011 at 6:00 PM, Michael Bayer <mik...@zzzcomputing.com> wrote:
>
> On Feb 14, 2011, at 5:53 AM, Tarek Ziadé wrote:
>
>> On Mon, Feb 14, 2011 at 11:44 AM, Wichert Akkerman <wic...@wiggy.net> wrote:
>> ..
>>>
>>>> you don't release at pypi a version that breaks the latest stable.  or
>>>> if you do, you check the "hidden" attribute on that release, to avoid
>>>> this problem with installers
>>>
>>> SQLAlchemy 0.7b1 is hidden. The hidden flag only hides it from humans
>>> though, not from setuptools.
>>
>> oh true...I forgot it's not hidden in the simple index :/
>
> OK so you're the expert - how does one release a beta on pypi without crashing everyone's "stable" install ?   should i just stick to sourceforge until final release ?

Unfortunately, Setuptools will pick the latest version and won't care
about beta tags (zc.buildout has such feature -- "prefer-final", and
Distutils2 too)

So I guess the best way with the current eco-system is to avoid
pushing any unstable release to PyPI

or... if you have the time to do so, push a new 0.7 beta that makes
sure people that run on the latest 0.6 can run it -- with deprecation
warnings all over the place :)


Cheers
Tarek

>
>
>
>
>
>>
>> --
>> You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
>> To post to this group, send email to sqlal...@googlegroups.com.
>> To unsubscribe from this group, send email to sqlalchemy+...@googlegroups.com.
>> For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
> To post to this group, send email to sqlal...@googlegroups.com.
> To unsubscribe from this group, send email to sqlalchemy+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
>
>

--
Tarek Ziadé | http://ziade.org

Michael Bayer

unread,
Feb 14, 2011, 12:18:09 PM2/14/11
to sqlal...@googlegroups.com

On Feb 14, 2011, at 12:11 PM, Tarek Ziadé wrote:

> On Mon, Feb 14, 2011 at 6:00 PM, Michael Bayer <mik...@zzzcomputing.com> wrote:
>>
>> On Feb 14, 2011, at 5:53 AM, Tarek Ziadé wrote:
>>
>>> On Mon, Feb 14, 2011 at 11:44 AM, Wichert Akkerman <wic...@wiggy.net> wrote:
>>> ..
>>>>
>>>>> you don't release at pypi a version that breaks the latest stable. or
>>>>> if you do, you check the "hidden" attribute on that release, to avoid
>>>>> this problem with installers
>>>>
>>>> SQLAlchemy 0.7b1 is hidden. The hidden flag only hides it from humans
>>>> though, not from setuptools.
>>>
>>> oh true...I forgot it's not hidden in the simple index :/
>>
>> OK so you're the expert - how does one release a beta on pypi without crashing everyone's "stable" install ? should i just stick to sourceforge until final release ?
>
> Unfortunately, Setuptools will pick the latest version and won't care
> about beta tags (zc.buildout has such feature -- "prefer-final", and
> Distutils2 too)
>
> So I guess the best way with the current eco-system is to avoid
> pushing any unstable release to PyPI
>
> or... if you have the time to do so, push a new 0.7 beta that makes
> sure people that run on the latest 0.6 can run it -- with deprecation
> warnings all over the place :)

I'll pull it off of Pypi.

We definitely do push for as much backwards compat as possible, but the release does remove / hard change things that were raising warnings throughout 0.6, as well as lots of apps rely upon undocumented behaviors which may have changed, so its inevitable that some apps will need adjustment.

But the code itself should not be considered as "bug-free" as a stable release so in any case its not appropriate to push it into installations without explicit consent.

Eric Lemoine

unread,
Feb 15, 2011, 9:27:20 AM2/15/11
to sqlal...@googlegroups.com
On Monday, February 14, 2011, Michael Bayer <mik...@zzzcomputing.com> wrote:
>
> On Feb 14, 2011, at 12:11 PM, Tarek Ziadé wrote:
>
>> On Mon, Feb 14, 2011 at 6:00 PM, Michael Bayer <mik...@zzzcomputing.com> wrote:
>>>
>>> On Feb 14, 2011, at 5:53 AM, Tarek Ziadé wrote:
>>>
>>>> On Mon, Feb 14, 2011 at 11:44 AM, Wichert Akkerman <wic...@wiggy.net> wrote:
>>>> ..
>>>>>
>>>>>> you don't release at pypi a version that breaks the latest stable.  or
>>>>>> if you do, you check the "hidden" attribute on that release, to avoid
>>>>>> this problem with installers
>>>>>
>>>>> SQLAlchemy 0.7b1 is hidden. The hidden flag only hides it from humans
>>>>> though, not from setuptools.
>>>>
>>>> oh true...I forgot it's not hidden in the simple index :/
>>>
>>> OK so you're the expert - how does one release a beta on pypi without crashing everyone's "stable" install ?   should i just stick to sourceforge until final release ?
>>
>> Unfortunately, Setuptools will pick the latest version and won't care
>> about beta tags (zc.buildout has such feature -- "prefer-final", and
>> Distutils2 too)
>>
>> So I guess the best way with the current eco-system is to avoid
>> pushing any unstable release to PyPI
>>
>> or... if you have the time to do so, push a new 0.7 beta that makes
>> sure people that run on the latest 0.6 can run it -- with deprecation
>> warnings all over the place :)
>
> I'll pull it off of Pypi.
>
> We definitely do push for as much backwards compat as possible,  but the release does remove / hard change things that were raising warnings throughout 0.6, as well as lots of apps rely upon undocumented behaviors which may have changed, so its inevitable that some apps will need adjustment.
>
> But the code itself should not be considered as "bug-free" as a stable release so in any case its not appropriate to push it into installations without explicit consent.


But aren't apps supposed to use <=0.6.99 to avoid backward compats
issues? Apps that don't will also break when 0.7 final is on pypi.

--
Eric Lemoine

Camptocamp France SAS
Savoie Technolac, BP 352
73377 Le Bourget du Lac, Cedex

Tel : 00 33 4 79 44 44 96
Mail : eric.l...@camptocamp.com
http://www.camptocamp.com

Tarek Ziadé

unread,
Feb 15, 2011, 9:47:54 AM2/15/11
to sqlal...@googlegroups.com
On Tue, Feb 15, 2011 at 3:27 PM, Eric Lemoine
<eric.l...@camptocamp.com> wrote
..

>
> But aren't apps supposed to use <=0.6.99 to avoid backward compats
> issues? Apps that don't will also break when 0.7 final is on pypi.

There are different things here:

1/ PyPI allows projects to publish any release, and easy_install will
pick the latest one, whether it's a "final" (==stable) or not. You can
publish your trunk if you want.

2/ An application that defines a dependency can define it in different flavors:

a - "Give me the latest release that was made available at PyPI"
b - "Give me the latest release from the 0.6.x series", it can use a
<0.7 or <0.6.99
c - "Give me version XX" <--- best practice once in production


For applications that are using 2.a, the interpretation of most people
is that "the latest release at PyPI" they are depending on is not a
development release. If they want a development release, they do it
explicitly in their environment to leave on the edge.

3/ a user types "easy_install SQLAlchemy" and wants the "latest stable"

So, yeah, when 0.7.1 final will be out, some apps will break -- but
they've been warned and they can choose to change their code or pin
their dependency to the 0.6.x series. But right now, it's a
development release that has been published for feedback as opposed to
a final release.

The less disruptive process (until distutils2 is available) in that
case is to let people opt in to be beta testers, and let "SQLAlchemy"
means "latest stable", whether it's called by "easy_install
SQLAlchemy" or in the install_requires option in setuptools.


Cheers
Tarek

Eric Lemoine

unread,
Feb 16, 2011, 2:35:06 AM2/16/11
to sqlal...@googlegroups.com


Thanks for the detailed response Tarek. It means that beta testers
won't be able to download SQLAlchemy betas from PyPI, and will have to
install betas from source?

Cheers,

Chris Withers

unread,
Feb 16, 2011, 3:20:56 AM2/16/11
to sqlal...@googlegroups.com
On 15/02/2011 14:27, Eric Lemoine wrote:
> But aren't apps supposed to use<=0.6.99 to avoid backward compats
> issues?

Well, I wish I could just say < 0.7 but I guess that would suck in 0.7
betas? Tarek?

That said, this is spot on. Honestly, it's up to the consuming
package/project to limit what versions it'll accept. So Michael releases
0.7 final, your app might still not have been upgraded to work with it,
is it now Michael's fault your build is broken or people complain that
they easy_install'ed your library and it sucked down the wrong version
of sqlalchemy?

I hope the rhetorical nature of that question is apparent, and I only
wish there was more support in the python community for specifying
sensible backstop version limits...

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk

Wichert Akkerman

unread,
Feb 16, 2011, 3:23:04 AM2/16/11
to sqlal...@googlegroups.com
On 2/16/11 09:20 , Chris Withers wrote:
> On 15/02/2011 14:27, Eric Lemoine wrote:
>> But aren't apps supposed to use<=0.6.99 to avoid backward compats
>> issues?
>
> Well, I wish I could just say < 0.7 but I guess that would suck in 0.7
> betas? Tarek?

< 0.7dev is the right restriction to use.

Wichert.

Michael Bayer

unread,
Mar 21, 2011, 11:41:21 AM3/21/11
to sqlal...@googlegroups.com

On Feb 14, 2011, at 5:22 AM, Tarek Ziadé wrote:

> userid = Column(Integer(11), primary_key=True, nullable=False)
> TypeError: __init__() takes exactly 1 argument (2 given)
>
>
> The code: http://hg.mozilla.org/services/server-storage/file/78762deede5d/syncstorage/storage/sqlmappers.py#l55

I didn't expect that anyone was passing arguments to types like Integer, the value you pass there is meaningless as the default Integer doesn't have any kind of length.

I will restore the "catchall" constructor to the base type class with a deprecation warning.

Reply all
Reply to author
Forward
0 new messages