Python 3 port
Arc Riley
We're looking for a database-agnostic ORM package to build the
standard Concordance database classes on. This requires Py3 support.
http://concordance-xmpp.org/
Yes, yes, there aren't many DB packages for Py3 yet. This is exactly
why we're looking for something that's database-agnostic; right now
we're limited to sqlite but preferably our classes shouldn't care what
database is being used and can just as readily be used with postgres
or mysql in the future. This is a chicken-and-egg problem that will
only be solved when existing Py3 code can take advantage of postgres
py3 bindings.
We have already ported Genshi to Py3 to the extent that it works for
our purposes, hosting our transformed branch on http://hg.concordance-xmpp.org/genshi-py3
Two of us spent a good deal of time last night banging out the
wrinkles left in sqlalchemy post-2to3, reducing the number of errors
reported by alltests from 2000+ to just over 100. All our patches can
be seen at http://hg.concordance-xmpp.org/sqlalchemy-py3 applied
against a mirror of your hg mirror.
Clearly we're not done yet but we've whittled down to the last few
pieces. While we don't mind hosting this branch until you're ready to
officially support Py3 it'd be nice to have all this work in a form
that you can readily pull in and use.
What can we do to help with that, and toward a near-future sqlalchemy
release of a -py3 versioned tarball.
Michael Bayer
>
> Yes, yes, there aren't many DB packages for Py3 yet. This is exactly
> why we're looking for something that's database-agnostic; right now
> we're limited to sqlite but preferably our classes shouldn't care what
> database is being used and can just as readily be used with postgres
> or mysql in the future. This is a chicken-and-egg problem that will
> only be solved when existing Py3 code can take advantage of postgres
> py3 bindings.
>
> We have already ported Genshi to Py3 to the extent that it works for
> our purposes, hosting our transformed branch on
> http://hg.concordance-xmpp.org/genshi-py3
>
> Two of us spent a good deal of time last night banging out the
> wrinkles left in sqlalchemy post-2to3, reducing the number of errors
> reported by alltests from 2000+ to just over 100. All our patches can
> be seen at http://hg.concordance-xmpp.org/sqlalchemy-py3 applied
> against a mirror of your hg mirror.
>
> Clearly we're not done yet but we've whittled down to the last few
> pieces. While we don't mind hosting this branch until you're ready to
> officially support Py3 it'd be nice to have all this work in a form
> that you can readily pull in and use.
>
> What can we do to help with that, and toward a near-future sqlalchemy
> release of a -py3 versioned tarball.
We have a mostly working py3 of SQLAlchemy which is targeted towards 0.6.
It includes a wrapper around 2to3 which allows us to manually control
sections of code for which 2to3 can't guess what we'd like to do. The
0.6 branch is available for review at
http://svn.sqlalchemy.org/sqlalchemy/branches/rel_0_6 , and we'll be
sprinting on it further at pycon. The sa2to3.py script detects blocks of
code marked as PY3K or PY2K and adjusts before sending off to 2to3.
The 0.6 series of SQLA is focused around a refactor of database dialects
and is appropriate for py3k since it decouples database compilers from
DBAPIs - the release can run MySQL against both MySQLdb as well as pyodbc,
and can run Postgres against either psycopg2 or pg8000 (the latter which
runs on py3k). Support for jython, ironpython, others is much more of a
drop-in as existing SQL compilation code can be reused.
It seems like most of the changes implemented in your branch of 0.5 are
already present in 0.6. If you'd like to work on 0.6 getting all tests to
pass, let me know. 0.6 is intended to be almost totally backwards compat
with 0.5, no huge ORM changes or anything like that, so the average
application should be able to switch to it without any significant
migration of their code. There might be some changes needed to external
packages that build upon schema compilation like Migrate.
Arc Riley
Ok. I can chalk this up to practice and a learning experience.
Yes - we're looking to use sqlalchemy in a Py3-only package set for first release sometime this week (we can depend on a svn tree for now). Is basic ORM, mapping, etc ready in svn rel_0_6? Backwards compatibility isn't needed since we're implementing new code.
Yes - we'd be happy to contribute towards getting most/all the tests to pass on Py3, especially as needed for our purposes but more generally toward helping with community migration.
So 0.6 is planned to have a tarball release specific for Py3, or otherwise installable for Py3 using setup.py?
Michael Bayer
> Grr - you mean trunk *isn't* the head development tree in svn?
> Frustrating
> in wasted time in duplicated effort. Can you please post this on the
> website
> so others don't make the same mistake?
trunk is the head dev tree in SVN, but we like to have trunk always
passing all tests. 0.6 is very recent and isn't quite passing everything
completely so has not been ready for the merge to trunk, but will be soon.
the site is in need of updating regarding 0.6's existence but usually
people come in and chat on the list or IRC before starting major
development initiatives, but sorry, we'll add it.
> Yes - we're looking to use sqlalchemy in a Py3-only package set for first
> release sometime this week (we can depend on a svn tree for now). Is
> basic
> ORM, mapping, etc ready in svn rel_0_6? Backwards compatibility isn't
> needed since we're implementing new code.
ORM and mapping code is the same code as in 0.5. We might remove the
ability to say session.flush([list of objects]) but I don't think too much
else is planned ORM-wise.
>
> Yes - we'd be happy to contribute towards getting most/all the tests to
> pass
> on Py3, especially as needed for our purposes but more generally toward
> helping with community migration.
>
> So 0.6 is planned to have a tarball release specific for Py3, or otherwise
> installable for Py3 using setup.py?
The 2to3 process is slow and finicky so when 0.6 is fully released, we'll
be releasing py2k and py3k tarballs separately.
Arc Riley
the site is in need of updating regarding 0.6's existence but usually
people come in and chat on the list or IRC before starting major
development initiatives, but sorry, we'll add it.
Actually I've been asking on IRC for two days re: Py3. Nobody seemed to know anything about this. Miscommunication all-around.
The 2to3 process is slow and finicky so when 0.6 is fully released, we'll
be releasing py2k and py3k tarballs separately.
Awesome. Kudos to you guys for this!
As I said, we'll help as we can and are needed.
Michael Bayer
ive been under crushing deadlines at work and haven't had any resources to
work on 0.6 lately, though Randall has been. feel free to check it out
from SVN and start running it.