Python 3 compatibility

[Thomas Kluyver]

I've got a branch of rdflib which can be installed under Python 3.
It's not quite passing all the tests yet, but some of the remaining
ones are just quirks of doctests. I hope to sort the remaining
problems out in the next few days.

To install it in Python 3, you need to have distribute installed (it
runs 2to3 as part of the build process).

I don't have versions of Python older than 2.6 handy to test, but I've
been checking the docs and trying to only use things that have been in
place since 2.4.

The code can be found here:
http://code.google.com/r/takowl-rdflib/source/list?name=py3

Thanks,
Thomas

[Gunnar Aastrand Grimnes]

Hi Thomas,

Good stuff!

Previously someone else made a py3 compatible rdflib, I've put it
here: https://bitbucket.org/gromgull/rdflib-python3

but I must admit I've not really tested or maintaned it.

You are saying that if we apply your handful of changesets rdflib will
run fine under py3 and 2.X?
And that you've not changed anything major?

I'll test your code with some projects of mine that use rdflib under
2.X and if everything works as expected, we could perhaps merge?
What do the rest of you rdflib'ers say? Graham?

- Gunnar

> --
> You received this message because you are subscribed to the Google Groups "rdflib-dev" group.
> To post to this group, send email to rdfli...@googlegroups.com.
> To unsubscribe from this group, send email to rdflib-dev+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/rdflib-dev?hl=en.
>
>

--
http://gromgull.net

[Thomas Kluyver]

Hi Gunnar,

On 16 November 2011 05:52, Gunnar Aastrand Grimnes <grom...@gmail.com> wrote:

Previously someone else made a py3 compatible rdflib, I've put it
here: https://bitbucket.org/gromgull/rdflib-python3

I came across that when I was investigating it - my understanding is that Michele did a pure Python 3 port. The downside of that, as you noticed, is that it has to be maintained separately, or it will fall behind the Python 2 version. My strategy was to produce a Python 2 version which, after running 2to3 on it, works in Python 3. It should be easy to maintain, if someone can run the test suite on Python 3 occasionally (I've added a shell script to convert and run the tests).
 

You are saying that if we apply your handful of changesets rdflib will
run fine under py3 and 2.X?

More or less. There's a few more changes I'll make, but most of the tests are passing now.
 

And that you've not changed anything major?

I don't think so. The diff is quite big, but it's not changing the structure of anything, it's just a lot of specific changes, mainly relating to unicode.

Thanks,
Thomas

[Thomas Kluyver]

On 16 November 2011 11:19, Thomas Kluyver <tak...@gmail.com> wrote:

More or less. There's a few more changes I'll make, but most of the tests are passing now.

Just to keep you updated on this: it's now failing just one test on Python 2, and three on Python 3. The test_rdfxml.testPositive failure appears with the main development branch as well.

Of the extra two failures on Python 3, one is a simple doctest issue that shouldn't affect functionality. The other has me completely stumped, but I think it must be affected by one of the previous tests - when I run the file (test_graph.py) individually, all its tests pass.

The full output of the tests on Python 3 is here: http://paste.pocoo.org/show/513229/

Thanks,
Thomas

[Graham Higgins]

Sorry about the delay in responding, I've been fully engaged on another matter ...

On Wednesday, 16 November 2011 05:52:50 UTC, Gunnar Aastrand Grimnes wrote:

Previously someone else made a py3 compatible rdflib, I've put it
here: https://bitbucket.org/gromgull/rdflib-python3

but I must admit I've not really tested or maintaned it.

As Thomas observed, Michele's port is py3-only. A little while ago I brought it up to date w.r.t. changes in rdflib that have been made since Michele contributed the code in early March of this year and I have recently included the code as a separate rdflib branch (py3port). I have it in a bitbucket repos [1] which is polled by a Jenkins instance (courtesy of shiningpanda) that shows the current state of test failures [2]. 

[1] https://bitbucket.org/gjhiggins/rdflib/changesets/tip/branch(%22py3port%22)

[2] https://jenkins.shiningpanda.com/rdflib/job/py3rdflib/13/console

Using py3 under shiningpanda's Jenkins revealed a related py2-py3 issue for rdflib - bsddb is no longer part of Python3 although an installable bsddb3 package is available. I had to nuke the sleepycat rdflib store plugin in order to get rdflib to run on shiningpanda's standard py3 distro. At the very least, this implies that the py3 version of rdflib has a bsddb3 dependency that needs to be reflected in the setup.py.

You are saying that if we apply your handful of changesets rdflib will
run fine under py3 and 2.X?
And that you've not changed anything major?

fwiw, I don't see anything major that's been changed. 

I'm seeing the same test results as those reported by Thomas and they broadly chime with the results from Michele's py3 port. The 13 skipped tests are from the swap_n3 test suite that I (unilaterally) added a few months ago in an attempt to extend the scope of the test harness. On reflection, I realise that particular test suite may or may not be considered pertinent to rdflib but either way, the skipped tests aren't really relevant to Thomas' port.

I should point out that Michele offered an alternative approach to tackling term.py [3] but so far it has inspired no comments that I can see.

[3] http://code.google.com/p/rdflib/source/browse/rdflib/term_new.py?name=py3port

The approach offered by Thomas' port seems to satisfy (the net effect of) the suggestions made by Niklas Lindström back in March [4]

[4] https://groups.google.com/d/msg/rdflib-dev/LFxzYEoNq_o/PzvCNsHyG10J

so yes, I think a merge would be useful.

Cheers,

Graham.

[Gunnar Aastrand Grimnes]

I finally got around to testing Thomas code with some existing
projects that use python2.6 and it seems to work fine!

Any reason why we do not just give Thomas commit right to rdflib and
let him merge it?

- Gunnar

> --
> You received this message because you are subscribed to the Google Groups
> "rdflib-dev" group.

> To view this discussion on the web visit
> https://groups.google.com/d/msg/rdflib-dev/-/FxlAWqP9jkMJ.

[Thomas Kluyver]

Thanks Graham,

On 28 November 2011 12:56, Graham Higgins <gjhi...@gmail.com> wrote:

Using py3 under shiningpanda's Jenkins revealed a related py2-py3 issue for rdflib - bsddb is no longer part of Python3 although an installable bsddb3 package is available. I had to nuke the sleepycat rdflib store plugin in order to get rdflib to run on shiningpanda's standard py3 distro. At the very least, this implies that the py3 version of rdflib has a bsddb3 dependency that needs to be reflected in the setup.py.

I also ran into this when testing, but I hadn't twigged that bsddb was part of the Python 2 standard library. Luckily Ubuntu has a python3-bsddb3 package, so I just installed that and carried on. Clearly that will need to be listed as an optional dependency.

I should point out that Michele offered an alternative approach to tackling term.py [3] but so far it has inspired no comments that I can see.

That's interesting. Comparing literals with different types is something I'm a bit uneasy about in my code. For now, I've simply tried to replicate the behaviour of the Python 2 code as closely as possible, but I'm not sure how much of it is specified behaviour, and how much is just incidental.

For instance, a string Literal with a datatype will sort out before a string literal without one, because the machinery ends up comparing a tuple to a unicode object, and Python 2 sorts types alphabetically, so 'tuple' < 'unicode'. At least one doctest relies on this behaviour, but I don't think it's intended to check that.

Best wishes,
Thomas

[Gunnar Aastrand Grimnes]

Thomas I've added you with your gmail address as a committed of rdflib
and rdfextras!

When you have time, you can pull in your own changes!

Cheers,
- Gunnar

[Thomas Kluyver]

Thanks, Gunnar, I'll try to get round to it soon.

Best wishes,
Thomas

[Thomas Kluyver]

A quick heads up - I've merged the Python 3 branch into trunk. I re-ran the tests, and things seem OK, but let me know if you spot anything that's broken.

I've added a shell script to run the tests. You'll need Python 3 installed, and the bsddb3 package (for the sleepycat tests - although I'll look into skipping those when it's missing). On Ubuntu/Debian, you can do:

    sudo apt-get install python3 python3-bsddb3

Now that things are basically working, there are some areas I'd like to improve, such as ordering of Literals, and parsing files opened in text mode.

Thanks,
Thomas