Fwd: Megatest on Mon Oct 20 2008 with 4 Errors
Christopher Lee
thanks for sending me this report! It's very easy to fix -- we'll
just add the clear_cache() method that Titus and I have discussed
previously on the forum. I'll do that.
I am trying to reproduce the blast cache problem you reported. Should
have some results soon...
-- Chris
Begin forwarded message from Namshin Kim:
>
> MEGATEST on Mon Oct 20 17:14:06 2008
>
> Traceback (most recent call last):
> File "/result/pygr_megatest/src_save/pygr/tests/protest.py", line
> 123, in <module>
> if do_test(sys.argv[2],sys.argv[3],sys.argv[4]):
> File "/result/pygr_megatest/src_save/pygr/tests/protest.py", line
> 33, in do_test
> m()
> File "/result/pygr_megatest/src_save/pygr/tests/
> annotation_dm2_megatest.py", line 105, in collectionannot_test
> exon_db.clear()
> File "/result/pygr_megatest/src_save/pygr/build/lib.linux-
> x86_64-2.5/pygr/seqdb.py", line 987, in clear
> raise NotImplementedError, "no deletions allowed"
> NotImplementedError: no deletions allowed
Christopher Lee
I added a fix for the megatest problem you were experiencing:
- added a clear_cache() method,
- switched annotation megatests to use that instead of clear().
So the annotation megatests should run now.
-- Chris
Namshin Kim
import pygrtest_common
File "/result/pygr_megatest/src_save/pygr/tests/pygrtest_common.py", line 21, in <module>
ver = ".".join(platform.python_version_tuple()[:2])
TypeError: sequence item 0: expected string, int found
ifile,ofile=os.popen2(cmd)
Traceback (most recent call last):
if do_test(sys.argv[2],sys.argv[3],sys.argv[4]):
m()
File "/result/pygr_megatest/src_save/pygr/tests/sequence_test.py", line 88, in blast_test
minAlignSize=14,pIdentityMin=0.5)
File "pygr.cnestedlist.pyx", line 599, in pygr.cnestedlist.NLMSASlice.edges
File "pygr.cnestedlist.pyx", line 831, in pygr.cnestedlist.NLMSASlice.groupByIntervals
File "pygr.cnestedlist.pyx", line 882, in pygr.cnestedlist.NLMSASlice.filterIvalConservation
TypeError: 'dict' object doesn't support item deletion
Tue Oct 21 16:52:05 PDT 2008
.......................................SSSSSSSSSSS.......SSSSSSSS..................................................SSSSSSSS
============================================================
SKIPPED: graph_test.SQLGraph_Test.basicquery_test()
SKIPPED: graph_test.SQLGraph_Test.cyclicacyclicquery_test()
SKIPPED: graph_test.SQLGraph_Test.cyclicquery_test()
SKIPPED: graph_test.SQLGraph_Test.delraise_test()
SKIPPED: graph_test.SQLGraph_Test.filteredquery_test()
SKIPPED: graph_test.SQLGraph_Test.graphdict_test()
SKIPPED: graph_test.SQLGraph_Test.graphedges_test()
SKIPPED: graph_test.SQLGraph_Test.headlessquery_test()
SKIPPED: graph_test.SQLGraph_Test.nodedel_test()
SKIPPED: graph_test.SQLGraph_Test.setitemraise_test()
SKIPPED: graph_test.SQLGraph_Test.symmetricquery_test()
SKIPPED: pygrdata_test.Seq_SQL2_Test.bind_test()
SKIPPED: pygrdata_test.Seq_SQL2_Test.dir_test()
SKIPPED: pygrdata_test.Seq_SQL2_Test.match_test()
SKIPPED: pygrdata_test.Seq_SQL2_Test.schema_test()
SKIPPED: pygrdata_test.Seq_SQL_Test.bind_test()
SKIPPED: pygrdata_test.Seq_SQL_Test.dir_test()
SKIPPED: pygrdata_test.Seq_SQL_Test.match_test()
SKIPPED: pygrdata_test.Seq_SQL_Test.schema_test()
SKIPPED: sqltable_test.SQLTable_Test.contains_test()
SKIPPED: sqltable_test.SQLTable_Test.get_test()
SKIPPED: sqltable_test.SQLTable_Test.has_key_test()
SKIPPED: sqltable_test.SQLTable_Test.items_test()
SKIPPED: sqltable_test.SQLTable_Test.iteritems_test()
SKIPPED: sqltable_test.SQLTable_Test.iterkeys_test()
SKIPPED: sqltable_test.SQLTable_Test.itervalues_test()
SKIPPED: sqltable_test.SQLTable_Test.keys_test()
FINAL: 0 error(s) in 124 tests, 27 skipped in 48.974 sec
/result/pygr_megatest/src_save/pygr/build/lib.linux-x86_64-2.5
...........
============================================================
Christopher Lee
On Oct 21, 2008, at 6:54 PM, Namshin Kim wrote:
> And I ran all tests for Python 2.6 and found some errors. Megatest
> has no errors for Python 2.6.
>
> ======== pygrtest_common.py error
> Traceback (most recent call last):
> File "protest.py", line 12, in <module>
> import pygrtest_common
> File "/result/pygr_megatest/src_save/pygr/tests/
> pygrtest_common.py", line 21, in <module>
> ver = ".".join(platform.python_version_tuple()[:2])
> TypeError: sequence item 0: expected string, int found
I guess they changed the python_version_tuple format...
> ======== fixes for pygrtest_common.py
> ver = ".".join(map(str, platform.python_version_tuple()[:2]))
Makes sense...
> ======== sequence_test.py error
> /result/pygr_megatest/src_save/pygr/build/lib.linux-x86_64-2.6/pygr/
> seqdb.py:309: DeprecationWarning:
> os.popen2 is deprecated. Use the subprocess module.
> ifile,ofile=os.popen2(cmd)
ignore for the moment. Unfortunately subprocess was only introduced
in Python 2.4.
>
> Traceback (most recent call last):
> File "protest.py", line 123, in <module>
> if do_test(sys.argv[2],sys.argv[3],sys.argv[4]):
> File "protest.py", line 33, in do_test
> m()
> File "/result/pygr_megatest/src_save/pygr/tests/sequence_test.py",
> line 88, in blast_test
> minAlignSize=14,pIdentityMin=0.5)
> File "pygr.cnestedlist.pyx", line 599, in
> pygr.cnestedlist.NLMSASlice.edges
> File "pygr.cnestedlist.pyx", line 831, in
> pygr.cnestedlist.NLMSASlice.groupByIntervals
> File "pygr.cnestedlist.pyx", line 882, in
> pygr.cnestedlist.NLMSASlice.filterIvalConservation
> TypeError: 'dict' object doesn't support item deletion
This error message is puzzling. The Python 2.6 docs explicitly
support the use of del on a dict object:
"del d[key]
Remove d[key] from d. Raises a KeyError if key is not in the map."
Also, it's hard to see how all the megatests could pass if such a
basic language feature were removed in 2.6. This message sounds like
a bug to me... but whose bug? My first guess is that this might be a
Python 2.6 bug, since it's still quite new...
C. Titus Brown
-> > File "protest.py", line 123, in <module>
-> > if do_test(sys.argv[2],sys.argv[3],sys.argv[4]):
-> > File "protest.py", line 33, in do_test
-> > m()
-> > File "/result/pygr_megatest/src_save/pygr/tests/sequence_test.py",
-> > line 88, in blast_test
-> > minAlignSize=14,pIdentityMin=0.5)
-> > File "pygr.cnestedlist.pyx", line 599, in
-> > pygr.cnestedlist.NLMSASlice.edges
-> > File "pygr.cnestedlist.pyx", line 831, in
-> > pygr.cnestedlist.NLMSASlice.groupByIntervals
-> > File "pygr.cnestedlist.pyx", line 882, in
-> > pygr.cnestedlist.NLMSASlice.filterIvalConservation
-> > TypeError: 'dict' object doesn't support item deletion
->
-> This error message is puzzling. The Python 2.6 docs explicitly
-> support the use of del on a dict object:
-> "del d[key]
-> Remove d[key] from d. Raises a KeyError if key is not in the map."
->
-> Also, it's hard to see how all the megatests could pass if such a
-> basic language feature were removed in 2.6. This message sounds like
-> a bug to me... but whose bug? My first guess is that this might be a
-> Python 2.6 bug, since it's still quite new...
This is almost certainly due to a Pyrex version upgrade; it's the same
kind of error I tracked down for the seqdb cache.
--titus
--
C. Titus Brown, c...@msu.edu
C. Titus Brown
->
-> -> > Traceback (most recent call last):
-> -> > File "protest.py", line 123, in <module>
-> -> > if do_test(sys.argv[2],sys.argv[3],sys.argv[4]):
-> -> > File "protest.py", line 33, in do_test
-> -> > m()
-> -> > File "/result/pygr_megatest/src_save/pygr/tests/sequence_test.py",
-> -> > line 88, in blast_test
-> -> > minAlignSize=14,pIdentityMin=0.5)
-> -> > File "pygr.cnestedlist.pyx", line 599, in
-> -> > pygr.cnestedlist.NLMSASlice.edges
-> -> > File "pygr.cnestedlist.pyx", line 831, in
-> -> > pygr.cnestedlist.NLMSASlice.groupByIntervals
-> -> > File "pygr.cnestedlist.pyx", line 882, in
-> -> > pygr.cnestedlist.NLMSASlice.filterIvalConservation
-> -> > TypeError: 'dict' object doesn't support item deletion
-> ->
-> -> This error message is puzzling. The Python 2.6 docs explicitly
-> -> "del d[key]
-> -> Remove d[key] from d. Raises a KeyError if key is not in the map."
-> ->
-> -> Also, it's hard to see how all the megatests could pass if such a
-> -> a bug to me... but whose bug? My first guess is that this might be a
-> -> Python 2.6 bug, since it's still quite new...
->
-> This is almost certainly due to a Pyrex version upgrade; it's the same
-> kind of error I tracked down for the seqdb cache.
OK, figured it out... this is a pyrex bug related to an optimization for
indexing maps with C integers.
The gory details follow.
--
I went through and ran the tests on each of the pyrex releases through
0.9.6.4, which was where the problem went away.
In Pyrex 0.9.7, I get this error:
File "pygr.cnestedlist.pyx", line 781, in
pygr.cnestedlist.NLMSASlice.groupByIntervals
TypeError: 'dict' object is unindexable
In Pyrex 0.9.7.1, you get this error:
File "pygr.cnestedlist.pyx", line 784, in
pygr.cnestedlist.NLMSASlice.groupByIntervals
TypeError: 'dict' object does not support item assignment
And in Pyrex 0.9.7.2, you get *this* error:
File "pygr.cnestedlist.pyx", line 884, in
pygr.cnestedlist.NLMSASlice.filterIvalConservation
TypeError: 'dict' object doesn't support item deletion
--
(easy_install makes it really easy to install multiple versions!)
--
Going into the pyrex CHANGES.txt,
http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/version/CHANGES.txt
I note further clues:
0.9.7.2 -- Another integer indexing problem fixed.
0.9.7.1 --
- The optimisation for indexing using a C int failed when the
object being indexed was a mapping rather than a sequence.
[Arc Riley]
--
So it's likely just to be a bug in Pyrex; I'll submit a bug report to
them once I can get the problem written up in a nice, small bit of pyx
code.
In the meantime, there are two possible fixes for pygr:
- don't declare targetID as a C integer (-> unoptimized performance)
- call 'seqIntervals.__delitem__(targetID)' instead of
'del seqIntervals[targetID]' .
See the attached diff for exact line numbers in cnestedlist.pyx.
Both seem to work -- as in, no complaints from Python or Pyrex in
running the tests -- and I'm slightly in favor of the first change.
Chris, let me know which you prefer and I'll wrap it up in a nice patch.
Alternatively you can just make the changes yourself and check it in:
the first fix is a one-line change, and the second is a two-line change.
sleuthing-ly yours,
C. Titus Brown
-> -> -> This error message is puzzling. The Python 2.6 docs explicitly
-> -> -> "del d[key]
-> -> -> Remove d[key] from d. Raises a KeyError if key is not in the map."
-> -> ->
-> -> -> Also, it's hard to see how all the megatests could pass if such a
-> -> -> a bug to me... but whose bug? My first guess is that this might be a
-> -> -> Python 2.6 bug, since it's still quite new...
-> ->
-> -> This is almost certainly due to a Pyrex version upgrade; it's the same
->
-> OK, figured it out... this is a pyrex bug related to an optimization for
-> indexing maps with C integers.
Specifically, this code fails:
--
cdef class Test:
def do_del(self): # fails: del d[key] with C int key
cdef int key
key = 0
d = {}
d[key] = 'foo'
del d[key]
--
I'll pass it on to the Pyrex list.
cheers,
--titus