Error compiling 'pyebset'

[Charlie Zha]

Hi, gang,

I am trying to install hustle on Ubuntu 14.04.  In the process, I encountered the following error, in short, error in compiling 'pyebset':

building 'pyebset' extension
i686-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DMODULE_VERSION="0.3.0" -I./libebset -I/usr/include/python2.7 -c libebset/pyebset.cpp -o build/temp.linux-i686-2.7/libebset/pyebset.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
libebset/pyebset.cpp: In function ‘PyObject* __pyx_gb_7pyebset_6BitSet_36generator(__pyx_GeneratorObject*, PyObject*)’:
libebset/pyebset.cpp:2422:31: error: no match for ‘operator=’ (operand types are ‘std::vector<long long unsigned int>::iterator {aka __gnu_cxx::__normal_iterator<long long unsigned int*, std::vector<long long unsigned int> >}’ and ‘std::vector<unsigned int>::iterator {aka __gnu_cxx::__normal_iterator<unsigned int*, std::vector<unsigned int> >}’)
...

Detailed output is attached to this message.

Anyone know how to fix this?

Best,
Charlie

[Tim Spurway]

Hi Charlie,

I haven't seen this one before, though we are trying to wrap a c++ library (ebset) with cython on a number of os/arch combos, so something is bound to go wrong :-)

* which version of cython is installed on your box (pip list | grep Cython)

* which version of gcc are you running (gcc --version)

The other c++ library that we wrap with cython is liblru.  I would like to see if that one works to eliminate the possibility that it is a libebset problem.  You can do this by editing hustle/deps/setup.py and commenting out the line

         ebitset,

(it should be line 106).

And try the install again.  Let me know if it dies trying to compile liblru.

tim

--
You received this message because you are subscribed to the Google Groups "hustle-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hustle-users...@googlegroups.com.
To post to this group, send email to hustle...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nan Jiang]

Hmm, what's your cython version?

Just compiled successfully on my Ubuntu 14.04 box.

$ g++ --version

g++ (Ubuntu 4.8.2-19ubuntu1) 4.8.2
$ cython --version

Cython version 0.19.1

[Charlie Zha]

Hi, Nan,

The following are versions of g++ and cython:

$ g++ --version
g++ (Ubuntu 4.8.2-19ubuntu1) 4.8.2

Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ cython --version
Cython version 0.21

It seems my cython is newer than your installation.

[Charlie Zha]

Hi, Tim,

After commenting out ebitset, everything else worked like magic.

So, it's ebitset only issue.

Best,
Charlie

[Charlie Zha]

Hi, Nan,

I got cython 0.21 by "sudo apt-get install".  Should I downgrade?

Best,
Charlie

[Tim Spurway]

Hi Charlie, 

I don't think downgrading is the issue, I am compiling this fine on my osx box with the same version of cython as you.  

Nan, could you upgrade to 21 and try again?

The next step would be to actually go into the cpp generated file and figure out which statement/declaration in the .pyx is generating the erroneous code.   Nan, do you know if there is a cython switch we can turn on to generate more info to help us with this?

tim

[Charlie Zha]

Hi, Tim,

Attached please find cython generated source file and messages from running "make".

Maybe a diff with your Mac OS generated source could reveal something.

Best,
Charlie

[Tim Spurway]

Thanks, Charlie

So I compared the osx output, and yeah, there is a significant difference between the two CPP files.

I also figured out where in the cython code it is finding the problem:

    def __richcmp__(BitSet l, BitSet r, int op):

        cdef bint e

        if op == 2:

            e = (deref(l.thisptr) == deref(r.thisptr))   # error #1 

        elif op == 3:

            e = (deref(l.thisptr) != deref(r.thisptr))   # error #2

        else:

            raise AttributeError("Unsupported operators.")

        return e

the errors are being flagged against the lines indicated.  I don't know enough about cython or how this is implemented.  Nan?

[Tim Spurway]

Charlie, 

Which version and branch of the Hustle repo are you using?

cd <hustle_directory>

git log

[Charlie Zha]

I encountered compilation problem from git cloned version. So, I went to find stable release and downloaded hustle-0.1.tar.gz.

I just git clone 0.2.6 and attached pybset.cpp and error messages to this message.

The following is output of 'git log' 0.2.6:

$ git log
commit b769b892a161d911b3c4d1db99d51ee0ab020278
Merge: 5f45ac8 d98c50c
Author: Tim Spurway <tspu...@gmail.com>
Date:   Wed Nov 5 15:47:34 2014 -0500

    Merge pull request #59 from oldmantaiter/update-repo-switch
   
    Remove HipChat link, point documentation to tspurway.github.io

commit d98c50c9e8651cadd36a1081380d11d808047571
Author: Tait Clarridge <ta...@clarridge.ca>
Date:   Wed Nov 5 12:53:18 2014 -0500

    Remove HipChat link, point documentation to tspurway.github.io
   
    The HipChat room is no longer being monitored, therefore we are removing it.

commit 5f45ac8ca0bb775f9b39b735de90e5b1386f7e26
Author: ncloudioj <njia...@gmail.com>
Date:   Tue Jul 29 16:38:53 2014 -0400

    Bump version to 0.2.6

[Charlie Zha]

I just ran diff of two versions of generated pyebset.py as follows, in deps directory of hustle 0.2.6:

diff libebset/pyebset.cpp ~/work/hustle-0.1/deps/libebset/pyebset.cpp > diff.txt

Please see attached for the diff output. Hope it's helpful.

[Tim Spurway]

Hi Charlie,

So you definitely need to be using the latest version of Hustle 0.2.6 (the HEAD of the master branch is OK, or the 0.2.6 tag, if you prefer).  If this is causing an error on installation, we should be working on that error.  Any tar/zip archives you find of Hustle are going to be very old and out of date.

tim

[Charlie Zha]

Tim,

Same issue with 0.2.6.  See attached files.

Best,

Charlie

[Nan Jiang]

Hey Charlie,

Just compiled successfully under Cython 0.21.

Looks like you are running a 32 bit OS. We'll take a look and fix it.

Note that for the performance concern, we'd recommend to use Hustle on the 64 bit box.

Cheers,

Nan

[Tim Spurway]

Good point, Nan.

I don't think we have tested hustle on 32 bit hardware at all, and I wouldn't expect it to run well in a 4GB address space.

[Charlie Zha]

Nan,

You are right on.

On the other hand, since it's in C and Python, I did not expect any issues with being 32-bit.

[Tim Spurway]

Hi Charlie,

We have a new release up on github (0.2.7) that is much friendlier to 32 bit systems.  Nan made the patch this morning.  Give it a try and tell us how it works out for you!

cheers,

tim

[Charlie Zha]

Hi, Tim and Nan,

Thank you for your prompt response.

I will download and try again later this afternoon.

There is few, if any, production systems with 32-bit architecture. Even 32-bit laptops are hard to come by. Still, your effort is appreciated.

Best,

Charlie

[Charlie Zha]

It worked!!!

Thank you, Nan and Tim.