About embedding cassandra thrift-gen code

[Daniel Lundin]

Considering pycassa is so tightly coupled with the thrift interface,
and there're no reliable eggs for python-cassandra, maybe including
the whole shebang (all three modules) isn't such a bad idea.

Check this out:

http://github.com/dln/pycassa/commit/ccbe3c540d25786fa88f2c44d7131dac3e0dbf9d

Too hacky?

I'm thinking it's still a better option than the current "cassandra on
the side" though, as an egg/bdist build
will always have the same contents (deprecating the --cassandra flag)
and won't pollute the global namespace, (site-packages/cassandra).

Once we sort out a "real" python-cassandra egg distribution, we can
presumably remove the bundled code altogether, but until then it makes
sense to provide good and explicit defaults.

What do you guys think?

/d

[Tyler Hobbs]

I think this sounds like a good idea for now.

- Tyler

[Jon Hermes]

Note this means a merge every time the Thrift API changes.
It can be easily scripted (compare my version to trunk's, if mismatch, ant gen-thrift-py, cp pyfiles pycassa/, commit), but I'm more worried about ever missing a version change and hearing user complaints/mystery bugs.

On Aug 19, 2010 7:14 PM, "Tyler Hobbs" <ty...@riptano.com> wrote:

I think this sounds like a good idea for now.

- Tyler

On Thu, Aug 19, 2010 at 5:23 PM, Daniel Lundin <d...@eintr.org> wrote:
>

> Considering pycassa is s...

[Eric Evans]

On Thu, Aug 19, 2010 at 5:23 PM, Daniel Lundin <d...@eintr.org> wrote:

> Considering pycassa is so tightly coupled with the thrift interface,
> and there're no reliable eggs for python-cassandra, maybe including
> the whole shebang (all three modules) isn't such a bad idea.

Yeah, this isn't ideal, but publishing an egg isn't either, not if the
idea is to support multiple versions and/or moving targets (current
stable and dev version). It might be a little wasteful and whatnot,
but I think embedding is the safest choice for now.

--
Eric Evans
john.er...@gmail.com

[Tyler Hobbs]

Yesterday I got an email saying that the bundled thrift-gen wasn't
working with setup.py.  So, I cleared out the relevant packages
from site-packages, ran setup.py, and cleared PYTHONPATH.
Importing pycassa failed on trying to import cassandra.ttypes.

I've tried adjusting setup.py to: treat cassandra as a different package,
treat bundled as a separate package, treat cassandra and bundled
as a separate package... none of it worked.

I think it comes down to this not working as expected:

try:
    from cassandra.ttypes import *
except ImportError:
    from bundled.ttypes import *

From what I understand, when we try to import our phony
cassandra.ttypes, that module gets listed in sys.module
before it even starts executing; this causes the import to
succeed after a lookup in sys.module and so an ImportError
is never raised.  I've also tried adding something like:

import cassandra.ttypes
if hasattr(cassandra.ttypes, 'Column'): then use cassandra.ttypes,
else: use bundled

with no success.

I think to fix this, we have to get rid of the phony cassandra
package, install bundled as a separate package, and change
a lot of how the imports are done inside of pycassa.

Before I start working on this: am I totally wrong here?  Does
anybody have any other ideas?

[Tyler Hobbs]

Actually, thinking about the problem more, it seems like if we are
tying specific pycassa versions to specific cassandra versions
with the new versioning scheme, we might as well bundle the
thrift-gen code and only use that; spefically, make the cassandra
package a subpackage of pycassa.  This is the approach that
telephus takes and I think that it could work well at least in the
short term.

Thoughts?
- Tyler

[Eric Evans]

On Thu, Sep 2, 2010 at 1:59 PM, Tyler Hobbs <ty...@riptano.com> wrote:
> Actually, thinking about the problem more, it seems like if we are
> tying specific pycassa versions to specific cassandra versions
> with the new versioning scheme, we might as well bundle the
> thrift-gen code and only use that; spefically, make the cassandra
> package a subpackage of pycassa.  This is the approach that
> telephus takes and I think that it could work well at least in the
> short term.
>
> Thoughts?

+1 (I thought that's what we were going to do before :o)

--
Eric Evans
john.er...@gmail.com

[Tyler Hobbs]

Hmm...

*scratches head*

Well, I suppose I'll make the changes.  Thanks for the feedback.

- Tyler