ore.xapian
kapil
i released the ore.xapian package to pypi a few weeks back, and after
a few iterations i've got in production on few small applications, its
a thin layer on top of xappy to give an indexing framework for zope3
based applications.
its pretty xapian agnostic.. its designed as an async indexing
framework, with abstractions for content indexers, content storage/
resolution, transactional flush into the indexing queue, manages
reopening search connections, etc.
the pypi page goes into a bit more detail (doctest style).
http://pypi.python.org/pypi/ore.xapian
i'm using it succesfully to index content from relational databases
and subversion with a zope3 front end. only real todo is to make the
index queue persistent for remote indexers, but to be useful that
would need corresponding support for remote search connections in
xappy. unfortunately i don't have the bandwidth for the latter atm.
cheers,
kapil
Richard Boulton
> hi folks,
>
> i released the ore.xapian package to pypi a few weeks back, and after
> a few iterations i've got in production on few small applications, its
> a thin layer on top of xappy to give an indexing framework for zope3
> based applications.
>
> its pretty xapian agnostic.. its designed as an async indexing
> framework, with abstractions for content indexers, content storage/
> resolution, transactional flush into the indexing queue, manages
> reopening search connections, etc.
That sounds excellent. The only warning I'd like to make is that the
xappy API isn't yet stable, and in particular the FieldAction stuff is
likely to be redesigned shortly (I'll send a summary of my thoughts on
what I want to do with this, and why, to this list in a bit, to ensure I
get the interface right this time). I'm hoping to have the API stable
in a couple of weeks, though.
From a quick peruse of the pypi page, I see that it exports the
FieldAction style interface, so you might want to add a note that this
is liable to change in the near future. More on this when I get a
chance to write up my plans properly.
For reference, I hope to make a 0.6 release of xappy in the next two
weeks, including a reworking of the field action interface. I'll then
work towards the 0.7 release, and at the 0.7 release I expect to declare
the API stable.
See http://code.google.com/p/xappy/issues/list?sort=milestone for
details of the issues currently planned to be addressed for each release.
> i'm using it succesfully to index content from relational databases
> and subversion with a zope3 front end. only real todo is to make the
> index queue persistent for remote indexers, but to be useful that
> would need corresponding support for remote search connections in
> xappy. unfortunately i don't have the bandwidth for the latter atm.
There is one change to xapian which is required to make it possible to
perform remote searches with xappy, which is to implement metadata
support for remote databases. This is not a very hard task, but I don't
have time to do it, right now. The appropriate xapian ticket is
http://trac.xapian.org/ticket/178, and this is marked for 1.1.0; I'll
endeavour to make sure this doesn't slip from the 1.1.0 release of xapian.
--
Richard
Kapil Thangavelu
That sounds excellent. The only warning I'd like to make is that the
kapil wrote:
> hi folks,
>
> i released the ore.xapian package to pypi a few weeks back, and after
> a few iterations i've got in production on few small applications, its
> a thin layer on top of xappy to give an indexing framework for zope3
> based applications.
>
> its pretty xapian agnostic.. its designed as an async indexing
> framework, with abstractions for content indexers, content storage/
> resolution, transactional flush into the indexing queue, manages
> reopening search connections, etc.
xappy API isn't yet stable, and in particular the FieldAction stuff is
likely to be redesigned shortly (I'll send a summary of my thoughts on
what I want to do with this, and why, to this list in a bit, to ensure I
get the interface right this time). I'm hoping to have the API stable
in a couple of weeks, though.
From a quick peruse of the pypi page, I see that it exports the
FieldAction style interface, so you might want to add a note that this
is liable to change in the near future. More on this when I get a
chance to write up my plans properly.
For reference, I hope to make a 0.6 release of xappy in the next two
weeks, including a reworking of the field action interface. I'll then
work towards the 0.7 release, and at the 0.7 release I expect to declare
the API stable.
See http://code.google.com/p/xappy/issues/list?sort=milestone for
details of the issues currently planned to be addressed for each release.
There is one change to xapian which is required to make it possible to
> i'm using it succesfully to index content from relational databases
> and subversion with a zope3 front end. only real todo is to make the
> index queue persistent for remote indexers, but to be useful that
> would need corresponding support for remote search connections in
> xappy. unfortunately i don't have the bandwidth for the latter atm.
perform remote searches with xappy, which is to implement metadata
support for remote databases. This is not a very hard task, but I don't
have time to do it, right now. The appropriate xapian ticket is
http://trac.xapian.org/ticket/178, and this is marked for 1.1.0; I'll
endeavour to make sure this doesn't slip from the 1.1.0 release of xapian.