Hypatia alternative?

12 views
Skip to first unread message

Thierry Florac

unread,
Apr 17, 2022, 7:14:37 AMApr 17
to pylons-...@googlegroups.com
Hi,

I'm actually using the Hypatia package for content indexing and I'm very happy with it, but this package doesn't seem to be updated for a very long time (2 years since last commit).
Actually, my own packages unit tests are failing when using Python 3.10 with Hypatia, so I was wondering if it is actually used and maintained, or is there any alternative ?

Best regards,
Thierry

Steve Piercy

unread,
Apr 17, 2022, 8:10:00 AMApr 17
to pylons-...@googlegroups.com, Thierry Florac

Steve Piercy

unread,
Apr 17, 2022, 8:21:04 AMApr 17
to pylons-...@googlegroups.com
Reading:

https://hypatia.readthedocs.io/en/latest/genealogy.html

Unfortunately, this ex-package appears to be pushing up the daisies, if it wasn't nailed to the perch.

You could also check out these two packages, which appear to be actively developed:

https://github.com/zopefoundation/zope.index

https://github.com/zopefoundation/zope.catalog

I would suggest asking whether there is any successor in Plone and Zope community (now that Plone absorbed Zope).

https://community.plone.org/

--steve


On 4/17/22 4:14 AM, Thierry Florac wrote:

Thierry Florac

unread,
Apr 17, 2022, 8:35:37 AMApr 17
to pylons-...@googlegroups.com
Hi Steve,
I'm aware of these packages, for using Zope for many years!
But Hypatia was providing some improvements (as far as I can estimate) over these packages, and was perfectly accorded to my own needs...
Am I the only one using this package?
Could it be possible to maintain this package and provide patches (just for last Python versions actually)?
Best regards,
Thierry


To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/01e6ac12-58b1-2ca6-338a-2e9276e2db2d%40gmail.com.

Laurent Daverio

unread,
Apr 17, 2022, 9:09:06 AMApr 17
to pylons-...@googlegroups.com
Hi Thierry,

when I started using Pyramid (Pylons, actually), my first idea was to
integrate the Xapian library, written in C++. Coming from the Zope
world, I tried to adopt some ideas from the ZCatalog. Xapian worked
great, except for one problem: it's a library, not a server, thus it
doesn't handle concurrent updates gracefully (doesn't handle them at
all, actually...).

So, after a little while, I decided to switch to Solr, which is a
great indexing engine written in Java. It takes a little while to get
familiar with it, but it's worth the effort. I used `scorched` for
interfacing with Python. Please note that the PyPI release doesn't
work with Python 3.10, but the Github version does. Scorched doesn't
seem to be actively maintained, unfortunately, but it works fine. I'm
aware that ElasticSearch might be more popular than Solr these days
(they're both based on the ame Apache Lucene libraries), but I haven't
had the motivation to try it out.

I you're looking for a pure Python solution, Whoosh could be a choice,
but it doesn't seem to be actively maintained either. At some point, I
tried to integrate it with Pyramid, but I didn't have enough
motivation to finalise it (I'm happy with Solr, it's superfast, and if
you're doing frontend programming, you can process its JSON responses
directly).

Please note that both Solr/ElasticSearch and Woosh offer faceting, a
feature missing from the ZCatalog.

Hope this helps,

Laurent.
> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com.

Steve Piercy

unread,
Apr 17, 2022, 9:11:22 AMApr 17
to pylons-...@googlegroups.com
I have no idea who uses Hypatia. I suppose you could use Google's Big Query to see how many PyPI downloads were made.

You could fork it, update it for Python 3.10 for your personal use, and submit pull requests back to it. I don't know if the maintainers are active.

--steve


On 4/17/22 5:35 AM, Thierry Florac wrote:
> Hi Steve,
> I'm aware of these packages, for using Zope for many years!
> But Hypatia was providing some improvements (as far as I can estimate) over these packages, and was perfectly accorded to my own needs...
> Am I the only one using this package?
> Could it be possible to maintain this package and provide patches (just for last Python versions actually)?
> Best regards,
> Thierry
> --
> https://www.ulthar.net <https://www.ulthar.net> -- http://pyams.readthedocs.io <http://pyams.readthedocs.io>
>
>
> Le dim. 17 avr. 2022 à 14:21, Steve Piercy <steve.pi...@gmail.com <mailto:steve.pi...@gmail.com>> a écrit :
>
> Reading:
>
> https://hypatia.readthedocs.io/en/latest/genealogy.html <https://hypatia.readthedocs.io/en/latest/genealogy.html>
>
> Unfortunately, this ex-package appears to be pushing up the daisies, if it wasn't nailed to the perch.
>
> You could also check out these two packages, which appear to be actively developed:
>
> https://github.com/zopefoundation/zope.index <https://github.com/zopefoundation/zope.index>
>
> https://github.com/zopefoundation/zope.catalog <https://github.com/zopefoundation/zope.catalog>
>
> I would suggest asking whether there is any successor in Plone and Zope community (now that Plone absorbed Zope).
>
> https://community.plone.org/ <https://community.plone.org/>
>
> --steve
>
>
> On 4/17/22 4:14 AM, Thierry Florac wrote:
> > Hi,
> >
> > I'm actually using the Hypatia package for content indexing and I'm very happy with it, but this package doesn't seem to be updated for a very long time (2 years since last commit).
> > Actually, my own packages unit tests are failing when using Python 3.10 with Hypatia, so I was wondering if it is actually used and maintained, or is there any alternative ?
> >
> > Best regards,
> > Thierry
> > --
> > https://www.ulthar.net <https://www.ulthar.net> <https://www.ulthar.net <https://www.ulthar.net>> -- http://pyams.readthedocs.io <http://pyams.readthedocs.io> <http://pyams.readthedocs.io <http://pyams.readthedocs.io>>
> >
> > --
> > You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com <mailto:pylons-discuss%2Bunsu...@googlegroups.com> <mailto:pylons-discus...@googlegroups.com <mailto:pylons-discuss%2Bunsu...@googlegroups.com>>.
> > To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com <https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com> <https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
>
> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com <mailto:pylons-discuss%2Bunsu...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/01e6ac12-58b1-2ca6-338a-2e9276e2db2d%40gmail.com <https://groups.google.com/d/msgid/pylons-discuss/01e6ac12-58b1-2ca6-338a-2e9276e2db2d%40gmail.com>.
>
> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com <mailto:pylons-discus...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWCmd_aCWaQrEXi5bSFvhH0D%2Bf%3DhrJ2sqUc%3DUqXGHZm4iw%40mail.gmail.com <https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWCmd_aCWaQrEXi5bSFvhH0D%2Bf%3DhrJ2sqUc%3DUqXGHZm4iw%40mail.gmail.com?utm_medium=email&utm_source=footer>.

Thierry Florac

unread,
Apr 17, 2022, 11:11:24 AMApr 17
to pylons-...@googlegroups.com
Hi Laurent,

I'm actually using Elasticsearch, with a custom integration package to include it in my framework!
It works very well, but I'm also looking for a default alternative, "internal" and lighter, as Hypatia does, which doesn't rely on an external service to handle objects cataloging, and which is really sufficient to handle indexing of small to medium websites...
Maybe I'll try to fork and make a merge request; Python 3.10 is not my actual target actually but it will probably be an option in the near future!

Best regards,
Thierry

Laurent Daverio

unread,
Apr 17, 2022, 12:11:07 PMApr 17
to pylons-...@googlegroups.com
Ah yes, that was my rationale for looking at Whoosh. I figured it
would be nice to have a self-contained solution which could run e.g.
on a Raspberry Pi, with SQLite instead of PostgreSQL, and Whoosh
instead of Solr. But I remember running into some interfacing problems
(Whoosh and Solr don't look quite the same from the outside), and the
work has been dormant for a couple of years.

Actually I'm not sure about the status of Whoosh - Just checked, the
last PyPI release was in 2016, but the last Github commit was only 3
months ago.
> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWDcwOnubxoVA0ga6hGn9VFRT-z3eJ%2BWtqdq16r7RLBndQ%40mail.gmail.com.

Chris McDonough

unread,
Apr 17, 2022, 9:15:59 PMApr 17
to pylons-...@googlegroups.com
I would likely be able to review any patches made to Hypatia

------- Original Message -------

Thierry Florac

unread,
Apr 18, 2022, 3:56:50 AMApr 18
to pylons-...@googlegroups.com
Hi Chris,
That would be very nice!  :)
Actually, my own CI tests are passing on Python 3.10 when using PURE_PYTHON environment variable; the problem seems to be in the Okapi scoring extension. I'll try to have a look at this but I'm not used to working on Python C extension...
Best regards,
Thierry

Chris McDonough

unread,
Apr 18, 2022, 4:40:35 AMApr 18
to pylons-...@googlegroups.com
Fwiw that extension was stolen from zope.catalog, it's probably fixed there.






-------- Original Message --------
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWA%2BxBNxvoL%3DqRjTj7%3DZmwR_O6kFGEzaVw%3DGsjHVsAHjUw%40mail.gmail.com.

Thierry Florac

unread,
Apr 18, 2022, 7:48:44 AMApr 18
to pylons-...@googlegroups.com
I have created a patch and a pull request...
If it's OK and if you could create a new release that would be very nice !!  :)

Best regards,
Thierry

Chris McDonough

unread,
Apr 18, 2022, 8:10:04 AMApr 18
to pylons-...@googlegroups.com



------- Original Message -------
On Monday, April 18th, 2022 at 7:48 AM, Thierry Florac <tfl...@gmail.com> wrote:


> I have created a patch and a pull request...If it's OK and if you could create a new release that would be very nice !! :)
Thanks for that! I'm installing all the necessary Python versions to check that it passes testing under all now.

Chris McDonough

unread,
Apr 18, 2022, 9:37:46 AMApr 18
to pylons-...@googlegroups.com
Gonna be a little while, the bitrot is pretty severe. :) But should be no (or only minor) issue with the code itself.

py27: commands succeeded
ERROR: py34: InvocationError for command /home/chrism/projects/hypatia/.tox/py34/bin/python -m pip install --no-deps -U .tox/.tmp/package/1/hypatia-0.4.dev0.zip (exited with code 1)
ERROR: py35: InvocationError for command /home/chrism/projects/hypatia/.tox/py35/bin/python -m pip install --no-deps -U .tox/.tmp/package/1/hypatia-0.4.dev0.zip (exited with code -11 (SIGSEGV)) (exited with code -11)
ERROR: py36: InvocationError for command /home/chrism/projects/hypatia/.tox/py36/bin/python -m pip install --no-deps -U .tox/.tmp/package/1/hypatia-0.4.dev0.zip (exited with code 1)
py37: commands succeeded
py38: commands succeeded
py39: commands succeeded
py310: commands succeeded
ERROR: cover: commands failed
ERROR: docs: commands failed

Chris McDonough

unread,
Apr 18, 2022, 1:12:58 PMApr 18
to pylons-...@googlegroups.com


> Gonna be a little while, the bitrot is pretty severe. :) But should be no (or only minor) issue with the code itself.
>
> py27: commands succeeded
> ERROR: py34: InvocationError for command /home/chrism/projects/hypatia/.tox/py34/bin/python -m pip install --no-deps -U .tox/.tmp/package/1/hypatia-0.4.dev0.zip (exited with code 1)
> ERROR: py35: InvocationError for command /home/chrism/projects/hypatia/.tox/py35/bin/python -m pip install --no-deps -U .tox/.tmp/package/1/hypatia-0.4.dev0.zip (exited with code -11 (SIGSEGV)) (exited with code -11)
> ERROR: py36: InvocationError for command /home/chrism/projects/hypatia/.tox/py36/bin/python -m pip install --no-deps -U .tox/.tmp/package/1/hypatia-0.4.dev0.zip (exited with code 1)
> py37: commands succeeded
> py38: commands succeeded
> py39: commands succeeded
> py310: commands succeeded
> ERROR: cover: commands failed
> ERROR: docs: commands failed

All green now on hypatia master (delta travis, which I forget how to use, but it doesn't matter, local tests and coverage pass), thanks for the patch! I'll try to remember how to make a release tonight. :)

Chris McDonough

unread,
Apr 18, 2022, 1:15:14 PMApr 18
to pylons-...@googlegroups.com
> All green now on hypatia master (delta travis, which I forget how to use, but it doesn't matter, local tests and coverage pass), thanks for the patch! I'll try to remember how to make a release tonight. :)

Ah, sorry, one other thing, could you make a PR adding your name to CONTRIBUTORS.rst?


Thierry Florac

unread,
Apr 19, 2022, 10:53:39 AMApr 19
to pylons-...@googlegroups.com
Done!

Best regards,
Thierry

Le lun. 18 avr. 2022 à 19:15, Chris McDonough <chr...@plope.com> a écrit :
> All green now on hypatia master (delta travis, which I forget how to use, but it doesn't matter, local tests and coverage pass), thanks for the patch! I'll try to remember how to make a release tonight. :)

Ah, sorry, one other thing, could you make a PR adding your name to CONTRIBUTORS.rst?


--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.

Peter Wilkinson

unread,
Apr 20, 2022, 1:57:29 AMApr 20
to pylons-...@googlegroups.com
Hi Thierry, Chris.

I’ve been working with a fork of Hypatia for a while, for our use case it is excellent - I’ve not felt comfortable pushing to ’take over’ in any way as my time to work on it comes and goes. I’ve updated for more recent python versions and also added a spatial index type. Feel free to have a look around in https://github.com/pfw/hypatia 

Peter W.



-- 
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.

Chris McDonough

unread,
Apr 23, 2022, 4:47:16 AMApr 23
to pylons-...@googlegroups.com
hypatia 0.4 should now be on PyPI.  Thanks!

------- Original Message -------
Reply all
Reply to author
Forward
0 new messages