Hypatia alternative?
30 views
Skip to first unread message
Thierry Florac
Apr 17, 2022, 7:14:37 AM4/17/22
to pylons-...@googlegroups.com
Hi,
I'm actually using the Hypatia package for content indexing and I'm very happy with it, but this package doesn't seem to be updated for a very long time (2 years since last commit).
Actually, my own packages unit tests are failing when using Python 3.10 with Hypatia, so I was wondering if it is actually used and maintained, or is there any alternative ?
Best regards,
Thierry
Steve Piercy
Apr 17, 2022, 8:10:00 AM4/17/22
to pylons-...@googlegroups.com, Thierry Florac
>
> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com <mailto:pylons-discus...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com <https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Steve Piercy
Apr 17, 2022, 8:21:04 AM4/17/22
to pylons-...@googlegroups.com
Reading:
https://hypatia.readthedocs.io/en/latest/genealogy.html
Unfortunately, this ex-package appears to be pushing up the daisies, if it wasn't nailed to the perch.
You could also check out these two packages, which appear to be actively developed:
https://github.com/zopefoundation/zope.index
https://github.com/zopefoundation/zope.catalog
I would suggest asking whether there is any successor in Plone and Zope community (now that Plone absorbed Zope).
https://community.plone.org/
--steve
On 4/17/22 4:14 AM, Thierry Florac wrote:
https://hypatia.readthedocs.io/en/latest/genealogy.html
Unfortunately, this ex-package appears to be pushing up the daisies, if it wasn't nailed to the perch.
You could also check out these two packages, which appear to be actively developed:
https://github.com/zopefoundation/zope.index
https://github.com/zopefoundation/zope.catalog
I would suggest asking whether there is any successor in Plone and Zope community (now that Plone absorbed Zope).
https://community.plone.org/
--steve
On 4/17/22 4:14 AM, Thierry Florac wrote:
Thierry Florac
Apr 17, 2022, 8:35:37 AM4/17/22
to pylons-...@googlegroups.com
Hi Steve,
I'm aware of these packages, for using Zope for many years!
But Hypatia was providing some improvements (as far as I can estimate) over these packages, and was perfectly accorded to my own needs...
Am I the only one using this package?
Could it be possible to maintain this package and provide patches (just for last Python versions actually)?
Best regards,
Thierry
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/01e6ac12-58b1-2ca6-338a-2e9276e2db2d%40gmail.com.
Laurent Daverio
Apr 17, 2022, 9:09:06 AM4/17/22
to pylons-...@googlegroups.com
Hi Thierry,
when I started using Pyramid (Pylons, actually), my first idea was to
integrate the Xapian library, written in C++. Coming from the Zope
world, I tried to adopt some ideas from the ZCatalog. Xapian worked
great, except for one problem: it's a library, not a server, thus it
doesn't handle concurrent updates gracefully (doesn't handle them at
all, actually...).
So, after a little while, I decided to switch to Solr, which is a
great indexing engine written in Java. It takes a little while to get
familiar with it, but it's worth the effort. I used `scorched` for
interfacing with Python. Please note that the PyPI release doesn't
work with Python 3.10, but the Github version does. Scorched doesn't
seem to be actively maintained, unfortunately, but it works fine. I'm
aware that ElasticSearch might be more popular than Solr these days
(they're both based on the ame Apache Lucene libraries), but I haven't
had the motivation to try it out.
I you're looking for a pure Python solution, Whoosh could be a choice,
but it doesn't seem to be actively maintained either. At some point, I
tried to integrate it with Pyramid, but I didn't have enough
motivation to finalise it (I'm happy with Solr, it's superfast, and if
you're doing frontend programming, you can process its JSON responses
directly).
Please note that both Solr/ElasticSearch and Woosh offer faceting, a
feature missing from the ZCatalog.
Hope this helps,
Laurent.
when I started using Pyramid (Pylons, actually), my first idea was to
integrate the Xapian library, written in C++. Coming from the Zope
world, I tried to adopt some ideas from the ZCatalog. Xapian worked
great, except for one problem: it's a library, not a server, thus it
doesn't handle concurrent updates gracefully (doesn't handle them at
all, actually...).
So, after a little while, I decided to switch to Solr, which is a
great indexing engine written in Java. It takes a little while to get
familiar with it, but it's worth the effort. I used `scorched` for
interfacing with Python. Please note that the PyPI release doesn't
work with Python 3.10, but the Github version does. Scorched doesn't
seem to be actively maintained, unfortunately, but it works fine. I'm
aware that ElasticSearch might be more popular than Solr these days
(they're both based on the ame Apache Lucene libraries), but I haven't
had the motivation to try it out.
I you're looking for a pure Python solution, Whoosh could be a choice,
but it doesn't seem to be actively maintained either. At some point, I
tried to integrate it with Pyramid, but I didn't have enough
motivation to finalise it (I'm happy with Solr, it's superfast, and if
you're doing frontend programming, you can process its JSON responses
directly).
Please note that both Solr/ElasticSearch and Woosh offer faceting, a
feature missing from the ZCatalog.
Hope this helps,
Laurent.
> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com.
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
Steve Piercy
Apr 17, 2022, 9:11:22 AM4/17/22
to pylons-...@googlegroups.com
I have no idea who uses Hypatia. I suppose you could use Google's Big Query to see how many PyPI downloads were made.
You could fork it, update it for Python 3.10 for your personal use, and submit pull requests back to it. I don't know if the maintainers are active.
--steve
On 4/17/22 5:35 AM, Thierry Florac wrote:
> Hi Steve,
> I'm aware of these packages, for using Zope for many years!
> But Hypatia was providing some improvements (as far as I can estimate) over these packages, and was perfectly accorded to my own needs...
> Am I the only one using this package?
> Could it be possible to maintain this package and provide patches (just for last Python versions actually)?
> Best regards,
> Thierry
> --
>
> https://github.com/zopefoundation/zope.catalog <https://github.com/zopefoundation/zope.catalog>
> > To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com <https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com> <https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/01e6ac12-58b1-2ca6-338a-2e9276e2db2d%40gmail.com <https://groups.google.com/d/msgid/pylons-discuss/01e6ac12-58b1-2ca6-338a-2e9276e2db2d%40gmail.com>.
You could fork it, update it for Python 3.10 for your personal use, and submit pull requests back to it. I don't know if the maintainers are active.
--steve
On 4/17/22 5:35 AM, Thierry Florac wrote:
> Hi Steve,
> I'm aware of these packages, for using Zope for many years!
> But Hypatia was providing some improvements (as far as I can estimate) over these packages, and was perfectly accorded to my own needs...
> Am I the only one using this package?
> Could it be possible to maintain this package and provide patches (just for last Python versions actually)?
> Best regards,
> Thierry
> --
> https://www.ulthar.net <https://www.ulthar.net> -- http://pyams.readthedocs.io <http://pyams.readthedocs.io>
>
>
> Le dim. 17 avr. 2022 à 14:21, Steve Piercy <steve.pi...@gmail.com <mailto:steve.pi...@gmail.com>> a écrit :
>
> Reading:
>
> https://hypatia.readthedocs.io/en/latest/genealogy.html <https://hypatia.readthedocs.io/en/latest/genealogy.html>
>
>
> Le dim. 17 avr. 2022 à 14:21, Steve Piercy <steve.pi...@gmail.com <mailto:steve.pi...@gmail.com>> a écrit :
>
> Reading:
>
>
> Unfortunately, this ex-package appears to be pushing up the daisies, if it wasn't nailed to the perch.
>
> You could also check out these two packages, which appear to be actively developed:
>
> https://github.com/zopefoundation/zope.index <https://github.com/zopefoundation/zope.index>
> Unfortunately, this ex-package appears to be pushing up the daisies, if it wasn't nailed to the perch.
>
> You could also check out these two packages, which appear to be actively developed:
>
>
> https://github.com/zopefoundation/zope.catalog <https://github.com/zopefoundation/zope.catalog>
>
> I would suggest asking whether there is any successor in Plone and Zope community (now that Plone absorbed Zope).
>
> https://community.plone.org/ <https://community.plone.org/>
> I would suggest asking whether there is any successor in Plone and Zope community (now that Plone absorbed Zope).
>
>
> --steve
>
>
> On 4/17/22 4:14 AM, Thierry Florac wrote:
> > Hi,
> >
> > I'm actually using the Hypatia package for content indexing and I'm very happy with it, but this package doesn't seem to be updated for a very long time (2 years since last commit).
> > Actually, my own packages unit tests are failing when using Python 3.10 with Hypatia, so I was wondering if it is actually used and maintained, or is there any alternative ?
> >
> > Best regards,
> > Thierry
> > --
> > https://www.ulthar.net <https://www.ulthar.net> <https://www.ulthar.net <https://www.ulthar.net>> -- http://pyams.readthedocs.io <http://pyams.readthedocs.io> <http://pyams.readthedocs.io <http://pyams.readthedocs.io>>
> --steve
>
>
> On 4/17/22 4:14 AM, Thierry Florac wrote:
> > Hi,
> >
> > I'm actually using the Hypatia package for content indexing and I'm very happy with it, but this package doesn't seem to be updated for a very long time (2 years since last commit).
> > Actually, my own packages unit tests are failing when using Python 3.10 with Hypatia, so I was wondering if it is actually used and maintained, or is there any alternative ?
> >
> > Best regards,
> > Thierry
> > --
> >
> > --
> > You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com <mailto:pylons-discuss%2Bunsu...@googlegroups.com> <mailto:pylons-discus...@googlegroups.com <mailto:pylons-discuss%2Bunsu...@googlegroups.com>>.
> > --
> > You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> > To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com <https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com> <https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com?utm_medium=email&utm_source=footer <https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWBOptsdANDDQZiZGEaXqVnWfv2Oa4aFty4YHD2%2BT-60_Q%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
>
> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com <mailto:pylons-discuss%2Bunsu...@googlegroups.com>.
> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/01e6ac12-58b1-2ca6-338a-2e9276e2db2d%40gmail.com <https://groups.google.com/d/msgid/pylons-discuss/01e6ac12-58b1-2ca6-338a-2e9276e2db2d%40gmail.com>.
>
> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> --
> You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com <mailto:pylons-discus...@googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWCmd_aCWaQrEXi5bSFvhH0D%2Bf%3DhrJ2sqUc%3DUqXGHZm4iw%40mail.gmail.com <https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWCmd_aCWaQrEXi5bSFvhH0D%2Bf%3DhrJ2sqUc%3DUqXGHZm4iw%40mail.gmail.com?utm_medium=email&utm_source=footer>.
Thierry Florac
Apr 17, 2022, 11:11:24 AM4/17/22
to pylons-...@googlegroups.com
Hi Laurent,
I'm actually using Elasticsearch, with a custom integration package to include it in my framework!
It works very well, but I'm also looking for a default alternative, "internal" and lighter, as Hypatia does, which doesn't rely on an external service to handle objects cataloging, and which is really sufficient to handle indexing of small to medium websites...
Maybe I'll try to fork and make a merge request; Python 3.10 is not my actual target actually but it will probably be an option in the near future!
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAB7cU6xX51RLsgetgTbWsgzJfzHaF_t0RcZVi2cPWVOTi0tEtQ%40mail.gmail.com.
Laurent Daverio
Apr 17, 2022, 12:11:07 PM4/17/22
to pylons-...@googlegroups.com
Ah yes, that was my rationale for looking at Whoosh. I figured it
would be nice to have a self-contained solution which could run e.g.
on a Raspberry Pi, with SQLite instead of PostgreSQL, and Whoosh
instead of Solr. But I remember running into some interfacing problems
(Whoosh and Solr don't look quite the same from the outside), and the
work has been dormant for a couple of years.
Actually I'm not sure about the status of Whoosh - Just checked, the
last PyPI release was in 2016, but the last Github commit was only 3
months ago.
> To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWDcwOnubxoVA0ga6hGn9VFRT-z3eJ%2BWtqdq16r7RLBndQ%40mail.gmail.com.
would be nice to have a self-contained solution which could run e.g.
on a Raspberry Pi, with SQLite instead of PostgreSQL, and Whoosh
instead of Solr. But I remember running into some interfacing problems
(Whoosh and Solr don't look quite the same from the outside), and the
work has been dormant for a couple of years.
Actually I'm not sure about the status of Whoosh - Just checked, the
last PyPI release was in 2016, but the last Github commit was only 3
months ago.
Chris McDonough
Apr 17, 2022, 9:15:59 PM4/17/22
to pylons-...@googlegroups.com
I would likely be able to review any patches made to Hypatia
------- Original Message -------
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWDcwOnubxoVA0ga6hGn9VFRT-z3eJ%2BWtqdq16r7RLBndQ%40mail.gmail.com.
Thierry Florac
Apr 18, 2022, 3:56:50 AM4/18/22
to pylons-...@googlegroups.com
Hi Chris,
That would be very nice! :)
Actually, my own CI tests are passing on Python 3.10 when using PURE_PYTHON environment variable; the problem seems to be in the Okapi scoring extension. I'll try to have a look at this but I'm not used to working on Python C extension...
Chris McDonough
Apr 18, 2022, 4:40:35 AM4/18/22
to pylons-...@googlegroups.com
Fwiw that extension was stolen from zope.catalog, it's probably fixed there.
-------- Original Message --------
-------- Original Message --------
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWA%2BxBNxvoL%3DqRjTj7%3DZmwR_O6kFGEzaVw%3DGsjHVsAHjUw%40mail.gmail.com.
Thierry Florac
Apr 18, 2022, 7:48:44 AM4/18/22
to pylons-...@googlegroups.com
I have created a patch and a pull request...
If it's OK and if you could create a new release that would be very nice !! :)
Chris McDonough
Apr 18, 2022, 8:10:04 AM4/18/22
to pylons-...@googlegroups.com
------- Original Message -------
On Monday, April 18th, 2022 at 7:48 AM, Thierry Florac <tfl...@gmail.com> wrote:
> I have created a patch and a pull request...If it's OK and if you could create a new release that would be very nice !! :)
Thanks for that! I'm installing all the necessary Python versions to check that it passes testing under all now.
Chris McDonough
Apr 18, 2022, 9:37:46 AM4/18/22
to pylons-...@googlegroups.com
py27: commands succeeded
ERROR: py34: InvocationError for command /home/chrism/projects/hypatia/.tox/py34/bin/python -m pip install --no-deps -U .tox/.tmp/package/1/hypatia-0.4.dev0.zip (exited with code 1)
ERROR: py35: InvocationError for command /home/chrism/projects/hypatia/.tox/py35/bin/python -m pip install --no-deps -U .tox/.tmp/package/1/hypatia-0.4.dev0.zip (exited with code -11 (SIGSEGV)) (exited with code -11)
ERROR: py36: InvocationError for command /home/chrism/projects/hypatia/.tox/py36/bin/python -m pip install --no-deps -U .tox/.tmp/package/1/hypatia-0.4.dev0.zip (exited with code 1)
py37: commands succeeded
py38: commands succeeded
py39: commands succeeded
py310: commands succeeded
ERROR: cover: commands failed
ERROR: docs: commands failed
Chris McDonough
Apr 18, 2022, 1:12:58 PM4/18/22
to pylons-...@googlegroups.com
> Gonna be a little while, the bitrot is pretty severe. :) But should be no (or only minor) issue with the code itself.
>
> py27: commands succeeded
> ERROR: py34: InvocationError for command /home/chrism/projects/hypatia/.tox/py34/bin/python -m pip install --no-deps -U .tox/.tmp/package/1/hypatia-0.4.dev0.zip (exited with code 1)
> ERROR: py35: InvocationError for command /home/chrism/projects/hypatia/.tox/py35/bin/python -m pip install --no-deps -U .tox/.tmp/package/1/hypatia-0.4.dev0.zip (exited with code -11 (SIGSEGV)) (exited with code -11)
> ERROR: py36: InvocationError for command /home/chrism/projects/hypatia/.tox/py36/bin/python -m pip install --no-deps -U .tox/.tmp/package/1/hypatia-0.4.dev0.zip (exited with code 1)
> py37: commands succeeded
> py38: commands succeeded
> py39: commands succeeded
> py310: commands succeeded
> ERROR: cover: commands failed
> ERROR: docs: commands failed
Chris McDonough
Apr 18, 2022, 1:15:14 PM4/18/22
to pylons-...@googlegroups.com
> All green now on hypatia master (delta travis, which I forget how to use, but it doesn't matter, local tests and coverage pass), thanks for the patch! I'll try to remember how to make a release tonight. :)
Ah, sorry, one other thing, could you make a PR adding your name to CONTRIBUTORS.rst?
Thierry Florac
Apr 19, 2022, 10:53:39 AM4/19/22
to pylons-...@googlegroups.com
Done!
Le lun. 18 avr. 2022 à 19:15, Chris McDonough <chr...@plope.com> a écrit :
> All green now on hypatia master (delta travis, which I forget how to use, but it doesn't matter, local tests and coverage pass), thanks for the patch! I'll try to remember how to make a release tonight. :)
Ah, sorry, one other thing, could you make a PR adding your name to CONTRIBUTORS.rst?
--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
Peter Wilkinson
Apr 20, 2022, 1:57:29 AM4/20/22
to pylons-...@googlegroups.com
Hi Thierry, Chris.
I’ve been working with a fork of Hypatia for a while, for our use case it is excellent - I’ve not felt comfortable pushing to ’take over’ in any way as my time to work on it comes and goes. I’ve updated for more recent python versions and also added a spatial index type. Feel free to have a look around in https://github.com/pfw/hypatia
Peter W.
--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
Chris McDonough
Apr 23, 2022, 4:47:16 AM4/23/22
to pylons-...@googlegroups.com
hypatia 0.4 should now be on PyPI. Thanks!
------- Original Message -------
To view this discussion on the web visit https://groups.google.com/d/msgid/pylons-discuss/CAPX_VWB9KD1bKNQ2fpFHEZ5ZeDHjkat-AkPqrB6qxaG9w3QDjQ%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages