Python Search Engine app
Harlin Seritt
Is anyone aware of an available open-source/free search engine app
(something similar to HTDig) written in Python that is out there?
Googling has turned up nothing. Thought maybe I'd mine some of you
guys' minds on this.
thanks,
Harlin Seritt
Internet Villa: www.seritt.org
Alan Meyer
"Harlin Seritt" <harlin...@yahoo.com> wrote in message
news:1126708775.6...@g43g2000cwa.googlegroups.com...
I'm not aware of such a thing.
I stand ready to be corrected, but I think Python would not be a
good language for writing search engines. In the ones I've written
for custom projects - in C or PL/1, it has been necessary to
perform very high speed operations on highly compressed binary
structures - which is not Python's forte.
You might be able to put a Python interface over an engine written
in another language.
Alan
gene tani
stemming" "stopwords" "text indexer". Maybe lucene has some python
bindings, hmm?
Will McGugan
> I'm not aware of such a thing.
>
> I stand ready to be corrected, but I think Python would not be a
> good language for writing search engines. In the ones I've written
> for custom projects - in C or PL/1, it has been necessary to
> perform very high speed operations on highly compressed binary
> structures - which is not Python's forte.
>
> You might be able to put a Python interface over an engine written
> in another language.
Wasn't Google's first search engine actualy written in Python?
Will McGugan
--
http://www.willmcgugan.com
"".join({'*':'@','^':'.'}.get(c,0) or chr(97+(ord(c)-84)%26) for c in
"jvyy*jvyyzpthtna^pbz")
Fredrik Lundh
> Is anyone aware of an available open-source/free search engine app
> (something similar to HTDig) written in Python that is out there?
> Googling has turned up nothing. Thought maybe I'd mine some of you
> guys' minds on this.
http://divmod.org/ has a couple of alternatives:
http://divmod.org/projects/lupy
http://divmod.org/projects/pyndex
http://divmod.org/projects/xapwrap
</F>
Kent Johnson
> Yes, there's a bunch. Google for "query parser" + python, "porter
> stemming" "stopwords" "text indexer". Maybe lucene has some python
> bindings, hmm?
At least two Python versions of Lucene:
http://pylucene.osafoundation.org/
http://divmod.org/projects/lupy
Kent
Alan Meyer
"Alan Meyer" <ame...@yahoo.com> wrote in message
news:BJmdnYaoCck...@comcast.com...
...
> I stand ready to be corrected, but I think Python would not be a
> good language for writing search engines. In the ones I've written
> for custom projects - in C or PL/1, it has been necessary to
> perform very high speed operations on highly compressed binary
> structures - which is not Python's forte.
>
> You might be able to put a Python interface over an engine written
> in another language.
It looks like the engines cited by the other responders to the query
are indeed mostly Python wrappings for C++ engines.
One that might not be, "lupy", notes under performance in the FAQ
the simple statement that "Java is faster".
As between Java and C++, I like Java much more as a programmer
friendly programming language, but I think I'd still choose C++ for
a search engine if the number of users or the size of the data was
large.
Alan
Peter Hansen
> Alan Meyer wrote:
>> I stand ready to be corrected, but I think Python would not be a
>> good language for writing search engines.
> Wasn't Google's first search engine actualy written in Python?
Not unless it was some kind of quickie prototype that was immediately
discarded in favour of an optimized C version once it was built. The
rumour you're probably referring to is the *web spider* part of the
system, while the indexing/searching stuff is written in C or
something close (memory fails me on that point, but I'm sure Google
itself could easily tell you more).
There's an interesting paper about the early architecture/history of
Google which was a published by the authors (when they were still
in figurative diapers) and which was referenced recently in this
newsgroup. The archives would quickly pull it up for you.
-Peter
Thomas Guettler
> Hi,
>
> Is anyone aware of an available open-source/free search engine app
> (something similar to HTDig) written in Python that is out there?
> Googling has turned up nothing. Thought maybe I'd mine some of you
> guys' minds on this.
Hi,
I use ZODB (not ZOPE) with ZCTextIndex.
HTH,
Thomas
--
Thomas Güttler, http://www.thomas-guettler.de/
E-Mail: guettli (*) thomas-guettler + de
Spam Catcher: niemand....@thomas-guettler.de
Jim
possibly make sense to use a dB? For an application of mine, I had
been looking at putting the text into PostGres, and using their
OpenFTS, at //http://openfts.sourceforge.net/ (there is a Python module
to talk to OpenFTS, although it is in an early stage of development).
Jim
asmlinkage
Will McGugan wrote:
>
> Wasn't Google's first search engine actualy written in Python?
I would'nt be surprised if they actually used python to do some
prototyping of their algorithms but I cannot imagine the search engine
actually being written in python.
Search engines consist of many sub-systems like spidering, tokenizing,
indexing, compression, ranking etc ... I think the main portion where
google or as a matter of fact many search companies would use python
is probably for spidering and for all the glue that supports the system.