full text search and/or external search services

[andreas schmid]

hi guys,

i really need to have a full text search on my app and i wanted to know what kind of external services you are using on your apps. or how you implement such a service.
i tried whoosh-appengine but first its a really old version and second it has huge issues with 1000+ indexed entities.

we don't know when the full text search will be released for GAE right?

thx

[PandaSuit]

If you are using Python then you can use Bill Katz solution until
Google releases theirs:
http://www.billkatz.com/2009/6/Simple-Full-Text-Search-for-App-Engine

[Pascal Voitot Dev]

you can also try to use compass also! I read somewhere it can work in GAE!

but knowing when GAE FT will be out would be great!

Pascal

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[andreas schmid]

thanks for the hint but im my app is written in python and it looks like compass is for java

On Jul 28, 2011, at 9:38 PM, Pascal Voitot Dev wrote:

compass

[Barry Hunter]

There is
http://www.houndsleuth.com/

I don't actully use it with AppEngine, but it is designed for it and
looks like it should work well. It has connectors to directly index
entities in the datastore.

[Robert Kluin]

Have you tried running whoosh (not whoosh-appengine) on a backend?
Backends are expensive, but might help you out a bit.

Robert

[andreas schmid]

i tried whoosh-appengine and as i mentioned before it fails after indexing a few 1000 because the way it works and it exceeds the soft process limit size.

never tried it on a backend. right now i customized bill katz's simple full text search and im really surprised how simple it is implemented and how easy it is to modify it to your needs.
i think ill go on with this solution until gaes full text search will be released.

thx

[Jon Romero Vlachogiannis]

We are using indextank.com for our service and it works perfectly.

On Aug 3, 10:32 am, andreas schmid <a.schmi...@gmail.com> wrote:
> i tried whoosh-appengine and as i mentioned before it fails after indexing a few 1000 because the way it works and it exceeds the soft process limit size.
>
> never tried it on a backend. right now i customized bill katz's simple full text search and im really surprised how simple it is implemented and how easy it is to modify it to your needs.
> i think ill go on with this solution until gaes full text search will be released.
>
> thx
>
> On Aug 3, 2011, at 7:17 AM, Robert Kluin wrote:
>
>
>
>
>
>
>
> > Have you tried running whoosh (not whoosh-appengine) on a backend?
> > Backends are expensive, but might help you out a bit.
>
> > Robert
>

> > On Thu, Jul 28, 2011 at 11:19, andreas schmid <a.schmi...@gmail.com> wrote:
> >> hi guys,
>
> >> i really need to have a full text search on my app and i wanted to know what kind of external services you are using on your apps. or how you implement such a service.
> >> i tried whoosh-appengine but first its a really old version and second it has huge issues with 1000+ indexed entities.
>
> >> we don't know when the full text search will be released for GAE right?
>
> >> thx
>
> >> --
> >> You received this message because you are subscribed to the Google Groups "Google App Engine" group.
> >> To post to this group, send email to google-a...@googlegroups.com.
> >> To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

> >> For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en.

[Robert Kluin]

On Wed, Aug 3, 2011 at 03:32, andreas schmid <a.sch...@gmail.com> wrote:
> i tried whoosh-appengine and as i mentioned before it fails after indexing a few 1000 because the way it works and it exceeds the soft process limit size.

Sorry, I was suggesting to *not* use whoosh-appengine -- rather try
'real' whoosh on a backend with more RAM. I've seen an example of
doing this before, though I don't recall where. I think it was quite
straightforward.

[andreas schmid]

i got that you weren't suggesting that :)

[Ikai Lan (Google)]

Sorry, we're working on getting this out as quickly as we can. We're not close to trusted tester/beta period yet. 

In the past, I've done App Engine talking HTTP over the public interwebs to an instance running Solr. The latency was acceptable for what I was doing: 20ms to either Slicehost or Linode (I forget which).

--

Ikai Lan 
Developer Programs Engineer, Google App Engine

plus.ikailan.com | twitter.com/ikai

[blackpawn]

I'm using http://www.elasticsearch.org/ hosted on Amazon EC2.  works great!

[Tim Hoffman]

Hi

We have been using solr on ec2 for www.polytechnic.wa.edu.au for nearly 2 years now, and we have never found the actual search latency

a problem.

Its running on a small ec2 instance along with plone (the cms for the appengine site)

Rgds

T

[PandaSuit]

From Ikai's post it sounds like we are a ways away from having
Google's search on GAE and I need some features that are going to be
annoying to add to Bill Katz solution so I decided to try out
indextank.com as Jon Romero Vlachogiannis suggested.

I am really impressed. It is easy to integrate and runs fast. All your
search and indexing processing is offloaded from your app and if you
have less than 100000 documents to index it is free.

I am in the process of converting my app over to using IndexTank right
now.

On Aug 3, 12:32 am, andreas schmid <a.schmi...@gmail.com> wrote:
> i tried whoosh-appengine and as i mentioned before it fails after indexing a few 1000 because the way it works and it exceeds the soft process limit size.
>
> never tried it on a backend. right now i customized bill katz's simple full text search and im really surprised how simple it is implemented and how easy it is to modify it to your needs.
> i think ill go on with this solution until gaes full text search will be released.
>
> thx
>
> On Aug 3, 2011, at 7:17 AM, Robert Kluin wrote:
>
>
>
>
>
>
>
> > Have you tried running whoosh (not whoosh-appengine) on a backend?
> > Backends are expensive, but might help you out a bit.
>
> > Robert
>

> > On Thu, Jul 28, 2011 at 11:19, andreas schmid <a.schmi...@gmail.com> wrote:
> >> hi guys,
>
> >> i really need to have a full text search on my app and i wanted to know what kind of external services you are using on your apps. or how you implement such a service.
> >> i tried whoosh-appengine but first its a really old version and second it has huge issues with 1000+ indexed entities.
>
> >> we don't know when the full text search will be released for GAE right?
>
> >> thx
>
> >> --
> >> You received this message because you are subscribed to the Google Groups "Google App Engine" group.
> >> To post to this group, send email to google-a...@googlegroups.com.
> >> To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

> >> For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en.

[andreas schmid]

ok after a while of using bill katzs full text search solution i switched to indextank.
and im really impressed how fast it is. ill be more than ok till GAEs full text search will be ready.
... and then we'll see if ill continue with indextank or GAEs solution.

[Bryce C]

I finished my conversion over to IndexTank and it has been running in
production for a few days now. It is too early to make grand
pronouncements but I am really happy with the performance so far,
especially compared to what I was using before. The faceting abilities
are really nice for generating reports on the types of data I am
storing. I used their jQuery auto-complete plugin to give my users a
better search experience and it was really easy to get integrated.

I am still definitely looking forward to the official GAE full text
search and especially how it will integrate with models and queries.

[Adam Sah]

we use SOLR on linode, and it's worked well for us (fast, cheap, scalable, ...) and avoids vendor downtime and lockin, while giving you complete control/flexibility over indexing.

that said, if I were starting over, I'd probably try indextank and then "graduate" to self-hosted SOLR when I needed the flexibility or they failed to deliver uptime.

hope this helps,

adam

[Brandon Wirtz]

Google Custom Search on a Paid Plan = Awesome.

 

Google Search Engine Appliance either using their hardware or the software version on your own, both work well.

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Jow2XZPVj3cJ.