New Search Engine For Wikidot

[Piotr Gabryjeluk]

Hi to you!

Lucene-based search - a brand new feature has been being introduced to
Wikidot code for quite a long time.

After that time of developing and dealing with performance problems
(searching the whole Wikidot in 3 seconds is too long!) it's time for
test this thing!

Everyone running their own Wikidot installation (like Ed), please visit
my blog: http://piotr.gabryjeluk.pl/dev:wikidot-search-ready-to-test
for more information on this update.

Current SVN revision: 369 (what a nice number) is stable! Check it
yourself. This is the version that powers WikiComplete.info right now.

Keep tuned for the more news on the Wikidot search engine and drop a
comment on my blog post (or reply to this message).
--
Piotr Gabryjeluk
Wikidot.com developer
http://piotr.gabryjeluk.pl/

[Piotr Gabryjeluk]

Hello again!

First of all, happy holidays everyone! Then, I would like you
check the new query syntax for the new search engine for Wikidot
(that I work on):

http://piotr.gabryjeluk.pl/search-syntax

I look forward to your opinions (both reply and comments).

I hope we will have the new search engine developed and running
with the very start of 2009!

Cheers!
Piotr

Dnia 2008-12-19, o godz. 00:42:20
Piotr Gabryjeluk <pi...@wikidev.lastlook.pl> napisał(a):

[Брунхилда од Аустразије]

Does this new serach engine works for all our wikidot sites, or is it only for community site?

Happy Hollidays to all!
Bruni

2008/12/25 Piotr Gabryjeluk <pi...@wikidev.lastlook.pl>

--
Brunehilda The Queen

[Piotr Gabryjeluk]

Hi Брунхилда

This new search is to work on whole Wikidot, i.e. Search All Sites
page. But this does not yet work anywhere but WikiComplete (and this
searches all WikiComplete sites):

http://www.wikicomplete.info/search:all

Dnia 2008-12-25, o godz. 12:45:02
"Брунхилда од Аустразије" <brunehild...@gmail.com> napisał(a):

[Брунхилда од Аустразије]

I am sorry, but I didn't understand very well... Do you mean that I this search mode will not work at my site, for example (when implemented, of course)?

2008/12/25 Piotr Gabryjeluk <pi...@wikidev.lastlook.pl>

--
Brunehilda The Queen

[Piotr Gabryjeluk]

First, it'll only for when searching in all sites (but you can search
for something site:your-site). Maybe then we replace the search this
site with the new engine.

Dnia 2008-12-25, o godz. 13:06:28

[Erich]

Will the new engine/syntax be able to
* search for an exact phrase, like "one two three"?
* search for "~~~~" (the current search engine can't -- see
http://community.wikidot.com/forum/t-35739)
* search for the user name (who posted, not names that were written
*in* the post itself). The current engine can't -- see
http://community.wikidot.com/forum/t-22894

Is the minimum-of-three-letters restriction absolutely necessary?
What if you'd have to search for e. g. "@@"?

[Piotr Gabryjeluk]

Dnia 2008-12-25, o godz. 05:40:35
Erich <erich.st...@gmail.com> napisał(a):

>
> Will the new engine/syntax be able to
> * search for an exact phrase, like "one two three"?

Items with "one two three" in title (!) should have the best rank on
the list when searching with phrase "one tho three". We can't search
for exact phrase in page content, because this would be slow.

> * search for "~~~~" (the current search engine can't -- see
> http://community.wikidot.com/forum/t-35739)

Only words are indexed and only words can be searched/found.
Performance matters.

> * search for the user name (who posted, not names that were written
> *in* the post itself). The current engine can't -- see
> http://community.wikidot.com/forum/t-22894

I think this is quite possible, but quite hard to do.

>
> Is the minimum-of-three-letters restriction absolutely necessary?
> What if you'd have to search for e. g. "@@"?

It's neccessary, if we want the search to work. Very short queries are
very resource-expensive.

Cheers!

[Ed Johnson]

Hi Piotr and Happy New Year!

I finally decided to roll up my sleeves and try to upgrade my server
at work to the latest rev (393 as of today) with Lighttpd and newer
version of Ubuntu server and PostgreSQL 8.3. The short version of my
story is that overall it worked pretty well considering I was
upgrading my database from somewhere around rev 195. I did end up with
a couple of issues with files/MIME types (I'm not sure exactly yet)
and searching is not working at all. I suspect the problem might be
with some missing tables since I restored from such an old revision. I
posted more details here:

http://my-wd-local.wikidot.com/blog:davanniswiki-server-upgrade

Could you give it a read when you have time to see if anything jumps
out as the possible problem(s)? So far I have not broken my live
install - I've been doing all of this on my second testing server.

Thanks!
-Ed

On Dec 25, 5:41 am, Piotr Gabryjeluk <pi...@wikidev.lastlook.pl>
wrote:

> Hello again!
>
> First of all, happy holidays everyone! Then, I would like you
> check the new query syntax for the new search engine for Wikidot
> (that I work on):
>
> http://piotr.gabryjeluk.pl/search-syntax
>
> I look forward to your opinions (both reply and comments).
>
> I hope we will have the new search engine developed and running
> with the very start of 2009!
>
> Cheers!
> Piotr
>
> Dnia 2008-12-19, o godz. 00:42:20
> Piotr Gabryjeluk <pi...@wikidev.lastlook.pl> napisał(a):
>
>
>
>
>
> > Hi to you!
>
> > Lucene-based search - a brand new feature has been being introduced to
> > Wikidot code for quite a long time.
>
> > After that time of developing and dealing with performance problems
> > (searching the whole Wikidot in 3 seconds is too long!) it's time for
> > test this thing!
>
> > Everyone running their own Wikidot installation (like Ed), please
> > visit my blog:

> >http://piotr.gabryjeluk.pl/dev:wikidot-search-ready-to-testfor more

[Piotr Gabryjeluk]

I tried to answer you competitively. See if those tips help!

Dnia 2008-12-26, o godz. 19:49:11
Ed Johnson <edjohn...@gmail.com> napisał(a):

[Erich]

Will the new search be able to (better) handle cases like the one
described in
http://community.wikidot.com/forum/t-27713/search-box:error-when-trying-to-search-for-string-site-changes-xml#post-344287
?

Thanks!