Duplicate lines of code returned from simple string query

[Chris Cleeland]

When I perform a simple string query through the web interface, any line of
code that contains the simple string appears in the list multiple times
consecutively. Most commonly I see it appear in triplicate, but I have
also seen it appear four times. Has anyone else ever encountered this, or
is it yet another artifact of a partially-failed (or partially successful)
installation?

In looking at the search.cgi and queries.cgi it appears that those lines
are coming out of the database, so I thought I would fire up sqlite3
command line and look at the database. However, I cannot successfully get
it to recognize the dxr-code-tokenizer so that it returns data from the fts
table. I used ".load" and even explicitly request that it run the same
init function that the python script runs, but when I "select * from fts;"
I get errors.

SQLite version 3.7.11 2012-03-20 11:35:50
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .load
/home/ccleeland/localbuilds/dxr/sqlite/libdxr-code-tokenizer.so sqlite>
select (select path from files where id = fts.rowid),fts.content,
offsets(fts) from fts where fts.content match "pthread_create";
Error: unknown tokenizer: dxrCodeTokenizer


Is anybody familiar with the magic incantations required to get this to
work and willing to share? Google seems to turn up nothing useful for me.

Thanks,

--
Chris Cleeland

[Taras Glek]

On 4/24/2012 2:21 PM, Chris Cleeland wrote:
> When I perform a simple string query through the web interface, any line of
> code that contains the simple string appears in the list multiple times
> consecutively. Most commonly I see it appear in triplicate, but I have
> also seen it appear four times. Has anyone else ever encountered this, or
> is it yet another artifact of a partially-failed (or partially successful)
> installation?

Carlos just fixed this last week on http://dxr.lanedo.com/. Carlos, can
you submit a pull request with your changes?

Taras

[Ehsan Akhgari]

I still see it, for example: <
http://dxr.lanedo.com/search.cgi?tree=mozilla-central&string=ReturnInParagraph
>

--
Ehsan
<http://ehsanakhgari.org/>

[Carlos Garnacho]

Hey,

On mar, 2012-04-24 at 15:05 -0700, Taras Glek wrote:
> On 4/24/2012 2:21 PM, Chris Cleeland wrote:
> > When I perform a simple string query through the web interface, any line of
> > code that contains the simple string appears in the list multiple times
> > consecutively. Most commonly I see it appear in triplicate, but I have
> > also seen it appear four times. Has anyone else ever encountered this, or
> > is it yet another artifact of a partially-failed (or partially successful)
> > installation?
> Carlos just fixed this last week on http://dxr.lanedo.com/. Carlos, can
> you submit a pull request with your changes?

I was going to do that from the lanedo clone, but accidentally pushed
directly... My bad

Carlos

[Carlos Garnacho]

Hey :),

On mar, 2012-04-24 at 21:40 -0400, Ehsan Akhgari wrote:

> On Tue, Apr 24, 2012 at 6:05 PM, Taras Glek <tg...@mozilla.com> wrote:
>
> > On 4/24/2012 2:21 PM, Chris Cleeland wrote:
> >
> >> When I perform a simple string query through the web interface, any line
> >> of
> >> code that contains the simple string appears in the list multiple times
> >> consecutively. Most commonly I see it appear in triplicate, but I have
> >> also seen it appear four times. Has anyone else ever encountered this, or
> >> is it yet another artifact of a partially-failed (or partially successful)
> >> installation?
> >>
> > Carlos just fixed this last week on http://dxr.lanedo.com/. Carlos, can
> > you submit a pull request with your changes?
> >
>

> I still see it, for example: <
> http://dxr.lanedo.com/search.cgi?tree=mozilla-central&string=ReturnInParagraph

The server has gone through a couple of hiccups during the last day... I
went chasing a big performance drop where dxr-index went I/O bound
rather than CPU bound, so I temporarily reverted to an earlier HEAD. We
later realized it was googlebot thrashing the server, so there's now a
robots.txt and this fix is back again :)

Carlos