Datetime driven deltas

[anibal]

Hey,

I'm trying to introduce the new datetime-driven deltas in my project
but something very strange is happening. This is the situation:

I create 3 objects 'Topic1', 'Topic2' and 'Topic3' and run a full
index (rake ts:index). Then I create another object 'Topic4' and try
searching all of them by their name. The sphinx query log reads like
this:

[Thu Jan 15 18:26:51.152 2009] 0.000 sec [all/2/rel 1 (0,20)] [*]
Topic1
[Thu Jan 15 18:26:54.239 2009] 0.001 sec [all/2/rel 1 (0,20)] [*]
Topic2
[Thu Jan 15 18:26:56.320 2009] 0.001 sec [all/2/rel 1 (0,20)] [*]
Topic3
[Thu Jan 15 18:27:02.144 2009] 0.000 sec [all/2/rel 0 (0,20)] [*]
Topic4

Which is perfect because 'Topic1', 'Topic2' and 'Topic3' are indexed
but 'Topic4' is not.

Then I run 'thinking_sphinx:index:delta' to merge the latest changes
and try searching all of them again. The query log reads like this:

[Thu Jan 15 18:28:10.400 2009] 0.000 sec [all/2/rel 0 (0,20)] [*]
Topic1
[Thu Jan 15 18:28:12.848 2009] 0.000 sec [all/2/rel 0 (0,20)] [*]
Topic2
[Thu Jan 15 18:28:14.839 2009] 0.000 sec [all/2/rel 0 (0,20)] [*]
Topic3
[Thu Jan 15 18:28:17.024 2009] 0.000 sec [all/2/rel 1 (0,20)] [*]
Topic4

Basically, only the new object 'Topic4' just merged is found but none
of the previous objects are found.

Just to add more information, if I run an empty query:
ThinkingSphinx::Search.search(''), sphinx finds all of them (the 4
topics):

[Thu Jan 15 18:28:38.544 2009] 0.000 sec [scan/2/rel 4 (0,20)] [*]

I was using the old delta mechanism and everything worked fine but I
really wanna start using the new approach because is a lot better in
my opinion.

Hope someone could help me and thanks in advance,
Anibal

[Pat Allan]

Hi Anibal

That behaviour isn't what we want... I'll try to add some more tests,
see if I can reproduce the situation, and we'll go from there. I'm not
convinced with how effective Sphinx's merge is, hence why it's not yet
implemented with the basic delta approach.

Will let you know what I find. In the meantime, what does your
define_index block look like? Just as a reference point.

Cheers

--
Pat

[Anibal Cucco]

Thanks Pat, I'll continue trying and seeing if I can find out what's wrong. This is the define_index block:

  define_index do
    indexes [ submitter.name, submitter.email ], :as => :submitter
    indexes forum.name, :as => :forum
    indexes posts.body, :as => :text
    indexes current_tags, :as => :tags   
    indexes title
    indexes updated_at, :sortable => true
    has account_id, forum_id, is_public, organization_id, is_pinned, submitter_id, is_syndicated, updated_at
    set_property :delta => :datetime, :threshold => 15.minutes
  end

-Anibal

[Anibal Cucco]

Hey Pat,

Just to let you know that we found the problem. I was using Sphinx 0.9.8-release (r1371) and there's something wrong with the merge in that version. Upgraded to Sphinx 0.9.8.1-release (r1533) and the problem is gone.

Thanks,
Anibal

[Pat Allan]

Ah, that's good to know. I've been meaning to restructure the TS site
for a while now, make it easier to keep the docs updated (perhaps
through a wiki? Haven't decided yet) - and that tidbit should
definitely be part of the new information.

Cheers

--
Pat

On 24/01/2009, at 12:14 AM, Anibal Cucco wrote:

> Hey Pat,
>
> Just to let you know that we found the problem. I was using Sphinx
> 0.9.8-release (r1371) and there's something wrong with the merge in
> that version. Upgraded to Sphinx 0.9.8.1-release (r1533) and the
> problem is gone.
>
> Thanks,
> Anibal
>
> On Mon, Jan 19, 2009 at 11:57 AM, Anibal Cucco
> <aniba...@gmail.com> wrote:
> Thanks Pat, I'll continue trying and seeing if I can find out what's
> wrong. This is the define_index block:
>
> define_index do
> indexes [ submitter.name, submitter.email ], :as => :submitter
> indexes forum.name, :as => :forum
> indexes posts.body, :as => :text
> indexes current_tags, :as => :tags
> indexes title
> indexes updated_at, :sortable => true
> has account_id, forum_id, is_public, organization_id, is_pinned,
> submitter_id, is_syndicated, updated_at
> set_property :delta => :datetime, :threshold => 15.minutes
> end
>
> -Anibal
>
>
>

> On Sun, Jan 18, 2009 at 11:45 PM, Pat Allan <pat@freelancing-