incremental indexing and merging.

[mohitnegi]

How can i do daily hourly or daily incremental indexing by using thinking sphinx.

[Pat Allan]

Incremental indexing isn't something that Thinking Sphinx does by default. The ts-datetime-delta gem does it, but I know in the past merging hasn't been entirely reliable with Sphinx (though that could quite possibly have changed by now - I've not tested it thoroughly for a couple of years).

What's prompted you to ask this, though? Why does the standard indexing approach not suit your setup?

Cheers

--
Pat
e: p...@freelancing-gods.com || m: +614 1327 3337
w: http://freelancing-gods.com || t: twitter.com/pat
bounce: http://trampolineday.com || skype: patallan

On 23/10/2011, at 8:59 PM, mohitnegi wrote:

> How can i do daily hourly or daily incremental indexing by using thinking sphinx.
>

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.
> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/WwDr-7Q1ctMJ.
> To post to this group, send email to thinkin...@googlegroups.com.
> To unsubscribe from this group, send email to thinking-sphi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.

[mohitnegi]

I have an index on my table which has reference to many other table and those get updated frequently,

so i want my index to be up to date.

So i was looking for incremental indexing and merging the incremental index with the index.

[Pat Allan]

Sounds like ts-datetime-delta may do what you want - give it a try.
https://github.com/freelancing-god/ts-datetime-delta

Cheers

--
Pat

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/VTMzdEU3s5UJ.

[mohitnegi]

ts-datetime-delta appears to solve my problem.

Does it takes care of merging the incremental index with the main index or merging logic we have to do it separately!!!!!

[Pat Allan]

It does merging as well.

-- 

Pat

On 24/10/2011, at 12:03 PM, mohitnegi wrote:

ts-datetime-delta appears to solve my problem.

Does it takes care of merging the incremental index with the main index or merging logic we have to do it separately!!!!!

--
You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.

To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/N2dlIc22y30J.

[mohitnegi]

tried couple of times but seems it's not working out for me.

threshold value is set as 1.hour

set_property :delta => :datetime,  :threshold    => 1.hour

now modifying a record.

running the rake task for delta indexing

bundle exec rake thinking_sphinx:index:delta

result is same as before running the task.

Has to rebuild the whole index again, then only the expected result is gained.

[Pat Allan]

What's the output when you run the thinking_sphinx:index:delta task?

--
Pat

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/IVFlYrgJ4LUJ.

[mohitnegi]

I upgrade my ts to 2.0.1 and now it's working!!!!! :)

Thanks Pat

[Pat Allan]

Ah, that's great - and do you mean Sphinx 2.0.1? Or Thinking Sphinx 2.0.1?

--
Pat

On 07/11/2011, at 3:02 PM, mohitnegi wrote:

> I upgrade my ts to 2.0.1 and now it's working!!!!! :)
>
> Thanks Pat
>

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/ltsgUY9FPFIJ.

[mohit negi]

Thinking Sphinx 2.0.1 :)

--
With Warm Regards,
Mohit Negi
+91-9945528639
mohit...@gmail.com

[Pat Allan]

Oh, that's not so cool - more recent versions didn't work?

--
Pat

[mohitnegi]

ohh sorry, my mistake its sphinx 2.0.1

[Pat Allan]

Ah, that's much better :)

--
Pat

On 08/11/2011, at 6:02 PM, mohitnegi wrote:

> ohh sorry, my mistake its sphinx 2.0.1
>

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/CksDl-9WOoYJ.

[mohitnegi]

Can i keep my threshold value in sphinx.yml file, or in some config file such that  Programatically my cron job and sphinx index should fetch it from the file.

I will be using 'whenever' for my cron job, so is it possible to define the threshold value somewhere such that both my index and 'whenever' fetches the value from there.

[Pat Allan]

I'm afraid the only place you can set the threshold value is in the index definition. Of course, you could define a constant somewhere and have that referenced within the index block *and* in the whenever configuration. Maybe worth giving that a shot?

--
Pat

On 16/11/2011, at 4:45 PM, mohitnegi wrote:

> Can i keep my threshold value in sphinx.yml file, or in some config file such that Programatically my cron job and sphinx index should fetch it from the file.
> I will be using 'whenever' for my cron job, so is it possible to define the threshold value somewhere such that both my index and 'whenever' fetches the value from there.
>

> --
> You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/thinking-sphinx/-/zu8UtZYYedgJ.