Slow indexing on latest unstable build 2.5 (2587)

[Paul Hinett]

Hi,

I have noticed a huge slow down in indexing speed in the most recent 2.5
build, i was using build 2587 to index around 25 various types of
indexes with 400,000 docs total, from the stats page it appeared on the
graph to be doing around 100 docs/second, i left this running for about
an hour but all 25 still showed as stale.

Downgrading to build 2580 made a big difference and finished remaining
indexing in minutes with the graphs showing indexing rate in the
thousands/second.

Not sure if anything has changed since these 2 builds to affect this, i
think i seen a message on the groups with someone else noticing similar
issues but not sure if there was an outcome to it.

Paul

[Oren Eini (Ayende Rahien)]

We modified the logic for index batches a bit, it might be related.

Can you send us a repro?

Paul


--
You received this message because you are subscribed to the Google Groups "ravendb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Paul Hinett]

Not had chance to package up a repro at the moment, but i'm noticing a
huge difference in indexing speed again with the most recent unstable
builds.

Downgrading to a lower build (in this case 2567) mad a drastic
difference again.

It's as if it's indexing a small handful of documents at a time, after
downgrading, in the studio logs it now says it's indexing 16,000+ and
32,000+ documents at time, where as with latest builds this was 1
document, or 4 documents etc.

I'll find some time tonight to try and reproduce this locally, however
i'm surprised no one else has picked this up.

> send an email to ravendb+unsubscribe@__googlegroups.com
> <mailto:ravendb%2Bunsu...@googlegroups.com>.
> For more options, visit https://groups.google.com/__groups/opt_out
> <https://groups.google.com/groups/opt_out>.

>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "ravendb" group.
> To unsubscribe from this group and stop receiving emails from it, send

> an email to ravendb+u...@googlegroups.com.

[Oren Eini (Ayende Rahien)]

Thanks

[Arkadiusz Paliński]

Hi Paul,

when you started the indexing process, did you have all documents already in a database? If not how did you put documents - by using bulk insert or session.SaveChanges? Did you have any concurrent document writes (from different threads or multiple clients) while the indexation was running?

Arek

>     <mailto:ravendb%2B...@googlegroups.com>.

[Paul Hinett]

They was already in the database, then around 3 indexes reset (out of
about 28 total indexes).

360,000 docs roughly.

> > <pa...@ukcreativedesigns.com <javascript:>

> <http://googlegroups.com>
> > <mailto:ravendb%2B...@googlegroups.com <javascript:>>.

> > For more options, visit
> https://groups.google.com/__groups/opt_out

> <https://groups.google.com/__groups/opt_out>
> > <https://groups.google.com/groups/opt_out

> <https://groups.google.com/groups/opt_out>>.
> >
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "ravendb" group.
> > To unsubscribe from this group and stop receiving emails from
> it, send

> > an email to ravendb+u...@googlegroups.com <javascript:>.
> > For more options, visit https://groups.google.com/groups/opt_out

[Jahmai Lay]

Does this help?

RavenConfiguration ravenConfiguration = new ...;

ravenConfiguration.Settings["Raven/DisableDocumentPreFetchingForIndexing"] = "true";

I'm asking because I had a problem with similar symptoms and this helped. If it helps you perhaps it narrows the problem.