Optimizing QueryParsingUtils.Compile()

[Damian Hickey]

Hi all,

I've got a significant number (>1000) of automated acceptance tests that utilize the embeddable document store. As the number of indexes we defined grew, the test speeds when running more than 1 test (i.e. CI / NCrunch) started to deteriorate. Poking around with dotTrace show a large percentage of time (>50%) inside QueryParsingUtils.Compile(). So I wrote a brief test that created an embeddable document store with 2 indexes and then disposed it 10 times to get some initial timings in milliseconds:

1: 785; 2: 232; 3: 229; 4: 228; 5: 220; 6: 225; 7: 227; 8: 226; 9: 216; 10: 227

I then hacked in a dictionary cache into QueryParsingUtils.Compile() just to see what difference it would make when I ran the same test:

1: 856; 2: 68; 3: 67; 4: 54; 5: 56; 6: 56; 7:80; 8:63; 9: 52; 10: 55

Ignoring the first loop (assembly loading etc), this is ~4 times faster. While I can't really see the benefit in optimizing this method from a production pov, it would have a big affect on accumlative test times. For me anyway at least. Is this something we can do?

Btw, I'm making no assertions as the correctness & quality of my hack :)

Cheers,

- Damian

[Oren Eini (Ayende Rahien)]

That is a very good point. It is actually _is_ suitable for prod, if you have more than a single database that have same indexes.

I added that, with the usual caveats about emptying it and limiting the size,

[Oren Eini (Ayende Rahien)]

And I think it just took 5 - 7 minutes out of our build process.

[Damian Hickey]

Nice!

And cheers for implementing :)

On Tuesday, 27 March 2012 13:57:44 UTC+1, Oren Eini wrote:

And I think it just took 5 - 7 minutes out of our build process.