Faster iteration performance with BDB 5.0.34

[ijuma]

Hey all,

The following may be of interest to many of you:

Use DiskOrderedCursor to improve iteration performance for bdb storage engine (affects rebalancing performance)

http://code.google.com/p/project-voldemort/issues/detail?id=373

Best,

Ismael

[Alex Feinberg]

Yes. It's on our list to investigate this for LinkedIn, although we
have higher priority items at this time (we are currently migrating
clusters to SSDs, which greatly increases scan performance even with
random seeks).

I have a test branch where I am investigating this, but there's a few
issues (it isn't as simple as just changing the methods/objects used).

Thanks,
- Alex

> --
> You received this message because you are subscribed to the Google Groups
> "project-voldemort" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/project-voldemort/-/N4KgJa5MyckJ.
> To post to this group, send email to project-...@googlegroups.com.
> To unsubscribe from this group, send email to
> project-voldem...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/project-voldemort?hl=en.

[ijuma]

Hi Alex,

On Monday, 5 December 2011 21:50:01 UTC, Alex Feinberg wrote:

Yes. It's on our list to investigate this for LinkedIn, although we
have higher priority items at this time (we are currently migrating
clusters to SSDs, which greatly increases scan performance even with
random seeks).

Interesting.

 

I have a test branch where I am investigating this, but there's a few
issues (it isn't as simple as just changing the methods/objects used).

Yes, I outlined them in the issue (biggest one is having to run a tool on repositories with duplicate keys). Are there others that I missed?

Best,

Ismael

[Liam Slusser]

We also run our V cluster all on SSDs as well. Outstanding
performance improvement from spinning disks. Just make sure you use
high quality enterprise SSDs and not the cheapo desktop stuff and
you'll love it.

liam