Hi,
We run ddfs on 4 machines. We insert about 15000 tags per day and remove the same number of tags from the past.
We need to keep a few days back but then old data can be deleted.
Each tag contains data divided into about 3 chunks of 6MB each (compressed). Everything is replicated 3 times.
We observe issues with ddfs performance after some time. If the storage is empty all operations are very quick. However after a few weeks it
slows down. ddfs ls may take about a minute and getting single data file several seconds.
We try to play with various options (mentioned here:
https://groups.google.com/forum/#!searchin/disco-dev/slow/disco-dev/AmwN72XUp4A/m7iJUN4rnjsJ)
but it doesn't help much.
GC also takes a few hours to perform single run.
What can we do to solve data accessing time problem?
Regards,
Krzysztof Kaczmarski