Performance degradation when ingesting multiple items

[frank asseg]

Hola Guys!

I was conducting some JMeter tests with RDFDatabank and came across the following numbers when running the tests on my local linux box:

https://docs.google.com/file/d/0B5nd_qlYdcqyS2haMlZLajRMc2M/edit

JMeter project: https://github.com/futures/ff-databank-jmeter

This test ingested ~350 items from the EDRM test data into a single Databank dataset.

http://www.edrm.net/projects/dataset

As you can see the time required for ingesting a single item seems to grow linear from ~80ms to ~700ms, while the throughput seems to decay exponentially.

Have you seen this behaviour before and can I tweak some settings to achieve more pleasing results?

Thanks!

Frank

Test Environment:

• Linux 3.6.8-1-ARCH #1 SMP PREEMPT x86_64 GNU/Linux
  
• Postgresql 9.2
• Python 2.7.3
• mod_wsgi 2 3.4
• Apache Web Server 2.2.23
• Apache Tomcat 7.0.19
• Apache Solr 3.6.2

[frank asseg]

Actually I think this was related to a data set sorted by size, although this would not explain the decrease in throughput it certainly is part of the request duration increase.

I will rerun these tests with a more homogenous dataset and write an update...

[frank asseg]

On Monday, January 14, 2013 1:20:45 PM UTC+1, frank asseg wrote:

Actually I think this was related to a data set sorted by size, although this would not explain the decrease in throughput it certainly is part of the request duration increase.

I will rerun these tests with a more homogenous dataset and write an update...

After I reran the tests with 500 items of 1MB I got the following results which show the same decay in ingest times/throughput:

https://docs.google.com/file/d/0B5nd_qlYdcqybFk2MjRZdUFfX1k/edit