Inconsistency in trace file results

[Gandholi Sarat]

Hi,
I am trying to see Dynamorio's drcachesim trace overhead on I/O intensive workloads. So I was trying to get the offline trace for some file bench like load of reading files, writing, etc. (https://github.com/filebench/filebench/blob/master/workloads/filemicro_rwrite.f)

I am observing some inconsistencies in run time and the trace file size generated while running the same benchmark of write.

I am taking care of clearing the cache, stopping address space randomization, and also pinning to cores, etc. And the benchmark itself is consistent when running normally. So why could this inconsistency occur with Dynamorio?
Are there any features that help us to handle cases of tracing I/o intense workloads effectively? Or Writing such clients to effectively handle this would be useful?

Regards,
Sarat

[Kai Luk]

What's the range of run time, and file sizes?

Do you see the same variation with sequential write, i.e. filemicro_seqwrite.f ?

Kai

[Gandholi Sarat]

The range for rread was time 32-201 secs and trace file size was 12-15gb

Variation was comparatively less with seqwrite like from 104 - 150 secs in time and file size was also around 16gb always


Sarat

[Derek Bruening]

For overall offline tracing overhead, the type of storage device and the type of compression make huge differences.  Generally you want lz4 compression for the raw files: make sure you built with and enabled lz4.

For the variation, turn it from total (compressed?) size into record types: are there significant instruction count differences?  If so, best to compare the profiles and see what the app behavior differences are.  Does the app have any wall-clock-time components or self-measuring components that overhead affects?

--
You received this message because you are subscribed to the Google Groups "DynamoRIO Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dynamorio-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dynamorio-users/2d375ee3-8899-4d67-904d-3707c9818f42n%40googlegroups.com.

[Gandholi Sarat]

I will try the following suggestions and get back.
But wanted to ask you, if such a study of the bottleneck of dynamorio on tracing io intense workloads is useful. How useful is making an io-optimized client?
I am doing a project exploring dynamorio so you could suggest other useful analyses to be done if any more.

Your response is highly appreciated. Thank you

[Derek Bruening]

Any contributions to improve DynamoRIO are welcome.  There are filed issues on performance problems with online analysis which seem to be regressions: https://github.com/DynamoRIO/dynamorio/issues/3614.  Another suggestion is to search for the label "help-wanted": https://github.com/DynamoRIO/dynamorio/labels/help%20wanted.

To view this discussion on the web visit https://groups.google.com/d/msgid/dynamorio-users/CAC%3DDGeUwFJfR_a_54QnLyvfn-r%2BsL%2Bz%3DYh2FNDYs7VNoMAw4-w%40mail.gmail.com.