I/O and timing

[LBJ]

Hi all,
I have question about timing and I/O. I am running a red hat version of linux (forgot which version exactly, relatively recent) and have been timing programs I have written in a very straighforward way (ie getTimeOfDay at the beginning, getTimeOfDay at the end, subtract). I have noticed a lot of variability in the measurements I have been taking and I suspect it may be due to I/O. There is a lot of writing to file and socket communication going on, and there are other I/O intense processes running on the box in addition to the one I am timing. My questions are:
1) is it reasonable to assume that it is the i/o that is introducing a lot of timing variability from run to run
2) if so, how can I (easily) prove it?

Thanks for any insights!

[Bit Twister]

On Fri, 16 Nov 2012 21:32:50 -0800 (PST), LBJ wrote:
> Hi all,
> I have question about timing and I/O. I am running a red hat version
of linux (forgot which version exactly, relatively recent)

You can see it with
cat redhat-release or maybe
cat /etc/release

> and have been timing programs I have written in a very
> straighforward way (ie getTimeOfDay at the beginning, getTimeOfDay
> at the end, subtract). I have noticed a lot of variability in the
> measurements I have been taking and I suspect it may be due to
> I/O. There is a lot of writing to file and socket communication
> going on, and there are other I/O intense processes running on the
> box in addition to the one I am timing. My questions are:
> 1) is it reasonable to assume that it is the i/o that is introducing
> a lot of timing variability from run to run
> 2) if so, how can I (easily) prove it?

First you might want to run "top" just to see what is going on.

Next I would suggest you using "time" to get a more detailed idea of
what may be going on.

Do keep in mind there are hourly cron jobs, which you might want to
avoid. If box is not up 24/7 then anacron might be running jobs which
did not execute when the box was powered off.

cat /etc/crontab to see actual times for jobs when system is up
24/7. If no up 24/7 you could have nightly jobs running while you are
logged in.

[Richard Kettlewell]

It doesn’t seem like a very safe assumption. If there are other
processes running then the level of activity from them is likely to
affect the wall-clock runtime of the process you’re testing. BT’s
suggestion of ‘time’ will give you a much better idea of time spent
actually running your program.

--
http://www.greenend.org.uk/rjk/

[The Natural Philosopher]

yes.

> 2) if so, how can I (easily) prove it?
>

remove the IO!

> Thanks for any insights!
>
Or simply open a timing debug log file and take snapshots of the time
throughout the program.

I did that once when I have a vile case of 'takes far too long' : came
down to a badly indexed Mysql database.




--
Ineptocracy

(in-ep-toc’-ra-cy) – a system of government where the least capable to
lead are elected by the least capable of producing, and where the
members of society least likely to sustain themselves or succeed, are
rewarded with goods and services paid for by the confiscated wealth of a
diminishing number of producers.

[Johann Klammer]

sysprof is very good at this. It can show you where your box spends its
time. shows user calls and kernel functions. Pointy-clicky GUI or
console program. You may need to recompile your kernel as it depends on
some tracing stuff(don't remember which exactly).

[unruh]

No. Redhat is a multiuser, multiprogram system. Ie, your program can be
interrupted by other programs while it is running, and depending on the
time of day and who the users are that can vary wildly.
Now. you perhaps do not know about the command "time" whose purpose is
precisely to time programs and to compensate, at least in part, for that
variability.
man time

[philo]

Just a guess but I'm thinking that if you do not know how to determine
which version of an operating system you are using...
maybe you should not be the one to try your hand at troubleshooting.

<shrug>

--
https://www.createspace.com/3707686