Background thread I/O priority

[Olivier Guilyardi]

Hi,

I have a background thread which performs quite long and intensive disk access.
This tends to freeze the system. I tried to use setpriority() but it doesn't
help unless I set it to 10, which makes the job almost 4x slower to complete.

I have obtained good results by timing disk access, and using something like
usleep(0.7 * buffer_read_time) after reading each buffer. My benchmarks show
that it only makes the whole job about 1.3x slower and freezes get pretty rare.

But it still can freeze some time or make the UI slow to respond.

How can I solve this problem?

Is using ionice an option?

TIA

--
Olivier

[Olivier Guilyardi]

A discussion regarding this actually took place in another thread about "SQLite
optimization and async I/O".

I am forwarding/quoting some answers here because that's where they really belong.

Here's my OP:

Regarding ionice, Dianne, in the thread about "SQLite optimization and async
I/O", you mention "Most flash fs drivers I believe don't do IO prioritization at
this point.".

So ionice doesn't appear to be an option. That said, as a side note, I've seen
that some system processes are ionice'd in various ways (lower I/O priority, RT,
etc..).

Tim, you suggested using mmap, but IIRC I tried that and it didn't help.

Dianne, you asked:
On 06/25/2011 09:06 AM, Dianne Hackborn wrote:

> What priority are the two threads running at? If you are doing background work you should make sure to set your thread to background priority. You also should do this using the android.os.Process API since that takes care of adjusting cgroups or whatever mechanism may be used to better control scheduling.
>
> Other information that would be useful -- how long the two versions take to complete, how much CPU the two versions use while running.
>
> And of course you can go and look at the code of FileChannel to see how that is implemented (I don't know off-hand).

All my threads use the default priority. But as mentioned above in the OP, I
experimented with setpriority() for the concerned background thread, and that
didn't really help. It is inappropriate IMO, since the overhead seems to be
caused by disk I/O, and changing the CPU priority is ineffective unless very low
(high value), which makes the job very slow.

I haven't tried Process.setThreadPriority() yet, and I would need to experiment,
but I think this is inappropriate too, since this problem isn't about the CPU load.

About "how long the two versions take to complete", some timings are mentioned
in my OP.

Yes, I think that I will look at FileChannel sources. But I have the intuition
that it may not help.

The thing about my tests with FileChannel is that they are about copying a file,
which means reading (fast) and writing (slow).

And the background thread mentioned in my OP doesn't write anything. It scans a
big file, that is, it only reads. I would need to try and do this with Java to
see if it works better. I will try and post some more info in the future.

Thanks

Olivier