[ANNOUNCE] BFQ-v7 for 3.2.0 - 3.13.0 is out!

Showing 1-9 of 9 messages
[ANNOUNCE] BFQ-v7 for 3.2.0 - 3.13.0 is out! paolo 1/28/14 2:51 AM
Hi,
believe it or not, BFQ-v7 is out! It has been a remarkable effort, but it seems like we made it.

One of the main improvements over v6r2 is that we found, or, more honestly, we bumped into a way to preserve a high responsiveness also in the presence of write hogs. Maybe some of you remember the discussion we had on this ml about the bad things that happen even if you just copy a large file: once the configured threshold of dirty pages is reached, a storm of asynchronous write requests starts. In particular, the rate at which writes are issued happen to be higher than that at which the device can serve them. To handle this kind of situations, the block layer uses a simple blocking scheme: the maximum number of overall pending I/O request for a device is limited, and, if a process tries to make a new I/O request when the pool is exhausted, that process just blocks, waiting for a request to be available.

Many problems follow from this fact:
1) If the unlucky process is updating metadata, the access to the whole filesystem may then be blocked until this process completes its job
2) If the unlucky process is executing an interactive task, such loading its own image and related libraries to start, you risk to wait for a while, until it 'grabs' all the I/O requests it needs.

In the end, the system, i.e., you, experience low responsiveness or even whole-system starvation. I let you imagine what happens if more than one process is writing greedily …

Well, BFQ-v7 contains an improvements that seems to recover responsiveness also in these situations! If you are curious about how this is achieved, have a look at the changelog, or, if you are brave enough, to the code itself.

Other relevant improvements are:
- better (and fixed) detection of soft real-time applications, which now filters out unwanted intruders and hence improves latency for the applications that really need it
- better handling of NCQ-enabled rotational disks, which boosts the throughput on these devices more than before (well, honestly this was a middle way between a bugfix and an improvement :) )
- a simple solution to make low-latency heuristics more robust inside a QEMU virtual machine and on an embedded system.

I wish to thank Virtual Open Systems, a company that, as part of a contract, allowed us to test BFQ also on new medias, such as microSDHC Cards, eMMCs and 1.8-inch HDDs. They also contributed to the above last improvement to BFQ.

I am sorry for the long email, but, believe me on that there would be even more to say about this new version of BFQ. The main improvements that you should experience are, regardless of the background workload and regardless on what device or environment you are executing BFQ in: a lower and more 'stable' responsiveness, a lower latency for time-sensitive applications such as video players, a higher throughput on NCQ-enabled hard disks.

This time we have tested BFQ-v7 on really many devices: single high-speed hard disks, high-speed disks paired in a software RAID1 configuration, SSDs, plus the above mentioned medias. We should publish soon a new version of the "test results" page, reporting our last results on these devices.

To conclude, I wish to thank as usual Arianna for her fundamental help and, in particular, for preparing all the patches also this time. Should you need BFQ-v7 for a previous kernel release than 3.2, just ask.

That's all, I hope you will all enjoy the results of our last two years of work on BFQ. Any feedback is more than welcome,
Paolo
Re: [ANNOUNCE] BFQ-v7 for 3.2.0 - 3.13.0 is out! Oleg R 1/28/14 3:39 AM
A patch for kernel 3.1 would really be appreciated. Certain devices that we support can't be updated past 3.1 due to a U-Boot bug. Thanks again!
Re: [ANNOUNCE] BFQ-v7 for 3.2.0 - 3.13.0 is out! Arianna Avanzini 1/28/14 4:15 AM
Il giorno martedì 28 gennaio 2014 12:39:23 UTC+1, Oleg R ha scritto:
A patch for kernel 3.1 would really be appreciated. Certain devices that we support can't be updated past 3.1 due to a U-Boot bug. Thanks again!


Paolo Valente has just uploaded both the full patchset and the incremental patches for 3.1.

Thank you for your interest in BFQ, and please do not hesitate to report about any issue you might have.
Re: [ANNOUNCE] BFQ-v7 for 3.2.0 - 3.13.0 is out! post-factum 1/28/14 5:10 AM
Many thanks, will try it ASAP.

Does it mean that after publishing tests results you will finally try to get BFQ merged by upstream?

Вівторок, 28 січня 2014 р. 12:51:27 UTC+2 користувач paolo написав:
Re: [ANNOUNCE] BFQ-v7 for 3.2.0 - 3.13.0 is out! Sergei Chudakov 1/28/14 9:34 AM
Nice! Where in main Linux kernel and Ubuntu kernel?
Re: [ANNOUNCE] BFQ-v7 for 3.2.0 - 3.13.0 is out! tux9656 1/28/14 3:24 PM
Thank you very much.  I am building 3.13 with BFQ v7 now.  My desktop machine runs Debian sid with btrfs and jfs filesystems on a SATA Western Digital black hard drive, ext4 on a SATA Western Digital green hard drive, btrfs on an external USB hard drive, and btrfs on a SATA Toshiba solid state drive.  My home server run Debian sid with btrfs with automatic daily snapshots on a PATA Western Digital hard drive.  Daily snapshots have been occurring since November 11, 2013 so there is quite a bit of metadata on that drive.  I'll try out BFQ v7 on both systems for a while and let you know how the performance and responsiveness is.
Re: [ANNOUNCE] BFQ-v7 for 3.2.0 - 3.13.0 is out! paolo 1/29/14 1:05 AM

Il giorno 28/gen/2014, alle ore 14:10, post-factum ha scritto:

> Many thanks, will try it ASAP.
Thanks.
>
> Does it mean that after publishing tests results you will finally try to get BFQ merged by upstream?
It seems like you are trying to read our minds … and you are succeeding. Yes, this should be the plan, but, as usual, our time constants are quite stretched … :)

Paolo
> --
> You received this message because you are subscribed to the Google Groups "bfq-iosched" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to bfq-iosched...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANNOUNCE] BFQ-v7 for 3.2.0 - 3.13.0 is out! paolo 1/31/14 11:50 PM

Il giorno 28/gen/2014, alle ore 18:34, Sergei Chudakov ha scritto:

> Nice! Where in main Linux kernel and Ubuntu kernel?
Slowly proceeding in this direction … As usual, we prefer to use our little time to first release/update/maintain what really matters, i.e., bfq itself :)
Re: [ANNOUNCE] BFQ-v7 for 3.2.0 - 3.13.0 is out! paolo 1/31/14 11:50 PM

Il giorno 29/gen/2014, alle ore 00:24, tux9656 ha scritto:

> Thank you very much.  I am building 3.13 with BFQ v7 now.  My desktop machine runs Debian sid with btrfs and jfs filesystems on a SATA Western Digital black hard drive, ext4 on a SATA Western Digital green hard drive, btrfs on an external USB hard drive, and btrfs on a SATA Toshiba solid state drive.  My home server run Debian sid with btrfs with automatic daily snapshots on a PATA Western Digital hard drive.  Daily snapshots have been occurring since November 11, 2013 so there is quite a bit of metadata on that drive.  I'll try out BFQ v7 on both systems for a while and let you know how the performance and responsiveness is.
>
Thx.