Prioritize resilvering priority

[javocado]

Hi,

How does one go about prioritizing the resilvering process so it does not
overwhelm normal disk I/O on a FreeBSD (8.3amd) system? Further, can this
be altered in real time, or do the settings have to be in place prior to
the resilvering even starting?

Thanks!
_______________________________________________
freeb...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-fs
To unsubscribe, send any mail to "freebsd-fs-...@freebsd.org"

--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-...@muc.de

[Gary Palmer]

On Tue, Jul 21, 2015 at 04:06:05PM -0700, javocado wrote:
> Hi,
>
> How does one go about prioritizing the resilvering process so it does not
> overwhelm normal disk I/O on a FreeBSD (8.3amd) system? Further, can this
> be altered in real time, or do the settings have to be in place prior to
> the resilvering even starting?

You don't state, but is this on ZFS?

I assume they're on 8.3 also, but look at settings like
vfs.zfs.resilver_min_time_ms and vfs.zfs.resilver_delay in sysctl.
Searching for resilver_min_time_ms and resilver_delay may give
some hints, e.g.

http://broken.net/uncategorized/zfs-performance-tuning-for-scrubs-and-resilvers/

does the opposite to what you want, but it should give hints. Also

https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/zfs-advanced.html

I think they're dynamic. I tuned the scrub settings and they took effect
immediately.

Regards,

Gary

[Michelle Sullivan]

Gary Palmer wrote:
> On Tue, Jul 21, 2015 at 04:06:05PM -0700, javocado wrote:
>
>> Hi,
>>
>> How does one go about prioritizing the resilvering process so it does not
>> overwhelm normal disk I/O on a FreeBSD (8.3amd) system? Further, can this
>> be altered in real time, or do the settings have to be in place prior to
>> the resilvering even starting?
>>
>
> You don't state, but is this on ZFS?
>
> I assume they're on 8.3 also, but look at settings like
> vfs.zfs.resilver_min_time_ms and vfs.zfs.resilver_delay in sysctl.
> Searching for resilver_min_time_ms and resilver_delay may give
> some hints, e.g.
>
> http://broken.net/uncategorized/zfs-performance-tuning-for-scrubs-and-resilvers/
>
> does the opposite to what you want, but it should give hints. Also
>
> https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/zfs-advanced.html
>
> I think they're dynamic. I tuned the scrub settings and they took effect
> immediately.
>
>

They mostly the settings are dynamic (both the ones you have above are)
... at worst some settings only take effect at boot time, so if you
wanted to change them reboot. None of the settings have to be made
before resilvering commences (remember that when you reboot, the
resilver stops then is resumed on startup so any 'boot tunables' take
effect)


--
Michelle Sullivan
http://www.mhix.org/

[javocado]

Thanks for all that feedback. Yes, this is for zfs. It looks like some of
those sysctls are not available in 8.3:

# sysctl vfs.zfs.scrub_delay
sysctl: unknown oid 'vfs.zfs.scrub_delay'

# sysctl vfs.zfs.resilver_delay
sysctl: unknown oid 'vfs.zfs.resilver_delay'

# sysctl vfs.zfs.zfs_top_maxinflight
sysctl: unknown oid 'vfs.zfs.zfs_top_maxinflight'


But I do have:
vfs.zfs.vdev.max_pending: 10 (dynamic)
vfs.zfs.scrub_limit: 10 (loader)

So, I think I would want to lower one or both of these to increase I/O
responsiveness on the system. Correct? How would the 2 play together in
terms of which to adjust to achieve the best system performance at the
expense of a longer resilver?

[Paul Kraus]

On Jul 22, 2015, at 14:52, javocado <javo...@gmail.com> wrote:

> But I do have:
> vfs.zfs.vdev.max_pending: 10 (dynamic)
> vfs.zfs.scrub_limit: 10 (loader)
>
> So, I think I would want to lower one or both of these to increase I/O
> responsiveness on the system. Correct? How would the 2 play together in
> terms of which to adjust to achieve the best system performance at the
> expense of a longer resilver?

vfs.zfs.vdev.max_pending is the limit on the number of disk I/O that can be outstanding for a drive (or, IIRC, in this case a given vdev). There has been great debate over tuning this one years ago on the zfs list. The general consensus is that 10 is a good value for modern SATA drives. When I was running 4 SATA drives behind a port multiplier (not a great configuration) I tuned this down to 4 to keep from overwhelming the port multiplier. Tuning it _down_ will reduce overall throughput to a drive. It does not differentiate between production I/O and scrub / resilver I/O.

This post: https://forums.freebsd.org/threads/how-to-limit-scrub-bandwidth-vfs-zfs-scrub_limit.31628/

Implies that the vfs.zfs.scrub_limit parameter limits the number of outstanding I/O but just for scrub / resilver operations. I would start by tuning it down to 5 or so and watch carefully with iostat -x to see the effect.

Note that newer ZFS code addresses the scrub operation starving the rest of the system from I/O. I have not had a problem on either my FBSD 9 or 10 systems.

--
Paul Kraus
pa...@kraus-haus.org