file corruption solution (soft-update or ZFS)

saeedeh motlagh

unread,

May 23, 2013, 5:21:01 AM5/23/13

to

hello every body

i have a question about fixing file corruption in freebsd.

now i have freebsd8.2 and some times file corruption happened on it. this
issue has a heavy cost for me and i want to avoid it or fixit it
completely. so my question is:

is it better to upgrade my freebsd to 9.1 and use soft update or migrate
from UFS to ZFS?

i heard so much about soft update -that is added in freebsd9.1- which can
fix file corruption in acceptable way with low cost but i don't know how
much is reliable and efficient.

in the other hand, i think migration from UFS to ZFS can be another
solution. as i read ZFS is is created to solve all the problems related
integrity file system. is it reliable enough in comparison soft-update?

now, i want to know which solution is better and why?
thanks in advance
s.motlagh
_______________________________________________
freebsd-...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questi...@freebsd.org"

Warren Block

unread,

May 23, 2013, 7:58:45 AM5/23/13

to

On Thu, 23 May 2013, saeedeh motlagh wrote:

> hello every body
>
> i have a question about fixing file corruption in freebsd.
>
> now i have freebsd8.2 and some times file corruption happened on it. this
> issue has a heavy cost for me and i want to avoid it or fixit it
> completely. so my question is:
>
> is it better to upgrade my freebsd to 9.1 and use soft update or migrate
> from UFS to ZFS?

That's a judgement call, which means "it depends".

> i heard so much about soft update -that is added in freebsd9.1- which can
> fix file corruption in acceptable way with low cost but i don't know how
> much is reliable and efficient.

Several things:

Soft updates have been around for quite a while.
Soft updates journaling is the new addition.
Neither of these address file corruption. Their purpose is to make sure
the filesystem does not get corrupted, but individual files could still
contain bad data.

> in the other hand, i think migration from UFS to ZFS can be another
> solution. as i read ZFS is is created to solve all the problems related
> integrity file system. is it reliable enough in comparison soft-update?
>
> now, i want to know which solution is better and why?

Again, it depends. Does the target system have enough RAM for ZFS? If
the file corruption is due to a hardware problem or an application
writing bad data, no filesystem can prevent that.

saeedeh motlagh

unread,

May 23, 2013, 8:14:06 AM5/23/13

to

thanks for your reply.

you know i have a sensitive server and unfortunately it is located some
where that power outage happens much. so i want guarantee my data and avoid
data lost and file corruption in my server.

i do not have any problem in RAM and hardware.

i don't know which approach is more suitable for my server. using
soft-update or ZFS. please help me to select the best one.

thank you so much

--
*Sa.M*

Trond Endrestøl

unread,

May 23, 2013, 8:18:30 AM5/23/13

to

On Thu, 23 May 2013 16:44+0430, saeedeh motlagh wrote:

> thanks for your reply.
>
> you know i have a sensitive server and unfortunately it is located some
> where that power outage happens much. so i want guarantee my data and avoid
> data lost and file corruption in my server.

Maybe you should also invest in a decent UPS.

Warren Block

unread,

May 23, 2013, 8:33:00 AM5/23/13

to

On Thu, 23 May 2013, saeedeh motlagh wrote:

> you know i have a sensitive server and unfortunately it is located some where that power outage happens much. so i want guarantee my data and avoid data lost and file corruption in my
> server.
>

> i do not have any problem in RAM and hardware.

The lack of a UPS can be considered a hardware problem.

> i don't know which approach is more suitable for my server. using soft-update or ZFS. please help me to select the best one.

Please don't top-post, as it makes responding to your message more
difficult. One thing mentioned earlier is that ZFS wants lots of
memory. 4G-8G minimum, some might say as much as the server will hold.

But resilient filesystems still can't prevent data corruption. Fix the
power problem with a UPS.

Michael Sierchio

unread,

May 23, 2013, 11:09:27 AM5/23/13

to

On Thu, May 23, 2013 at 5:33 AM, Warren Block <wbl...@wonkity.com> wrote:

> ..

> One thing mentioned earlier is that ZFS wants lots of memory. 4G-8G
> minimum, some might say as much as the server will hold.
>
>

Not necessarily so - deduplication places great demands on memory, but that
can be satisfied with dedicated cache devices (on SSD for performance and
safety reasons). Without dedup, the requirements are more modest.

Softupdates guarantee metadata consistency, but do nothing to address data
integrity. ZFS has copy-on-write semantics (which solve a problem that even
hardware RAID can't), and end-to-end checksums to detect/prevent data
corruption (large drives will have uncorrectable bit errors over their
lifetime).

- M

Joshua Isom

unread,

May 23, 2013, 7:08:59 PM5/23/13

to

On 5/23/2013 7:14 AM, saeedeh motlagh wrote:
> thanks for your reply.
>

> you know i have a sensitive server and unfortunately it is located some
> where that power outage happens much. so i want guarantee my data and avoid
> data lost and file corruption in my server.

Get a good reliable UPS. Test it regularly, the batteries do fail.
Test to make sure that it will work, unplug it and let the computer
drain the battery to time it. Consider that the battery will degrade
over time. One thing google does is put a 12V battery inside the
chassis to help with the power backup, you might look into it.

> i do not have any problem in RAM and hardware.
>

> i don't know which approach is more suitable for my server. using
> soft-update or ZFS. please help me to select the best one.

If power failure is an issue, you have no guarantee of data loss
protection unless you use networked storage to a safe place. UFS soft
updates protects against file system corruption in case of power loss,
no guarantees of individual file consistency. ZFS guarantees no silent
failures, it doesn't guarantee protection, only that you'll know about
it. There is no filesystem that can guarantee you won't lose data in a
power failure. Hard drives are known to lie about what's been
physically synced to disk out of cache in order to improve speed. If
the power goes out at the wrong time, you can lose data. ZFS can find a
corrupted file and tell you, everything else won't. If you have a back
up of that file, you can restore it.

> thank you so much
>
>
>
> On Thu, May 23, 2013 at 4:28 PM, Warren Block <wbl...@wonkity.com> wrote:
>

>> On Thu, 23 May 2013, saeedeh motlagh wrote:
>>

Paul Kraus

unread,

May 25, 2013, 11:44:59 AM5/25/13

to

On May 23, 2013, at 11:09 AM, Michael Sierchio <ku...@tenebras.com> wrote:

> On Thu, May 23, 2013 at 5:33 AM, Warren Block <wbl...@wonkity.com> wrote:
>
>> ..
>
>> One thing mentioned earlier is that ZFS wants lots of memory. 4G-8G
>> minimum, some might say as much as the server will hold.
>>
>>
> Not necessarily so - deduplication places great demands on memory, but that
> can be satisfied with dedicated cache devices (on SSD for performance and
> safety reasons). Without dedup, the requirements are more modest.

The rule of thumb for DeDupe is 1GB physical RAM for every 1TB of capacity. The issue is that the DeDupe metadata table must live in the ARC for good performance. The discussion I have seen on the ZFS lists indicates that L2ARC is not really adequate for this, so adding cache devices (SSD's) don't really help.

On the other hand, you can use ZFS without DeDupe with as little as 2GB of total system RAM (depending on what else the system is doing). In my experience, the amount of RAM depends on the amount of I/O not the amount of storage. I find between 1GB and 3GB space for the ARC is adequate.

--
Paul Kraus
Deputy Technical Director, LoneStarCon 3
Sound Coordinator, Schenectady Light Opera Company