Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

fast method of cloning a hard drive?

5 views
Skip to first unread message

Orson Cart

unread,
Feb 5, 2012, 7:46:29 AM2/5/12
to

I try to clone a 250 GB disk PATA to a SATA drive with same CHS.
I use dd if=/dev/sdb of=/dev/sda
This is going for hours. In system monitor, I see one CPU going
about 30% for 20 seconds, then another CPU goes 30% for 30 seconds.
Just flip-flop forever. The HD LED is flashing.
System is Core 2 Quad with 6 GiB of RAM.
I think if I copied it to a USB disk it would have taken less time...

John Reiser

unread,
Feb 5, 2012, 9:59:59 AM2/5/12
to
> I try to clone a 250 GB disk PATA to a SATA drive with same CHS.
Are the PATA and SATA interfaces native to the same CPU motherboard?

> I use dd if=/dev/sdb of=/dev/sda
> This is going for hours. In system monitor, I see one CPU going
> about 30% for 20 seconds, then another CPU goes 30% for 30 seconds.
> Just flip-flop forever. The HD LED is flashing.
It should be ON nearly all the time. Any OFF duration should be
almost unnoticeable.

> System is Core 2 Quad with 6 GiB of RAM.
> I think if I copied it to a USB disk it would have taken less time...

250GB in + 250GB out ==> 500GB transferred; @50MB/s ==> 10,000 seconds;
about 2.8 hours. Near-perfect overlap of physical read and write
still requires almost 1.5 hours. The usual practice is to specify
"bs=32k" or "bs=64k" to reduce syscall overhead when using dd for
such a large copy. (The default blocksize for dd is bs=512 [bytes].)
Or, use ordinary /bin/cp instead of dd.

--


ray

unread,
Feb 5, 2012, 10:15:34 AM2/5/12
to
I doubt you have 250gb of data. It would probably have been much faster
to use partimage to backup the partition(s) to an external usb drive and
then restore to the new one.

Dan C

unread,
Feb 5, 2012, 11:30:28 AM2/5/12
to
On Sun, 05 Feb 2012 13:46:29 +0100, Orson Cart wrote:

Clonezilla.


--
"Ubuntu" -- an African word, meaning "Slackware is too hard for me".
"Bother!" said Pooh, as Lorena Bobbitt tied him to the bed.
Usenet Improvement Project: http://twovoyagers.com/improve-usenet.org/
Thanks, Obama: http://brandybuck.site40.net/pics/politica/thanks.jpg

Robert Heller

unread,
Feb 5, 2012, 11:55:11 AM2/5/12
to
Unless the source disk is 100% full, doing a file system copy (eg tar,
dump/restore, cpio, etc.) will almost allways be faster. Much of your
transfer file will be copying empty (unused) sectors (a complete waste
of time) and stuff like the swap partition(s) (also a complete waste of
time). Yes, going file system by file system is more tedious, but you
will save lots of time...

>

--
Robert Heller -- 978-544-6933 / hel...@deepsoft.com
Deepwoods Software -- http://www.deepsoft.com/
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments



unruh

unread,
Feb 5, 2012, 8:03:52 PM2/5/12
to
Use rsync not cp. Faster but also you will be sure that what is copied
is the same as what was original.


>

Johann Klammer

unread,
Feb 5, 2012, 9:13:33 PM2/5/12
to
Robert Heller wrote:
> At 5 Feb 2012 13:46:29 +0100 "Orson Cart"<ex-p...@parts.org> wrote:
>
>
> Unless the source disk is 100% full, doing a file system copy (eg tar,
> dump/restore, cpio, etc.) will almost allways be faster. Much of your
> transfer file will be copying empty (unused) sectors (a complete waste
> of time) and stuff like the swap partition(s) (also a complete waste of
> time). Yes, going file system by file system is more tedious, but you
> will save lots of time...
>

Hi there,

File system copy is definitely the way to go. But...
You have to be careful what and how you copy things.
I vaguely remember ending up with a unusable system after such a stunt.

The Hard-Disk-Upgrade HOWTO is a decent starting point for those cp...
incantations.

Regarding speed, the LINUX vm system is IMHO broken.
It likes to retain rarely used recently read pages in memory,
while happily swapping out application code.
This generates unnecessary pagefaults and slows down the system.
For sequential reads/writes this is somewhat deadly.

One may do
'watch echo 3 \> /proc/sys/vm/drop_caches'
as root to make the system more responsive in those cases.
The command drops clean filesystem caches every 2 seconds. It helps for
a lot of filesystem intensive things, unless you manage to fill up the
pagecache in less than two seconds.

When using dd, be sure to read its manpage, especially the part about
direct IO. The differences are significant.

regards,
JK


Johann Klammer

unread,
Feb 5, 2012, 9:17:59 PM2/5/12
to
Robert Heller wrote:
> At 5 Feb 2012 13:46:29 +0100 "Orson Cart"<ex-p...@parts.org> wrote:
>
>
> Unless the source disk is 100% full, doing a file system copy (eg tar,
> dump/restore, cpio, etc.) will almost allways be faster. Much of your
> transfer file will be copying empty (unused) sectors (a complete waste
> of time) and stuff like the swap partition(s) (also a complete waste of
> time). Yes, going file system by file system is more tedious, but you
> will save lots of time...
>

Orson Cart

unread,
Feb 6, 2012, 8:11:24 AM2/6/12
to

John Reiser <jrei...@comcast.net> wrote:
>> I try to clone a 250 GB disk PATA to a SATA drive with same CHS.
>Are the PATA and SATA interfaces native to the same CPU motherboard?
>
Um no, it has ICH8 or ICH9 which have no IDE anymore, so there is a Jmicron
chip for the legacy I/O.

patrick

unread,
Feb 6, 2012, 4:04:38 PM2/6/12
to
rsync -av source target is supposed to be equal to
cp -av source target


Both are quick.

The Natural Philosopher

unread,
Feb 6, 2012, 4:50:03 PM2/6/12
to
as long as the target is a formatted partition..

Pascal Hambourg

unread,
Feb 6, 2012, 5:38:22 PM2/6/12
to
Hello,

Robert Heller a écrit :
> At 5 Feb 2012 13:46:29 +0100 "Orson Cart" <ex-p...@parts.org> wrote:
>>
>> I try to clone a 250 GB disk PATA to a SATA drive with same CHS.
>> I use dd if=/dev/sdb of=/dev/sda
[...]
> Unless the source disk is 100% full, doing a file system copy (eg tar,
> dump/restore, cpio, etc.) will almost allways be faster.

Or unless the data on the filesystem are horribly fragmented, e.g. lots
of small files. Access time kills the throughput on magnetic disks.

The Natural Philosopher

unread,
Feb 6, 2012, 6:01:05 PM2/6/12
to
BUT at least you don't end up with a fragged target disk.

And its arguable with decent caching and RAM whether reading a fragged
disk is all that much slower anyway.

Pascal Hambourg

unread,
Feb 6, 2012, 6:17:50 PM2/6/12
to
The Natural Philosopher a écrit :
>
> And its arguable with decent caching and RAM whether reading a fragged
> disk is all that much slower anyway.

How does decent caching and RAM help in that matter ?

Eli the Bearded

unread,
Feb 6, 2012, 6:50:29 PM2/6/12
to
In comp.os.linux.misc, Robert Heller <hel...@deepsoft.com> wrote:
> At 5 Feb 2012 13:46:29 +0100 "Orson Cart" <ex-p...@parts.org> wrote:
> > I try to clone a 250 GB disk PATA to a SATA drive with same CHS.
> > I use dd if=/dev/sdb of=/dev/sda

Set the block size to match the blocks used by the actual devices. For
historical reasons dd defaults to a block size of 1/2 kilobyte. Modern
devices are more likely to use a block size of 8 kilobytes.

bs=8k

If the two devices have different block sizes, dd can rechunk for you
by setting separate ibs=N (input block size) and obs=M (output) values.

> Unless the source disk is 100% full, doing a file system copy (eg tar,
> dump/restore, cpio, etc.) will almost allways be faster. Much of your
> transfer file will be copying empty (unused) sectors (a complete waste

Faster != clone. If you are doing forensic analysis, the "empty" sectors
are unlikely a waste.

Faster is not certain. Some filesystems may be faster to copy as raw data
than as individual files. Consider ext2 with tens of thousands of files
in a single directory. I've seen cpio (and rsync) run out of memory when
trying to deal with filesystems that employ millions of hard linked
files: those programs try to cache the hard link information.

> of time) and stuff like the swap partition(s) (also a complete waste of
> time). Yes, going file system by file system is more tedious, but you
> will save lots of time...

dd can copy filesystems your kernel cannot understand. dd can copy
dying disks with the need to fsck on the bad drive. dd can swap byte
endianness for you (conv=swab).

Without more information about the original use case, you can't be
certain what the best approach is.

Elijah
------
has used dd for backing up Mac HFS volumes

The Natural Philosopher

unread,
Feb 6, 2012, 8:39:05 PM2/6/12
to
the subsystem reads whole blocks and holds them in RAM till they are
needed.

unruh

unread,
Feb 7, 2012, 4:43:55 AM2/7/12
to
but it is not. It is a different program from cp.

>
>
> Both are quick.
>

patrick

unread,
Feb 20, 2012, 11:20:01 PM2/20/12
to
You're right.

cp -av source target
rsync -aHAX -v source target

Nothing wrong with the cp command.

unruh

unread,
Feb 21, 2012, 12:40:11 AM2/21/12
to
except rsync also hashes the source and target to make sure that what is
transfered is the same as what was supposed to be transfered.

And if the target is almost the same as source, it transfers only the
parts that are different.

Pascal Hambourg

unread,
Feb 21, 2012, 3:27:04 PM2/21/12
to
Eli the Bearded a écrit :
>>> I try to clone a 250 GB disk PATA to a SATA drive with same CHS.
>>> I use dd if=/dev/sdb of=/dev/sda
>
> Set the block size to match the blocks used by the actual devices. For
> historical reasons dd defaults to a block size of 1/2 kilobyte. Modern
> devices are more likely to use a block size of 8 kilobytes.

Any source, please ?
AFAIK, the Advanced Format sector size is 4 KiB.
Anyway, even with 512-byte sector size, I observed it is faster to use a
bigger block size such as 4 KiB.
0 new messages