Hi,

I have a server running RH Linux 9 with two HDD slots.  What is the easiest
way to clone a disk, i.e. copy the whole and exact image of the disk in slot
1 to an identical disk in slot 2?

Thanks,

T.
--

On Thu, 2 Sep 2004 11:23:19 -0500, L staggered into the Black Sun and
said:

cp -a , or tar , or partimage ( http://partimage.com/ ).  If any
filesystem on disk 0 is mounted and is writeable, copying that
filesystem to disk 1 will be difficult because files on the mounted
filesystem may change while the copy/tar/partimage is running.

--
Matt G|There is no Darkness in Eternity/But only Light too dim for us to see
Brainbench MVP for Linux Admin /    mail: TRAP + SPAN don't belong
http://www.brainbench.com     /                Hire me!
-----------------------------/ http://crow202.dyndns.org/~mhgraham/resume

i have used cp -a (the -a preserves attributes and
ownerships) with great success to transfer the whole system
of a linux box to a newer one. Don't forget though to do
this for every partition (for example /boot and / ). You'll
also have to install the boot sector manually with lilo or grub.
to avoid the problem of files changing during the copy, i
booted with a live linux cd (i used SystemRescueCd).
Also, if the two disks are exactly the same, dd could also
be used i guess.

news:41375558$0$26992$626a14ce@news.free.fr...

Can I use /dev/sda (the source disk) and /dev/sdb (the dest disk) to copy
the whole and exact image from the src disk to the dest disk some how?

Thanks,

T.
--

news:2pp3q4Fn5l9mU1@uni-berlin.de...

Once boot up Linux from the source disk, can I use "dd if=/dev/sda
of=/dev/sdb" to copy the whole and exact image from the source disk
(/dev/sda) to the dest disk (/dev/sdb)?

Thanks,

T.
--

On Thu, 2 Sep 2004 12:42:52 -0500, L staggered into the Black Sun and
said:

Why did you post this twice?

You can, but it'd be a really dumb idea.  dd copies *everything*, including
all the unused space on the disk, so it's slower than using tar or cp -a
or partimage.  If the disks have different sizes, the partition table on
the destination disk will also be incorrect.  If you decide to use dd
anyway, don't say we didn't warn you, and make sure to pass the bs=32k
parameter, since dd uses a default blocksize of 512 bytes, and that will
make things take even longer.

Use tar or cp -a or partimage.  Really.  That's what they are designed
for.

--
Matt G|There is no Darkness in Eternity/But only Light too dim for us to see
Brainbench MVP for Linux Admin /    mail: TRAP + SPAN don't belong
http://www.brainbench.com     /                Hire me!
-----------------------------/ http://crow202.dyndns.org/~mhgraham/resume

L <l_x2...@yahoo.com> did eloquently scribble:

If the two disks have identical sizes and geometries
(sectors/tracks/cylinders), then yes.

dd if=/dev/sda of=/dev/sdb bs=512

If the second disk is bigger, you may have problems with that method, cos
your partitions will be the same sizes and you might not be able to make
more... I've never tried dd on different sized disks.
--
___________________________________________________________________________ ___
|   spi...@freenet.co.uk   | "I'm alive!!! I can touch! I can taste!         |
|Andrew Halliwell BSc(hons)|  I can SMELL!!!  KRYTEN!!! Unpack Rachel and    |
|            in            |  get out the puncture repair kit!"              |
|     Computer Science     |     Arnold Judas Rimmer- Red Dwarf              |
--------------------------------------------------------------------------- ---

/ ...

/ ...

Between posts, you added a condition that poses a problem. If you are using
the source disk as your active OS while copying, the destination copy will
not be identical to the source. The result can range between no noticeable
problems to a total and complete failure of the filesystem.

It is much better to boot from a bootable CD or other medium, and copy only
unmounted drives.

--
Paul Lutus
http://www.arachnoid.com

You can use dd if=/dev/sda of=/dev/sdb, but to be sure it copies
correctly, you should probably do a checksum. I've copied 1 MB
blocks and done the checksum using MD5 on each block transferred.
Look at the man page for MD5 to see how to do it. The man page for dd
will show you how to skip to the area you want to copy. Look for the
options for skip and seek (for the input and output) as well as set
the block size (bs=?).

Or you could just use Ghost (from Symantec) to do it. It does
essentially the same thing.

Jim

  Alexander Skwar <f...@alexander.skwar.name>,
  In a message on Thu, 02 Sep 2004 20:20:24 +0200, wrote :

AS>
AS> > If the two disks have identical sizes and geometries
AS> > (sectors/tracks/cylinders), then yes.
AS> >
AS> > dd if=/dev/sda of=/dev/sdb bs=512
AS>
AS> I'm curious: Why's everyone using dd for this and not the IMO simpler
AS> cat (as in "cat /dev/sda > /dev/sdb"). Is there any advantage in
AS> using dd?

Dd will work properly at the device level.  Cat is really only meant for
concatenating normal files.  DD can also perform buffering and
re-blocking (see 'man dd' for more details), which is sometimes needed
when moving data at this level.

Note that *any* I/O below the file system level is dangerous can extreme
care needs to be taken.

spike1's proviso 'If the two disks have identical sizes and geometries'
is a very important condition.  While it is possible to copy from a
smaller to a larger disk and 'get away with it', the reverse will be bad
news.  There can also be problems if one (or both) of the disks have bad
blocks that are mapped out at the FS level.

AS>
AS> Alexander Skwar
AS> --
AS> You can't start worrying about what's going to happen.  You get spastic
AS> enough worrying about what's happening now.
AS>          -- Lauren Bacall
AS> ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
AS>            

                                     \/
Robert Heller                        ||InterNet:   hel...@cs.umass.edu
http://vis-www.cs.umass.edu/~heller  ||            hel...@deepsoft.com
http://www.deepsoft.com              /\FidoNet:    1:321/153

  Paul Lutus <nos...@nosite.zzz>,
  In a message on Thu, 02 Sep 2004 11:24:08 -0700, wrote :

PL>
PL> / ...
PL>
PL> >> Can I use /dev/sda (the source disk) and /dev/sdb (the dest disk) to copy
PL> >> the whole and exact image from the src disk to the dest disk some how?
PL>
PL> / ...
PL>  
PL> > Once boot up Linux from the source disk, can I use "dd if=/dev/sda
PL> > of=/dev/sdb" to copy the whole and exact image from the source disk
PL> > (/dev/sda) to the dest disk (/dev/sdb)?
PL>
PL> Between posts, you added a condition that poses a problem. If you are using
PL> the source disk as your active OS while copying, the destination copy will
PL> not be identical to the source. The result can range between no noticeable
PL> problems to a total and complete failure of the filesystem.
PL>
PL> It is much better to boot from a bootable CD or other medium, and copy only
PL> unmounted drives.

It is quite possible to successfully 'clone' a running system, using
a disk-to-disk dump (eg: 'dump 0f - / | (cd /newdisk;restore rf -)',
repeat for /var, /usr, /home, etc.).  You do need some kind of boot
media (rescue floppy/CD-ROM) to re-install the boot loader, if needed.

Using dd to copy a 'live' system disk is likely to be bad news. It is
not a problem using dump/cpio/tar/cp/etc. since these are work at the FS
level and won't be affected by (normal) low-level changes to the
filesystem -- all that would be lost would be randomly updated system
files (eg log files), which won't affect the runablity of the cloned
system disk.

PL>
PL> --
PL> Paul Lutus
PL> http://www.arachnoid.com
PL>
PL>                                                                              

                                     \/
Robert Heller                        ||InterNet:   hel...@cs.umass.edu
http://vis-www.cs.umass.edu/~heller  ||            hel...@deepsoft.com
http://www.deepsoft.com              /\FidoNet:    1:321/153

On Thu, 02 Sep 2004 20:20:24 +0200, Alexander Skwar staggered into the
Black Sun and said:

Block size.  You can specify a block size to dd.  It's faster to do 1
read() of 32k than it is to do 64 read()s of 512 bytes.  cat probably
reads 4k at a time, and that's not optimal when you're copying a large
chunk of data.

And if you didn't know, dd is a *terrible* way to back up and/or mirror
something.  It copies everything, even unused space, so it's slower than
tar or cp -a or partimage if your filesystem isn't 100% full.  If the
second disk has a different size or a different geometry, the partition
table you copied from the first disk will be incorrect.  The only
advantage dd has over other backup/mirror methods is that you don't have
to reinstall GRUB or LILO.  Reinstalling GRUB takes 4 commands, which
you can put in a shell script like so:

#!/bin/bash
/sbin/grub --batch << EOF > /dev/null
root (hdX,Y)
setup (hdX)
quit
EOF
# modify X and Y for your particular setup.

...I forget what the proper commands are for LILO as it's been several
years since I've used it.

--
Matt G|There is no Darkness in Eternity/But only Light too dim for us to see
Brainbench MVP for Linux Admin /    mail: TRAP + SPAN don't belong
http://www.brainbench.com     /                Hire me!
-----------------------------/ http://crow202.dyndns.org/~mhgraham/resume

  Alexander Skwar <f...@alexander.skwar.name>,
  In a message on Thu, 02 Sep 2004 22:09:55 +0200, wrote :

AS>
AS> > Dd will work properly at the device level.  Cat is really only meant for
AS> > concatenating normal files.
AS>
AS> Okay, maybe I don't have enough experience, but did you ever have
AS> problems with cat, that were not present with dd?

Cat lacks some useful options:

dd if=boot.img of=/dev/fd0 bs=9216

Copies a floppy disk image and does the floppy I/O in track sized
buffers.  This is a bit faster than cat boot.img >/dev/fd0, which does
it byte-by-byte (yes, there is buffering inside the glibc code, but not
specifically floppy track sized).  Also dd's parameter methodology is
more convenient than cat when used with sudo. Compare:

sudo dd if=boot.img of=/dev/fd0

vs.

sudo sh -c 'cat  boot.img >/dev/fd0'

Cat is quite useful in many situations (I use it all of the time), but
dd is a more powerful command and has many advantages in *other*
situations.  Compare using a hammer to 'pound in' a screw.  This can be
done, but a screwdriver works better (to drive a screw).

AS>
AS> I haven't.
AS>
AS>
AS> Alexander Skwar
AS> --
AS> I must Create a System, or be enslav'd by another Man's;
AS> I will not Reason and Compare; my business is to Create.
AS>          -- William Blake, "Jerusalem"
AS> ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
AS>                                                                                

                                     \/
Robert Heller                        ||InterNet:   hel...@cs.umass.edu
http://vis-www.cs.umass.edu/~heller  ||            hel...@deepsoft.com
http://www.deepsoft.com              /\FidoNet:    1:321/153

  Dances With Crows <danSPANceswitTRAPhcr...@gmail.com>,
  In a message on 2 Sep 2004 20:21:19 GMT, wrote :

DWC> On Thu, 02 Sep 2004 20:20:24 +0200, Alexander Skwar staggered into the
DWC> Black Sun and said:

DWC> >> If the two disks have identical sizes and geometries
DWC> >> (sectors/tracks/cylinders), then yes.  dd if=/dev/sda of=/dev/sdb
DWC> >> bs=512
DWC> > I'm curious: Why's everyone using dd for this and not the IMO simpler
DWC> > cat (as in "cat /dev/sda > /dev/sdb"). Is there any advantage in using
DWC> > dd?
DWC>
DWC> Block size.  You can specify a block size to dd.  It's faster to do 1
DWC> read() of 32k than it is to do 64 read()s of 512 bytes.  cat probably
DWC> reads 4k at a time, and that's not optimal when you're copying a large
DWC> chunk of data.
DWC>
DWC> And if you didn't know, dd is a *terrible* way to back up and/or mirror
DWC> something.  It copies everything, even unused space, so it's slower than
DWC> tar or cp -a or partimage if your filesystem isn't 100% full.  If the
DWC> second disk has a different size or a different geometry, the partition
DWC> table you copied from the first disk will be incorrect.  The only
DWC> advantage dd has over other backup/mirror methods is that you don't have
DWC> to reinstall GRUB or LILO.  Reinstalling GRUB takes 4 commands, which
DWC> you can put in a shell script like so:
DWC>
DWC> #!/bin/bash
DWC> /sbin/grub --batch << EOF > /dev/null
DWC> root (hdX,Y)
DWC> setup (hdX)
DWC> quit
DWC> EOF
DWC> # modify X and Y for your particular setup.
DWC>
DWC> ...I forget what the proper commands are for LILO as it's been several
DWC> years since I've used it.

Assuming that /etc/lilo.conf exists (it would be copied with
tar/cp/cpio/dump/etc.), then it is just:

/sbin/lilo -r /newdisk

Otherwise lilo takes ALL of the config options directly on the command
line.  So, lilo always takes one command, which may be simple or
complex, depending.

DWC>
DWC> --
DWC> Matt G|There is no Darkness in Eternity/But only Light too dim for us to see
DWC> Brainbench MVP for Linux Admin /    mail: TRAP + SPAN don't belong
DWC> http://www.brainbench.com     /                Hire me!
DWC> -----------------------------/ http://crow202.dyndns.org/~mhgraham/resume
DWC>        

                                     \/
Robert Heller                        ||InterNet:   hel...@cs.umass.edu
http://vis-www.cs.umass.edu/~heller  ||            hel...@deepsoft.com
http://www.deepsoft.com              /\FidoNet:    1:321/153

Okay.

Is it? Some "benchmarks". Those are all filesystems on a LVM.
The fs were NOT mounted (neither nor anytime at all after boot).
So the blocks should not be cached.

The filesystems are of the same size and should not be
"interleaved".

server read-test # time cat /dev/system_vg/ext2 > cat ; sync ; time dd if=/dev/system_vg/ext3 of=dd-32k bs=32k

real    1m55.575s
user    0m0.136s
sys     0m9.025s

real    2m21.200s
user    0m0.087s
sys     0m8.795s

I did more tests with other filesystems of the same size
and always get about the same times - dd with bs=32k is
slower than cat.

Am I doing something wrong?

Uhm, how do you make an *IMAGE* with tar or cp? I use those only
for making backups, but not for IMAGES. The good thing about dd/cat is,
that it reads the whole partition/harddisk - INCLUDING the
free space. This way, you get an EXACT copy. Especially useful, if
you make an image of the whole hard drive, because you then
also have a copy of the MBR and partitioning. Thus, you'll only
need to do a "dd if=backup-file of=/dev/hda" to get a fully
working systme.

With tar/cp, you'll also need to have some documentation about
how the system was partitioned and how to install the boot loader.

Yes, since partimage (*VERY* roughly) does a dd of the hard drive
and thus cannot be compared at all to a backup of the files on
the filesystem, which is what tar and cp do.

Alexander Skwar
--
Do Miami a favor.  When you leave, take someone with you.
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

It depends on the number of files, their total size, the distribution of
their sizes, and the existing fragmentation.  I have seen ext2/ext3 file-
systems that are only 85% full but which have over 40% non-contiguous files
(used for software development), and dd is faster than tar, cp, or partimage
because of fewer seeks: one seek per cylinder for dd, zillions of seeks for
anything that deals with all the files.  If the filesystem is nearly full,
then dd is attractive for shortest clock time.  On the other hand, a file-
by-file backup-then-restore gives "free" defragmentation if you endure
the seeks during backup.

--

  Alexander Skwar <f...@alexander.skwar.name>,
  In a message on Thu, 02 Sep 2004 23:13:30 +0200, wrote :

AS> Dances With Crows wrote:
AS> > On Thu, 02 Sep 2004 20:20:24 +0200, Alexander Skwar staggered into the
AS> > Black Sun and said:
AS> >

AS> >>
AS> >>>If the two disks have identical sizes and geometries
AS> >>>(sectors/tracks/cylinders), then yes.  dd if=/dev/sda of=/dev/sdb
AS> >>>bs=512
AS> >>
AS> >>I'm curious: Why's everyone using dd for this and not the IMO simpler
AS> >>cat (as in "cat /dev/sda > /dev/sdb"). Is there any advantage in using
AS> >>dd?
AS> >
AS> >
AS> > Block size.  You can specify a block size to dd.
AS>
AS> Okay.
AS>
AS> > reads 4k at a time, and that's not optimal when you're copying a large
AS> > chunk of data.
AS>
AS> Is it? Some "benchmarks". Those are all filesystems on a LVM.
AS> The fs were NOT mounted (neither nor anytime at all after boot).
AS> So the blocks should not be cached.
AS>
AS> The filesystems are of the same size and should not be
AS> "interleaved".
AS>
AS> server read-test # time cat /dev/system_vg/ext2 > cat ; sync ; time dd if=/dev/system_vg/ext3 of=dd-32k bs=32k
AS>
AS> real    1m55.575s
AS> user    0m0.136s
AS> sys     0m9.025s
AS>
AS> real    2m21.200s
AS> user    0m0.087s
AS> sys     0m8.795s
AS>
AS> I did more tests with other filesystems of the same size
AS> and always get about the same times - dd with bs=32k is
AS> slower than cat.
AS>
AS> Am I doing something wrong?
AS>
AS> > And if you didn't know, dd is a *terrible* way to back up and/or mirror
AS> > something.  It copies everything, even unused space, so it's slower than
AS> > tar or cp -a
AS>
AS> Uhm, how do you make an *IMAGE* with tar or cp? I use those only
AS> for making backups, but not for IMAGES. The good thing about dd/cat is,
AS> that it reads the whole partition/harddisk - INCLUDING the
AS> free space. This way, you get an EXACT copy. Especially useful, if
AS> you make an image of the whole hard drive, because you then
AS> also have a copy of the MBR and partitioning. Thus, you'll only
AS> need to do a "dd if=backup-file of=/dev/hda" to get a fully
AS> working systme.

Image backups can be seriously bad news, since they are not 'portable'.
If I make a tar backup of my Linux box (of some *random* version of the
O/S with some random version of ext2 or ext3 or whatever file system), I
can view/extract/whatever elements of that tar file on some other
system, including not only *any* other UNIX flavor, but ALSO MS-Windows
or MacOS (classic OR OSX) -- WinZIP understands tar files, so does
StuffitExpander.  If I have an image backup, I can only make use of it
by getting and *exactly* twin of the original system (same O/S, same fs
version, and importantly, the *same* disk).

AS>
AS> With tar/cp, you'll also need to have some documentation about
AS> how the system was partitioned and how to install the boot loader.
AS>

/sbin/fdisk -l | lpr -

lpr /etc/lilo.conf
lpr /etc/grub.conf              (guessing -- I don't use grub myself).

AS> > partimage
AS>
AS> Yes, since partimage (*VERY* roughly) does a dd of the hard drive
AS> and thus cannot be compared at all to a backup of the files on
AS> the filesystem, which is what tar and cp do.
AS>
AS> Alexander Skwar
AS> --
AS> Do Miami a favor.  When you leave, take someone with you.
AS> ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
AS>                                                                                                    

                                     \/
Robert Heller                        ||InterNet:   hel...@cs.umass.edu
http://vis-www.cs.umass.edu/~heller  ||            hel...@deepsoft.com
http://www.deepsoft.com              /\FidoNet:    1:321/153

Hmmm... actually, all the options to dd make it look more
like a Swiss Army Knife :)

Where I found dd the most useful, in the dim dark past
where computers exchanged data mostly via tapes,
was its capability to convert formats.

Consider reading an EBCDIC 9-track tape created on an IBM
mainframe:

dd if=/dev/rmt0n of=tape_header conv=ascii,lcase cbs=80 count=1
# the obligatory tape header read in through the
# "no rewind" minor-device of the driver
dd if=/dev/rmt0n of=/dev/null ibs=128 skip=10000
# skip first 10000 records of 128 bytes each, no rewind
dd if=/dev/rmt0 of=tape_file conv=ascii,lcase cbs=128

(Obviously, all of the above was in a script, because
I wasn't about to type that at the console every time.
If I had been able to "cat" those steenking tapes, I would
have :)

Nowadays, the need for this has almost disappeared,
I'd guess, but dd is still around.

It all depends on the problem you're trying to solve.
If your problem is strictly copying one physical
device to another identical physical device, then,
by all means, use "cat", since it doesn't carry all
that baggage in it's code of converting and variable
block sizes and such.

If you're going to back up to a device which is not
identical in all respects, tarballs and/or cpio
(whichever you feel most comfortable with) are
probably more appropriate.

If you have something out of the ordinary, like
creating a floppy with some odd-ball block size,
then the "swiss army knife" is probably the way to
go.

Use the right tool for the right job.

NPL

--
"It is impossible to make anything foolproof
because fools are so ingenious"
  - A. Bloch