What's the best way to move data via ZFS ?
Dave
boot), the other a new Sun Ultra 27 (OpenSolaris 06/2009, ZFS boot).
I'd like to copy a lot of the files from the Blade 2000 to the Ultra 27.
The Blade 2000 has about 1 TB of data in total. Most of that is on external USB
disks drives, though some is on SCSI disk. There is a USB 2.0 card in the Blade
2000 used for the external drives.
The Ultra 27 has only internal SATA drives.
I've stuck a Quad Gigabit ethernet card in the Blade 2000, so the network
transfer rate should I believe be limited by the speed of the SCSI disks, rather
than the network. (I assume the SCSI disks work at 20 or 40 MB/s in the Blade
2000 - nothing like the 320 MB/s disk speed).
What is the best way to move the data?
I suspect using ZFS send/receive is probably best, but are not exactly sure the
best way to do it.
One option is to physically remove the external USB drives from the Blade 2000,
and put them on the Ultra 27. I do not however believe that is going to gain me
anything, as the Gigabit Ethernet should be faster than USB 2.0.
This is the disk layout of the Blade 2000, with 3 SCSI disks, and two SATA disks.
drkirkby@swan:[~] $ zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
fcaldisk 136G 23.9G 112G 17% ONLINE -
mypool 928G 693G 235G 74% ONLINE -
rpool 68G 8.69G 59.3G 12% ONLINE -
s6pool 68G 4.57G 63.4G 6% ONLINE -
Filesystem size used avail capacity Mounted on
rpool/ROOT/s10s_u7wos_08
67G 7.7G 56G 13% /
/devices 0K 0K 0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 8.1G 416K 8.1G 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
sharefs 0K 0K 0K 0% /etc/dfs/sharetab
/platform/sun4u-us3/lib/libc_psr/libc_psr_hwcap1.so.1
64G 7.7G 56G 13%
/platform/sun4u-us3/lib/libc_psr.so.1
/platform/sun4u-us3/lib/sparcv9/libc_psr/libc_psr_hwcap1.so.1
64G 7.7G 56G 13%
/platform/sun4u-us3/lib/sparcv9/libc_psr.so.1
fd 0K 0K 0K 0% /dev/fd
swap 8.1G 104K 8.1G 1% /tmp
swap 8.1G 40K 8.1G 1% /var/run
/dev/dsk/c2t1d0s0 19G 17G 1.8G 91% /oldroot
/dev/dsk/c2t1d0s3 108G 82G 25G 77% /usr/local
rpool/export 67G 20K 56G 1% /export
fcaldisk 134G 24G 110G 18% /export/home
mypool 914G 224G 220G 51% /mypool
mypool/big 914G 14G 220G 7% /mypool/big
mypool/chess 914G 200G 220G 48% /mypool/chess
mypool/downloads 914G 11G 220G 5% /mypool/downloads
mypool/drkirkby.bak 914G 18K 220G 1% /mypool/drkirkby.bak
mypool/kirkby 914G 18K 220G 1% /mypool/kirkby
mypool/maths 914G 7.8G 220G 4% /mypool/maths
mypool/my-old-stuff 914G 6.3G 220G 3% /mypool/my-old-stuff
mypool/onxxx 914G 2.8G 220G 2% /mypool/onxxx
mypool/ufsdumps 914G 144G 220G 40% /mypool/ufsdumps
mypool/usr 914G 20K 220G 1% /mypool/usr
mypool/usr/local 914G 83G 220G 28% /mypool/usr/local
rpool 67G 94K 56G 1% /rpool
s6pool 67G 92K 59G 1% /s6pool
s6pool/ROOT 67G 18K 59G 1% /s6pool/ROOT
The Ultra 27 does not have much on it yet. The root pool is a single 500 GB disk
(I'll mirror that later today, as I now have a second disk). The other pool is a
pair of 2 TB Hitachi UltraStars.
drkirkby@hawk:~$ /usr/sbin/zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
bpool 1.81T 16.3G 1.80T 0% ONLINE -
rpool 464G 12.7G 451G 2% ONLINE -
drkirkby@hawk:~$ df -h
Filesystem size used avail capacity Mounted on
rpool/ROOT/opensolaris
457G 6.4G 438G 2% /
/devices 0K 0K 0K 0% /devices
/dev 0K 0K 0K 0% /dev
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 6.3G 312K 6.3G 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
sharefs 0K 0K 0K 0% /etc/dfs/sharetab
/usr/lib/libc/libc_hwcap1.so.1
445G 6.4G 438G 2% /lib/libc.so.1
fd 0K 0K 0K 0% /dev/fd
swap 6.4G 97M 6.3G 2% /tmp
swap 6.3G 48K 6.3G 1% /var/run
bpool 1.8T 19K 1.8T 1% /bpool
rpool/export 457G 21K 438G 1% /export
rpool/export/home 457G 22K 438G 1% /export/home
bpool/users 1.8T 13G 1.8T 1% /export/home/drkirkby
bpool/local 1.8T 2.2G 1.8T 1% /usr/local
bpool/users/jas 1.8T 20K 1.8T 1% /export/home/jas
bpool/zones 1.8T 21K 1.8T 1% /zones
Can anyone suggest the best ways to move this data, with the minimum of hassle?
webjuan
If minimum hassle is what you want, I would just use the USB method of
moving the USB drive from one machine to the other. Alternatively, if
you want to use ZFS, you will have to create snapshots of all your
pools then use use the "send and receive" zfs option to move your
data.
At a high level, you can do something similar to the following:
# zfs snapshot -r yourpoolname@today
# zfs send -R yourpoolname@today | ssh otherserver "zfs recv -vFd
poolname"
Of course, your mileage may vary. Manpage covers all the zfs recv
options.
Hope that points you in the right direction.
Juan
Dave
>> Can anyone suggest the best ways to move this data, with the minimum of hassle?
>
> If minimum hassle is what you want, I would just use the USB method of
> moving the USB drive from one machine to the other. Alternatively, if
> you want to use ZFS, you will have to create snapshots of all your
> pools then use use the "send and receive" zfs option to move your
> data.
The thing is I want to move the data off of the cheap consumer grade USB disks,
where the power lead often falls out, and put the data on the internal disks on
the Sun Ultra 27. Physically unplugging them did intially look attractive, as
the speed should be faster than the 10/100 ethernet in the Blade 2000. But I got
a quad Gigabit card off of eBay, so I do not feel there is anything to gain from
physically moving the disks over.
> At a high level, you can do something similar to the following:
>
> # zfs snapshot -r yourpoolname@today
> # zfs send -R yourpoolname@today | ssh otherserver "zfs recv -vFd
> poolname"
>
> Of course, your mileage may vary. Manpage covers all the zfs recv
> options.
>
> Hope that points you in the right direction.
>
> Juan
Thank you. I'll give the zfs/ssh options a go.
I assume it's possible to do this on a file system basis, rather than the whole
pool? For example, one of the pools has a /usr/local file system on it. Since
that it going to be almost all binaries, there is no point copying them over. So
I'd rather omit that file system. But other file systems on the same pool need
to be copied.
Dave
Richard B. Gilbert
> I've got two machines. One is an old Sun Blade 2000 (Solaris 10, update
> 7, ZFS boot), the other a new Sun Ultra 27 (OpenSolaris 06/2009, ZFS boot).
>
> I'd like to copy a lot of the files from the Blade 2000 to the Ultra 27.
>
> The Blade 2000 has about 1 TB of data in total. Most of that is on
> external USB disks drives, though some is on SCSI disk. There is a USB
> 2.0 card in the Blade 2000 used for the external drives.
>
> The Ultra 27 has only internal SATA drives.
>
> I've stuck a Quad Gigabit ethernet card in the Blade 2000, so the
> network transfer rate should I believe be limited by the speed of the
> SCSI disks, rather than the network. (I assume the SCSI disks work at 20
> or 40 MB/s in the Blade 2000 - nothing like the 320 MB/s disk speed).
>
> What is the best way to move the data?
>
> I suspect using ZFS send/receive is probably best, but are not exactly
> sure the best way to do it.
>
> One option is to physically remove the external USB drives from the
> Blade 2000, and put them on the Ultra 27. I do not however believe that
> is going to gain me anything, as the Gigabit Ethernet should be faster
> than USB 2.0.
PLEASE SET MIND IN MOTION BEFORE TOUCHING KEYBOARD!
Gigabit Ethernet won't necessarily do a thing for you! You are limited
by the speed at which you can read those disks. It's unlikely that the
files are contiguous on disk which means you are going to be doing a lot
of seeks and waiting for the proper sector to come under the heads.
IOW, it's going to be slow going
BTW, I believe that 320 MB/s is the peek speed at which you can transfer
data once you get the heads positioned and the proper sector under the
heads.
AZ Nomad
gigabit ethernet is good for about 85MB/s. 320MB sounds like buffer
speed, not media speed.
Of course, ten times the time needed for the transfer has been pissed
away trying to figure out the fastest method.
1 TB takes about half a day at USB2 speeds to a consumer grade SATA
drive. (1TB / 30MB/s = 10 hours)