Migrate zfs iocage jail to new machine

[billw...@gmail.com]

I apologize in advance, as I am a big fan of RTFM.  I did not see the answer, and my question is as likely a ZFS question as an iocage one, but I am a small business owner in accounting and tech is my hobby / passion.

I would like to take an iocage ZFS jail and move it to a new server.  Snapshots seem to rely on prior instances, and exports seem to take the source offline.  Is it easy to, and if so can you, explain how to take a jail on one machine and move it to another (same environment ground-up)?  I suspect it might be as easy (given the exact same environment) as TGZ'ing the /iocage/jails/<UUID> and move it, but I can't find confirmation.

Again, thanks for your help.  I am in a bit of a pinch as I am moving a self-hosted environment to a data center as I have to move the servers out of my office this weekend and don't have much of a margin for error!

Thanks!
Bill Warren

[Kai Gallasch]

On 28.01.2016 01:21 billw...@gmail.com wrote:
> I would like to take an iocage ZFS jail and move it to a new server.

Example: Copy an iocage jail from server A to server B.

1) stop the jail you want to copy over and snapshot it.

zfs snapshot -r rpool/iocage/jails/943d33f3-ebf1-11e4-8a0b-00144fcaa900@now

2) host B: start netcat on the remote host B, listening on port 1337 for
30 seconds.

nc -w 30 -l 1337 | zfs receive
rpool/iocage/jails/943d33f3-ebf1-11e4-8a0b-00144fcaa900


3) host A: zfs send the snapshot to host B.

zfs send -R rpool/iocage/jails/943d33f3-ebf1-11e4-8a0b-00144fcaa900@now
| nc my.destination.host 1337

4) host B: Set hostid of the moved jail to the hostid of host B.
(cat /etc/hostid)

"iocage set hostid $HOSTID_OF_HOST_B"


If you transfer iocage jails over unsafe networks, you should use SSH
and not netcat.

Regards,
Kai.

--
PGP-KeyID = 0x70654D7C4FB1F588

[billw...@gmail.com]

Kai - Thank you so very much!

[ch...@cwharton.com]

When moving over the jail none of the setting for the jail move over with it.

[Vick Khera]

> On Feb 18, 2016, at 13:41, ch...@cwharton.com wrote:
>
> When moving over the jail none of the setting for the jail move over with it.

What commands did you use to copy it?

[ch...@cwharton.com]

I apologize. I'm still new to zfs and it was a error on my end.

[le...@lebel.org]

When I follow what was posted, I get when I send the zfs the following:

root@moya:~ # zfs send -Rv zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e@now | ssh root@sulaco 'zfs receive zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e'
send from @ to zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e@now estimated size is 30K
send from @ to zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e/root@now estimated size is 646M
send from @ to zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e/root/data@now estimated size is 40.5K
total estimated size is 646M
TIME        SENT   SNAPSHOT
TIME        SENT   SNAPSHOT
cannot receive: local origin for clone zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e@now does not exist
warning: cannot send 'zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e/root@now': Broken pipe

Either I use ssh or nc, I get the same behavior?

[ch...@cwharton.com]

This is the same problem I was running into. I believe your issue is that your jail is a lightweight jail (cloned). If you create a full jail you should be able to send it with no problems. I'm sure there is a way to send clones in zfs but I have not had the time to research it yet.

[David Lebel]

They are indeed lightweight jail. 

--

// david lebel <le...@lebel.org>
// http://www.lebel.org/
// https://keybase.io/lebel

[David Lebel]

Sorta newbie here then, the source server is on its last mile, and I'd love to move the 6 jails I have it on another host.  The jails are all lightweight. 

[Vick Khera]

On Thu, Mar 10, 2016 at 9:49 AM, David Lebel <le...@lebel.org> wrote:

Sorta newbie here then, the source server is on its last mile, and I'd love to move the 6 jails I have it on another host.  The jails are all lightweight. 

Then zfs send the *whole* iocage data set at once, using the usual incremental update after the initial copy to minimize downtime.

[David Lebel]

How? My zfs knowledge is still limited.

--
You received this message because you are subscribed to a topic in the Google Groups "iocage" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/iocage/nTcpXp7LiW0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to iocage+un...@googlegroups.com.
To post to this group, send email to ioc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/iocage/CALd%2BdcfU_OZ0j-Z_XrYn-%2BNFoaghoELCpPR8WyCnyjLe0QZTTA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Vick Khera]

On Thu, Mar 10, 2016 at 11:03 AM, David Lebel <le...@lebel.org> wrote:

How? My zfs knowledge is still limited.

basically

zfs snapshot -r zfspool/iocage@now1

then zfs send/receive over your ssh or nc however you wish. presumably this will take a while.

then stop all your jails

zfs snapshot -r zfspool/iocage@now2

then send the delta snapshot from now1 to now2 to the new machine. presumably this will take just a few seconds

start your jails on the new host.  destroy your temporary snapshots on the new host, and destroy your iocage data set on the old host.

[David Lebel]

I already have a few jails on my destination. Won't this overwrite? 

--
You received this message because you are subscribed to a topic in the Google Groups "iocage" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/iocage/nTcpXp7LiW0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to iocage+un...@googlegroups.com.
To post to this group, send email to ioc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/iocage/CALd%2BdceKtrUzmAAwSqqS7z4ew3FF1mTjtnriy5hmusyC7pY6Hg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[le...@lebel.org]

OK, I sent a snapshot of the jail uuid, root and root/data:

zfs send -v zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e@now | ssh root@sulaco 'zfs receive zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e/root/data'
zfs send -v zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e/root@now | ssh root@sulaco 'zfs receive zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e/root/data'
zfs send -v zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e/root/data@now | ssh root@sulaco 'zfs receive zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e/root/data'

and after that on the destination host I have:

JID   UUID                                  BOOT  STATE  TAG
-     -  -     down   -
2     ab9c4fbb-c4a0-11e5-b4f1-00248c8c97fe  on    up     plex

I'm confused, I guess iocage requires more than just the three zfs per jails to exists.

[mark....@gmail.com]

Did you find a solution to migrate a jail to another host?

[Predrag Punosevac]

Sir this is exactly what I was looking for!!! Works perfectly. I apologize to everyone for my previous post in particular Mr. Nick Taylor who offered the original help. I made the post out of frustration after trying over 8h to get this to work the way I imagine in my head.

Thank you!

Predrag

On Thursday, January 28, 2016 at 2:14:16 AM UTC-5, Kai Gallasch wrote:

[Vick Khera]

On Thursday, March 10, 2016 at 9:26:09 AM UTC-5, le...@lebel.org wrote:

When I follow what was posted, I get when I send the zfs the following:

root@moya:~ # zfs send -Rv zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e@now | ssh root@sulaco 'zfs receive zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e'
send from @ to zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e@now estimated size is 30K
send from @ to zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e/root@now estimated size is 646M
send from @ to zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e/root/data@now estimated size is 40.5K
total estimated size is 646M
TIME        SENT   SNAPSHOT
TIME        SENT   SNAPSHOT
cannot receive: local origin for clone zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e@now does not exist
warning: cannot send 'zroot/iocage/jails/1659c263-c3a1-11e5-9b11-bc5ff488480e/root@now': Broken pipe

I finally actually needed to do this specific migration. It turned out to be not so hard after all to migrate a base jail. The trick is to skip zfs sending the cloned data sets. (The brilliant insights that come to you while sitting in the dentist's chair....)

Example:

# zfs get -r origin /iocage/jails/2988c469-6127-11e5-921c-5fc322bdfad1 | awk '$3 == "-" {print $1}'

zfspool/iocage/jails/2988c469-6127-11e5-921c-5fc322bdfad1

zfspool/iocage/jails/2988c469-6127-11e5-921c-5fc322bdfad1/root

zfspool/iocage/jails/2988c469-6127-11e5-921c-5fc322bdfad1/root/data

zfspool/iocage/jails/2988c469-6127-11e5-921c-5fc322bdfad1/root/usr

Any data set that has a VALUE for the origin property, we do not want to copy, since it is a clone. So now we just individually send each of the above 4 data sets, using "-p" to preserve the properties which is how iocage knows all the settings.

# iocage snapshot sandbox@copy1

# zfs send -p zfspool/iocage/jails/2988c469-6127-11e5-921c-5fc322bdfad1@copy1 | ssh -c none $REMOTE 'zfs receive zfspool/iocage/jails/2988c469-6127-11e5-921c-5fc322bdfad1'

# zfs send -p zfspool/iocage/jails/2988c469-6127-11e5-921c-5fc322bdfad1/root@copy1 | ssh -c none $REMOTE 'zfs receive zfspool/iocage/jails/2988c469-6127-11e5-921c-5fc322bdfad1/root'

# zfs send -p zfspool/iocage/jails/2988c469-6127-11e5-921c-5fc322bdfad1/root/data@copy1 | ssh -c none $REMOTE 'zfs receive zfspool/iocage/jails/2988c469-6127-11e5-921c-5fc322bdfad1/root/data'

# zfs send -p zfspool/iocage/jails/2988c469-6127-11e5-921c-5fc322bdfad1/root/usr@copy1 | ssh -c none $REMOTE 'zfs receive zfspool/iocage/jails/2988c469-6127-11e5-921c-5fc322bdfad1/root/usr'

The receive data set obviously needs to use the proper pool name where you have already configured iocage. If you are crossing an insecure network, obviously use a real cipher instead of "-c none" to ssh (just remove that option entirely to get the default).

Now on the $REMOTE host reset the hostid and set the new IP (if necessary), and get rid of the snapshot.

# iocage set hostid=`cat /etc/hostid` sandbox

# iocage set ip4_addr='em0|192.168.96.6' sandbox

# iocage snapremove sandbox@copy1

# iocage start sandbox

could not find any snapshots to destroy; check snapshot names.

* 2988c469-6127-11e5-921c-5fc322bdfad1 is a basejail, re-cloning jail..

* Starting 2988c469-6127-11e5-921c-5fc322bdfad1 (sandbox)

  + Started (shared IP mode) OK

  + Starting services        OK

The first time you start this copied base jail, iocage will complain it could not delete the old snapshot that it expects to exist in /iocage/base from where the basejail was originally cloned. It doesn't matter, because it will then proceed to make clones of the other data sets, completing your file system for your newly copied jail and just continue to work.

Easy peasy. :)