WW + golden image
476 views
Skip to first unread message
Meij, Henk
Sep 7, 2015, 2:19:20 PM9/7/15
to ware...@lbl.gov
Is there some clever way I can grab an existing node's image with dd/cpio and toss it into my chroots location, edit the config files in question and then provision nodes in a statefull manner from there (doing the relevant steps in the database)?
I have some nodes with all the X11 and tons of other graphical libs running and dread thinking about building a chgroot environment from the ground up.
-Henk
Jason Stover
Sep 7, 2015, 2:33:54 PM9/7/15
to ware...@lbl.gov
There should be a golden-image (or similarly named) template for
wwmkchroot. It basically just does an rsync of a node file system.
-J
> --
> You received this message because you are subscribed to the Google Groups
> "Warewulf" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to warewulf+u...@lbl.gov.
> To post to this group, send email to ware...@lbl.gov.
> To view this discussion on the web visit
> https://groups.google.com/a/lbl.gov/d/msgid/warewulf/8688BD11DAC0574AA90295127E9E9F4AC220311C%40exchangewes8.wesad.wesleyan.edu.
> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
wwmkchroot. It basically just does an rsync of a node file system.
-J
> You received this message because you are subscribed to the Google Groups
> "Warewulf" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to warewulf+u...@lbl.gov.
> To post to this group, send email to ware...@lbl.gov.
> To view this discussion on the web visit
> https://groups.google.com/a/lbl.gov/d/msgid/warewulf/8688BD11DAC0574AA90295127E9E9F4AC220311C%40exchangewes8.wesad.wesleyan.edu.
> For more options, visit https://groups.google.com/a/lbl.gov/d/optout.
Meij, Henk
Sep 7, 2015, 3:14:15 PM9/7/15
to ware...@lbl.gov
oh, tell me more, where do I find info on this? how does an rsync operation preserve boot partition and all that...
-Henk
________________________________________
From: Jason Stover [jason....@gmail.com]
Sent: Monday, September 07, 2015 2:33 PM
To: ware...@lbl.gov
Subject: Re: [Warewulf] WW + golden image
To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAGfAqt81hRjhJwK7HWf%2Bn6709aU4yjmnrbNwHc6%2BNvJ1ZvAjvQ%40mail.gmail.com.
-Henk
________________________________________
From: Jason Stover [jason....@gmail.com]
Sent: Monday, September 07, 2015 2:33 PM
To: ware...@lbl.gov
Subject: Re: [Warewulf] WW + golden image
Jason Stover
Sep 7, 2015, 3:52:52 PM9/7/15
to ware...@lbl.gov
The file you'll want to look at is:
/usr/libexec/warewulf/wwmkchroot/golden-system.tmpl
You'll run it something like (or export it):
SOURCEADDR=node0001 wwmkchroot golden-system /var/chroots/node0001.chroot
Where 'SOURCEADDR' is the node that is your "golden-system".
In the tmpl file, you'll find the buildchroot() function which has the
rsync command that's executed. Beyond that, the rsync command is not
limited to a single file system, but it is doing some excludes. So
anything that can be seen on the node will be included. This includes
NFS mounts, so be careful there. ;)
Beyond that ... there isn't a lot to what it does. It's just a
scripted way to do an rsync. I would probably exclude /dev as well,
but it shouldn't hurt anything if that gets copied over as well.
-J
/usr/libexec/warewulf/wwmkchroot/golden-system.tmpl
You'll run it something like (or export it):
SOURCEADDR=node0001 wwmkchroot golden-system /var/chroots/node0001.chroot
Where 'SOURCEADDR' is the node that is your "golden-system".
In the tmpl file, you'll find the buildchroot() function which has the
rsync command that's executed. Beyond that, the rsync command is not
limited to a single file system, but it is doing some excludes. So
anything that can be seen on the node will be included. This includes
NFS mounts, so be careful there. ;)
Beyond that ... there isn't a lot to what it does. It's just a
scripted way to do an rsync. I would probably exclude /dev as well,
but it shouldn't hurt anything if that gets copied over as well.
-J
Meij, Henk
Dec 8, 2015, 3:19:16 PM12/8/15
to ware...@lbl.gov
Finally getting back to this project ...
1. When deploying a golden image I'm assuming I'm not building a VNFS of my golden image chroot but then proceed with the statefull instructions, then when done set bootlocal=1
2. If the disk golden image has LVM on it and fdisk -l shows what is listed below what do the filesystem settings look like.
Do I introduce sda3 as swap or is there a way to handle LVM?
-Henk
Disk /dev/sda: 80.0 GB, 80026361856 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 64 512000 83 Linux
/dev/sda2 64 9730 77637632 8e Linux LVM
Disk /dev/mapper/vg_b0-lv_root: 77.4 GB, 77351354368 bytes
Disk /dev/mapper/vg_b0-lv_swap: 2147 MB, 2147483648 bytes
Sent: Monday, September 07, 2015 3:52 PM
1. When deploying a golden image I'm assuming I'm not building a VNFS of my golden image chroot but then proceed with the statefull instructions, then when done set bootlocal=1
2. If the disk golden image has LVM on it and fdisk -l shows what is listed below what do the filesystem settings look like.
Do I introduce sda3 as swap or is there a way to handle LVM?
-Henk
Disk /dev/sda: 80.0 GB, 80026361856 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 64 512000 83 Linux
/dev/sda2 64 9730 77637632 8e Linux LVM
Disk /dev/mapper/vg_b0-lv_root: 77.4 GB, 77351354368 bytes
Disk /dev/mapper/vg_b0-lv_swap: 2147 MB, 2147483648 bytes
Sent: Monday, September 07, 2015 3:52 PM
--
You received this message because you are subscribed to the Google Groups "Warewulf" group.
To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov.
To post to this group, send email to ware...@lbl.gov.
To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAGfAqt_OzOVSB1kt%2Btn%3DC0Y5y0BP_Nsi7n77edq4ZHaHXmGEDA%40mail.gmail.com.
You received this message because you are subscribed to the Google Groups "Warewulf" group.
To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov.
To post to this group, send email to ware...@lbl.gov.
Henk
Dec 9, 2015, 3:43:09 PM12/9/15
to Warewulf
I'm missing something...I have test "b" nodes, so I made b3 stateless against centos-6 chroot, then added b6 so I could work out some script deployment issues
Then I deleted object b6 and build a b0.chroot golden image from that node's OS. Then I configured boot[loader|format|partition] and filesystem for b6.
But when I PXE boot b6 it still is stateless and the hdd is not freshly rebuilt, should I delete the b6 provision vnfs and bootstrap?
-Henk
[root@petaltail b0.chroot]# wwsh object print b6 -p :all
#### node b6.cluster ##########################################################
30: NAME = b6,b6.cluster
30: BOOTLOADER = sda
30: BOOTSTRAPID = 6
30: DISKFORMAT = sda1,sda2,sda3
30: DISKPARTITION = sda
30: DOMAIN = cluster
30: FILEIDS = 16,19,2,20,22,23,28,4,5
30: FILESYSTEM = mountpoint=/boot:dev=sda1:type=ext4:size=500,dev=sda2:type=swap:size=2147,mountpoint=/:dev=sda3:type=ext4:size=fill
30: NETDEVS = ObjectSet
NETDEVS.eth0.NAME = eth0
NETDEVS.eth0.HWADDR = 00:13:d3:56:9a:a8
NETDEVS.eth0.IPADDR = 192.168.1.12
NETDEVS.eth0.NETMASK = 255.255.0.0
NETDEVS.eth0.NETWORK = 255.255.0.0
NETDEVS.eth1.NAME = eth1
NETDEVS.eth1.HWADDR = 00:13:d3:56:9a:a9
NETDEVS.eth1.IPADDR = 10.10.100.12
NETDEVS.eth1.NETMASK = 255.255.0.0
NETDEVS.eth1.NETWORK = 255.255.0.0
30: NODENAME = b6
30: VNFSID = 14
[root@petaltail b0.chroot]# wwsh provision print b6
#### b6.cluster ###############################################################
b6.cluster: BOOTSTRAP = 2.6.32-504.8.1.el6.x86_64
b6.cluster: VNFS = centos-6
b6.cluster: FILES = bashrc,dynamic_hosts,group,hosts,ifcfg-eth1.ww,network.ww,passwd,profile,shadow
b6.cluster: PRESHELL = FALSE
b6.cluster: POSTSHELL = FALSE
b6.cluster: IFRENAME = FALSE
b6.cluster: CONSOLE = UNDEF
b6.cluster: PXELINUX = UNDEF
b6.cluster: SELINUX = DISABLED
b6.cluster: KARGS = "quiet"
b6.cluster: DISKFORMAT = sda1,sda2,sda3
b6.cluster: DISKPARTITION = sda
b6.cluster: BOOTLOADER = sda
b6.cluster: BOOTLOCAL = FALSE
#### node b6.cluster ##########################################################
30: NAME = b6,b6.cluster
30: BOOTLOADER = sda
30: BOOTSTRAPID = 6
30: DISKFORMAT = sda1,sda2,sda3
30: DISKPARTITION = sda
30: DOMAIN = cluster
30: FILEIDS = 16,19,2,20,22,23,28,4,5
30: FILESYSTEM = mountpoint=/boot:dev=sda1:type=ext4:size=500,dev=sda2:type=swap:size=2147,mountpoint=/:dev=sda3:type=ext4:size=fill
30: NETDEVS = ObjectSet
NETDEVS.eth0.NAME = eth0
NETDEVS.eth0.HWADDR = 00:13:d3:56:9a:a8
NETDEVS.eth0.IPADDR = 192.168.1.12
NETDEVS.eth0.NETMASK = 255.255.0.0
NETDEVS.eth0.NETWORK = 255.255.0.0
NETDEVS.eth1.NAME = eth1
NETDEVS.eth1.HWADDR = 00:13:d3:56:9a:a9
NETDEVS.eth1.IPADDR = 10.10.100.12
NETDEVS.eth1.NETMASK = 255.255.0.0
NETDEVS.eth1.NETWORK = 255.255.0.0
30: NODENAME = b6
30: VNFSID = 14
[root@petaltail b0.chroot]# wwsh provision print b6
#### b6.cluster ###############################################################
b6.cluster: BOOTSTRAP = 2.6.32-504.8.1.el6.x86_64
b6.cluster: VNFS = centos-6
b6.cluster: FILES = bashrc,dynamic_hosts,group,hosts,ifcfg-eth1.ww,network.ww,passwd,profile,shadow
b6.cluster: PRESHELL = FALSE
b6.cluster: POSTSHELL = FALSE
b6.cluster: IFRENAME = FALSE
b6.cluster: CONSOLE = UNDEF
b6.cluster: PXELINUX = UNDEF
b6.cluster: SELINUX = DISABLED
b6.cluster: KARGS = "quiet"
b6.cluster: DISKFORMAT = sda1,sda2,sda3
b6.cluster: DISKPARTITION = sda
b6.cluster: BOOTLOADER = sda
b6.cluster: BOOTLOCAL = FALSE
Jason Stover
Dec 9, 2015, 4:34:57 PM12/9/15
to ware...@lbl.gov
Hi Henk,
" FILESYSTEM =
mountpoint=/boot:dev=sda1:type=ext4:size=500,dev=sda2:type=swap:size=2147,mountpoint=/:dev=sda3:type=ext4:size=fill"
The variable is FILESYSTEMS ... not FILESYSTEM
-J
" FILESYSTEM =
mountpoint=/boot:dev=sda1:type=ext4:size=500,dev=sda2:type=swap:size=2147,mountpoint=/:dev=sda3:type=ext4:size=fill"
-J
> --
> You received this message because you are subscribed to the Google Groups
> "Warewulf" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to warewulf+u...@lbl.gov.
> To post to this group, send email to ware...@lbl.gov.
> To view this discussion on the web visit
> https://groups.google.com/a/lbl.gov/d/msgid/warewulf/32b710f6-f34b-42c5-95b1-050747b0ce2a%40lbl.gov.
> You received this message because you are subscribed to the Google Groups
> "Warewulf" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to warewulf+u...@lbl.gov.
> To post to this group, send email to ware...@lbl.gov.
> To view this discussion on the web visit
Meij, Henk
Dec 10, 2015, 10:43:53 AM12/10/15
to ware...@lbl.gov
Ayi! (me bangs head on desk). But that did not fix it. It's the getvnfs during PXE boot I think is the problem.
Do I need to change b6's VNFS setting or make b0.root VNFS?
Here is my history
591 SOURCEADDR=b0 wwmkchroot golden-system /var/chroots/goldimages/b0.chroot
593 du -hs b0.chroot/
595 wwsh object modify -s bootloader=sda b6
598 wwsh object modify -s diskformat=sda1,sda3 b6
599 wwsh object modify -s filesystems="mountpoint=/boot:dev=sda1:type=ext4:size=500,dev=sda2:type=swap:size=2147,mountpoint=/:dev=sda3:type=ext4:size=fill" b6
602 service dhcpd start
PXE boot b6 still ends up at centos-6(.6) stateless while b0(.chroot) should be centos-6(.5) with and a reformatted disk.
-Henk
[root@petaltail goldimages]# wwsh object print b6 -p :all
#### node b6.cluster ##########################################################
32: NAME = b6,b6.cluster
32: BOOTLOADER = sda
32: BOOTSTRAPID = 6
32: DISKFORMAT = sda1,sda3
32: DOMAIN = cluster
32: FILEIDS = 16,19,2,20,22,23,28,4,5
32: FILESYSTEMS = dev=sda2:type=swap:size=2147,mountpoint=/:dev=sda3:type=ext4:size=fill,mountpoint=/boot:dev=sda1:type=ext4:size=500
32: GROUPS = wwnodes
32: NETDEVS = ObjectSet
32: VNFSID = 14
[root@petaltail goldimages]# wwsh provision list b6
NODE VNFS BOOTSTRAP FILES
================================================================================
b6.cluster centos-6 2.6.32-504.8.1.el6... bashrc,dynamic_hos...
Sent: Wednesday, December 09, 2015 4:34 PM
To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAGfAqt_HBbinGJYK3mGy%2BEazQdXgNHRUcF6hEx3eqg8qnLMcgg%40mail.gmail.com.
Do I need to change b6's VNFS setting or make b0.root VNFS?
Here is my history
591 SOURCEADDR=b0 wwmkchroot golden-system /var/chroots/goldimages/b0.chroot
593 du -hs b0.chroot/
595 wwsh object modify -s bootloader=sda b6
598 wwsh object modify -s diskformat=sda1,sda3 b6
599 wwsh object modify -s filesystems="mountpoint=/boot:dev=sda1:type=ext4:size=500,dev=sda2:type=swap:size=2147,mountpoint=/:dev=sda3:type=ext4:size=fill" b6
602 service dhcpd start
PXE boot b6 still ends up at centos-6(.6) stateless while b0(.chroot) should be centos-6(.5) with and a reformatted disk.
-Henk
[root@petaltail goldimages]# wwsh object print b6 -p :all
#### node b6.cluster ##########################################################
32: NAME = b6,b6.cluster
32: BOOTLOADER = sda
32: BOOTSTRAPID = 6
32: DISKFORMAT = sda1,sda3
32: DOMAIN = cluster
32: FILEIDS = 16,19,2,20,22,23,28,4,5
32: FILESYSTEMS = dev=sda2:type=swap:size=2147,mountpoint=/:dev=sda3:type=ext4:size=fill,mountpoint=/boot:dev=sda1:type=ext4:size=500
32: GROUPS = wwnodes
32: NETDEVS = ObjectSet
NETDEVS.eth0.NAME = eth0
NETDEVS.eth0.HWADDR = 00:13:d3:56:9a:a8
NETDEVS.eth0.IPADDR = 192.168.1.12
NETDEVS.eth0.NETMASK = 255.255.0.0
NETDEVS.eth0.NETWORK = 255.255.0.0
NETDEVS.eth1.NAME = eth1
NETDEVS.eth1.HWADDR = 00:13:d3:56:9a:a9
NETDEVS.eth1.IPADDR = 10.10.100.12
NETDEVS.eth1.NETMASK = 255.255.0.0
NETDEVS.eth1.NETWORK = 255.255.0.0
32: NODENAME = b6
NETDEVS.eth0.HWADDR = 00:13:d3:56:9a:a8
NETDEVS.eth0.IPADDR = 192.168.1.12
NETDEVS.eth0.NETMASK = 255.255.0.0
NETDEVS.eth0.NETWORK = 255.255.0.0
NETDEVS.eth1.NAME = eth1
NETDEVS.eth1.HWADDR = 00:13:d3:56:9a:a9
NETDEVS.eth1.IPADDR = 10.10.100.12
NETDEVS.eth1.NETMASK = 255.255.0.0
NETDEVS.eth1.NETWORK = 255.255.0.0
32: VNFSID = 14
[root@petaltail goldimages]# wwsh provision list b6
NODE VNFS BOOTSTRAP FILES
================================================================================
b6.cluster centos-6 2.6.32-504.8.1.el6... bashrc,dynamic_hos...
Sent: Wednesday, December 09, 2015 4:34 PM
Jason Stover
Dec 10, 2015, 11:16:53 AM12/10/15
to ware...@lbl.gov
I don't see a:
wwvnfs --chroot=/var/chroots/b0.chroot --name=centos-6 (or whatever)
That command may be 592/594 though... Also, can you look through (I
think): /var/log/warewulf/provision on the node when it comes up?
Hopefully that copy code is in the version you're running anyway.
There may be an error in 20-filesystems that wasn't fatal ... but
caused something to get messed up.
You may also want to run:
wwsh pxe update
wwsh dhcp update
When you use the 'object' command, it doesn't run the events as it
would if you made a change with 'wwsh provision' or 'wwsh node'. It's
a direct modification with no niceties attached.
-J
wwvnfs --chroot=/var/chroots/b0.chroot --name=centos-6 (or whatever)
That command may be 592/594 though... Also, can you look through (I
think): /var/log/warewulf/provision on the node when it comes up?
Hopefully that copy code is in the version you're running anyway.
There may be an error in 20-filesystems that wasn't fatal ... but
caused something to get messed up.
You may also want to run:
wwsh pxe update
wwsh dhcp update
When you use the 'object' command, it doesn't run the events as it
would if you made a change with 'wwsh provision' or 'wwsh node'. It's
a direct modification with no niceties attached.
-J
Meij, Henk
Dec 10, 2015, 3:55:42 PM12/10/15
to ware...@lbl.gov
Got it working! Thanks Jason and all. I will put a doc together on how I got there and close this thread.
One oddity I'm working on is
mountpoint=/:dev=sda3:type=ext4:size=70000" b6
yields sda1 at 100M and sda3 at 2GB. fdisk -l reveals sda1 thru sda7 are created, odd. I can live with this as I just pick the right partition for file systems I want. size=fill does not work for me. (WW v3.6.99)
-Henk
Sent: Thursday, December 10, 2015 11:16 AM
One oddity I'm working on is
wwsh object modify -s filesystems="mountpoint=/boot:dev=sda1:type=ext4:size=500,
dev=sda2:type=swap:size=2048,
mountpoint=/:dev=sda3:type=ext4:size=70000" b6
yields sda1 at 100M and sda3 at 2GB. fdisk -l reveals sda1 thru sda7 are created, odd. I can live with this as I just pick the right partition for file systems I want. size=fill does not work for me. (WW v3.6.99)
-Henk
Sent: Thursday, December 10, 2015 11:16 AM
--
You received this message because you are subscribed to the Google Groups "Warewulf" group.
To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov.
To post to this group, send email to ware...@lbl.gov.
To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAGfAqt8upMqFvVwUQqVyDTXsRkxRjKzswYFw2WCi3uDMxp5HvA%40mail.gmail.com.
You received this message because you are subscribed to the Google Groups "Warewulf" group.
To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov.
To post to this group, send email to ware...@lbl.gov.
Jason Stover
Dec 10, 2015, 4:08:57 PM12/10/15
to ware...@lbl.gov
Hrmm... something seems really off on it as WW should be wiping the
partition table. :/ Glad you got it partially working. You can also
try: 'size=+' ... that *should* do the same thing as size=fill.
-J
partition table. :/ Glad you got it partially working. You can also
try: 'size=+' ... that *should* do the same thing as size=fill.
-J
Meij, Henk
Dec 11, 2015, 1:43:03 PM12/11/15
to ware...@lbl.gov
Giving back
https://dokuwiki.wesleyan.edu/doku.php?id=cluster:144&#warewulf_golden_image
Hope it helps somebody. Thanks for all the help list members, especially Jason.
Sent: Thursday, December 10, 2015 4:08 PM
https://dokuwiki.wesleyan.edu/doku.php?id=cluster:144&#warewulf_golden_image
Hope it helps somebody. Thanks for all the help list members, especially Jason.
Sent: Thursday, December 10, 2015 4:08 PM
--
You received this message because you are subscribed to the Google Groups "Warewulf" group.
To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov.
To post to this group, send email to ware...@lbl.gov.
To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAGfAqt9JMzFwoicTwFr7%2B1H%3DsUfRu0g9w%3DsJQv-Qb4bcgMkD8g%40mail.gmail.com.
You received this message because you are subscribed to the Google Groups "Warewulf" group.
To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov.
To post to this group, send email to ware...@lbl.gov.
Novosielski, Ryan
Dec 11, 2015, 2:02:51 PM12/11/15
to ware...@lbl.gov
I would guess a filesystem.log should say what the trouble is, from the /var/log/warewulf directory on the node.
____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
|| \\UTGERS |---------------------*O*---------------------
||_// Biomedical | Ryan Novosielski - Senior Technologist
|| \\ and Health | novo...@rutgers.edu - 973/972.0922 (2x0922)
|| \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
`'
> --
> You received this message because you are subscribed to the Google Groups "Warewulf" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov.
> To post to this group, send email to ware...@lbl.gov.
> To view this discussion on the web visit https://groups.google.com/a/lbl.gov/d/msgid/warewulf/CAGfAqt9JMzFwoicTwFr7%2B1H%3DsUfRu0g9w%3DsJQv-Qb4bcgMkD8g%40mail.gmail.com.
> You received this message because you are subscribed to the Google Groups "Warewulf" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to warewulf+u...@lbl.gov.
> To post to this group, send email to ware...@lbl.gov.
____ *Note: UMDNJ is now Rutgers-Biomedical and Health Sciences*
|| \\UTGERS |---------------------*O*---------------------
||_// Biomedical | Ryan Novosielski - Senior Technologist
|| \\ and Health | novo...@rutgers.edu - 973/972.0922 (2x0922)
|| \\ Sciences | OIRT/High Perf & Res Comp - MSB C630, Newark
`'
Meij, Henk
Dec 11, 2015, 2:32:40 PM12/11/15
to ware...@lbl.gov
attached if you want to have a look, seems entirely normal, picking up right sizes/partitions etc but that is not how the disk ends up
-Henk
________________________________________
From: Novosielski, Ryan [novo...@ca.rutgers.edu]
Sent: Friday, December 11, 2015 2:02 PM
Reply all
Reply to author
Forward
0 new messages