Unable to Fetch Kickstart with latest 5.0 release

78 views
Skip to first unread message

Mike Walters

unread,
Dec 13, 2017, 3:35:40 PM12/13/17
to stacki
I am back again :( with the same issue but with a freshly installed 5.0 frontend. The backend node (VM) starts the PXE boot, but fails midway with message: failed to fetch kickstart.


Configuration:

OS: CentOS 7
Frontend: 192.168.55.141
Backend: 192.168.55.142

==> Spreadsheet File: <==
NAME
,INTERFACE HOSTNAME,DEFAULT,APPLIANCE,RACK,RANK,IP,MAC,INTERFACE,NETWORK,CHANNEL,OPTIONS,VLAN,INSTALLACTION,RUNACTION,GROUPS,BOX
backendnode01
,backendnode01,TRUE,backend,1,1,192.168.55.142,00:15:5D:0E:FC:50,eth0,private,,,,default,default,,default

[root@stacki stacki]# stack list network
NETWORK ADDRESS      MASK          GATEWAY      MTU  ZONE DNS   PXE
private 192.168.55.0 255.255.255.0 192.168.55.1 1500 ---- False True


[root@stacki stacki]# stack list host
HOST          RACK RANK APPLIANCE OS     BOX     ENVIRONMENT OSACTION INSTALLACTION STATUS COMMENT
stacki        
0    0    frontend  redhat default ----------- default  default       up
backendnode01
1    1    backend   redhat default ----------- default  default       ------


[root@stacki stacki]# stack list pallet
NAME    VERSION              RELEASE ARCH   OS     BOXES
os      
7.4_20171128         redhat7 x86_64 redhat
stacki  
5.0_20171128_b0ed4e3 redhat7 x86_64 redhat default
Updates 7                    7.x     x86_64 redhat default
CentOS  7                    redhat7 x86_64 redhat default
EPEL    
7                    7.x     x86_64 redhat default


[root@stacki stacki]# stack list host boot
HOST          ACTION  NUKEDISKS NUKECONTROLLER
stacki        
------- False     False
backendnode01 install
True      False

[root@stacki stacki]# stack list storage partition
SCOPE         DEVICE PARTID MOUNTPOINT SIZE  FSTYPE   OPTIONS
global        sda    1      biosboot   1     biosboot --------------------------------------------------------- 1
global        sda    2      /          30000 ext4     --------------------------------------------------------- 2
global        sda    3      /var       30000 ext4     --------------------------------------------------------- 3
global        sda    4      swap       32768 swap     --------------------------------------------------------- 4
global        sda    5      /scratch   0     xfs      --------------------------------------------------------- 5
backendnode01 myvg   5      /          30000 xfs      --name=lv_root                                            5
backendnode01 myvg   6      swap       16384 xfs      --name=lv_swap                                            6
backendnode01 myvg   7      /home      5000  xfs      --name=lv_home --fsoptions=defaults,nodev,nosuid          7
backendnode01 myvg   8      /tmp       5000  xfs      --name=lv_tmp --fsoptions=defaults,nodev,noexec,nosuid    8
backendnode01 myvg   9      /var/log   5000  xfs      --name=lv_varlog --fsoptions=defaults,nodev,noexec,nosuid 9
backendnode01 myvg   10     /var       20000 xfs      --name=lv_var --fsoptions=defaults,nodev --grow           10
backendnode01 pv.01  4      myvg       0     volgroup --------------------------------------------------------- 4
backendnode01 sda    1      biosboot   1     biosboot --------------------------------------------------------- 1
backendnode01 sda    2      /boot      500   xfs      --------------------------------------------------------- 2
backendnode01 sda    3      pv.01      1     lvm      --grow                                                    3




Logs During Failed message:

==> /var/log/httpd/ssl_access_log <==
192.168.55.142 - - [13/Dec/2017:15:07:11 -0500] "GET /install/sbin/profile.cgi?os=redhat&arch=x86_64&np=2 HTTP/1.1" 500 527

==> /var/log/httpd/ssl_request_log <==
[13/Dec/2017:15:07:11 -0500] 192.168.55.142 TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 "GET /install/sbin/profile.cgi?os=redhat&arch=x86_64&np=2 HTTP/1.1" 527

==> /var/log/httpd/ssl_error_log <==
[Wed Dec 13 15:07:19.403981 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:   File "/opt/stack/bin/stack", line 155, in run_command
[Wed Dec 13 15:07:19.404031 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:     rc = command.runWrapper(name, args[i:])
[Wed Dec 13 15:07:19.404059 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:   File "/opt/stack/lib/python3.6/site-packages/stack/commands/__init__.py", line 2254, in runWrapper
[Wed Dec 13 15:07:19.404072 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:     rc = self.run(self._params, self._args)
[Wed Dec 13 15:07:19.404100 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:   File "/opt/stack/lib/python3.6/site-packages/stack/commands/list/host/xml/__init__.py", line 60, in run
[Wed Dec 13 15:07:19.404114 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:     xml = self.command('list.node.xml', args)
[Wed Dec 13 15:07:19.404139 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:   File "/opt/stack/lib/python3.6/site-packages/stack/commands/__init__.py", line 1668, in command
[Wed Dec 13 15:07:19.404155 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:     self.rc = o.runWrapper(name, args, self.level + 1)
[Wed Dec 13 15:07:19.404182 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:   File "/opt/stack/lib/python3.6/site-packages/stack/commands/__init__.py", line 2254, in runWrapper
[Wed Dec 13 15:07:19.404195 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:     rc = self.run(self._params, self._args)
[Wed Dec 13 15:07:19.404223 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:   File "/opt/stack/lib/python3.6/site-packages/stack/commands/list/node/xml/__init__.py", line 312, in run
[Wed Dec 13 15:07:19.404235 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:     handler.parseNode(node, doEval, self)
[Wed Dec 13 15:07:19.404259 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:   File "/opt/stack/lib/python3.6/site-packages/stack/profile.py", line 247, in parseNode
[Wed Dec 13 15:07:19.404267 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:     parser.feed(xml)
[Wed Dec 13 15:07:19.404287 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:   File "/opt/stack/lib/python3.6/xml/sax/expatreader.py", line 214, in feed
[Wed Dec 13 15:07:19.404299 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:     self._err_handler.fatalError(exc)
[Wed Dec 13 15:07:19.404320 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:   File "/opt/stack/lib/python3.6/xml/sax/handler.py", line 38, in fatalError
[Wed Dec 13 15:07:19.404328 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215:     raise exception
[Wed Dec 13 15:07:19.404350 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] AH01215: stack.commands.list.host.xml: SAXParseException -- <unknown>:59:0: unbound prefix
[Wed Dec 13 15:07:19.680468 2017] [cgi:error] [pid 1018] [client 192.168.55.142:1] End of script output before headers: profile.cgi


==> /var/log/messages <==
Dec 13 15:18:12 stacki in.tftpd[2489]: Client 192.168.55.142 finished pxelinux.0
Dec 13 15:18:12 stacki in.tftpd[2490]: RRQ from 192.168.55.142 filename pxelinux.cfg/a1ebda7d-fd63-3248-9990-c72318d01bae
Dec 13 15:18:12 stacki in.tftpd[2490]: Client 192.168.55.142 File not found pxelinux.cfg/a1ebda7d-fd63-3248-9990-c72318d01bae
Dec 13 15:18:12 stacki in.tftpd[2491]: RRQ from 192.168.55.142 filename pxelinux.cfg/01-00-15-5d-0e-fc-50
Dec 13 15:18:12 stacki in.tftpd[2491]: Client 192.168.55.142 File not found pxelinux.cfg/01-00-15-5d-0e-fc-50
Dec 13 15:18:12 stacki in.tftpd[2492]: RRQ from 192.168.55.142 filename pxelinux.cfg/C0A8378E
Dec 13 15:18:12 stacki in.tftpd[2492]: Client 192.168.55.142 finished pxelinux.cfg/C0A8378E
Dec 13 15:18:12 stacki in.tftpd[2493]: RRQ from 192.168.55.142 filename vmlinuz-5.0_20171128_b0ed4e3-redhat7-x86_64
Dec 13 15:18:17 stacki in.tftpd[2493]: Client 192.168.55.142 finished vmlinuz-5.0_20171128_b0ed4e3-redhat7-x86_64
Dec 13 15:18:17 stacki in.tftpd[2494]: RRQ from 192.168.55.142 filename initrd.img-5.0_20171128_b0ed4e3-redhat7-x86_64
Dec 13 15:19:06 stacki in.tftpd[2494]: Client 192.168.55.142 finished initrd.img-5.0_20171128_b0ed4e3-redhat7-x86_64
Dec 13 15:19:18 stacki systemd: Starting Cleanup of Temporary Directories...
Dec 13 15:19:18 stacki systemd: Started Cleanup of Temporary Directories.
Dec 13 15:19:25 stacki dhcpd: DHCPDISCOVER from 00:15:5d:0e:fc:50 via eth0
Dec 13 15:19:25 stacki dhcpd: DHCPOFFER on 192.168.55.142 to 00:15:5d:0e:fc:50 via eth0
Dec 13 15:19:25 stacki dhcpd: DHCPREQUEST for 192.168.55.142 (192.168.55.141) from 00:15:5d:0e:fc:50 via eth0
Dec 13 15:19:25 stacki dhcpd: DHCPACK on 192.168.55.142 to 00:15:5d:0e:fc:50 via eth0
Dec 13 15:19:28 stacki mysqld: 2017-12-13 15:19:28 139636574422784 [Warning] Aborted connection 8 to db: 'cluster' user: 'apache' host: 'localhost' (Got an error reading communication packets)
Dec 13 15:19:28 stacki mysqld: 2017-12-13 15:19:28 139636574422784 [Warning] Aborted connection 9 to db: 'cluster' user: 'apache' host: 'localhost' (Got an error reading communication packets)
Dec 13 15:19:28 stacki mysqld: 2017-12-13 15:19:28 139636574422784 [Warning] Aborted connection 10 to db: 'cluster' user: 'apache' host: 'localhost' (Got an error reading communication packets)
Dec 13 15:19:29 stacki mysqld: 2017-12-13 15:19:29 139636574422784 [Warning] Aborted connection 11 to db: 'cluster' user: 'apache' host: 'localhost' (Got an error reading communication packets)
Dec 13 15:19:29 stacki mysqld: 2017-12-13 15:19:29 139636574119680 [Warning] Aborted connection 13 to db: 'cluster' user: 'apache' host: 'localhost' (Got an error reading communication packets)
Dec 13 15:19:29 stacki mysqld: 2017-12-13 15:19:29 139636574119680 [Warning] Aborted connection 14 to db: 'cluster' user: 'apache' host: 'localhost' (Got an error reading communication packets)
Dec 13 15:19:30 stacki mysqld: 2017-12-13 15:19:30 139636574119680 [Warning] Aborted connection 15 to db: 'cluster' user: 'apache' host: 'localhost' (Got an error reading communication packets)
Dec 13 15:19:30 stacki mysqld: 2017-12-13 15:19:30 139636574119680 [Warning] Aborted connection 16 to db: 'cluster' user: 'apache' host: 'localhost' (Got an error reading communication packets)
Dec 13 15:19:30 stacki mysqld: 2017-12-13 15:19:30 139636574422784 [Warning] Aborted connection 12 to db: 'cluster' user: 'apache' host: 'localhost' (Got an error reading communication packets)


Command I run on frontend before trying another backend install:

stack set host boot backendend01 action=os
stack set host boot backendend01 action=install
stack set host attr backendnode01 attr=nukedisks value=true

Backend Monitor - error before failure loop:

Auto Generated Inline Image 1

Joe Kaiser

unread,
Dec 13, 2017, 3:41:22 PM12/13/17
to sta...@googlegroups.com
Hi,

Please send the output of

# stack report system


Also check:

# stack list host profile backend-0-0

Do you have a cart on this?

Thanks,

Joe


--
You received this message because you are subscribed to the Google Groups "stacki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to stacki+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Joe Kaiser

unread,
Dec 13, 2017, 4:11:51 PM12/13/17
to sta...@googlegroups.com
Hyper-V right? How much memory on the backend and how big is the disk?

What nic emulation is it using?

Thanks,

Joe

Mike Walters

unread,
Dec 13, 2017, 6:09:18 PM12/13/17
to stacki
Hi Joe,

Here are the requested items:

[root@stacki ~]# stack report system
=========================================================================================================================== test session starts ============================================================================================================================
platform linux
-- Python 3.6.1, pytest-3.3.0, py-1.5.2, pluggy-0.6.0 -- /opt/stack/bin/python3
cachedir
: .cache
rootdir
: /opt/stack/lib/python3.6/site-packages/stack/commands/report/system, inifile:
plugins
: testinfra-1.8.0
collected
15 items

tests
/services.py::test_apache_enabled_and_running[local] PASSED                                                                                                                                                                                                     [  6%]
tests
/services.py::test_mariadb_enabled_and_running[local] PASSED                                                                                                                                                                                                    [ 13%]
tests
/services.py::test_tftpd_enabled_and_running[local] PASSED                                                                                                                                                                                                      [ 20%]
tests
/services.py::test_ludicrous_server_enabled_and_running[local] PASSED                                                                                                                                                                                           [ 26%]
tests
/services.py::test_dhcp_enabled_and_running[local] PASSED                                                                                                                                                                                                       [ 33%]
tests
/services.py::test_rmq_processor_enabled_and_running[local] PASSED                                                                                                                                                                                              [ 40%]
tests
/services.py::test_rmq_producer_enabled_and_running[local] PASSED                                                                                                                                                                                               [ 46%]
tests
/services.py::test_rmq_publisher_enabled_and_running[local] PASSED                                                                                                                                                                                              [ 53%]
tests
/df.py::test_slash_not_full[local] PASSED                                                                                                                                                                                                                       [ 60%]
tests
/df.py::test_export_not_full[local] PASSED                                                                                                                                                                                                                      [ 66%]
tests
/df.py::test_var_not_full[local] PASSED                                                                                                                                                                                                                         [ 73%]
tests
/stacki.py::test_stack_list_boxes

NAME    OS     PALLETS                     CARTS
default redhat stacki-5.0_20171128_b0ed4e3-redhat7 Updates-7-7.x CentOS-7-redhat7 EPEL-7-7.x rs-users rs-config
PASSED                                                                                                                                                                                                                        
[ 80%]
tests
/stacki.py::test_stack_list_pallets


NAME   VERSION RELEASE ARCH   OS     BOXES
os
7.4_20171128 redhat7 x86_64 redhat
stacki
5.0_20171128_b0ed4e3 redhat7 x86_64 redhat default

Updates 7 7.x x86_64 redhat default
CentOS 7 redhat7 x86_64 redhat default
EPEL
7 7.x x86_64 redhat default

PASSED                                                                                                                                                                                                                      
[ 86%]
tests
/stacki.py::test_stack_list_networks


NETWORK ADDRESS     MASK          GATEWAY      MTU  ZONE  DNS   PXE
private 192.168.55.0 255.255.255.0 192.168.55.1 1500  False True
PASSED                                                                                                                                                                                                                    
[ 93%]
tests
/stacki.py::test_stack_list_host


HOST RACK RANK APPLIANCE OS BOX ENVIRONMENT OSACTION INSTALLACTION STATUS
stacki 0 0 frontend redhat default None default default up None
backendnode01
1 1 backend redhat default None default default None None
PASSED                                                                                                                                                                                                                        
[100%]

======================================================================================================================== 15 passed in 0.92 seconds ==========


Maybe this is the issue?
[root@stacki ~]# stack list host profile backendnode01

 
File "/opt/stack/bin/stack", line 155, in
run_command
    rc
= command.runWrapper(name, args[i:])

 
File "/opt/stack/lib/python3.6/site-packages/stack/commands/__init__.py", line 2254, in
runWrapper
    rc
= self.run(self._params, self._args)
 
File "/opt/stack/lib/python3.6/site-packages/stack/commands/list/host/profile/__init__.py", line 132, in run
    xmlinput
= self.command('list.host.xml', [ host ])

 
File "/opt/stack/lib/python3.6/site-packages/stack/commands/__init__.py", line 1668, in
command
   
self.rc = o.runWrapper(name, args, self.level + 1)

 
File "/opt/stack/lib/python3.6/site-packages/stack/commands/__init__.py", line 2254, in
runWrapper
    rc
= self.run(self._params, self._args)

 
File "/opt/stack/lib/python3.6/site-packages/stack/commands/list/host/xml/__init__.py", line 60, in
run
    xml
= self.command('list.node.xml', args)

 
File "/opt/stack/lib/python3.6/site-packages/stack/commands/__init__.py", line 1668, in
command
   
self.rc = o.runWrapper(name, args, self.level + 1)

 
File "/opt/stack/lib/python3.6/site-packages/stack/commands/__init__.py", line 2254, in
runWrapper
    rc
= self.run(self._params, self._args)

 
File "/opt/stack/lib/python3.6/site-packages/stack/commands/list/node/xml/__init__.py", line 312, in
run
    handler
.parseNode(node, doEval, self)

 
File "/opt/stack/lib/python3.6/site-packages/stack/profile.py", line 247, in
parseNode
    parser
.feed(xml)

 
File "/opt/stack/lib/python3.6/xml/sax/expatreader.py", line 214, in
feed
   
self._err_handler.fatalError(exc)

 
File "/opt/stack/lib/python3.6/xml/sax/handler.py", line 38, in
fatalError
   
raise exception
stack
.commands.list.host.profile: SAXParseException -- <unknown>:59:0: unbound prefix


I have minimal carts from previous install:
[root@stacki ~]# stack list cart
NAME      BOXES
rs
-users  default
rs
-config default


Anoop Rajendra

unread,
Dec 13, 2017, 6:15:31 PM12/13/17
to sta...@googlegroups.com
Hi Mike,

You have an XML error in one of the carts.

Run "stack list host profile backendnode01" with only one of the carts enabled, to isolate which cart has the XML error in it. Then eyeball the cart node xml files to see which one might have the error.

There's a good chance that there is a "<" sign or an "&" in one of your XML files between tags.

-Anoop

Joe Kaiser

unread,
Dec 13, 2017, 6:16:05 PM12/13/17
to sta...@googlegroups.com
cat the cart*xml in the nodes directory of each of those and I'll tell you how to fix that.

The language has changed and I haven't finished documenting it yet.

Thanks,

Joe


Mike Walters

unread,
Dec 13, 2017, 6:16:55 PM12/13/17
to stacki
Yes Hyper-V

Backend Specs:
4GB Ram
100GB Disk
Auto provisioned stackios

Nic: Hyper-V Legacy Network Adapter


Mike Walters

unread,
Dec 13, 2017, 6:41:29 PM12/13/17
to stacki
Hi Joe,

Attached are the two carts, which I have taken out our keys and usernames.


cart01
cart02

Joe Kaiser

unread,
Dec 13, 2017, 6:51:33 PM12/13/17
to sta...@googlegroups.com
In each of these files there exists the opening xml line and the open and closing kickstart line:

<?xml version="1.0" standalone="no"?>
<kickstart>
.
.
.
.
</kickstart>

Take out the "xml version" line.

and change the kickstart tags to:

<stack:stack>
.
.
.

</stack:stack>

Then try the "stack list host profile nodename" again and see if it completes without error.

We changed the syntax for the xml to make it universal to that xml carts are now OS agnostic (well, for the most part).

The docs for these changes are still being written. So questions are welcomed for certain.

Thanks,

Joe

On Wed, Dec 13, 2017 at 4:41 PM, Mike Walters <wmi...@gmail.com> wrote:
Hi Joe,

Attached are the two carts, which I have taken out our keys and usernames.


Mike Walters

unread,
Dec 15, 2017, 3:57:26 PM12/15/17
to stacki
Thank you Joe, your changes fixed my issue.
To unsubscribe from this group and stop receiving emails from it, send an email to stacki+un...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages