[9fans] Plan 9 on Xen 3.2.1 / 32-bit PAE Kernel / 64-bit Hypervisor
Kenneth L Van Alstyne
64-bit hypervisor? It's my understanding that the PAE kernel should work,
but upon creation of the DomU, the machine quickly panics. Example output
is below:
wired:~# xm create -c /home/kvanals/plan9/boot.conf
Using config file "/home/kvanals/plan9/boot.conf".
Started domain plan9.kvanals.org
Plan 9
127 holes free
0018b000 00dba000 12775424
12775424 bytes free
cpu0: 2792MHz AuthenticAMD AMD64 (cpuid: AX 0x20F51 DX 0x78BFBFF)
panic: xenupdatema - pte 80185010 value 269a5027
panic: xenupdatema - pte 80185010 value 269a5027
dumpstack
ktrace /kernel/path 8001444a 80002e2c <<EOF
estackx 80003000
80002dcc=8001428b 80002dd8=80040514 80002ddc=8006b902 80002de4=8006b963
80002df0=80072b4a 80002e00=8004065b 80002e10=8006cef8 80002e14=8001444a
80002e28=8001444a 80002e2c=8001428f 80002e34=80040a81 80002e48=8006a91a
80002e84=8007380a 80002e90=80074682 80002ea4=80073a82 80002ed0=8006cf37
80002ed4=8006a91a 80002edc=80072b4a 80002ef0=80070000 80002ef4=80070044
80002f00=8006cef8 80002f0c=8006cef8 80002f10=80070276 80002f18=8007055a
80002f24=8006a91a 80002f38=80070661 80002f44=8006c3e7 80002f60=8006ca30
80002f8c=80011cc0 80002fac=80011be5 80002fb8=80012208 80002fc4=80012254
80002fe4=00000000 80002fe8=801c6ff8 80002fec=80010a37 80002ff0=800b209f
80002ff4=00000000 80002ff8=8001001f 80002ffc=00000000
EOF
cpu0: exiting
cpu0: spurious interrupt 102, last 0
cpu0: spurious interrupt 102, last 0
cpu0: spurious interrupt 102, last 0
cpu0: spurious interrupt 102, last 0
cpu0: spurious interrupt 102, last 0
cpu0: spurious interrupt 102, last 0
cpu0: spurious interrupt 102, last 0
...
Thanks ahead of time for any assistance anyone is able to provide.
Thanks,
Kenny
Wolfgang Kunz
The Plan 9 Xen PAE kernels runs fine on my NetBSD 64 bit Xen
server. My
Xen version is 3.3.x but 3.2.1 should works too. I could post my
Xen config file here if you like?
-Wolfgang
Richard Miller
> 64-bit hypervisor? It's my understanding that the PAE kernel should work,
> but upon creation of the DomU, the machine quickly panics.
I've updated /n/sources/xen/xen3 source and sample kernels with a small
mmu correction which was needed for Xen 3.4 (thanks to Peter Bosch for
tracking down the bug). Maybe this will help for your configuration too.
How much physical RAM do you have? I'm not aware of the PAE code having
been tested with more than 4GB. If your machine is bigger, you could
try booting Xen with RAM restricted to, say, 2GB (should be a linux boot
parameter for this) and see if it makes a difference.
erik quanstrom
> mmu correction which was needed for Xen 3.4 (thanks to Peter Bosch for
> tracking down the bug). Maybe this will help for your configuration too.
>
> How much physical RAM do you have? I'm not aware of the PAE code having
> been tested with more than 4GB. If your machine is bigger, you could
> try booting Xen with RAM restricted to, say, 2GB (should be a linux boot
> parameter for this) and see if it makes a difference.
could you explain this a little further. pae would normally
not be useful with 2gb of ram and is often of dubious utility
on a 4gb machine. what's different about xen?
- erik
Richard Miller
> not be useful with 2gb of ram and is often of dubious utility
> on a 4gb machine. what's different about xen?
PAE is of no utility at all as far as I can see, but sometimes
we have to run xen guests on somebody else's hypervisor (think
virtual hosting companies). If the hypervisor is configured
with PAE (or 64-bit, it seems), the guest has to run in PAE mode.
I hacked the Plan 9 xen kernel to support PAE for this reason,
but since my xen test machine has only a couple of gigs I have
never actually seen it work in the non-trivial case where the
upper half of a 64-bit page address is non-zero. If our friend
has more than 4GB and the panic disappears when he uses less
than 2GB, that might indicate why he's having problems that
nobody else has seen.
Kenneth L Van Alstyne
For what it's worth, the following is my Xen config:
kernel = "/home/kvanals/plan9/9xeninst-pae.gz"
memory = 32
vcpus = '1'
name = "plan9.kvanals.org"
vif = [ 'mac=00:16:3E:9F:FF:01' ]
disk = [ 'file:/home/kvanals/plan9/plan9.img,sda,w',
'file:/home/kvanals/plan9/plan9.iso,sdb,r' ]
reboot = 'never'
extra="""
nobootprompt=local!/boot/bzroot
bootfile=sd01!cdboot!bootdisk.img
"""
Furthermore, this box only has 4GB of physical memory:
wired:~# xm info | grep total_memory
total_memory : 4095
Mr. Miller: I attempted to use your newly uploaded kernel, but was
greeted with a very similar panic:
wired:~# xm create -c /home/kvanals/plan9/boot.conf
Using config file "/home/kvanals/plan9/boot.conf".
Plan 9
127 holes free
0018a000 00dba000 12779520
12779520 bytes free
cpu0: 2792MHz AuthenticAMD AMD64 (cpuid: AX 0x20F51 DX 0x78BFBFF)
Started domain plan9.kvanals.org
panic: xenupdatema - pte 80184010 value
23c9b027
panic: xenupdatema - pte 80184010 value 23c9b027
dumpstack
ktrace /kernel/path 8001444c 80002e2c <<EOF
estackx 80003000
80002dcc=8001428d 80002dd8=80040836 80002ddc=8006bac4 80002de4=8006bb25
80002df0=80072d0c 80002e00=8004097d 80002e10=8006d0ba 80002e14=8001444c
80002e28=8001444c 80002e2c=80014291 80002e34=80040da3 80002e48=8006aa5a
80002e84=80073a0a 80002e90=80074844 80002ea4=80073c44 80002ed0=8006d0f9
80002ed4=8006aa5a 80002edc=80072d0c 80002ef0=80070200 80002ef4=80070206
80002f00=8006d0ba 80002f0c=8006d0ba 80002f10=80070438 80002f18=8007071c
80002f24=8006aa5a 80002f38=80070823 80002f44=8006c5a9 80002f60=8006cbf2
80002f8c=80011cc5 80002fac=80011be5 80002fb8=8001220a 80002fc4=80012256
80002fe4=00000000 80002fe8=801c5ff8 80002fec=80010a37 80002ff0=800b0147
80002ff4=00000000 80002ff8=8001001f 80002ffc=00000000
EOF
cpu0: exiting
cpu0: spurious interrupt 102, last 0
cpu0: spurious interrupt 102, last 0
cpu0: spurious interrupt 102, last 0
cpu0: spurious interrupt 102, last 0
...
Upon doing some "Googling" I came across one other mention of this problem
on 9fans, but it was never solved:
http://9fans.net/archive/2009/12/67
Thanks,
Kenny
Richard Miller
> kernel = "/home/kvanals/plan9/9xeninst-pae.gz"
> memory = 32
32MB might be a bit tight - try 64.
> cpu0: spurious interrupt 102, last 0
You can stop these messages after a crash by adding *debug=1 to the
plan9.ini section of your config file.
I've put a new 9xeninst-pae.gz on sources with more diagnostic
output to help track down the panic.
Let's continue this off-list because not many people are
interested in xen.
Jano
> > For what it's worth, the following is my Xen config:
> > memory 32
>
> 32MB might be a bit tight - try 64.
>
> > cpu0: spurious interrupt 102, last 0
>
> plan9.ini section of your config file.
>
> I've put a new 9xeninst-pae.gz on sources with more diagnostic
> output to help track down the panic.
>
> Let's continue this off-list because not many people are
> interested in xen.
oh? really?
Well, for one i am interested, so if you have continued off-list,
please cc me, i still have my amd64 xen host ready to host an plan9
guest if possible! I'd be so <stoked> to be able to finally have a
plan9 install!
Thanks,
John
Richard Miller
> ... if you have continued off-list,
> please cc me
Will do. I'll also announce to the list when (if) we get it working.
Richard Miller
It's working now. Turned out to be a known error in xen 3.2 (hypervisor
was initialising the mmu page directory table incorrectly). 9xen kernels
on sources now have a simple workaround for this.
Jano
> > I'll also announce to the list when (if) we get it working.
>
> was initialising the mmu page directory table incorrectly). Â 9xen kernels
> on sources now have a simple workaround for this.
wow awesome!
I'll try it soon..
Thanks,
John