pmap vs. large Zorro III space
Radosław Kujawa
Yesterday I've installed Matay Prometheus PCI, which allocates very large Zorro III space (512MB).
As soon as kernel was loaded, panic occured with "uvm_km_suballoc: unable to allocate space in parent map" error. Looks like the problem is during pmap_init. I think this is the same problem we've seen on the list in december. I was able to reproduce this on a recent GENERIC kernel (from releng.netbsd.org).
See this blurry pic for the full trace: http://c0ff33.net/pmappanic1.jpg
I checked out pmap_init and I see only two places where uvm_km_suballoc is called. One is allocation of segment table map, and the other is allocation of page tables map. Michael Hitch suggested, that it is probaby related to page tables map allocation and that I might want to increase VM_KERNEL_PT_PAGES (defined in vmparam.h). I've tried increasing it a bit (to 12, 15, 24), but no change was visible. Increasing the value too much (for example to 32) leads to a different kind of panic: http://c0ff33.net/pmappanic2.jpg
I won't act like I know what's going on in pmap code, because I don't know. Any sugestions would be highly appreciated.
--
Best regards,
Radosław Kujawa
--
Posted automagically by a mail2news gateway at muc.de e.V.
Please direct questions, flames, donations, etc. to news-...@muc.de
Frank Wille
Rados?aw Kujawa <radosla...@c0ff33.net> wrote:
> Yesterday I've installed Matay Prometheus PCI, which allocates very large
> Zorro III space (512MB).
>
> As soon as kernel was loaded, panic occured with "uvm_km_suballoc:
> unable to allocate space in parent map" error.
> [...]
> probaby related to page tables map allocation and that I might want to
> increase VM_KERNEL_PT_PAGES (defined in vmparam.h).
Yes. I doubt that there is enough space for page tables to map the whole
512MB area.
> I've tried increasing
> it a bit (to 12, 15, 24), but no change was visible. Increasing the value
> too much (for example to 32) leads to a different kind of panic:
Hmm. I cannot say that I understand the pmap module either, but you
might have seen that VM_KERNEL_PT_PAGES is assigned to Sysptsize in
m68k/m68k/pmap_motorola.c?
The key to understand all the pmap allocation stuff is to go through
start_c() in amiga_init.c. Here Sysptsize will be used to allocate space
for the page tables, and the initial MMU setup occurs.
To map 512MB you would need 512MB/4096 = 131072 page descriptors, which
fit into 131072/(4096/4) = 128 pages. That's a lot more than the default
of 8 for VM_KERNEL_PT_PAGES.
On the other hand 128 pages are (only) 512k. So it should be possible
somehow, provided you have enough RAM...?
--
Frank Wille
Michael van Elst
>On Fri, 22 Jul 2011 20:09:17 +0200
>Rados?aw Kujawa <radosla...@c0ff33.net> wrote:
>> Yesterday I've installed Matay Prometheus PCI, which allocates very large
>> Zorro III space (512MB).
>>
>> As soon as kernel was loaded, panic occured with "uvm_km_suballoc:
>> unable to allocate space in parent map" error.
>> [...]
>> probaby related to page tables map allocation and that I might want to
>> increase VM_KERNEL_PT_PAGES (defined in vmparam.h).
>Yes. I doubt that there is enough space for page tables to map the whole
>512MB area.
NetBSD 5.99.51 (DUMMY) #0: Sun May 1 13:43:56 CEST 2011
mlelstv@henery:/home/netbsd-current/obj.amiga/home/netbsd-current/src/sys/arch/amiga/compile/DUMMY
total memory = 2048 MB
avail memory = 2021 MB
memory segment 0 at 40000000 size 80000000
memory segment 1 at 00000000 size 00200000
Why does it not panic here?
# pmap 0
00000000 24824K read/write/exec [ stack ]
0183E000 8K read/write/exec [ stack ]
01840000 6616K read/write/exec [ stack ]
01EB6000 1184K read/write/exec [ stack ]
01FDE000 64K read/write/exec [ stack ]
01FEE000 132520K read/write/exec [ kmem_map ]
0A158000 16384K read/write/exec [ pager_map ]
0B158000 416K read/write/exec [ anon ]
0B1C0000 16384K read/write/exec [ phys_map ]
0C1C0000 4856K read/write/exec [ anon ]
0C67E000 8192K read/write/exec [ ubc_pager ]
0CE7E000 360K read/write/exec [ anon ]
0CED8000 16K read/write/exec [ uvm_aobj ]
0CEDC000 400K read/write/exec [ anon ]
0CF40000 256K read/write/exec [ uvm_aobj ]
0CF80000 32K read/write/exec [ anon ]
0CF90000 192K read/write/exec [ anon ]
10000000 303104K read/write/exec [ pt_map ]
total 515808K
# vmstat -s
8192 bytes per page
1 page color
259566 pages managed
257652 pages free
282 pages active
...
Greetings,
--
Michael van Elst
Internet: mle...@serpens.de
"A potential Snark may lurk in every tree."
--
--
Michael van Elst
Internet: mle...@serpens.de
"A potential Snark may lurk in every tree."
Michael van Elst
> Could you try creating large (512MB) Z III I/O board in your emulated environment? It should trigger the panic.
Looks like it:
panic: uvm_km_suballoc: unable to allocate space in parent map
Stopped in pid 0.1 (system) at netbsd:cpu_Debugger+0x6: unlk a6
db> bt
cpu_Debugger(12800000,38c2a0,431f38,431edc,431ef8) + 6
panic(326653,431f34,15,20e94300,431f38) + 1d0
uvm_km_suballoc(37fa00,431f38,431f34,12800000,0) + c8
pmap_init(20c96000,ff000000,431f64,431f60,39054c) + 286
uvm_init(38750c,2,0,0,0) + 5c
main(431fb4) + 4a
start() + 286
--
Michael van Elst
Internet: mle...@serpens.de
"A potential Snark may lurk in every tree."
--
Michael L. Hitch
> I checked out pmap_init and I see only two places where uvm_km_suballoc
> is called. One is allocation of segment table map, and the other is
> allocation of page tables map. Michael Hitch suggested, that it is
> probaby related to page tables map allocation and that I might want to
> increase VM_KERNEL_PT_PAGES (defined in vmparam.h). I've tried
> increasing it a bit (to 12, 15, 24), but no change was visible.
> Increasing the value too much (for example to 32) leads to a different
> kind of panic: http://c0ff33.net/pmappanic2.jpg
>
> I won't act like I know what's going on in pmap code, because I don't
> know. Any sugestions would be highly appreciated.
One other thing I was thinking is that the user page tables start at a
fixed address of 0x10000000 (see M68K_PTBASE in
amiga/include/pte_motorola.h). There may not be enough address space
between the kernel and M68K_PTBASE to fit everything.
You might try moving that to 0x20000000.
---
Michael L. Hitch mhi...@montana.edu
Operations Consulting, Information Technology Center
Montana State University, Bozeman, MT USA
Radoslaw Kujawa
On Jul 25, 2011, at 7:15 PM, Michael L. Hitch wrote:
>
> One other thing I was thinking is that the user page tables start at a fixed address of 0x10000000 (see M68K_PTBASE in amiga/include/pte_motorola.h). There may not be enough address space between the kernel and M68K_PTBASE to fit everything.
>
> You might try moving that to 0x20000000.
Looks like you are right :-). I still need to do some additional testing, but probably that's the case here.
--
Best regards,
Radoslaw Kujawa
Michael L. Hitch
> On Jul 25, 2011, at 7:15 PM, Michael L. Hitch wrote:
>>
>> One other thing I was thinking is that the user page tables start at a
>> fixed address of 0x10000000 (see M68K_PTBASE in
>> amiga/include/pte_motorola.h). There may not be enough address space
>> between the kernel and M68K_PTBASE to fit everything.
>>
>> You might try moving that to 0x20000000.
>
> Looks like you are right :-). I still need to do some additional testing, but probably that's the case here.
I had thought I had remembered seeing something in amiga_init.c that was
dynamically setting the user page table base, and I was correct. Some
time ago, the amiga pmap was changed to use a computed base, but that
capability disappeared when I switched to the common m68k pmap.
Mike
---
Michael L. Hitch mhi...@montana.edu
Operations Consulting, Information Technology Center
Montana State University, Bozeman, MT USA
--
Radoslaw Kujawa
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
2006, 2007, 2008, 2009, 2010, 2011
The NetBSD Foundation, Inc. All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
NetBSD 5.99.45 (A3K_MPPB_DEV) #22: Thu Jul 28 15:30:03 CEST 2011
rku...@Radoslaws-MacBook-Pro.local:/Users/rkujawa/repos/NetBSD-current/
src/sys/arch/amiga/compile/obj/A3K_MPPB_DEV
Amiga 3000 (68060 rev.1 CPU/MMU/FPU)
total memory = 127 MB
avail memory = 123 MB
memory segment 0 at 08000000 size 07f80000
memory segment 1 at 07c00000 size 00400000
memory segment 2 at 00000000 size 00200000
sysctl_createv: sysctl_create(no_sa_support) returned 22
timecounter: Timecounters tick every 10.000 msec
mainbus0 (root)
clock0 at mainbus0: CIA B system hz 100 hardware hz 709379
timecounter: Timecounter "CIA B" frequency 709379 Hz quality 100
Calibrating delay loop... 21/1024 us
a34kbbc0 at mainbus0
ser0 at mainbus0: input fifo 512 output fifo 32
kbd0 at mainbus0: CIA A type Amiga
ms0 at mainbus0
grfcc0 at mainbus0
grf0 at grfcc0: width 640 height 400 colors 4
ite0 at grf0: rows 50 cols 80 repeat at (30/100)s next at (10/100)s has keyboard
fdc0 at mainbus0: dmabuf pa 0x1e4bf0: dmabuf ka 0x44abf0
fd0 at fdc0 unit 0: 3.5dd 80 cyl, 2 head, 11 sec [9 sec], 512 bytes/sec
ahsc0 at mainbus0
scsibus0 at ahsc0: 8 targets, 8 luns per target
aucc0 at mainbus0
audio0 at aucc0: half duplex, playback, capture
zbus0 at mainbus0: i/o size 0x22000000
board at zbus0: pa 0x40000000 man/pro 2206/62 not configured
board at zbus0: pa 0x41000000 man/pro 3643/16 not configured
board at zbus0: pa 0xe90000 man/pro 3643/19 not configured
xsurf at zbus0: pa 0xea0000 man/pro 4626/23 not configured
mppb0 at zbus0 pa 0x60000000 man/pro 44359/1: Matay Prometheus PCI bridge
pci0 at mppb0 bus 0
pci0: i/o space, memory space enabled
3Dfx Interactive Voodoo3 (VGA display, revision 0x01) at pci0 dev 0 function 0 not configured
cbiiisc0 at zbus0 pa 0xf01060 man/pro 8512/100
cbiiisc0: no SCSI termination, host adapter deactivated.
scsibus1 at cbiiisc0: 0 targets, 8 luns per target
timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
scsibus0: waiting 2 seconds for devices to settle...
scsibus1: waiting 2 seconds for devices to settle...
4 views configured
root device:
Radoslaw Kujawa
Best regards,
Radoslaw Kujawa
On Jul 28, 2011, at 12:23 AM, Michael L. Hitch wrote:
> On Wed, 27 Jul 2011, Radoslaw Kujawa wrote:
>
>> On Jul 25, 2011, at 7:15 PM, Michael L. Hitch wrote:
>>>
>>> One other thing I was thinking is that the user page tables start at a fixed address of 0x10000000 (see M68K_PTBASE in amiga/include/pte_motorola.h). There may not be enough address space between the kernel and M68K_PTBASE to fit everything.
>>>
>>> You might try moving that to 0x20000000.
>>
>> Looks like you are right :-). I still need to do some additional testing, but probably that's the case here.
>
> I had thought I had remembered seeing something in amiga_init.c that was dynamically setting the user page table base, and I was correct. Some time ago, the amiga pmap was changed to use a computed base, but that capability disappeared when I switched to the common m68k pmap.
It appears to work after moving M68K_PTBASE, but I have trouble accessing the VA-mapped Zorro III space.
My Prometheus is configured at PA 0x40000000 and is successfully mapped to VA 0x66a000. However, accessing it always results in a bus error. Examining any virtual address from 0x66a000 - 0x2066a3000 range in DDB results in bus error too.
I think there's still some problem… but I don't know if this is related or not.
zbus0 at mainbus0: i/o size 0x22000000
zbusmap: pa 0x40000000, size 536870912, avail 570425344, kva 0x66a000, nextkva 0x2066a000
physaccess: pte @ 0xff000cd4 kvtopte(0x66a000)
pte @ 0xff000cd4 = PA 40000000
pte @ 0xff000cd8 = PA 40002000
pte @ 0xff000cdc = PA 40004000
(…)
40000000 = kvtop(0x66a000) ok
mppb: matched by Zorro ID 44359, 1
mppb0 at zbus0 pa 0x40000000 man/pro 44359/1: Matay Prometheus PCI bridge
mppb: mapped conf 400f0000 -> 75a000, io 40000000 -> 66a000, mem 40100000 -> 76a000
pci0 at mppb0 bus 0
will read from pc->pci_conf_ioh: 75a000 + c
trap type 0, code = 1050020, v = 75a00c
pid = 0, lid = 1, pc = 0000FA36, ps = 2700, sfc = 1, dfc = 1
Registers:
0 1 2 3 4 5 6 7
dreg: 00000000 00000000 0000000C 00000000 2CBB2C30 0014F3F8 00000000 00000000
areg: 0075A00C 001F9A34 2BA90FB8 000D388A 2CBAE000 2CBB2C30 001F9A10 2DFFFFFC
Kernel stack (001F9918):
1F9918: 000ED344 001F99BC 00000080 0016FFF8 00000000 01050020 0075A00C 00000000
1F9938: 001F99A4 000EDB0C 00000000 01050020 0075A00C 001F99BC 0000000C 00000000
1F9958: 2CBB2C30 0014F3F8 00000000 2BA90FB8 000D388A 2CBAE000 2CBB2C30 00000001
1F9978: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
1F9998: 00000000 00000000 00000000 001F9A10 00002060 001F99BC 00000000 01050020
1F99B8: 0075A00C 00000000 00000000 0000000C 00000000 2CBB2C30 0014F3F8 00000000
1F99D8: 00000000 0075A00C 001F9A34 2BA90FB8 000D388A 2CBAE000 2CBB2C30 001F9A10
1F99F8: 2DFFFFFC 00000000 27000000 FA364008 0075A00C 01050020 001F9A3C 00093020
1F9A18: 0075A000 0000000C 00000000 00000000 2BA90FB8 001F9B98 00000000 00000000
1F9A38: 00000000 001F9A88 000A3DB6 2BA90FB8 00000000 0000000C 2BA90FB8 00000000
1F9A58: 00000000 00000000 00000002 00000000 2CBB2C30 00000000 00000000 00000002
1F9A78: 2CBAE000 001F9B98 000D358E 2CBB2C30 001F9AA4 000A3F28 2CBAE000 0014F3F8
1F9A98: 00000000 00000000 2CBB2C30 001F9AE8 000A423C 2CBB2C30 00142A6D 0014F3F8
1F9AB8: 2CBB2C30 000A43C4 2BA90FB8 00000000 0014EC91 2CBAC21C 2CBAC200 001F9B98
1F9AD8: FFFFFFF7 0017BB10 00142A6D 00000000 001F9B20 000C8748 2CBAC200 2CBB2C30
1F9AF8: 001F9B98 2CBB2C30 001F9B98 001F9B98 2CBAC200 00000000 000A7F14 000D388A
panic: Bus error
Stopped in pid 0.1 (system) at netbsd:cpu_Debugger+0x6: unlk a6
db> x 75a00c
0x75a00c:Skipping crash dump on recursive panic
panic: Bus error
Stopped in pid 0.1 (system) at netbsd:cpu_Debugger+0x6: u
nlk a6
db> x 75c00c
0x75c00c:Skipping crash dump on recursive panic
panic: Bus error
Stopped in pid 0.1 (system) at netbsd:cpu_Debugger+0x6: u
nlk a6
db> x 75e00c
0x75e00c:Skipping crash dump on recursive panic
panic: Bus error
Stopped in pid 0.1 (system) at netbsd:cpu_Debugger+0x6: u
nlk a6
db> x 76000c
0x76000c:Skipping crash dump on recursive panic
panic: Bus error
Stopped in pid 0.1 (system) at netbsd:cpu_Debugger+0x6: u
nlk a6
--
Best regards,
Radosław Kujawa