FreeBSD and 6GB RAM
Marek Gorka
i have an intel 4400 platform equipped with 4 xeon processors and 6GB ram
(based on the chipset ServerWorks ServerSet II HE). i have fbsd 4.3 working
on it.
unfortunately, the system is unable to work with this big amount of ram.
there are two problems:
first of all, when the kernel is loaded, it says:
--
2GB of memory above 4GB ignored
--
after which the booting process continues, but a while after hangs with the
following message:
--
panic: swap_pager_swap_init: swap_zone == NULL
--
all i can do is reboot.
the server can be launched when i set the hw.physmem parameter to either 1G
or 2G in the bootloader.
the swap partition is fully operational (4G).
i think we might exclude hardware problems here, as the SMP works perfectly
fine (what means that the mainboard is supported). i had windows 2000
advanced server installed before on this machine and there were no problems
with ram.
does any one have experience in configuring such big machines? i'd be glad
for any suggestions as i've run out of ideas.
regards,
--
[......Marek Gorka, tp.internet.......]
[..........[ mar...@tpi.pl ]..........]
David Malone
>after which the booting process continues, but a while after hangs with the
>following message:
>--
>panic: swap_pager_swap_init: swap_zone == NULL
>--
>all i can do is reboot.
I think this message means that there was too much swap space
configured. There have been threads on the mailing list explaining
what kernel parameters to avoid this. Unfortunately I can't rememer
exactly when, but I think Matt Dillon would have written some of
the replies.
David.
Marek Gorka
> I think this message means that there was too much swap space
> configured.
i don't thinks so. during various tests i have also turned the swap off.
this message still remained.
Robert Picard
it or its not there, perhaps you have the swap drive configured incorrectly
(aslo dosnt bsd have a limitation that the swap space and boot secotrs have
to be below the 2gb limit?)
"Marek Gorka" <marek...@tpi.pl> wrote in message
news:3B541FBC...@tpi.pl...
David Malone
>David Malone wrote:
>> I think this message means that there was too much swap space
>> configured.
>i don't thinks so. during various tests i have also turned the swap off.
>this message still remained.
I think the amount it tries to allocate is based on the number of
pages in the system. Try looking for the line:
n = cnt.v_page_count * 2;
in /usr/src/sys/vm/swap_pager.c and changing the "* 2" to "/ 2".
David.
Marek Gorka
> its just me but if it says swap space = null, it means it either can't
> find it or its not there, perhaps you have the swap drive configured
> incorrectly (aslo dosnt bsd have a limitation that the swap space and boot
> secotrs have to be below the 2gb limit?)
well, i am sure that the swap configuration IS proper. swapinfo shows the
proper output. moreover, even if i turn the swap off (completely remove it
from the system, delete entries in /etc/fstab) the problem still remains.
i have compiled the kernel with the option:
options MAXMEM="(6144*1024)"
but it didn't help.
what is interesting, as i have written above, fbsd at booting says "ignoring
2G memory above 4G limit". when in bootloader i set the parameter hw.physmem
to 4G, immediately after launching the kernel it hangs.
i get some information about missing page before the system hangs.
running out of ideas.
Marek Gorka
> 2G memory above 4G limit". when in bootloader i set the parameter hw.physmem
> to 4G, immediately after launching the kernel it hangs.
hm, one more observation:
when the machine starts, bios tests 6G ram. next the bootloader is called, i
choose which partition to boot from. then bootloader shows the following
message:
--
BIOS 634kB/3767232kB available memory
--
and yes, when i set the hw.physmem to 3G, everything is okay. at the same
time, the kernel keeps showing the following message:
--
2GB of memory above 4GB ignored
--
so, maybe it is some kind of incompatibility between the bootloader and the
bios?
mark tinguely
>hello,
>
>i have an intel 4400 platform equipped with 4 xeon processors and 6GB ram
>(based on the chipset ServerWorks ServerSet II HE). i have fbsd 4.3 working
>on it.
>
>unfortunately, the system is unable to work with this big amount of ram.
>there are two problems:
>
>first of all, when the kernel is loaded, it says:
>
>--
>2GB of memory above 4GB ignored
>--
currently, the PDE/PTE in the Intel pmap.c/machdep.c is configured to map
4GB. I would have to look at the code some more, but in the quick look this
morning, I did not see the code adjust the physmap entry lower to 4GB.
Whether this could mess up the swap_zone mapping would also have to be
seen.
I would suggest that you specify your MAXMEM to 4GB.
--mark tinguely
mic...@lpthe.jussieu.fr
> i have compiled the kernel with the option:
> options MAXMEM="(6144*1024)"
> but it didn't help.
I am not expert about such stuff but i was convinced that the x86 architecture
limited memory size to 4gigs. In fact more or less 2 Gigs for a process and 2
Gigs for the kernel (in Linux you can choose 3 - 1 or the like). If this is
the case your panic is quite natural. There are VM experts lurking on the
group now, perhaps they will give a correct answer.
> what is interesting, as i have written above, fbsd at booting says "ignoring
> 2G memory above 4G limit". when in bootloader i set the parameter hw.physmem
> to 4G, immediately after launching the kernel it hangs.
> i get some information about missing page before the system hangs.
> running out of ideas.
> --
> [......Marek Gorka, tp.internet.......]
> [..........[ mar...@tpi.pl ]..........]
--
Michel Talon
Kirk Strauser
At 2001-07-17T09:44:50Z, Marek Gorka <marek...@tpi.pl> writes:
> i have an intel 4400 platform equipped with 4 xeon processors and 6GB ram
> (based on the chipset ServerWorks ServerSet II HE). i have fbsd 4.3
> working on it.
Have you tried any other operating system? It was my understanding that the
x86 (and other 32-bit) architecture could only address 2^32 (4G) bytes of
memory. Am I wrong?
--
Kirk Strauser
WJ Suijlen
> It was my understanding that the
>x86 (and other 32-bit) architecture could only address 2^32 (4G) bytes of
>memory. Am I wrong?
Because of the 16-bit heritage of the x86 processors, 386 (and higher)
can operate with 48-bit addresses.
In the age of 16-bit computing they segmented the memory in order to
work with memories > 64 kB. This segmenting tool can also be used to
extend 32-bit addresses > 4 GB.
Wijnand Suijlen
jpd
Holy schroothoop, that's some way to ignore the actual architecture
of i386+. sheesh. segment registers don't actually add bits, they're
indexes to a table. Still only 32 bits there.
AFAIK, any single i386+ can address only 4G mem.
But, using the mem control t00lz provided with the SMP setup of
said SMP machines, you can `translate' certain parts of memory
for each processor. So they can't all `see' the whole range at
once. But with some tricks, they all memory gets usable.
--
j p d (at) d s b (dot) t u d e l f t (dot) n l .
Alexander Viro
jpd <data...@elsewhere.net> wrote:
>On Tue, 17 Jul 2001 21:07:50 GMT, WJ Suijlen <w.jsu...@freeler.nl> wrote:
>>
>>
>>> It was my understanding that the
>>>x86 (and other 32-bit) architecture could only address 2^32 (4G) bytes of
>>>memory. Am I wrong?
>>
>> Because of the 16-bit heritage of the x86 processors, 386 (and higher)
>> can operate with 48-bit addresses.
>> In the age of 16-bit computing they segmented the memory in order to
>> work with memories > 64 kB. This segmenting tool can also be used to
>> extend 32-bit addresses > 4 GB.
Segments have nothing with that.
>Holy schroothoop, that's some way to ignore the actual architecture
>of i386+. sheesh. segment registers don't actually add bits, they're
>indexes to a table. Still only 32 bits there.
>
>AFAIK, any single i386+ can address only 4G mem.
>But, using the mem control t00lz provided with the SMP setup of
>said SMP machines, you can `translate' certain parts of memory
>for each processor. So they can't all `see' the whole range at
>once. But with some tricks, they all memory gets usable.
Not quite. 686 and higher have a bastardized version of the thing Sun
had done for quite a while. They support 36-bit physical address space.
You can switch the MMU on these boxen to 3-level page tables (instead of
normal 2-level ones). Normal split is 10:10:12 - 4Kb pages and 32-bit
page table entries. 686 allows 2:9:9:12 - still 32bit address space, but
page table entries are 64-bit.
IOW, you can have everything except the top-level table (it's location is
stored in 32-bit register) anywhere in physical memory - up to 64Gb.
Each process is still limited to 4Gb. However, between different contexts
you get more memory to play with. It has nothing to do with SMP, BTW -
context switch simply sets the pointer to top-level table (as usual)
and that changes the virtual->physical mapping. The only difference
from the usual case is that range of these mapping is bigger. Domain
remains the same, indeed.
--
"You're one of those condescending Unix computer users!"
"Here's a nickel, kid. Get yourself a better computer" - Dilbert.
Stephen E. Halpin
> In article <slrn9l9kq0.2bb...@oli252.rolahola.tudelft.nl>,
> jpd <data...@elsewhere.net> wrote:
> >On Tue, 17 Jul 2001 21:07:50 GMT, WJ Suijlen <w.jsu...@freeler.nl> wrote:
> >>
> >>
> >>> It was my understanding that the
> >>>x86 (and other 32-bit) architecture could only address 2^32 (4G) bytes of
> >>>memory. Am I wrong?
> >>
> >> Because of the 16-bit heritage of the x86 processors, 386 (and higher)
> >> can operate with 48-bit addresses.
> >> In the age of 16-bit computing they segmented the memory in order to
> >> work with memories > 64 kB. This segmenting tool can also be used to
> >> extend 32-bit addresses > 4 GB.
>
> Segments have nothing with that.
Segments would give you more than 4GB of virtual memory per process.
As you point out, Intel extended the architecture to address 64GB of physical
memory. Some processors which can address 64GB only cache 4GB
(see http://developer.intel.com/design/PentiumII/prodbref/index.htm as
an example.)
I don't see anything in the FreeBSD pages to indicate that it can support
this form of addressing. Solaris 8 and the 2.4 Linux kernel are both capable
of addressing 64GB:
http://soldc.sun.com/support/drivers/hcl/8/401/files/c0109.htm
http://sourceforge.net/foundry/linuxkernel/
-Steve
Marek Gorka
> Have you tried any other operating system?
as i have written in my first post, there was windows 2000 advanced server
intalled by default. it supported 6G ram.
mic...@lpthe.jussieu.fr
Sure, but neither Linux nor FreeBSD use the segment register as far as i know,
because in particular it is incredibly inefficient performance wise.
> Wijnand Suijlen
--
Michel Talon
Nick Hilliard
> I am not expert about such stuff but i was convinced that the x86 architecture
> limited memory size to 4gigs. In fact more or less 2 Gigs for a process and 2
> Gigs for the kernel (in Linux you can choose 3 - 1 or the like). If this is
> the case your panic is quite natural. There are VM experts lurking on the
> group now, perhaps they will give a correct answer.
My understanding is that you can get 386 and greater chips to address more than
4Gb using much the same segmentation tricks that were used in 80{,1,2}86 chips,
but I could be mistaken on this. There was a discussion about it recently on
one of the mailing lists.
Nick
mark tinguely
Stephen E. Halpin <s...@quadrizen.com> wrote:
>
>I don't see anything in the FreeBSD pages to indicate that it can support
>this form of addressing. Solaris 8 and the 2.4 Linux kernel are both capable
>of addressing 64GB:
>
> http://soldc.sun.com/support/drivers/hcl/8/401/files/c0109.htm
> http://sourceforge.net/foundry/linuxkernel/
you are correct, FreeBSD does not currently support three level
page tables for the Intel platform.
--mark tinguely
Acid
<mic...@lpthe.jussieu.fr> wrote in message
news:9j1lm6$319$1...@rose.lpthe.jussieu.fr...
> Marek Gorka <marek...@tpi.pl> wrote:
>
> > i have compiled the kernel with the option:
>
> > options MAXMEM="(6144*1024)"
>
> > but it didn't help.
>
> I am not expert about such stuff but i was convinced that the x86
architecture
> limited memory size to 4gigs. In fact more or less 2 Gigs for a process
and 2
Well "unconvince" yourself of that limit. Windows 2000 Advanced Server
handles 8GB, and DataCenter can do up to 64GB of RAM, all of this on x86
Intel platform. If anything, it is a limit with FreeBSD.
Stephen E. Halpin
You can't avoid the segment registers in the 32-bit world. Most systems
provide a 32-bit environment equivalent to the "tiny" model from DOS,
where everything appears in a single segment, but they are still there.
-Steve
Alexander Viro
I suspect that in case of FreeBSD it should be easier than in Linux -
we used to have all physical memory permanently mapped in kernel
space and getting rid of that was basically all it took. FreeBSD
has pmap. I.e. most of the problems should be already solved.
Dejan Oljaca
wrote:
>Marek Gorka <marek...@tpi.pl> wrote:
>
>> i have compiled the kernel with the option:
>
>> options MAXMEM="(6144*1024)"
This one is not a good idea...at least if you take a look at the
kernel source code. Keep this below 4GB as it denotes the virtual
memory limit per process.
>I am not expert about such stuff but i was convinced that the x86 architecture
>limited memory size to 4gigs. In fact more or less 2 Gigs for a process and 2
>Gigs for the kernel (in Linux you can choose 3 - 1 or the like). If this is
>the case your panic is quite natural. There are VM experts lurking on the
>group now, perhaps they will give a correct answer.
There's a difference between virtual memory (32-Bit) and physical
memory. To be able to use more than 4GB physical memory you need the
PAE feature coming with newer CPUs (36-Bit Address Space).
For the later I'm not sure if it is supported by FreeBSD.
Dejan Oljaca
>
>This one is not a good idea...at least if you take a look at the
>kernel source code. Keep this below 4GB as it denotes the virtual
>memory limit per process.
>
still valid...
Marek Gorka
> There's a difference between virtual memory (32-Bit) and physical
> memory. To be able to use more than 4GB physical memory you need the
> PAE feature coming with newer CPUs (36-Bit Address Space).
>
> For the later I'm not sure if it is supported by FreeBSD.
well, my cpus support PAE. i was told that fbsd doesn't support this
feature. and, therefore, it cannot handle more than 4G ram.
true?
mark tinguely
Alexander Viro <vi...@weyl.math.psu.edu> wrote:
>
>I suspect that in case of FreeBSD it should be easier than in Linux -
>we used to have all physical memory permanently mapped in kernel
>space and getting rid of that was basically all it took. FreeBSD
>has pmap. I.e. most of the problems should be already solved.
I would think a new architecture type should be defined to not penalize
systems that do no use over 4G with the additional Page Table and
address space overhead.
--mark tinguely
Alexander Viro
<shrug> You already handle Alpha, so code supporting 3-level page tables
shouldn't be hard to find. I suspect that it would be pretty much
limited to pmap.c, and not hard to merge with i386/i386/pmap.c at that.
ifdefs are evil, indeed, but in that case they might be pretty localized.
Keep in mind that top table is pretty much fixed due to hardware problems
on 686 - changing any of its entries requires full TLB flush. So it might
be a reasonable policy to consider top-level table + 4 second-level ones
as a single unit replacing the normal top-level of 386. IOW, you can allocate
4 pages + top-level table when you create a context and then consider
it not as 2:9:9:12 but as 11:9:12 with slightly different entry layout.
JD
"Alexander Viro" <vi...@weyl.math.psu.edu> wrote in message news:9j74oj$9...@weyl.math.psu.edu...
> In article <9j6lpq$ll5$1...@news.ndsu.nodak.edu>,
> mark tinguely <ting...@web.cs.ndsu.nodak.edu> wrote:
> >In article <9j4cut$5...@weyl.math.psu.edu>,
> >Alexander Viro <vi...@weyl.math.psu.edu> wrote:
> >>
> >>I suspect that in case of FreeBSD it should be easier than in Linux -
> >>we used to have all physical memory permanently mapped in kernel
> >>space and getting rid of that was basically all it took. FreeBSD
> >>has pmap. I.e. most of the problems should be already solved.
> >
> >I would think a new architecture type should be defined to not penalize
> >systems that do no use over 4G with the additional Page Table and
> >address space overhead.
>
> <shrug> You already handle Alpha, so code supporting 3-level page tables
> shouldn't be hard to find. I suspect that it would be pretty much
> limited to pmap.c, and not hard to merge with i386/i386/pmap.c at that.
> ifdefs are evil, indeed, but in that case they might be pretty localized.
>
the X86. It isn't brain surgery, but as suggested, there could be some subtile
differences in semantics (flushing schemes, etc.)
John
Dejan Oljaca
>feature. and, therefore, it cannot handle more than 4G ram.
>
>true?
It's true for now. Do not know, if other *BSDs support PAE. Maybe
Linux?
JD
"mark tinguely" <ting...@web.cs.ndsu.nodak.edu> wrote in message
news:9j1h03$2t9$1...@news.ndsu.nodak.edu...
It is ALOT more than just changing the configuration, but the code might have
been upgraded since I wrote it!!!
John
Dejan Oljaca
>> 4GB. I would have to look at the code some more, but in the quick look this
>> morning, I did not see the code adjust the physmap entry lower to 4GB.
>> Whether this could mess up the swap_zone mapping would also have to be
>> seen.
>>
>> I would suggest that you specify your MAXMEM to 4GB.
>>
>The version of PDE/PTEs that I remember supporting >4GB are twice as large???
>
>It is ALOT more than just changing the configuration, but the code might have
>been upgraded since I wrote it!!!
>
>John
>
As you said, it's not just a different mapping and a quick code
change. PAE requires to think of segmentet memory again. Which in
practice means, that we need some sort of library help to map
applications 32-Bit virtual flat address space into 36-Bit physical
address space (otherwise we still have the 32-Bit restriction per
process). The same problem the x86 architecture encountered when
switching to more than 1MB of RAM on 8-Bit plain DOS (e.g. XMMS, EMS).
mark tinguely
Dejan Oljaca <dw...@NOSPAM.myokay.net> wrote:
>As you said, it's not just a different mapping and a quick code
>change. PAE requires to think of segmentet memory again. Which in
>practice means, that we need some sort of library help to map
>applications 32-Bit virtual flat address space into 36-Bit physical
>address space (otherwise we still have the 32-Bit restriction per
>process).
From the Intel docs, we could use more than 4GB of memory in one pmap
by segmenting on either 4GB boundaries, by replacing the PDPT in the CR3,
or on 1GB boundaries, by replacing the PDT in the PDPT. Both methods
would make cases of memory to memory copies that are difficult to impossible.
This would also break any programs that have assumptions on the
sizeof(int) == sizeof(*int) -- no loss here, they should be fixed.
Also there could be some drivers that assume that Intel addressing
uses 32 bit memory addresses (DMA, etc) -- could be solved with
hacked bounced buffers in < 4GB addressed RAM with high order bits
ignored.
it probably has not been done yet because no one see it being useful.
especially since there are other capiable processors that have MMUs
that work correctly.
--mark tinguely.
glen herrmannsfeldt
Almost. The physical address bus is now 36 bits, (64GB), but the
bus going into the memory manager is only 32 bits. So a single process
can only access up to 4GB, usually some allocated to the OS, but
the total for all processes can be more.
As far as I have heard, Solaris 2.8 has the best implementation for
over 4GB of all x86 OS's. You might try that. I believe it is
free for research and development uses, and about $50 otherwise.
-- glen
glen herrmannsfeldt
>On Tue, 17 Jul 2001 21:07:50 GMT, WJ Suijlen <w.jsu...@freeler.nl> wrote:
>>
>>> It was my understanding that the x86 (and other 32-bit)
>>>architecture could only address 2^32 (4G) bytes of memory. Am I wrong?
>>
>> Because of the 16-bit heritage of the x86 processors, 386 (and higher)
>> can operate with 48-bit addresses.
>> In the age of 16-bit computing they segmented the memory in order to
>> work with memories > 64 kB. This segmenting tool can also be used to
>> extend 32-bit addresses > 4 GB.
>Holy schroothoop, that's some way to ignore the actual architecture
>of i386+. sheesh. segment registers don't actually add bits, they're
>indexes to a table. Still only 32 bits there.
As they say, bits is bits. It means that the virtual address space is
48 bits. Well, some are ring and global/local bits, so maybe 45 bits.
The physical address bus is 36 bits. However, the MMU bus is only 32
bits. So, to use the 45 bit virtual space the OS has to change segment
table entries at the appropriate time. Note that the 45 bit
virtual address space is available even if you have less than 4GB memory,
as long as you have enough disk. Most OS don't know how to allocate
segment selectors, though. It might be that OS/2 4.0 does.
The Watcom C compiler can do large mode (16+32) addressing.
-- glen
glen herrmannsfeldt
>WJ Suijlen <w.jsu...@freeler.nl> wrote:
Not as inefficient as the interrupt required for the OS to change
the tables. Segment registers are inefficient if you have to load
them too often. With 4GB segments, you shouldn't have to do that.
-- glen