i386 with 4GB RAM: less than 2GB available on A2SAV (Intel Atom E3940)

[Andre Albsmeier]

I have a lot of machines running with 4 GB physical RAM and, for
some reasons, I still have to use a 32 bits OS.

All of them show something between 3 and 3.5 GB of RAM available
in dmesg but the brand new Supermicro A2SAV really shocked me:

FreeBSD 11.1-STABLE #0: Mon Jan 15 06:57:10 CET 2018
...
real memory = 4294967296 (4096 MB)
avail memory = 1939558400 (1849 MB)
...

So do people have any ideas how I might get a bit closer to at least
3 GB? I assume there are no FreeBSD knobs which might help but hope
dies last...
_______________________________________________
freebsd...@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stabl...@freebsd.org"

[Eugene Grosbein]

28.01.2018 21:57, Andre Albsmeier wrote:

> I have a lot of machines running with 4 GB physical RAM and, for
> some reasons, I still have to use a 32 bits OS.
>
> All of them show something between 3 and 3.5 GB of RAM available
> in dmesg but the brand new Supermicro A2SAV really shocked me:
>
> FreeBSD 11.1-STABLE #0: Mon Jan 15 06:57:10 CET 2018
> ...
> real memory = 4294967296 (4096 MB)
> avail memory = 1939558400 (1849 MB)
> ...
>
> So do people have any ideas how I might get a bit closer to at least
> 3 GB? I assume there are no FreeBSD knobs which might help but hope
> dies last...

First, try to decrease amount of RAM dedicated to integrated video, if any (BIOS Setup).

Also, I'd like to know reasons that made you stick to 32 bit OS
as we have pretty good support for 32 bit applications running under 64 bit system.

[Dimitry Andric]

On 28 Jan 2018, at 15:57, Andre Albsmeier <Andre.A...@siemens.com> wrote:
> I have a lot of machines running with 4 GB physical RAM and, for
> some reasons, I still have to use a 32 bits OS.
>
> All of them show something between 3 and 3.5 GB of RAM available
> in dmesg but the brand new Supermicro A2SAV really shocked me:
>
> FreeBSD 11.1-STABLE #0: Mon Jan 15 06:57:10 CET 2018
> ...
> real memory = 4294967296 (4096 MB)
> avail memory = 1939558400 (1849 MB)
> ...
>
> So do people have any ideas how I might get a bit closer to at least
> 3 GB? I assume there are no FreeBSD knobs which might help but hope
> dies last...

This is a common problem on i386. Most likely some ranges are reserved
for I/O mappings, such as video cards. If you boot with -v, I think the
kernel prints an overview of the physical ram chunks available? I don't
know of any other way to get such an overview.

Another option is to try PAE, but I have no idea how stable that is...

-Dimitry

[Mike Karels]

> On 28 Jan 2018, at 15:57, Andre Albsmeier <Andre.A...@siemens.com> =

> wrote:
> > I have a lot of machines running with 4 GB physical RAM and, for
> > some reasons, I still have to use a 32 bits OS.

> >=20

> > All of them show something between 3 and 3.5 GB of RAM available
> > in dmesg but the brand new Supermicro A2SAV really shocked me:

> >=20

> > FreeBSD 11.1-STABLE #0: Mon Jan 15 06:57:10 CET 2018
> > ...

> > real memory =3D 4294967296 (4096 MB)
> > avail memory =3D 1939558400 (1849 MB)
> > ...
> >=20

> > So do people have any ideas how I might get a bit closer to at least
> > 3 GB? I assume there are no FreeBSD knobs which might help but hope
> > dies last...

> This is a common problem on i386. Most likely some ranges are reserved
> for I/O mappings, such as video cards. If you boot with -v, I think the
> kernel prints an overview of the physical ram chunks available? I don't
> know of any other way to get such an overview.

> Another option is to try PAE, but I have no idea how stable that is...

> -Dimitry

I suspect that the unavailable RAM has been mapped above 4 GB by the BIOS.

About PAE: at $JOB, we have a FreeBSD 8.2 system that has been running
PAE reliably since 8.2 was new. Also, we ship amd64 systems that run
mostly 32-bit binaries, which works well.

Mike

[Eric van Gyzen]

sysctl machdep.smap on BIOS, machdep.efi_map on UEFI.

Eric

[Andre Albsmeier]

On Sun, 28-Jan-2018 at 22:51:04 +0700, Eugene Grosbein wrote:
> 28.01.2018 21:57, Andre Albsmeier wrote:
>
> > I have a lot of machines running with 4 GB physical RAM and, for
> > some reasons, I still have to use a 32 bits OS.
> >
> > All of them show something between 3 and 3.5 GB of RAM available
> > in dmesg but the brand new Supermicro A2SAV really shocked me:
> >
> > FreeBSD 11.1-STABLE #0: Mon Jan 15 06:57:10 CET 2018
> > ...
> > real memory = 4294967296 (4096 MB)
> > avail memory = 1939558400 (1849 MB)
> > ...
> >
> > So do people have any ideas how I might get a bit closer to at least
> > 3 GB? I assume there are no FreeBSD knobs which might help but hope
> > dies last...
>
> First, try to decrease amount of RAM dedicated to integrated video, if any (BIOS Setup).

Done that. I have set everything as small as possible but this didn't help.
After a BIOS upgrade, I found the promising option

MAX TOLUD

which was set to 2GB. I changed it to 3GB but nothing changed.

>
> Also, I'd like to know reasons that made you stick to 32 bit OS
> as we have pretty good support for 32 bit applications running under 64 bit system.

I (still) have 32 bit machines and don't want to maintain 2 userlands.
Each machine has its own kernel but userland (updated via nfs) must
remain 32 bit.

Or is it possible to boot a 64 bit kernel and have everything else in
32 bit?

[Andre Albsmeier]

On Sun, 28-Jan-2018 at 17:05:54 +0100, Dimitry Andric wrote:
> On 28 Jan 2018, at 15:57, Andre Albsmeier <Andre.A...@siemens.com> wrote:
> > I have a lot of machines running with 4 GB physical RAM and, for
> > some reasons, I still have to use a 32 bits OS.
> >
> > All of them show something between 3 and 3.5 GB of RAM available
> > in dmesg but the brand new Supermicro A2SAV really shocked me:
> >
> > FreeBSD 11.1-STABLE #0: Mon Jan 15 06:57:10 CET 2018
> > ...
> > real memory = 4294967296 (4096 MB)
> > avail memory = 1939558400 (1849 MB)
> > ...
> >
> > So do people have any ideas how I might get a bit closer to at least
> > 3 GB? I assume there are no FreeBSD knobs which might help but hope
> > dies last...
>
> This is a common problem on i386. Most likely some ranges are reserved
> for I/O mappings, such as video cards. If you boot with -v, I think the
> kernel prints an overview of the physical ram chunks available? I don't

Yes, it does:

real memory = 4294967296 (4096 MB)

Physical memory chunk(s):
0x0000000000001000 - 0x000000000009afff, 630784 bytes (154 pages)
0x0000000000100000 - 0x00000000003fffff, 3145728 bytes (768 pages)
0x0000000000c28000 - 0x000000001fffffff, 524124160 bytes (127960 pages)
0x0000000022151000 - 0x0000000075733fff, 1398681600 bytes (341475 pages)
0x000000007998e000 - 0x0000000079a5efff, 856064 bytes (209 pages)
0x000000007a151000 - 0x000000007a4bffff, 3600384 bytes (879 pages)
0x000000007a4eb000 - 0x000000007aae2fff, 6258688 bytes (1528 pages)
0x000000007aae5000 - 0x000000007afeffff, 5287936 bytes (1291 pages)
avail memory = 1939800064 (1849 MB)

-Andre

> know of any other way to get such an overview.
>
> Another option is to try PAE, but I have no idea how stable that is...
>
> -Dimitry
>

--
Win98: useless extension to a minor patch release for 32-bit extensions
and a graphical shell for a 16-bit patch to an 8-bit operating
system originally coded for a 4-bit microprocessor, written by
a 2-bit company that can't stand for 1 bit of competition.

[Andre Albsmeier]

But can the entire userland be 32 bit only?

Maybe I'll try the PAE thing...

-Andre

[Eugene Grosbein]

On 30.01.2018 13:59, Andre Albsmeier wrote:

>> Also, I'd like to know reasons that made you stick to 32 bit OS
>> as we have pretty good support for 32 bit applications running under 64 bit system.
>
> I (still) have 32 bit machines and don't want to maintain 2 userlands.
> Each machine has its own kernel but userland (updated via nfs) must
> remain 32 bit.
>
> Or is it possible to boot a 64 bit kernel and have everything else in
> 32 bit?

I have not tried "everything else in 32 bit", there may be some rough edges
dealing with run-time linker but you can try.

/sbin/init is statically linked and it should work with kernel having option COMPAT_FREEBSD32.
/bin/sh may be OK too provided /libexec/ld-elf32.so.1 is in place.

You should really consider switching to 64 bit kernel for such hardware.

[Boris Samorodov]

30.01.2018 09:59, Andre Albsmeier пишет:

> After a BIOS upgrade,

Did you try to reset BIOS to default settings? A BIOS upgrade
is the right time to try this.

> I found the promising option
>
> MAX TOLUD
>
> which was set to 2GB. I changed it to 3GB but nothing changed.

Hm, TOLUD is Top Of Lower Usable Dram. That memory is often used
for GPU (minering). I'd say that you may be interested in *decreasing*
it as it is effectively memory that is not used by an OS. There may be
an AUTO choice to try as well.

--
WBR, Boris Samorodov (bsam)
FreeBSD Committer, http://www.FreeBSD.org The Power To Serve

[Mike Karels]

> On 30.01.2018 13:59, Andre Albsmeier wrote:

> >> Also, I'd like to know reasons that made you stick to 32 bit OS
> >> as we have pretty good support for 32 bit applications running under 64 bit system.
> >
> > I (still) have 32 bit machines and don't want to maintain 2 userlands.
> > Each machine has its own kernel but userland (updated via nfs) must
> > remain 32 bit.
> >
> > Or is it possible to boot a 64 bit kernel and have everything else in
> > 32 bit?

> I have not tried "everything else in 32 bit", there may be some rough edges
> dealing with run-time linker but you can try.

> /sbin/init is statically linked and it should work with kernel having option COMPAT_FREEBSD32.
> /bin/sh may be OK too provided /libexec/ld-elf32.so.1 is in place.

> You should really consider switching to 64 bit kernel for such hardware.

You definitely cannot run all of userland in 32-bit mode. There are many
sysadin programs that have incompatible syscall interfaces, starting with
mount, ifconfig, ps, route, netstat, etc (probably 50 total). Unless they
were all statically linked, you would have to install the 64-bit shared
libraries, moving the 32-bit libraries to /lib32 and /usr/lib32, and
switching around /libexec/ld-elf*.

Or, if you really want the userland to be the same, you could use a PAE
kernel.

Mike

[Alan Somers]

Sure. I do this with jails. It's no problem to have a 32-bit jail on a
64-bit kernel. Kernel modules would be an issue, though. If you need any,
you'll have to find a way for the 64-bit machines to find 64-bit kernel
modules.

>
> Maybe I'll try the PAE thing...
>

[Konstantin Belousov]

On Tue, Jan 30, 2018 at 07:54:41AM -0600, Mike Karels wrote:
> > On 30.01.2018 13:59, Andre Albsmeier wrote:
>
> > >> Also, I'd like to know reasons that made you stick to 32 bit OS
> > >> as we have pretty good support for 32 bit applications running under 64 bit system.
> > >
> > > I (still) have 32 bit machines and don't want to maintain 2 userlands.
> > > Each machine has its own kernel but userland (updated via nfs) must
> > > remain 32 bit.
> > >
> > > Or is it possible to boot a 64 bit kernel and have everything else in
> > > 32 bit?
>
> > I have not tried "everything else in 32 bit", there may be some rough edges
> > dealing with run-time linker but you can try.
>
> > /sbin/init is statically linked and it should work with kernel having option COMPAT_FREEBSD32.
> > /bin/sh may be OK too provided /libexec/ld-elf32.so.1 is in place.
>
> > You should really consider switching to 64 bit kernel for such hardware.
>
> You definitely cannot run all of userland in 32-bit mode. There are many
> sysadin programs that have incompatible syscall interfaces, starting with
> mount, ifconfig, ps, route, netstat, etc (probably 50 total). Unless they
> were all statically linked, you would have to install the 64-bit shared
> libraries, moving the 32-bit libraries to /lib32 and /usr/lib32, and
> switching around /libexec/ld-elf*.

ps(8) compatibility for compat32 is nearly perfect, mount (in its nmount
syscall variant) also works. There is no issue with /libexec/ld-elf.so.1
being 32bit, whatever rumors whoever tried to spread.

Yes, networking administrative interfaces are not functional for compat32.
This precludes both ifconfig(8) and route(8) from operating. Also, firewall
management tools, for all three FreeBSD firewalls, can only work with
matching kernel.

[Brooks Davis]

There are some deficiencies in the management space[0], but it should
generally work. Where it doesn't it's a bug and usually not too hard to
fix.

-- Brooks

[0] e.g. https://reviews.freebsd.org/D13459

[Andre Albsmeier]

On Sun, 28-Jan-2018 at 10:32:44 -0600, Mike Karels wrote:

Finally I found some time to play with the PAE option. I added

option PAE
option KVA_PAGES=1024

and the A2SAV booted and gave me

real memory = 4294967296 (4096 MB)
avail memory = 4048207872 (3860 MB)

Very encouraging, this is double of what I had before! So I decided to try
this on my desktop machine which also boots but loading the i915kms and drm2
stuff fails with

info: [drm] Initialized drm 1.1.0 20060810
drmn0: <Intel IvyBridge> on vgapci0
error: [drm:pid949:i915_gem_gtt_init] *ERROR* Scratch setup failed
device_attach: drmn0 attach returned 12

The KMS stuff doesn't support the A2SAV anyway so I can't test how
it would behave there...