How to activate swap?

[Alain Knaff]

Hi,

Does anybody know how to activate swap on the 3B1?

I have 2 disks, with each a 4MB swap partition on it, but apparently
there is only around 2.5MB of memory available for the applications
(pool for *all* of them).


# iv -tv /dev/rfp000
Winchester disk
Volume Name: WINCHE
1400 Cylinders. 16 Heads per Cylinder.
There are 17 Physical Sectors (of 512 bytes) per Track.
272 Physical Sectors per Cylinder, 380800 Physical Sectors per Disk.
There are 8 Logical Blocks (of 1024 bytes) per Track.
128 Logical Blocks per Cylinder, 179200 Logical Blocks per Disk.
The Step Rate supplied to the Controller is 0.
Partition 0: start Track=0, size (in Blocks)=128
Partition 1: start Track=16, size (in Blocks)=4000
Partition 2: start Track=516, size (in Blocks)=175072
Loader starts at Block 2 (size=23 Blocks).
Bad Block Table starts at Block 1 (size=1 Blocks).
The Bad Block Table contains 0 entries.

# iv -tv /dev/rfp010
Second winchester disk
Volume Name: WINCHE
989 Cylinders. 5 Heads per Cylinder.
There are 17 Physical Sectors (of 512 bytes) per Track.
85 Physical Sectors per Cylinder, 84065 Physical Sectors per Disk.
There are 8 Logical Blocks (of 1024 bytes) per Track.
40 Logical Blocks per Cylinder, 39560 Logical Blocks per Disk.
The Step Rate supplied to the Controller is 0.
Partition 0: start Track=0, size (in Blocks)=40
Partition 1: start Track=5, size (in Blocks)=4000
Partition 2: start Track=505, size (in Blocks)=35520
Bad Block Table starts at Block 1 (size=1 Blocks).
The Bad Block Table contains 0 entries.

iv doesn't seem to show the type of the partitions, but I created the
swap partition on the second hard disk was created using the
01_Diagnosic_Disk_Ver_3.51 boot floppy, so it is for sure indeed a
swap partition :-)

To test whether swap is indeed available, I made a program that
gobbles up as much memory as it can (in chunks of 100K), and as soon
as it doesn't get any, breaks out of the loop, and waits for 10
seconds.

The first instance of the program gets 2.3MB. The second one gets
0.1MB.

I conclude by this that only about 2.5MB are available in *total*
i.e. only system memory, and no swap.


Questions:

1. Is there any command to activate the existing swap partitions (like
"swapon" on Linux)?

2. Is there any command to easily display available memory (like "free"
on Linux)

Anything else that I might be missing here?

Thanks,

--
Alain

[J Booth]

Hi Alain.

>>I conclude by this that only about 2.5MB are available in *total*

So it hit me today, I recall seeing the memory map for the 4MB RAM address space (0x0 - 0x3FFFFF), and sure enough, only around 2.5 MB allocated to user processes:

KERNEL ONLY = 00_00_00 – 07_FF_FF (512 KB)
User Process = 08_00_00 – 2F_FF_FF (2.5 MB)
Shared Lib = 30_00_00 – 37_FF_FF (512 KB)
Kernel VM = 38_00_00 – 3F_FF_FF (512 KB)

So I don't think increasing swap is going to help much given the limited address space.

Jesse

[Alain Knaff]

Hi Jesse,

On 24/10/2022 07:45, J Booth wrote:

> Hi Alain.
>
>>> I conclude by this that only about 2.5MB are available in *total*
> So it hit me today, I recall seeing the memory map for the 4MB RAM address space (0x0 - 0x3FFFFF), and sure enough, only around 2.5 MB allocated to user processes:
>
> KERNEL ONLY = 00_00_00 – 07_FF_FF (512 KB)
> User Process = 08_00_00 – 2F_FF_FF (2.5 MB)
> Shared Lib = 30_00_00 – 37_FF_FF (512 KB)

Yes, I have figured this out since then

> Kernel VM = 38_00_00 – 3F_FF_FF (512 KB)

That bit I didn't know yet. As far as I know, Linux does the same (i.e.
the kernel exists in the application's address space, even though
inaccessible to the application, obviously)

>
> So I don't think increasing swap is going to help much given the limited address space.

Actually, the problem that I used to have was running "many" *processes*
concurrently.

When configuring a software package using GNU autoconf, at some point in
time there's a multi-step pipeline and each step implies needs an
additional shell process. With ksh or bash (containing a full load of
config variables), and just 3 or 4 steps of pipeline, this was enough to
run out of VM. Adding more swap resolved the issue. Back then, I did add
more swap "the hard way" by re-installing from floppies and choosing
more swap at the appropriate step. But nowadays, I just use dgesswein's
s4vol tool
Next challenge is to get gcc-3.3.6 (running on UnixPC) to compile a
program longer than a simple hello-world. It runs out of memory pretty
quickly, and as this is one process, swap does not help that use case.

>
> Jesse
>

Regards,

Alain

[Peter Schmidt]

The ported memory architecture is a sad waste of a 32 bit microprocessor's potential. Only machines I've ever had where physical memory was greater than per process virtual memory.

[J Booth]

On Monday, October 24, 2022 at 7:19:38 AM UTC-4, Peter Schmidt wrote:
> The ported memory architecture is a sad waste of a 32 bit microprocessor's potential. Only machines I've ever had where physical memory was greater than per process virtual memory.

Yeah does seem like a waste to have a full 4MB physical memory on this machine.

This mapmem tool will give more info on how memory is being used:
http://www.unixpc.org/3b1/osu/misc/mapmem.cpio.gz

[J Booth]

On Monday, October 24, 2022 at 2:40:22 AM UTC-4, Alain Knaff wrote:

> > Kernel VM = 38_00_00 – 3F_FF_FF (512 KB)
> That bit I didn't know yet. As far as I know, Linux does the same (i.e.
> the kernel exists in the application's address space, even though
> inaccessible to the application, obviously)

Ahh ok, I wasn't sure what that Kernel VM address space was.

> Actually, the problem that I used to have was running "many" *processes*
> concurrently.
>

You've probably already found, but there is ktune which allows you to adjust some kernel parameters, such as nproc (number of processes, default: 100), e.g. ktune nproc=200

But it sounds like that's not the issue, just purely running out of memory.

> Adding more swap resolved the issue.

Interesting to hear that helped. I would have thought the default 4000 KB would have sufficed, unless yours was smaller than that.

> Next challenge is to get gcc-3.3.6 (running on UnixPC) to compile a
> program longer than a simple hello-world. It runs out of memory pretty
> quickly, and as this is one process, swap does not help that use case.
>

Good luck! I really appreciate the work you and Mike have done on the cross compiler. A great resource for anyone interested in UNIX PC.

[J Booth]

> You've probably already found, but there is ktune which allows you to adjust some kernel parameters, such as nproc (number of processes, default: 100), e.g. ktune nproc=200
>

I just tried ktune nproc=120 and then rebooted which did indeed increase nproc to 120 (as reported by ktune -d). The naming in /usr/include/sys/tune.h with NPROCMAX = 100 made me wonder if you could go above 100, but the documentation on ktune says "up to system capacity" whatever that might be... The MAX values in tune.h appear to be the default values, which matches the ktune documentation.

> Interesting to hear that helped. I would have thought the default 4000 KB would have sufficed, unless yours was smaller than that.

Nevermind -- I just saw your other note about having pushed the swap partition 100+ MB, nice.

[Alain Knaff]

Thanks. This is indeed interesting, and should make debugging much easier.

With a 105M swap partition mapmem shows 25300 4K "blocks" available

=> so it does look like it makes use of more than 64M of swap after all.
And indeed, I could launch 40 of my 2.5M processes this time.

Not sure what happened with my earlier test, where I seemed to be
limited to 23 processes.

Regards,

Alain

[Alain Knaff]

On 24/10/2022 20:27, J Booth wrote:
> On Monday, October 24, 2022 at 2:40:22 AM UTC-4, Alain Knaff wrote:
>>> Kernel VM = 38_00_00 – 3F_FF_FF (512 KB)
>> That bit I didn't know yet. As far as I know, Linux does the same (i.e.
>> the kernel exists in the application's address space, even though
>> inaccessible to the application, obviously)
>
> Ahh ok, I wasn't sure what that Kernel VM address space was.

However, at a second glance, I'm wondering about this now. In-Kernel
addresses are much lower that 0x380000. Indeed everything seems to be
below 0x20500 (you can apply nm to /UNIX3.51m to check). Unless it's
about kmalloc'ed dynamic memory.

>
>> Actually, the problem that I used to have was running "many" *processes*
>> concurrently.
>>
>
> You've probably already found, but there is ktune which allows you to adjust some kernel parameters, such as nproc (number of processes, default: 100), e.g. ktune nproc=200

Actually, I haven't known about this yet, thanks for the pointer. So far
I had resorted to using a hex editor for this kind of stuff :-)

ktune sounds easier, at least for those variables that are available
through it.


swapdev does not seem to be available unfortunately. However, with the
hexeditor, I managed to set the swapdev variable at 0x1ed64 (0x1ee0c in
the UNIX3.51m file) to 0x11, and could swap from the second disk that
way. Given that on a second disk there are no longer any concerns of
pushing the root partition with the kernel image too far out, I managed
to set up a swap partition of 184M (beyond, the disk would be full).

mapmem did indeed show the full 44334 blocks, and I managed to spawn 69
instances of my 2.5M test process :-)

>
> But it sounds like that's not the issue, just purely running out of memory.

Indeed.

>
>> Adding more swap resolved the issue.
> Interesting to hear that helped. I would have thought the default 4000 KB would have sufficed, unless yours was smaller than that.
>
>> Next challenge is to get gcc-3.3.6 (running on UnixPC) to compile a
>> program longer than a simple hello-world. It runs out of memory pretty
>> quickly, and as this is one process, swap does not help that use case.
>>
> Good luck!

Thanks :-)

> I really appreciate the work you and Mike have done on the cross compiler. A great resource for anyone interested in UNIX PC.
>

It's a pleasure to hear that you like it, thanks :-)

Regards,

Alain