exceeding 64k

[muta...@gmail.com]

With recent (approx a year ago) enhancements to
SubC, the text segment exceeded 64k, and since
SubC only supports small memory model for DOS,
that meant that DOS stopped being self-hosting.

The solution is to move to medium/large, but I'm
wondering whether anything other than the 8086
supported more than 64k of code while still using
16-bit registers.

I know some things like the Commodore 128
allow memory to be banked out to access more
than 64k, but correct me if I'm wrong, but I assume
that means getting the C programmer to add
bank-switching code to their program, instead of it
being transparent like it is on the 8086 (the compiler
takes care of generating segment loads).

And I know that "overlays" exist, but once again it is
something the programmer needs to be involved in
(I think - I don't know much about overlays).

For a non-kludged solution, is the 8086 all there is?

Thanks. Paul.

[Alexei A. Frounze]

One typical solution is to split the compiler into several
stages, each executing the other as a separate program
with the communication between them done via files
containing intermediate results.
E.g. one stage preprocesses the input, another is
the front end that parses declarations and statements
and produces intermediate representations of those
and then there's the back end that takes that and
turns it into machine code.
Try to split the compiler into more stages.

This is very much like overlays, but it uses only conventional
primitives (file I/O, system()/exec*()/spawn*()) available in
any semi-decent OS.

Alex

[muta...@gmail.com]

On Saturday, April 23, 2022 at 2:25:13 AM UTC+10, Alexei A. Frounze wrote:

> One typical solution is to split the compiler into several
> stages, each executing the other as a separate program
> with the communication between them done via files
> containing intermediate results.

Ok, but that requires the programmer to change the
way they program, because the code they would
naturally write exceeds 64k of machine code.

My question is - is the 8086 the only processor that
still has 16-bit registers, but allows the naturally-written
program with more than 64k of machine code, to
be built by a compiler?

Or is there some other technique that would work on
the Commodore 128 or some other machine that
allowed more than 64k of memory with only 16-bit
or 8-bit registers?

Thanks. Paul.

[Alexei A. Frounze]

On Friday, April 22, 2022 at 1:41:02 PM UTC-7, muta...@gmail.com wrote:
> On Saturday, April 23, 2022 at 2:25:13 AM UTC+10, Alexei A. Frounze wrote:
>
> > One typical solution is to split the compiler into several
> > stages, each executing the other as a separate program
> > with the communication between them done via files
> > containing intermediate results.
> Ok, but that requires the programmer to change the
> way they program, because the code they would
> naturally write exceeds 64k of machine code.

Correct.

> My question is - is the 8086 the only processor that
> still has 16-bit registers, but allows the naturally-written
> program with more than 64k of machine code, to
> be built by a compiler?
>
> Or is there some other technique that would work on
> the Commodore 128 or some other machine that
> allowed more than 64k of memory with only 16-bit
> or 8-bit registers?

Dunno. There's yet another approach, which I did on a MIPS32
microcontroller with just 128KB of RAM. I made a small
MIPS32 emulator that demand-loads portions of the large
code section, caches them, and emulates the cached code.
Virtual memory it is.
This makes things pretty slow, but not too slow if your CPU
runs at about 100 MHz (the microcontroller ran at 80 MHz)
and you do those things relatively infrequently (which compiling is).

In principle, you could introduce a wider "virtual" IP/PC (or
introduce fake 8086-like segmentation) to extend the allowed
code size beyond 64 KB, and then do something like I described
above.

Btw, Smaller C's huge memory model supports code size
larger than 64 KB, the only restriction being that individual
subroutines each fit into 64 KB. With this every C function
resides in a separate segment and is far-callable.
You could turn SubC's pointers into far pointers (and probably
make plain ints 32-bit as well as I think the two types are
expected to be the same size just as in Smaller C) and solve
the problem similarly. IOW, just use far pointers everywhere
while restricting individual object and function size to 64 KB
(ditto for arrays).

Alex

[anti...@math.uni.wroc.pl]

There were word addressable machines. You still have
64k limit, but in 16-bit words, so effectively twice
as large program. There are machines with separate
data and code address spaces. For example Atmel
processors have 8 bit registers, 64k bytes data space
and 64k words program space. So maximal program
effectively can use 192 kB memory (128kB code and
64kB data).

MSP-430X is cheating: while normal data instructions
treat registers as 16 bit there are extra four bits
available to special instructions and addressing.
So it looks mostly like 16-bit processor but really
is 20-bit.

STM8 has (single) 8-bit data register, 8bit conditon
code register, 2 16-bit index registers, logically
16-bit stack pointer and logically 24-bit program
counter. I write "logically" above because in
practice stack pointer and program counter need
only bits corresponding to existing memory. STM8
uses indirect addressing which can use up to 3
bytes as address, so can address 16 MB of memory.

You probably will disqualify STM8 as despite 16-bit
registers it allows 24-bit linear address. But note
that STM8 is logically rather similar to 6502
(which was used in Commodore 64). My guess is that
STM8 could be implemented with complexity similar
to Z80 (Z80 was approximatly twice as large as 6502)
and much smaller than 8086.

Note: STM8 is relatively modern design (probably
around 2006), but historically there where several
processors similar to 6502 but having bigger (
but within 16-bit limit) registers. I do not know
if any of them allowed addressing more than 65kB of
memory, but that would be natural extention.

OTOH current market for small, but bigger than 64 kB
processors is dominated by ARM, with MIPS and RISCV
as main competitors. Each of the above has 32-bit
registers.

--
Waldek Hebisch