Self-decompressing COM files?

[Steve Nickolas]

OK, this is going to get weird. The tl;dr of it is that I'm wondering if
there's a tool to create self-decompressing (like what upx does) binaries
for CP/M-80 (i.e., a CP/M executable cruncher). I'll get into the gories.

I've been working on porting some games to a system (Nabu) that runs CP/M
3.1 - the call at 5 is to $BA06. (There are two other unofficial ports of
CP/M, both of 2.2, and one of them I'll probably need to figure out the
equivalent value; the other I don't care about.)

For one of them, I've been working on trying to expand it from the source
version's (MSX) 76 levels to the full 150. I don't think there's memory
to have all 150 and still enough room to load the program from CP/M. (The
level storage format is kind-of sparse and wasteful, but easy to figure
out. Basically, it's BCD.) I decided to include 120 levels.

But I was wondering about whether, using an executable cruncher, I could
try to include the whole set of 150.

(The other method - to store them in a file and load them on the fly, like
the original game on the Apple ][ - would require a lot more refactoring
of the code to move where scratch RAM is located.)

-uso.

[Jacob Nevins]

Steve Nickolas <usot...@buric.co> writes:
>OK, this is going to get weird. The tl;dr of it is that I'm wondering if
>there's a tool to create self-decompressing (like what upx does) binaries
>for CP/M-80 (i.e., a CP/M executable cruncher).

PopCom! is one such.

Apparently I've lost track of where CP/M archive mirrors are these days,
but you can download a copy, and documentation in Japanese (looks like
Shift-JIS encoding), from
https://www.asahi-net.or.jp/~am9y-mn/fswlist.htm

>I've been working on porting some games to a system (Nabu) that runs CP/M
>3.1 - the call at 5 is to $BA06. (There are two other unofficial ports of
>CP/M, both of 2.2, and one of them I'll probably need to figure out the
>equivalent value; the other I don't care about.)
>
>For one of them, I've been working on trying to expand it from the source
>version's (MSX) 76 levels to the full 150. I don't think there's memory
>to have all 150 and still enough room to load the program from CP/M.

[...]

>But I was wondering about whether, using an executable cruncher, I could
>try to include the whole set of 150.

I'm not sure how much PopCom will help you increase your maximum program
size on your CP/M 3.1 system, though?
(Disclaimer: I only ever used a banked 3.1 system with loads of memory,
so I had a massive TPA and never had to understand any of this stuff
properly. So I may have misunderstood. Also I haven't thought about any
of this for a very long time.)

Machine-translating the PopCom docs:

"When a programme compressed with PopCom! is executed, it is not limited
by the TPA size when PopCom! is executed."

Looking at this CP/M 2 description
http://www.gaby.de/cpm/manuals/archive/cpm22htm/ch5.htm
I can see how PopCom can increase effective program size, because the
definition of TPA (limiting the loaded program's size) doesn't include
the CCP, but your program can re-use the CCP's space once it's loaded.
So I guess PopCom lets you increase your program size by the size of the
CCP (however much that is), since PopCom can uncompress into space that
was occupied by the CCP during loading.

However, on CP/M 3, I think that restriction on loaded program size
is reduced to the LOADER (not the whole CCP), which is documented as
being "three pages long" (= 384 bytes IIRC).
(http://www.cpm.z80.de/manuals/cpm3-pgr.pdf pp17-8)
So I think PopCom can only increase your maximum program size by that
much on CP/M 3? (And I don't know how much of that PopCom's own
overheads might eat.)

Other tidbits machine-translated from POPCOM.DOC, in case they avoid
wasting your time:
"PopCom! can be used with CP/M of 47K or more (TPA 42K or more).
However, a CPU equivalent to Z80 or higher is required."
"It also uses only the minimum amount of memory required for
decompression, so if the source programme is one that does not destroy
CCP, the compressed programme will also work without destroying CCP in
most cases."

It looks like your TPA is 46.25 kbyte or thereabouts, so I guess PopCom
will work on your system.

[Steve Nickolas]

On Wed, 3 Jan 2024, Jacob Nevins wrote:

> Steve Nickolas <usot...@buric.co> writes:
>> OK, this is going to get weird. The tl;dr of it is that I'm wondering if
>> there's a tool to create self-decompressing (like what upx does) binaries
>> for CP/M-80 (i.e., a CP/M executable cruncher).
>
> PopCom! is one such.
>
> Apparently I've lost track of where CP/M archive mirrors are these days,
> but you can download a copy, and documentation in Japanese (looks like
> Shift-JIS encoding), from
> https://www.asahi-net.or.jp/~am9y-mn/fswlist.htm

Yeah, it's SJIS. My Japanese isn't great but I do speak it a little... ;p

>> I've been working on porting some games to a system (Nabu) that runs CP/M
>> 3.1 - the call at 5 is to $BA06. (There are two other unofficial ports of
>> CP/M, both of 2.2, and one of them I'll probably need to figure out the
>> equivalent value; the other I don't care about.)
>>
>> For one of them, I've been working on trying to expand it from the source
>> version's (MSX) 76 levels to the full 150. I don't think there's memory
>> to have all 150 and still enough room to load the program from CP/M.
> [...]
>> But I was wondering about whether, using an executable cruncher, I could
>> try to include the whole set of 150.
>
> I'm not sure how much PopCom will help you increase your maximum program
> size on your CP/M 3.1 system, though?
> (Disclaimer: I only ever used a banked 3.1 system with loads of memory,
> so I had a massive TPA and never had to understand any of this stuff
> properly. So I may have misunderstood. Also I haven't thought about any
> of this for a very long time.)

I'm actually kicking CP/M out after I load the program. But the program
won't load right if it's big enough to step on the BDOS in the process.

> Other tidbits machine-translated from POPCOM.DOC, in case they avoid
> wasting your time:
> "PopCom! can be used with CP/M of 47K or more (TPA 42K or more).
> However, a CPU equivalent to Z80 or higher is required."
> "It also uses only the minimum amount of memory required for
> decompression, so if the source programme is one that does not destroy
> CCP, the compressed programme will also work without destroying CCP in
> most cases."
>
> It looks like your TPA is 46.25 kbyte or thereabouts, so I guess PopCom
> will work on your system.

Nod. It's an unbanked 64K system.

I'm just using CP/M as a loader, so if the decompressor stomps on CP/M, I
don't really care. Might have to make it DI first though, if it doesn't
already.

-uso.

[Steve Nickolas]

Just tested it - had to go into an actual CP/M environment, but that was
fine - crunched the game I was working on (Lode Runner btw) and tried it,
and it worked.

-uso.

[Jacob Nevins]

\o/

Re running in actual CP/M environment: for what it's worth, I find that
utilities like PMEXT run fine under zxcc, which means I can conveniently
run them in my host (Linux) environment.
https://www.seasip.info/Unix/Zxcc/index.html
POPCOM may work this way too. (In a quick test, it seemed to basically
work, although there was a "cpmredir: Corrupt FCB" glitch on exit.)

[Steve Nickolas]

On Wed, 3 Jan 2024, Jacob Nevins wrote:

Perhaps. I tried the Lopushinsky emulators in DOSBOX. (I also run on a
Linux host.)

-uso.

[Tony Nicholson]

On 4/01/2024 3:51 am, Jacob Nevins wrote:
> Steve Nickolas <usot...@buric.co> writes:

> Re running in actual CP/M environment: for what it's worth, I find that
> utilities like PMEXT run fine under zxcc, which means I can conveniently
> run them in my host (Linux) environment.
> https://www.seasip.info/Unix/Zxcc/index.html
> POPCOM may work this way too. (In a quick test, it seemed to basically
> work, although there was a "cpmredir: Corrupt FCB" glitch on exit.)

I've collected some updates and patches for John Elliot's ZXCC which
you'll find on GitHub at

https://github.com/agn453/ZXCC

Be sure to read the README for the complete list of changes.

Tony