extending parameters

[muta...@gmail.com]

Looking at the code for PDPCLIB and PDOS,
it seems that when a program does a:

system("fred abc");

that the maximum length of "abc" is governed
by what can fit in the PSP, starting at 0x80,
which means from 128 bytes you need to
subtract 1 byte for the length indicator itself,
and 1 byte for a compulsory '\r', so there are
126 bytes for the caller, and that is what is put
in the first byte.

I'd like to support more than that. Actually the
limit of a single byte is enough for me at the
moment, ie 254, but maybe it should be extended
even further, ie if someone puts 255 in that spot,
it means that the real length is in cmd[1] and
cmd[2]. And if both of those are x'FF' then the
real length is in cmd[3..6].

Anyway, if the length is anything above 126, then
how about calling a new INT 21H function to
retrieve an "extended parameter"?

I see there is already an extension for PDOS/386,
and it is isolated in the AH=F6H for PDOS/x86
extensions. Maybe that's where it should stay.

Any suggestions on the best way forward?

Is there any prior art for this? Maybe I can just
copy an existing solution. Unless it has some
disadvantage.

Thanks. Paul.


/* F6,3F - Get Command Line String For The Current Process */
char *PosGetCommandLine(void)
{
union REGS regsin;
union REGS regsout;

regsin.h.ah = 0xF6;
regsin.h.al = 0x3F;

int86(0x21, &regsin, &regsout);
return ((char *)(regsout.d.eax));
}

[JJ]

On Tue, 27 Apr 2021 01:35:38 -0700 (PDT), muta...@gmail.com wrote:
>
> I see there is already an extension for PDOS/386,
> and it is isolated in the AH=F6H for PDOS/x86
> extensions. Maybe that's where it should stay.
>
> Any suggestions on the best way forward?
>
> Is there any prior art for this? Maybe I can just
> copy an existing solution. Unless it has some
> disadvantage.

MS-DOS 7.x already specify a standard for this via CMDLINE environment
variable. It's also supported by the already popular 4DOS software. So, it
should be supported by all new DOS compatible OS projects.

[Alexei A. Frounze]

On Tuesday, April 27, 2021 at 1:35:39 AM UTC-7, muta...@gmail.com wrote:
> Looking at the code for PDPCLIB and PDOS,
> it seems that when a program does a:
>
> system("fred abc");
>
> that the maximum length of "abc" is governed
> by what can fit in the PSP, starting at 0x80,
> which means from 128 bytes you need to
> subtract 1 byte for the length indicator itself,
> and 1 byte for a compulsory '\r', so there are
> 126 bytes for the caller, and that is what is put
> in the first byte.
>
> I'd like to support more than that. Actually the
> limit of a single byte is enough for me at the
> moment, ie 254, but maybe it should be extended
> even further, ie if someone puts 255 in that spot,
> it means that the real length is in cmd[1] and
> cmd[2]. And if both of those are x'FF' then the
> real length is in cmd[3..6].

...

> Any suggestions on the best way forward?

DJGPP had a few ways of doing it, look it up.
Smaller C uses the one where @foo on the command line
expands into the contents of file foo.

Alex

[muta...@gmail.com]

On Tuesday, April 27, 2021 at 9:23:10 PM UTC+10, JJ wrote:

> MS-DOS 7.x already specify a standard for this via CMDLINE environment
> variable. It's also supported by the already popular 4DOS software. So, it
> should be supported by all new DOS compatible OS projects.

Ok, thanks. I found that referenced here:

https://en.wikipedia.org/wiki/Environment_variable

But I haven't yet found the rules for setting the PSP.

BFN. Paul.

[JJ]

On Tue, 27 Apr 2021 05:52:40 -0700 (PDT), muta...@gmail.com wrote:
>
> Ok, thanks. I found that referenced here:
>
> https://en.wikipedia.org/wiki/Environment_variable
>
> But I haven't yet found the rules for setting the PSP.
>
> BFN. Paul.

Ralf Browns's PSP table described it in details.

[muta...@gmail.com]

On Wednesday, April 28, 2021 at 4:20:15 PM UTC+10, JJ wrote:

> > But I haven't yet found the rules for setting the PSP.

> Ralf Browns's PSP table described it in details.

Ok, now I know the format:

http://www.ctyme.com/intr/rb-2682.htm

But who is responsible for setting this? I assume there
is no new INT 21H call to replace AH=4BH. So it falls
to the system() implementer to set the parmblock
according to the above rules, and also set the CMDLINE
environment variable.

I noticed that there doesn't seem to be a facility to
set the environment variable. I know how to read
environment variables (the pointer is provided in
the PSP). Looking at this:

http://www.ctyme.com/intr/rb-2939.htm

I'm guessing system() needs to allocate a new area
of memory for environment variables, copy the old
environments, add/replace CMDLINE, and put the
segment into the parameter block.

BFN. Paul.

[JJ]

On Wed, 28 Apr 2021 01:05:37 -0700 (PDT), muta...@gmail.com wrote:
>
> But who is responsible for setting this?

It's COMMAND.COM, actually - and unfortunately.

Don't know why it's not implemented within the DOS interrupt service itself.
Probably because it's problematic enough, or it was a rushed solution.

> I'm guessing system() needs to allocate a new area
> of memory for environment variables, copy the old
> environments, add/replace CMDLINE, and put the
> segment into the parameter block.

Yes.

[muta...@gmail.com]

On Thursday, April 29, 2021 at 8:09:44 PM UTC+10, JJ wrote:

> > But who is responsible for setting this?

> It's COMMAND.COM, actually - and unfortunately.
>
> Don't know why it's not implemented within the DOS interrupt service itself.
> Probably because it's problematic enough, or it was a rushed solution.

Could you please explain what you think would
have been a better design?

Thanks. Paul.

[JJ]

On Thu, 29 Apr 2021 08:38:46 -0700 (PDT), muta...@gmail.com wrote:
>
> Could you please explain what you think would
> have been a better design?

I'd keep the CMDLINE environment variable as the full command line storage.
Mainly because it's MS-DOS v7.x compatible, and it can also be used in
earlier and other DOS versions to run programs which support the variable.

That will make other programs which support the variable, easy to port to
other DOS-like OSes, because the command line storage is the same.

But implement the code which prepare the CMDLINE variable in the DOS
interrupt service itself (i.e. in Int-214B), rather than the command
interpreter, so that it's OS native. Otherwise, if the user decide to use
other command interpreter which doesn't support the CMDLINE environment
variable. Long command line won't be available for programs which already
support the CMDLINE variable.

[muta...@gmail.com]

On Friday, April 30, 2021 at 7:58:20 PM UTC+10, JJ wrote:

> But implement the code which prepare the CMDLINE variable in the DOS
> interrupt service itself (i.e. in Int-214B), rather than the command
> interpreter, so that it's OS native. Otherwise, if the user decide to use
> other command interpreter which doesn't support the CMDLINE environment
> variable. Long command line won't be available for programs which already
> support the CMDLINE variable.

The way things currently work, one long-parm-aware
program can invoke another long-parm-aware program
regardless of MSDOS version.

If the change was made in the MSDOS interrupt itself,
the calling program's system() would STILL need to
do something special, to invoke the new functionality
of Int-214B, and the called program would STILL need
to do something special to receive it, AND you need
to be on a particular version of MSDOS that knows
about this new facility.

How would this be better? Or am I missing something?

BFN. Paul.

[JJ]

On Fri, 30 Apr 2021 05:34:54 -0700 (PDT), muta...@gmail.com wrote:
>
> If the change was made in the MSDOS interrupt itself,
> the calling program's system() would STILL need to
> do something special, to invoke the new functionality
> of Int-214B, and the called program would STILL need
> to do something special to receive it, AND you need
> to be on a particular version of MSDOS that knows
> about this new facility.
>
> How would this be better? Or am I missing something?

Aw, crap. I overlooked the fact that Int-214B separates between the program
name and (the limited) command tail.

I'm beginning to believe that Microsoft's implementation is actually a
rushed solution. They should really have added an additional field in the
Parameter Block for the full command line or extended command tail. Albeit
requires that other programs handle program execution differently.

[muta...@gmail.com]

Once again, an additional field in the parameter block
would require all 3 components (caller, OS and called)
to be aware of the additional field. How is that better
than CMDLINE?

Regardless, for PDOS/386 there is an opportunity for a
fresh start. The "environment" field, a 2-byte segment,
cannot be used by hook or crook, and needs to be a
4-byte pointer instead. And everything necessarily needs
to be recompiled for 80386 anyway, so there is no harm
in doing it differently.

But the objective is still to make it easy for programmers
to target both 16-bit MSDOS and a theoretical 32-bit
MSDOS. Here is what I currently have:

typedef struct {
#ifdef __32BIT__
char *env;
#else
short env;
#endif
unsigned char *cmdtail;
char *fcb1;
char *fcb2;
} POSEXEC_PARMBLOCK;

void PosExec(char *prog, POSEXEC_PARMBLOCK *parmblock)

{
union REGS regsin;
union REGS regsout;

struct SREGS sregs;

regsin.h.ah = 0x4b;
regsin.h.al = 0;
#ifdef __32BIT__
regsin.d.edx = (int)prog;
regsin.d.ebx = (int)parmblock;
#else
sregs.ds = FP_SEG(prog);
regsin.x.dx = FP_OFF(prog);
sregs.es = FP_SEG(parmblock);
regsin.x.bx = FP_OFF(parmblock);
#endif
int86x(0x21, &regsin, &regsout, &sregs);
return;
}

I'm thinking 32-bit should drop the fcb1/2, as they seem to
be outdated, and specify that the cmdtail doesn't have a
length in the first byte, and just relies on NUL-termination.
And no '\r' either.

BFN. Paul.

[R.Wieser]

Muta,

> Once again, an additional field in the parameter
> block would require all 3 components (caller, OS
> and called)

Why do you think so ? AFAIKS the *pointer* field doesn't care how long the
pointed-to string is ...

Add a "give me a pointer to the extended command-tail string" for the called
program (the PSP can only hold 126 chars so it needs to be stored somewhere
else) and it should be enough - if it fails than there is no support. It
can be used by the caller, to determine if it should apply the CMDLINE
environment variable hack, as well as by the callee, for the same reason.

> I'm thinking 32-bit should drop the fcb1/2, as they seem
> to be outdated, and specify that the cmdtail doesn't have
> a length in the first byte, and just relies on NUL-termination.
> And no '\r' either.

As long as you're busy upgrading stuff, do consider changing 2109 to a
zero-terminated string too.

Regards,
Rudy Wieser

[R.Wieser]

> Add a "give me a pointer to the extended command-tail string"

Forgot to include the word INT at the the above end.

Regards,
Rudy Wieser

[muta...@gmail.com]

On Sunday, May 2, 2021 at 1:17:04 AM UTC+10, R.Wieser wrote:

Hi Rudy. Are you going to answer my question about
the technical barrier to ctrl-c checking for programs
that loop without doing output? I'd really like to know
that.

> > Once again, an additional field in the parameter
> > block would require all 3 components (caller, OS
> > and called)

> Why do you think so ?

For the same reason I gave before?

> AFAIKS the *pointer* field doesn't care how long the
> pointed-to string is ...

Sure.

> Add a "give me a pointer to the extended command-tail string (INT)" for the called

> program (the PSP can only hold 126 chars so it needs to be stored somewhere
> else) and it should be enough - if it fails than there is no support. It
> can be used by the caller, to determine if it should apply the CMDLINE
> environment variable hack, as well as by the callee, for the same reason.

So both caller and called require two techniques
instead of one, and the OS indeed can have 0 or 1.

This can in fact be added to the existing system
at any time if you think it is a good idea.

> > I'm thinking 32-bit should drop the fcb1/2, as they seem
> > to be outdated, and specify that the cmdtail doesn't have
> > a length in the first byte, and just relies on NUL-termination.
> > And no '\r' either.

> As long as you're busy upgrading stuff, do consider changing 2109 to a
> zero-terminated string too.

Currently I can go:

PosDisplayString("Hello$");

and it will work on both 16-bit and 32-bit MSDOS.

What is the advantage of forcing programmers to code:

#ifdef __32BIT__
PosDisplayString("Hello");
#else
PosDisplayString("Hello$");
#endif

?

More code to achieve what?

BFN. Paul.

[R.Wieser]

Muta,

> Hi Rudy. Are you going to answer my question about
> the technical barrier to ctrl-c checking for programs
> that loop without doing output? I'd really like to know
> that.

Done.

>> > Once again, an additional field in the parameter
>> > block would require all 3 components (caller, OS
>> > and called)
>
>> Why do you think so ?
>
> For the same reason I gave before?

No, not who should be aware of that extra field, but why that extra field is
needed in the first place.

> So both caller and called require two techniques
> instead of one, and the OS indeed can have 0 or 1.

Required ? No. A good idea to have them both *if* you want to have your
program to also run on DOSes which do not support the extended command-tail
? Yes.

Did I already mention that I regard the CMDLINE method to be an hack (and an
ugly one to boot) ? And have you ever thought about how clumsy changing
the environment and retrieving from it is for a to-be-executed program is in
a plain DOS version ?

Than again, most of that is hidden from you by the C{something} environment
you're programming in ...

>> As long as you're busy upgrading stuff, do consider changing 2109 to a
>> zero-terminated string too.
>
> Currently I can go:
>
> PosDisplayString("Hello$");
>
> and it will work on both 16-bit and 32-bit MSDOS.

Yeah, it does. You know what /won't/ work ? Displaying the current, or
your proposed extended command-tail that way.

But lets put it differently : you're introducing a /third/ string form. Why
? And of you think that a zero-terminated string works better than why not
update that ancient '$' ended string too ? Better yet, change all three to
the same, zero-terminated form.

Than again, if you would do that you would lose compatibility with all of
the other DOSes ... so maybe don't.

> More code to achieve what?

Nope, less code. One set of functions that will handle the single string
form, instead of having three different ones. Also, no permutations needed
to combine/append one string form with/to another.

Regards,
Rudy Wieser

[muta...@gmail.com]

On Sunday, May 2, 2021 at 7:00:00 AM UTC+10, R.Wieser wrote:

> Did I already mention that I regard the CMDLINE method to be an hack (and an
> ugly one to boot) ? And have you ever thought about how clumsy changing
> the environment and retrieving from it is for a to-be-executed program is in
> a plain DOS version ?

I think we need to distinguish between:

1. How MSDOS would ideally have been written starting with version 1.0.

2. How to maintain compatibility with the CMDLINE solution
introduced in MSDOS 7.0.

3. What alternative could have been chosen knowing the
"mistakes" of 1.0 and before the 7.0 solution has been
set in stone.

> > PosDisplayString("Hello$");
> >
> > and it will work on both 16-bit and 32-bit MSDOS.

> Yeah, it does. You know what /won't/ work ? Displaying the current, or
> your proposed extended command-tail that way.

Sure. This is not the function for that. There isn't any
choice but to use an alternative. Is that a problem?
You can't go changing the MSDOS API, at least not
for the 16-bit version, at least not if you want to
maintain compatibility with any version of genuine
MSDOS at all. Or any existing version of Freedos for
that matter.

> But lets put it differently : you're introducing a /third/ string form. Why
> ?

Because the '\r' is useless either way and needs to
be removed. I'm only talking about the command
line processing, which is necessarily different
between 16-bit and 32-bit anyway.

There's no need for the CMDLINE kludge either in the
32-bit version. The PSP needs to be changed, and
everything needs to be recompiled anyway.

If we (as I do) theorize that Microsoft had created a
nice C API like the Pos* functions, instead of only
providing an assembler interface, what would that
API actually look like?

The Pos* functions:

https://sourceforge.net/p/pdos/gitcode/ci/master/tree/src/pos.c

are my guess at answering that question, but I'm open
to suggestions.

> And of you think that a zero-terminated string works better than why not
> update that ancient '$' ended string too ? Better yet, change all three to
> the same, zero-terminated form.
>
> Than again, if you would do that you would lose compatibility with all of
> the other DOSes ... so maybe don't.

Correct.

I guess we need to ask what a sensible objective is in
the first place. I just have a vague idea of "32-bit MSDOS".

And part of that is an expectation that you can display
a "$" terminated string on both real 16-bit MSDOS and
newly-invented 32-bit MSDOS.

> > More code to achieve what?

> Nope, less code. One set of functions that will handle the single string
> form, instead of having three different ones. Also, no permutations needed
> to combine/append one string form with/to another.

I am not providing an API to print a NUL-terminated
string. I'm just setting rules for INT 21H 4BH.

At the moment, anyway.

BFN. Paul.

[R.Wieser]

Muta,

> I think we need to distinguish between:
>
> 1. How MSDOS would ideally have been written starting with version 1.0.

...

> 3. What alternative could have been chosen knowing the
> "mistakes" of 1.0 and before the 7.0 solution has been
> set in stone.

No, definitily not. I have no wish to talk that way about stuff I cannot
alter. Also, it would be bashing those thanwhile choices with our current
knowledge. Thats simply not fair. Also, in your case its would be tainted
with what you think your own OS should be capable of / looking like.

> 2. How to maintain compatibility with the CMDLINE
> solution introduced in MSDOS 7.0.

Can you come up with a way to make a program incompatible with it ? If
not than that requirement is moot.

> Sure. This is not the function for that. There isn't any
> choice but to use an alternative. Is that a problem?

Lol. *you* are introducing a new sting type, and than claim you have no
choice to use another function ? What about *NOT* introducing a new
string type ?

> Because the '\r' is useless either way and needs
> to be removed.

The same goes for that '$' terminating a string. Your point ?

But as I already mentioned, such a change (for either or both of the above
strings) would have dire consequences. Which might easily be prohibitonary.

> There's no need for the CMDLINE kludge either in
> the 32-bit version.

Thanks. That means I can just ignore any references you make to the 32-bit
version.

> If we (as I do) theorize that Microsoft had created
> a nice C API like the Pos* functions, instead of only
> providing an assembler interface, what would that
> API actually look like?

Nope, not going to engage in that. For several reasons.

One question though : whats, according to you, the difference between the
above "C API" and "assembler interface" ?

> I guess we need to ask what a sensible objective is in
> the first place. I just have a vague idea of "32-bit MSDOS".

And as that one, as you have just mentioned yourself, doesn't need the
CMDLINE kludge it means that our current conversation is a waste of our
time. :-(

> I am not providing an API to print a NUL-terminated
> string.

Which makes the problem bigger. Introducing something new, but not adding
support for it.

> I'm just setting rules for INT 21H 4BH.

And I've mentioned (and explained!) to you that changes to support an
extended command-tail are not needed . If you think otherwise I would like
to hear why.

Regards,
Rudy Wieser

[JJ]

On Sat, 1 May 2021 06:41:26 -0700 (PDT), muta...@gmail.com wrote:
>
> Once again, an additional field in the parameter block
> would require all 3 components (caller, OS and called)
> to be aware of the additional field.

Yes, I'm aware of that now.

> How is that better than CMDLINE?

It's not. That isn't the point. It's a better way (than what MS-DOS 7.x) to
implement the code which provide CMDLINE. So that it doesn't depend on the
command interpreter to do it. It also off-load the code to the DOS kernel,
making programs a bit smaller. Yes, it does require the 3 components. That
can not be helped.

> Regardless, for PDOS/386 there is an opportunity for a
> fresh start. The "environment" field, a 2-byte segment,
> cannot be used by hook or crook, and needs to be a
> 4-byte pointer instead. And everything necessarily needs
> to be recompiled for 80386 anyway, so there is no harm
> in doing it differently.

If backward compatibility is not kept, it'll add more work to port existing
softwares.

> But the objective is still to make it easy for programmers
> to target both 16-bit MSDOS and a theoretical 32-bit
> MSDOS.

[snip]

> I'm thinking 32-bit should drop the fcb1/2, as they seem to
> be outdated, and specify that the cmdtail doesn't have a
> length in the first byte, and just relies on NUL-termination.
> And no '\r' either.

Yes, but that only applies to programs which are created from scratch. It
doesn't apply to existing programs.

[muta...@gmail.com]

On Sunday, May 2, 2021 at 9:57:06 PM UTC+10, JJ wrote:

> > How is that better than CMDLINE?

> It's not. That isn't the point. It's a better way (than what MS-DOS 7.x) to
> implement the code which provide CMDLINE. So that it doesn't depend on the
> command interpreter to do it. It also off-load the code to the DOS kernel,
> making programs a bit smaller. Yes, it does require the 3 components. That
> can not be helped.

It can be helped. You can avoid the middle component,
the OS, needing to be changed.

And the command interpreter is not necessarily on
either side of that exec() call either.

> > Regardless, for PDOS/386 there is an opportunity for a
> > fresh start. The "environment" field, a 2-byte segment,
> > cannot be used by hook or crook, and needs to be a
> > 4-byte pointer instead. And everything necessarily needs
> > to be recompiled for 80386 anyway, so there is no harm
> > in doing it differently.

> If backward compatibility is not kept, it'll add more work to port existing
> softwares.

Going from 8080 CP/M to 8086 MSDOS required conversion
of code at the assembler source level.

How do you propose doing a transition from 16-bit
MSDOS to 32-bit theoretical MSDOS?

Starting with applications that are calling exec()
in either assembler or C.

> > But the objective is still to make it easy for programmers
> > to target both 16-bit MSDOS and a theoretical 32-bit
> > MSDOS.
> [snip]
> > I'm thinking 32-bit should drop the fcb1/2, as they seem to
> > be outdated, and specify that the cmdtail doesn't have a
> > length in the first byte, and just relies on NUL-termination.
> > And no '\r' either.

> Yes, but that only applies to programs which are created from scratch. It
> doesn't apply to existing programs.

Ok, I forgot to mention something there.

I am talking about 16-bit MSDOS programs that were
written with a reasonably clean (I'm willing to negotiate,
nothing is set in stone) Pos* interface, and then
porting those to 32-bit MSDOS.

There is a reasonably clean PosExec() for 16-bit MSDOS,
but the caller and called will both need to adjust for
32-bit MSDOS. I don't see any reasonable way of avoiding
that, and I don't know what the proposed alternative is
anyway.

BFN. Paul.

[muta...@gmail.com]

On Sunday, May 2, 2021 at 7:52:14 PM UTC+10, R.Wieser wrote:

> > I think we need to distinguish between:
> >
> > 1. How MSDOS would ideally have been written starting with version 1.0.
> ...
> > 3. What alternative could have been chosen knowing the
> > "mistakes" of 1.0 and before the 7.0 solution has been
> > set in stone.

> No, definitily not. I have no wish to talk that way about stuff I cannot
> alter. Also, it would be bashing those thanwhile choices with our current
> knowledge. Thats simply not fair.

I don't consider it to be bashing. I'd like to understand what
the ideal solution would have been, understand what was
actually done, potentially bridge any gap, and write down
a set of principles if history ever repeats for some reason.

> > Sure. This is not the function for that. There isn't any
> > choice but to use an alternative. Is that a problem?

> Lol. *you* are introducing a new sting type, and than claim you have no
> choice to use another function ? What about *NOT* introducing a new
> string type ?

For 16-bit MSDOS you have to use cmdtail[0] to get
to the '\r' so that you can remove it.

For 32-bit MSDOS there is no cmdtail[0], and there's
no particular reason to insert a useless '\r' so that
someone can then remove it.

I don't see any issue.

But if you can explain why putting a useless '\r' so that
people have to remove it is actually helpful for
compatibility with something, I'm all ears.

> > Because the '\r' is useless either way and needs
> > to be removed.

> The same goes for that '$' terminating a string. Your point ?

$-terminated strings are a separate issue, and the
solution is identical for 16-bit and 32-bit MSDOS.
There is no issue whatsoever there.

The only issue is exec()/PosExec().

> But as I already mentioned, such a change (for either or both of the above
> strings) would have dire consequences. Which might easily be prohibitonary.

The parameter block to exec() in 32-bit has to change.
And recompilation has to be done too.

What specifically are you proposing for 32-bit programming
under 32-bit MSDOS?

> > If we (as I do) theorize that Microsoft had created
> > a nice C API like the Pos* functions, instead of only
> > providing an assembler interface, what would that
> > API actually look like?

> Nope, not going to engage in that. For several reasons.
>
> One question though : whats, according to you, the difference between the
> above "C API" and "assembler interface" ?

The C API merely requires a recompile for almost
all INT 21H services when switching from 16-bit
MSDOS to 32-bit MSDOS.

> > I am not providing an API to print a NUL-terminated
> > string.

> Which makes the problem bigger. Introducing something new, but not adding
> support for it.

It's all new. 32-bit exec() is all new. Necessarily.

> > I'm just setting rules for INT 21H 4BH.

> And I've mentioned (and explained!) to you that changes to support an
> extended command-tail are not needed . If you think otherwise I would like
> to hear why.

To start with, the parameter block to exec() has a
2-byte "segment" pointing to an environment block.

That can't work in a 32-bit flat address space. You
need a 4-byte flat pointer.

Maybe the source of confusion is that you think I am
talking about 32-bit applications calling real 16-bit
MSDOS. That is not the case. I am talking about
32-bit applications calling 32-bit MSDOS. Microsoft
never produced such a thing, but I have attempted to.
It's called PDOS/386.

BFN. Paul.

[JJ]

On Sun, 2 May 2021 06:46:17 -0700 (PDT), muta...@gmail.com wrote:
>
> It can be helped. You can avoid the middle component,
> the OS, needing to be changed.

If you don't want to offload the code to the kernel, then yes.

> And the command interpreter is not necessarily on
> either side of that exec() call either.

Not the command line interpreting part. That's irrelevant. It's the program
executing part.

> Going from 8080 CP/M to 8086 MSDOS required conversion
> of code at the assembler source level.

Yes, assuming that the program was created using pure Assembly.

> How do you propose doing a transition from 16-bit
> MSDOS to 32-bit theoretical MSDOS?
>
> Starting with applications that are calling exec()
> in either assembler or C.

...

> Ok, I forgot to mention something there.
>
> I am talking about 16-bit MSDOS programs that were
> written with a reasonably clean (I'm willing to negotiate,
> nothing is set in stone) Pos* interface, and then
> porting those to 32-bit MSDOS.
>
> There is a reasonably clean PosExec() for 16-bit MSDOS,
> but the caller and called will both need to adjust for
> 32-bit MSDOS. I don't see any reasonable way of avoiding
> that, and I don't know what the proposed alternative is
> anyway.

The pointer interpretation is entirely different for both platforms. You'll
juat have to treat pointers differently for each platform. e.g. for 16-bit
DOS, HiWord of the pointer is the Segment, and LoWord of the pointer is the
Offset; and for 32-bit DOS, HiWord is the upper 16 bits of the 32-bit flat
address, and LoWord is the lower 16 bits of the 32-bit flat address.
Assuming that pointer for the 32-bit DOS is a flat address.

[muta...@gmail.com]

On Tuesday, May 4, 2021 at 1:47:20 AM UTC+10, JJ wrote:

> > It can be helped. You can avoid the middle component,
> > the OS, needing to be changed.

> If you don't want to offload the code to the kernel, then yes.

The entirety of the code can't be offloaded from
either the caller or the callee. The only thing you
can entirely offload is the kernel, and that
actually maximizes the usefulness of it.

> > And the command interpreter is not necessarily on
> > either side of that exec() call either.

> Not the command line interpreting part. That's irrelevant. It's the program
> executing part.

I'm talking about the program execution.

PROGA can exec() PROGB without either the kernel
or command.com being involved.

> > There is a reasonably clean PosExec() for 16-bit MSDOS,
> > but the caller and called will both need to adjust for
> > 32-bit MSDOS. I don't see any reasonable way of avoiding
> > that, and I don't know what the proposed alternative is
> > anyway.

> The pointer interpretation is entirely different for both platforms. You'll
> juat have to treat pointers differently for each platform. e.g. for 16-bit
> DOS, HiWord of the pointer is the Segment, and LoWord of the pointer is the
> Offset; and for 32-bit DOS, HiWord is the upper 16 bits of the 32-bit flat
> address, and LoWord is the lower 16 bits of the 32-bit flat address.
> Assuming that pointer for the 32-bit DOS is a flat address.

There is no LoWord in the existing 16-bit MSDOS
exec() parameter block.

In any proposed solution of "how to write MSDOS
programs" I'm expecting to produce a 16-bit
executable that actually works on say real
MSDOS 3.2.

Starting from the above, how do you minimize the
amount of changes required to support a future
32-bit MSDOS? ie the year is 1986. MSDOS 3.2
was just released, and so was the 80386. Gates
just announced he will produce a 32-bit version
of MSDOS, but the exact details haven't been
announced yet.

You're a friend of Gates. You can influence him on
both the design of 32-bit MSDOS and on coding
standards for 16-bit MSDOS programming to make
the transition back and forth to 32-bit as easy as
possible. In any application there will be two distinct
executables created - one for 8086 16-bit MSDOS
and one for 80386 32-bit MSDOS.

Do you have a proposal?

Note that OS/2 1.0 had a "family API" for the same
executable to work on both MSDOS and OS/2, but
that's not a goal here.

BFN. Paul.

[JJ]

On Mon, 3 May 2021 09:11:22 -0700 (PDT), muta...@gmail.com wrote:
>
> The entirety of the code can't be offloaded from
> either the caller or the callee.

Not the entire code. Only part of it. The CMDLINE creation part.

> The only thing you
> can entirely offload is the kernel, and that
> actually maximizes the usefulness of it.

That would be no different that a bootable application which use its own
private OS. Basically an OS, but isolated.

> I'm talking about the program execution.
>
> PROGA can exec() PROGB without either the kernel
> or command.com being involved.

Offloading the CMDLINE creation code to the kernel which makes the kernel
part of the components, was just a suggestion. It's up to you to decide
whether to choose bigger kernel, or bigger applications; because the code
has to be placed somewhere.

> There is no LoWord in the existing 16-bit MSDOS
> exec() parameter block.

I was referring to your PDOS' pointer type in general.

> Starting from the above, how do you minimize the
> amount of changes required to support a future
> 32-bit MSDOS? ie the year is 1986. MSDOS 3.2
> was just released, and so was the 80386. Gates
> just announced he will produce a 32-bit version
> of MSDOS, but the exact details haven't been
> announced yet.

It's why I asked about the pointer type of your PDOS.

[muta...@gmail.com]

On Tuesday, May 4, 2021 at 9:50:58 PM UTC+10, JJ wrote:

> > The entirety of the code can't be offloaded from
> > either the caller or the callee.

> Not the entire code. Only part of it. The CMDLINE creation part.

Ok, true, you can do that. But if you do that, you can
avoid the CMDLINE altogether and just create a new
INT 21H function, can't you? And maybe that is the
superior solution, so that we can fix this "envseg"
issue in the parm block at the same time.

> > The only thing you
> > can entirely offload is the kernel, and that
> > actually maximizes the usefulness of it.

> That would be no different that a bootable application which use its own
> private OS. Basically an OS, but isolated.

Well I guess that's what it is. It's a very small OS that
knows about CMDLINE, even though MSDOS 3.2
does not.

Maybe at some level there is no distinction between
OS and application?

> > I'm talking about the program execution.
> >
> > PROGA can exec() PROGB without either the kernel
> > or command.com being involved.

I misspoke above. The kernel is still involved, it just
doesn't know about CMDLINE.

> Offloading the CMDLINE creation code to the kernel which makes the kernel
> part of the components, was just a suggestion. It's up to you to decide
> whether to choose bigger kernel, or bigger applications; because the code
> has to be placed somewhere.

I don't care about the size of either. What I care about is
compatibility.

> > There is no LoWord in the existing 16-bit MSDOS
> > exec() parameter block.

> I was referring to your PDOS' pointer type in general.

> > Starting from the above, how do you minimize the
> > amount of changes required to support a future
> > 32-bit MSDOS? ie the year is 1986. MSDOS 3.2
> > was just released, and so was the 80386. Gates
> > just announced he will produce a 32-bit version
> > of MSDOS, but the exact details haven't been
> > announced yet.

> It's why I asked about the pointer type of your PDOS.

Well, I'm willing to negotiate, but my opening offer, which
is what the PDOS/386 source code currently in sourceforge
does, now that I have fixed binutils 2.14a to stop stripping
relocation information, so that I could disable the VM code
that Alica wrote, is at the same level as 16-bit MSDOS - the
OS and applications all see the exact same flat memory
space, starting at location 0 (ie where you will find real
mode interrupt vectors, which are not the protected mode
interrupt vectors, which are currently not in a fixed location
so can't be directly manipulated by applications - but that's
not necessarily a bad thing).

Very simple. No memory protection. You can write directly
to 0xb8000 and the SVGA graphics card without any fuss
at all. As you can in 16-bit MSDOS if you make it
0xb800:0x0000

This is 32-bit MSDOS, not 32-bit Windows.

Actually I have a 32-bit Windows proposal too, but forget
about that for now, and let's agree what 32-bit MSDOS
should look like.

BFN. Paul.

[JJ]

On Tue, 4 May 2021 08:39:23 -0700 (PDT), muta...@gmail.com wrote:
>
> Ok, true, you can do that. But if you do that, you can
> avoid the CMDLINE altogether and just create a new
> INT 21H function, can't you? And maybe that is the
> superior solution, so that we can fix this "envseg"
> issue in the parm block at the same time.

Sure, the CMDLINE can be omitted and be replaced with a more elegant method,
but didn't you said you want to make it easy to port programs to your OS? If
you remove CMDLINE, you'll have more work porting those programs.

> I don't care about the size of either. What I care about is
> compatibility.

In your case, there's no way to keep 100% compatibility due to platform
difference. What you can do is to minimize the incompatibilities due to that
- if you care about compatibility.

> Well, I'm willing to negotiate, but my opening offer, which
> is what the PDOS/386 source code currently in sourceforge
> does, now that I have fixed binutils 2.14a to stop stripping
> relocation information, so that I could disable the VM code
> that Alica wrote, is at the same level as 16-bit MSDOS - the
> OS and applications all see the exact same flat memory
> space, starting at location 0 (ie where you will find real
> mode interrupt vectors, which are not the protected mode
> interrupt vectors, which are currently not in a fixed location
> so can't be directly manipulated by applications - but that's
> not necessarily a bad thing).
>
> Very simple. No memory protection. You can write directly
> to 0xb8000 and the SVGA graphics card without any fuss
> at all. As you can in 16-bit MSDOS if you make it
> 0xb800:0x0000
>
> This is 32-bit MSDOS, not 32-bit Windows.
>
> Actually I have a 32-bit Windows proposal too, but forget
> about that for now, and let's agree what 32-bit MSDOS
> should look like.

OK.

So, what if you treat segment fields as the upper 16-bit of the 32-bit flat
address?

[muta...@gmail.com]

On Wednesday, May 5, 2021 at 8:05:40 PM UTC+10, JJ wrote:

> > Ok, true, you can do that. But if you do that, you can
> > avoid the CMDLINE altogether and just create a new
> > INT 21H function, can't you? And maybe that is the
> > superior solution, so that we can fix this "envseg"
> > issue in the parm block at the same time.

> Sure, the CMDLINE can be omitted and be replaced with a more elegant method,
> but didn't you said you want to make it easy to port programs to your OS? If
> you remove CMDLINE, you'll have more work porting those programs.

Sure, it's a tradeoff. I'm not expecting every MSDOS
program to work unchanged in the transition from
16-bit to 32-bit.

But e.g. every MSDOS program that was written in C90
will port with zero changes whatsoever, other than the
C library itself, which is to be expected.

But C90 doesn't give you the ability to retrieve file
attributes. But can that be done in a portable manner,
sticking reasonable close to INT 21H AH=43H. All it
needs is a wrapper function, ie:

int PosGetFileAttributes(const char *fnm,int *attr)

That seems perfectly reasonable to me. Gates could
have created that API function above and told everyone
to start using that instead of setting ds/dx manually,
in preparation for a move to 32-bit, and even a move
to 64-bit. The function above will work even for 256-bit.

So to the question of CMDLINE - I don't think much
application code is spend doing direct exec() calls
with long parameters. It is more likely to be hidden
in a system() call, so you just need the C runtime
library author to make that change, once.

I really don't care how difficult it is for a C runtime
library author to convert from 16-bit MSDOS to
32-bit and 64-bit. I only care about the application
programmer.

> > I don't care about the size of either. What I care about is
> > compatibility.

> In your case, there's no way to keep 100% compatibility due to platform
> difference. What you can do is to minimize the incompatibilities due to that

That's true for the 32-bit move, but the CMDLINE
solution works even for old versions of 16-bit
MSDOS, which seems like a good solution to me.

> > Very simple. No memory protection. You can write directly
> > to 0xb8000 and the SVGA graphics card without any fuss
> > at all. As you can in 16-bit MSDOS if you make it
> > 0xb800:0x0000
> >
> > This is 32-bit MSDOS, not 32-bit Windows.
> >
> > Actually I have a 32-bit Windows proposal too, but forget
> > about that for now, and let's agree what 32-bit MSDOS
> > should look like.

> OK.
>
> So, what if you treat segment fields as the upper 16-bit of the 32-bit flat
> address?

Is this just for exec() or in general?

It's a lot of work to go to just for exec(), and it still won't
work anyway. Because if you only provide the upper
16 bits, it means you need to be able to allocate memory
on a 64k boundary, requiring a non-standard call. As soon
as you start doing non-standard stuff, you may as well
just write the conditional code.

In addition, when you move to 64-bit, you will be presented
with the same problem. Are you going to make the segment
the upper 16 bits of a 64-bit address? Or perhaps arrange
for memory allocation below 4 GB so that the upper 32 bits
are all 0, the next 16 bits will come from the segment, and
the lower 16 bits will be 0. This manipulation requires
conditional compilation. And basically buys nothing.

BFN. Paul.

[JJ]

On Wed, 5 May 2021 04:52:25 -0700 (PDT), muta...@gmail.com wrote:
>
> I really don't care how difficult it is for a C runtime
> library author to convert from 16-bit MSDOS to
> 32-bit and 64-bit. I only care about the application
> programmer.

Well you can't have both an entirely different specification, and
compatibility. You'll have to choose one and sacrifice the other. Whichever
is easier for software developers.

> Is this just for exec() or in general?

I meant DOS data structures which have segment fields instead of pointer
fields. IOTW, special handling for those type of DOS structures only.
Nothing more.

> It's a lot of work to go to just for exec(), and it still won't
> work anyway. Because if you only provide the upper
> 16 bits, it means you need to be able to allocate memory
> on a 64k boundary, requiring a non-standard call.

That is the downside, unfortunately.

Either way, it's like what I've mentioned earlier. You can't have both.
You'll have to choose one. Do you want to keep DOS compatibility, or (third
party) software compatibility? Whichever it's easier for the software
developer.

[muta...@gmail.com]

On Thursday, May 6, 2021 at 8:58:56 PM UTC+10, JJ wrote:

> > I really don't care how difficult it is for a C runtime
> > library author to convert from 16-bit MSDOS to
> > 32-bit and 64-bit. I only care about the application
> > programmer.

> Well you can't have both an entirely different specification, and
> compatibility. You'll have to choose one and sacrifice the other. Whichever
> is easier for software developers.

I'm having trouble understanding your position.

What options are available for software developers,
just considering these two "concepts":

int PosGetFileAttributes(const char *fnm, int *attr)
0x43

void PosExec(char *prog, POSEXEC_PARMBLOCK *parmblock)

0x4b

Noting that we want to support 16, 32 and 64-bit
versions of MSDOS, and we expect a recompile
of applications for each of those 3 targets.

> > Is this just for exec() or in general?

> I meant DOS data structures which have segment fields instead of pointer
> fields. IOTW, special handling for those type of DOS structures only.
> Nothing more.

I don't think there are many of those.

> > It's a lot of work to go to just for exec(), and it still won't
> > work anyway. Because if you only provide the upper
> > 16 bits, it means you need to be able to allocate memory
> > on a 64k boundary, requiring a non-standard call.

> That is the downside, unfortunately.

The downside is the same either way, so you may as
well use the opportunity to do it properly.

> Either way, it's like what I've mentioned earlier. You can't have both.
> You'll have to choose one. Do you want to keep DOS compatibility, or (third
> party) software compatibility? Whichever it's easier for the software
> developer.

It depends what you mean by "DOS compatibility".

I'm not expecting 32-bit software to run on 16-bit
hardware, and vice-versa.

That's an assumption I didn't think to voice out, sorry,
so may alter your answer.

BFN. Paul.

[JJ]

On Thu, 6 May 2021 09:12:40 -0700 (PDT), muta...@gmail.com wrote:
>
> It depends what you mean by "DOS compatibility".
>
> I'm not expecting 32-bit software to run on 16-bit
> hardware, and vice-versa.

If you think "DOS compatibility" is a hardware related, then I give up.

[muta...@gmail.com]

On Friday, May 7, 2021 at 10:35:58 PM UTC+10, JJ wrote:

> > It depends what you mean by "DOS compatibility".
> >
> > I'm not expecting 32-bit software to run on 16-bit
> > hardware, and vice-versa.

> If you think "DOS compatibility" is a hardware related, then I give up.

I have no idea what you are talking about.

BFN. Paul.