ANN: STATE-smartness: Applications, Pitfalls, Alternatives

[Anton Ertl]

@Unpublished{ertl01state,
author = {M. Anton Ertl},
title = {STATE-smartness: Applications, Pitfalls, Alternatives},
note = {This paper is an updated and significantly revised
version of my EuroForth '98 paper \cite{ertl98}; I
submitted this draft version to JFAR in 2001, but it
was not processed, so I finally put it online.},
year = {2001},
url = {http://www.complang.tuwien.ac.at/papers/ertl01state.pdf},
abstract = {STATE-smart words provide a number of unpleasant
surprises to their users. They are applied in two
contexts, and they fail in both: 1) for implementing
words like \texttt{s"} that provide an arbitrary
combination of interpretation and compilation
semantics (combined words); 2) for optimizing using
a special implementation of the (default)
compilation semantics. This paper discusses these
issues and shows programmers and system implementors
how to avoid STATE-smart words. It also reports our
experiences in converting the STATE-smart words in
Gforth into a clean solution: little work and few
problems.}
}

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

[A. K.]

Thanks for publishing the revised paper.

Although I have no problems with state-smartness (just being careful),
those who deal more with Forth compiler functions will sure find it informative.

[Krishna Myneni]

On 9/25/20 10:21 AM, Anton Ertl wrote:
> @Unpublished{ertl01state,
> author = {M. Anton Ertl},
> title = {STATE-smartness: Applications, Pitfalls, Alternatives},
> note = {This paper is an updated and significantly revised
> version of my EuroForth '98 paper \cite{ertl98}; I
> submitted this draft version to JFAR in 2001, but it
> was not processed, so I finally put it online.},
> year = {2001},
> url = {http://www.complang.tuwien.ac.at/papers/ertl01state.pdf},
> abstract = {STATE-smart words provide a number of unpleasant
> surprises to their users. They are applied in two
> contexts, and they fail in both: 1) for implementing
> words like \texttt{s"} that provide an arbitrary
> combination of interpretation and compilation
> semantics (combined words); 2) for optimizing using
> a special implementation of the (default)
> compilation semantics. This paper discusses these
> issues and shows programmers and system implementors
> how to avoid STATE-smart words. It also reports our
> experiences in converting the STATE-smart words in
> Gforth into a clean solution: little work and few
> problems.}
> }
>

Thank you for the nice paper, which gives a fairly clear description of
the problems associated with use of state-smart words for certain
specialized programming cases. There are also several specialized terms
most of which are clearly defined in the paper. IMO, the first step to
resolving the problems with state-smartness is to have a clear
understanding of these terms, which presents no minor challenge for both
Forth programmers and for system implementors:

1. interpretation semantics
2. compilation semantics
3. execution semantics
4. run-time semantics
5. initiation semantics

I don't question the need for these terms; the complexity is warranted
by the programming which is possible to do with Forth. The flexibility
of Forth gives rise to the complexity, which must be managed for
predictable behavior. On top of these terms, the paper also uses:

6. normal words
7. immediate words
8. combined words

Your argument is that using state-smartness in the definition of a word,
i.e. giving the word distinct execution behavior based on the value of
STATE at run-time, is a particular approach to writing "combined words",
with ambiguities which arise when the state-smart word names are
operated upon by tick, bracket-tick, and postpone.

The claim is that combined words, which make use of dual xt's, one for
interpretation semantics and a separate one for compilation semantics,
will not have the same ambiguities with respect to words like tick,
bracket-tick, and POSTPONE. It seems like

The other point is that binding of xt's can be done at parse time rather
than at run-time, which sounds like an advantage.

In my own Forth system, I blur the concept of interpretation state and
compilation state somewhat through the use of deferred execution, so I
need to consider the consequences of the combined word approach for my
system. S" is given as an example of a word which is sometimes
implemented as a state-smart word, but then has problems with tick and
postpone. In my system, S" is implemented without reference to STATE and
works in both interpretation and compilation states. Whether or not it
has problems with tick and POSTPONE is not something I have had to consider.

For example, the following two examples work in my system:

: t" ['] s" execute ; immediate
t" hello there" type

or,

: t" postpone s" ; immediate
t" hello there" type


The former example also works in Gforth, but the latter does not. There
is no state-smartness in Gforth's implementation, so why does it not work?

: t" postpone s" ; immediate ok
t" hello there" type
*the terminal*:2:17: error: Stack underflow
t" hello there" >>>type<<<
Backtrace:
kernel/io.fs:32:5: 0 $7FEE341F1EF0 dup
1 $7FEE341F2348

Krishna

[Ruvim]

On 2020-09-25 21:51, Krishna Myneni wrote:
> On 9/25/20 10:21 AM, Anton Ertl wrote:
>>    title =        {STATE-smartness: Applications, Pitfalls,
>> Alternatives},

>>    url =
>> {http://www.complang.tuwien.ac.at/papers/ertl01state.pdf},

[...]

> In my own Forth system, I blur the concept of interpretation state and
> compilation state somewhat through the use of deferred execution, so I
> need to consider the consequences of the combined word approach for my
> system. S" is given as an example of a word which is sometimes
> implemented as a state-smart word, but then has problems with tick and
> postpone. In my system, S" is implemented without reference to STATE and
> works in both interpretation and compilation states. Whether or not it
> has problems with tick and POSTPONE is not something I have had to
> consider.
>
> For example, the following two examples work in my system:
>
> : t" ['] s" execute ; immediate
> t" hello there" type

The question is how this word t" works in compilation state.

According to the standard, it should be an ambiguous condition.
So the following line of code

: foo t" test" type ; 123 foo .

is allowed to print "test 123", as well as to fail.


But by Anton's view it should fail, and the following:

: bar t" test" [ type ] ;

should print "test".


An excerpt:

| ’, [’] : XT represents interpretation semantics ³
|
| ³ This follows from the last sentence of
| [ANS94, Section 6.1.0070]

Actually, it doesn't follow.

The mentioned last sentence of 6.1.0070 is:
| When interpreting, ' xyz EXECUTE is equivalent to xyz.
(1)


If (³) is true then

' xyz ] [EXECUTE] [
is equivalent to
xyz
(2)


But (2) is false in the general case, and therefore (³) is false.

The "bar" example above relies on (2), so it's also false.

> or,
>
> : t" postpone s" ; immediate
> t" hello there" type

According to Q99-027 (RFI by Philip Preston), it's an ambiguous condition.

But the usual expectation is that this t" generates code even in
interpretation state:

: baz [ t" test" ] type ; 123 baz .

prints "test 123"

> The former example also works in Gforth, but the latter does not. There
> is no state-smartness in Gforth's implementation, so why does it not work?

Since it meets the common expectation that the code generated by
postpone doesn't depend on state, and it does in interpretation state
the same thing that it does in compilation state.


--
Ruvim

[dxforth]

On 26/09/2020 04:45, A. K. wrote:
>
> Although I have no problems with state-smartness (just being careful),
> those who deal more with Forth compiler functions will sure find it informative.
>

So it's aimed at beginners who haven't been around forth as long you to
know better. Such an audience may well believe state-smart words are
illegal under ANS-Forth when told - not "UB" at all, but an alternative
truth that would be good for everyone if adopted. Floats on the data
stack were evil too.

[Krishna Myneni]

On 9/25/20 9:29 PM, Ruvim wrote:
> On 2020-09-25 21:51, Krishna Myneni wrote:

...

>> : t" postpone s" ; immediate
>> t" hello there" type
>
> According to Q99-027 (RFI by Philip Preston), it's an ambiguous condition.
>
> But the usual expectation is that this t" generates code even in
> interpretation state:
>
>   : baz [ t" test" ] type ; 123 baz .
>
> prints "test 123"
>
>
>> The former example also works in Gforth, but the latter does not.
>> There is no state-smartness in Gforth's implementation, so why does it
>> not work?
>
> Since it meets the common expectation that the code generated by
> postpone doesn't depend on state, and it does in interpretation state
> the same thing that it does in compilation state.
>

...

I see. The discrepancy comes about because "interpretation" in my system
means something different than what is commonly understood in Forth.

KM

[A. K.]

What audience are you making up?

The Forthers that I met in my professional life were all seasoned engineers
and software guys. No beginners. They just didn't fiddle around with compiler
entrails but got their job done by programming in Forth for signal processing.

That you are speaking of 'truth' and 'evil' here speaks more of your own
mind state.

[Anton Ertl]

These terms are there mostly to support a number of implementation
approaches. In particular, the Forth-94 TC wanted to cover both the
traditional single-xt+immediate-flag and cmForth's approach. They did
almost, but not quite succeed. In particular, FILE S" as specified in
the Forth-94 cannot be implemented on a single-xt+immediate-flag
system. That could be fixed with a simple proposal, but nobody has
made the proposal, so it seems that people implementing
single-xt+immediate-flag systems don't care.

In any case, if we look at the "copy-pasteability" requirement below,
then the terminology of the standard transcends these origins. It is
a good way to specify these words. And as a system implementor, the
best way is to design your Forth system to implement these concepts.
"Copy-pasteability" means being able to copy and paste code that
contains, e.g., 's" bla"' from compiled code to interpreted code.

However, a significant number of people seem to have a deep attachment
to single-xt+immediate-flag implementations, and waste countless hours
on trying to reinterpret the standard or find or invent ambiguous
conditions in order to declare code that would unveil this
implementation approach as non-compliant. For some reason they prefer
these discussions over the pragmatic approach of Forth, Inc. (ignore
the issue until a customer complains), or investing a few hours in
implementing the concepts above (the first implementation in Gforth
took an afternoon), or invest some more hours in making proposals to
allow the STATE-smart implementation of S" and S\".

Anyway, back to the semantics above: As I see below, despite the paper
the semantics concepts have not been understood. So let me give an
example:

Consider 11.6.1.2165 S":

|Interpretation:
|( "ccc<quote>" -- c-addr u )
|
|Parse ccc delimited by " (double quote). Store the resulting string in
|a transient buffer c-addr u.
|
|Compilation:
|( "ccc<quote>" -- )
|
|Parse ccc delimited by " (double quote). Append the run-time semantics
|given below to the current definition.
|
|Run-time:
|( -- c-addr u )
|
|Return c-addr and u that describe a string consisting of the
|characters ccc.

Now consider the following definitions:

: s"i '"' parse save-mem ;
: s"c '"' parse postpone sliteral ; immediate
: foo s"c bla" ;
simple-see foo

\ output:
$7F463A9027D0 lit
$7F463A9027D8 $7F463A67C6D8
$7F463A9027E0 lit
$7F463A9027E8 3
$7F463A9027F0 ;s

S"I is an implementation of the interpretation semantics of S"
S"C is an implementation of the compilation semantics of S"

Inside FOO you see the run-time semantics that S"C appended to the
definition of FOO: push c-addr ($7F463A67C6D8) and u (3) on the stack.

> On top of these terms, the paper also uses:
>
>6. normal words
>7. immediate words
>8. combined words

For some reason, "combined word" has not caught on. Stephen Pelc has
introduced "NDCS word" and later "dual word". We will see if that
catches on. Anyway, I'll use "dual word" in the rest of this posting.

>Your argument is that using state-smartness in the definition of a word,
>i.e. giving the word distinct execution behavior based on the value of
>STATE at run-time, is a particular approach to writing "combined words",
>with ambiguities which arise when the state-smart word names are
>operated upon by tick, bracket-tick, and postpone.

No ambiguities. It's unambiguously defined what they do, and in some
cases they do not do what combined words should do. So it's not an
approach to writing dual words, STATE-smart words are an
approximation.

>The other point is that binding of xt's can be done at parse time rather
>than at run-time, which sounds like an advantage.

Yes, that's the decisive difference. There are many ways to implement
dual words correctly, but they all select the xt at parse time.

>In my system, S" is implemented without reference to STATE and
>works in both interpretation and compilation states. Whether or not it
>has problems with tick and POSTPONE is not something I have had to consider.
>
>For example, the following two examples work in my system:
>
>: t" ['] s" execute ; immediate
>t" hello there" type
>
>or,
>
>: t" postpone s" ; immediate
>t" hello there" type
>
>
>The former example also works in Gforth, but the latter does not. There
>is no state-smartness in Gforth's implementation, so why does it not work?

You apparently expect that T" performs the interpretation semantics of
S", but if you look at the specification POSTPONE, you find that it
appends the compilation semantics of S" to the current definition.
And that's what happens on Gforth:

: bla t" bla" ;
bla type \ prints "bla"

In the interpretive usage above, T" also performs the compilation
semantics of S", and appends the run-time semantics of S" to the
current definition; of course, because there is no current definition,
there is no way to perform this compiled code. TYPE then finds that
there is nothing on the stack and produces a stack underflow.

[Anton Ertl]

Ruvim <ruvim...@gmail.com> writes:
>On 2020-09-25 21:51, Krishna Myneni wrote:
>> For example, the following two examples work in my system:
>>
>> : t" ['] s" execute ; immediate
>> t" hello there" type
>
>The question is how this word t" works in compilation state.
>
>According to the standard, it should be an ambiguous condition.

Which ambiguous condition are you referring to?

In any case, trying it on Gforth, iForth, SwiftForth, and VFX, this
outputs "hello there".

>An excerpt:
>
> | ’, [’] : XT represents interpretation semantics ³
> |
> | ³ This follows from the last sentence of
> | [ANS94, Section 6.1.0070]
>
>Actually, it doesn't follow.
>
>The mentioned last sentence of 6.1.0070 is:
> | When interpreting, ' xyz EXECUTE is equivalent to xyz.
>(1)
>
>
>If (³) is true then
>
> ' xyz ] [EXECUTE] [
>is equivalent to
> xyz
>(2)

If you mean interpreted XYZ in the last line, the behaviour of
STATE-smart words differs between these two cases.

[me]

On Sunday, September 27, 2020 at 11:37:41 AM UTC-5, Anton Ertl wrote:

Structure a word for two semantics, one for compile the
other for interpretation.
INTERPRET determines, by state, which semantics to use when presented with the word; state is not in the word.

: foo (compile semantics)
(execution semantics) ; \ no state in foo,
\ just two vectors.

foo \ run foo's execution semantics
: bar foo ; \ compile foo's compile semantics
postpone foo \ if foo immediate, compile foo's compile
\ semantic
\ else compile foo (no semantics
\ determined at this point)

I could be off a bit on the postpone part but the main thing is there is no need for word doubles such as ['] [char] ... [foo] .

If so, looks worth pursuing.

--
me

[none albert]

In article <2020Sep2...@mips.complang.tuwien.ac.at>,
Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
<SNIP>

>These terms are there mostly to support a number of implementation
>approaches. In particular, the Forth-94 TC wanted to cover both the
>traditional single-xt+immediate-flag and cmForth's approach. They did
>almost, but not quite succeed. In particular, FILE S" as specified in
>the Forth-94 cannot be implemented on a single-xt+immediate-flag
>system. That could be fixed with a simple proposal, but nobody has
>made the proposal, so it seems that people implementing
>single-xt+immediate-flag systems don't care.

They can be implemented on single-xt+immediate-flag+allocate systems.
Where I've implemented allocate on top of single-xt+immediate-flag
system, one could maintain that FILE S" can be implemented on top
of single-xt+immediate-flag systems.

>- anton

Groetjes Albert
--
This is the first day of the end of your life.
It may not kill you, but it does make your weaker.
If you can't beat them, too bad.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[Bruce McFarling]

On Monday, September 28, 2020 at 12:37:41 AM UTC+8, Anton Ertl wrote:

> However, a significant number of people seem to have a deep attachment
> to single-xt+immediate-flag implementations, and waste countless hours
> on trying to reinterpret the standard or find or invent ambiguous
> conditions in order to declare code that would unveil this
> implementation approach as non-compliant.

That can only be explained by a confusion regarding the status a RFI replies. There is absolutely no need to find or invent ambiguous conditions when in the standard

1. It is an ambiguous condition for a program to perform compilation semantics in the interpretation state.
2. It is an ambiguous condition for a program to append semantics to the current definition in the interpretation state.

Since in A99-027 it was asked whether that is the case and the TC response confirmed that that is the case. Following that TC decision, it is settled that any interpretation of the standard that requires either or both of those to not be the case is a misinterpretation of the intent of the TC when they reached consensus on the standard, and therefore is a misinterpretation of the standard.

As for the source of that confusion, one possible source is the repeated insistence of a proponent of what the following discussion explains is a misinterpretation of the standard that he is a better judge of the intent of the TC than the explicit consensus of the TC regarding their intent.

Virtually,
BruceMcF, Beijng

[me]

On Sunday, September 27, 2020 at 3:20:54 PM UTC-5, me wrote:
> If so, looks worth pursuing.
>

/*
Simulation of word with combined semantics being run and
compiled.
*/

\ A word with combined semantics
: foo"
[ ' branch cfa , ahead ] ( compilation semantics )
asc " parse ' (.") cfa , HERE place here c@ 1+ allot align
exit [ resolve ] ( execution semantics )
asc " parse type
;

\ Simulation of new INTERPRET
: koo
-find
nip ( for now not deal with immediate )
if ( word found )
state @ if 2 cells + fi
enter
ii ( word not found )
drop
fi
; immediate
cr koo foo" Hello World "
\. ==> Hello World
cr
: bar koo foo" Hello Chuck " ." Now what? " ;
bar
\. ==> Hello Chuck Now what?

/*
That worked.
*/

--
me

[me]

On Monday, September 28, 2020 at 8:32:15 AM UTC-5, sdwj...@gmail.com wrote:
> On Sunday, September 27, 2020 at 3:20:54 PM UTC-5, me wrote:
> > If so, looks worth pursuing.
> >
>
> /*
> Simulation of word with combined semantics being run and
> compiled.
> */

> /*
> That worked.
> */
>

/*
(Before anyone wets their panties, this is simulation
and evaluation only, certainly not a suggestion)
It also worked for the following word type:
*/

\ A word with combined and shared semantics

: foo"
[ ' branch cfa , ahead ] ( compilation semantics )

back ' (.") cfa , HERE place here c@ 1+ allot align
trek
[ ' branch cfa , ahead swap ]

[ resolve ] ( execution semantics )

back type
trek
[ resolve ] ( common semantics )
asc " parse
;

--
me

[me]

On Monday, September 28, 2020 at 9:48:22 AM UTC-5, sdwj...@gmail.com wrote:
> On Monday, September 28, 2020 at 8:32:15 AM UTC-5, sdwj...@gmail.com wrote:>
> \ A word with combined and shared semantics

/*
Add some sugar.
*/

: COMPILE: compile branch ahead ; immediate
: RUN: compile branch ahead swap resolve ; immediate
: COMMON: resolve ; immediate

\ A word with combined and shared semantics
: foo"

COMPILE: ( compilation semantics )

back ' (.") cfa , HERE place here c@ 1+ allot align
trek

RUN: ( execution semantics )
back type
trek
COMMON: ( common semantics )
asc " parse
;

--
me

[Ruvim]

On 2020-09-27 18:18, Anton Ertl wrote:
> Krishna Myneni <krishna...@ccreweb.org> writes:
[...]

>> IMO, the first step to
>> resolving the problems with state-smartness is to have a clear
>> understanding of these terms, which presents no minor challenge for both
>> Forth programmers and for system implementors:
>>
>> 1. interpretation semantics
>> 2. compilation semantics
>> 3. execution semantics
>> 4. run-time semantics
>> 5. initiation semantics
>>
>> I don't question the need for these terms; the complexity is warranted
>> by the programming which is possible to do with Forth.
>
> These terms are there mostly to support a number of implementation
> approaches.



> In particular, the Forth-94 TC wanted to cover both the
> traditional single-xt+immediate-flag and cmForth's approach.

It's true. It means that actually the standard is a generalization of
these two approaches. So the standard is an approximation of these
approaches, and not vice versa.

Also it means that some practice that are correct in one approach, or in
another approach, are not correct in the generalized approach.


As a result of this generalization, the standard also allows other
approaches (e.g. single-nt with dual-xt).

I believe, we should support this generalization and make it better, and
perhaps tune the different approaches to better support the general
conceptions.

But some last movements are directed to support only one implementation,
and disregard other implementations. E.g. the new word NAME>INTERPRET is
for a dual-xt approaches only. It cannot return a correct result for TO
that is implemented as a single-xt immediate word.

> They did almost, but not quite succeed. > In particular, FILE S" as specified in the Forth-94 cannot
> be implemented on a single-xt+immediate-flag system.

The implementation at https://git.io/JvctZ is for a
single-xt+immediate-flag system. Is it right that by your view, this
implementation is not standard compliant?



--
Ruvim

[none albert]

In article <6c9a171b-e2e2-42db...@googlegroups.com>,

Bruce McFarling <bruce.m...@gmail.com> wrote:
>On Monday, September 28, 2020 at 12:37:41 AM UTC+8, Anton Ertl wrote:
>
>> However, a significant number of people seem to have a deep attachment
>> to single-xt+immediate-flag implementations, and waste countless hours
>> on trying to reinterpret the standard or find or invent ambiguous
>> conditions in order to declare code that would unveil this
>> implementation approach as non-compliant.
>
>That can only be explained by a confusion regarding the status a RFI replies.
>There is absolutely no need to find or invent ambiguous conditions when in the
>standard
>
>1. It is an ambiguous condition for a program to perform compilation semantics in
>the interpretation state.
>2. It is an ambiguous condition for a program to append semantics to the current
>definition in the interpretation state.

Can you discuss what this means for the now infamous case of a state-smart
." , in particular postponeing it?

>Virtually,
>BruceMcF, Beijng

[Bruce McFarling]

On Tuesday, September 29, 2020 at 5:26:59 PM UTC+8, none albert wrote:
> In article <6c9a171b-e2e2-42db...@googlegroups.com>,
> Bruce McFarling <bruce.m...@gmail.com> wrote:
> >There is absolutely no need to find or invent ambiguous conditions when in the
> >standard

> >1. It is an ambiguous condition for a program to perform compilation semantics in
> >the interpretation state.
> >2. It is an ambiguous condition for a program to append semantics to the current
> >definition in the interpretation state.
> Can you discuss what this means for the now infamous case of a state-smart
> ." , in particular postponeing it?

Standard ." has undefined interpretation semantics, and compiler semantics of:
QUOTE

Compilation: ( "ccc<quote>" -- )

Parse ccc delimited by " (double-quote). Append the run-time semantics given below to the current definition.

Run-time: ( -- )

Display ccc.

CLOSE QUOTE

So if you postpone in compilation state, that appends those compilation semantics to the word. You now have a word that does not have a defined standard behavior in interpretation state, but in compilation state, it ought to parse ccc delimited by " and append the runtime semantics of displaying ccc to the current definition.

It seems like a state smart immediate ." that an implementation's POSTPONE simply compiles into the word rather than executing should be fine if that defined word is then used in compilation state. And the standard only specifies its behavior if it is executed in compilation state.

If that word IS executed in interpretation state, compiling a runtime at HERE and then embedding the string in the form assumed by the runtime would be allowed, as would be parsing ccc and then displaying it, because that is what "is an ambiguous condition" MEANS ... that the behavior of a compliant implementation is not constrained in that circumstance.

It may be that the limitation in the scope of the standard for definitions that have had compilation semantics appended by POSTPONE is an inconvenience and having another standard which specifies what they do would be a convenience, but that would be a matter for that hypothetical other standard to take up.

Virtually
BruceMcF, Beijing

[Bruce McFarling]

On Tuesday, September 29, 2020 at 5:26:59 PM UTC+8, none albert wrote:

> Can you discuss what this means for the now infamous case of a state-smart
> ." , in particular postponeing it?

The issue would not be POSTPONE ." but ' ." which is supposed to give the "execution" semantics. A POSTPONEment only has to append compilation semantics so that the resulting works when compiling. Similarly for [COMPILE] which for words with non-standard compilation semantics only has to append those compilation semantics so that the resulting word can work when compiling.

But a tick has to EXECUTE correctly and COMPILE, correctly. And it could be EXECUTEd in interpret state or, if the EXECUTE occurs within an IMMEDIATE word executed in compilation state, so it has to EXECUTE correctly in interpret state and EXECUTE correctly in compile state.

The thing about ." is that it does not have defined interpretation behavior, so whatever it does when interpreting (and whether or not that is well advised), the interpretation behavior is specified to be outside the scope of the standard. The interpreter can refuse to perform it, the word can throw an exception if used in interpret state, it can go ahead and try to compile even at the risk of messing things up, and it can do what .( does except with a " delimiter.

Or it can play the national anthem of the country within which it is executed.

RFI 0007 which was passed 12Y:0N:1A:1NV clarifies all of the things an implementation must see to when adding implementation-specific behavior to a word with defined interpretation and compilation semantics:
(1) It must compile correctly
(2) It must interpret correctly
(3) (what it compiles) must execute correctly
(4) Its tick, when EXECUTEd or COMPILE,d must execute correctly
(5) [COMPILE] must work correctly with it.

(2) is not constrained for .", for (3) only executing in compilation state is constrained, for the tick, for (4) only what it does when EXECUTE is performed in compilation state and when the word into which it is COMPILE,ed may be executed in compilation state, and for (5) any word containing a [COMPILE] with a word with distinct specified compilation behavior is only constrained as to what it does when compiling.

The problem with S" is that the execution behavior of FILE S" is to use the buffer if interpreting and to embed a string which has a ca u reference placed on the stack if compiling, and it is impossible to know which of the two will be performed by EXECUTE at the time that the tick is performed, since it might be executed inside a word that is IMMEDIATE and is executing in compilation state. So FILE S" seems to mandate a state-smart definition, with execution-time binding of the behavior of S", or else some clever way to make an early bound system mimic the behavior of a late-bound system.

However, if state-smart definitions are bad, you want to avoid execution-time binding of the behavior of S" and would much prefer to have straightforward and direct access-time (when interpreting, compiling, POSTPONEing, [COMPILE]ing or ticking) binding of the behavior of S". You don't WANT to mimic the behavior of a late-binding system, because that is the behavior that is causing the problems you are trying to avoid.

And it seemed to be the intent of the TC to permit such early-bound systems.

It might be argued that such early bound systems should not implement FILE S" because their xt's when ticked are not capable of EXECUTEing ' S" correctly in both states. But that is a far from satisfactory state of affairs.

Virtually,
BruceMcF, Beijing

[me]

/*
Modified interpreter to handle stateless state-smart words.

In compiling branch added check for CBRAN which indicates
a stateless state-smart word. The CBRAN is same code as
BRANCH to be used only by COMPILE: . All stateless state-smart
begin with COMPILE:

MFA converts the PFA to the address following the CBRAN and
its offset. That is the address of the compile semantics in
the stateless state-smart word.

*/

\ Begins a stateless state-smart word
: COMPILE: compile CBRAN ahead ; immediate

\ convert pfa to macro field area
: MFA [ 2 CELLS ] LITERAL + ;

: INTERP
BEGIN
-FIND IF ( word found )
STATE @ < IF ( compiling )
DUP @ CBRAN = \ added
IF ( run compile part ) \ added
MFA ENTER \ added
II ( compile )
CFA , CONTINUE
FI
II ( run )
CFA EXEC
FI
II ( word not found )
HERE NUMBER
DPL @ 1+ IF ( double number )
PPN DLITERAL
II ( single number )
DROP PPN LITERAL
FI
FI
?STACK
AGAIN
;

--
me

[Ruvim]

On 2020-09-27 19:53, Anton Ertl wrote:
> Ruvim <ruvim...@gmail.com> writes:
>> On 2020-09-25 21:51, Krishna Myneni wrote:
>>> For example, the following two examples work in my system:
>>>
>>> : t" ['] s" execute ; immediate
>>> t" hello there" type
>>
>> The question is how this word t" works in compilation state.
>>
>> According to the standard, it should be an ambiguous condition.
>
> Which ambiguous condition are you referring to?

Applying tick to a word with undefined execution semantics is implicitly
ambiguous since the returned xt *may* identify the different behavior in
the different systems.

Therefore it should be explicitly declared that an ambiguous condition
exists if Tick is applied to a word with undefined execution semantics,
or undefined interpretation semantics. The second part can be later
removed, when "run-time semantics" will be used instead of "execution
semantics" in glossary entries for the certain words with undefined
interpretation semantics.

> In any case, trying it on Gforth, iForth, SwiftForth, and VFX, this
> outputs "hello there".

I raised a question how does 't"' (defined above) work in *compilation*
state.

The phrase:
] t" hello there" type [

doesn't output anything in SwiftForth, in contrast to Gforth (ⁱ).

Both behaviors are allowed, since if the standard doesn't specify
execution semantics, any behavior is allowed. Hence, an ambiguous
condition exists in this case.


You can claim that the execution semantics of a dual behavior word
should be equal to the interpretation semantics. But it means that this
word has default interpretation semantics! The fact of specifying
interpretation semantics separately says that they are not default, and
they may distinct from the implemented execution semantics.

Moreover, if the standard doesn't say that by default execution
semantics are equal to interpretation semantics — a program may not
relay on such an assumption!




The last sentence of 6.1.0070 only makes sense when execution semantics
are defined and interpretation semantics are defined for XYZ.

If interpretation semantics are undefined for XYZ, then the right part
is ambiguous (it may do anything), and then the left part is ambiguous too.

If execution semantics are undefined for XYZ, then the left part is
ambiguous (it may do anything), and then the *implication* is false
(since it doesn't mean that the right part is ambiguous too).

Actually, the fact that the ambiguity of the right part doesn't follow
from the the ambiguity of the left part means that this expression isn't
a material biconditional ( ( ↔ or ⇔ ), but a material implication ( → )
that can be expressed in the full form as:

*If* interpreting of « ' xyz EXECUTE » is unambiguous *then*
interpreting of « xyz » is unambiguous too and they behavior
are equal.

>> An excerpt:
>>
>> | ’, [’] : XT represents interpretation semantics ³
>> |
>> | ³ This follows from the last sentence of
>> | [ANS94, Section 6.1.0070]
>>
>> Actually, it doesn't follow.
>>
>> The mentioned last sentence of 6.1.0070 is:
>> | When interpreting, ' xyz EXECUTE is equivalent to xyz.
>> (1)
>>
>>
>> If (³) is true then
>>
>> ' xyz ] [EXECUTE] [
>> is equivalent to
>> xyz
>> (2)
>
> If you mean interpreted XYZ in the last line,

Of course, there is "when interpreting" in (2), as well as in (1).

> the behaviour of STATE-smart words differs between these two cases.

Yes, moreover, the behavior differs for any STATE-dependent immediate
word. And it means that (³) is false in the general case.


If a dual semantics word is implemented as a dual-xt word (⁴), the
standard interpretation semantics for this word can be easily
represented by an xt, but the expected STATE-dependent execution
semantics for this word *cannot* be easily represented by an xt.


If a dual semantics word is implemented as an immediate word (⁵), the
expected STATE-dependent execution semantics for this word can be easily
represented by an xt, but the standard interpretation semantics for this
word *cannot* be easily represented by an xt.


Therefore neither expected execution semantics nor standard
interpretation semantics can be got for a dual semantics word in a
portable way.



If we say that applying Tick to a dual semantics word should return an
xt for the execution semantics of *the word*, the result *may* (due to
undefined execution semantics) be different for the cases (⁴) and (⁵) —
see example (ⁱ) above. Hence it's ambiguous.

If we say that applying Tick to a dual semantics word should return an
xt that represents the standard interpretation semantics for the word,
it violates the explicit wording in the standard (that Tick returns the
execution semantics for /name/), and it doesn't work in the case (⁵).
Hence it's ambiguous too.

If we change the corresponding wording in the standard, it excludes
approach (⁵) and makes too many systems non-standard.


Therefore, the assumption (³) (that an xt returned by Tick always
represents the interpretation semantics for the corresponding word) is
false and should remain false.

Tick returns execution semantics, and applying Tick to a dual semantics
word is ambiguous.


--
Ruvim

[Ruvim]

On 2020-10-01 10:57, Ruvim wrote:
> On 2020-09-27 19:53, Anton Ertl wrote:
>> Ruvim <ruvim...@gmail.com> writes:
>>> On 2020-09-25 21:51, Krishna Myneni wrote:
>>>> For example, the following two examples work in my system:
>>>>
>>>> : t" ['] s" execute ; immediate
>>>> t" hello there" type
>>>
>>> The question is how this word t" works in compilation state.

[...]

>> In any case, trying it on Gforth, iForth, SwiftForth, and VFX, this
>> outputs "hello there".
>
> I raised a question how does 't"' (defined above) work in *compilation*
> state.
>
> The phrase:
>   ] t" hello there" type [
>
> doesn't output anything in SwiftForth, in contrast to Gforth (ⁱ).

Correction:

0 0 ] t" hello there" [ type

Outputs "hello there" in Gforth and nothing in SwiftForth.


--
Ruvim

[Bruce McFarling]

On Thursday, October 1, 2020 at 3:57:08 PM UTC+8, Ruvim wrote:
> The last sentence of 6.1.0070 only makes sense when execution semantics
> are defined and interpretation semantics are defined for XYZ.

"is equivalent to" makes sense. If "xyz" when interpreting has a specified behavior,
' xyz EXECUTE when interpreting has the same behavior, if "xyz" when interpreting
has an implementation dependent behavior, ' xyz EXECUTE when interpreting has
the same implementation dependent behavior. That behavior may, of course, be:

? xyz

> If interpretation semantics are undefined for XYZ, then the right part
> is ambiguous (it may do anything), and then the left part is ambiguous too.

In an implementation, interpretation semantics won't be undefined, so
' xyz EXECUTE when interpreting should do the same thing as xyz

> *If* interpreting of « ' xyz EXECUTE » is unambiguous *then*
> interpreting of « xyz » is unambiguous too and they behavior
> are equal.

With respect to WHAT it does, if there is no defined interpretation
behavior, then the implementation is free to have that be whatever
the implementer desires or whatever happens as an accidental
consequences of some implementation design feature(s) or
whatever it may be.

We do have that: "Unless otherwise specified in an Interpretation: section
of the glossary entry, the interpretation semantics of a Forth definition are
its execution semantics." so the execution semantics are only uniquely
the interpretation semantics WHEN not otherwise specified via a distinct
"Interpretation" section.

> If a dual semantics word is implemented as a dual-xt word (⁴), the
> standard interpretation semantics for this word can be easily
> represented by an xt, but the expected STATE-dependent execution
> semantics for this word *cannot* be easily represented by an xt.

This is the issue with dual XT words. For FILE S", if ['] S" EXECUTE is
executed while compiling because it is contained in an IMMEDIATE
word, then it's execution semantics are specified to be:

QUOTE: Compilation: ( "ccc<quote>" -- )
Parse ccc delimited by " (double-quote). Append the run-time semantics given
below to the current definition.

Run-time: ( -- c-addr u )

Return c-addr and u describing a string consisting of the characters ccc. A
program shall not alter the returned string. UNQUOTE

> If a dual semantics word is implemented as an immediate word (⁵), the
> expected STATE-dependent execution semantics for this word can be easily
> represented by an xt, but the standard interpretation semantics for this
> word *cannot* be easily represented by an xt.

But is that true? When ' S" EXECUTE is performed while interpreting,
it behaves the same as when S" is performed in interpreting. If
['] S" EXECUTE is compiled and then performed when compiling,
it behaves the same as when S" is compiling. If POSTPONEd or
[COMPILE] the defined word is only constrained by the standard
when executing while compiling and will perform the S"
compilation semantics.

If ['] S" COMPILE, the word defined has defined compilation semantics,
and no defined interpretation semantics. The defined word will do
interpretation semantics of S" if it executes while interpreting, and
the compiler semantics of S" if it executes while compiling, which
sounds like the execution semantics of S" as specified has been
appended to the word that COMPILE, has executed on.

Virtually,
BruceMcF, Beijing

[Ruvim]

On 2020-10-01 16:54, Bruce McFarling wrote:
> On Thursday, October 1, 2020 at 3:57:08 PM UTC+8, Ruvim wrote:
>> The last sentence of 6.1.0070 only makes sense when execution semantics
>> are defined and interpretation semantics are defined for XYZ.
>
> "is equivalent to" makes sense. If "xyz" when interpreting has a specified behavior,
> ' xyz EXECUTE when interpreting has the same behavior, if "xyz" when interpreting
> has an implementation dependent behavior, ' xyz EXECUTE when interpreting has
> the same implementation dependent behavior. That behavior may, of course, be:
>
> ? xyz

It's wrong in the general case.

For example, TO has a specified behavior when interpreting. But it
doesn't mean that it may be ticked. An implementation may throw an
exception on ticking TO. In this implementation the phrase
«TO X» works, but «' TO EXECUTE X» fails. And it's allowed.

Let xyz is "abc" (a string literal) that has implementation defined
behavior when interpreting. But it doesn't mean that ticking should
work. «"abc" type» prints "abc", but «' "abc" execute type» prints error
message. Again, they are not equivalent.

>> If interpretation semantics are undefined for XYZ, then the right part
>> is ambiguous (it may do anything), and then the left part is ambiguous too.
>
> In an implementation, interpretation semantics won't be undefined, so
> ' xyz EXECUTE when interpreting should do the same thing as xyz

Well, even if some implementation defines some meaningful interpretation
semantics for EXIT (e.g. to exit from the parent QUIT), it doesn't mean
that «' EXIT EXECUTE» does the same thing as «EXIT», e.g. Tick may
throw an exception in this case. And it's allowed and compliant.

>> *If* interpreting of « ' xyz EXECUTE » is unambiguous *then*
>> interpreting of « xyz » is unambiguous too and they behavior
>> are equal.
>
> With respect to WHAT it does, if there is no defined interpretation
> behavior, then the implementation is free to have that be whatever
> the implementer desires or whatever happens as an accidental
> consequences of some implementation design feature(s) or
> whatever it may be.

Yes.

If there is no defined interpretation behavior (i.e., defined by the
standard) then this case is *ambiguous*. The formula above is true for
*unambiguous* cases only.

[...]

--
Ruvim

[me]

On Tuesday, September 29, 2020 at 3:57:33 PM UTC-5, me wrote:

Built S" and LITERAL (called FooLit) with composite semantics.
I only show below the run for FooLit.

It appears (not noticeable from this) that INTERPRET doesn't need to
be concerned with the immediate flag; just concerns itself with
state and if in the compile state if it's dealing with composite
semantics or not. The immediate flag is just the concern of
POSTPONE as deciding what to present to the interpreter.

h1 FooLit a stateless state-smart Literal
/*
Build a stateless state smart literal and name it
FooLit
*/
: FooLit COMPILE: compile LIT , COMMON: ;
cr
h2 Run mode
cr 42 FooLit
42 assert \. pass
cr
h2 Compile mode
cr { [ 42 ] FooLit } run1
42 assert \. pass
cr
h2 Postpone
/*
FooLit is postpned by use of [COMPILE] .
The one-shot { ... } run1 ticks TASK and returns
its PFA as a literal.
*/
cr { [compile] ' [compile] FooLit } run1 task
' task assert \. pass
/*
Note
Stateless state-smart words are non-immediate but
still use [COMPILE] for postponement because they perform
as immediate when in the compile state.
*/

--
me

[Bruce McFarling]

On Friday, October 2, 2020 at 1:09:09 AM UTC+8, Ruvim wrote:
> On 2020-10-01 16:54, Bruce McFarling wrote:
> > On Thursday, October 1, 2020 at 3:57:08 PM UTC+8, Ruvim wrote:
> >> The last sentence of 6.1.0070 only makes sense when execution semantics
> >> are defined and interpretation semantics are defined for XYZ.
> >
> > "is equivalent to" makes sense. If "xyz" when interpreting has a specified behavior,
> > ' xyz EXECUTE when interpreting has the same behavior, if "xyz" when interpreting
> > has an implementation dependent behavior, ' xyz EXECUTE when interpreting has
> > the same implementation dependent behavior. That behavior may, of course, be:
> >
> > ? xyz
> It's wrong in the general case.
>
> For example, TO has a specified behavior when interpreting. But it
> doesn't mean that it may be ticked. An implementation may throw an
> exception on ticking TO. In this implementation the phrase
> «TO X» works, but «' TO EXECUTE X» fails. And it's allowed.

Where is that specified? The ambiguity on POSTPONE TO and [COMPILE] TO is in the definition of TO ... where is the ambiguity on ticking TO stated?

Virtually,
BruceMcF, Beijing

[Ruvim]

In it's glossary entry 6.2.2295

| An ambiguous condition exists if any of POSTPONE, [COMPILE],
| ' or ['] are applied to TO.

-- https://forth-standard.org/standard/core/TO


The only dual semantics word that is not explicitly disallowed to be
ticked is FILE S". Comparing to other cases it looks like an
inconsistency in the text/wording.

This flaw can be explained by the fact that the glossary entry of FILE
S" is based on the glossary entry of CORE S", for which ticking is
disallowed in another place.

My conclusion is that ticking of any word with undefined execution
semantics is intrinsically ambiguous. We have also a number of other
cases in the Forth-2012 standard when something is intrinsically
ambiguous but isn't explicitly declared as ambiguous.

Just for example:
- appending some semantics to the current definition when the current
definition is absent;
- executing IMMEDIATE when the current definition is presence;
- ticking of a local variable;
- applying POSTPONE to a local variable;

All these cases are intrinsically ambiguous but it isn't declared
explicitly. Such things can be made better in the next versions of the
standard.


--
Ruvim

[Ruvim]

On 2020-09-29 19:58, Bruce McFarling wrote:
> On Tuesday, September 29, 2020 at 5:26:59 PM UTC+8, none albert wrote:
>> Can you discuss what this means for the now infamous case of a state-smart
>> ." , in particular postponeing it?
>
> The issue would not be POSTPONE ." but ' ." which is supposed to give the "execution" semantics.

Tick is not applicable to the '."' word, — it's explicitly expressed as
an ambiguous condition in 4.1.2:

| attempting to obtain the execution token, [...] of a definition
| with undefined interpretation semantics;



BTW, I have noticed an interesting part in the rationale A.6.1.0190

| a Standard Program may not compile POSTPONE ." inside a new word,
| and then use that word while interpreting.

It closely recalls a99-027.



[...]

> RFI 0007 which was passed 12Y:0N:1A:1NV clarifies all of the things an implementation must see to when adding implementation-specific behavior to a word with defined interpretation and compilation semantics:

Looks like a typo: "defined" -> "default"

RFI 007 talks only about words with *default* interpretation and
compilation semantics:

| ... when extending normally "default" compilation
| and/or interpretation semantics of compliant definitions.

| ... it is not possible to define words of the sort
| that normally have "default" compilation and
| interpretation semantics such that ...

[...]

> The problem with S" is that the execution behavior of FILE S" is

Execution semantics for FILE S" are not specified by the standard.
Therefore, a standard program cannot have any problem that is connected
with execution behavior of S"

> to use the buffer if interpreting and to embed a string which has a ca u reference placed on the stack if compiling, and it is impossible to know which of the two will be performed by EXECUTE at the time that the tick is performed, since it might be executed inside a word that is IMMEDIATE and is executing in compilation state.

But these execution semantics are not specified! Certainly, a Forth
system may define this behavior, but a standard program is not allowed
to rely on this behavior, since a Forth system is not obligated to
provide namely this behavior.

> So FILE S" seems to mandate a state-smart definition, with execution-time binding of the behavior of S", or else some clever way to make an early bound system mimic the behavior of a late-bound system.

It's a curious reasoning. But it is not grounded in the standard.

In any case, I think we have to assume that the standard allows the
different approaches for implementing dual semantics words, and a
standard program cannot rely on any specific approach.

Therefore, concerning Tick, the only robust decision is that a standard
program may not apply Tick to a dual semantics word.

>
> However, if state-smart definitions are bad, you want to avoid execution-time binding of the behavior of S" and would much prefer to have straightforward and direct access-time (when interpreting, compiling, POSTPONEing, [COMPILE]ing or ticking) binding of the behavior of S". You don't WANT to mimic the behavior of a late-binding system, because that is the behavior that is causing the problems you are trying to avoid.
>

> And it seemed to be the intent of the TC to permit such early-bound systems..

Yes.

>
> It might be argued that such early bound systems should not implement FILE S" because their xt's when ticked are not capable of EXECUTEing ' S" correctly in both states. But that is a far from satisfactory state of affairs.

The general approach in the standard is that the operations that may be
incorrect in some allowed implementation, are just disallowed for a
standard program by declaration of the corresponding ambiguous conditions.



--
Ruvim

[Bruce McFarling]

On Friday, October 2, 2020 at 4:13:44 PM UTC+8, Ruvim wrote:
> On 2020-09-29 19:58, Bruce McFarling wrote:
> > On Tuesday, September 29, 2020 at 5:26:59 PM UTC+8, none albert wrote:
> >> Can you discuss what this means for the now infamous case of a state-smart
> >> ." , in particular postponeing it?

> > The issue would not be POSTPONE ." but ' ." which is supposed to give the "execution" semantics.
> Tick is not applicable to the '."' word, — it's explicitly expressed as
> an ambiguous condition in 4.1.2:

> | attempting to obtain the execution token, [...] of a definition
> | with undefined interpretation semantics;

Yes, that specifies that the system should specify what happens when you do that. It is ambiguous because the execution semantics are only specified when compiling, so what happens when you EXECUTE that tick would only be defined under the standard if in compilation mode.

> BTW, I have noticed an interesting part in the rationale A.6.1.0190
>
> | a Standard Program may not compile POSTPONE ." inside a new word,
> | and then use that word while interpreting.
>
> It closely recalls a99-027.

Yes, that reflects the same understanding of "append compilation semantics" as was clarified in a99-027: if some of the semantics of appended to the word are not defined when intepreting, but only when compiling, then executing the defined word in interpreter mode would be ambiguous.

> [...]
> > RFI 0007 which was passed 12Y:0N:1A:1NV clarifies all of the things an implementation must see to when adding implementation-specific behavior to a word with defined interpretation and compilation semantics:
> Looks like a typo: "defined" -> "default"

> RFI 007 talks only about words with *default* interpretation and
> compilation semantics:

It explains the case of default interpretation and compilation semantics. But it does help clarify all of the things an implementation must see to when adding implementation specific behavior to a word with defined interpretation and compilation semantics.

>
> | ... when extending normally "default" compilation
> | and/or interpretation semantics of compliant definitions.
>
> | ... it is not possible to define words of the sort
> | that normally have "default" compilation and
> | interpretation semantics such that ...
>
> [...]
> > The problem with S" is that the execution behavior of FILE S" is
> Execution semantics for FILE S" are not specified by the standard.

Of course it is. It specifies what happens when FILE S" is executed while compiling, and it specifies what happens when FILE S" is executed while interpreting, so it explains the behavior of the Forth word FILE S" when it is executed.

QUOTE: execution semantics:
The behavior of a Forth definition when it is executed. UNQUOTE

What is that behavior?
QUOTE: 11.6.1.2165 S"
s-quote FILE
Extend the semantics of 6.1.2165 S" to be:

Interpretation: ( "ccc<quote>" -- c-addr u )

Parse ccc delimited by " (double quote). Store the resulting string c-addr u at a temporary location. The maximum length of the temporary buffer is implementation-dependent but shall be no less than 80 characters. Subsequent uses of S" may overwrite the temporary buffer. At least one such buffer shall be provided.

Compilation: ( "ccc<quote>" -- )
Parse ccc delimited by " (double quote). Append the run-time semantics given below to the current definition.
Run-time: ( -- c-addr u )
Return c-addr and u that describe a string consisting of the characters ccc.

UNQUOTE

> Therefore, a standard program cannot have any problem that is connected
> with execution behavior of S"

You seem to be redefining the term "execution semantics" to mean something other than what is stated in its definition.

> In any case, I think we have to assume that the standard allows the
> different approaches for implementing dual semantics words, and a
> standard program cannot rely on any specific approach.

I think we can say that they attempted to. It is not axiomatic that they succeeded in specifying the standard in such a way to allowing every approach for implementing dual semantics words that modern users may wish to employ. If their buffer word had been FN" or some such, with simple default semantics, rather than a dual semantic word, it seems like they would have left more leeway, by virtue of the ambiguity of the standard of ticking a compiling word with incompletely specified execution semantics.

>
> Therefore, concerning Tick, the only robust decision is that a standard
> program may not apply Tick to a dual semantics word.
> >
> > However, if state-smart definitions are bad, you want to avoid execution-time binding of the behavior of S" and would much prefer to have straightforward and direct access-time (when interpreting, compiling, POSTPONEing, [COMPILE]ing or ticking) binding of the behavior of S". You don't WANT to mimic the behavior of a late-binding system, because that is the behavior that is causing the problems you are trying to avoid.
> >
> > And it seemed to be the intent of the TC to permit such early-bound systems..
>
> Yes.
> >
> > It might be argued that such early bound systems should not implement FILE S" because their xt's when ticked are not capable of EXECUTEing ' S" correctly in both states. But that is a far from satisfactory state of affairs.

> The general approach in the standard is that the operations that may be
> incorrect in some allowed implementation, are just disallowed for a
> standard program by declaration of the corresponding ambiguous conditions.

I think it is important to avoid the kind of retconning in reading the standard that Anton Ertl engages on in the piece referred to in the OP of this thread, where instead of accepting the TC explanation that his interpretation was a mis-interpretation of what they intended, he implies that they lies about that and what they did was to vote to change their mind about what they intended while simply pretending that that was what they intended all along.

That is, the direct contradiction between the TC discussion:

QUOTE: The TC recognizes that the terminology it used in 1994 to circumscribe the usage of the Forth compiler by a standard program has led to some significant misinterpretations of TC intent. The above answers state the intent of the TC. ... The TC does, however, with to state categorically that its original intent was to make it clear that standard programs without environmental dependencies shall not assume access to functions specific to the Forth compiler except while actually in the compilation state. UNQUOTE

And Anton Ertl's discussion:
QUOTE: The ANS Forth committee has relaxed (from a system view; restricted from a program view) its
model lately to allow state-smart implementations of certain words [X3J99] (but not all [X3J96]), so
programmers trying to comply with the standard lose some of the advantages of the original model.
However, for system implementors it is a good idea to try to comply with the original model, to avoid
bugs in their user's programs, and to hopefully establish more practice in the original model, such
that it will become standard again some day. UNQUOTE

The "original model" in Anton Ertl's discussion is what the TC is referring to as a "significant misinterpretation", and the TC insisted that it was never the ANS Forth model.

Virtually,
BruceMcF, Beijing

[Ruvim]

On 2020-10-02 14:48, Bruce McFarling wrote:
> On Friday, October 2, 2020 at 4:13:44 PM UTC+8, Ruvim wrote:
>> On 2020-09-29 19:58, Bruce McFarling wrote:
[...]

>>> The problem with S" is that the execution behavior of FILE S" is
>> Execution semantics for FILE S" are not specified by the standard.
>
> Of course it is. It specifies what happens when FILE S" is executed while compiling, and it specifies what happens when FILE S" is executed while interpreting, so it explains the behavior of the Forth word FILE S" when it is executed.

It doesn't specify what happens when S" is executed. But what happens
when S" name is *encountered* by the Forth text interpreter in
interpretation state, and what happens when S" name is *encountered* by
the text interpreter in compilation state.

See "interpretation semantics" and "compilation semantics" in 2.1
Definitions of terms
https://forth-standard.org/standard/notation#notation:terms


[...]

--
Ruvim

[dxforth]

Constantine the Great removed ambiguity. Ambiguity was the price ANS paid
to keep coalition partners together. Forth never had a Constantine. Still
doesn't.

[Bruce McFarling]

I did see them, but one DOES have to READ them.

execution semantics:
The behavior of a Forth definition when it is executed.

interpretation semantics:
The behavior of a Forth definition when its name is encountered by the text interpreter in interpretation state.
compilation semantics:
The behavior of a Forth definition when its name is encountered by the text interpreter in compilation state.

The claim that interpretation semantics and compilation semantics are NOT specific cases of execution semantics requires that the Forth definition is not EXECUTED when it is encountered by the text intepreter, and if it is not EXECUTED, then it HAS NO BEHAVIOR, so that is an internal contradiction of being the behavior of the Forth definition.

Wrt the following comment by our esteemed author of dxforth, that is not "ambiguity", that is straining to get the standard to say what it doesn't actually say. In these three definitions, interpretation semantics and compilation semantics are quite simply and directly defined to be specific cases of the execution semantics of words when they are encountered by the text interpreter.

We do have an extant argument to the contrary, but the TC pointed out 20 years ago that it was a misinterpretation of what they mean from the outset, and following along a misinterpretation does not seem to me to be a promising approach.

Virtually,
BruceMcF, Beijing

[Ruvim]

On 2020-10-03 06:30, Bruce McFarling wrote:
> On Friday, October 2, 2020 at 8:37:07 PM UTC+8, Ruvim wrote:
>> On 2020-10-02 14:48, Bruce McFarling wrote:
>>> On Friday, October 2, 2020 at 4:13:44 PM UTC+8, Ruvim wrote:
>>>> On 2020-09-29 19:58, Bruce McFarling wrote:
>> [...]
>>
>>>>> The problem with S" is that the execution behavior of FILE S" is
>>>> Execution semantics for FILE S" are not specified by the standard.
>
>>> Of course it is. It specifies what happens when FILE S" is executed while compiling, and it specifies what happens when FILE S" is executed while interpreting, so it explains the behavior of the Forth word FILE S" when it is executed.
>
>> It doesn't specify what happens when S" is executed. But what happens
>> when S" name is *encountered* by the Forth text interpreter in
>> interpretation state, and what happens when S" name is *encountered* by
>> the text interpreter in compilation state.
>
>> See "interpretation semantics" and "compilation semantics" in 2.1
>> Definitions of terms
>> https://forth-standard.org/standard/notation#notation:terms
>
> I did see them, but one DOES have to READ them.
>
> execution semantics:
> The behavior of a Forth definition when it is executed.
> interpretation semantics:
> The behavior of a Forth definition when its name is encountered by the text interpreter in interpretation state.
> compilation semantics:
> The behavior of a Forth definition when its name is encountered by the text interpreter in compilation state.
>
> The claim that interpretation semantics and compilation semantics are NOT specific cases of execution semantics requires that the Forth definition is not EXECUTED when it is encountered by the text intepreter,

Both claims are wrong: that they are NOT specific cases, and that they
ARE specific cases. You seem to confuse necessity and possibility.

It is possible that interpretation semantics and compilation semantics
ARE specific cases of execution semantics (identified by a single xt),
but it is NOT necessary. For example, it isn't the case for the ordinary
words. Also, it isn't the case for dual words in dual-xt systems.



It looks like some people believe that only single-xt+immediate-flag
systems can be strictly compliant. And other people believe that only
dual-xt systems can be strictly compliant. But the intention and wording
of the standard is to cover the different approaches in implementations
and make sure that a standard program correctly works independently of
these implementation details.

> and if it is not EXECUTED, then it HAS NO BEHAVIOR, so that is an internal contradiction of being the behavior of the Forth definition.

It's wrong. E.g., what is EXECUTED to perform the compilation semantics
for the "DUP" word?

>
> Wrt the following comment by our esteemed author of dxforth, that is not "ambiguity", that is straining to get the standard to say what it doesn't actually say.


> In these three definitions, interpretation semantics and compilation semantics are quite simply and directly defined to be specific cases of the execution semantics of words when they are encountered by the text interpreter.

See the opposite examples above.

>
> We do have an extant argument to the contrary, but the TC pointed out 20 years ago that it was a misinterpretation of what they mean from the outset, and following along a misinterpretation does not seem to me to be a promising approach.

The question (q99-027) was about *programs*. And as Anton correctly
noted, the TC has restricted a program view, and relaxed a system view.
But you seem to want to restrict the system view even more increasingly
that it was before.


--
Ruvim

[Ruvim]

On 2020-10-02 14:48, Bruce McFarling wrote:

[...]

> That is, the direct contradiction between the TC discussion:
>
> QUOTE: The TC recognizes that the terminology it used in 1994 to circumscribe the usage of the Forth compiler by a standard program has led to some significant misinterpretations of TC intent. The above answers state the intent of the TC. ... The TC does, however, with to state categorically that its original intent was to make it clear that standard programs without environmental dependencies shall not assume access to functions specific to the Forth compiler except while actually in the compilation state. UNQUOTE
>
> And Anton Ertl's discussion:

Just for reference:
http://www.complang.tuwien.ac.at/papers/ertl01state.pdf

> QUOTE: The ANS Forth committee has relaxed (from a system view; restricted from a program view) its
> model lately to allow state-smart implementations of certain words [X3J99] (but not all [X3J96]), so
> programmers trying to comply with the standard lose some of the advantages of the original model.
> However, for system implementors it is a good idea to try to comply with the original model, to avoid
> bugs in their user's programs, and to hopefully establish more practice in the original model, such
> that it will become standard again some day. UNQUOTE

> The "original model" in Anton Ertl's discussion is what the TC is referring to as a "significant misinterpretation", and the TC insisted that it was never the ANS Forth model.

Anton is essentially right in this place, if "original model" is the
*abstract model* that is described by the *normative parts* of the
Forth-94 standard (I suppose, Rationale is a non normative part).

In the normative parts, nothing preclude a program to execute in
interpretation state the code that is generated by POSTPONE (i.e., it's
a one of advantages of the original model).

Yes, TC didn't have such intention. But they wrote what they wrote. And
it was interpreted correctly by itself. And now the expected behavior is
that the code generated by POSTPONE behaves as in compilation state, but
independently of STATE.

However a standard program without environmental dependencies cannot
rely on such behavior due to the official clarification a99-027.


So I don't see any contradiction in the above excerpts.


--
Ruvim

[Krishna Myneni]

On 9/27/20 11:53 AM, Anton Ertl wrote:
> Ruvim <ruvim...@gmail.com> writes:
>> On 2020-09-25 21:51, Krishna Myneni wrote:
>>> For example, the following two examples work in my system:
>>>
>>> : t" ['] s" execute ; immediate
>>> t" hello there" type
>>
>> The question is how this word t" works in compilation state.
>>

>> According to the standard, it should be an ambiguous condition.
>
> Which ambiguous condition are you referring to?
>

> In any case, trying it on Gforth, iForth, SwiftForth, and VFX, this
> outputs "hello there".
>

>> An excerpt:
>>
>> | ’, [’] : XT represents interpretation semantics ³
>> |
>> | ³ This follows from the last sentence of

>> | [ANS94, Section 6.1.0070]
>>
>> Actually, it doesn't follow.
>>
>> The mentioned last sentence of 6.1.0070 is:
>> | When interpreting, ' xyz EXECUTE is equivalent to xyz.
>> (1)
>>
>>

>> If (³) is true then

>>
>> ' xyz ] [EXECUTE] [
>> is equivalent to
>> xyz
>> (2)
>

> If you mean interpreted XYZ in the last line, the behaviour of

> STATE-smart words differs between these two cases.
>

Not having thought through the dual-xt approach in detail before, my
question to Anton and Ruvim is the following:

Imagine for the moment that the standard explicitly requires a dual-xt
implementation, and disallows the traditional single xt + immediate
flag. Does it then follow that

1) The word IMMEDIATE may be removed from the standard,

and

2) All ambiguous conditions for use of ' (tick), ['], and POSTPONE with
any standard word or user-defined colon definitions, in either
compilation or interpretation state, may be eliminated,

and

3) user-defined words never need reference STATE within their definitions?


If not, what type of system can achieve the above goals?

Krishna



Krishna

[Anton Ertl]

Krishna Myneni <krishna...@ccreweb.org> writes:
>Imagine for the moment that the standard explicitly requires a dual-xt
>implementation, and disallows the traditional single xt + immediate
>flag.

You mean, for all words? With, e.g.,

' s"int ' s"comp interpret/compile: s"

>Does it then follow that
>
>1) The word IMMEDIATE may be removed from the standard,

You still want to be able to load existing programs, so that probably
will not happen.

If you mean:

Can you replace

: foo ... ; immediate

with

:noname ... ; dup interpret/compile: foo

then yes, that would work.

>2) All ambiguous conditions for use of ' (tick), ['], and POSTPONE with
>any standard word or user-defined colon definitions, in either
>compilation or interpretation state, may be eliminated,

Those that come to my mind at the moment, yes.

>3) user-defined words never need reference STATE within their definitions?

They have not needed it before (for some meaning of "need"), unless
they were user-defined text interpreters (and they still need it). If
your question is:

Can you implement a dual word, say, TO, properly rather than
implementing an approximation that uses STATE (and relies on
ambiguous conditions to cover up the difference between the
approximation and the proper behaviour)?

then yes, you can. A VALUE and it's corresponding (VALUE-only) TO can
be implemented as:

: value ( x "name" -- )
create ,
does> ( -- x1 )
@ ;

: to-int ( x1 "name" -- )
' >body ! ;
: to-comp ( compilation: "name" -- ; run-time: x1 -- )
' >body postpone literal postpone ! ;
' to-int ' to-comp interpret/compile: to

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

[Bruce McFarling]

I am not talking about either necessity nor possibility, I am talking about the meaning of the terms as defined in Section 2. A normal Forth definition is executed when it is encountered when interpreting, it's execution semantics and interpretation semantics are identical in that case. A normal Forth definition has its execution semantics appended to the word being defined when it is encountered when compiling, so it is executed when the defined word is executed.

It is not necessarily the case that a system is limited to a single TOKEN referring to the execution behavior, but the tokens are discussed in terms of the behavior, the behavior is not discussed in terms of the tokens. A definition in an implementation may have, as A.6.2.0945 puts it, the "normal execution token" that is used while interpreting or by EXECUTE and another compilation token that is used while compiling. However, there is nothing there to suggest that the compilation semantics will fail to be PERFORMED if compiling using the "normal execution token", rather that the compiled result might not be as EFFICIENT as if the compilation token is used: "In these implementations, COMPILE, might not generate code that is as efficient as normally compiled code."

> It is possible that interpretation semantics and compilation semantics
> ARE specific cases of execution semantics (identified by a single xt),
> but it is NOT necessary.

Your parenthetical statement has the ANS Forth system reversed. The definition of execution semantics is the behavior of the Forth definition when it is executed. Section 2 is the scope within which the term is used.

They are using the terms AS THEY DEFINED THEM: there may be different WAYS an implementation ensures that compilation semantics OCCURS, and if you want to get the optimal result AND know the Forth definition that you will be using at the time of defining the compiling word, POSTPONE may have a more optimal result than COMPILE, ... but COMPILE, will still see to the compilation semantics being performed in standard compliant Forth.

> For example, it isn't the case for the ordinary words.

In what sense? The interpretation and compiler semantics for ordinary words are defined in terms of the single execution semantics section of every ordinary word.

> Also, it isn't the case for dual words in dual-xt systems.

Yes, that is an implementation issue for the dual-xt systems, how it is that they can infer the full standard execution behavior from the execution token delivered by ticking the sole standard dual word.

If it is not normally something that the implementation can do, then the most straightforward approach may be to save the execution token that executes the interpretation mode behavior, and when compiling trap that xt and perform the compilation behavior that will append the correct CORE S" execution sementics instead.

If the ANS Standard can constrain the applicability of POSTPONE and [COMPILE] to TO in the definition of TO, with no mentioning of the constraint in the definition of POSTPONE and [COMPILE], then surely it is fair game for any NEW dual-words that implement the dual-xt system to specify that the compilation semantics for POSTPONE and [COMPILE] and when encountered while compiling are one thing, and the compilation semantics when the execution token is executed in compilation mode is something else.

> It looks like some people believe that only single-xt+immediate-flag
> systems can be strictly compliant.

This does not follow. The discussion regarding the correct handling of compilation of optimized machine language in a subroutine threaded Forth notes that extra information handling may be required in order to make sure that everything is handled correctly.

But on more careful consideration, this understanding of execution semantics implies that ticking words with no defined interpretation behavior is indeed outside the scope of the standard,

So because of the blunder in the definition of FILE S", it may be that extra work is required for a dual-xt implementation which aspires to standard compliance while including FILE S" that is not required of a compliant dual-xt implementation that does not implement it, nor of an implementation willing to depart from standard compliance on this point.

> And other people believe that only dual-xt systems can be strictly compliant.

The extant argument on that front was identified as a misinterpretation back in 1999, so it seems fairly safe to set aside.

> But the intention and wording of the standard is to cover the different
> approaches in implementations and make sure that a standard program
> correctly works independently of these implementation details.

The RFI0006 quite clearly established that it is not the case that every
specific implementation of every implementation strategy is compliant.

> > and if it is not EXECUTED, then it HAS NO BEHAVIOR, so that is an internal contradiction of being the behavior of the Forth definition.
> It's wrong. E.g., what is EXECUTED to perform the compilation semantics
> for the "DUP" word?

Yes, you are correct, I mispoke, the behavior of compilation is normally to cause the execution semantics to occur when the defined word is executed. But there are exceptions to that, which is why they defined it more generally.

> The question (q99-027) was about *programs*. And as Anton correctly
> noted, the TC has restricted a program view, and relaxed a system view.

I understand and sympathize with a system implementer who sees in a text what they had hoped to find, only to be disappointed when the text is explained and it turns out that they had misread it.

Anton claimed it was a change by the TC. The TC claimed Anton's interpretation was a mis-interpretation, and they were specifying what they had always intended to mean, which directly implies that they were not changing anything. I do not accept that Anton is a stronger authority on what the TC intended by the standard than the TC was.

Sincerely,
BruceMcF, Beijing

[Bruce McFarling]

On Saturday, October 3, 2020 at 8:16:14 PM UTC+8, Krishna Myneni wrote:
> 3) user-defined words never need reference STATE within their definitions?

Is it the case that user defined words NEED to reference STATE within their definitions in a single-xt+immediate system? Simply do not find standard words without defined interpretation semantics when in interpretation state, or else make all standard words without interpretation semantics COMPILE-ONLY and the only outlier is FILE S", which can have its regrettably overloaded execution semantics handled in some other way.
-- The most straightforward is to simply not implement ANS File S" but to implement " with the S" interpreter semantics in its place. The argument over " was between implementations which returned the address of a counted string and those that returned the address of the text string and the count on the stack, and the urgency for some new implementation maintaining compatibility with the former is much lower in 2020 than it was in the mid 90s.
-- If FILE S" is really desired, and avoiding state-smart words is also really desired, if one switches compilation state by vectoring word handling between COMPILE and INTERPET one could vector EXECUTE at the same time, and have the COMPILE mode EXECUTE check whether the XT is the S" XT and do CORE S" instead of interpretation-mode FILE S" ... then in interpret mode EXECUTE doesn't have to carry that overhead.
- Forth implementers are quite clever, there are surely more that haven't occurred to me off the top of my head.

IMMEDIATE cannot be dispensed with, where in a dual-xt it may be, though for portability with existing code bases might not be.

Implementations shouldn't have ambiguous conditions, they should do something specific in specific conditions. "Ambiguous conditions" in the standard are where WHAT they do is not constrained by the standard. A single-xt+immediate implementation will do something specific when it ticks, if the tick succeeds when it executes, when it postpones, when it [COMPILE]s, etc., just as a a dual-xt word will do.

One thing that makes the dual-xt approach appealing is that often the most straightforward way to set up the single-xt system is to say "no you can't do that" when the standard says it doesn't HAVE to do something, when it is more straightforward to set up a dual-xt system so that it can say, "yeah, you can do that too, don't worry about it".

Virtually,
BruceMcF, Beijing

[Ruvim]

On 2020-10-03 15:16, Krishna Myneni wrote:

> Imagine for the moment that the standard explicitly requires a dual-xt
> implementation, and disallows the traditional single xt + immediate
> flag. Does it then follow that
>
> 1) The word IMMEDIATE may be removed from the standard,
>
> and
>
> 2) All ambiguous conditions for use of ' (tick), ['],  and POSTPONE with
> any standard word or user-defined colon definitions, in either
> compilation or interpretation state, may be eliminated,
>
> and
>
> 3) user-defined words never need reference STATE within their definitions?
>
>
> If not, what type of system can achieve the above goals?

The ambiguous conditions concerning POSTPONE can be removed without any
additional explicit requirements to implementations. POSTPONE can be
correctly implemented in any approach.


If the standard requires dual-xt and disallows single-xt, it is still
not enough to meet the points 1,2,3.

1. IMMEDIATE is required for back-compatibility of programs. Otherwise,
such standard should de-standardize some programs and provide another
alternative mechanism.

2. To eliminate ambiguous conditions concerning Tick, it should specify
execution semantics for dual words, and for words with undefined
interpretation semantics (with corresponding rewording in some places).
Otherwise these ambiguous conditions still remain.

3. It should provide an alternative mechanism to define dual words
without STATE, and a user-defined text interpreter without STATE (if any).


OTOH, if the standard requires single-xt and disallows dual-xt, all the
ambiguous conditions concerning Tick are eliminated. But the execution
semantics in interpretation state for some words remain implementation
defined.

Concerning some well-known problems with immediate words, they are
eliminated by the corresponding implementation of POSTPONE.


--
Ruvim

[Krishna Myneni]

On 10/3/20 9:23 AM, Anton Ertl wrote:
> Krishna Myneni <krishna...@ccreweb.org> writes:
>> Imagine for the moment that the standard explicitly requires a dual-xt
>> implementation, and disallows the traditional single xt + immediate
>> flag.
>
> You mean, for all words? With, e.g.,
>
> ' s"int ' s"comp interpret/compile: s"
>

Yes. I don't recall the meanings of the above words, but all words have
an xt1 for interpretation state and an xt2 for compilation state.

>> Does it then follow that
>>
>> 1) The word IMMEDIATE may be removed from the standard,
>
> You still want to be able to load existing programs, so that probably
> will not happen.
>

Under the hypothetical standard, IMMEDIATE should be definable as word
which retrieves the last named definition's interpretation semantics and
sets the word's compilation semantics to the same xt. So that's not
really an issue for backwards compatibility.

> If you mean:
>
> Can you replace
>
> : foo ... ; immediate
>
> with
>
> :noname ... ; dup interpret/compile: foo
>
> then yes, that would work.

And default compilation semantics would just be a normal colon
definition without the use of IMMEDIATE. Ok, so under a dual-xt system,
there is no need for IMMEDIATE except for backwards compatibility, and
it can be defined in Forth source assuming we have a word which can
retrieve the interpretation semantics of the last defined word.

>
>> 2) All ambiguous conditions for use of ' (tick), ['], and POSTPONE with
>> any standard word or user-defined colon definitions, in either
>> compilation or interpretation state, may be eliminated,
>
> Those that come to my mind at the moment, yes.
>

Good. It is possible that some words have one or the other xt set to
NOOP (no-operation), but that's fine.

>> 3) user-defined words never need reference STATE within their definitions?
>
> They have not needed it before (for some meaning of "need"), unless
> they were user-defined text interpreters (and they still need it). If
> your question is:
>
> Can you implement a dual word, say, TO, properly rather than
> implementing an approximation that uses STATE (and relies on
> ambiguous conditions to cover up the difference between the
> approximation and the proper behaviour)?
>
> then yes, you can. A VALUE and it's corresponding (VALUE-only) TO can
> be implemented as:
>
> : value ( x "name" -- )
> create ,
> does> ( -- x1 )
> @ ;
>
> : to-int ( x1 "name" -- )
> ' >body ! ;
> : to-comp ( compilation: "name" -- ; run-time: x1 -- )
> ' >body postpone literal postpone ! ;
> ' to-int ' to-comp interpret/compile: to
>

Yes, that's the sort of thing I had in mind. So 3) should not be a
restriction, but a special case for when one needs to write a custom
interpreter.

Krishna

[Krishna Myneni]

On 10/3/20 11:25 AM, Ruvim wrote:
> On 2020-10-03 15:16, Krishna Myneni wrote:
>
>> Imagine for the moment that the standard explicitly requires a dual-xt
>> implementation, and disallows the traditional single xt + immediate
>> flag. Does it then follow that
>>
>> 1) The word IMMEDIATE may be removed from the standard,
>>
>> and
>>
>> 2) All ambiguous conditions for use of ' (tick), ['],  and POSTPONE
>> with any standard word or user-defined colon definitions, in either
>> compilation or interpretation state, may be eliminated,
>>
>> and
>>
>> 3) user-defined words never need reference STATE within their
>> definitions?
>>
>>
>> If not, what type of system can achieve the above goals?
>
>
> The ambiguous conditions concerning POSTPONE can be removed without any
> additional explicit requirements to implementations. POSTPONE can be
> correctly implemented in any approach.
>

Ok. That's fair, considering my system(s) are single-xt + immediate
flag, and I believe POSTPONE works per the standard on them.

>
> If the standard requires dual-xt and disallows single-xt, it is still
> not enough to meet the points 1,2,3.
>
> 1. IMMEDIATE is required for back-compatibility of programs. Otherwise,
> such standard should de-standardize some programs and provide another
> alternative mechanism.
>

IMMEDIATE should be definable in Forth source, if needed, in a dual-xt
system -- see my reply to Anton above.

> 2. To eliminate ambiguous conditions concerning Tick, it should specify
> execution semantics for dual words, and for words with undefined
> interpretation semantics (with corresponding rewording in some places).
> Otherwise these ambiguous conditions still remain.
>

Sure, it should be possible to set either interpretation semantics or
compilation semantics to a NO-OP.

> 3. It should provide an alternative mechanism to define dual words
> without STATE, and a user-defined text interpreter without STATE (if any).
>

I'm not saying that STATE should be eliminated, just wondering whether
or not it is still needed for the types of uses we encounter typically
in single-xt+immediate flag systems. Additional words to get and set the
interpretation semantics and compilation semantics of a word are needed.

>
> OTOH, if the standard requires single-xt and disallows dual-xt, all the
> ambiguous conditions concerning Tick are eliminated. But the execution
> semantics in interpretation state for some words remain implementation
> defined.
>

Your implication is that a dual-xt standard still can have ambiguities
about the behavior of a word in interpretation state and/or compilation
state. I'm not sure why that would be the case. Can you give an example?


Krishna

[Krishna Myneni]

On 10/3/20 11:00 AM, Bruce McFarling wrote:
> On Saturday, October 3, 2020 at 8:16:14 PM UTC+8, Krishna Myneni wrote:
>> 3) user-defined words never need reference STATE within their definitions?
>
> Is it the case that user defined words NEED to reference STATE within their definitions in a single-xt+immediate system? Simply do not find standard words without defined interpretation semantics when in interpretation state, or else make all standard words without interpretation semantics COMPILE-ONLY and the only outlier is FILE S", which can have its regrettably overloaded execution semantics handled in some other way.

Perhaps one does not "need" to do it, but it is common-place in
single-xt+immediate flag systems. That indicates to me that the present
standard Forth is getting in the way of what programmers want to do.

> -- The most straightforward is to simply not implement ANS File S" but to implement " with the S" interpreter semantics in its place. The argument over " was between implementations which returned the address of a counted string and those that returned the address of the text string and the count on the stack, and the urgency for some new implementation maintaining compatibility with the former is much lower in 2020 than it was in the mid 90s.

My argument is not with S" . It's trying to imagine a new standard in
which the behavior of a word, any word, is not ambiguous in either
interpretation state or compilation state. That seems to me a worthwhile
goal for writing portable programs.


> ...

> IMMEDIATE cannot be dispensed with, where in a dual-xt it may be, though for portability with existing code bases might not be.
>

IMMEDIATE can be dealt with easily in the imagined "standard dual-xt
system". It requires an additional standard word or two.

> Implementations shouldn't have ambiguous conditions, they should do something specific in specific conditions. "Ambiguous conditions" in the standard are where WHAT they do is not constrained by the standard. A single-xt+immediate implementation will do something specific when it ticks, if the tick succeeds when it executes, when it postpones, when it [COMPILE]s, etc., just as a a dual-xt word will do.
>
> One thing that makes the dual-xt approach appealing is that often the most straightforward way to set up the single-xt system is to say "no you can't do that" when the standard says it doesn't HAVE to do something, when it is more straightforward to set up a dual-xt system so that it can say, "yeah, you can do that too, don't worry about it".
>

A future standard should work towards allowing powerful forms of
expression in the programming which don't allow for ambiguities. In
order to do so, it seems like we have to pick one or the other. Anton
makes a strong case for the dual-xt system.

Krishna

[Anton Ertl]

Krishna Myneni <krishna...@ccreweb.org> writes:
>On 10/3/20 9:23 AM, Anton Ertl wrote:
>> Krishna Myneni <krishna...@ccreweb.org> writes:
>>> Imagine for the moment that the standard explicitly requires a dual-xt
>>> implementation, and disallows the traditional single xt + immediate
>>> flag.
>>
>> You mean, for all words? With, e.g.,
>>
>> ' s"int ' s"comp interpret/compile: s"
>>
>
>Yes. I don't recall the meanings of the above words, but all words have
>an xt1 for interpretation state and an xt2 for compilation state.

S"INT is a word with an xt that performs the interpretation semantics
of S". S"COMP is a word with an xt that performs the compilation
semantics of S". The implementation may be:

: s"int '"' parse save-mem ;
: s"comp '"' parse postpone sliteral ;

INTERPRET/COMPILE: ( xt-int xt-comp "name" -- ) is a word that defines
"name" with the interpretation semantics being to perform xt-int, and
the compilation semantics being to perform xt-comp.

Your description and INTEPRET/COMPILE: suggest an implementation where
the compilation semantics are always represented by a single xt.
E.g., you might have a separated-header design, where the header of S"
above looks like:

link field: address of the previously defined header
name field: string 'S"'
int field: xt of S"INT
comp field: xt of S"COMP

An immediate word like '(' is easy to define with this header
structure:

:noname ')' parse 2drop ; dup interpret/compile: (

link field: address of the previously defined header
name field: string '('
int field: xt of the :noname-defined word above
comp field: xt of the :noname-defined word above

A normal word like 2DUP is slightly harder to define; for each such
word you have to define an xt that implements the compilation
semantics:

: 2dup-int over over ;
: 2dup-comp ['] 2dup-int compile, ;
' 2dup-int ' 2dup-comp interpret/compile: 2dup

(The names 2DUP-INT and 2DUP-COMP are used here for clarity. In
practice, one would use :NONAME definitions, because they do not
introduce more headers that would need additional compilation
semantics definitions, which would lead to endless recursion.).

Because normal words are so frequent, AFAIK no system has taken this
pure dual-xt approach. This is also catered for in the stack effect
of

NAME>COMPILE ( nt -- xt1 xt2 )

For normal words, you can pass the content of the int field as xt1 and
the xt of COMPILE, as xt2, avoiding the need to have a separate compilation semantics implementation.

For immediate words, you can again pass the content of the int field
as xt1, but you pass the xt of EXECUTE as xt2.

For dual words, you can pass the xt in the comp field as xt1, and the
xt of EXECUTE as xt2.

So how do you know which of these cases you have? A straighforward
way is to design the header so you do not need to know:

link field
name field
int field
comp-xt1 field
comp-xt2 field

A normal definition stores the same xt in the int field and in
comp-xt1, and stores the xt of COMPILE, in comp-xt2. IMMEDIATE writes
the xt of EXECUTE into the comp-xt2 field. A word defined with
INTERPRET/COMPILE: stores xt-int in the int field, xt-comp in the
comp-xt1 field, and the xt of EXECUTE in the comp-xt2 field.

The new Gforth header [paysan19] takes a different approach, though.
It has fields that contain the xts for performing NAME>INTERPRET and
NAME>COMPILE on this particular header; e.g., NAME>COMPILE is
implemented like

: name>compile ( nt -- xt1 xt2 ) dup name>compile-field @ execute ;

This allows to implement them very flexibly. For normal words, the
NAME>INTERPRET field contains ' NOOP (the nt and the xt point to the
same place for normal words in the new header), and the NAME>COMPILE
field contains

: default-name>comp ( nt -- xt1 xt2 ) ['] compile, ;

i.e., xt1=nt, nt2=COMPILE,

For immediate words, the NAME>INTERPRET field contains ' NOOP, and the
NAME>COMPILE field contains the xt of

: imm>comp ( nt -- xt1 xt2 ) name>int ['] execute ;

For a word defined with INTERPRET/COMPILE:, such as S", the body
contains an int field and a comp field (similar to the straightforward
header above), and the NAME>INTERPRET field contains the xt of

: a>int ( nt -- xt-int ) >body @ ;

and the NAME>COMPILE field contains the xt of

: i/c>comp ( nt -- xt1 xt2 )
>body cell+ @ ['] execute ;

The new Gforth header is very flexible, also supporting standard features
like "to name" semantics, synonyms and a number of non-standard
features like per-task deferred words (defined with udefer) without
needing to add flag after flag to the header and corresponding ifs to
the code dealing with headers. Read all about it in:

@InProceedings{paysan19,
author = {Bernd Paysan and M. Anton Ertl},
title = {The new {Gforth} Header},
crossref = {euroforth19},
pages = {5--20},
url = {http://www.euroforth.org/ef19/papers/paysan.pdf},
url-slides = {http://www.euroforth.org/ef19/papers/paysan-slides.pdf},
video = {https://wiki.forth-ev.de/doku.php/events:ef2019:header},
OPTnote = {refereed},
abstract = {The new Gforth header is designed to directly
implement the requirements of Forth-94 and
Forth-2012. Every header is an object with a fixed
set of fields (code, parameter, count, name, link)
and methods (\texttt{execute}, \texttt{compile,},
\texttt{(to)}, \texttt{defer@}, \texttt{does},
\texttt{name>interpret}, \texttt{name>compile},
\texttt{name>string}, \texttt{name>link}). The
implementation of each method can be changed
per-word (prototype-based object-oriented
programming). We demonstrate how to use these
features to implement optimization of constants,
\texttt{fvalue}, \texttt{defer}, \texttt{immediate},
\texttt{to} and other dual-semantics words, and
\texttt{synonym}.}

}

>>> Does it then follow that
>>>
>>> 1) The word IMMEDIATE may be removed from the standard,
>>
>> You still want to be able to load existing programs, so that probably
>> will not happen.
>>
>
>Under the hypothetical standard, IMMEDIATE should be definable as word
>which retrieves the last named definition's interpretation semantics and
>sets the word's compilation semantics to the same xt. So that's not
>really an issue for backwards compatibility.

So the question is "Can IMMEDIATE be defined on such a dual-xt
system?" Yes, it can. In the straightforward dual-xt system that
would be

: immediate latest dup int-field @ swap comp-field ! ;

In the variant with comp-xt1 and comp-xt2 it would be

: immediate ['] execute latest comp-xt2-field ! ;

In development Gforth, it is

: imm>comp name>int ['] execute ;
: immediate ( -- ) \ core
['] imm>comp set->comp ;

>>> 2) All ambiguous conditions for use of ' (tick), ['], and POSTPONE with
>>> any standard word or user-defined colon definitions, in either
>>> compilation or interpretation state, may be eliminated,
>>
>> Those that come to my mind at the moment, yes.
>>
>
>Good. It is possible that some words have one or the other xt set to
>NOOP (no-operation), but that's fine.

NOOP does not make much sense as the interpretation semantics or
compilation semantics of all words but NOOP. If the interpretation
semantics are undefined (e.g., for IF), you can put

: no-intsem -14 throw ;

in the int field. However, our experience is that a number of users
expect that these words have an interpretation semantics that's the
same as the defined compilation semantics, i.e., that these words are
defined like

: if postpone ?branch >mark? ; immediate

so in the current development version that is what we do.

[Ulrich Hoffmann]

Anton Ertl schrieb am Montag, 5. Oktober 2020 um 12:15:40 UTC+2:
...

> So how do you know which of these cases you have? A straighforward
> way is to design the header so you do not need to know:
>
> link field
> name field
> int field
> comp-xt1 field
> comp-xt2 field
>
> A normal definition stores the same xt in the int field and in
> comp-xt1, and stores the xt of COMPILE, in comp-xt2. IMMEDIATE writes
> the xt of EXECUTE into the comp-xt2 field. A word defined with
> INTERPRET/COMPILE: stores xt-int in the int field, xt-comp in the
> comp-xt1 field, and the xt of EXECUTE in the comp-xt2 field.

What other systems besides (classic) Gforth implement this dual-xt header structure?
I'm only aware of the original Holon for DOS that uses this approach. Are there others?

Regards,
Ulrich

[dxforth]

VFX for the last few years (?)

[Anton Ertl]

Ulrich Hoffmann <ulrich.e...@gmail.com> writes:
>Anton Ertl schrieb am Montag, 5. Oktober 2020 um 12:15:40 UTC+2:
>...
>> So how do you know which of these cases you have? A straighforward
>> way is to design the header so you do not need to know:
>>
>> link field
>> name field
>> int field
>> comp-xt1 field
>> comp-xt2 field
>>
>> A normal definition stores the same xt in the int field and in
>> comp-xt1, and stores the xt of COMPILE, in comp-xt2. IMMEDIATE writes
>> the xt of EXECUTE into the comp-xt2 field. A word defined with
>> INTERPRET/COMPILE: stores xt-int in the int field, xt-comp in the
>> comp-xt1 field, and the xt of EXECUTE in the comp-xt2 field.
>
>What other systems besides (classic) Gforth implement this dual-xt header structure?

I don't know of a system that implements this structure.

Before the new header, Gforth used a header that was pretty close to
the classical fig-Forth header with a few additional flags, and dealt
with the various requirements with lots of conditionals, based either
on the flags or on the contents of the code field:

* It had an alias flag, and associated handling in NAME>INTERPRET and
NAME>COMPILE (actually NAME>INT, NAME?INT and NAME>COMP).

* It had a compile-only (aka restrict) flag, and associated handling
in NAME>INTERPRET.

* Dual words had a special code field, with the two xts being in the
body. NAME>INTERPRET and NAME>COMPILE check for this special code
field, and do the appropriate thing if they see this code field.

* The intelligent COMPILE, was implemented by inspecting the code
field in a CASE construct, and doing the appropriate thing; e.g.,
when it saw a constant, the value of the constant was compiled as a
literal. Of course, this CASE statement is not extensible, so you
could not implement SET-OPTIMIZER.

* TO was also implemented by inspecting the code field in a case
construct, and doing the approriate thing. Of course, this is not
extensible, either.

* Nameless words just did not have the name and link field, but there
was no reliable way to determine whether a word is nameless.

Overall, this was a mess of special cases, and every new requirement
meant that something new had to be added. E.g., we could not
implement SYNONYM properly with the old header structure. I guess
that some will cite Thinking Forth and claim that that is a benefit of
the old header structure.

The new header structure eliminates nearly* all of these conditionals
and is very flexible. Of course I don't know if it will be flexible
enough to cover future requirements, but it's goal is that it covers
the existing requirements elegantly, and it does that nicely.

[*] We still have the compile-only flag and associated IFs.

>I'm only aware of the original Holon for DOS that uses this approach.

Where do I find anything about its header structure?

[Anton Ertl]

As far as I understand the presentations and the papers given by
Stephen Pelc about this topic in the last few years, VFX does not
implement such a structure. It has a header that includes a field for
the "intelligent COMPILE,", and has had this header field since the
inception of VFX. VFX recently has added a flag that tells all
affected words that the word is a dual word, and that the content of
that field is no longer used to implement "COMPILE,", but to implement
compilation semantics. I have not evaluated the implementation to see
whether all affected words act correctly when this flag is set.

[A. K.]

Am Samstag, 3. Oktober 2020 20:58:50 UTC+2 schrieb Krishna Myneni:
> On 10/3/20 11:00 AM, Bruce McFarling wrote:
> > On Saturday, October 3, 2020 at 8:16:14 PM UTC+8, Krishna Myneni wrote:
> >> 3) user-defined words never need reference STATE within their definitions?
> >
> > Is it the case that user defined words NEED to reference STATE within their definitions in a single-xt+immediate system? Simply do not find standard words without defined interpretation semantics when in interpretation state, or else make all standard words without interpretation semantics COMPILE-ONLY and the only outlier is FILE S", which can have its regrettably overloaded execution semantics handled in some other way.
>
> Perhaps one does not "need" to do it, but it is common-place in
> single-xt+immediate flag systems. That indicates to me that the present
> standard Forth is getting in the way of what programmers want to do.
>

I always wondered why one should jump through hoops and loops just because of
that one misconceptioned word POSTPONE...

[dxforth]

"POSTPONEd Words are Evil" :)

[Alex McDonald]

My system WF32. https://github.com/alextangent/wf32

begin-structure head% \ length to head.nt
6 cells - 2- 1- \ most words return head.nt
lfield: head.link \ [ link field ] lfa
lfield: head.ct \ [ ' compile, ] <---+ 1 ct:
compile, or execute (if immediate)
lfield: head.xtptr \ +---- [ xt ptr field ] | 2 xt-ptr
lfield: head.comp \ | [ ' xt-call, ] | how to
compile, normally xt-call,
lfield: head.rec \ | [ recognizer ] | currently 0
wfield: head.vfa \ | [ view field ] | vfa (line#)
wfield: head.ofa \ | [ optimize field ] | ofa
(length of code for inline)
wfield: head.stk \ | [ stack effects ] | ste (will
be removed)
bfield: head.tfa \ | [ type flag ] | tfa
0 +field head.nt \ | [ the name letters ] | counted
string; the name token
6 cells 2+ + \ | |
end-structure \ | |
\ | |
\ | [ ct-off ] ----+ rel offset
to head.ct
\ +---> [ xt field ] code (the xt)




--
Alex

[Stephen Pelc]

On Sat, 3 Oct 2020 13:58:46 -0500, Krishna Myneni
<krishna...@ccreweb.org> wrote:

>On 10/3/20 11:00 AM, Bruce McFarling wrote:
>> On Saturday, October 3, 2020 at 8:16:14 PM UTC+8, Krishna Myneni wrote:
>>> 3) user-defined words never need reference STATE within their definitions?
>>
>> Is it the case that user defined words NEED to reference STATE within their definitions in a single-xt+immediate system? Simply do not find standard words without defined interpretation semantics when in interpretation state, or else make all standard words without interpretation semantics COMPILE-ONLY and the only outlier is FILE S", which can have its regrettably overloaded execution semantics handled in some other way.
>
>Perhaps one does not "need" to do it, but it is common-place in
>single-xt+immediate flag systems. That indicates to me that the present
>standard Forth is getting in the way of what programmers want to do.

VFX Forth is at heart a "single-xt+immediate flag" system. There's an
extra header field that was originally only for code generators. Then
it wandered a bit until we discovered that the 94 standard considers
immediacy to be a special case of "non-default compilation semantics"
(NDCS).

At this point we added an NDCS bit and the code generator field either
has a code generator or an NDCS handler. That's the current situation
and we could probably simplify it to a "single-xt+NDCS flag" system.

The second xt in the header is not used by application code, only
by words that deal with code generation and NDCS issues. We made a
house rule that the primary xt is the usual interpretation/execution
action. For COMPILE-ONLY words, the interpretation action is to THROW
or to perform some special case action.

It's not complicated at all. It just required us to read the standard
carefully and to ignore all cries of complexity. Part of the problem
is that the standard probably does not say what the then TC intended
... and I doubt that any standard does. Another part of the problem
for us was that the solution to describing and defining NDCS actions
has to read well.

Read the VFX sources in kernel64.fth and you'll find that it's all
very plain vanilla.

Stephen


--
Stephen Pelc, ste...@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612
web: http://www.mpeforth.com - free VFX Forth downloads

[Stephen Pelc]

On Tue, 06 Oct 2020 13:07:39 GMT, ste...@mpeforth.com (Stephen Pelc)
wrote:

It should also be noted that the Forth 94 committee fell into the
trap of trying to make cmForth standard. This lead to a whole slew
of nasties of which the most horrible was that FIND could return
different xts according to whether the system is interpreting
or compiling.

CMForth was abandoned by its author. The concession by the
committee broke the rule of common practice for adoption into
a standard.

Many introductions that did not have common practice have lead
to problems, including POSTPONE.

[Anton Ertl]

[Fixed formatting: 101 chars of width required]

So:

head.ct corresponds to the comp-xt2 field above.
head.xtptr corresponds to the int field above.
I don't see anything corresponding to comp-xt1 above.

[none albert]

In article <5f7c6d11...@news.eternal-september.org>,

Stephen Pelc <ste...@mpeforth.com> wrote:
>On Tue, 06 Oct 2020 13:07:39 GMT, ste...@mpeforth.com (Stephen Pelc)
>wrote:
>
>It should also be noted that the Forth 94 committee fell into the
>trap of trying to make cmForth standard. This lead to a whole slew
>of nasties of which the most horrible was that FIND could return
>different xts according to whether the system is interpreting
>or compiling.
>
>CMForth was abandoned by its author. The concession by the
>committee broke the rule of common practice for adoption into
>a standard.
>
>Many introductions that did not have common practice have lead
>to problems, including POSTPONE.

So I'm not the only one.

Where POSTPONE is only used by immediate system words that are entitled to
carnal knowledge anyway, I'm going to replace those
POSTPONE DLITERAL
by the shorter
'DLITERAL ,
just to show the proponents it is superfluous.

>
>Stephen

Groetjes Albert
--
This is the first day of the end of your life.
It may not kill you, but it does make your weaker.
If you can't beat them, too bad.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[Ulrich Hoffmann]

Anton Ertl schrieb am Dienstag, 6. Oktober 2020 um 10:16:21 UTC+2:
> Ulrich Hoffmann <ulrich.e...@gmail.com> writes:
...

> >I'm only aware of the original Holon for DOS that uses this approach.
> Where do I find anything about its header structure?

As far as I know, there is no published document describing its header structure but Wolf Wejgaard released the source code at GitHub [1]. Although you typically access the source via Holon's Smalltalk like code browser, the source has also been exported for inspection [2]. There you find:

in STRUKTUR.MTX
> WortEinträge
> /Vorgänger/Nachfolger/alfa/text/dcode/run/comp/flags/class/$name/
> alfa = Link zu vorigem Wort in Alfaliste
> text = Zeiger auf Wortrecord im Textfile (Page)
> dcode = enthält zca oder Datum (32 bit)
> intp = xt von Intpcode
> comp = xt von Compcode
> flags: bit0..9=zCodelänge, bit10..14=xCodelänge, bit15=geladen?
> class = Zeigt auf Klassendefinition
> Der Eintrag wirkt als Actor, je nach State wird Intp- oder Compcode
> ausgeführt.

and in COMPILER.MTX
> : ?Target
> tFind ?dup
> if State if CompFeld else IntpFeld then @s execute
> 2r> 2drop
> then ;

Looks like dual-xt to me.

Regards,
Ulrich

[1] https://github.com/wejgaard/Holonforth-DOS
[2] https://github.com/wejgaard/Holonforth-DOS/tree/master/Holon86/HOST86/Exported%20Modules

[Anton Ertl]

Ulrich Hoffmann <ulrich.e...@gmail.com> writes:
[Holon source code:]
>in STRUKTUR.MTX
>> WortEintr=C3=A4ge
>> /Vorg=C3=A4nger/Nachfolger/alfa/text/dcode/run/comp/flags/class/$name/
>> alfa =3D Link zu vorigem Wort in Alfaliste
>> text =3D Zeiger auf Wortrecord im Textfile (Page)
>> dcode =3D enth=C3=A4lt zca oder Datum (32 bit)
>> intp =3D xt von Intpcode
>> comp =3D xt von Compcode
>> flags: bit0..9=3DzCodel=C3=A4nge, bit10..14=3DxCodel=C3=A4nge, bit15=3D=
>geladen?
>> class =3D Zeigt auf Klassendefinition

>> Der Eintrag wirkt als Actor, je nach State wird Intp- oder Compcode

>> ausgef=C3=BChrt.

>
>and in COMPILER.MTX
>> : ?Target
>> tFind ?dup
>> if State if CompFeld else IntpFeld then @s execute
>> 2r> 2drop
>> then ;
>
>Looks like dual-xt to me.

Yes, it's a pure dual-xt system. The use of "xt" in the documentation
indicates that this version is already influenced by Forth-94, but I
assume that the first version (1989 or so) already had a similar
structure. The first commit on github is from 2016, and it already
talks about xts.

>[1] https://github.com/wejgaard/Holonforth-DOS
>[2] https://github.com/wejgaard/Holonforth-DOS/tree/master/Holon86/HOST86/E=
>xported%20Modules

[none albert]

In article <2020Sep2...@mips.complang.tuwien.ac.at>,

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>Ruvim <ruvim...@gmail.com> writes:
>>On 2020-09-25 21:51, Krishna Myneni wrote:
>>> For example, the following two examples work in my system:
>>>
>>> : t" ['] s" execute ; immediate
>>> t" hello there" type
>>
>>The question is how this word t" works in compilation state.
>>
>>According to the standard, it should be an ambiguous condition.
>
>Which ambiguous condition are you referring to?
>
>In any case, trying it on Gforth, iForth, SwiftForth, and VFX, this
>outputs "hello there".

And even on ciforth.
Also
: test T" aap noot mies" TYPE ;
gives
OK test
aap noot mies
OK

>- anton

[none albert]

In article <rlah4f$qol$1...@dont-email.me>,
Krishna Myneni <krishna...@ccreweb.org> wrote:
<SNIP>

>I'm not saying that STATE should be eliminated, just wondering whether
>or not it is still needed for the types of uses we encounter typically
>in single-xt+immediate flag systems. Additional words to get and set the
>interpretation semantics and compilation semantics of a word are needed.

All this talk about eliminating STATE , seems to gloss over the fact
that state can not be eliminated.
I have demonstrated that one can eliminate STATE, at least in ciforth,
but then still the interpreter is aware of the difference between
compilation and interpretation modes. So in compilation mode
the interpreter compiles LITERAL after any (immediate) word that leaves
something on the stack such as ". ] and [ become messages towards the
interpreter object, to change mode.

This kind of "information hiding" is theoretically attractive, but
in a Forth system, you don't want to hide your tools.
Information hiding is for hiding details, not for hiding how
the very essential feature of Forth that the compiler is extensible
can be invoked.

>Krishna

Groetjes Albert

[none albert]

In article <9e9b9140-15cd-4d6c...@googlegroups.com>,
Bruce McFarling <bruce.m...@gmail.com> wrote:
>On Tuesday, September 29, 2020 at 5:26:59 PM UTC+8, none albert wrote:
>> Can you discuss what this means for the now infamous case of a state-smart
>> ." , in particular postponeing it?
>
>The issue would not be POSTPONE ." but ' ." which is supposed to give
>the "execution" semantics. A POSTPONEment only has to append compilation
>semantics so that the resulting works when compiling. Similarly for
>[COMPILE] which for words with non-standard compilation semantics only
>has to append those compilation semantics so that the resulting word can
>work when compiling.
>
>But a tick has to EXECUTE correctly and COMPILE, correctly. And it could
>be EXECUTEd in interpret state or, if the EXECUTE occurs within an
>IMMEDIATE word executed in compilation state, so it has to EXECUTE
>correctly in interpret state and EXECUTE correctly in compile state.
>
>The thing about ." is that it does not have defined interpretation
>behavior, so whatever it does when interpreting (and whether or not that
>is well advised), the interpretation behavior is specified to be outside
>the scope of the standard. The interpreter can refuse to perform it, the
>word can throw an exception if used in interpret state, it can go ahead
>and try to compile even at the risk of messing things up, and it can do
>what .( does except with a " delimiter.
>
>Or it can play the national anthem of the country within which it is executed.
>
>RFI 0007 which was passed 12Y:0N:1A:1NV clarifies all of the things an
>implementation must see to when adding implementation-specific behavior
>to a word with defined interpretation and compilation semantics:
>(1) It must compile correctly
>(2) It must interpret correctly
>(3) (what it compiles) must execute correctly
>(4) Its tick, when EXECUTEd or COMPILE,d must execute correctly
>(5) [COMPILE] must work correctly with it.
>
>(2) is not constrained for .", for (3) only executing in compilation
>state is constrained, for the tick, for (4) only what it does when
>EXECUTE is performed in compilation state and when the word into which
>it is COMPILE,ed may be executed in compilation state, and for (5) any
>word containing a [COMPILE] with a word with distinct specified
>compilation behavior is only constrained as to what it does when
>compiling.
>
>The problem with S" is that the execution behavior of FILE S" is to use
>the buffer if interpreting and to embed a string which has a ca u
>reference placed on the stack if compiling, and it is impossible to know
>which of the two will be performed by EXECUTE at the time that the tick
>is performed, since it might be executed inside a word that is IMMEDIATE
>and is executing in compilation state. So FILE S" seems to mandate a
>state-smart definition, with execution-time binding of the behavior of
>S", or else some clever way to make an early bound system mimic the
>behavior of a late-bound system.
>
>However, if state-smart definitions are bad, you want to avoid
>execution-time binding of the behavior of S" and would much prefer to
>have straightforward and direct access-time (when interpreting,
>compiling, POSTPONEing, [COMPILE]ing or ticking) binding of the behavior
>of S". You don't WANT to mimic the behavior of a late-binding system,
>because that is the behavior that is causing the problems you are trying
>to avoid.
>
>And it seemed to be the intent of the TC to permit such early-bound systems.
>
>It might be argued that such early bound systems should not implement
>FILE S" because their xt's when ticked are not capable of EXECUTEing '
>S" correctly in both states. But that is a far from satisfactory state
>of affairs.

I sidestepped this issue by just having S" always compiling a skip and
storing a permanent string at HERE , compilation mode or not.
This is one of those small incompatibilities because S" in interpretation
mode is not supposed to eat dictionary space.
This is why ciforth is called ciforth, (close to iso).
(In an answer to Anton Ertl, I explained that one can use ALLOCATE
to the same effect, and stay totally in line.)

It wastes space, but I never regretted it. Since that decision,
my memory has grown from 32 Mbyte to 32 Gbyte.

>
>Virtually,
>BruceMcF, Beijing

[dxforth]

On 7/10/2020 21:06, albert wrote:
> ...

> This is why ciforth is called ciforth, (close to iso).

Would that be 'International Space Order'? The logo was emblazoned
on the helmets of astronauts in a sci-fi flick I watched last night.
Not to draw too fine a point, Forth too is looking increasingly like
a bad B movie starring old actors in the twilight of their career.

[Anton Ertl]

albert@cherry.(none) (albert) writes:
>I sidestepped this issue by just having S" always compiling a skip and
>storing a permanent string at HERE , compilation mode or not.
>This is one of those small incompatibilities because S" in interpretation
>mode is not supposed to eat dictionary space.

Core S" has undefined interpretation semantics. An S" that implements
the interpretation semantics to be the same as the compilation
semantics (i.e., an IMMEDIATE S") is certainly a valid implementation
of that. Note that you should not claim that ciforth provides S" from
the File Access word set, nor that it is "providing the File
Access word set".

[me]

I ran a test script for the infamous state-smart 2DUP example and
found that FigForth had correct behavior for all ten cases in
section 2.0 that alleged that state-smart would fail for four of the
cases.

I ran another test script for the specific cases in sections 2.1,
2.2 2.3 and 2.4 and found a state-smart 2dup could be used correctly
for each case providing the code was made suitable for a word having
composite semantics. The alleged problems were not with state-smart
but in coding that was not applicable to a word of composite
semantics. It is a silly notion that a word should be able to
replace another word of different coding and to have the same behavior
in all cases without adaption.

In short, state-smart works quite well in FigForth; no exorcism
needed.

I also simulated building and running words of composite semantics.
( I use the term composite semantics as it includes dual semantics
and also possible dual semantics with shared common semantics. ) I
found a case, a dual semantic literal, where a state check was
needed to be placed in the run semantics because which semantics to
use had to be made before it knew what state it would eventually be
running in. Neither was checking precedence of any help in this
case. Nor would dual words fare any better because to use FOO or
[FOO] at this point cannot be determined. This is even more
problematic for words that must parse and decide if to compile the
content or not when it doesn't know if the word in which it's used
will want the content collected and compiled at compile time or just
typed.

Use of any word with composite semantics will have to be used with
care. State-smart provides an easy and flexible solution.

Don't take any of the above as gospel, I'm no expert on this. Just
relating what I've done and what I see at the present. If it's all
wrong, it won't be the first time nor last. I shall now be quite
on this subject and endeavor to not add any more noise.

--
me

[Krishna Myneni]

On 10/7/20 4:46 AM, albert wrote:
> In article <rlah4f$qol$1...@dont-email.me>,
> Krishna Myneni <krishna...@ccreweb.org> wrote:
> <SNIP>
>> I'm not saying that STATE should be eliminated, just wondering whether
>> or not it is still needed for the types of uses we encounter typically
>> in single-xt+immediate flag systems. Additional words to get and set the
>> interpretation semantics and compilation semantics of a word are needed.
>
> All this talk about eliminating STATE , seems to gloss over the fact

> that state can not be eliminated. ...

It's weird that you clipped my sentence explicitly stating that my
discussion is not about eliminating STATE and then go on to say, "all
this talk about eliminating STATE ..."

No one in this discussion is talking about eliminating STATE -- the
discussion is only about reducing the number of words which need to use
STATE and therefore have problems with ' (tick), ['], and POSTPONE.

KM

[Krishna Myneni]

On 10/7/20 4:46 AM, albert wrote:

> In article <rlah4f$qol$1...@dont-email.me>,
> Krishna Myneni <krishna...@ccreweb.org> wrote:
> <SNIP>
>> I'm not saying that STATE should be eliminated, just wondering whether
>> or not it is still needed for the types of uses we encounter typically
>> in single-xt+immediate flag systems. Additional words to get and set the
>> interpretation semantics and compilation semantics of a word are needed.
>
> All this talk about eliminating STATE , seems to gloss over the fact
> that state can not be eliminated.

[Ulrich Hoffmann]

Anton Ertl schrieb am Mittwoch, 7. Oktober 2020 um 11:27:21 UTC+2:
> Ulrich Hoffmann <ulrich.e...@gmail.com> writes:
> [Holon source code:]

...

> >and in COMPILER.MTX
> >> : ?Target
> >> tFind ?dup
> >> if State if CompFeld else IntpFeld then @s execute
> >> 2r> 2drop
> >> then ;
> >
> >Looks like dual-xt to me.
> Yes, it's a pure dual-xt system. The use of "xt" in the documentation
> indicates that this version is already influenced by Forth-94, but I
> assume that the first version (1989 or so) already had a similar
> structure.

Right - a pure dual-xt system. What other systems use this implementation technique?

> The first commit on github is from 2016, and it already talks about xts.

The reason why is that that source was proprietary for a long time and only
then was published more or less unchanged as I understand.

> >[1] https://github.com/wejgaard/Holonforth-DOS

Regards,
Ulrich

[a...@littlepinkcloud.invalid]

albert <albert@cherry.(none)> wrote:
> In article <rlah4f$qol$1...@dont-email.me>,
> Krishna Myneni <krishna...@ccreweb.org> wrote:
> <SNIP>
>>I'm not saying that STATE should be eliminated, just wondering whether
>>or not it is still needed for the types of uses we encounter typically
>>in single-xt+immediate flag systems. Additional words to get and set the
>>interpretation semantics and compilation semantics of a word are needed.
>
> All this talk about eliminating STATE , seems to gloss over the fact
> that state can not be eliminated.
> I have demonstrated that one can eliminate STATE, at least in ciforth,
> but then still the interpreter is aware of the difference between
> compilation and interpretation modes.

Not exactly, no: there's an interpreter and a compiler. They are
completely unaware of each other's existence. Each only knows how to
do its own job: interpreter interprets, compiler compiles.

> So in compilation mode the interpreter compiles LITERAL after any
> (immediate) word that leaves something on the stack such as ". ] and
> [ become messages towards the interpreter object, to change mode.

They shouldn't be messages: ] is the name of the compiler, and it gets
executed.

> This kind of "information hiding" is theoretically attractive, but
> in a Forth system, you don't want to hide your tools.

Sure, but there's a huge difference between exposing an
implementation, which you should do, and specifying a language.

Andrew.

[Anton Ertl]

Ulrich Hoffmann <ulrich.e...@gmail.com> writes:
>Anton Ertl schrieb am Mittwoch, 7. Oktober 2020 um 11:27:21 UTC+2:
>> I
>> assume that the first version (1989 or so) already had a similar
>> structure.

...

>> The first commit on github is from 2016, and it already talks about xts.
>The reason why is that that source was proprietary for a long time and only
>then was published more or less unchanged as I understand.

Yes, and Wolf Wejgaard probably did not use CVS or something, so he
could not just convert his repository to git like I have done for some
projects.

BTW, Gforth's git only reflects the history after we started with CVS.
We have 2 years of using RCS before that, but don't know how to tack
that in front of the existing git history now. Does anybody have an
idea?

[a...@littlepinkcloud.invalid]

Krishna Myneni <krishna...@ccreweb.org> wrote:
>
> No one in this discussion is talking about eliminating STATE -- the
> discussion is only about reducing the number of words which need to use
> STATE and therefore have problems with ' (tick), ['], and POSTPONE.

I'll grant you this is problematic when implementing standard Forth
because of words like S".

But when writing Forth applications there is no problem at all. No
word needs to use STATE. To eliminate STATE, find every word which
uses STATE and remove its use. This in turn removes the need for
dial-XT systems, etc.

Andrew.

[none albert]

In article <2020Oct...@mips.complang.tuwien.ac.at>,

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>Ulrich Hoffmann <ulrich.e...@gmail.com> writes:
>>Anton Ertl schrieb am Mittwoch, 7. Oktober 2020 um 11:27:21 UTC+2:
>>> I
>>> assume that the first version (1989 or so) already had a similar
>>> structure.
>...
>>> The first commit on github is from 2016, and it already talks about xts.
>>The reason why is that that source was proprietary for a long time and only
>>then was published more or less unchanged as I understand.
>
>Yes, and Wolf Wejgaard probably did not use CVS or something, so he
>could not just convert his repository to git like I have done for some
>projects.
>
>BTW, Gforth's git only reflects the history after we started with CVS.
>We have 2 years of using RCS before that, but don't know how to tack
>that in front of the existing git history now. Does anybody have an
>idea?

I used RCS before CVS, and there is no conversion because it uses
the same data format. So you just could have started CVS with your
RCS files, like I did. I will never convert from cvs/rcs format
to git, because the git format cannot guarantee that you can
recover any old version (and it is too complicated in my book.)
I fear that many valuable archives will be lost due to the usage of git.
Look at the oldest versions of the Koran, most pages are lost, but those
that are not, are at least partly readable. Now imagine the first Koran
was saved using git.

The best thing would be to start with the old RCS, store the git files
on top, and from then on use CVS.

>
>- anton

[none albert]

In article <jKOdnVLtWcqzf-PC...@supernews.com>,

Also see my proposal for systems where STATE is only used inside
] [ and INTERPRET.

To elaborate on your example. The real problem with e.g. ' is that
it parses ahead in the input stream. So in fact
: ' PARSE-NAME FOUND ;
By always splitting off PARSE-NAME whenever you're inclined to use
' the problem goes away.

>
>Andrew.

[none albert]

In article <jKOdnVPtWcqpfOPC...@supernews.com>,

<a...@littlepinkcloud.invalid> wrote:
>albert <albert@cherry.(none)> wrote:
>> In article <rlah4f$qol$1...@dont-email.me>,
>> Krishna Myneni <krishna...@ccreweb.org> wrote:
>> <SNIP>
>>>I'm not saying that STATE should be eliminated, just wondering whether
>>>or not it is still needed for the types of uses we encounter typically
>>>in single-xt+immediate flag systems. Additional words to get and set the
>>>interpretation semantics and compilation semantics of a word are needed.
>>
>> All this talk about eliminating STATE , seems to gloss over the fact
>> that state can not be eliminated.
>> I have demonstrated that one can eliminate STATE, at least in ciforth,
>> but then still the interpreter is aware of the difference between
>> compilation and interpretation modes.
>
>Not exactly, no: there's an interpreter and a compiler. They are
>completely unaware of each other's existence. Each only knows how to
>do its own job: interpreter interprets, compiler compiles.

That could be implemented, and as far as I remember, the implementations
were very contorted. Having a separate interpreter and compiler is just
one of the many ways a system can handle this. Still the system is
in a different state while the interpreter is running then when
the compiler is running. We all know what state actually means:
a different reaction to the same events. Take the even that
the input stream contains "DROP". QED.

>
>> So in compilation mode the interpreter compiles LITERAL after any
>> (immediate) word that leaves something on the stack such as ". ] and
>> [ become messages towards the interpreter object, to change mode.
>
>They shouldn't be messages: ] is the name of the compiler, and it gets
>executed.

Why would you impose an implementation on me? There surely is a Forth
in smalltalk that works in this way, using messages. Your reasoning
doesn't convince me that a Forth in smalltalk couldn't possibly be standard.

<SNIP>

>
>Andrew.

[Krishna Myneni]

On 10/8/20 4:55 AM, a...@littlepinkcloud.invalid wrote:
> Krishna Myneni <krishna...@ccreweb.org> wrote:
>>
>> No one in this discussion is talking about eliminating STATE -- the
>> discussion is only about reducing the number of words which need to use
>> STATE and therefore have problems with ' (tick), ['], and POSTPONE.
>
> I'll grant you this is problematic when implementing standard Forth
> because of words like S".
>

Also an optimized "TO" which works in both interpretation and
compilation state and which can be ticked or postponed -- in general,
parsing words which have immediate precedence in a standard system, I
think, will have the same problems.

> But when writing Forth applications there is no problem at all. No
> word needs to use STATE. To eliminate STATE, find every word which
> uses STATE and remove its use. This in turn removes the need for
> dial-XT systems, etc.
>

Using S" in interpretation mode is quite common within an application,
for declaring a set of strings. And then one needs a null-terminated
string when working with C library interfaces, so I often define the
word ' Z" ' by postponing S" within its definition and making it
immediate -- no STATE is needed to do this. So it does become necessary
to invoke STATE at some point within a definition when we want to define
words such as Z" within a standard single xt+immediate flag system.

What can we gain with dual-xt systems? Here's my understanding at present:

1. We can eliminate IMMEDIATE while being able to define it if one needs
backwards compatibility.

2. We can write "dual words" which now would be written by invoking
STATE in a single xt+immediate system, without the need for STATE and,
hence, avoid all problems related to tick, ['], and POSTPONE.
Furthermore, the run-time semantics are defined at the time of
definition rather than during execution.

3. We will be able to use ' (tick), ['], and POSTPONE without ambiguity,
allowing for new Forth idioms which presently would break across
systems. The value of such new Forth expressions are not entirely clear
to me at this point, but I don't rule them out.


Krishna











> Andrew.
>

[Anton Ertl]

Krishna Myneni <krishna...@ccreweb.org> writes:
>3. We will be able to use ' (tick), ['], and POSTPONE without ambiguity,
>allowing for new Forth idioms which presently would break across
>systems.

If you look at systems like Gforth, VFX, and SwiftForth, there is not
much STATE-smartness there, and not much that breaks on them even now:

* Gforth has eliminated STATE-smartness in 1996.

* VFX has eliminated STATE-smartness also a long time ago. VFX had
some words where COMPILE,ing th xt would do the wrong thing, but
AFAIK this is fixed in VFX 5.

* SwiftForth has not that many STATE-smart words (e.g., no STATE-smart .").

>The value of such new Forth expressions are not entirely clear
>to me at this point, but I don't rule them out.

They are not new. If nobody had used them before, nobody would have
tripped over STATE-smart words, and we would not be having this
discussion.

The value is that you don't need to worry about STATE when performing
a Forth word, whichever way you perform it.

[none albert]

In article <2020Oct...@mips.complang.tuwien.ac.at>,
Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:

>Krishna Myneni <krishna...@ccreweb.org> writes:
>>3. We will be able to use ' (tick), ['], and POSTPONE without ambiguity,
>>allowing for new Forth idioms which presently would break across
>>systems.
>
>If you look at systems like Gforth, VFX, and SwiftForth, there is not
>much STATE-smartness there, and not much that breaks on them even now:
>
>* Gforth has eliminated STATE-smartness in 1996.
>
>* VFX has eliminated STATE-smartness also a long time ago. VFX had
> some words where COMPILE,ing th xt would do the wrong thing, but
> AFAIK this is fixed in VFX 5.
>
>* SwiftForth has not that many STATE-smart words (e.g., no STATE-smart .").
>

* ciforth has only ." as a state-smart word as a concession to compiling
very old programs.
ciforth discourages the use of any words that parses the input stream
unless it results in a literal where the intended behaviour in compile
mode is obvious (like $123 'a' etc.). 1] So in newer programs you will
find
"AAP" TYPE
instead of
." AAP" / .( AAP)

There is an unfortunate exception with [DEFINED] . It is used to add
words, possibly string handling ones, to a minimal system.
It cannot possibly use string defining words, so it has to parse the
input stream itself.

<SNIP>
>
>- anton

1] In most Forths this is handled by the interpreter itself, and would not
even be called a parsing of the input stream.

[Krishna Myneni]

On 10/9/20 3:27 AM, Anton Ertl wrote:
> Krishna Myneni <krishna...@ccreweb.org> writes:
>> 3. We will be able to use ' (tick), ['], and POSTPONE without ambiguity,
>> allowing for new Forth idioms which presently would break across
>> systems.
>

...

>> The value of such new Forth expressions are not entirely clear
>> to me at this point, but I don't rule them out.
>
> They are not new. If nobody had used them before, nobody would have
> tripped over STATE-smart words, and we would not be having this
> discussion.
>

...

I'm considering the more exotic cases when we switch back and forth
between interpretation and compilation states within a word. But, yes,
that's a valid point. It would be useful to see a large set of examples
which currently have different behavior under existing Forth systems,
and which would have unambiguous behavior within a dual-xt system.

KM

[A. K.]

It would be far easier to agree on a set of test cases. For instance
Standard draft 18.1 comprises extracts from the test suite plus some tests for
some newer words.

When a Forth system passes the tests, then it should be good enough.
Whoever wants to cover all exotic corner cases on top of that, they shall
be declared 'application specific'.

Otherwise these discussions will go on for another half century.
I can't even yawn anymore about state 'smartness'.

[Ruvim]

On 2020-10-03 21:49, Krishna Myneni wrote:
> On 10/3/20 11:25 AM, Ruvim wrote:
>> On 2020-10-03 15:16, Krishna Myneni wrote:
>>
>>> 2) All ambiguous conditions for use of ' (tick), ['],  and POSTPONE
>>> with any standard word or user-defined colon definitions, in either
>>> compilation or interpretation state, may be eliminated,


>> The ambiguous conditions concerning POSTPONE can be removed without
>> any additional explicit requirements to implementations. POSTPONE can
>> be correctly implemented in any approach.
>>
>
> Ok. That's fair, considering my system(s) are single-xt + immediate
> flag, and I believe POSTPONE works per the standard on them.

I think, it works per the current standard, that has an ambiguous
condition concerning POSTPONE. But not for the unambiguous variant.
Although, it also can be adapted.



>> If the standard requires dual-xt and disallows single-xt, it is still
>> not enough to meet the points 1,2,3.
>>
>> 1. IMMEDIATE is required for back-compatibility of programs.
>> Otherwise, such standard should de-standardize some programs and
>> provide another alternative mechanism.
>>
>
> IMMEDIATE should be definable in Forth source, if needed, in a dual-xt
> system -- see my reply to Anton above.

It some approaches it's not too easy to get this information back, i.e.
to implement SEARCH-WORDLIST that returns the original immediacy flag.


>> 2. To eliminate ambiguous conditions concerning Tick, it should
>> specify execution semantics for dual words, and for words with
>> undefined interpretation semantics (with corresponding rewording in
>> some places). Otherwise these ambiguous conditions still remain.
>>
> Sure, it should be possible to set either interpretation semantics or
> compilation semantics to a NO-OP.

Such approach is error prone too much, I think.
And it tights an implementation also too much.


>> OTOH, if the standard requires single-xt and disallows dual-xt, all
>> the ambiguous conditions concerning Tick are eliminated. But the
>> execution semantics in interpretation state for some words remain
>> implementation defined.
>>
>
> Your implication is that a dual-xt standard still can have ambiguities
> about the behavior of a word in interpretation state and/or compilation
> state. I'm not sure why that would be the case. Can you give an example?


There are many ways to implement dual-xt approach. E.g.: the cmForth
approach (dual-nt in the different word lists), the approach of the
system of Mark William Humphries (dual-nt with the different flags), the
approach of Gforth (single-nt with dual-xt).


And at least in some of them Tick cannot guarantee a meaningful result
when it's applied to a word with undefined interpretation semantics.

It means that without fixing more implementation details, it cannot
specify, e.g., the behavior of «' IF» phrase. And then it's an
ambiguous condition.


--
Ruvim

[Ruvim]

On 2020-10-06 16:19, Stephen Pelc wrote:
> On Tue, 06 Oct 2020 13:07:39 GMT, ste...@mpeforth.com (Stephen Pelc)
> wrote:
>
> It should also be noted that the Forth 94 committee fell into the
> trap of trying to make cmForth standard. This lead to a whole slew
> of nasties of which the most horrible was that FIND could return
> different xts according to whether the system is interpreting
> or compiling.
>
> CMForth was abandoned by its author. The concession by the
> committee broke the rule of common practice for adoption into
> a standard.

cmForth approach is actually very useful. E.g., it is used in the cross
compilers.

In addition, thanks to allowing this approach in a standard system,
along with the classic single-xt+immediate-flag approach, we have the
elegant language that describes the different semantics, and we have
dual-xt Forth systems.

As I see it, if TC had abandoned cmForth, we only had
single-xt+immediate-flag systems.

> Many introductions that did not have common practice have lead
> to problems, including POSTPONE.

If the specification of POSTPONE was tighter, and guaranteed that the
generated code has its behavior in interpretation state the same as in
the compilation state — what problem we could have? I think, we did not
have any problems.



--
Ruvim

[peter....@gmail.com]

I have had a dual-xt system for over 20 years. None of your points above
was behind my decision to develop it. As I was developing a code generator
I needed to execute the code generator for each word while compiling.
I started with a CMFORTH style compiling vocabulary. That worked fine until
I wanted to use vocabularies also for its intended purpose. From there on
the dual-xt was implemented. It is only an implementation detail, a user of
my system will not notice that it has dual-xts.

That it also were easier to implement S" TO etc was discovered later.

I would never eliminate IMMEDIATE! That is the way to set the int-xt as
also the comp-xt

My new 64 bit system has today a single-xt but will probably go to
triple-xts when I finish the code generator.

BR
Peter

[Stephen Pelc]

On Fri, 9 Oct 2020 20:01:32 +0300, Ruvim <ruvim...@gmail.com>
wrote:

>cmForth approach is actually very useful. E.g., it is used in the cross
>compilers.

It was used by *some* cross-compilers (including MPE's) from the
early 1980s onwards, i.e. the approach was used *before* cmForth.

Having used both solutions, in designing a new cross-compiler, I found
myself sticking to the single xt and NDCS flag approach. Why? Because
it is simpler and easier to understand.

Dual xt (as opposed to dual behaviour) is just one implementation
approach. What is important in dual behaviour is the realisation
that IMMEDIATE and non-default compilation semantics are not the
same thing.

>As I see it, if TC had abandoned cmForth, we only had
>single-xt+immediate-flag systems.

No VFX has never (and I hope never will) return different xts in
interpretation and compilation states.

>> Many introductions that did not have common practice have lead
>> to problems, including POSTPONE.
>
>If the specification of POSTPONE was tighter, and guaranteed that the
>generated code has its behavior in interpretation state the same as in
>the compilation state — what problem we could have? I think, we did not
>have any problems.

The big problem with POSTPONE is that nobody understood it, until
people claimed that they did. The amount of verbiage caused by
POSTPONE is amazing, whereas introductions with common use (in other
languages) such as CATCH and THROW have generated little other than
surprise.

Stephen


--
Stephen Pelc, ste...@vfxforth.com <<< NEW
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612
web: http://www.mpeforth.com - free VFX Forth downloads

[Ruvim]

On 2020-10-09 20:39, Stephen Pelc wrote:
[...]

> Dual xt (as opposed to dual behaviour) is just one implementation
> approach.

So can you agree that the words that are immediate and STATE-dependent,
are essentially dual-behavior words?

> What is important in dual behaviour is the realisation
> that IMMEDIATE and non-default compilation semantics are not the
> same thing.

Yes. An immediate word is certainly a word with non-default compilation
semantics, but if a word has non-default compilation semantics it
doesn't mean that this word is necessarily immediate.

In other words, compilation semantics of immediate words are just an
instance of non-default compilation semantics. And it's a single
instance that is available for a standard program. Other variants are
totally implementation dependent.


In the same time, optimization is not an instance of any compilation
semantics (neither non-default, nor default). Optimization takes place
under the hood, and a program (or a user) doesn't need to know anything
in this concern.

>> As I see it, if TC had abandoned cmForth, we only had
>> single-xt+immediate-flag systems.
>
> No VFX has never (and I hope never will) return different xts in
> interpretation and compilation states.

It just means that VFX is based on the single-xt approach.

In a system that utilizes the dual-xt approach, FIND should return the
different results in interpretation and compilation states for some
words at least.

>>> Many introductions that did not have common practice have lead
>>> to problems, including POSTPONE.
>>
>> If the specification of POSTPONE was tighter, and guaranteed that the
>> generated code has its behavior in interpretation state the same as in
>> the compilation state — what problem we could have? I think, we did not
>> have any problems.
>
> The big problem with POSTPONE is that nobody understood it, until
> people claimed that they did. The amount of verbiage caused by
> POSTPONE is amazing, whereas introductions with common use (in other
> languages) such as CATCH and THROW have generated little other than
> surprise.

POSTPONE is a tool of metaprogramming. And meta-programming is a
difficult thing by itself. So POSTPONE cannot be as simple as DUP.

But it could be easier than it was.
I can imagine something like the following.


Interpretation:
Parse /name/, find /name/. Perform the run-time semantics
given bellow.

Compilation:
Parse /name/, find /name/. Append the run-time semantics
given bellow to the current definition.

Run-time:
Perform the compilation semantics for /name/
(that shall be the same independently of
the current state).


These consistent interpretation semantics allow people to test and learn
how POSTPONE works.


Conceptually,

: foo bar baz ;

is equal to

: foo [ ' bar compile, ' baz compile, ] ;

(where bar and baz are ordinary words)
(NB: immediate words are also non-ordinary)

that is equal to

: foo [ postpone bar postpone baz ] ;

(where bar and baz now may be any words, including any non-ordinary words)

that in turn is equal to

: foo, postpone bar postpone baz ;

: foo [ foo, ] ;

and that is equal to

: [foo,] postpone bar postpone baz ; immediate

: foo [foo,] ;



--
Ruvim

[a...@littlepinkcloud.invalid]

Krishna Myneni <krishna...@ccreweb.org> wrote:
> On 10/8/20 4:55 AM, a...@littlepinkcloud.invalid wrote:
>> Krishna Myneni <krishna...@ccreweb.org> wrote:
>>>
>>> No one in this discussion is talking about eliminating STATE -- the
>>> discussion is only about reducing the number of words which need to use
>>> STATE and therefore have problems with ' (tick), ['], and POSTPONE.
>>
>> I'll grant you this is problematic when implementing standard Forth
>> because of words like S".
>>
>
> Also an optimized "TO" which works in both interpretation and
> compilation state and which can be ticked or postponed

I'm equally sure that no application actually needs to tick or
postpone TO.

> What can we gain with dual-xt systems? Here's my understanding at present:
>
> 1. We can eliminate IMMEDIATE while being able to define it if one needs
> backwards compatibility.
>
> 2. We can write "dual words" which now would be written by invoking
> STATE in a single xt+immediate system, without the need for STATE and,
> hence, avoid all problems related to tick, ['], and POSTPONE.

This is of course true, but it makes no difference to my point. This
is a way to do neat stuff, but the actual value of the stuff is not
clear. I'm not denying that if you really want to be able to write
"dual words" then dual-XT is a neat way to do it.

> The value of such new Forth expressions are not entirely clear
> to me at this point, but I don't rule them out.

Andrew.

[a...@littlepinkcloud.invalid]

albert <albert@cherry.(none)> wrote:
> In article <jKOdnVPtWcqpfOPC...@supernews.com>,
> <a...@littlepinkcloud.invalid> wrote:
>>albert <albert@cherry.(none)> wrote:
>>> In article <rlah4f$qol$1...@dont-email.me>,
>>> Krishna Myneni <krishna...@ccreweb.org> wrote:
>>> <SNIP>
>>>>I'm not saying that STATE should be eliminated, just wondering whether
>>>>or not it is still needed for the types of uses we encounter typically
>>>>in single-xt+immediate flag systems. Additional words to get and set the
>>>>interpretation semantics and compilation semantics of a word are needed.
>>>
>>> All this talk about eliminating STATE , seems to gloss over the fact
>>> that state can not be eliminated.
>>> I have demonstrated that one can eliminate STATE, at least in ciforth,
>>> but then still the interpreter is aware of the difference between
>>> compilation and interpretation modes.
>>
>>Not exactly, no: there's an interpreter and a compiler. They are
>>completely unaware of each other's existence. Each only knows how to
>>do its own job: interpreter interprets, compiler compiles.
>
> That could be implemented, and as far as I remember, the
> implementations were very contorted.

That's not what I remember.

>>> So in compilation mode the interpreter compiles LITERAL after any
>>> (immediate) word that leaves something on the stack such as ". ] and
>>> [ become messages towards the interpreter object, to change mode.
>>
>>They shouldn't be messages: ] is the name of the compiler, and it gets
>>executed.
>
> Why would you impose an implementation on me?

I'm not imposing anything: I am saying that IMVHO it's the right
Forthly thing to do.

> There surely is a Forth in smalltalk that works in this way, using
> messages. Your reasoning doesn't convince me that a Forth in
> smalltalk couldn't possibly be standard.

You can drive a nail with a screwdriver if you want.

What do standards have to do with this, anyway? Nothing AFAICS.

Andrew.

[Anton Ertl]

a...@littlepinkcloud.invalid writes:
>I'm equally sure that no application actually needs to tick or
>postpone TO.

I dimly remember wanting to POSTPONE TO in objects2. I then worked
around the restriction not to do that, so as far as "actually needs",
you are right.

[a...@littlepinkcloud.invalid]

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> a...@littlepinkcloud.invalid writes:
>>I'm equally sure that no application actually needs to tick or
>>postpone TO.
>
> I dimly remember wanting to POSTPONE TO in objects2. I then worked
> around the restriction not to do that, so as far as "actually needs",
> you are right.

With regard to locals, POSTPONE TO is very awkward. I'm not at all
sure what it would really mean. However,

S" to x" evaluate

is well-defined in a Standard Forth System, so POSTPONE TO isn't
needed for any application. AFAICS...

Andrew.

[Anton Ertl]

a...@littlepinkcloud.invalid writes:
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>> a...@littlepinkcloud.invalid writes:
>>>I'm equally sure that no application actually needs to tick or
>>>postpone TO.
>>
>> I dimly remember wanting to POSTPONE TO in objects2.

My memory is getting brighter (thanks to
<2015Jan...@mips.complang.tuwien.ac.at>). There are two
potential ways to implement THIS (and this is already relevant for
objects.fs
<http://www.complang.tuwien.ac.at/forth/objects/objects.fs>)

1) Implement THIS as a local. This is in many ways the better approach,
but cannot be used in a satisfactory way on, e.g., VFX. If THIS was
implemented as a local, then we could just write

: m: :noname s" this" (local) ;

There would be no need for EXITM, ;M, or for redefining CATCH.

2) Implement THIS as a value (or, on multitasking systems, a uvalue).
Then we have to write

0 value this
: to-this to this ;

: m: :noname POSTPONE this POSTPONE >r POSTPONE to-this ;

and define exitm, ;m, and redefine catch to restore THIS. One problem
with this approach is that non-inlining Forth systems perform an extra
call to TO-THIS.

Case 2 is the one that's interesting here. Allowing POSTPONE TO does
not help in a satisfactory way. You could eliminate the extra call at
run-time by writing it as

: comp-to postpone to ;

: m: :noname POSTPONE this POSTPONE >r s" this" ['] comp-to execute-parsing ;

But this relies on the intended THIS being visible in the search order
when M: is called (similar to using EVALUATE, see below). However, in
Gforth with the to-recognizer we can write

: m: :noname POSTPONE this POSTPONE >r POSTPONE ->this ;

or shorter:

: m: :noname ]] this >r ->this [[ ;

Bottom line: POSTPONE TO does not really help here. And it's probably
better to just go with approach 1 (using a local THIS). And if
somebody has a problem with the limitations of his Forth system, they
should complain to the implementor of that system.

>With regard to locals, POSTPONE TO is very awkward. I'm not at all
>sure what it would really mean.

If the sentence about "ambiguous condition" was not there, it would be
well-defined. I even gave an example in a reply
<2018May...@mips.complang.tuwien.ac.at> to one of your postings.

>However,
>
> S" to x" evaluate
>
>is well-defined in a Standard Forth System

But it introduces dependencies on the current search order, on the
contents of the wordlists in the current search order, and on STATE.

[a...@littlepinkcloud.invalid]

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> a...@littlepinkcloud.invalid writes:
>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>> a...@littlepinkcloud.invalid writes:
>>>>I'm equally sure that no application actually needs to tick or
>>>>postpone TO.
>>>
>>> I dimly remember wanting to POSTPONE TO in objects2.
>

>>However,
>>
>> S" to x" evaluate
>>
>>is well-defined in a Standard Forth System
>
> But it introduces dependencies on the current search order, on the
> contents of the wordlists in the current search order, and on STATE.

OK, I don't get this. What is the dependency on STATE? And I don't
quite see how there's a dependency on the contents of the wordlists in
the current search order, unless someone has been foolish enough to
redefine TO, in which case you no longer have a Standard Forth System.

Andrew.

[a...@littlepinkcloud.invalid]

a...@littlepinkcloud.invalid wrote:
> Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>> a...@littlepinkcloud.invalid writes:
>>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>> a...@littlepinkcloud.invalid writes:
>>>>>I'm equally sure that no application actually needs to tick or
>>>>>postpone TO.
>>>>
>>>> I dimly remember wanting to POSTPONE TO in objects2.
>>
>>>However,
>>>
>>> S" to x" evaluate
>>>
>>>is well-defined in a Standard Forth System
>>
>> But it introduces dependencies on the current search order, on the
>> contents of the wordlists in the current search order, and on STATE.
>
> OK, I don't get this. What is the dependency on STATE?

Sorry, I do get this. EVALUATE has an implicit dependency on
STATE.

Andrew.

[Anton Ertl]

a...@littlepinkcloud.invalid writes:
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>> a...@littlepinkcloud.invalid writes:

>>> S" to x" evaluate
...

>And I don't
>quite see how there's a dependency on the contents of the wordlists in
>the current search order, unless someone has been foolish enough to
>redefine TO, in which case you no longer have a Standard Forth System.

Whether the result is not a standard system is irrelevant; this is
allowed in a standard program.

The other dependency on the contents of the wordlists in the search
order is about the word X.

<http://www.complang.tuwien.ac.at/forth/why-evaluate-is-bad> in only
25 years old.

[a...@littlepinkcloud.invalid]

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> a...@littlepinkcloud.invalid writes:
>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>> a...@littlepinkcloud.invalid writes:
>>>> S" to x" evaluate
> ...
>>And I don't quite see how there's a dependency on the contents of
>>the wordlists in the current search order, unless someone has been
>>foolish enough to redefine TO, in which case you no longer have a
>>Standard Forth System.
>
> Whether the result is not a standard system is irrelevant; this is
> allowed in a standard program.

I see. However, this is exactly what I mean by "application need." The
natural Forth practitioner's response to "What if TO has been
redefined?" is "Don't do that, then," and quite rightly so. Forth was
never intended for ivory-tower academics. It's just not that kind of
language.

> The other dependency on the contents of the wordlists in the search
> order is about the word X.

But in this dicussion X is a local, and may not even be in a wordlist.
The only context in which S" TO X" EVALUATE is executed must be while
the definition in which X occurs is being compiled.

> <http://www.complang.tuwien.ac.at/forth/why-evaluate-is-bad> in only
> 25 years old.

See above.

Andrew.

[Anton Ertl]

a...@littlepinkcloud.invalid writes:
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>> a...@littlepinkcloud.invalid writes:
>>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>> a...@littlepinkcloud.invalid writes:
>>>>> S" to x" evaluate
>> ...
>>>And I don't quite see how there's a dependency on the contents of
>>>the wordlists in the current search order, unless someone has been
>>>foolish enough to redefine TO, in which case you no longer have a
>>>Standard Forth System.
>>
>> Whether the result is not a standard system is irrelevant; this is
>> allowed in a standard program.
>
>I see. However, this is exactly what I mean by "application need." The
>natural Forth practitioner's response to "What if TO has been
>redefined?" is "Don't do that, then," and quite rightly so. Forth was
>never intended for ivory-tower academics. It's just not that kind of
>language.

That's contrafactual. Most languages designed by academics (e.g.,
Algol 60, Pascal, Haskell) have reserved words, and disallow defining
the same more than once. Forth is not that kind of language. It does
not have reserved words, and allows defining words with the same name
multiple times.

And defining standard names in a non-standard way is not at all
unusual, actually even standard systems do it; e.g., SwiftForth, VFX,
and Gforth on IA-32/AMD64 come with words called AND that implement
the core word AND, as well as with words called AND that behave
completely differently.

Blaming the user with the flimsiest (or, in this case, contrafactual)
supporting argument is a frequent behaviour of programmers whose
programs or programming practices fail in some circumstances. In the
Forth world "Don't do that" is a frequent way to express that.

In most cases, there is a way to make the use of EVALUATE robust
against search order and wordlist content problems: Define synonyms in
a wordlist for the words that are used in the string, then put that
wordlist on the top of the search order, then perform the EVALUATE.

In the present case, where you want to perform the compilation
semantics of TO, the not-quite Forth-94 way to do it would be

get-current wordlist dup constant eval-private set-current
: to postpone to ; immediate
\ ... possibly the same for x
set-current

: to-x
get-order eval-private swap 1+ set-order
\ ... also make STATE-robust
s" to x" evaluate
\ ... restore STATE
previous ;

and the use TO-X instead of S" TO X" EVALUATE.

However, because POSTPONE TO is ambiguous, this code is not Forth-94
(nor Forth-2012) compliant. So here we have an example where POSTPONE
TO would be useful.

In Forth-2012 we can use SYNONYM instead of the definition of TO
above.

In any case, the cumbersomeness of this workaround suggests that maybe
we should look for a way that does not involve EVALUATE.

However, if X is a local, POSTPONE ->X is not a way to do it, either.
Locals and meta-programming don't work well together at the moment.
One could work on that, but for now it's not clear that people want to
combine these features to make such work worthwhile.

>> The other dependency on the contents of the wordlists in the search
>> order is about the word X.
>
>But in this dicussion X is a local,

Maybe in your mind, but that's not obvious from what you wrote. But
yes, given the rules of locals visibility, if we know that X is a
local, there is no need to protect against othere words X shadowing
the local.

>> <http://www.complang.tuwien.ac.at/forth/why-evaluate-is-bad> in only
>> 25 years old.
>
>See above.

The "Don't do that" excuse was unconvincing then, and has not become
any more convincing in the meantime. And it's covered in the linked
page:

|Now don't tell me that the user has no business defining UNTIL. First
|of all this is Forth, not Ada's school for politically correct
|programmers. Second, there are completely legitimate reasons for
|defining standard words, e.g., in an application wor list for users
|that do not come into contact with the more down-to-earth facts of
|Forth (like UNTIL).

[none albert]

In article <2020Oct1...@mips.complang.tuwien.ac.at>,
Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
<SNIP>

>|Now don't tell me that the user has no business defining UNTIL. First
>|of all this is Forth, not Ada's school for politically correct
>|programmers. Second, there are completely legitimate reasons for
>|defining standard words, e.g., in an application wor list for users
>|that do not come into contact with the more down-to-earth facts of
>|Forth (like UNTIL).

There is one reason to redefine a word that is always legitimate,
that is debugging.
Unlike other languages Forth explicitly condones this.
A weird word like TO could need severe debugging because it often
leads to unexpected results.

>
>- anton

Groetjes Albert
--

[a...@littlepinkcloud.invalid]

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> a...@littlepinkcloud.invalid writes:
>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>> a...@littlepinkcloud.invalid writes:
>>>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>>> a...@littlepinkcloud.invalid writes:
>>>>>> S" to x" evaluate
>>> ...
>>>>And I don't quite see how there's a dependency on the contents of
>>>>the wordlists in the current search order, unless someone has been
>>>>foolish enough to redefine TO, in which case you no longer have a
>>>>Standard Forth System.
>>>
>>> Whether the result is not a standard system is irrelevant; this is
>>> allowed in a standard program.
>>
>>I see. However, this is exactly what I mean by "application need." The
>>natural Forth practitioner's response to "What if TO has been
>>redefined?" is "Don't do that, then," and quite rightly so. Forth was
>>never intended for ivory-tower academics. It's just not that kind of
>>language.
>
> That's contrafactual. Most languages designed by academics (e.g.,
> Algol 60, Pascal, Haskell) have reserved words, and disallow defining
> the same more than once. Forth is not that kind of language. It does
> not have reserved words, and allows defining words with the same name
> multiple times.

Er, what? The first sentence disagrees with what I wrote, but the
later ones agree. In other words, the first sentence of that paragraph
is contrafactual.

> And defining standard names in a non-standard way is not at all
> unusual, actually even standard systems do it; e.g., SwiftForth, VFX,
> and Gforth on IA-32/AMD64 come with words called AND that implement
> the core word AND, as well as with words called AND that behave
> completely differently.

Sure, in the assembler that is common. But the programmer there *knows
what they are doing, and why*. It is not in any way an argument
against my point.

> Blaming the user with the flimsiest (or, in this case, contrafactual)
> supporting argument is a frequent behaviour of programmers whose
> programs or programming practices fail in some circumstances. In the
> Forth world "Don't do that" is a frequent way to express that.

And quite rightly so. Don't do that.

>>> The other dependency on the contents of the wordlists in the search
>>> order is about the word X.
>>
>>But in this dicussion X is a local,
>
> Maybe in your mind,

In the surrounding context of this thread. We were talking about
locals.

> but that's not obvious from what you wrote. But yes, given the
> rules of locals visibility, if we know that X is a local, there is
> no need to protect against othere words X shadowing the local.

>>> <http://www.complang.tuwien.ac.at/forth/why-evaluate-is-bad> in only
>>> 25 years old.
>>
>>See above.
>
> The "Don't do that" excuse was unconvincing

to you

> then, and has not become any more convincing

to you

> in the meantime.

OK, but convincing you isn't the point.

> |Now don't tell me that the user has no business defining UNTIL.

I'm not telling you that. Do that if you must, but be it on your own
head. The user can do anything they want, but library code doesn't
have to cope with

: 2 3 ;

either.

Andrew.

[Krishna Myneni]

Actually, the above is not true. A common use case is defining an inline
macro such as the following:

: fsquare ( F: r1 -- r2 ) postpone fdup postpone f* ; immediate

Now, FSQUARE works as intended when it is used inside of a definition,
namely it will compile the sequence FDUP F* into the definition.
However, FSQUARE will not be usable at the interpreter in Standard
Forth. There are two ways to make it usable in Standard Forth for both
cases:

1) Your suggestion of

: fsquare s" fdup f*" evaluate ; immediate

2) Write a STATE-smart word

: fsquare STATE @ IF postpone fdup postpone f*
ELSE fdup f* THEN ; immediate

The first approach is a bad idea in general, for the cases when the
string contains words which may be redefined -- the evaluation occurs at
run-time not when FSQUARE is defined.

The second case is bad due to being STATE-smart and the associated
restrictions on what we can subsequently do with FSQUARE as a result.

A dual-xt implementation of FSQUARE would not suffer these drawbacks.
IMO, it's absurd to define two words with two different names, one for
interpretation and one for compilation, in order to satisfy Standard
Forth and avoid being STATE-smart.

This sort of use case is very common, at least in my programming.

Krishna

[Anton Ertl]

Krishna Myneni <krishna...@ccreweb.org> writes:
>A common use case is defining an inline
>macro such as the following:
>
>: fsquare ( F: r1 -- r2 ) postpone fdup postpone f* ; immediate
>
>Now, FSQUARE works as intended when it is used inside of a definition,
>namely it will compile the sequence FDUP F* into the definition.
>However, FSQUARE will not be usable at the interpreter in Standard
>Forth.

...

>A dual-xt implementation of FSQUARE would not suffer these drawbacks.

You can use a dual-xt approach for that, and that's what cmForth does;
if you implement [COMPILE], you have to worry about knowing whether
the second xt performs default compilation semantics or not.

An alternative for this optimization task (but not for implementing
non-default compilation semantics) is to use the intelligent COMPILE,.
In ordinary compilation, the difference is not very obvious: Instead
of EXECUTEing this compile xt, you COMPILE, the xt representing
interpretation/execution semantics, and the COMPILE, then executes
just the same code that would otherwise be EXECUTED.

One (minor) difference is that the intelligent COMPILE, applies the
optimization when the xt is COMPILE,d, while the dual-xt word does not.

In Gforth, you can get an optimizing FSQUARE as follows:

: fsquare fdup f* ;
[: drop ]] fdup f* [[ ;] set-optimizer

[Krishna Myneni]

On 10/13/20 4:04 PM, Anton Ertl wrote:
> Krishna Myneni <krishna...@ccreweb.org> writes:
>> A common use case is defining an inline
>> macro such as the following:
>>
>> : fsquare ( F: r1 -- r2 ) postpone fdup postpone f* ; immediate
>>
>> Now, FSQUARE works as intended when it is used inside of a definition,
>> namely it will compile the sequence FDUP F* into the definition.
>> However, FSQUARE will not be usable at the interpreter in Standard
>> Forth.
> ...
>> A dual-xt implementation of FSQUARE would not suffer these drawbacks.
>
> You can use a dual-xt approach for that, and that's what cmForth does;

...

> In Gforth, you can get an optimizing FSQUARE as follows:
>
> : fsquare fdup f* ;
> [: drop ]] fdup f* [[ ;] set-optimizer
>

I have to admit I don't understand the above code.

Why can't you simply use the method you posted earlier, involving
INTERPRET/COMPILE: ?

:noname fdup f* ;
:noname postpone fdup postpone f* ;
interpret/compile: fsquare

This seems to work as intended under Gforth:

: test 5e fsquare ;
test f. 25. ok
5e fsquare f. 25. ok

KM

[Anton Ertl]

Krishna Myneni <krishna...@ccreweb.org> writes:
>On 10/13/20 4:04 PM, Anton Ertl wrote:
>> Krishna Myneni <krishna...@ccreweb.org> writes:
>>> A common use case is defining an inline
>>> macro such as the following:
>>>
>>> : fsquare ( F: r1 -- r2 ) postpone fdup postpone f* ; immediate
>>>
>>> Now, FSQUARE works as intended when it is used inside of a definition,
>>> namely it will compile the sequence FDUP F* into the definition.
>>> However, FSQUARE will not be usable at the interpreter in Standard
>>> Forth.
>> ...
>>> A dual-xt implementation of FSQUARE would not suffer these drawbacks.
>>
>> You can use a dual-xt approach for that, and that's what cmForth does;
>...
>> In Gforth, you can get an optimizing FSQUARE as follows:
>>
>> : fsquare fdup f* ;
>> [: drop ]] fdup f* [[ ;] set-optimizer
>>
>
>I have to admit I don't understand the above code.

The first line is the definition. By default it's COMPILE,
implementation is to CALL the definition. The second line changes the
COMPILE, implementation to [: drop ]] fdup f* [[ ;]. The DROP is
there because COMPILE, gets the xt of the word, and this
implementation does not need the xt.

[: ... ;] is a quotation, and in Gforth you can use that
interpretively; the difference from using :NONAME ... ; is that ;]
restores the old "current definition", so SET-OPTIMIZER changes
FSQUARE, not the quotation. ]] ... [[ is a shorthand for POSTPONEing
all the words in between.

A closer-to-standard way of writing this is:

: compile,-fsquare ( xt -- ) drop postpone fdup postpone f* ;
: fsquare fdup f* ;
' compile,-fsquare set-compiler

>Why can't you simply use the method you posted earlier, involving
>INTERPRET/COMPILE: ?
>
>:noname fdup f* ;
>:noname postpone fdup postpone f* ;
>interpret/compile: fsquare

[COMPILE] would not work correctly for it; it assumes that every
INTERPRET/COMPILE: word has non-default compilation semantics:

:noname fdup f* ;
:noname postpone fdup postpone f* ;
interpret/compile: fsquare

: fsquare1 [compile] fsquare ;
5e fsquare f. \ should output "25.", outputs "5.".

Also, in Gforth we have and need an intelligent COMPILE, anyway
(because the primitive-centric code needs it). This is the right
place for optimizations (and only optimizations), so of course we use
it for that purpose.

If you have a classic threaded-code system (where : COMPILE, , ;
works), and don't implement [COMPILE], or have a way to tell it that
the compilation action actually implements the default compilation
semantics, using the dual-xt mechanism for optimization is perfectly
fine.

[a...@littlepinkcloud.invalid]

Krishna Myneni <krishna...@ccreweb.org> wrote:
> Actually, the above is not true. A common use case is defining an inline
> macro such as the following:
>
> : fsquare ( F: r1 -- r2 ) postpone fdup postpone f* ; immediate
>
> Now, FSQUARE works as intended when it is used inside of a definition,
> namely it will compile the sequence FDUP F* into the definition.
> However, FSQUARE will not be usable at the interpreter in Standard
> Forth.

That looks like a mistake to me. The definition of FSQUARE should be

: fsquare ( F: r1 -- r2 ) fdup f* ;

That's Forth. Simple, elegant, and obviously correct. Why do anything
else?

> This sort of use case is very common, at least in my programming.

That's a shame. Would it not make more sense to concentrate on writing
an inilning pass if this sort of stuff is very common? Or perhaps
investigate the overhead of calls?

Andrew.

[a...@littlepinkcloud.invalid]

a...@littlepinkcloud.invalid wrote:
> Krishna Myneni <krishna...@ccreweb.org> wrote:
>

>> This sort of use case is very common, at least in my programming.
>
> That's a shame. Would it not make more sense to concentrate on

> writing an [inlining] pass if this sort of stuff is very common? Or

> perhaps investigate the overhead of calls?

Just to expland on this some more, and hopefully make it sound less
like an _ex cathedra_ pronouncement:

Forth was designed to have the lowest-possible overhead for calls, to
allow the practice of fine-grained factoring at a level beyond
anything common at the time:

"... In contrast, a subroutine call constructed by most compilers
requires instructions for handling the calling sequence before and
after a CALL or JSR instruction and address, and typically save and
restore registers within the subroutine. Forth’s stack architecture
obviates the need for an explicit calling sequence ..."

Chuck Moore's practice has always been to write lots of small words,
one or two lines, and not worry about call/ret overhead. That's also
what I've seen from the work of the other great Forth programmers
influenced by him, and who influenced me: they just factor, factor,
and factor some more, not worrying about the cost of : and ; .

Andrew.

[Anton Ertl]

a...@littlepinkcloud.invalid writes:

>a...@littlepinkcloud.invalid wrote:
>> That's a shame. Would it not make more sense to concentrate on
>> writing an [inlining] pass if this sort of stuff is very common? Or
>> perhaps investigate the overhead of calls?
>
>Just to expland on this some more, and hopefully make it sound less
>like an _ex cathedra_ pronouncement:
>
>Forth was designed to have the lowest-possible overhead for calls, to
>allow the practice of fine-grained factoring at a level beyond
>anything common at the time:

And yet a considerable number of Forth programmers worry about the
call overhead, and use various ways to achieve some kind of inlining
even on systems that do not have inlining built-in. Evidence can be
found in various postings by Wil Baden, in the RFI 7 question by
Wonyong Koh, in the posting that lead to this subthread, and in other
postings here over the years.

The best answer is certainly a Forth system that inlines, like VFX and
iForth. And if you use one that does not inline, is it really worth
complicating your program to cater for that system?

The answer to that question still seems to be "yes" for many. In a
non-inlining case, this syndrome affected me, too: I complicated my
matrix multiplication code with loop unrolling to cater for the slow
DO...LOOP implementation of pretty much all Forth systems.

Has this never afflicted you?

>Chuck Moore's practice has always been to write lots of small words,
>one or two lines, and not worry about call/ret overhead.

And yet cmForth is not subroutine-threaded, but native code, and has
the dual-wordlist mechanism mainly to implement native-code
compilation. Machine Forth (and, I expect, colorForth) is native code
rather than subroutine-threaded, too.