POSTPONE and semantics of immediate words

[Ruvim]

This message continues in a new thread the topic:
"Custom xt compiler and compilation semantics (was: Extended VALUEs in
Forth 200x)" news://thi90o$2ofsa$1...@dont-email.me
https://groups.google.com/g/comp.lang.forth/c/dh347IHLDtw/m/tv7Z8QsfBAAJ




Some time ago Anton defended one peculiar understanding (interpretation)
of immediate words and their semantics, namely that for any immediate
word the interpretation semantics are equivalent to the compilation
semantics, which in turn are equivalent to the execution semantics (¹).
There were many discussions in this regard [1]. I don't remember anyone
else defending this understanding, but it seems it has its share in the
community. This message is addressed to the people who share this
understanding.

[1] Just one example:
https://groups.google.com/g/comp.lang.forth/c/ildjR6oy6hg/m/caArstkVBgAJ



One consequence of the understanding (¹) is that compilation semantics
can depend on STATE. Since, if the execution semantics for an immediate
word are STATE-dependent, then the compilation semantics are either
STATE-dependent too or not equivalent to the execution semantics. If you
don't accept their inequivalence, you have to admit STATE-dependent
compilation semantics.

\ An example of immediate word for which execution semantics
\ are dependent on STATE, i.e. they produce different results
\ in interpretation state and in compilation state
: foo ( -- 132 | ) s" 123" evaluate ; immediate
' foo constant xt-foo

\ if xt-foo is executed interpretively, it places 123 on the stack
\ if xt-foo is executed compilatively, it compiles 123 as a literal


By this understanding, if I ask you to perform the compilation semantics
for an immediate word, you can do it by performing the execution
semantics of this word in interpretation state as well as in compilation
state.

But the results will vary if these execution semantics are
STATE-dependent. Should I really add that I ask for the variant of
compilation semantics that takes place namely in compilation state?

Actually, no. If I ask to perform the compilation semantics for a word,
I mean to perform that very behavior that takes place when the Forth
text interpreter encounters this word name in compilation state. No
other options.

Ditto for interpretation semantics.


Thus, the understanding (¹) produces a bunch of such excessive notions as:
- STATE-dependent compilation semantics,
- compilation semantics in compilation state,
- compilation semantics in interpretation state,
- STATE-dependent interpretation semantics,
- interpretation semantics in interpretation state,
- interpretation semantics in compilation state.

But they are not needed. They are nonsense. This whole understanding of
immediate words is very contrived and totally useless.



The only possible reason for the understanding (¹) was to justify a
particular implementation of "postpone" (i.e., to proof that the
Forth-94 standard allegedly allows such implementations):

: postpone ( "name" -- )
bl word find case
-1 of lit, ['] compile, compile, endof
1 of compile, endof
-13 throw
endcase
; immediate
(²)

This "postpone", when it's applied to an immediate word, appends to the
current definition the execution semantics of the word (by applying
"compile," to their identifier), instead of the compilation semantics
(as it's defined in 2.1 Definitions of terms).


If you want to allow this implementation, it's better to just say that
"postpone", when it's applied to an immediate word, appends its
execution semantics. There's no need to skew the notion of compilation
semantics making it STATE-dependent.

But, anyway, if you allow this implementation, you introduce some
problems. And these problems remain regardless what and how you call.

The Forth-94 TC realized these problems and, in their clarification RFI
Q99-027/A99-027, chose to restrict standard programs to avoid these
problems (NB: at the moment, it is not a normative part of any Forth
standard issued after 1984).



If you don't restrict programs, the behavior variations of the
"postpone" implementation (²) when it is applied to immediate and
nonimmediate dual-semantics words become observable for programs. Hence,
the programmer has to know how an argument of "postpone" is implemented.
And it makes "postpone" *fail* in what it was designed for.

A.6.1.2033 POSTPONE
https://forth-standard.org/standard/rationale#rat:core:POSTPONE

| POSTPONE replaces most of the functionality
| of COMPILE and [COMPILE]. COMPILE and [COMPILE]
| are used for the same purpose: postpone
| the compilation behavior of the next word
| in the parse area. COMPILE was designed to be applied
| to non-immediate words and [COMPILE] to immediate words.
|
| This burdens the programmer with needing to know
| which words in a system are immediate. Consequently,
| Forth standards have had to specify the immediacy
| or non-immediacy of all words covered by the standard.
| This unnecessarily constrains implementors.


A.6.1.1550 FIND
https://forth-standard.org/standard/rationale#rat:core:FIND

| POSTPONE allowed de-specification of immediacy
| or non-immediacy for all but a few Forth words
| whose behavior must be STATE-independent.


So, if implementation (²) for "postpone" is allowed, the programmer is
burdened again with needing to know which words are immediate.

Therefore, the implementation (²) is not acceptable without restriction
on programs. And even the understanding (¹) cannot change that. Hence,
there is no reasons to stick with this contrived understanding at all.



It's very easy to implement (or redefine) "postpone" that correctly deal
with immediate words and does not impose any restriction on programs.

See: https://github.com/ForthHub/discussion/discussions/103#solution





On 2022-10-04 21:36Z Ruvim wrote:
> On 2022-10-03 21:52Z, Krishna Myneni wrote:
>
>> NAME>COMPILE should always return the compilation xt, which can be
>> independently set on a dual token system, or on a single token system
>> which supports "non-default compilation semantics".
>
>
>> In contrast, ticking a word on a standard system will return an
>> execution token corresponding to its interpretation semantics.
>
> It sounds incorrectly. By this wording you revoke access to the
> identifier of the execution semantics for a word.
>
> Actually, ticking a word on a standard system returns the execution
> token that identifies the execution semantics for *this* word.
>
> If the standard does not specify execution semantics for the word then
> the returned xt (if any) identifies implementation-dependent execution
> semantics for this word.
>
> Yes, usually, this execution token is used by the system to *perform*
> the interpretation semantics for the word.
>
> But, for some words, this the same execution token may be also used to
> perform the compilation semantics for the word. Namely, if the word is
> an immediate word.
>
> Since, to perform the compilation semantics for an immediate word, its
> execution token shall be executed in compilation state, to perform the
> interpretation semantics — it shall be executed in interpretation state.
>
> When you refer this execution token as one corresponding to the
> interpretation semantics of a word, you also make an impression that
> executing this token performs the interpretation semantics for the word
> regardless of STATE. But it's false in the general case.
>
> --
> Ruvim


--
Ruvim

[Stephen Pelc]

On 6 Oct 2022 at 15:21:43 CEST, "Ruvim" <ruvim...@gmail.com> wrote:

> This message continues in a new thread the topic:
> "Custom xt compiler and compilation semantics (was: Extended VALUEs in
> Forth 200x)" news://thi90o$2ofsa$1...@dont-email.me
> https://groups.google.com/g/comp.lang.forth/c/dh347IHLDtw/m/tv7Z8QsfBAAJ
>
>
>
>
> Some time ago Anton defended one peculiar understanding (interpretation)
> of immediate words and their semantics, namely that for any immediate
> word the interpretation semantics are equivalent to the compilation
> semantics, which in turn are equivalent to the execution semantics (¹).
> There were many discussions in this regard [1]. I don't remember anyone
> else defending this understanding, but it seems it has its share in the
> community. This message is addressed to the people who share this
> understanding.
>
> [1] Just one example:
> https://groups.google.com/g/comp.lang.forth/c/ildjR6oy6hg/m/caArstkVBgAJ
>

....

>
> It's very easy to implement (or redefine) "postpone" that correctly deal
> with immediate words and does not impose any restriction on programs.
>
> See: https://github.com/ForthHub/discussion/discussions/103#solution
>

> Ruvim

IMMEDIATE and immediacy are very old ideas that long precede POSTPONE.
Everything thta you say about IMMEDIATE words is (probably) true. However, the
Forth community is relatively slow moving when adopting new concepts and you
will have to provide an easy migration path.

My NDCS papers allow you to handle non-default compilation semantics easily
and in a standard-compliant way with a readable notation. See my recent
EuroForth papers. The ideas are in all current versions of VFX Forth which are
supplied with full source code. See Kernel/Common/kernel.fth and kernel64.fth.
The source code is high-level Forth.

Stephen
--
Stephen Pelc, ste...@vfxforth.com
MicroProcessor Engineering, Ltd. - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612,
+34 649 662 974
http://www.mpeforth.com - free VFX Forth downloads

[Anton Ertl]

Ruvim <ruvim...@gmail.com> writes:
>This message continues in a new thread the topic:
>"Custom xt compiler and compilation semantics (was: Extended VALUEs in
>Forth 200x)" news://thi90o$2ofsa$1...@dont-email.me
>https://groups.google.com/g/comp.lang.forth/c/dh347IHLDtw/m/tv7Z8QsfBAAJ
>
>
>
>
>Some time ago Anton defended one peculiar understanding (interpretation)
>of immediate words and their semantics, namely that for any immediate
>word the interpretation semantics are equivalent to the compilation
>semantics, which in turn are equivalent to the execution semantics (¹).

That follows straight from the standard:

A) Equivalence of execution and compilation semantics for immediate
words:

|2.1 Definitions of terms
|
|immediate word: A Forth word whose compilation semantics are to
|perform its execution semantics.

Do you want to argue that this does not mean that the compilation
semantics are equivalent to the execution semantics?

B) Equivalence of execution and interpretation semantics for immediate
words.

|3.4.3.2 Interpretation semantics
|
|Unless otherwise specified in an "Interpretation:" section of the
|glossary entry, the interpretation semantics of a Forth definition are
|its execution semantics.

This makes the execution and interpretation semantics of all
user-defined words equivalent; as for standard-defined words, none
those that are specified as immediate have an Interpretation: section
that specifies something else. I.e., for all immediate words, the
execution and interpretation semantics are equivalent.

C) Equivalence of interpretation and compilation semantics for
immediate words.

Follows from A, B, and the transitivity of equivalence.

>One consequence of the understanding (¹) is that compilation semantics
>can depend on STATE. Since, if the execution semantics for an immediate
>word are STATE-dependent, then the compilation semantics are either
>STATE-dependent too or not equivalent to the execution semantics. If you
>don't accept their inequivalence, you have to admit STATE-dependent
>compilation semantics.
>
> \ An example of immediate word for which execution semantics
> \ are dependent on STATE, i.e. they produce different results
> \ in interpretation state and in compilation state
> : foo ( -- 132 | ) s" 123" evaluate ; immediate
> ' foo constant xt-foo
>
> \ if xt-foo is executed interpretively, it places 123 on the stack
> \ if xt-foo is executed compilatively, it compiles 123 as a literal

Please clarify "execute interpretively" and "execute compilatively".
Do you also use phrases like "execute decimally", "execute octally" or
the like to discuss the context-dependent (and often unwanted)
properties of this definition?

>By this understanding, if I ask you to perform the compilation semantics
>for an immediate word, you can do it by performing the execution
>semantics of this word in interpretation state as well as in compilation
>state.
>
>But the results will vary if these execution semantics are
>STATE-dependent. Should I really add that I ask for the variant of
>compilation semantics that takes place namely in compilation state?

If you want to ask for that, then yes. Of course, the better
alternative is to avoid STATE-smart (and, by extension,
STATE-dependent) words.

>Actually, no. If I ask to perform the compilation semantics for a word,
>I mean to perform that very behavior that takes place when the Forth
>text interpreter encounters this word name in compilation state. No
>other options.
>
>Ditto for interpretation semantics.

I assume with "this very behaviour" you mean that you want that
compilation semantics is performed only in compilation state, so you
get protected from the STATE-smart properties of the word.

And likewise you want interpretation semantics to be performed only in
interpretation state.

The preferable alternative is to write your words such that they do
not depend on STATE. This gives you what you want without deviating
from common system implementation practice.

Just to check what systems do, I have written a small test (made
somewhat larger by prepending an implementation of FIND-NAME-IN:

---------------
\ find-name-in implementation from
\ <http://www.forth200x.org/reference-implementations/find-name.fs>
: >lower ( c1 -- c2 )
dup 'A' 'Z' 1+ within bl and or ;
: istr= ( addr1 u1 addr2 u2 -- flag )
rot over <> IF 2drop drop false EXIT THEN
bounds ?DO
dup c@ >lower I c@ >lower <> IF drop false unloop EXIT THEN
1+
LOOP drop true ;

: find-name-in-helper ( addr u wid -- nt / 0 )
dup >r name>string 2over istr= IF
rot drop r> -rot false
ELSE r> drop true THEN ;

: find-name-in ( addr u wid -- nt / 0 )
>r 0 -rot r>
['] find-name-in-helper
swap traverse-wordlist 2drop ;

\ the actual test
: [.state] state @ . ; immediate
: [execute] execute ; immediate
[.state] \ 0
] [.state] [ \ -1
s" [.state]" forth-wordlist find-name-in constant [.state]-nt
[.state]-nt name>interpret ] [execute] [ \ -1
[.state]-nt name>compile execute \ 0
----------------

I have run this on gforth 0.7.9_20221005, lxf 1.6-982-823, SwiftForth
4.0.0-RC52, vfx 5.11RC2, and they all print "0 -1 -1 0", as I
expected. What you want is that they print "0 -1 0 -1", i.e., no
system behaves as you want.

And actually even fans of STATE-smartness don't want the behaviour you
advocate. They do things like define a state-smart LITERAL and then


: literal state @ if postpone literal then ; immediate
: ascii parse-name drop c@ postpone literal ; immediate

and expect ASCII to inherit the state-smartness of their LITERAL.

>Thus, the understanding (¹) produces a bunch of such excessive notions as:
> - STATE-dependent compilation semantics,
> - compilation semantics in compilation state,
> - compilation semantics in interpretation state,
> - STATE-dependent interpretation semantics,
> - interpretation semantics in interpretation state,
> - interpretation semantics in compilation state.
>
>But they are not needed. They are nonsense.

Certainly. Just don't use STATE-smart words, and you avoid this
nonsense.

>The only possible reason for the understanding (¹) was to justify a
>particular implementation of "postpone" (i.e., to proof that the
>Forth-94 standard allegedly allows such implementations):
>
> : postpone ( "name" -- )
> bl word find case
> -1 of lit, ['] compile, compile, endof
> 1 of compile, endof
> -13 throw
> endcase
> ; immediate
>(²)
>
>This "postpone", when it's applied to an immediate word, appends to the
>current definition the execution semantics of the word (by applying
>"compile," to their identifier), instead of the compilation semantics
>(as it's defined in 2.1 Definitions of terms).
>
>
>If you want to allow this implementation, it's better to just say that
>"postpone", when it's applied to an immediate word, appends its
>execution semantics. There's no need to skew the notion of compilation
>semantics making it STATE-dependent.

This is not limited to POSTPONE. Note that the [.STATE] example does
not use POSTPONE, and the POSTPONEs in the ASCII example can be
replaced with [COMPILE] without changing a thing. Note especially
that the style of the definition of ASCII predates POSTPONE.

And your position about interpretation semantics obviously has nothing
to do with POSTPONE.

>But, anyway, if you allow this implementation, you introduce some
>problems. And these problems remain regardless what and how you call.

What are those problems? STATE-smart words have been problematic and
have been recognized as problematic since at least the early 1980s
(not by all, but that has not changed).

For those who are aware of the problems, several solutions have been
found:

1) Avoid state-smart words and instead use pairs of words, e.g. ' [']

2) Avoid state-smart words in general, but define them in a few places
where the standard defines non-immediate non-default compilation
semantics, e.g., the implementation of S" in SwiftForth.

3) Avoid state-smart words by using recognizers instead of S" and TO,
e.g., the string-recognizer and the to-recognizer in Gforth.

4) Provide a way to define words with arbitrary combinations of
interpretation and compilation semantics, e.g., interpret/compile:
and compsem: in Gforth.

As far as the standard is concerned, I think we need a few
improvements to support approach 2) a little better, but I would
expect the system implementors of such systems to show some
initiative. That work is certainly no on to top of *my* ToDo list.

>If you don't restrict programs, the behavior variations of the
>"postpone" implementation (²) when it is applied to immediate and
>nonimmediate dual-semantics words become observable for programs. Hence,
>the programmer has to know how an argument of "postpone" is implemented.
>And it makes "postpone" *fail* in what it was designed for.
>
>A.6.1.2033 POSTPONE
>https://forth-standard.org/standard/rationale#rat:core:POSTPONE
>
>| POSTPONE replaces most of the functionality
>| of COMPILE and [COMPILE]. COMPILE and [COMPILE]
>| are used for the same purpose: postpone
>| the compilation behavior of the next word
>| in the parse area. COMPILE was designed to be applied
>| to non-immediate words and [COMPILE] to immediate words.
>|
>| This burdens the programmer with needing to know
>| which words in a system are immediate. Consequently,
>| Forth standards have had to specify the immediacy
>| or non-immediacy of all words covered by the standard.
>| This unnecessarily constrains implementors.

POSTPONE works nicely in that capacity. I have not needed to use
[COMPILE] or COMPILE except for test programs since POSTPONE has been
introduced.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2022: https://euro.theforth.net

[Ruvim]

On 2022-10-09 10:07, Anton Ertl wrote:
> Ruvim <ruvim...@gmail.com> writes:
>> This message continues in a new thread the topic:
>> "Custom xt compiler and compilation semantics (was: Extended VALUEs in
>> Forth 200x)" news://thi90o$2ofsa$1...@dont-email.me
>> https://groups.google.com/g/comp.lang.forth/c/dh347IHLDtw/m/tv7Z8QsfBAAJ
>>
>>
>>
>>
>> Some time ago Anton defended one peculiar understanding (interpretation)
>> of immediate words and their semantics, namely that for any immediate
>> word the interpretation semantics are equivalent to the compilation
>> semantics, which in turn are equivalent to the execution semantics (¹).
>
> That follows straight from the standard:
>
> A) Equivalence of execution and compilation semantics for immediate
> words:
>
> |2.1 Definitions of terms
> |
> |immediate word: A Forth word whose compilation semantics are to
> |perform its execution semantics.
>
> Do you want to argue that this does not mean that the compilation
> semantics are equivalent to the execution semantics?

Yes, it does not mean that.

>
> B) Equivalence of execution and interpretation semantics for immediate
> words.
>
> |3.4.3.2 Interpretation semantics
> |
> |Unless otherwise specified in an "Interpretation:" section of the
> |glossary entry, the interpretation semantics of a Forth definition are
> |its execution semantics.
>
> This makes the execution and interpretation semantics of all
> user-defined words equivalent; as for standard-defined words, none
> those that are specified as immediate have an Interpretation: section
> that specifies something else. I.e., for all immediate words, the
> execution and interpretation semantics are equivalent.
>
> C) Equivalence of interpretation and compilation semantics for
> immediate words.
>
> Follows from A, B, and the transitivity of equivalence.

No. I already shown a mistake in this reasoning and provided my
reasoning, and did it twice [1,2]. No answers on that.

You just repeat your arguments and ignore my counter-arguments.
I hope to see your answers to my reasoning eventually.


[1] Re: Semantics rethinking, 2019-08-04 10:41Z
news://qi6con$1bdk$1...@gioia.aioe.org
https://groups.google.com/g/comp.lang.forth/c/vrq2f2XXy1Q/m/tiuq7hJTFgAJ

[2] Re: Semantics of POSTPONE and immediate words, 2020-09-23 00:19Z
news://rke4a2$i5i$1...@dont-email.me
https://groups.google.com/g/comp.lang.forth/c/ildjR6oy6hg/m/caArstkVBgAJ




We may always substitute a term definition name (definiendum) with the
term definition body (definiens) in a place where this term is used.

Let's do it for the terms "interpretation semantics" and "compilation
semantics" in the fragments mentioned by you. The replacements are
marked by curly brackets:

in 2.1:
| immediate word:
| A Forth word whose { behavior when its name is encountered
| by the text interpreter *in compilation state* } is to perform its
| execution semantics.

in 3.4.3.2:

| Unless otherwise specified in an "Interpretation:" section of the

| glossary entry, { the behavior of a Forth definition when its name
| is encountered by the text interpreter *in interpretation state* }
| is its execution semantics.


In the first case it means that execution semantics are performed in
compilation state.

In the seconds case it means that execution semantics are performed in
interpretation state.


Are you agree that a function is not equivalent to a partial application
of this function in the general case? I.e., it's easy to provide
examples when they are not equivalent.

Ditto, execution semantics in a given state are not equivalent to the
original execution semantics. Since execution semantics in a given state
is like a partially applied function.


What are your counter-arguments to my reasoning?


[...]

--
Ruvim

[Ruvim]

On 2022-10-09 10:07, Anton Ertl wrote:
> Ruvim <ruvim...@gmail.com> writes:
>>

>> Some time ago Anton defended one peculiar understanding (interpretation)
>> of immediate words and their semantics, namely that for any immediate
>> word the interpretation semantics are equivalent to the compilation
>> semantics, which in turn are equivalent to the execution semantics (¹).

[...]

>> One consequence of the understanding (¹) is that compilation semantics
>> can depend on STATE. Since, if the execution semantics for an immediate
>> word are STATE-dependent, then the compilation semantics are either
>> STATE-dependent too or not equivalent to the execution semantics. If you
>> don't accept their inequivalence, you have to admit STATE-dependent
>> compilation semantics.
>>
>> \ An example of immediate word for which execution semantics
>> \ are dependent on STATE, i.e. they produce different results
>> \ in interpretation state and in compilation state
>> : foo ( -- 132 | ) s" 123" evaluate ; immediate
>> ' foo constant xt-foo
>>
>> \ if xt-foo is executed interpretively, it places 123 on the stack
>> \ if xt-foo is executed compilatively, it compiles 123 as a literal
>
> Please clarify "execute interpretively" and "execute compilatively".

It's just abbreviations for "execute in interpretation state" and
"execute in compilation state" correspondingly.

I supposed it should be obvious, since such wording is used in the
community. For example, you wrote on 2022-04-28: "the execution
semantics of EXIT when used interpretively".

> Do you also use phrases like "execute decimally", "execute octally" or
> the like to discuss the context-dependent (and often unwanted)
> properties of this definition?

I could if it makes sense. But it doesn't matter for this topic.



[...]


--
Ruvim

[Krishna Myneni]

On 10/16/22 15:42, Ruvim wrote:
> On 2022-10-09 10:07, Anton Ertl wrote:
>> Ruvim <ruvim...@gmail.com> writes:

..

>>> Some time ago Anton defended one peculiar understanding (interpretation)
>>> of immediate words and their semantics, namely that for any immediate
>>> word the interpretation semantics are equivalent to the compilation
>>> semantics, which in turn are equivalent to the execution semantics (¹).
>>
>> That follows straight from the standard:
>>
>> A) Equivalence of execution and compilation semantics for immediate
>> words:
>>
>> |2.1 Definitions of terms
>> |
>> |immediate word: A Forth word whose compilation semantics are to
>> |perform its execution semantics.
>>
>> Do you want to argue that this does not mean that the compilation
>> semantics are equivalent to the execution semantics?
>
> Yes, it does not mean that.

> ...

I'm trying to understand your interpretation. If I define the following
word,

: FOO STATE @ IF ." BAR" ELSE ." FOO" THEN ; IMMEDIATE

can you identify what you consider to be its execution semantics and
what you consider to be its compilation semantics?

--
Krishna

[Ruvim]

FOO Execution ( -- )
If the system in compilation state, then display "BAR", otherwise
display "FOO".

FOO Compilation ( -- )
Display "BAR".



NB: Any behaviors can be specified in a natural language, and in many
different but equivalent formulations. And a Forth definition with the
same behavior can be implemented in many different ways.


The execution token (xt) of the word "FOO" identifies the execution
semantics specified above.

If we have this xt (let's say xt.foo), and we want to perform the
compilation semantics for the word "FOO", we have to execute xt.foo in
compilation state.

We can create a separate definition that performs the compilation
semantics for the word "FOO", and we can do it in many different ways, e.g.:

: C-FOO ( -- ) ." BAR" ;

: C-FOO ( -- ) ['] FOO EXECUTE-COMPILING ;

: C-FOO ( -- ) POSTPONE FOO ; \ POSTPONE is fully compliant

The execution token (xt) of the word "C-FOO" identifies the execution
semantics for "C-FOO", which are *equivalent* to the compilation
semantics for the word "FOO". As an abbreviation we can say that this xt
also identifies the compilation semantics for "FOO".

If we have the xt of the word "C-FOO" (let's say xt.c-foo), and we want
to perform compilation semantics for the word "FOO", we have to just
execute this xt.c-foo, regardless of whether the system in
interpretation state or in compilation state.

With some reservations, we can thought of execution tokens as partially
defined functions on the Forth system's phase space (and which have its
codomain equals to its domain).

Then the function xt.c-foo is a partially applied function of xt.foo
that has fixed STATE argument.


--
Ruvim

[Ruvim]

On 2022-10-09 10:07, Anton Ertl wrote:
> Ruvim <ruvim...@gmail.com> writes:
>>

>> Some time ago Anton defended one peculiar understanding (interpretation)
>> of immediate words and their semantics, namely that for any immediate
>> word the interpretation semantics are equivalent to the compilation
>> semantics, which in turn are equivalent to the execution semantics (¹).

>> One consequence of the understanding (¹) is that compilation semantics
>> can depend on STATE.

[...]

>> If I ask to perform the compilation semantics for a word,
>> I mean to perform that very behavior that takes place when the Forth
>> text interpreter encounters this word name in compilation state. No
>> other options.
>>
>> Ditto for interpretation semantics.
>
> I assume with "this very behaviour" you mean that you want that
> compilation semantics is performed only in compilation state,

No. I mean the following.

If the word "compile-name ( i*x nt -- j*x )" is a way to ask the system
to perform the compilation semantics for the word identified by an nt,
then execution of "compile-name" shall demonstrate the *same* behavior
*regardless* whether the system is in compilation state or in
interpretation state before this execution. And it is the very behavior
that the system demonstrates when it encounters the nt word name in
compilation state (³).

And it's a system's internal problem whether it needs (or doesn't need)
to enter in compilation state to perform the semantics that I ask.

And if you ask the system to perform the compilation semantics for a
word, you never need the system to perform a behavior that distinct from
(³). Hence, it's excessive to specify that you ask namely for the
behavior (³).

Counter examples are welcome.



[...]



--
Ruvim

[none albert]

In ciforth DO is a normal IMMEDIATE word, and ISO doesn't specify
what it is supposed to do unless in deferring mode, building a definition.

I added a facility to be able to use control structures while
interpreting: "-scripting-" WANTED

\ define a buffer for numbers not divisable by 3 or 5.
CREATE iets 100 DO I 3 MOD I 5 MOD OR IF I , THEN LOOP

Now I don't want to even think about this facility outside
of simple scripting situations. I don't care if it is creating
a execution token of sorts. It probably interacts with my
definition of classes, that likewise temporarily compiles to an
alternative space, in this case the same alternative area.
Standing back, you realize that you have extended the system
in two ways, exerting your right as a system programmer.
But you have to take your responsability to document/care
the interacting between the two facilities, if any.
Could one abstractify my classes and my scripting in a way
that it is guaranteed to work defining and juggling ever more
tokens? Maybe.

A BASIC programmer could say that DO .. LOOP "behaves" the same
in interpretation or compilation mode. You have a hard time
argue with that, but it is not a cosher argument.

You can find ADA if you need it.

>--
>Ruvim

Groetjes Albert
--
"in our communism country Viet Nam, people are forced to be
alive and in the western country like US, people are free to
die from Covid 19 lol" duc ha
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[Ruvim]

[...]
I don't see a counter example in that. Please, clarify.

> A BASIC programmer could say that DO .. LOOP "behaves" the same
> in interpretation or compilation mode. You have a hard time
> argue with that, but it is not a cosher argument.

They behaves the same in *run-time*.

In Forth, compilation semantics is not about run-time, but about
compilation time — namely, it's about a behavior that the system
demonstrates when the Forth text interpreter *encounters* a word name in
compilation state. In more precisely formulation, it's about
*detectable* changes in the system between two points of time:
1) a point just after the word name is parsed in compilation state;
2) a point just before the next lexeme is parsed.


--
Ruvim

[Ruvim]

On 2022-10-09 10:07, Anton Ertl wrote:
> Ruvim <ruvim...@gmail.com> writes:
>>

>> Some time ago Anton defended one peculiar understanding (interpretation)
>> of immediate words and their semantics, namely that for any immediate
>> word the interpretation semantics are equivalent to the compilation
>> semantics, which in turn are equivalent to the execution semantics (¹).

>> One consequence of the understanding (¹) is that compilation semantics
>> can depend on STATE.

[...]

Not quite, since this result is practically impossible concerning
"name>interpret".


My strong premise is that Forth-94 allows the single-xt approach as well
as the dual-nt (and, hence, the dual-xt) approach in implementation.

In some places the standard is not formulated quite well to perfectly
cover one or another from these approaches, — it should be corrected. In
Forth-2012 new such places were introduced.


One such problem was with "find" — its glossary entry doesn't
sufficiently specify how to perform the interpretation semantics for a
word, and how to perform the compilation semantics for a word,
regardless of the employed implementation approach in the system.

NB: by "interpretation semantics" and "compilation semantics" I mean the
standard notion according to the term definitions.


The words "name>interpret" and "name>compile" (along with "find-name")
were introduced as a replacement for "find".

But, the word "name>interpret" is also specified insufficiently in
Forth-2012 (so it's the same problem as for "find" in Forth-94). The
glossary entry for "name>interpret" does not describe how the returned
value should be used to perform the behavior that takes place when the
Forth text interpreter encounters the word name in interpretation state
— i.e., how to perform the interpretation semantics for a word
(regardless of the employed implementation approach in the system).


From various discussions I concluded, "name>interpret" should return
the same xt that Tick returns for a word (when it's applicable). It
means that "name>interpret" (despite of its name) returns the xt that
identifies the execution semantics for the word, but this xt cannot be
used to perform the compilation semantics for the word in the general case.

And, usually, "name>interpret" is implemented namely in this way. And
it's difficult to implement it in another way (at least, in a
straightforward single-xt system).



The word "[.state]" has STATE-dependent execution semantics, which
"name>interpret" returns. If we perform these semantics in compilation
state, then it prints -1, in interpretation state it prints 0 — and your
test shows that.





The word "name>compile" is specified better, but many implementations
fall into the same problem as for "postpone" — they provide
STATE-dependent execution semantics in some cases. Therefore, a portable
way to perform the compilation semantics for a word via "name>compile"
is to execute the returned xt in compilation state.

But it's a bad API, since it forces the user to specify his intention
twice — via "name>compile", and via "state". It should not be acceptable.

OTOH, nobody has provided an example when a user need to use
"name>compile" to perform the behavior for a word that Forth system
demonstrates when it encounters this word in *interpretation* state.





Concerning testing implementations in general.

No new implementations can be used to justify a peculiar interpretation
of the standard.

To validate our interpretation of the standard, with some reservations,
we can only use old implementations, which were known in the time when
the Forth-94 standard was developed.




[...]

--
Ruvim

[Ruvim]

On 2022-10-09 10:07, Anton Ertl wrote:
> Ruvim <ruvim...@gmail.com> writes:
>>

>> Some time ago Anton defended one peculiar understanding (interpretation)
>> of immediate words and their semantics, namely that for any immediate
>> word the interpretation semantics are equivalent to the compilation
>> semantics, which in turn are equivalent to the execution semantics (¹).

>> One consequence of the understanding (¹) is that compilation semantics
>> can depend on STATE.

[...]

>
> And actually even fans of STATE-smartness don't want the behaviour you
> advocate. They do things like define a state-smart LITERAL and then
>
>
> : literal state @ if postpone literal then ; immediate
> : ascii parse-name drop c@ postpone literal ; immediate
>
> and expect ASCII to inherit the state-smartness of their LITERAL.

They may do it, since it's not a standard code, but a part of a system
implementation, and so they *may* rely on carnal knowledge.

"ascii" was known long before Forth-94 (it's known in the "Uncontrolled
reference words" chapter of Forth-83 and Forth-79).

More likely this word was defined using "[compile]" as:

: ascii ( "ccc" -- char | )
bl word 1+ c@ [compile] literal
; immediate

And it was absolutely correct definition since the word "[compile]" from
Forth-83 appends the *execution semantics* of the next word to the
current definition (though, a different terminology was used).

Without knowledge how "literal" is implemented, and how "postpone" is
implemented, "[compile]" *cannot* be replaced by "postpone" in the
above definition of "ascii".


In interesting example is the rationale for the word '."' in Forth-94, a
citation:

A.6.1.0190 ."
Typical use:
: X ... ." ccc" ... ;
An implementation may define interpretation semantics for ." if desired.
In one plausible implementation, interpreting ." would display the
delimited message. In another plausible implementation, interpreting ."
would compile code to display the message later. In still another
plausible implementation, interpreting ." would be treated as an
exception. Given this variation a Standard Program may not use ." while
interpreting. Similarly, a Standard Program may not compile POSTPONE ."
inside a new word, and then use that word while interpreting.

End of the citation.


This rationale implies that the word '."' may be implemented as an
immediate word, and that "postpone" *may* append the execution semantics
of this word to the current definition, and, hence, it will be ambiguous
to execute the containing word in interpretation state.

I.e.:

: foo" postpone ." ;

foo" xxx" \ this is ambiguous.


This rationale is applicable to any word for which interpretation
semantics are undefined by the standard. For example:

: if, postpone if ;
: bar [ if, ] ; \ this is ambiguous
\ according to A.3.4.3.2 and A.6.1.0190

And it is what RFI Q99-027/A99-027 says.


The only way to discard this restriction — is to implement a fully
compliant "postpone". And the dual-xt approach cannot solve this problem.

A good news is that a strong implementation for "postpone" can be
included as a short polyfill[1] into any system that provides a
restricted "postpone".


[1] About POSTPONE semantics in edge cases
https://github.com/ForthHub/discussion/discussions/103




[...]

--
Ruvim

[Krishna Myneni]

On 10/25/22 06:04, Ruvim wrote:
..

>
> The words "name>interpret" and "name>compile" (along with "find-name")
> were introduced as a replacement for "find".
>
> But, the word "name>interpret" is also specified insufficiently in
> Forth-2012 (so it's the same problem as for "find" in Forth-94). The
> glossary entry for "name>interpret" does not describe how the returned
> value should be used to perform the behavior that takes place when the
> Forth text interpreter encounters the word name in interpretation state
> — i.e., how to perform the interpretation semantics for a word
> (regardless of the employed implementation approach in the system).
>

Really? Are there system implementations with different behaviors for
NAME>INTERPRET. The systems I've tried appear to do the same thing --
return xt-interp. Which means perform EXECUTE after NAME>INTERPRET if
you want to perform the interpretations semantics.

>
> From various discussions I concluded, "name>interpret" should return
> the same xt that Tick returns for a word (when it's applicable). It
> means that "name>interpret" (despite of its name) returns the xt that
> identifies the execution semantics for the word, but this xt cannot be
> used to perform the compilation semantics for the word in the general case.
>

On a dual-semantics system, NAME>INTERPRET returns the interpretation
semantics, which is what is executed by the text interpreter in STATE
zero. Why would you want to use NAME>INTERPRET or "'"(tick) to obtain
the compilation semantics?

..

>
> The word "name>compile" is specified better, but many implementations
> fall into the same problem as for "postpone" — they provide
> STATE-dependent execution semantics in some cases. Therefore, a portable
> way to perform the compilation semantics for a word via "name>compile"
> is to execute the returned xt in compilation state.
>

Which implementations require STATE to be set in order to use
NAME>COMPILE correctly? The standard does not say that STATE must be set
to compile in order to obtain the compilation semantics.

15.6.2.1909.10 NAME>COMPILE
“name-to-compile”
TOOLS EXT
( nt – – x xt )
x xt represents the compilation semantics of the word nt. The returned
xt has the stack effect ( i * x x – – j * x ) . Executing xt consumes x
and performs the compilation semantics of the word represented by nt.

> But it's a bad API, since it forces the user to specify his intention
> twice — via "name>compile", and via "state". It should not be acceptable.
>

The API is more cumbersome than for NAME>INTERPRET because it needs to
work with single xt systems (since xt-comp does not exist in those
systems; except that it is equal to xt-interp for immediate words). For
a dual-xt system the only necessity is to return xt-comp (in the form of
x xt) which may EXECUTEd to perform the compilation semantics. The
present API allows for use on both single-xt and dual-semantics systems.

> OTOH, nobody has provided an example when a user need to use
> "name>compile" to perform the behavior for a word that Forth system
> demonstrates when it encounters this word in *interpretation* state.
>

NAME>COMPILE does not return xt-interp, except for the case of immediate
words. If you want to perform the behavior of a word that the Forth
system demonstrates when it encounters the word in interpretation state,
use NAME>INTERPRET. This is unambiguous and I fail to see what your
difficulty is with NAME>COMPILE and NAME>INTERPRET.

--
Krishna

[Krishna Myneni]

On 10/25/22 10:43, Ruvim wrote:
..

>
>   : foo" postpone ." ;
>
>   foo" xxx"  \ this is ambiguous.
>
>
> This rationale is applicable to any word for which interpretation
> semantics are undefined by the standard. For example:
>
>   : if, postpone if ;
>   : bar [ if, ] ; \ this is ambiguous
>                   \ according to A.3.4.3.2 and A.6.1.0190
>
> And it is what RFI Q99-027/A99-027 says.
>
>
> The only way to discard this restriction — is to implement a fully
> compliant "postpone". And the dual-xt approach cannot solve this problem.
>

..

I've already shown how the above problem can be solved with a
dual-semantics (not necessarily dual-xt) approach.

--
Krishna

[Krishna Myneni]

On 10/25/22 16:59, Krishna Myneni wrote:
> On 10/25/22 10:43, Ruvim wrote:
> ..
>>
>>    : foo" postpone ." ;
>>
>>    foo" xxx"  \ this is ambiguous.
>>
>>

..

>>
>> The only way to discard this restriction — is to implement a fully
>> compliant "postpone". And the dual-xt approach cannot solve this problem.
>>
> ..
>
> I've already shown how the above problem can be solved with a
> dual-semantics (not necessarily dual-xt) approach.
>

In Gforth,

require set-compsem.fs ok
: foo" -14 throw ; ok
compsem: postpone ." ; ok
foo" xxx"
*the terminal*:4:1: error: Interpreting a compile-only word
>>>foo"<<< xxx"
Backtrace:
*terminal*:2:12: 0 $7FF8D948B7E8 throw
: test foo" hello" ; ok
test hello ok


In Vfx Forth:

: foo" -14 throw ; ok
ndcs: postpone ." ; ok
ok
foo" hello"
Err# -14 ERR: Attempt to interpret a compile only definition.
-> foo" hello"
^
: test foo" hello" ; ok
test hello ok


The examples show how to write a compile-only word.

--
Krishna

[Ruvim]

On 2022-10-25 21:57, Krishna Myneni wrote:
> On 10/25/22 06:04, Ruvim wrote:
>>
>> The words "name>interpret" and "name>compile" (along with "find-name")
>> were introduced as a replacement for "find".

[...]

>> The word "name>compile" is specified better, but many implementations
>> fall into the same problem as for "postpone" — they provide
>> STATE-dependent execution semantics in some cases. Therefore, a
>> portable way to perform the compilation semantics for a word via
>> "name>compile" is to execute the returned xt in compilation state.
>>
>
> Which implementations require STATE to be set in order to use
> NAME>COMPILE correctly? The standard does not say that STATE must be set
> to compile in order to obtain the compilation semantics.

Not to obtain, but to perform. It doesn't say that, and it's a problem.


[...]

>
> I fail to see what your difficulty is with NAME>COMPILE and NAME>INTERPRET.

Let I want to define a word "compile-word ( i*x sd.name -- j*x )" that
performs the compilation for a word named sd.name.

Where sd.name is a pair (c-addr u), which is a string of a word name,
and "compilation semantics" is a standard term according to the term
definitions (i.e. this behavior does not depend on STATE).

Usage example:

s" foo" compile-word

This phrase should perform the compilation semantics for the word "foo",
regardless of state.

How "compile-word" can be defined via "name>compile" or "name>interpret"?




--
Ruvim

[Krishna Myneni]

The following example illustrates:

: foo ." hello" ; ok
: compile-word ( caddr u -- ) find-name name>compile execute ;
immediate ok
: test1 [ s" foo" compile-word ] ; ok
: test2 [ s" foo" ] compile-word ; ok
test1 hello ok
test2 hello ok

see test1
: test1
foo ; ok
see test2
: test2
foo ; ok

--
Krishna

[Ruvim]

Applying "immediate" is incorrect. I specify only the execution
semantics for "compile-word" ("When a definition has only one specified
behavior, the label is omitted" by 3.4.3.1). It means this word is an
ordinary word, which have default compilation semantics according to
3.4.3.3.

: bar s" foo" compile-word ;

bar \ it should perform the compilation semantics for the word "foo"
\ according to the current search order.

> : test1 [ s" foo" compile-word ] ;  ok
> : test2 [ s" foo" ] compile-word ;  ok

Well, let's test your definition of "compile-word"

Let "foo" is defined as following:

: foo s" 123 . " evaluate ; immediate

: test1 [ s" foo" compile-word ] ;
\ prints "123"
: test2 [ s" foo" ] compile-word ;
\ prints nothing

test1 \ prints nothing
test2 \ prints "123"


The results are different. So, your definitions of "compile-word"
violates the conditions of my challenge.




--
Ruvim

[Ruvim]

Well, I see, this my wording is slightly confusing, if you don't align
it with the above specification.

I wanted to say that execution semantics of this phrase are to perform
the compilation semantics for the word "foo", and produce the same
results regardless of state.

In other words, it means, performing xt of
[: s" foo" compile-word ;]

performs the compilation semantics for the word "foo", and produce the
same results regardless of state.


Nevertheless, then I tested your variant.

>>>
>>> How "compile-word" can be defined via "name>compile" or
>>> "name>interpret"?
>>>
>>
>> The following example illustrates:
>>
>> : foo ." hello" ;  ok
>> : compile-word ( caddr u -- ) find-name name>compile execute ;
>> immediate  ok
>
> Applying "immediate" is incorrect. I specify only the execution
> semantics for "compile-word" ("When a definition has only one specified
> behavior, the label is omitted" by 3.4.3.1). It means this word is an
> ordinary word, which have default compilation semantics according to
> 3.4.3.3.
>
>   : bar s" foo" compile-word ;
>
>   bar \ it should perform the compilation semantics for the word "foo"
>       \ according to the current search order.

--
Ruvim