Extended VALUEs in Forth 200x

[Krishna Myneni]

Forth 2012 standardizes the words FVALUE and 2VALUE which, like VALUE,
work with TO. Thus, the typed value contains in addition to the datum
another data field indicating the execution semantics for TO. Recently I
revised the implementation of TO and VALUE in kForth-64, using
standardized Forth 200x source, to support FVALUE and 2VALUE. It turned
out to be beneficial to use the name token (nt) instead of using the
(xt) for informing TO of how to compile the execution semantics in
kForth. This is because, for primitive words (those built-in to the
dictionary), the sequence

( xt addr -- ) POSTPONE LITERAL POSTPONE LITERAL POSTPONE EXECUTE

generates less efficient code for primitive words than the sequence

( nt addr -- ) POSTPONE LITERAL NAME>COMPILE EXECUTE

when TO executes in compilation state. In the stack diagram, addr is the
address of the VALUE's data, xt is the execution semantics for TO, and
nt is the name token for a word which provides execution semantics for
TO. The improved efficiency is realized because NAME>COMPILE can
recognize a primitive word and generate byte code for a direct call to
the primitive word rather than indirect execution via EXECUTE --
basically NAME>COMPILE permits a compiler-specific optimization in
kForth, which would otherwise not be possible if one were dealing with xt's.

The following implementation might serve as a reference implementation
for Forth 200x systems -- I say Forth 200x rather than 2012 because
FIND-NAME is required. FIND-NAME was only standardized in 2018 and does
not appear in the Forth 2012 standard.

Note the two kForth-specific definitions A@ and NONDEFERRED have simple
standard definitions:

------
: NONDEFERRED ;
SYNONYM

: TO ( i*x "name" -- ) \ or ( F: i*r -- ) ( "name" -- )
' >body
dup a@ swap cell+
state @ IF
postpone literal
name>compile execute
ELSE
swap name>interpret execute
THEN ; immediate

: xVALUE ( i*x nt-put usize -- ) ( F: j*r -- )
create 1 cells + allot? \ -- i*x nt-put a
2dup ! cell+ swap name>interpret execute immediate nondeferred
;

: VALUE ( n "name" -- )
[ s" !" find-name ] literal 1 cells xVALUE
does> cell+ state @ if postpone literal postpone @
else @ then ;

: 2VALUE ( x1 x2 "name" -- )
[ s" 2!" find-name ] literal 2 cells xVALUE
does> cell+ state @ if postpone literal postpone 2@
else 2@ then ;

: FVALUE ( F: r -- ) ( "name" -- )
[ s" F!" find-name ] literal 1 floats xVALUE
does> cell+ state @ if postpone literal postpone f@
else f@ then ;
------

Note that xVALUE is used as a common factor in defining the various
VALUE types. It may be used to write a user-defined value type such as
structure values. It takes as arguments the data associated with the
value (could be a combination of cell size, double cell size, and
floating point data), a name token which provides the execution
semantics for TO and the size in cells for the value. Besides
associating the nt with the value, xVALUE also uses the sequence
NAME>INTERPRET EXECUTE to initialize the named value.


The primitive words associated with execution semantics of TO for VALUE,
2VALUE, and FVALUE are "!" , "2!", and "F!", respectively. The nt's for
these words are stored in the created values of those types. Such an
approach won't be practical on embedded systems with tightly limited
memory, but it should be fine otherwise. I have done some tests on this
code under Gforth to ensure it works as expected, but it is still
experimental. The revised TO and VALUE and FVALUE and 2VALUE are now
included in the kForth-64 master branch, in ans-words.4th.

A few tests:
------------
5 value n1 ok
n1 . 5 ok

7 to n1 ok
n1 . 7 ok

: test n1 n1 + to n1 ; ok
test ok
n1 . 14 ok

3.14e fvalue almostPI ok
almostPI f. 3.14 ok

3.14159e to almostPI ok
almostPI f. 3.14159 ok

-1 0 2value d1 ok
d1 d. 18446744073709551615 ok

: test2
cr ['] almostPI execute f.
cr ['] n1 execute .
cr ['] d1 execute d.
cr ;
ok

test2
3.14159
14
18446744073709551615
ok

: test3 postpone almostPI postpone n1 postpone d1 ; immediate ok
: test4 test3 cr d. cr . cr f. cr ; ok

test4
18446744073709551615
14
3.14159
ok
-----

-- Krishna Myneni

[Krishna Myneni]

On 9/28/22 19:39, Krishna Myneni wrote:
> ------
> : NONDEFERRED ;
> SYNONYM

The above line should have been

SYNONYM A@ @

--
KM

[minf...@arcor.de]

Thanks for posting this!

It is refreshing to read some substantial contribution again
here in c.l.f., instead of all those silly rantings of the last months.

I was only wondering if your idea could be extended to +TO.

[none albert]

In article <th2pfa$fmrn$1...@dont-email.me>,

Krishna Myneni <krishna...@ccreweb.org> wrote:
>Forth 2012 standardizes the words FVALUE and 2VALUE which, like VALUE,
>work with TO. Thus, the typed value contains in addition to the datum
>another data field indicating the execution semantics for TO. Recently I
>revised the implementation of TO and VALUE in kForth-64, using
>standardized Forth 200x source, to support FVALUE and 2VALUE. It turned
>out to be beneficial to use the name token (nt) instead of using the
>(xt) for informing TO of how to compile the execution semantics in
>kForth. This is because, for primitive words (those built-in to the
>dictionary), the sequence
>
>( xt addr -- ) POSTPONE LITERAL POSTPONE LITERAL POSTPONE EXECUTE
>
>generates less efficient code for primitive words than the sequence
>
>( nt addr -- ) POSTPONE LITERAL NAME>COMPILE EXECUTE

I have taken the stand that xt should be ditched for nt (dea, Dictionary
Entry Address). All what is accomplished by xt can be done better by nt,
because there is more information available.
To those who are concerned with efficiency (read speed), they are
invited to add some optimizing to the resulting code.

The xt is a half assed concept introduced by ISO 93, when we did not
know better.

<SNIP>

>-- Krishna Myneni
>
--
"in our communism country Viet Nam, people are forced to be
alive and in the western country like US, people are free to
die from Covid 19 lol" duc ha
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[dxforth]

On 29/09/2022 6:54 pm, albert wrote:
> ...

> I have taken the stand that xt should be ditched for nt (dea, Dictionary
> Entry Address). All what is accomplished by xt can be done better by nt,
> because there is more information available.
> To those who are concerned with efficiency (read speed), they are
> invited to add some optimizing to the resulting code.
>
> The xt is a half assed concept introduced by ISO 93, when we did not
> know better.

Needs vary. Nearly every application I write ditches the dictionary in
favour of the space it occupied. That's a 35% gain in free memory at
run-time.

[none albert]

That is a non-sequitur. I can shrink the space that executables occupy on
disk far more than 35%. In view of typical 64 mbyte dictionary space
or 8 Gbyte if needed, and Terabyte hard disks that is not a worth while
endeavour. 1)
I can choose between manipulating the dictionary for 30 K gain
or configuring lina64 for one Gigabyte more.
The latter is far more easy.

Groetjes Albert

1) Maybe I should do it. Not merely throwing away headers, but
all the code of words I don't use in the application.
For the fun of it. And to spite Java and c programmers.

[Krishna Myneni]

On 9/29/22 03:54, albert wrote:
> In article <th2pfa$fmrn$1...@dont-email.me>,
> Krishna Myneni <krishna...@ccreweb.org> wrote:
>> Forth 2012 standardizes the words FVALUE and 2VALUE which, like VALUE,
>> work with TO. Thus, the typed value contains in addition to the datum
>> another data field indicating the execution semantics for TO. Recently I
>> revised the implementation of TO and VALUE in kForth-64, using
>> standardized Forth 200x source, to support FVALUE and 2VALUE. It turned
>> out to be beneficial to use the name token (nt) instead of using the
>> (xt) for informing TO of how to compile the execution semantics in
>> kForth. This is because, for primitive words (those built-in to the
>> dictionary), the sequence
>>
>> ( xt addr -- ) POSTPONE LITERAL POSTPONE LITERAL POSTPONE EXECUTE
>>
>> generates less efficient code for primitive words than the sequence
>>
>> ( nt addr -- ) POSTPONE LITERAL NAME>COMPILE EXECUTE
>
> I have taken the stand that xt should be ditched for nt (dea, Dictionary
> Entry Address). All what is accomplished by xt can be done better by nt,
> because there is more information available.

...

I wouldn't go quite that far. xt's do not need to be associated with
dictionary entries at all, while nt's require a dictionary entry/header.
But the general principle is correct that nt provides more information
for smarter compilation.

--
Krishna

[Mike]

So how come there is a data type smart word (TO) in Forth? Are there any others ?

I have implemented 2TO instead. Is the smart TO in the standard ? Are locals the reason for it?

[Krishna Myneni]

Yes, it can; however, if you need to use +TO my advice would be to not
use a VALUE (or FVALUE or other extended VALUE) but go back to
VARIABLEs. One can use of +! , F+! (not currently standard, but
necessary), etc. with corresponding variables. A drawback of TO and
extensions such as +TO is that various Forth operations such as '
(tick), ['], POSTPONE, etc. on them results in an ambiguous condition in
standard Forth.

Having said that, if I were to implement +TO , I would set up a table
which contains nt's for various TO operators (TO is really !TO). Then,
the extended VALUEs only need to provide an index into the table, from
which TO and +TO could look up the corresponding nt's and from those
obtain the necessary execution semantics at compile time. Then xVALUE
will have to be given a set of nt's for initialization of the table
entries when the value is created, corresponding to implemented TO
operations such as +TO. This is a lot of work, and I don't think the
reward is worth it.

Nick Nelson gave a talk at EuroForth 2020 on extensions to VALUEs. The
video may be found here.

https://www.youtube.com/watch?v=X-qD5PKdCcI

--
Krishna

[Krishna Myneni]

On 9/29/22 05:05, Mike wrote:
> So how come there is a data type smart word (TO) in Forth? Are there any others ?
>
> I have implemented 2TO instead. Is the smart TO in the standard ? Are locals the reason for it?

The implementation of TO is not data-type smart -- it can be used on any
arbitrary data type of VALUE. The execution semantics for TO are
obtained from the value, i.e. the information about how to store the a
new value is contained into the named value is contained in the named value.

The Forth 2012 standard specifies that TO should work on cell size
values (VALUE), double cell values (2VALUE), and floating point values
(FVALUE).

--
Krishna

[Krishna Myneni]

On 9/29/22 05:28, Krishna Myneni wrote:
> On 9/29/22 05:05, Mike wrote:
>> So how come there is a data type smart word (TO) in Forth? Are there
>> any others ?
>>
>> I have implemented 2TO instead. Is the smart TO in the standard ? Are
>> locals the reason for it?
>
> The implementation of TO is not data-type smart -- it can be used on any
> arbitrary data type of VALUE. The execution semantics for TO are
> obtained from the value, i.e. the information about how to store the a
> new value is contained into the named value is contained in the named
> value.

Don't know what happened with my editing. Here is what was intended:

... i.e. the information about how to store a new value is contained in
the named value.

--
KM

[Zbig]

> So how come there is a data type smart word (TO) in Forth? Are there any others ?

So state-smart words are bad — but data-type-smart words are (still?) good? ;)

[none albert]

In article <647d397d-bba8-496a...@googlegroups.com>,

!!!!!!

[dxforth]

On 29/09/2022 7:55 pm, albert wrote:
> In article <th3om4$10lr$1...@gioia.aioe.org>, dxforth <dxf...@gmail.com> wrote:
>> On 29/09/2022 6:54 pm, albert wrote:
>>> ...
>>> I have taken the stand that xt should be ditched for nt (dea, Dictionary
>>> Entry Address). All what is accomplished by xt can be done better by nt,
>>> because there is more information available.
>>> To those who are concerned with efficiency (read speed), they are
>>> invited to add some optimizing to the resulting code.
>>>
>>> The xt is a half assed concept introduced by ISO 93, when we did not
>>> know better.
>>
>> Needs vary. Nearly every application I write ditches the dictionary in
>> favour of the space it occupied. That's a 35% gain in free memory at
>> run-time.
>>
>>
> That is a non-sequitur.

Not unless "we" means you.

[Anton Ertl]

Krishna Myneni <krishna...@ccreweb.org> writes:
>------
>: NONDEFERRED ;
>SYNONYM A@ @

Vade retro, STATE-smartness!

For comparison, here's a simplified version of Gforth's implementation
of TO:

--------------------
\ First, the method implementation of the method (TO) for VALUEs:
: value-to ( n value-xt -- ) \ gforth-internal
\g this is the TO-method for normal values
>body ! ;

: value ( n "name" -- )
create ,
['] @ set-does> \ like DOES> @ ;
['] value-to set-to \ The method implementation for (TO) is now VALUE-TO
;

: name>(to) ( nt -- xt-(to-implementation )
\ given an nt, the following magic incantation produces the xt of
\ the (TO) implementation of the word represented by nt.
>namehm @ >hmto @ ;

\ Now we (re)implement the method selector (TO)
: (to) ( v nt -- )
dup name>(to) execute ;

\ interpretation semantics for TO; (') produces the nt of "name"
: <TO> ( v "name" -- ) \ gforth
(') (to) ;

\ compilation semantics for TO
: [TO] ( compilation "name" -- ; run-time v -- ) \ gforth bracket-is
(') ]] literal (to) [[ ; immediate

\ now combine them into the word TO
' <TO> ' [TO] interpret/compile: TO ( value "name" -- ) \ core-ext
\g changes the value of @var{name} to @var{value}
------------------------

Note that this relies on the new Gforth header [paysan&ertl19], where
each word can have a (TO) method implementation, and the
interpretation semantics <TO> of TO and the compilation semantics [TO]
of TO both perform the method (to) for the word. Values use the
implementation VALUE-TO above, while 2values use one that performs a
2!, and Fvalues use one that performs an F!. The ordinary execution
semantics of a value does not deal with TO in any way, the (TO) method
is separate from the execution semantics.

The code above does not show the optimizations that allow to eliminate
basically all overhead. I posted a (longer) version with
optimizations in <2022Mar1...@mips.complang.tuwien.ac.at>, and
showed the result of compilation:

0 value x
: to-x to x ;
see to-x \ this decompiles : to-x ['] x ! ;

Note that the address stored to is the body address of X; in the
version of Gforth used, the xt and body of X are the same, so the body
is decompiled as ['] X.

Concerning +TO, the actual implementation of Gforth supports +TO
(unlike the implementation above), but the implementation is not
pretty and introduces another complication. In principle it's
possible to have a separate method (+TO), but the current Gforth
implementation instead hangs it all onto the (TO) implementation in an
ugly, but extensible way, that I am too embarrased about to explain
it.

@InProceedings{paysan&ertl19,
author = {Bernd Paysan and M. Anton Ertl},
title = {The new {Gforth} Header},
crossref = {euroforth19},
pages = {5--20},
url = {http://www.euroforth.org/ef19/papers/paysan.pdf},
url-slides = {http://www.euroforth.org/ef19/papers/paysan-slides.pdf},
video = {https://wiki.forth-ev.de/doku.php/events:ef2019:header},
OPTnote = {refereed},
abstract = {The new Gforth header is designed to directly
implement the requirements of Forth-94 and
Forth-2012. Every header is an object with a fixed
set of fields (code, parameter, count, name, link)
and methods (\texttt{execute}, \texttt{compile,},
\texttt{(to)}, \texttt{defer@}, \texttt{does},
\texttt{name>interpret}, \texttt{name>compile},
\texttt{name>string}, \texttt{name>link}). The
implementation of each method can be changed
per-word (prototype-based object-oriented
programming). We demonstrate how to use these
features to implement optimization of constants,
\texttt{fvalue}, \texttt{defer}, \texttt{immediate},
\texttt{to} and other dual-semantics words, and
\texttt{synonym}.}
}

@Proceedings{euroforth19,
title = {35th EuroForth Conference},
booktitle = {35th EuroForth Conference},
year = {2019},
key = {EuroForth'19},
url = {http://www.euroforth.org/ef19/papers/proceedings.pdf}
}

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2022: https://euro.theforth.net

[Anton Ertl]

Mike <oh2...@gmail.com> writes:
>Is the smart TO in the standard ?

Yes.

>Are locals the reason for it?

In Forth-94 TO works on VALUEs and locals, so one might answer "yes"
to your question; or one might say that VALUEs are the reason for it.

It certainly is interesting that people went for using TO for all
kinds of value-flavoured words, while we have ! 2! F! C! etc. for
variable-flavoured words. I guess the reason is that TO takes a word,
and the Forth system can determine what kind of word it is, while !
etc. take an address on the stack, and a traditional Forth system
cannot determine what kind of address it is.

One could still argue that TO 2TO FTO etc. would be simpler, but
apparently people are prepared to pay the price of a little complexity
for the convenience of a polymorphic TO.

[Anton Ertl]

Zbig <zbigni...@gmail.com> writes:
>So state-smart words are bad

They are evil! [ertl98]

>=E2=80=94 but data-type-smart words are (still=
>?) good? ;)

The polymorphic TO is fine. No such problems as with STATE-smartness.
It feels somewhat alien in Forth, though. It took me and many others
a few decades to warm up to it, but in recent years I hear more and
more people sing the praises of value-flavoured words and TO, e.g.,
Joerg Voelker and Nick Nelson (I can give references if there is
demand).

@InProceedings{ertl98,
author = {M. Anton Ertl},
title = {\texttt{State}-smartness --- Why it is Evil and How
to Exorcise it},
booktitle = {EuroForth'98 Conference Proceedings},
year = {1998},
address = {Schlo\ss{} Dagstuhl},
url = {http://www.complang.tuwien.ac.at/papers/ertl98.ps.gz},
abstract = {\texttt{State}-smart words provide a number of unpleasant
surprises to their users. They are applied in two
contexts, and they fail in both: 1) for providing an
arbitrary combination of interpretation and
compilation semantics; 2) for optimizing with a
special implementation of the (default) compilation
semantics. This paper discusses these issues and
shows programmers and system implementors how to
avoid \texttt{state}-smart words. It also reports our
experiences in converting the \texttt{state}-smart
words in Gforth into a clean solution: little work
and few problems.}

[Zbig]

> more people sing the praises of value-flavoured words and TO

Well, when I'm sure I won't use the address, then indeed using VALUE
means one „funny character” less ('@').
Anyway VARIABLE is still most universal.

[P Falth]

On Thursday, 29 September 2022 at 19:42:58 UTC+2, Anton Ertl wrote:
> Mike <oh2...@gmail.com> writes:
> >Is the smart TO in the standard ?
> Yes.
> >Are locals the reason for it?
> In Forth-94 TO works on VALUEs and locals, so one might answer "yes"
> to your question; or one might say that VALUEs are the reason for it.
>
> It certainly is interesting that people went for using TO for all
> kinds of value-flavoured words, while we have ! 2! F! C! etc. for
> variable-flavoured words. I guess the reason is that TO takes a word,
> and the Forth system can determine what kind of word it is, while !
> etc. take an address on the stack, and a traditional Forth system
> cannot determine what kind of address it is.
>
> One could still argue that TO 2TO FTO etc. would be simpler, but
> apparently people are prepared to pay the price of a little complexity
> for the convenience of a polymorphic TO.
> - anton

I was not prepared to pay that price!
Instead I stopped using values of any kind!
And I have not ever looked back for them.
I still provide them to be able to include other peoples code.

On my systems variables and values also produce the same code
This is from the tokenized code of nft64/lxf64
0 value v1 ok
variable v2 ok
: t1 111 to v1 ; ok
: t2 111 v2 ! ; ok

see t1 9 bytes starting at $00A0´A538
Address OP Instruction
$00A0´A538 26 LIT1 $006F 111
$00A0´A53A 29 LIT4 $0040´60E8 4´219´112
$00A0´A53F 21 !
$00A0´A540 25 RET
ok
see t2 9 bytes starting at $00A0´A558
Address OP Instruction
$00A0´A558 26 LIT1 $006F 111
$00A0´A55A 29 LIT4 $0040´60F0 4´219´120
$00A0´A55F 21 !
$00A0´A560 25 RET
ok

In fact TO is implemented as first compiling the value
and then exchanging the @, 2@, F@ to a !, 2!, F!
If state is compiling it is ready there
If state is interpreting a ret is compiled, the
code is executed and finally it is removed by minus alloting.

BR
Peter

[Krishna Myneni]

On 9/28/22 19:39, Krishna Myneni wrote:

> ... The improved efficiency is realized because NAME>COMPILE can

> recognize a primitive word and generate byte code for a direct call to
> the primitive word rather than indirect execution via EXECUTE --
> basically NAME>COMPILE permits a compiler-specific optimization in
> kForth, which would otherwise not be possible if one were dealing with
> xt's.
>

...

Two-separate extended value implementations, one based on xt's only, and
one using nt's, are benchmarked in kForth-64:


Extended Values: nt version

Elapsed times in ms shown below -- lower is better.

VALUEs benchmark: 1999

VARIABLEs benchmark: 1995
ok

In the above version using nt's, VALUEs incur little or no overhead
compared to using VARIABLES when invoking them or using TO on them in a
definition.

Extended Values: xt version

Elapsed times in ms shown below -- lower is better.

VALUEs benchmark: 2529

VARIABLEs benchmark: 1992
ok

A significant penalty for use of VALUEs and TO in a definition is
incurred with the version using xt's only in kForth-64.

The test code and the two implementations of extended values (xt and nt
versions) are given below.

--
Krishna Myneni

Two implementations of extended values using standard Forth only:
-----
\ values.4th
\
\ Two reference implementations of extended values for
\ Forth 200x.
\
\ K. Myneni, 2022-09-29

0 [IF]

: TO ( i*x "name" -- ) \ or ( F: i*r -- ) ( "name" -- )
' >body
dup a@ swap cell+
state @ IF
postpone literal

postpone literal
postpone execute
ELSE
swap execute
THEN ; immediate

: VALUE ( x "name" -- )
create 2 cells allot?
['] ! over ! cell+ ! immediate

does> cell+ state @ if
postpone literal postpone @
else @ then ;

: 2VALUE ( x1 x2 "name" -- )

create 3 cells allot?
['] 2! over ! cell+ 2! immediate

does> cell+ state @ if
postpone literal postpone 2@
else 2@ then ;

: FVALUE ( F: r -- ) ( "name" -- )

create 1 cells 1 floats + allot?
['] F! over ! cell+ F! immediate

does> cell+ state @ if
postpone literal postpone f@
else f@ then ;

cr cr .( Extended Values: xt version ) cr

[ELSE]


\ Use following compatible defns of A@ and NONDEFERRED for
\ standard Forth systems:
\
\ : NONDEFERRED ;
\ SYNONYM A@ @

: TO ( i*x "name" -- ) \ or ( F: i*r -- ) ( i*x "name" -- )

' >body
dup a@ swap cell+
state @ IF
postpone literal
name>compile execute
ELSE
swap name>interpret execute
THEN ; immediate

: xVALUE ( i*x nt-put usize "name" -- ) ( F: j*r -- )

create 1 cells + allot? \ -- i*x nt-put a
2dup ! cell+ swap name>interpret execute

immediate nondeferred \ -- ( -- ) ( F: -- )

;

: VALUE ( n "name" -- )
[ s" !" find-name ] literal 1 cells xVALUE
does> cell+ state @ if postpone literal postpone @
else @ then ;

: 2VALUE ( x1 x2 "name" -- )
[ s" 2!" find-name ] literal 2 cells xVALUE
does> cell+ state @ if postpone literal postpone 2@
else 2@ then ;

: FVALUE ( F: r -- ) ( "name" -- )
[ s" F!" find-name ] literal 1 floats xVALUE
does> cell+ state @ if postpone literal postpone f@

else a@ then ;

cr cr .( Extended Values: nt version ) cr

[THEN]
-------

Test code for kForth-64:
-------
\ test-values.4th
\
\ Benchmark of kForth implementation of extended values vs
\ variables.

include ans-words
include values

200000000 constant N_iter

0 value jval
2 value kval

: val-bench ( -- )
1 to jval
begin
jval kval + to jval
jval N_iter >
until
;

cr .( Elapsed times in ms shown below -- lower is better. ) cr
cr .( VALUEs benchmark: )
ms@ val-bench ms@ swap - . cr


VARIABLE jvar
VARIABLE kvar
0 jvar !
2 kvar !

: var-bench ( -- )
1 jvar !
begin
jvar @ kvar @ + jvar !
\ kvar @ jvar +!
jvar @ N_iter >
until
;

cr .( VARIABLEs benchmark: )
ms@ var-bench ms@ swap - . cr
-------

[Krishna Myneni]

On 9/29/22 11:56, Anton Ertl wrote:
> Krishna Myneni <krishna...@ccreweb.org> writes:
>> ------
>> : NONDEFERRED ;

>> SYNONYM A@ @ ...

>
> Vade retro, STATE-smartness!
>
> For comparison, here's a simplified version of Gforth's implementation
> of TO:
>
> --------------------
> \ First, the method implementation of the method (TO) for VALUEs:
> : value-to ( n value-xt -- ) \ gforth-internal
> \g this is the TO-method for normal values
> >body ! ;
>
> : value ( n "name" -- )
> create ,
> ['] @ set-does> \ like DOES> @ ;
> ['] value-to set-to \ The method implementation for (TO) is now VALUE-TO
> ;
>
> : name>(to) ( nt -- xt-(to-implementation )
> \ given an nt, the following magic incantation produces the xt of
> \ the (TO) implementation of the word represented by nt.
> >namehm @ >hmto @ ;
>

...

Yes, my implementation is state-smart. Is there a way to implement TO
VALUE 2VALUE and FVALUE per the standard using *only* standard Forth
words and no state-smart definitions?

--
Krishna

[dxforth]

On 30/09/2022 3:43 am, Anton Ertl wrote:
> Zbig <zbigni...@gmail.com> writes:
>> So state-smart words are bad
>
> They are evil! [ertl98]

Follow the money.

If the practitioners of evil refuse to stop, then they too must
be evil and eliminated.

[Anton Ertl]

Krishna Myneni <krishna...@ccreweb.org> writes:
>Is there a way to implement TO
>VALUE 2VALUE and FVALUE per the standard using *only* standard Forth
>words and no state-smart definitions?

I don't see the value of the restriction "using *only* standard Forth
words", but actually there is:

-------------------
variable val-action 0 val-action !

: to 1 cells val-action ! ;

create value-actions ' @ , ' ! ,
create 2value-actions ' 2@ , ' 2! ,
create fvalue-actions ' f@ , ' f! ,

: value ( x "name" -- )
create ,
does> ( name: -- x; to name: x -- )
value-actions val-action @ + @ execute
0 val-action ! ;

: 2value ( x1 x2 "name" -- )
create ,
does> ( name: -- x1 x2; to name: x1 x2 -- )
2value-actions val-action @ + @ execute
0 val-action ! ;

: fvalue ( r "name" -- )
create ,
does> ( name: -- r; to name: r -- )
fvalue-actions val-action @ + @ execute
0 val-action ! ;
-------------------

This is not particularly efficient. For an efficient implementation,
drop the restriction, and look at Gforth's implementation (or my
postings <2022Sep2...@mips.complang.tuwien.ac.at> and
<2022Mar1...@mips.complang.tuwien.ac.at>.

[Anton Ertl]

P Falth <peter....@gmail.com> writes:
>On my systems variables and values also produce the same code
>This is from the tokenized code of nft64/lxf64
>0 value v1 ok
>variable v2 ok
>: t1 111 to v1 ; ok
>: t2 111 v2 ! ; ok

On Gforth this also produces essentially the same code:

simple-see t1
$7FC1CA0AD518 lit
$7FC1CA0AD520 <111>
$7FC1CA0AD528 lit
$7FC1CA0AD530 <v1>
$7FC1CA0AD538 !
$7FC1CA0AD540 ;s ok
simple-see t2
$7FC1CA0AD570 lit
$7FC1CA0AD578 <111>
$7FC1CA0AD580 lit
$7FC1CA0AD588 <v2>
$7FC1CA0AD590 !
$7FC1CA0AD598 ;s ok

For more advanced compilers (more advanced than we have in Forth at
the moment), value-flavoured words have an advantage in making it at
least easier to reorder accesses or allocating it to a register.
E.g., if you have a loop that stores to memory and (only) reads from a
value v, you can rely on the stores not changing the value (I assume
that the Forth system does not allow ADDR v), and therefore load it
into a register before the loop instead of fetching it from memory in
every iteration (as would be necessary if v was a variable).

[none albert]

In article <th5dms$psea$1...@dont-email.me>,
Krishna Myneni <krishna...@ccreweb.org> wrote:
<SNIP>

>Yes, my implementation is state-smart. Is there a way to implement TO
>VALUE 2VALUE and FVALUE per the standard using *only* standard Forth
>words and no state-smart definitions?

It is easy and it is trivial, but not fast.
See VALUE in the .lab blocks.
It has long appreciated that values are poor man's
message interpretating objects. See the documentation
for tforth (A.D. 1994).

-------------------
( VALUE TO FROM ) \ AvdH B2aug07


VARIABLE TO-MESSAGE \ 0 : FROM , 1 : TO .
DATA _value_jumps '@ , '! , '+! ,
: FROM 0 TO-MESSAGE ! ;
\ ISO
: TO 1 TO-MESSAGE ! ;
\ Signal that we want to add to value
: +TO 2 TO-MESSAGE ! ;

\ ISO
: VALUE CREATE , DOES> _value_jumps TO-MESSAGE @ CELLS +
@ EXECUTE FROM ;

-------------------

There was a discussion about the original ANSI 93 rule
that "TO must parse". However it has been established that
there is no standard conforming program that can test
whether that is the case. Shooting down/ignoring that
remark is part of the argument that non state smart
implementations stands.

>Krishna

Groetjes Albert

[Krishna Myneni]

...

Ok, thanks. The value in being able to implement with only standard
Forth code is that the functionality can be added quickly to a Forth
system, without tampering with the system itself, and used to check
whether or not there will be any side effects from the extensions to
existing Forth programs.

Apart from the minor errors in your definitions of 2VALUE and FVALUE
involving "CREATE ,", which may be easily replaced by use of standard
Forth words which perform the proper initialization, the code above is
easy to follow and has the benefit of not being state-smart. The
tradeoff is that the compilation behavior of TO cannot be optimized.

A dual semantics system such as Gforth, or a system which can override
default compilation semantics (e.g. I believe VFX has a word called
NDCS: for doing this) is helpful to overcome the tradeoff.

--
Krishna

[Krishna Myneni]

Yes, I was stuck on the notion of a parsing TO which led to my
particular implementation. The restriction in the Forth 2012 standard
that "An ambiguous condition exists if any of POSTPONE, [COMPILE], '
(tick) or ['] are applied to TO." (6.2.2295) is apparently a concession
to the allowance for implementing a parsing TO.

--
Krishna

[Stephen Pelc]

On 30 Sep 2022 at 13:05:51 CEST, "Krishna Myneni" <krishna...@ccreweb.org>
wrote:

> A dual semantics system such as Gforth, or a system which can override
> default compilation semantics (e.g. I believe VFX has a word called
> NDCS: for doing this) is helpful to overcome the tradeoff.

<font color="#000000">I have written several EuroForth papers about
implementing words with non-default compilation semantics in a
standards-compliant way. NDCS: is one of the support words. The other change
you would probably need to make is to have an NDCS bit in the dictionary
header.</font>
<font color="#000000"></font>
<font color="#000000">One of the interesting impacts of this is that the
IMMEDIATE bit can go away, and an IMMEDIATE word is just an NDCS word with the
same actions in interpretation and compilation states. </font>
<font color="#000000"></font>
<font color="#000000">The simplicity of the final solution implemented in VFX
(al versions) is one of the things that convinces me that the solution is
correct. VFX is supplied with full source code.</font>
<font color="#000000"></font>
<font color="#000000">Stephen</font>
<font color="#000000"></font>
Stephen Pelc, ste...@vfxforth.com
MicroProcessor Engineering, Ltd. - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612,
+34 649 662 974
http://www.mpeforth.com - free VFX Forth downloads

[Stephen Pelc]

A previous response seems to have become garbled, so here it is again.

On 30 Sep 2022 at 13:05:51 CEST, "Krishna Myneni" <krishna...@ccreweb.org>
wrote:

> A dual semantics system such as Gforth, or a system which can override
> default compilation semantics (e.g. I believe VFX has a word called
> NDCS: for doing this) is helpful to overcome the tradeoff.

I have written several EuroForth papers about implementing words with
non-default compilation semantics in a standards-compliant way. NDCS: is one
of the support words. The other change you would probably need to make is to
have an NDCS bit in the dictionary header.

One of the interesting impacts of this is that the IMMEDIATE bit can go away,
and an IMMEDIATE word is just an NDCS word with the same actions in
interpretation and compilation states.

The simplicity of the final solution implemented in VFX (all versions) is one

of the things that convinces me that the solution is correct. VFX is supplied

with full source code. See kernel.fth or kernel64.fth.

Stephen

--

[Krishna Myneni]

On 9/30/22 10:44, Stephen Pelc wrote:
> A previous response seems to have become garbled, so here it is again.
>
> On 30 Sep 2022 at 13:05:51 CEST, "Krishna Myneni" <krishna...@ccreweb.org>
> wrote:
>> A dual semantics system such as Gforth, or a system which can override
>> default compilation semantics (e.g. I believe VFX has a word called
>> NDCS: for doing this) is helpful to overcome the tradeoff.
>
> I have written several EuroForth papers about implementing words with
> non-default compilation semantics in a standards-compliant way. NDCS: is one
> of the support words. The other change you would probably need to make is to
> have an NDCS bit in the dictionary header.
>
> One of the interesting impacts of this is that the IMMEDIATE bit can go away,
> and an IMMEDIATE word is just an NDCS word with the same actions in
> interpretation and compilation states.
>
> The simplicity of the final solution implemented in VFX (all versions) is one
> of the things that convinces me that the solution is correct. VFX is supplied
> with full source code. See kernel.fth or kernel64.fth.
>

If I wanted to use my implementation of extended VALUEs with NDCS: to
remove state-smart words, how would the following words be modified?

: TO ( i*x "name" -- ) \ or ( F: i*r -- ) ( "name" -- )
' >body
dup a@ swap cell+
state @ IF
postpone literal
name>compile execute
ELSE
swap name>interpret execute
THEN ; immediate

: VALUE ( n "name" -- )
[ s" !" find-name ] literal 1 cells xVALUE
does> cell+ state @ if postpone literal postpone @
else @ then ;

( xVALUE is not a state smart word ).

I think it would be useful for many of the readers here on clf to see a
side by side comparison of my state-smart implementation above and the
non-state-smart implementations using VFX's NDCS: and Gforth's
dual-semantics approach.

--
Krishna

[Anton Ertl]

Krishna Myneni <krishna...@ccreweb.org> writes:
>If I wanted to use my implementation of extended VALUEs with NDCS: to
>remove state-smart words, how would the following words be modified?
>
>
>: TO ( i*x "name" -- ) \ or ( F: i*r -- ) ( "name" -- )
> ' >body
> dup a@ swap cell+
> state @ IF
> postpone literal
> name>compile execute
> ELSE
> swap name>interpret execute
> THEN ; immediate

This is a parsing word. Here you want compilation semantics that
differ from both default and immediate compilation semantics, so a
word like COMPSEM: (or the more beautiful VFX name NDCS:) is
appropriate. You can define TO as follows (untested):

require set-compsem.fs

: to ( i*x "name" -- )
' >body dup a@ swap cell+ swap name>interpret execute ;
compsem: ' >body dup a@ swap cell+ postpone literal name>compile execute ;

>: VALUE ( n "name" -- )
> [ s" !" find-name ] literal 1 cells xVALUE
> does> cell+ state @ if postpone literal postpone @
> else @ then ;

The word defined by a VALUE has default compilation semantics, so
COMPSEM: is wrong (and STATE-smartness is wrong, too; it's not hard to
construct an example where this VALUE misbehaves).

What you are trying to do here is to optimize the compiled code, so
you can define value as follows:

: value-does> does> cell+ @ ;

: value ( n "name" -- )

[ s" !" find-name ] literal 1 cells xVALUE

value-does>
[: >body cell+ postpone literal postpone @ ;] set-optimizer ;

In VFX SET-OPTIMIZER is called SET-COMPILER, which misleads people
into believing that it changes the compilation semantics.

[Stephen Pelc]

: operator \ n --
\ *G Define an operator with the given number.
create
here swap , OperatorChain @ , OperatorChain !
['] opCompiler set-compiler
interp>
@ OperatorType ! ;
: BAD-METHOD \ --
#-404 throw
;

: (s_val,) \ caddr --
\ Given the data address, compile the run time action for a VALUE.
[compile] literal
case OperatorType @off
0 of postpone @ endof
1 of postpone ! endof
2 of endof
3 of postpone incr endof
4 of postpone decr endof
5 of postpone +! endof
6 of postpone off endof
7 of postpone -! endof
8 of postpone drop postpone cell endof
9 of postpone on endof
bad-method
endcase
;

: valInterp \ addr --
\ Given a data address, interpret the VALUE.
case OperatorType @off
0 of @ endof
1 of ! endof
2 of endof
3 of incr endof
4 of decr endof
5 of +! endof
6 of off endof
7 of -! endof
8 of drop cell endof
9 of on endof
bad-method
endcase
;

: value \ n -- ; ??? -- ??? ; 6.2.2405
\ *G Create a variable with an initial value. When the *\fo{VALUE}'s
\ ** name is referenced, the value is returned. Precede the name
\ ** with *\fo{TO} or *\fo{->} to store to it. Precede the name
\ ** with *\fo{ADDR} to get the address of the data. The full list
\ ** of operators is displayed by *\fo{.OPERATORS ( -- )}.
\ *E 5 VALUE FOO \ initial value of FOO is 5
\ ** FOO . \ will give 5
\ ** 6 TO FOO \ new value is 6
\ ** FOO . \ will give 6
\ ** ADDR FOO @ . \ will give 6
create
, ['] valComp, set-compiler
interp>
valInterp
;

INTERP> is like DOES> but specifies only the interpretation behaviour of the
child, whereas SET-COMPILER specifies how the child is compiled.

For plain words, here's the definition of S" using NDCS:

: S" \ Comp: "ccc<quote>" -- ; Run: -- c-addr u 6.1.2165
\ *G Describe a string. Text is taken up to the next double-quote
\ ** character. The address and length of the string are returned.
[char] " parse >syspad
;
ndcs: ( -- ) discard-sinline postpone (s") ", ;

: IF \ C: -- orig ; Run: x -- 6.1.1700
\ *G Mark the start of an *\fo{IF ... [ELSE] ... THEN} conditional block.
\ ** *\fo{ELSE} is optional.
NoInterp
;
ndcs: ( -- ) s_?br>, ?checking if 2 then ;

[Krishna Myneni]

Something is not quite right with the compilation semantics of your
revised TO.

From Gforth:

5 value n1 ok
n1 . 5 ok
7 to n1 ok
n1 . 7 ok
: test n1 n1 + to n1 ;
*the terminal*:8:22: error: Control structure mismatch
: test n1 n1 + to n1 >>>;<<<

Here's the code:
----
\ values2.4th

require set-compsem.fs
include contrib/kforth-compat.fs

: to ( i*x "name" -- )
' >body dup a@ swap cell+ swap name>interpret execute ;

compsem: ' >body dup a@ swap cell+ postpone literal
name>compile execute ;

: xVALUE ( i*x nt-put usize -- ) ( F: j*r -- )
create 1 cells + allot? \ -- i*x nt-put a

2dup ! cell+ swap name>interpret execute immediate ;

: value-does> does> cell+ @ ;

: value ( n "name" -- )
[ s" !" find-name ] literal 1 cells xVALUE
value-does>
[: >body cell+ postpone literal postpone @ ;] set-optimizer ;

----


--
KM

[Anton Ertl]

Krishna Myneni <krishna...@ccreweb.org> writes:
>Something is not quite right with the compilation semantics of your
>revised TO.

It turns out that the compilation semantics of the words created with
xVALUE was wrong (it created immediate values).

Here's a working version:

\ values2.4th

require set-compsem.fs
\ include contrib/kforth-compat.fs

synonym a@ @

: to ( i*x "name" -- )
' >body dup a@ swap cell+ swap name>interpret execute ;

compsem: ' >body dup a@ swap cell+ postpone literal
name>compile execute ;

: xVALUE ( i*x nt-put usize -- ) ( F: j*r -- )

create 1 cells + here >r allot r> \ -- i*x nt-put a
2dup ! cell+ swap name>interpret execute ;

: value-does> does> cell+ @ ;

: value ( n "name" -- )
[ s" !" find-name ] literal 1 cells xVALUE
value-does>
[: >body cell+ postpone literal postpone @ ;] set-optimizer ;

\ tests
5 value n1
n1 .
7 to n1
n1 .
: test1 n1 . ;
cr test1
see test1

: test n1 n1 + to n1 ;

cr test n1 .
see test

[Krishna Myneni]

I've extended the definitions for 2VALUE and FVALUE based on your
examples, and under Gforth they pass all of the tests in my original post :

----

5 value n1 ok
n1 . 5 ok
7 to n1 ok
n1 . 7 ok

: test n1 n1 + to n1 ; ok
test ok
n1 . 14 ok
ok
3.14e fvalue almostPI ok
almostPI f. 3.14 ok
3.14159e to almostPI ok
almostPI f. 3.14159 ok
ok
-1 0 2value d1 ok
d1 d. 18446744073709551615 ok
ok
: test2
cr ['] almostPI execute f.
cr ['] n1 execute .
cr ['] d1 execute d.
cr ; ok
ok
test2
3.14159
14
18446744073709551615
ok
ok
: test3 postpone almostPI postpone n1 postpone d1 ; immediate ok
: test4 test3 cr d. cr . cr f. cr ; ok
ok
test4
18446744073709551615
14
3.14159
ok
----

The efficiency of the redefined VALUE appears to be the same to your
built-in VALUE in Gforth.

----
\ values2.4th

require set-compsem.fs
synonym a@ @
: allot? ( u -- a ) here swap allot ;

: to ( i*x "name" -- )
' >body dup a@ swap cell+ swap name>interpret execute ;

compsem: ' >body dup a@ swap cell+ postpone literal
name>compile execute ;

: xVALUE ( i*x nt-put usize -- ) ( F: j*r -- )

create 1 cells + allot? \ -- i*x nt-put a

2dup ! cell+ swap name>interpret execute ;

: value-does> does> cell+ @ ;

: value ( n "name" -- )
[ s" !" find-name ] literal 1 cells xVALUE
value-does>
[: >body cell+ postpone literal postpone @ ;] set-optimizer ;

: 2value-does> does> cell+ 2@ ;

: 2value ( n1 n2 "name" -- )

[ s" 2!" find-name ] literal 2 cells xVALUE

2value-does>
[: >body cell+ postpone literal postpone 2@ ;] set-optimizer ;

: fvalue-does> does> cell+ f@ ;

: fvalue ( F: r -- ) ( "name" -- )
[ s" f!" find-name ] literal 1 floats xVALUE
fvalue-does>
[: >body cell+ postpone literal postpone f@ ;] set-optimizer ;
----

I still don't get how SET-OPTIMIZER works. Earlier you stated,

"In VFX SET-OPTIMIZER is called SET-COMPILER, which misleads people
into believing that it changes the compilation semantics."

Doesn't SET-OPTIMIZER change the compilation semantics of the words
created by VALUE 2VALUE etc.?

--
Krishna

[Krishna Myneni]

On 9/30/22 17:02, Krishna Myneni wrote:
> On 9/30/22 15:56, Anton Ertl wrote:

...

>> Here's a working version:
>>
>> \ values2.4th
>>
>> require set-compsem.fs
>> \ include contrib/kforth-compat.fs
>> synonym a@ @
>>
>> : to ( i*x "name" -- )
>>       ' >body dup a@ swap cell+ swap name>interpret execute ;
>>
>> compsem: ' >body dup a@ swap cell+ postpone literal
>>            name>compile execute ;
>>
>> : xVALUE ( i*x nt-put usize -- ) ( F: j*r -- )
>>       create 1 cells + here >r allot r>  \ -- i*x nt-put a
>>       2dup ! cell+ swap name>interpret execute ;
>>
>> : value-does> does> cell+ @ ;
>>
>> : value ( n "name" -- )
>>      [ s" !" find-name ] literal 1 cells xVALUE
>>      value-does>
>>      [: >body cell+ postpone literal postpone @ ;] set-optimizer ;
>>

One may also eliminate the intermediate value-does> word(s) using
quotations.

----
\ values2.4th

require set-compsem.fs
synonym a@ @
: allot? ( u -- a ) here swap allot ;

: to ( i*x "name" -- )
' >body dup a@ swap cell+ swap name>interpret execute ;

compsem: ' >body dup a@ swap cell+ postpone literal
name>compile execute ;

: xVALUE ( i*x nt-put usize -- ) ( F: j*r -- )
create 1 cells + allot? \ -- i*x nt-put a
2dup ! cell+ swap name>interpret execute ;

: value ( n "name" -- )
[ s" !" find-name ] literal 1 cells xVALUE

[: does> cell+ @ ;] execute

[: >body cell+ postpone literal postpone @ ;] set-optimizer ;

: 2value ( n1 n2 "name" -- )
[ s" 2!" find-name ] literal 2 cells xVALUE

[: does> cell+ 2@ ;] execute

[: >body cell+ postpone literal postpone 2@ ;] set-optimizer ;

: fvalue ( F: r -- ) ( "name" -- )
[ s" f!" find-name ] literal 1 floats xVALUE

[: does> cell+ f@ ;] execute

[: >body cell+ postpone literal postpone f@ ;] set-optimizer ;
----

This implementation under Gforth also passes all of my original post tests.

--
KM

[Anton Ertl]

Krishna Myneni <krishna...@ccreweb.org> writes:
>I still don't get how SET-OPTIMIZER works. Earlier you stated,
>
>"In VFX SET-OPTIMIZER is called SET-COMPILER, which misleads people
>into believing that it changes the compilation semantics."
>
>Doesn't SET-OPTIMIZER change the compilation semantics of the words
>created by VALUE 2VALUE etc.?

No. SET-OPTIMIZER sets the implementation of COMPILE, ( xt -- ) for
the current word. A correct implementation of COMPILE, does not
change the semantics in any way, only the implementation of the
semantics. So if you instead do

[: postpone literal postpone execute ;] set-optimizer

the behaviour stays the same.

If you want to change the compilation semantics, Gforth has SET->COMP,
which sets the implementation of NAME>COMPILE for the current word.
Gforth also (after loading set-compsem.fs) has the higher-level words
SET-COMPSEM and COMPSEM:. The difference is that you have to pass an
xt with the stack effect ( nt -- xt1 xt2 ) to SET->COMP. By contrast,
with SET-COMPSEM you pass xt1; xt2 is the xt of EXECUTE and nt is not
needed for the intended uses of SET-COMPSEM (if you want to change the
semantics in a way that makes use of the nt, use SET->COMP). With
COMPSEM: the following code becomes the xt passed to SET-COMPSEM.

[Anton Ertl]

Krishna Myneni <krishna...@ccreweb.org> writes:
>One may also eliminate the intermediate value-does> word(s) using
>quotations.

...

>: value ( n "name" -- )
> [ s" !" find-name ] literal 1 cells xVALUE
> [: does> cell+ @ ;] execute
> [: >body cell+ postpone literal postpone @ ;] set-optimizer ;

Another way to do that is to use SET-DOES> (an xt-taking variant of
DOES>):

: value ( n "name" -- )
[ s" !" find-name ] literal 1 cells xVALUE

[: cell+ @ ;] set-does>

[: >body cell+ postpone literal postpone @ ;] set-optimizer ;

[Anton Ertl]

Krishna Myneni <krishna...@ccreweb.org> writes:
>On 9/30/22 04:30, albert wrote:
>> There was a discussion about the original ANSI 93 rule
>> that "TO must parse".

There is no such rule. Forth-94 and 2012 describe TO as parsing
words, allowing the implementation of a parsing TO. But, as you
mention, there is no standard program that could determine if TO does
not parse, so a standard system can also have a non-parsing TO.

>Yes, I was stuck on the notion of a parsing TO which led to my
>particular implementation. The restriction in the Forth 2012 standard
>that "An ambiguous condition exists if any of POSTPONE, [COMPILE], '
>(tick) or ['] are applied to TO." (6.2.2295) is apparently a concession
>to the allowance for implementing a parsing TO.

In Forth-2012 this is intended to allow STATE-smart implementations of
TO (intended for parsing implementations of TO).

Forth-94 only makes POSTPONEing and [COMPILE]ing TO ambiguous. My
explanation used to be that this was there to allow state-smart
implementations of TO, and they just missed to also mention ' and ['].
But somebody came up with a plausible alternative explanation (which I
unfortunately I don't remember), so it might also be that other
reason.

[Anton Ertl]

Now that you have dropped the restriction, I also did a variant of
that using SET-OPTIMIZER:

----------------
variable val-action 0 val-action !

: to 1 cells val-action ! ; immediate

create value-actions ' @ , ' ! ,
create 2value-actions ' 2@ , ' 2! ,
create fvalue-actions ' f@ , ' f! ,

: value ( x "name" -- ) ( name: -- x; to name: x -- )
create ,
[: value-actions val-action @ + @ execute 0 val-action ! ;] set-does>
[: >body postpone literal value-actions val-action @ + @ compile,
0 val-action ! ;] set-optimizer
;

: 2value ( x "name" -- ) ( name: -- x; to name: x -- )
create 2,
[: 2value-actions val-action @ + @ execute 0 val-action ! ;] set-does>
[: >body postpone literal 2value-actions val-action @ + @ compile,
0 val-action ! ;] set-optimizer
;

: fvalue ( x "name" -- ) ( name: -- x; to name: x -- )
create f,
[: fvalue-actions val-action @ + @ execute 0 val-action ! ;] set-does>
[: >body postpone literal fvalue-actions val-action @ + @ compile,
0 val-action ! ;] set-optimizer
;
-------------------

Factoring opportunities abound and are left to the reader.

This passes a simple test, but the question is: is it correct? This
code relies on being able to pass VAL-ACTION during compilation with
the same semantics as if it had been passed at run-time. Of course,
for code like

0 value x
: foo 5 to val-action . x ;
foo

you will see a difference to the unoptimized code, but if you
guarantee correctness only for standard programs (which may not access
VAL-ACTION and may not put other code between TO and X), I think this
is correct. At least I don't see a way it can go wrong (which is not
saying much).

AFAIK some Forth systems use a TO that passes the action index during
compilation rather than during execution, so at least in their eyes
this kind of implementation is good enough (which is not saying much).

The nice thing here is that no COMPSEM: or the like is needed (but you
do need IMMEDIATE).

[Ruvim]

On 2022-09-30 04:36, Krishna Myneni wrote:
> On 9/29/22 11:56, Anton Ertl wrote:
>> Krishna Myneni <krishna...@ccreweb.org> writes:

[...]
>>>
>>> : VALUE ( n "name" -- )

>>> [ s" !" find-name ] literal 1 cells xVALUE

>>> does> cell+ state @ if postpone literal postpone @
>>> else @ then ;
>>>

[...]
>>> ------
>>
>> Vade retro, STATE-smartness!

>>
> ...
>
> Yes, my implementation is state-smart.

"STATE-smartness" is a vague term, since the same word can be classified
as STATE-smart or not STATE-smart depending on intention of the author [1].


The actual problem is that your implementation is not
standard-compliant. Namely, the words defined by this VALUE have
STATE-dependent execution semantics (according to the does-part), while
the standard doesn't specify that [2]:

| name Execution:
| ( -- x )
|
| Place x on the stack. The value of x is that given
| when name was created, until the phrase x TO name
| is executed, causing a new value of x to be assigned
| to name.

For example:

123 value foo
' foo ( xt )
\ Executing of this xt shall place 123 on the stack
\ regardless of STATE

> Is there a way to implement TO VALUE 2VALUE and FVALUE
> per the standard using *only* standard Forth
> words and no state-smart definitions?

Yes, and there are even several different approaches (for example, see [3]).

NB: "TO" may can perform optimization in any approach (my example [4]).

NB: "TO" that is implemented on the standard basis cannot be applied to
local variables that the system provides (if any). If you want to apply
your "TO" to local variables, you should either use system's "TO" under
the hood, or implement local variables on the standard basis too and
rely on the carnal knowledge of your implementation.



[1] Re: Semantics of POSTPONE and immediate words, 2020-09-22
https://groups.google.com/g/comp.lang.forth/c/ildjR6oy6hg/m/uwg4YMnnBQAJ

[2] 6.2.2405 VALUE
https://forth-standard.org/standard/core/VALUE

[3] VALUE and TO, 2021-10-14
https://github.com/ForthHub/discussion/discussions/108

[4] Dual-semantics words via the immediacy mechanism
https://gist.github.com/ruv/8999398a106a362579586b464b13ce98


--
Ruvim

[Krishna Myneni]

On 10/1/22 05:12, Ruvim wrote:

...

> The actual problem is that your implementation is not
> standard-compliant. Namely, the words defined by this VALUE have
> STATE-dependent execution semantics (according to the does-part), while
> the standard doesn't specify that [2]:
>
>   | name Execution:
>   | ( -- x )
>   |
>   | Place x on the stack. The value of x is that given
>   | when name was created, until the phrase x TO name
>   | is executed, causing a new value of x to be assigned
>   | to name.
>
> For example:
>
>   123 value foo
>   ' foo ( xt )
>   \ Executing of this xt shall place 123 on the stack
>   \ regardless of STATE
>
>

Do you mean the following?

Using Gforth's native VALUE

5 value n1
: [execute] execute ; immediate
: test [ ' n1 ] [execute] [ . ] ; \ prints 5 upon Enter in Gforth

while my original implementation gives the following under Gforth

5 value n1
: [execute] execute ; immediate
: test [ ' n1 ] [execute] [ . ] ;
140095166437768
*the terminal*:5:33: error: Control structure mismatch
: test [ ' n1 ] [execute] [ . ] >>>;<<<
Backtrace:
kernel/cond.fs:119:26: 0 $7F6A72A1F1B0 throw
glocals.fs:582:5: 1 $7F6A72A30DD0 ?struc
kernel/comp.fs:781:5: 2 $7F6A72A15B78 ;-hook


Under kForth-64 my original implementation gives
Line 15: VM Error(-4): Stack underflow
: test [ ' n1 ] [execute] [ . ] ;

--
Krishna

[Ruvim]

Yes, implementations which fail this test case (i.e., don't print 5) are
incorrect.


--
Ruvim

[Ruvim]

On 2022-09-29 21:43, Anton Ertl wrote:
> Zbig <zbigni...@gmail.com> writes:
>> So state-smart words are bad
>
> They are evil! [ertl98]

In this regard the paper shows that it's incorrect to implement an
ordinary word as an immediate word. So, it's not just evil, it's a mistake.

But concerning 's"' (and other combined words) the paper only says (in
the section 3, on page 3):

| This definition behaves correctly as long as it
| is only processed by the text interpreter,
| but it fails with ', postpone, etc.,
| as discussed in Section 2.

This reasoning is wrong:
1. actually, it does not fail with ' (Tick);
2. if it fails with "postpone", then a problem is in "postpone", and
the problem should be corrected in "postpone".

I explained [1] why it doesn't fail with Tick:

It's obvious that in the general case, performing execution semantics in
compilation state may be inequivalent to performing these execution
semantics in interpretation state. And we don't have any ground to say
that they should be equivalent in the case of the s" word, since the
execution semantics for s" are not specified by the standard.

[1] https://forth-standard.org/standard/file/Sq#contribution-201



>> but data-type-smart words are (still?)
>> good? ;)
>
> The polymorphic TO is fine. No such problems as with STATE-smartness.
> It feels somewhat alien in Forth, though. It took me and many others
> a few decades to warm up to it, but in recent years I hear more and
> more people sing the praises of value-flavoured words and TO, e.g.,
> Joerg Voelker and Nick Nelson (I can give references if there is
> demand).


An alternative to the TO-based approach is to just define two words:
getter and setter. So, instead of a phrase "to foo" you use a single
word "to-foo". I mean a real ordinary word, so "' to-foo execute"
performs execution semantics of "to-foo". The word "to-foo" is created
automatically along with "foo".

I don't see any advantage of "to foo" over "to-foo" on the source code
level. But the "to-foo" approach has some advantages:
- no need to have the word "to" in the context,
- a setter has an xt ("to foo" doesn't have an xt),
- a setter can be easy redefined,
- it works for any new types (for example, for strings).

I use this approach, with "set-" prefix. So, for a getter "foo" the
setter "set-foo" is also created.

For dynamic strings, a setter frees the previous string (if any) before
store the new string (or resizes the memory region to make it fit the
new string). It can be implemented in the standard basis, without any
carnal knowledge of a particular Forth system.



The bottom line: "TO" is evil. Especially, in APIs.


--
Ruvim

[Krishna Myneni]

On 10/1/22 11:44, Ruvim wrote:
> On 2022-10-01 19:13, Krishna Myneni wrote:
>> On 10/1/22 05:12, Ruvim wrote:
>>
...

>>>    123 value foo
>>>    ' foo ( xt )
>>>    \ Executing of this xt shall place 123 on the stack
>>>    \ regardless of STATE
>>>
>>>
>>
>> Do you mean the following?
>>
>> Using Gforth's native VALUE
>>
>> 5 value n1
>> : [execute] execute ; immediate
>> : test [ ' n1 ] [execute] [ . ] ;   \ prints 5 upon Enter in Gforth
>>

...

>
>
> Yes, implementations which fail this test case (i.e., don't print 5) are
> incorrect.
>

Both of Anton's implementations, the variation of the one in my original
post using COMPSEM and SET-OPTIMIZER (Gforth words) and Anton's
non-parsing one using only SET-OPTIMIZER pass the above tests under
Gforth. See

https://groups.google.com/g/comp.lang.forth/c/dh347IHLDtw/m/GFLLhsrnAgAJ

and

https://groups.google.com/g/comp.lang.forth/c/dh347IHLDtw/m/SvheOJULAwAJ


The functionality of the words COMPSEM and SET-OPTIMIZER are presently
missing in kForth, and the state-smart version (I think the term
"state-smart" is appropriate for the version in the original post) is a
stopgap measure until such functionality is added. It is clear to me
that expressions such as

[ ' n1 ] [execute] [ . ]

have unambiguous meaning and should work consistently under Forth.

--
Krishna

[Krishna Myneni]

On 9/30/22 15:11, Stephen Pelc wrote:
...

> : value \ n -- ; ??? -- ??? ; 6.2.2405
> \ *G Create a variable with an initial value. When the *\fo{VALUE}'s
> \ ** name is referenced, the value is returned. Precede the name
> \ ** with *\fo{TO} or *\fo{->} to store to it. Precede the name
> \ ** with *\fo{ADDR} to get the address of the data. The full list
> \ ** of operators is displayed by *\fo{.OPERATORS ( -- )}.
> \ *E 5 VALUE FOO \ initial value of FOO is 5
> \ ** FOO . \ will give 5
> \ ** 6 TO FOO \ new value is 6
> \ ** FOO . \ will give 6
> \ ** ADDR FOO @ . \ will give 6
> create
> , ['] valComp, set-compiler
> interp>
> valInterp
> ;
>
> INTERP> is like DOES> but specifies only the interpretation behaviour of the
> child, whereas SET-COMPILER specifies how the child is compiled.
>

...Ok, I think I understand from this example how to implement my
extended VALUEs using INTERP> and SET-COMPILER . Will try this out on
VFX and see what the results are.

--
Krishna

[Krishna Myneni]

On 10/1/22 02:06, Anton Ertl wrote:
> Krishna Myneni <krishna...@ccreweb.org> writes:
>> I still don't get how SET-OPTIMIZER works. Earlier you stated,
>>
>> "In VFX SET-OPTIMIZER is called SET-COMPILER, which misleads people
>> into believing that it changes the compilation semantics."
>>
>> Doesn't SET-OPTIMIZER change the compilation semantics of the words
>> created by VALUE 2VALUE etc.?
>
> No. SET-OPTIMIZER sets the implementation of COMPILE, ( xt -- ) for
> the current word. A correct implementation of COMPILE, does not
> change the semantics in any way, only the implementation of the
> semantics. So if you instead do
>
> [: postpone literal postpone execute ;] set-optimizer
>
> the behaviour stays the same.
>

I not seeing the difference. I can put any arbitrary sequence inside the
quotation for SET-OPTIMIZER. Then the compilation semantics (behavior)
can be different from that of a word defined without SET-OPTIMIZER.

: xVALUE ( i*x nt-put usize -- ) ( F: j*r -- )
create 1 cells + allot? \ -- i*x nt-put a
2dup ! cell+ swap name>interpret execute ;

: value1 ( n "name" -- )

[ s" !" find-name ] literal 1 cells xVALUE

does> cell+ @ ;

: value2 ( n "name" -- )

[ s" !" find-name ] literal 1 cells xVALUE

[: does> cell + @ ;] execute
[: >body @ postpone literal ;] set-optimizer ;

----
5 value1 n1 ok
5 value2 n2 ok
n1 . 5 ok
n2 . 5 ok
: test1 n1 ; ok
test1 . 5 ok
: test2 n2 ; ok
test2 . 140038354505904 ok
----

Thus, it appears that SET-OPTIMIZER has been used to change the
compilation semantics (behavior) of the word VALUE2.

--
Krishna

[Anton Ertl]

Krishna Myneni <krishna...@ccreweb.org> writes:
>On 10/1/22 02:06, Anton Ertl wrote:
>> SET-OPTIMIZER sets the implementation of COMPILE, ( xt -- ) for
>> the current word. A correct implementation of COMPILE, does not
>> change the semantics in any way, only the implementation of the
>> semantics. So if you instead do
>>
>> [: postpone literal postpone execute ;] set-optimizer
>>
>> the behaviour stays the same.
>>
>
>I not seeing the difference. I can put any arbitrary sequence inside the
>quotation for SET-OPTIMIZER. Then the compilation semantics (behavior)
>can be different from that of a word defined without SET-OPTIMIZER.

Yes, you can shoot yourself in the foot with SET-OPTIMIZER. This is
Forth. But once you have done that, COMPILE, does no longer work as
specified (and, more importantly, as used in most places where
COMPILE, is used).

And you won't see it in the text interpreter, because the use of
COMPILE, in the text interpreter is one of the few places where you
cannot tell the difference. But in most other places that I inspected
where COMPILE, is used (e.g., in mini-oof and other object-oriented
packages), it is used as a more efficient way of saying POSTPONE
LITERAL POSTPONE EXECUTE, and that's what is specified for it:

|Append the execution semantics of the definition represented by xt to
|the execution semantics of the current definition.

And the rationale says:

|COMPILE, is the compilation equivalent of EXECUTE.

Why do you not see a difference in the text interpreter?

A modern text interpreter first gets the nt, and in compile state then
performs the compilation semantics as follows:

( nt ) name>compile ( xt1 xt2 ) execute

For a word with default compilation semantics like a value v, xt1 is
the xt of the word and xt2 is the xt of COMPILE,, so the compilation
semantics is to COMPILE, the xt of the word. In a traditional text
interpreter, the COMPILE, is even more explicit (the following code
shows only the case where the word is found and the text interpreter
is in compile state, as above):

( xt +-1 ) 0< if compile, else execute then

You are not the first one to make the mistake of thinking that
changing what COMPILE, does is an appropriate way of changing the
compilation semantics: After all, it works in the text interpreter.
The problem is that this does not work in most other uses.

The appropriate way to change compilation semantics in a modern text
interpreter is to change what NAME>COMPILE does. For the traditional
text interpreter, you would have to change what FIND does (possibly
making FIND results depend on STATE, as explicitly allowed by the
standard).

[Krishna Myneni]

On 10/2/22 00:00, Anton Ertl wrote:
> Krishna Myneni <krishna...@ccreweb.org> writes:
>> On 10/1/22 02:06, Anton Ertl wrote:
>>> SET-OPTIMIZER sets the implementation of COMPILE, ( xt -- ) for
>>> the current word. A correct implementation of COMPILE, does not
>>> change the semantics in any way, only the implementation of the
>>> semantics. So if you instead do
>>>
>>> [: postpone literal postpone execute ;] set-optimizer
>>>
>>> the behaviour stays the same.
>>>
>>
>> I not seeing the difference. I can put any arbitrary sequence inside the
>> quotation for SET-OPTIMIZER. Then the compilation semantics (behavior)
>> can be different from that of a word defined without SET-OPTIMIZER.
>
> Yes, you can shoot yourself in the foot with SET-OPTIMIZER. This is
> Forth. But once you have done that, COMPILE, does no longer work as
> specified (and, more importantly, as used in most places where
> COMPILE, is used).
>

..

Clearly there is a lot to think about here. Given that SET-COMPSEM takes
a new compilation xt and sets it as the new compilation semantics for
the last definition, the following does not work

: value ( n "name" -- )

[ s" !" find-name ] literal 1 cells xVALUE

[: does> cell+ @ ;] execute
[: >body cell+ postpone literal postpone @ ;] set-compsem ;

In the above, SET-OPTIMIZER is replaced by SET-COMPSEM. The reason that
the new definition fails (in compilation mode) appears to be that there
is no xt for the value on the stack for the new compilation semantics to
operate on with >BODY. Is there a way to place the xt of the last
definition on the stack, so that SET-COMPSEM can be used in the
definition of VALUE?

--
Krishna

[Anton Ertl]

Why? SET-OPTIMIZER was appropriate here: values have default
compilation semantics. And the implementation was correct.

>Is there a way to place the xt of the last
>definition on the stack, so that SET-COMPSEM can be used in the
>definition of VALUE?

You can use Gforth's closures to pass a stack item from closure
construction time to closure run-time.

: broken-value ( n "name" -- )

[ s" !" find-name ] literal 1 cells xVALUE
[: does> cell+ @ ;] execute

lastxt >body cell+ [n:d postpone literal postpone @ ;] set-compsem ;

In the following, let's assume you have a word defined with

0 broken-value n2

BROKEN-VALUE and the words created by it have the following
disadvantages compared to the VALUE that uses SET-OPTIMIZER:

* [COMPILE] N2 has the same effect as POSTPONE N2, not the same effect
as just compiling N2, which a standard value would have. The reason
is that [COMPILE] thinks it has non-default compilation semantics;
if you want to stick with default compilation semantics, don't use
SET-COMPSEM! This is the reason why I called this word
BROKEN-VALUE.

* COMPILE,ing the xt of N2 now does not optimize. The result is
correct, but not as fast as what you get with the SET-OPTIMIZER
variant.

* The closure costs some memory (3 cells or so).

[Ruvim]

I see, these conceptions are quite confusing.

"SET-OPTIMIZER" sets what the system does when "COMPILE," is applied to
the corresponding xt (actually, it's a specific variant of "COMPILE,"
and so xt is passed as an argument — to have the same interface as
"COMPILE,").

"SET-COMPSEM" sets what the system does when the corresponding word is
encountered by the Forth text interpreter in compilation state. Usually
this method is used to set non default compilation semantics, that is a
unique behavior which cannot be generalized. Therefore, no sense to pass
an xt as an argument.


A reason of confusing is that "SET-COMPSEM" can be also used to make
optimization (i.e., to make the system do compilation of a more
efficient code than usual for this particular word). But this method
cannot cover the cases when "COMPILE," is applied to the xt of an
ordinary word. However "SET-OPTIMIZER" is suitable for these cases.


Actually, these methods ("SET-COMPSEM" and "SET-OPTIMIZER") are mutual
exclusive for a word, and usually are not interchangeable.

If a word is ordinary, you have to use "SET-OPTIMIZER" to cover all
cases when the word is compiled, and it's enough — "SET-COMPSEM" is not
needed.

Hence, "SET-COMPSEM" is used for non ordinary words only. In the same
time, for these words either "COMPILE," is never used, or compilation
cannot be optimized. So, you don't use "SET-OPTIMIZER" for non ordinary
words.

Q.E.D.

> The reason that
> the new definition fails (in compilation mode) appears to be that there
> is no xt for the value on the stack for the new compilation semantics to
> operate on with >BODY.

It's absent by design, since this xt is not required on implied use cases.

See also the example in [1]:

\ :noname ." compiling" ;
\ : foo ." interpreting" ; set-compsem


Take into account that the anonymous definition intended to perform the
compilation semantics for "foo" is defined *before* "foo".
It's because "set-compsem" changes the last defined word, and an
anonymous definition is also a word in Gforth (1)

Actually, it's inconvenient. A better approach is to use the latest word
defined in the compilation word list (2). So you are able to create an
anonymous definition and use it for the word defined right before that.


Also, I considered the following interface:

: foo ." interpreting" ;

compiling: foo ." compiling" ;

"compiling:" finds "foo" in the compilation word list, and
changes/redefines it to ensure that the given behavior will takes place
when the system encounters "foo" in compilation state.

> Is there a way to place the xt of the last
> definition on the stack, so that SET-COMPSEM can be used in the
> definition of VALUE?

A closure (namely, a partial application [2]) could be used for that,
but the limitation (1) makes it more difficult, since a generated
anonymous definition becomes last definition. A possible solution is to
employ predefined execution tokens. But (2) is better.



[1]
https://github.com/forthy42/gforth/blob/0.7.9_20220929/set-compsem.fs#L25
[2] https://en.wikipedia.org/wiki/Partial_application

--
Ruvim

[Krishna Myneni]

Yes, this is what I want.

> In the following, let's assume you have a word defined with
>
> 0 broken-value n2
>
> BROKEN-VALUE and the words created by it have the following
> disadvantages compared to the VALUE that uses SET-OPTIMIZER:
>
> * [COMPILE] N2 has the same effect as POSTPONE N2, not the same effect
> as just compiling N2, which a standard value would have. The reason
> is that [COMPILE] thinks it has non-default compilation semantics;
> if you want to stick with default compilation semantics, don't use
> SET-COMPSEM! This is the reason why I called this word
> BROKEN-VALUE.
>

[COMPILE] is obsolete and need not be discussed. It used to exist in
kForth and has been removed.

> * COMPILE,ing the xt of N2 now does not optimize. The result is
> correct, but not as fast as what you get with the SET-OPTIMIZER
> variant.
>

This is a more fundamental issue because "COMPILE," is specified to work
with an xt. On a system which supports name tokens, "COMPILE," can still
work with the xt (performing LITERAL and compiling EXECUTE) but it
points to the absence of a compilation word which takes an nt as an
argument and performs LITERAL and compiles NAME>COMPILE and EXECUTE .
Then it will be optimized.

In short, I think using SET-COMPSEMP is a more transparent way of
achieving optimization and your BROKEN-VALUE definition would be my
preferred way to go about it. On a dual token system (one which provides
a reference to interpretation semantics and a separate reference to
compilation semantics) I want to avoid thinking about whether the
compilation semantics are default or non-default. There is only
"compilation semantics" for a word, at any given instance in time. The
system provides compilation semantics for the word upon definition but
it may be changed later, if desired. Similarly, for words which CREATE
other words, there should be a mechanism for setting the compilation
semantics of the created words immediately after creation, if desired.

--
Krishna

[Anders Koburg]

True.

An admttedly not very important and very personal side observation:
I find all those differing syntaxes confusing and more often than not plain ugly.

In my dual-xt system use just these words
COMPILES> (aligned "esthetically" to DOES>)
COMPILES (aligned "esthetically" to IS)
IMMEDIATE
COMPILE-ONLY
to control the compilation.

Usage:

: DUALWORD <compilation semantic> COMPILES> <execution sematic> ;
to define a dual-xt word in one go

:NONAME <compilation semantic> ; COMPILES OTHERWORD
(or similarly through ticking) to add/overwrite OTHERWORD's
compilation-xt in its header

IMMEDIATE
to make last word's compilation-xt and execution-xt identical in its header

COMPILE-ONLY
to move execution-xt to compilation-xt in last word's header for normal words
(resp. set execution-xt to zero when last word is already immediate).

Of course the text interpreter is adapted to check the presence of both xt's
in the header of found names. Up to now this simple scheme has served me
well, but perhaps only because I don't play funny things with the Forth compiler
like others seem to do.

[Ruvim]

It looks like a Gforth's closure is a definition (and an associated xt)
which is created in run-time, but which does not become the last
definition. It's a confusing inconsistency.

>
> Yes, this is what I want.
>
>> In the following, let's assume you have a word defined with
>>
>> 0 broken-value n2
>>
>> BROKEN-VALUE and the words created by it have the following
>> disadvantages compared to the VALUE that uses SET-OPTIMIZER:
>>
>> * [COMPILE] N2 has the same effect as POSTPONE N2, not the same effect
>>    as just compiling N2, which a standard value would have.  The reason
>>    is that [COMPILE] thinks it has non-default compilation semantics;
>>    if you want to stick with default compilation semantics, don't use
>>    SET-COMPSEM!  This is the reason why I called this word
>>    BROKEN-VALUE.
>>
>
> [COMPILE] is obsolete and need not be discussed. It used to exist in
> kForth and has been removed.
>


>> * COMPILE,ing the xt of N2 now does not optimize.  The result is
>>    correct, but not as fast as what you get with the SET-OPTIMIZER
>>    variant.
>>
>
> This is a more fundamental issue because "COMPILE," is specified to work
> with an xt. On a system which supports name tokens, "COMPILE," can still
> work with the xt (performing LITERAL and compiling EXECUTE) but it
> points to the absence of a compilation word which takes an nt as an
> argument and performs LITERAL and compiles NAME>COMPILE and EXECUTE .
> Then it will be optimized.

Taking into account that "compile," can be implemented as:

: compile, ( xt -- ) postpone literal postpone execute ;


It seems you assume that the following phrases are equivalent for an
ordinary word (for which xt and nt are defined):

postpone literal postpone execute
\ run-time: ( xt -- )

postpone literal postpone name>compile postpone execute
\ run-time: ( nt -- )

But they are not equivalent.
— The first phrase in run-time *performs* the compilation semantics for
the word (and later "execute" executes the word) .
— The second phrase in run-time *appends* the compilation semantics for
the word to the current definition (and later "execute" executes the
compilation semantics for the word).

> In short, I think using SET-COMPSEMP is a more transparent way of
> achieving optimization and your BROKEN-VALUE definition would be my
> preferred way to go about it.

Probably you actually thought about something like "compile-nt"

: compile-nt ( i*x nt -- j*x ) name>compile execute ;


But this word cannot replace "compile," due to the following reasons:

1. There is a common standard-compliant pattern to compile an ordinary word:
['] foo compile,
And this pattern should remain valid.

2. Anonymous definitions don't have associated nt (name token), and the
only way to compile them is to use "compile,".

BTW, in my system I associate a custom compilation procedure not with
nt, but with xt. So, an optimized compiler can be specified for an
anonymous definition too.

Also, it's less confusing, since the notion of compilation semantics is
not applicable to execution tokens. And when you set a compiler for xt,
you cannot think like you define compilation semantics by that for some
word at all.

> On a dual token system (one which provides
> a reference to interpretation semantics and a separate reference to
> compilation semantics) I want to avoid thinking about whether the
> compilation semantics are default or non-default. There is only
> "compilation semantics" for a word, at any given instance in time. The
> system provides compilation semantics for the word upon definition but
> it may be changed later, if desired. Similarly, for words which CREATE
> other words, there should be a mechanism for setting the compilation
> semantics of the created words immediately after creation, if desired.

In such a case you have to eliminate execution tokens, "execute" and
"compile," from your system (i.e., make them inaccessible for a user).


But the notion of anonymous definition and execution semantics are
essential in Forth. It's a rock other conceptions are based on.


--
Ruvim

[none albert]

In article <theadt$24khr$1...@dont-email.me>,
Ruvim <ruvim...@gmail.com> wrote:
<SNIP>

>
>
>But the notion of anonymous definition and execution semantics are
>essential in Forth. It's a rock other conceptions are based on.

No it isn't. Anything accomplished by an anonymous definition can be
accomplished by a definition that has a name and can be properly
found in the dictionary.

It is an other matter if we redesign Forth and go the other way
around. I can define :NONAME as
:NONAME ": NONAME" EVALUATE ;
but that is cheating. I do that in ciforth because I see no
value in a :NONAME that saves a couple of bytes from my Gbytes.

"It's a rock other conceptions are based on."

If only that were true.
A redesigned forth from the ground up looks like lucky7.
In lucky7 the fundamental construction is
{ ... } that leaves a behaviour token ("xt" not a "nt")
than can be run ("execute").
It is a compile time constant that automatically is LITERALized as
were it a number, so usable in deferred/compile mode.
lucky7 add new definitions with ``:''
{ "This is lucky" TYPE CR } : lucky
`` ; '' is freed to designate comment.
So the behaviour is given a name by `` : ''
`` ; '' is freed to designate comment.
There are other defining words, of course.

The glossary of lucky7 fits on a stamp:

; { } [ ] run
|{ }|{ }| {| | |} do ix
poke peek bpoke bpeek
+ - * / % negate or and xor invert << >>
< > = <> >= <= false true not
drop dup over nip swap pdup pdrop spswap
>r r> r
. x. $.
here , reserve
: data variable meta

include want from: import
lsn clr shw

Literals are introduced by a digit or a special character
sequence ( 123 0x123 "ape" ) .
`` meta '' replaces CREATE/DOES>.

>--
>Ruvim

Groetjes Albert
--
"in our communism country Viet Nam, people are forced to be
alive and in the western country like US, people are free to
die from Covid 19 lol" duc ha
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[Ruvim]

On 2022-10-03 13:35, Ruvim wrote:
> On 2022-10-03 01:08, Krishna Myneni wrote:
>> On 10/2/22 11:14, Anton Ertl wrote:
>>> Krishna Myneni <krishna...@ccreweb.org> writes:
>>>> On 10/2/22 00:00, Anton Ertl wrote:
>>>>> Krishna Myneni <krishna...@ccreweb.org> writes:
>>>>>> On 10/1/22 02:06, Anton Ertl wrote:
>>>>>>> SET-OPTIMIZER sets the implementation of COMPILE, ( xt -- ) for
>>>>>>> the current word.  A correct implementation of COMPILE, does not
>>>>>>> change the semantics in any way, only the implementation of the
>>>>>>> semantics.

(¹)

>>>>>>> So if you instead do
>>>>>>>
>>>>>>> [: postpone literal postpone execute ;] set-optimizer
>>>>>>>
>>>>>>> the behaviour stays the same.
>>>>>>>
>>>>>>
>>>>>> I not seeing the difference.
>>>>>> I can put any arbitrary sequence inside the
>>>>>> quotation for SET-OPTIMIZER. Then the compilation
>>>>>> semantics (behavior) can be different from that
>>>>>> of a word defined without SET-OPTIMIZER.

(²)

>>>>>
>>>>> Yes, you can shoot yourself in the foot with SET-OPTIMIZER.  This is
>>>>> Forth.  But once you have done that, COMPILE, does no longer work as
>>>>> specified (and, more importantly, as used in most places where
>>>>> COMPILE, is used).

[...]

>>> * COMPILE,ing the xt of N2 now does not optimize.  The result is
>>>    correct, but not as fast as what you get with the SET-OPTIMIZER
>>>    variant.
>>>

[...]

>> In short, I think using SET-COMPSEMP is a more transparent way of
>> achieving optimization and your BROKEN-VALUE definition would be my
>> preferred way to go about it.
>
>
> Probably you actually thought about something like "compile-nt"
>
>   : compile-nt ( i*x nt -- j*x ) name>compile execute ;
>
>
> But this word cannot replace "compile," due to the following reasons:
>
> 1. There is a common standard-compliant pattern to compile an ordinary
> word:
>     ['] foo compile,
> And this pattern should remain valid.
>
> 2. Anonymous definitions don't have associated nt (name token), and the
> only way to compile them is to use "compile,".
>
> BTW, in my system I associate a custom compilation procedure not with
> nt, but with xt. So, an optimized compiler can be specified for an
> anonymous definition too.
>
> Also, it's less confusing, since the notion of compilation semantics is
> not applicable to execution tokens. And when you set a compiler for xt,
> you cannot think like you define compilation semantics by that for some
> word at all.

A compilation procedure for an xt shall be observably equivalent to
apply "compile," to this xt, which in turn shall be observably
equivalent to apply the definition (quotation)
[: postpone literal postpone execute ;]
to this xt.

A definition is not a compilation procedure for an xt, if the above are
not held.

Thus, the behavior of a compilation procedure is well defined and
determined by the execution semantics identified by the xt for which
this procedure is intended — i.e., execution of code generated by this
procedure shall be equivalent to execution of this xt. You have no room
for *observable* variations. Therefore, it's impossible to *change*
compilation semantics of some word by assign a custom compilation
procedure to some xt.


I think, in these terms the statements (¹) and (²) marked above should
be more clear.






[...]

>> On a dual token system (one which provides a reference to
>> interpretation semantics and a separate reference to compilation
>> semantics) I want to avoid thinking about whether the compilation
>> semantics are default or non-default.

The notion of compilation semantics is only applicable to Forth words
(named Forth definitions).

As a user, you cannot avoid the notion of /default/ compilation
semantics, since it's compilation semantics assigned by the system for a
new ordinary word (e.g., for an ordinary colon definition).


If you provide the system with a definition that should be used to
perform the compilation semantics for some word, you still have to think
whether you change the previously defined compilation semantics for this
word or not.

If the semantics are changed, the new semantics should be reflected in
the documentation and tests for the word.

If the semantics are not changed, the documentation and tests for the
word may remain unchanged. But then, if this word is an ordinary word,
the compilation semantics for this word can still be performed via
"compile," without involving your definition to perform the compilation
semantics for this word. You should not ignore this fact.

If you think that "compile," can just unconditionally use your
definition to perform the compilation semantics for the word — it's
wrong, since your definition may consume and leave additional stack
items, but "compile," — cannot.

[Ruvim]

On 2022-10-03 16:59, albert wrote:
> In article <theadt$24khr$1...@dont-email.me>,
> Ruvim <ruvim...@gmail.com> wrote:
> <SNIP>
>>
>>
>> But the notion of anonymous definition and execution semantics are
>> essential in Forth. It's a rock other conceptions are based on.
>
> No it isn't. Anything accomplished by an anonymous definition can be
> accomplished by a definition that has a name and can be properly
> found in the dictionary.

I talk about notions. I don't mean Forth systems or implementations.

"Forth definition" is a more basic notion than "named Forth definition".



[...]

> "It's a rock other conceptions are based on."
> If only that were true.
> A redesigned forth from the ground up looks like lucky7.
> In lucky7 the fundamental construction is
> { ... } that leaves a behaviour token ("xt" not a "nt")
> than can be run ("execute").

In Forth it is [: ... ;]


I think, a method to create a named Forth definition from an execution
token should be standardized.


--
Ruvim

[Gerry Jackson]

On 01/10/2022 09:45, Anton Ertl wrote:
> Krishna Myneni<krishna...@ccreweb.org> writes:
>> On 9/30/22 04:30, albert wrote:
>>> There was a discussion about the original ANSI 93 rule
>>> that "TO must parse".
> There is no such rule. Forth-94 and 2012 describe TO as parsing
> words, allowing the implementation of a parsing TO. But, as you
> mention, there is no standard program that could determine if TO does
> not parse, so a standard system can also have a non-parsing TO.
>

But a non-standard program may detect a non-parsing TO e.g.
where TO sets a flag that a following value tests to return or change
its value.

\ --------------------
VFX Forth 64 for Windows x64
© MicroProcessor Engineering Ltd, 1998-2022

Version: 5.20 [build 4077]
Build date: 1 August 2022

Free dictionary = 6966470 bytes [6803kb]


0 value x x . 0 ok
123 to to x x . 123 ok
234 to to to to x x . 234 ok
345 to ok-1
x x . 345 ok
\ ----------------------------

--
Gerry

[Gerry Jackson]

On 29/09/2022 11:20, Krishna Myneni wrote:
> On 9/29/22 02:24, minf...@arcor.de wrote:
>> Krishna Myneni schrieb am Donnerstag, 29. September 2022 um 02:43:10
>> UTC+2:
>>> On 9/28/22 19:39, Krishna Myneni wrote:
>>>> ------
>>>> : NONDEFERRED ;
>>>> SYNONYM
>>>
>>> The above line should have been
>>>
>>> SYNONYM A@ @
>>>
>>> --
>>
>> Thanks for posting this!
>>
>> It is refreshing to read some substantial contribution again
>> here in c.l.f., instead of all those silly rantings of the last months.
>>
>> I was only wondering if your idea could be extended to +TO.
>
> Yes, it can; however, if you need to use +TO my advice would be to not
> use a VALUE (or FVALUE or other extended VALUE) but go back to
> VARIABLEs. One can use of +! , F+! (not currently standard, but
> necessary), etc. with corresponding variables. A drawback of TO and
> extensions such as +TO is that various Forth operations such as '
> (tick), ['], POSTPONE, etc. on them results in an ambiguous condition in
> standard Forth.
>

If I define +TO as (using standard Forth):

: +to ( n "name" -- )
parse-name 2>r ( -- n ) ( R: -- ca u )
2r@ s" + to " 2r@
<# holds holds holds 2r> #> evaluate
; immediate

which given
+to x
on a value x evaluates
x + to x

' ['] and POSTPONE can be applied to +TO. It may need special
compilation semantics for some cases but that's beside the point.

--
Gerry

[Krishna Myneni]

Ruvim,

I'll have to address your statements out of order.

1.

>> On a dual token system (one which provides a reference to
>> interpretation semantics and a separate reference to compilation
>> semantics) I want to avoid thinking about whether the compilation
>> semantics are default or non-default. There is only "compilation
>> semantics" for a word, at any given instance in time. The system
>> provides compilation semantics for the word upon definition but it may
>> be changed later, if desired. Similarly, for words which CREATE other
>> words, there should be a mechanism for setting the compilation
>> semantics of the created words immediately after creation, if desired.
>
>
> In such a case you have to eliminate execution tokens, "execute" and
> "compile," from your system (i.e., make them inaccessible for a user).
>

No, a dual token system by definition contains execution tokens, one for
interpretation and one for compilation. There is no suggestion that
execution tokens should be eliminated. It would be silly to eliminated
EXECUTE therefore. I also made no suggestion for eliminating COMPILE,
and explicitly stated that COMPILE, is necessary when it is desired to
compile the execution behavior returned by ordinary '(tick).

Also see my reply to albert earlier in this thread, in which I stated
that xt's cannot be eliminated in a standard Forth system.

2.

> Probably you actually thought about something like "compile-nt"
>
> : compile-nt ( i*x nt -- j*x ) name>compile execute ;
>
>
> But this word cannot replace "compile," due to the following reasons:
>
> 1. There is a common standard-compliant pattern to compile an ordinary
> word:
> ['] foo compile,
> And this pattern should remain valid.
>
> 2. Anonymous definitions don't have associated nt (name token), and the
> only way to compile them is to use "compile,".
>
> BTW, in my system I associate a custom compilation procedure not with
> nt, but with xt. So, an optimized compiler can be specified for an
> anonymous definition too.
>
> Also, it's less confusing, since the notion of compilation semantics is
> not applicable to execution tokens. And when you set a compiler for xt,
> you cannot think like you define compilation semantics by that for some
> word at all.
>

I actually have such a word in kForth (which is not presently a
dual-token system) called "COMPILE-NAME", which performs the equivalent
of NAME>COMPILE EXECUTE . One can still use expressions such as "['] FOO
COMPILE," or " ... [ ' FOO ] LITERAL COMPILE,".

On a dual-token system, COMPILE-NAME (or whatever name one wants to give
it) would be guaranteed to compile the xt referencing the extra token
referencing the compilation semantics of a word. The sequence "[']
<name>" would still perform its usual action of returning the
interpretation xt for the word, so there are no changes in this regard.
'(tick) and ['] return what are expected in standard Forth.

3.

> It seems you assume that the following phrases are equivalent for an
> ordinary word (for which xt and nt are defined):
>
> postpone literal postpone execute
> \ run-time: ( xt -- )
>
> postpone literal postpone name>compile postpone execute
> \ run-time: ( nt -- )
>
> But they are not equivalent.
> — The first phrase in run-time *performs* the compilation semantics for
> the word (and later "execute" executes the word) .
> — The second phrase in run-time *appends* the compilation semantics for
> the word to the current definition (and later "execute" executes the
> compilation semantics for the word).
>
>

No, I don't assume that. NAME>COMPILE should always return the
compilation xt, which can be independently set on a dual token system,
or on a single token system which supports "non-default compilation
semantics". In contrast, ticking a word on a standard system will return
an execution token corresponding to its interpretation semantics.

--
Krishna

[Krishna Myneni]

On 10/3/22 04:07, Anders Koburg wrote:
> Krishna Myneni schrieb am Sonntag, 2. Oktober 2022 um 23:08:43 UTC+2:

..

I like your names. They are more aesthetic. I suggest you use
"interpretation-xt" in place of "execution-xt". Both xt's are execution
tokens.

The examples given where state-smart words can fail are a bit contrived
for simplicity; however, when you try to do non-trivial things, you want
to be able to analyze the behavior of the code, and always knowing the
compilation and interpretation behavior of words will help to predict
what the code will do. Most routine coding is trivial to understand.

--
Krishna

[Ruvim]

On 2022-10-04 01:52, Krishna Myneni wrote:
> On 10/3/22 04:35, Ruvim wrote:
>> On 2022-10-03 01:08, Krishna Myneni wrote:
>
> 1.
>>> On a dual token system (one which provides a reference to
>>> interpretation semantics and a separate reference to compilation
>>> semantics) I want to avoid thinking about whether the compilation
>>> semantics are default or non-default. There is only "compilation
>>> semantics" for a word, at any given instance in time. The system
>>> provides compilation semantics for the word upon definition but it may
>>> be changed later, if desired. Similarly, for words which CREATE other
>>> words, there should be a mechanism for setting the compilation
>>> semantics of the created words immediately after creation, if desired.
>>
>>
>> In such a case you have to eliminate execution tokens, "execute" and
>> "compile," from your system  (i.e., make them inaccessible for a user).
>>
>
> No, a dual token system by definition contains execution tokens, one for
> interpretation and one for compilation. There is no suggestion that
> execution tokens should be eliminated.

I just wanted to say that it's impossible to avoid the notion of default
compilation semantics without avoiding ordinary words. I also mentioned
this idea my another message yesterday:
https://groups.google.com/g/comp.lang.forth/c/dh347IHLDtw/m/_C_XjZ60AwAJ

> 2.
>
>> Probably you actually thought about something like "compile-nt"
>>
>>    : compile-nt ( i*x nt -- j*x ) name>compile execute ;
>>
>>
>> But this word cannot replace "compile," due to the following reasons:
>>
>> 1. There is a common standard-compliant pattern to compile an ordinary
>> word:
>>      ['] foo compile,
>> And this pattern should remain valid.
>>
>> 2. Anonymous definitions don't have associated nt (name token), and the
>> only way to compile them is to use "compile,".

> I actually have such a word in kForth (which is not presently a
> dual-token system) called "COMPILE-NAME", which performs the equivalent
> of NAME>COMPILE EXECUTE . One can still use expressions such as "['] FOO
> COMPILE," or " ... [ ' FOO ] LITERAL COMPILE,".
>
> On a dual-token system, COMPILE-NAME (or whatever name one wants to give
> it) would be guaranteed to compile the xt referencing the extra token
> referencing the compilation semantics of a word.

Should it compile the xt, or perform the compilation semantics for the
word? Sometimes these actions are different.

> The sequence "['] <name>"
> would still perform its usual action of returning the
> interpretation xt for the word, so there are no changes in this regard.
> '(tick) and ['] return what are expected in standard Forth.

[...]

> NAME>COMPILE should always return the
> compilation xt, which can be independently set on a dual token system,
> or on a single token system which supports "non-default compilation
> semantics".


> In contrast, ticking a word on a standard system will return
> an execution token corresponding to its interpretation semantics.

It sounds incorrectly. By this wording you revoke access to the
identifier of the execution semantics for a word.

Actually, ticking a word on a standard system returns the execution
token that identifies the execution semantics for *this* word.

If the standard does not specify execution semantics for the word then
the returned xt (if any) identifies implementation-dependent execution
semantics for this word.

Yes, usually, this execution token is used by the system to *perform*
the interpretation semantics for the word.

But, for some words, this the same execution token may be also used to
perform the compilation semantics for the word. Namely, if the word is
an immediate word.

Since, to perform the compilation semantics for an immediate word, its
execution token shall be executed in compilation state, to perform the
interpretation semantics — it shall be executed in interpretation state.

When you refer this execution token as one corresponding to the
interpretation semantics of a word, you also make an impression that
executing this token performs the interpretation semantics for the word
regardless of STATE. But it's false in the general case.



--
Ruvim

[Ruvim]

Having an opaque implementation for "+to", Tick should not be applied to
"+to" in a standard program while execution semantics are not specified
for "+to" in the documentation.


"postpone" may be applied to "+to" in any case. But the behavior of the
containing definition in interpretation state is ambiguous if the system
provides not fully compliant "postpone".

A test case:

: compiling(+to) postpone +to ;
0 value x
: test-x 10 [ compiling(+to) x ] x ;
x . test-x . \ should print "0 10"

If this test fails, the system /effectively/ imposes an environmental
restriction: a program should not start performing compilation semantics
in interpretation state.


--
Ruvim

[Ruvim]

Good point! Since "execution-xt" means "execution-execution-token".


These execution tokens should be thought as the execution tokens of
supportive definitions that are used to *perform* the interpretation
semantics and compilation semantics for a word correspondingly.

In the case of an immediate word (or STATE-dependent execution
semantics) these execution tokens do the intended things in the
corresponding state only.

So, in the general case (when you don't know whether a word is immediate
or not, or whether their execution semantics are STATE-dependent) they
should be also executed in the corresponding state only.

> The examples given where state-smart words can fail are a bit contrived
> for simplicity; however, when you try to do non-trivial things, you want
> to be able to analyze the behavior of the code, and always knowing the
> compilation and interpretation behavior of words will help to predict
> what the code will do. Most routine coding is trivial to understand.
>
> --
> Krishna

--
Ruvim

[Krishna Myneni]

On 10/4/22 16:36, Ruvim wrote:
> On 2022-10-04 01:52, Krishna Myneni wrote:

..

>> No, a dual token system by definition contains execution tokens, one
>> for interpretation and one for compilation. There is no suggestion
>> that execution tokens should be eliminated.
>
> I just wanted to say that it's impossible to avoid the notion of default
> compilation semantics without avoiding ordinary words. I also mentioned
> this idea my another message yesterday:
> https://groups.google.com/g/comp.lang.forth/c/dh347IHLDtw/m/_C_XjZ60AwAJ
>
>

You may want to call it default compilation semantics, but on a dual
token system I would simply think of it as initial compilation semantics
assigned by the system when the word has been defined, i.e. when the
colon definition specifies the interpretation semantics.

Desired compilation semantics can be bound to the word at any later
time, including by IMMEDIATE which copies the interpretation semantics
to the compilation semantics.

But this is simply a fine point about terminology. On any Forth system,
dual-token or not, it is possible to make a definition with no semantics
at all (no behavior in interpretation state or in compilation state), e.g.

: MY-WORD ;

In a dual-token system, it should be possible to subsequently set either
or both interpretation and compilation semantics afterwards, any time
the word can be found in the search order using FIND-NAME.

>
>
>
>> 2.
>>
>>> Probably you actually thought about something like "compile-nt"
>>>
>>>    : compile-nt ( i*x nt -- j*x ) name>compile execute ;
>>>
>>>
>>> But this word cannot replace "compile," due to the following reasons:
>>>
>>> 1. There is a common standard-compliant pattern to compile an ordinary
>>> word:
>>>      ['] foo compile,
>>> And this pattern should remain valid.
>>>
>>> 2. Anonymous definitions don't have associated nt (name token), and the
>>> only way to compile them is to use "compile,".
>
>
>> I actually have such a word in kForth (which is not presently a
>> dual-token system) called "COMPILE-NAME", which performs the
>> equivalent of NAME>COMPILE EXECUTE . One can still use expressions
>> such as "['] FOO COMPILE," or " ... [ ' FOO ] LITERAL COMPILE,".
>>
>> On a dual-token system, COMPILE-NAME (or whatever name one wants to
>> give it) would be guaranteed to compile the xt referencing the extra
>> token referencing the compilation semantics of a word.
>
> Should it compile the xt, or perform the compilation semantics for the
> word?  Sometimes these actions are different.
>

The compilation semantics of the word corresponding to nt is performed
when COMPILE-NAME executes.

>
>
>> The sequence "['] <name>" would still perform its usual action of
>> returning the interpretation xt for the word, so there are no changes
>> in this regard. '(tick) and ['] return what are expected in standard
>> Forth.
>
>
>
> [...]
>
>> NAME>COMPILE should always return the compilation xt, which can be
>> independently set on a dual token system, or on a single token system
>> which supports "non-default compilation semantics".
>
>
>> In contrast, ticking a word on a standard system will return an
>> execution token corresponding to its interpretation semantics.
>
> It sounds incorrectly. By this wording you revoke access to the
> identifier of the execution semantics for a word.
>

Not really. In a dual-token system there are explicitly two types of
execution semantics: "interpretation semantics" or "compilation
semantics." It is easier to understand which one we are talking about.
In contrast, for a traditional system, we have to determine whether the
single xt representing execution semantics for a word corresponds to
compilation semantics or to interpretation semantics by checking the
precedence flag for the word.

..

> Yes, usually, this execution token is used by the system to *perform*
> the interpretation semantics for the word.
>
> But, for some words, this the same execution token may be also used to
> perform the compilation semantics for the word. Namely, if the word is
> an immediate word.

> Since, to perform the compilation semantics for an immediate word, its
> execution token shall be executed in compilation state, to perform the
> interpretation semantics — it shall be executed in interpretation state.
>
> When you refer this execution token as one corresponding to the
> interpretation semantics of a word, you also make an impression that
> executing this token performs the interpretation semantics for the word
> regardless of STATE. But it's false in the general case.
>

Note that I said "usually" in referring to the xt on a traditional Forth
system as representing the interpretation semantics. This is because
there are far fewer immediate words in a typical Forth program or in the
standard set of words than those which are not.

Your points above about execution token not always representing the
interpretation semantics are valid for a traditional Forth system, but
when you tick a word name, how do you know which semantics are being
returned? In contrast, with the sequence "nt NAME>INTERPRET" or "nt
NAME>COMPILE", you know precisely to which semantics the result corresponds.

The notion of a word having two execution tokens, xt-interp and xt-comp,
is far more flexible way of achieving precise behavior of words in a
Forth system than the single execution token method. It is also simpler
to understand (unless you have been exposed to the single xt system with
its precedence flag; then you have to unlearn this convoluted way of
doing things). One can use the same word in both interpretation and
compilation state without side effects associated with state-smartness.
A particular example of where this is important is array indexing, e.g.

A{ I }

with the indexing word "}" -- one needs "}" to work in both the
interpreter and from within a definition, but the compilation semantics
usually need to be different from the interpretation semantics to obtain
the best possible efficiency.

--
Krishna

[none albert]

In article <thj4hv$2tjb2$4...@dont-email.me>,

Krishna Myneni <krishna...@ccreweb.org> wrote:
>On 10/4/22 16:36, Ruvim wrote:
>> On 2022-10-04 01:52, Krishna Myneni wrote:
>..
>>> No, a dual token system by definition contains execution tokens, one
>>> for interpretation and one for compilation. There is no suggestion
>>> that execution tokens should be eliminated.
>>
>> I just wanted to say that it's impossible to avoid the notion of default
>> compilation semantics without avoiding ordinary words. I also mentioned
>> this idea my another message yesterday:
>> https://groups.google.com/g/comp.lang.forth/c/dh347IHLDtw/m/_C_XjZ60AwAJ
>>
>>
>
>You may want to call it default compilation semantics, but on a dual
>token system I would simply think of it as initial compilation semantics
>assigned by the system when the word has been defined, i.e. when the
>colon definition specifies the interpretation semantics.

Regarding those two token system. Chuck Moore experimented with that,
and has abandoned in favor of a multi color system.
In more mundane terms every object has multiple methods identified
by color. Two is not enough.

I want to go back to recipees. Recipees have no name and can be coupled
to a name by a dictionary. (Sort of noname definitions.)
A recipee can (but need not be) be coupled to one name, but subject to
manipulation and search order. Recipees can be immediate which affects
deferred mode, but they behave the same in either mode.
No smart words, only smart literals like numbers.
Note that a denotation for a recipee is also a literal, ("quotation").

>--
>Krishna

[minf...@arcor.de]

Good points!
But Chuck's multi-color system didn't go far either ... so ...

The question remains where you get the payoff for increasing complexity in
the compiler (by adding "multiple methods") only to cover some obscure
corner cases that nobody uses in reality (I don't count introspection into
one's own Forth system here)

Simplicity is a virtue by itself!

[Gerry Jackson]

Sorry, I thought the semantics of +TO was well understood so
documentation was unnecessary in a c.l.f discussion

>
>
> "postpone" may be applied to "+to" in any case. But the behavior of the
> containing definition in interpretation state is ambiguous if the system
> provides not fully compliant "postpone".
>
> A test case:
>
>   : compiling(+to) postpone +to ;
>   0 value x
>   : test-x 10 [ compiling(+to) x ] x ;
>   x . test-x . \ should print "0 10"
>
> If this test fails, the system /effectively/ imposes an environmental
> restriction: a program should not start performing compilation semantics
> in interpretation state.
>
>
> --
> Ruvim

Yes my system and others failed test-x. To make this work something like
a compile only word [+TO] needs to be defined, such as

: [+to] state @ >r ] postpone +to r> state ! ; immediate

I realise that STATE ! makes it non-standard but surely if its previous
value is restored the standard is being unnecessarily restrictive.
Anyway it can be replaced by ... 0= IF [ THEN ...
Presumably [+TO] isn't state smart but state neutral

Your test:
: compiling[+to] postpone [+to] ;
0 value y
: test-y 20 [ compiling[+to] y ] y ;
y . test-y . \ displays 0 20

: a [+to] y y ;
y . 5 a . \ displays 20 25, works in a colon definition

Similar code works for postponing S"

--
Gerry

[Ruvim]

What are well understood are the interpretation semantics and
compilation semantics for "+to", but not execution semantics.

If the phrase:
' +to
returns an xt, this xt identifies some execution semantics. And these
execution semantics may vary, if you don't specify it in a
documentation/standard/etc. (NB: I consider a case of opaque
implementations).

>>
>> "postpone" may be applied to "+to" in any case. But the behavior of
>> the containing definition in interpretation state is ambiguous if the
>> system provides not fully compliant "postpone".
>>
>> A test case:
>>
>>    : compiling(+to) postpone +to ;
>>    0 value x
>>    : test-x 10 [ compiling(+to) x ] x ;
>>    x . test-x . \ should print "0 10"
>>
>> If this test fails, the system /effectively/ imposes an environmental
>> restriction: a program should not start performing compilation
>> semantics in interpretation state.
>>
>

> Yes my system and others failed test-x. To make this work something like
> a compile only word [+TO] needs to be defined

To make this work something like a fully compliant "postpone" should be
defined.

See: https://github.com/ForthHub/discussion/discussions/105



--
Ruvim

[minf...@arcor.de]

As long as the Forth-2012 Test Suite is passed successfully,
a system's POSTPONE implementation is good enough.

If not, a reference implementation should be added here:
https://forth-standard.org/standard/implement

[Ruvim]

The Test Suit doesn't cover all the cases.

I created a new test case. At the moment, no one pointed out a mistake
in my reasoning, or shown that this test is incorrect.

https://github.com/ForthHub/discussion/discussions/103


Anyway, it's OK to stick with the mentioned environmental restriction,
if it's good enough for you.

>
> If not, a reference implementation should be added here:
> https://forth-standard.org/standard/implement
>

Reference implementations are not normative and optional:

https://forth-standard.org/proposals/reference-implementations-are-not-normative#reply-772


--
Ruvim

[minf...@arcor.de]

You're right. I any case, hair splitting has become too precious with my
few remaining hairs. :o)

[Krishna Myneni]

On 10/2/22 11:14, Anton Ertl wrote:

..

>
> : broken-value ( n "name" -- )
> [ s" !" find-name ] literal 1 cells xVALUE
> [: does> cell+ @ ;] execute
> lastxt >body cell+ [n:d postpone literal postpone @ ;] set-compsem ;
>
> In the following, let's assume you have a word defined with
>
> 0 broken-value n2
>
> BROKEN-VALUE and the words created by it have the following
> disadvantages compared to the VALUE that uses SET-OPTIMIZER:
>

..

>
> * COMPILE,ing the xt of N2 now does not optimize. The result is
> correct, but not as fast as what you get with the SET-OPTIMIZER
> variant.
>

I see that Gforth provides a word to convert an xt to an nt with the
word XT>NAME. Therefore, you should be able to obtain proper compilation
of N2 with "COMPILE," simply with

: compile-name ( nt -- ) name>compile execute ;
: compile, ( xt -- ) xt>name compile-name ;

This should make SET-OPTIMIZER unnecessary since the compilation
semantics set with SET-COMPSEM will then be used by the new "COMPILE,".
The redefinition of "COMPILE," will still work on a single-xt system.

--
Krishna

[Krishna Myneni]

On 10/6/22 08:39, Krishna Myneni wrote:
> On 10/2/22 11:14, Anton Ertl wrote:
> ..
>>
>> : broken-value ( n "name" -- )
>>       [ s" !" find-name ] literal 1 cells xVALUE
>>       [: does> cell+ @ ;] execute
>>       lastxt >body cell+ [n:d postpone literal postpone @ ;]
>> set-compsem ;
>>

..
>>
>> * COMPILE,ing the xt of N2 now does not optimize.  The result is
>>    correct, but not as fast as what you get with the SET-OPTIMIZER
>>    variant.
>>
>
> I see that Gforth provides a word to convert an xt to an nt with the
> word XT>NAME. Therefore, you should be able to obtain proper compilation
> of N2 with "COMPILE," simply with
>
> : compile-name ( nt -- ) name>compile execute ;
> : compile, ( xt -- ) xt>name compile-name ;
>
> This should make SET-OPTIMIZER unnecessary since the compilation
> semantics set with SET-COMPSEM will then be used by the new "COMPILE,".
> The redefinition of "COMPILE," will still work on a single-xt system.
>

Spoke too soon. The above is not a good idea, since COMPILE, will no
longer simply compile the semantics of xt without translation. There is
still a need for the standard behavior of COMPILE, .

--
Krishna

[Krishna Myneni]

Also, SET-OPTIMIZER is not a good idea for the same reason. It bypasses
what is intended by "COMPILE," which is simply to compile the execution
semantics represented by xt . If SET-OPTIMIZER is used to alter the
compilation semantics of a word, and a programmer wants to compile the
interpretation semantics of a word using, say,

['] <name> compile,

the above will not compile the interpretation semantics (as the
programmer might assume).

The cleaner solution, in my opinion, would be to use nt-based
compilation when dealing with named words. Then the explicit semantics,
interpretation or compilation, can be specified for compilation.

--
Krishna


--
Krishna

[Ruvim]

On 2022-10-05 09:26, Krishna Myneni wrote:
> On 10/4/22 16:36, Ruvim wrote:
>> On 2022-10-04 01:52, Krishna Myneni wrote:

[...]

>>>> Probably you actually thought about something like "compile-nt"
>>>>
>>>>    : compile-nt ( i*x nt -- j*x ) name>compile execute ;
>>>

>>> I actually have such a word in kForth (which is not presently a
>>> dual-token system) called "COMPILE-NAME", which performs the
>>> equivalent of NAME>COMPILE EXECUTE . One can still use expressions
>>> such as "['] FOO COMPILE," or " ... [ ' FOO ] LITERAL COMPILE,".
>>>
>>> On a dual-token system, COMPILE-NAME (or whatever name one wants to
>>> give it) would be guaranteed to compile the xt referencing the extra
>>> token referencing the compilation semantics of a word.
>>
>> Should it compile the xt, or perform the compilation semantics for the
>> word?  Sometimes these actions are different.
>>
>
> The compilation semantics of the word corresponding to nt is performed
> when COMPILE-NAME executes.
>

Does it perform compilation semantics regardless of the STATE?

For example:

: translate-123 s" 123" evaluate ; immediate

: test-123
[ "translate-123" find-name compile-name ]
;
0 test-123 . . \ should print "0 123"


If no, your system forces me to specify my intention twice — via
"compile-name", and via "state". It's not acceptable for a good API, I
think.

>>> NAME>COMPILE should always return the compilation xt, which can be
>>> independently set on a dual token system, or on a single token system
>>> which supports "non-default compilation semantics".
>>>
>>> In contrast, ticking a word on a standard system will return an
>>> execution token corresponding to its interpretation semantics.

What does "corresponding" mean? How is this execution token connected
with the execution semantics of the word? How can the interpretation
semantics be performed? (again, should interpretation state be set or not).

>>
>> It sounds incorrectly. By this wording you revoke access to the
>> identifier of the execution semantics for a word.
>>
>
> Not really. In a dual-token system there are explicitly two types of
> execution semantics: "interpretation semantics" or "compilation
> semantics." It is easier to understand which one we are talking about.
> In contrast, for a traditional system, we have to determine whether the
> single xt representing execution semantics for a word corresponds to
> compilation semantics or to interpretation semantics by checking the
> precedence flag for the word.

The standard allows the single-xt approach in implementations as well as
dual-xt or dual-nt approach. And it specifies an abstract model over all
possible implementation variants. And it specifies (should specify) only
APIs that don't depend on a particular implementation or approach. And a
standard program is not bothered whether it runs on a single-xt system,
or on a dual-xt system.

The standard specifies the execution semantics term, and the Tick method
(among other) to obtain an execution token for a word that identifies
the execution semantics of the word. To perform the interpretation
semantics for the word, "execute" should be applied to this token in
interpretation state. It's true for any standard system, regardless
whether it's a single-xt system or dual-xt system.



Sure, we can classify execution semantics by some types, according to
their internal properties. For example, execution semantics can be
either STATE-dependent or STATE-independent.

But "interpretation semantics" and "compilation semantics" are not types
of execution semantics. These terms are already occupied and defined in
the standard. Don't clutter them with other meanings. Choose other names.




[...]

> Your points above about execution token not always representing the
> interpretation semantics are valid for a traditional Forth system, but
> when you tick a word name, how do you know which semantics are being
> returned?

Actually, I may tick only ordinary or immediate word.
Anyway, when I Tick a word, it returns an identifier of the execution
semantics for the word.

> In contrast, with the sequence "nt NAME>INTERPRET" or "nt
> NAME>COMPILE", you know precisely to which semantics the result
> corresponds.
>

It's questionable, but my questions are above.

[...]



--
Ruvim

[Krishna Myneni]

On 10/6/22 15:40, Ruvim wrote:
> On 2022-10-05 09:26, Krishna Myneni wrote:

>>>>>
>>>>>    : compile-nt ( i*x nt -- j*x ) name>compile execute ;
>>>>
>>>> I actually have such a word in kForth (which is not presently a
>>>> dual-token system) called "COMPILE-NAME", which performs the
>>>> equivalent of NAME>COMPILE EXECUTE . One can still use expressions
>>>> such as "['] FOO COMPILE," or " ... [ ' FOO ] LITERAL COMPILE,".
>>>>

> Does it perform compilation semantics regardless of the STATE?
>

As I have stated above, COMPILE-NAME may be represented exactly in
standard code as your COMPILE-NT.

>>>> NAME>COMPILE should always return the compilation xt, which can be
>>>> independently set on a dual token system, or on a single token
>>>> system which supports "non-default compilation semantics".
>>>> In contrast, ticking a word on a standard system will return an
>>>> execution token corresponding to its interpretation semantics.
>
> What does "corresponding" mean?  How is this execution token connected
> with the execution semantics of the word? How can the interpretation
> semantics be performed? (again, should interpretation state be set or not).
>

I was a bit sloppy in my description above. In a single-xt system like
kForth, NAME>COMPILE returns the compilation semantics as follows:

For nt corresponding to an IMMEDIATE word, the returned compilation
semantics are given by two xt's:

( nt-word -- xt-word xt-execute )

When EXECUTE is applied to the stack diagram above, xt-word is
immediately executed.

For nt corresponding to a non-immediate word, the returned compilation
semantics are given by an nt and an xt:

( nt-word -- nt xt-compile-name )

When EXECUTE is applied to the above stack diagram, the system compiles
the execution semantics for xt-word. Thus far no use has been made of STATE.


In contrast, in dual-xt systems, there is no need for the term
"execution semantics". We can more precisely talk about either
interpretation semantics (xt-interp) or compilation semantics (xt-comp).
Which semantics is relevant to the text interpreter -inter- when it
encounters the word, if STATE is zero, it will obtain the interpretation
semantics (xt-interp), else it will obtain the compilation semantics
(xt-comp). In a dual-xt system The text interpreter does not need to
examine a precedence flag for the word.

For both types of systems, single-xt with precedence and dual-xt, we can
write a simple one line text interpreter as follows.

: interp ( -- )
begin
parse-name find-name
?dup if state @ if name>compile else name>interpret then
then execute then
again ;

This interpreter won't recognize numbers, but can be demonstrated like this:

interp : test true abs dup + false do i . loop ; cr test cr abort

It should define a word and execute it, printing "0 1". Note that we are
using only standard words in the definition of the one-line interpreter.

Yes, both "interpretation semantics" and "compilation semantics" are
types of execution semantics, which should be obvious by the fact that
they can be represented by tokens on the stack which, when EXECUTEd will
perform the behavior required in the appropriate STATE, as illustrated
by the one-line text interpreter.

>
>
>
> [...]
>> Your points above about execution token not always representing the
>> interpretation semantics are valid for a traditional Forth system, but
>> when you tick a word name, how do you know which semantics are being
>> returned?
>
> Actually, I may tick only ordinary or immediate word.
> Anyway, when I Tick a word, it returns an identifier of the execution
> semantics for the word.
>

In a dual-xt system, which xt do you expect tick or ['] to return?
xt-interp or xt-comp? It should return xt-interp for backward
compatibility with single-xt systems.

>
>> In contrast, with the sequence "nt NAME>INTERPRET" or "nt
>> NAME>COMPILE", you know precisely to which semantics the result
>> corresponds.
>>
>
> It's questionable, but my questions are above.
>

Maybe my example helps to clarify.

--
Krishna

[Krishna Myneni]

On 10/6/22 22:01, Krishna Myneni wrote:
..

> I was a bit sloppy in my description above. In a single-xt system like
> kForth, NAME>COMPILE returns the compilation semantics as follows:
>
> For nt corresponding to an IMMEDIATE word, the returned compilation
> semantics are given by two xt's:
>
> ( nt-word -- xt-word xt-execute )
>
> When EXECUTE is applied to the stack diagram above, xt-word is
> immediately executed.
>
> For nt corresponding to a non-immediate word, the returned compilation
> semantics are given by an nt and an xt:
>
> ( nt-word -- nt xt-compile-name )
>

..

The stack diagram above should be

( nt-word -- nt-word xt-compile-name )

--
KM

[Ruvim]

On 2022-10-07 03:01, Krishna Myneni wrote:
> On 10/6/22 15:40, Ruvim wrote:
>> On 2022-10-05 09:26, Krishna Myneni wrote:
>
>>>>>>
>>>>>>    : compile-nt ( i*x nt -- j*x ) name>compile execute ;
>>>>>
>>>>> I actually have such a word in kForth (which is not presently a
>>>>> dual-token system) called "COMPILE-NAME", which performs the
>>>>> equivalent of NAME>COMPILE EXECUTE . One can still use expressions
>>>>> such as "['] FOO COMPILE," or " ... [ ' FOO ] LITERAL COMPILE,".
>>>>>
>
>> Does it perform compilation semantics regardless of the STATE?
>>
>
> As I have stated above, COMPILE-NAME may be represented exactly in
> standard code as your COMPILE-NT.

It does not answer to my question, since your "name>compile" can return
xt that always identifies STATE-independent execution semantics.

Therefore, I provided a test to check how it works:

: translate-123 s" 123" evaluate ; immediate

: test-123
[ "translate-123" find-name compile-name ]
;
0 test-123 . . \ should print "0 123"

I guess, this test fails in your system.


So, if a user wants to perform th compilation semantics for a word, what
is your reasoning to force the user to specify his intention *twice* in
the same time — via "compile-name" and via entering into compilation
state before call "compile-name"?

>>>>> NAME>COMPILE should always return the compilation xt, which can be
>>>>> independently set on a dual token system, or on a single token
>>>>> system which supports "non-default compilation semantics".
>>>>> In contrast, ticking a word on a standard system will return an
>>>>> execution token corresponding to its interpretation semantics.
>>
>> What does "corresponding" mean?  How is this execution token connected
>> with the execution semantics of the word? How can the interpretation
>> semantics be performed? (again, should interpretation state be set or
>> not).
>>
>
> I was a bit sloppy in my description above. In a single-xt system like
> kForth, NAME>COMPILE returns the compilation semantics as follows:
>
> For nt corresponding to an IMMEDIATE word, the returned compilation
> semantics are given by two xt's:
>
> ( nt-word -- xt-word xt-execute )
>
> When EXECUTE is applied to the stack diagram above, xt-word is
> immediately executed.

Then, to perform the compilation semantics for an immediate word, the
user should use "name>compile" and enter into compilation state — again,
he should specify his intention twice.

Also, the user should know whether the word is immediate or not.

But, having the nt of a word, there is no a standard way to detect
whether the corresponding word is immediate or not.


Actually, an API should not require user to know whether a word is
immediate or not. See A.6.1.2033 — why POSTPONE was introduced.

> For nt corresponding to a non-immediate word, the returned compilation
> semantics are given by an nt and an xt:
>
> ( nt-word -- nt xt-compile-name )
>
> When EXECUTE is applied to the above stack diagram, the system compiles
> the execution semantics for xt-word. Thus far no use has been made of
> STATE.

The compilation semantics for an ordinary word are to *append* its
execution semantics to the current definition.

The compilation semantics for an immediate word are to *performs* its
execution semantics in compilation state.

In the general case, there can be standard words, which the system
provides, that are neither immediate nor ordinary.

For example, the word S" in a standard system can be implemented in such
a way that if you tick this word, and executes the returned xt in
compilation state — it will not perform the compilation semantics for
this word. Hence, this word is nonimmediate in such a case.


[...]


--
Ruvim

[Ruvim]

On 2022-10-07 09:38, Ruvim wrote:
> On 2022-10-07 03:01, Krishna Myneni wrote:
>> On 10/6/22 15:40, Ruvim wrote:
>>> On 2022-10-05 09:26, Krishna Myneni wrote:
>>
>>>>>>>
>>>>>>>    : compile-nt ( i*x nt -- j*x ) name>compile execute ;
>>>>>>
>>>>>> I actually have such a word in kForth (which is not presently a
>>>>>> dual-token system) called "COMPILE-NAME", which performs the
>>>>>> equivalent of NAME>COMPILE EXECUTE . One can still use expressions
>>>>>> such as "['] FOO COMPILE," or " ... [ ' FOO ] LITERAL COMPILE,".
>>>>>>
>>
>>> Does it perform compilation semantics regardless of the STATE?
>>>
>>
>> As I have stated above, COMPILE-NAME may be represented exactly in
>> standard code as your COMPILE-NT.
>
> It does not answer to my question, since your "name>compile" can return
> xt that always identifies STATE-independent execution semantics.
>
> Therefore, I provided a test to check how it works:
>
>   : translate-123 s" 123" evaluate ; immediate
>
>   : test-123
>     [ "translate-123" find-name compile-name ]
>   ;
>   0 test-123 . . \ should print "0 123"

Typo correction:

0 test-123 . . \ should print "123 0"



--
Ruvim

[Stephen Pelc]

On 6 Oct 2022 at 16:13:56 CEST, "Krishna Myneni" <krishna...@ccreweb.org>
wrote:

> The cleaner solution, in my opinion, would be to use nt-based
> compilation when dealing with named words. Then the explicit semantics,
> interpretation or compilation, can be specified for compilation.

I cannot resist a chuckle when I find that in the current gxxxxx the headers
have been changed so that nt=xt.

All that effort and people are back where we started. "Simplify and add
lightness" is always a good engineering maxim.

Stephen
--
Stephen Pelc, ste...@vfxforth.com
MicroProcessor Engineering, Ltd. - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612,
+34 649 662 974
http://www.mpeforth.com - free VFX Forth downloads

[Krishna Myneni]

On 10/7/22 06:35, Stephen Pelc wrote:
> On 6 Oct 2022 at 16:13:56 CEST, "Krishna Myneni" <krishna...@ccreweb.org>
> wrote:
>> The cleaner solution, in my opinion, would be to use nt-based
>> compilation when dealing with named words. Then the explicit semantics,
>> interpretation or compilation, can be specified for compilation.
>
> I cannot resist a chuckle when I find that in the current gxxxxx the headers
> have been changed so that nt=xt.
>
> All that effort and people are back where we started. "Simplify and add
> lightness" is always a good engineering maxim.
>

Gforth's nt = xt-interp. While Gforth provides for dual semantics words,
it's not entirely clear from their paper [1] that xt-comp (referencing
the compilation semantics) is stored separately for all words. It
appears that xt-comp may be computed in some cases.

A simple header for a dual-xt system would only require storage in the
header for another execution token (xt-comp), compared to the single-xt
system. Also, the precedence flag can be remove from the dictionary header.

A dual-xt system does not need a complex header, although the Gforth 1.0
header appears to be quite complex. I haven't digested this paper to
understand the reasons for its complexity, but I suspect they don't have
to do with the fact that it supports words with dual semantics and more
likely the reasons have to do with optimization.

--
Krishna


Refs.

1. B. Paysan and M. A. Ertl, "The new Gforth Header",
https://publik.tuwien.ac.at/files/publik_287599.pdf

[Ron Braithwaite]

Thank you all for an interesting discussion and thread. I have grown very frustrated over time with the constant bickering and occasionally consider unsubscribing from this group. But it is discussions like this that continue to make comp.lang.forth worth following.

I just wanted to stick in my totally unrelated 2 cents worth.

-RonBraithwaite

[minf...@arcor.de]

r...@braithwaites.net schrieb am Freitag, 7. Oktober 2022 um 20:30:26 UTC+2:
> Thank you all for an interesting discussion and thread. I have grown very frustrated over time with the constant bickering and occasionally consider unsubscribing from this group. But it is discussions like this that continue to make comp.lang.forth worth following.
>
> I just wanted to stick in my totally unrelated 2 cents worth.

You're not alone with this opinion. While some "participants" here behave like Statler
and Waldorf from Muppet show, just don't feed them. And perhaps you could start
an interesting discussion, there are some really smart and experienced guys in this
forum.

[Krishna Myneni]

On 10/7/22 04:38, Ruvim wrote:
> On 2022-10-07 03:01, Krishna Myneni wrote:
>> On 10/6/22 15:40, Ruvim wrote:
>>> On 2022-10-05 09:26, Krishna Myneni wrote:
>>
>>>>>>>
>>>>>>>    : compile-nt ( i*x nt -- j*x ) name>compile execute ;
>>>>>>
>>>>>> I actually have such a word in kForth (which is not presently a
>>>>>> dual-token system) called "COMPILE-NAME", which performs the
>>>>>> equivalent of NAME>COMPILE EXECUTE . One can still use expressions
>>>>>> such as "['] FOO COMPILE," or " ... [ ' FOO ] LITERAL COMPILE,".
>>>>>>
>>
>>> Does it perform compilation semantics regardless of the STATE?
>>>
>>
>> As I have stated above, COMPILE-NAME may be represented exactly in
>> standard code as your COMPILE-NT.
>
> It does not answer to my question, since your "name>compile" can return
> xt that always identifies STATE-independent execution semantics.
>
> Therefore, I provided a test to check how it works:
>
>   : translate-123 s" 123" evaluate ; immediate
>
>   : test-123
>     [ "translate-123" find-name compile-name ]
>   ;
>   0 test-123 . . \ should print "0 123"
>
>
> I guess, this test fails in your system.
>

Yes, it does fail:

----

: translate-123 s" 123" evaluate ; immediate

ok
: test-123 [ s" translate-123" find-name compile-name ] ;
ok
0 test-123 .
0 ok

[Krishna Myneni]

Hmm... my newsreader is acting funny. My reply to you got truncated. The
intended reply goes on to say that this is an implementation deficiency
in COMPILE-NAME and it can be fixed. COMPILE-NAME like "COMPILE," should
be able to execute the compilation semantics of the word referenced by
nt from interpretation state when there is an unclosed definition.

--
Krishna

[P Falth]

Also my two systems ntf/lxf a dual xt system and ntf64/lxf64 a single xt system
fails this test.
I think that is correct as the test in my opinion is flawed!
translate-123 will correctly get executed by compile-name.
As the system is in interpretative state when this happens
123 just gets placed on the stack. This is the correct behavior.

To pass Ruvim's test you can define compile-name as

: compile-name name>compile ] execute postpone [ ;

Better would be to define : [execute] execute ; immediate
and write the test as
: test-123b [ s" translate-123" find-name name>compile ] [execute] ;

But the standard way would be

: test-123c translate-123 ;

Writing this in a single xt system like

: test-123d [ ' translate-123 compile, ] ;
It looks to pass the test, but has instead just postponed the translate-123.
As translate-123 is immediate it should be written
: test-123d [ ' translate-123 execute ] ;
and it will fail again

Krishna, I introduced the two XT header over 20 years ago. It works like you
have described. No set-optimizer and a compile, that just compiles a call to the xt.
The syntax is different:

:p name ...... ;p defines the compilation semantics
:r name ....... ;r defines the interpretative semantics

As the name is repeated they do not need to be defined at the same time.
In fact the compilation semantics are defined in the meta compilation and
the interpretative part when the new system is extended.

Best Regards
Peter Fälth

[Krishna Myneni]

On 10/7/22 13:30, Ron Braithwaite wrote:
> On Friday, October 7, 2022 at 7:12:22 AM UTC-7, Krishna Myneni wrote:
>> On 10/7/22 06:35, Stephen Pelc wrote:
>>> On 6 Oct 2022 at 16:13:56 CEST, "Krishna Myneni" <krishna...@ccreweb.org>
>>> wrote:
>>>> The cleaner solution, in my opinion, would be to use nt-based
>>>> compilation when dealing with named words. Then the explicit semantics,
>>>> interpretation or compilation, can be specified for compilation.
>>>
>>> I cannot resist a chuckle when I find that in the current gxxxxx the headers
>>> have been changed so that nt=xt.
>>>
>>> All that effort and people are back where we started. "Simplify and add
>>> lightness" is always a good engineering maxim.
>>>
>> Gforth's nt = xt-interp. While Gforth provides for dual semantics words,
>> it's not entirely clear from their paper [1] that xt-comp (referencing
>> the compilation semantics) is stored separately for all words. It
>> appears that xt-comp may be computed in some cases.
>>

..

>
> Thank you all for an interesting discussion and thread. I have grown very frustrated over time with the constant bickering and occasionally consider unsubscribing from this group. But it is discussions like this that continue to make comp.lang.forth worth following.
>
> I just wanted to stick in my totally unrelated 2 cents worth.
>

The noise in comp.lang.forth has gone up and down over time. I'm not
sure if selective memory filters out the noise of the past and gives the
illusion that the present is much worse. We do seem to be near a minimum
at present, though.

--
Krishna

[Krishna Myneni]

That would certainly be the correct behavior when there is no open
definition. In kForth COMPILE-NAME is a primitive and can use
system-specific knowledge to determine whether or not a definition is in
progress and compile into the current definition when that is the case,
as with "COMPILE,". It simply means that COMPILE-NAME cannot be defined
in standard Forth.

> To pass Ruvim's test you can define compile-name as
>
> : compile-name name>compile ] execute postpone [ ;
>

..

>
> Krishna, I introduced the two XT header over 20 years ago. It works like you
> have described. No set-optimizer and a compile, that just compiles a call to the xt.
> The syntax is different:
>
> :p name ...... ;p defines the compilation semantics
> :r name ....... ;r defines the interpretative semantics
>

Returning to the original post, FVALUE and 2VALUE can be implemented
without STATE. But in my system there is a significant penalty for doing
so. The elegance, simplicity, and flexibility of a dual-xt system -- yes
simplicity -- truly appeals to me. Your experience, and that of others,
with implementing such a system gives me additional motivation to make
an experimental branch of kForth-64 which provides for dual semantics.

I understand that commercial vendors of Forth systems have to take a
more conservative approach to the evolution of their systems. This is
where open-source/free Forths can contribute to exploring evolutionary
paths for future Forth standards.

--
Krishna

[dxforth]

Some carry on as if they were owed entertainment and not received it. It's most
amusing to watch.

[none albert]

In article <thos39$3ji4p$1...@dont-email.me>,
Ruvim <ruvim...@gmail.com> wrote:
<SNIP>

>
>In the general case, there can be standard words, which the system
>provides, that are neither immediate nor ordinary.
>
>For example, the word S" in a standard system can be implemented in such
>a way that if you tick this word, and executes the returned xt in
>compilation state — it will not perform the compilation semantics for
>this word. Hence, this word is nonimmediate in such a case.

That makes the case that S" should be forbidden. It is not needed in
view of recognizers/prefixes.


>--
>Ruvim

[none albert]

In article <thp2um$3m5gm$1...@dont-email.me>,

Stephen Pelc <ste...@vfxforth.com> wrote:
>On 6 Oct 2022 at 16:13:56 CEST, "Krishna Myneni" <krishna...@ccreweb.org>
>wrote:
>> The cleaner solution, in my opinion, would be to use nt-based
>> compilation when dealing with named words. Then the explicit semantics,
>> interpretation or compilation, can be specified for compilation.
>
>I cannot resist a chuckle when I find that in the current gxxxxx the headers
>have been changed so that nt=xt.

Really? That is old fashioned. In 2001 I argued that there is no reason
why a dea (dictionary entry address) couldn't be used as an xt (for EXECUTE).
I implemented such a system.
The rest is optimisation.

>
>All that effort and people are back where we started. "Simplify and add
>lightness" is always a good engineering maxim.
>
>Stephen

[Ruvim]

On 2022-10-07 03:01, Krishna Myneni wrote:

> On 10/6/22 15:40, Ruvim wrote:
>> On 2022-10-05 09:26, Krishna Myneni wrote:

[...]

Well, then please provide definitions of these types, and criteria — how
to detect whether an xt identifies execution semantics of one or another
type.



[...]


--
Ruvim

[Ruvim]

On 2022-10-07 03:01, Krishna Myneni wrote:

> On 10/6/22 15:40, Ruvim wrote:
>> On 2022-10-05 09:26, Krishna Myneni wrote:

[...]

>
> In contrast, in dual-xt systems, there is no need for the term
> "execution semantics".

Even if some standard dual-xt system doesn't need the term "execution
semantics" under the hood, it shall map the standard notions of
"execution semantics" and "identifier of execution semantics" to its
internal notions.

Have a look:

:noname 123 . ; ( xt )


For a standard program it does not matter what this xt means in your
system under the hood, and how you call it internally.

For a standard program this xt is an identifier of execution semantics,
namely of the execution semantics that are defined in the body of this
unnamed definition.

[...]
>>> Your points above about execution token not always representing the
>>> interpretation semantics are valid for a traditional Forth system,
>>> but when you tick a word name, how do you know which semantics are
>>> being returned?
>>
>> Actually, I may tick only ordinary or immediate word.
>> Anyway, when I Tick a word, it returns an identifier of the execution
>> semantics for the word.
>>
>
> In a dual-xt system, which xt do you expect tick or ['] to return?
> xt-interp or xt-comp? It should return xt-interp for backward
> compatibility with single-xt systems.

The standard does not specify a notion of xt-interp or xt-comp.

On any standard system Tick shall return an xt that identifies the
execution semantics of a word.

When a user defines a word via Colon, he defines the execution semantics
for this word.

For example, see 6.1.0450 : (colon)

| The execution semantics of /name/ will be determined
| by the words compiled into the body of the definition.


For the most standard words the execution semantics are specified.

When execution semantics are not specified for a standard word, the
system may provide implementation-dependent execution semantics for the
word. But it's ambiguous to perform these execution semantics in the
general case, and it's ambiguous even to apply Tick to such a standard word.


In a dual-xt system I would make the first xt identify the execution
semantics for the word. The second xt — identifies the execution
semantics of a definition that is used to perform the special
compilation semantics for a word (if any).


I would not provide the "name>interpret" and "name>compile" words since
they are flawed by design and insufficiently specified to be usable on
any standard system.


I would provide "find", "find-name", and words as:

find-semantics ( sd.name -- xt1 xt2 ) \ stateless
compile-name ( i*x nt -- j*x )
interpret-name ( i*x nt -- j*x )


Well, having internal words like

name>xt ( nt -- xt true | nt false )
name>xt2 ( nt -- xt true | nt false )


The words of an public API can be defined as follows:

: compilation ( -- flag ) state @ 0<> ;
: leave-compilation ( -- ) ] ;
: enter-compilation ( -- ) postpone [ ;


: execute-interpretively ( i*x xt -- j*x )
compilation 0= if execute exit then
leave-compilation execute enter-compilation
;
: execute-compilatively ( i*x xt -- j*x )
compilation if execute exit then
enter-compilation execute leave-compilation
;
: interpret-name ( i*x nt -- j*x )
name>xt 0= -14 and throw execute-interpretively
;
: compile-name ( i*x nt -- j*x )
name>xt2 if execute-compilatively exit then
name>xt 0= -14 and throw compile,
;


: postpone-name ( nt -- )
\ portable implementation
lit, ['] compile-name compile,
;
: postpone-name ( nt -- )
\ system-specific implementation, which is slightly more efficient
name>xt2 if lit, ['] execute-compilatively compile, exit then
name>xt 0= -32 and throw lit, ['] compile, compile,
;
: postpone ( "name" -- )
parse-name find-name dup 0= -32 and throw postpone-name
; immediate



: find-semantics ( sd.name -- xt.word|0 xt.special-com|0 )
\ stateless
find-name dup 0= if 0 exit then
dup name>xt and swap name>xt2 and
;

: find-word? ( sd.name -- xt flag.special true | false )
\ stateful
find-semantics compilation if
dup if nip true true exit then drop
dup if false true exit then ( false )
exit
then
over 0= if 2drop false exit then 0<> true
;
: find-word ( sd.name -- xt flag.special true | sd.name false )
\ stateful
2dup 2>r find-word? dup if 2rdrop exit then drop 2r> false
;
: find ( c-addr -- c-addr 0 | xt 1 | xt -1 )
\ stateful
dup >r count find-word? 0= if r> 0 exit then rdrop
if 1 else -1 then
;


: immediate ( -- )
latest-name name>xt -32 ?throw name-xt2!
;

: compile: ( "name" -- ) ( C: -- colon-sys )
parse-name find-name dup -13 ?throw >r
['] :noname execute-balance 1- pick r> name-xt2!
;


Where

\ latest-name ( -- nt )
\ Return the name token nt for the definition
\ whose name was placed into the compilation word list most recently,
\ or throw an exception if there is no such a definition.


: execute-balance ( i*x xt -- j*x n )
\ Take xt, execute it and place on the stack
\ the stack depth difference
depth 1- >r execute depth r> -
;



--
Ruvim

[Ruvim]

What make you think so?

> translate-123 will correctly get executed by compile-name.
> As the system is in interpretative state when this happens
> 123 just gets placed on the stack. This is the correct behavior.

When I apply "compile-name" to an nt, my intention (and expectation) is
to perform the behavior that takes place when the Forth text interpreter
encounters the corresponding word name in compilation state. (¹)

Otherwise, if I want to perform this behavior, you forced me to specify
my intention *twice* — via enter into compilation state and via use of
"compile-name". It would be a flawed API.

And, there are no cases when you need to perform via "compile-name" a
behavior that will not be equivalent to the behavior (¹).
Counterexamples are welcome.

>
> To pass Ruvim's test you can define compile-name as
>
> : compile-name name>compile ] execute postpone [ ;

Yes, but a little more sophisticated.

You should not leave compilation state if the system was in compilation
state before executing the xt.



--
Ruvim

[Anton Ertl]

Ruvim <ruvim...@gmail.com> writes:
>On 2022-09-29 21:43, Anton Ertl wrote:
>> Zbig <zbigni...@gmail.com> writes:
>>> So state-smart words are bad
>>
>> They are evil! [ertl98]
>
>In this regard the paper shows that it's incorrect to implement an
>ordinary word as an immediate word. So, it's not just evil, it's a mistake.

There are several aspects why state-smart words are worse than many
other mistakes:

1) They work as intended much of the time, and only behave wrongly in
corner cases.

2) This seduces some people to try to argue (by language lawyering or
by declaring the standar to be buggy) that not the state-smart words,
but the corner case is a mistake; i.e., indemnify the implementation
by blaming the programmer.

3) In your case it's even worse, because AFAICS you try to declare not
just the corner case usage, but also all widely used systems as buggy
through language-lawyering.

>But concerning 's"' (and other combined words) the paper only says (in
>the section 3, on page 3):
>
> | This definition behaves correctly as long as it
> | is only processed by the text interpreter,
> | but it fails with ', postpone, etc.,
> | as discussed in Section 2.
>
>This reasoning is wrong:
> 1. actually, it does not fail with ' (Tick);
> 2. if it fails with "postpone", then a problem is in "postpone", and
>the problem should be corrected in "postpone".
>
>I explained [1] why it doesn't fail with Tick:
>
>It's obvious that in the general case, performing execution semantics in
>compilation state may be inequivalent to performing these execution
>semantics in interpretation state.

Apart from the difference in the contents of STATE, there must not be
a difference between the behaviour. S" is not specified as a
STATE-dependent word, so for it there should be no difference.

>And we don't have any ground to say
>that they should be equivalent in the case of the s" word, since the
>execution semantics for s" are not specified by the standard.

Possibly, but ' arguably should produce an execution token for the
interpretation semantics of a word. Arguments for that position:

1) 6.1.0070 ' says:

|When interpreting, ' xyz EXECUTE is equivalent to xyz.

So when interpreting,

' S" execute bla"

is equivalent to

S" bla"

2) "4.1.2 Ambiguous conditions" explicitly disallows

|attempting to obtain the execution token, (e.g., with 6.1.0070 ',
|6.1.1550 FIND, etc. of a definition with undefined interpretation
|semantics;

but not of words without execution semantics.

>An alternative to the TO-based approach is to just define two words:
>getter and setter. So, instead of a phrase "to foo" you use a single
>word "to-foo". I mean a real ordinary word, so "' to-foo execute"
>performs execution semantics of "to-foo". The word "to-foo" is created
>automatically along with "foo".

Gforth has a to-recognizer, and there you can write ->FOO, eliminating
the need for TO. You cannot ' ->FOO, however, unless you first define

: ->foo \ a real, tickable word
->foo \ only the to-recognizer with FOO
;

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: https://forth-standard.org/
EuroForth 2022: https://euro.theforth.net

[Anton Ertl]

Ruvim <ruvim...@gmail.com> writes:
>Actually, these methods ("SET-COMPSEM" and "SET-OPTIMIZER") are mutual
>exclusive for a word

No:

require set-compsem.fs
: foo ." interpretation semantics" ;
opt: drop ]] "interpretation semantics" type [[ ;
compsem: ." compilation semantics" ;

foo
: bar foo [ ' foo compile, ] ; \ prints "compilation semantics"
bar \ prints "interpretation semantics"
see bar \ shows that bar contains '"interpretation semantics" type'

However, as you can see from the example above, you don't get the OPT:
(SET-OPTIMIZER) implementation of the interpretation semantics when
text-interpreting FOO in either interpret or compile state; you have
to explicitly COMPILE, the xt representing the execution semantics.
It's unlikely that you need to optimize this case, so you will not
normally use OPT: and COMPSEM: (or SET-OPTIMIZER and SET-COMPSEM)
together.

>> The reason that
>> the new definition fails (in compilation mode) appears to be that there
>> is no xt for the value on the stack for the new compilation semantics to
>> operate on with >BODY.
>
>It's absent by design, since this xt is not required on implied use cases.

There is also no way to get it there. The text interpreter performs
compilation semantics as follows:

( nt ) name>compile ( xt1 xt2 ) execute

( xt3 ) SET-COMPSEM changes NAME>COMPILE for the word to produce xt3 xt-ex,
where xt-ex is the xt of EXECUTE.

>See also the example in [1]:
>
>\ :noname ." compiling" ;
>\ : foo ." interpreting" ; set-compsem
>
>
>Take into account that the anonymous definition intended to perform the
>compilation semantics for "foo" is defined *before* "foo".
>It's because "set-compsem" changes the last defined word, and an
>anonymous definition is also a word in Gforth (1)

Alternatively, you can use interpretive quotations, because
interpretive quotations, like their compilation counterparts restore
the previous "current definition" when they end.

require set-compsem.fs
: foo ." interpreting" ;
[: ." compiling" ;] set-compsem
foo \ prints "interpreting"
: bar foo ; \ prints "compiling"

Or you use COMPSEM: as shown above.

>A closure (namely, a partial application [2]) could be used for that,
>but the limitation (1) makes it more difficult, since a generated
>anonymous definition becomes last definition.

Closures, like quotations, save the current definition at the start
and restore it at the end.

[Anton Ertl]

Krishna Myneni <krishna...@ccreweb.org> writes:
>[COMPILE] is obsolete and need not be discussed. It used to exist in
>kForth and has been removed.

On such a system you can use SET-COMPSEM for optimization, but
SET-OPTIMIZER covers more cases.

>In short, I think using SET-COMPSEMP is a more transparent way of
>achieving optimization and your BROKEN-VALUE definition would be my
>preferred way to go about it.

This follows the progression of Forth systems: In the beginning Forth
systems had no optimization. Then came cmForth which optimized using
compilation semantics. Then came systems like SwiftForth, VFX, and
Gforth that optimize with COMPILE,.

>On a dual token system (one which provides
>a reference to interpretation semantics and a separate reference to
>compilation semantics) I want to avoid thinking about whether the
>compilation semantics are default or non-default.

Once you have eliminated [COMPILE], there is no need to.

[Anton Ertl]

Gerry Jackson <do-no...@swldwa.uk> writes:
>On 01/10/2022 09:45, Anton Ertl wrote:
>> there is no standard program that could determine if TO does
>> not parse, so a standard system can also have a non-parsing TO.
>>
>
>But a non-standard program may detect a non-parsing TO e.g.
>where TO sets a flag that a following value tests to return or change
>its value.

Certainly, but the standard does not specify what standard systems do
when encountering non-standard programs.

>\ --------------------
> VFX Forth 64 for Windows x64
> © MicroProcessor Engineering Ltd, 1998-2022
>
> Version: 5.20 [build 4077]
> Build date: 1 August 2022
>
> Free dictionary = 6966470 bytes [6803kb]
>
>
>0 value x x . 0 ok
>123 to to x x . 123 ok
>234 to to to to x x . 234 ok
>345 to ok-1
>x x . 345 ok
>\ ----------------------------

One case where my run-time flag-setting variant differs from my
optimization of it is

0 value x
0 value y
: foo to if x then y ;
5 0 foo .s
y .

Which shows that my optimization is correct only for standard usage,
which is not ideal.

[Anton Ertl]

Gerry Jackson <do-no...@swldwa.uk> writes:
>If I define +TO as (using standard Forth):
>
>: +to ( n "name" -- )
> parse-name 2>r ( -- n ) ( R: -- ca u )
> 2r@ s" + to " 2r@
> <# holds holds holds 2r> #> evaluate
>; immediate

For resilience against incompatible redefinitions of + and against not
having them in the search order, you should have a wordlist that
contains just synonyms of + and TO, and put that wordlist on the
search order before the EVALUATE, and remove it afterwards.

>' ['] and POSTPONE can be applied to +TO.

But this +TO is state-smart thanks to using EVALUATE, so I would
recommend against ticking and POSTPONEing it.