POSTPONEing literals?

[dxforth]

I've found an essentially no-cost way of implementing it.
Other than the novelty of being able to write ]] 123 [[
what are the reasons/examples for having it?

[Hugh Aguilar]

My MACRO: and POST: have worked with literals for quite some time.
This is a comment from my code:
\ POST: LIT: DLIT: FLIT: "LIT: are used inside of immediate words that meta-compile.
\ This is similar to ]] but a lot better because it supports literals
\ and immediate words such as [CHAR] POSTPONE etc. that take data out of the input stream.

Why are you interested in Anton Ertl's ]] construct?
It is worthless from a technical standpoint. It was invented primarily as a workaround
for Anton Ertl's lack of disambiguifiers that Elizabeth Rather forbids him from using.
Anybody who uses ]] is just brown-nosing Anton Ertl who is brown-nosing
Elizabeth Rather --- an example of: "meta-brown-nosing" --- this is a pun, because my
MACRO: and POST: are used for meta-compiling (I explain this in case
any of the ANS-Forth cult didn't get the pun).

Note that my MACRO: and POST: are ANS-Forth code.
Your as-yet-undescribed way to POSTPONE literals is not going to be ANS-Forth.
Because of this, the ANS-Forth cult will not be interested in it.

I really doubt that anybody in the ANS-Forth cult understand meta-compiling.

[dxforth]

It may interest forth implementers that provide POSTPONE. ISTM an
oversight that POSTPONE shouldn't handle both 'words and numbers'
directly when any forth interpreter can.

A fig-style interpreter looks something like:

: interpret ( -- )
begin
bl word dup c@
while
find ?dup if
state @ if compile, else execute then
else
( c-addr) number
then ?stack
repeat drop ;

Given 'number', we can define:

: POSTPONE
?comp bl word find ?dup if
0< if compile compile then compile,
end number ; immediate

Such a POSTPONE will handle any literal that 'number' can.

\ Simple POSTPONE macro
: ]]
begin >in @ bl word count ?dup
while s" [[" compare
while >in ! [compile] postpone
repeat drop end 2drop ; immediate

[Anton Ertl]

dxforth <dxf...@gmail.com> writes:
>It may interest forth implementers that provide POSTPONE. ISTM an
>oversight that POSTPONE shouldn't handle both 'words and numbers'
>directly when any forth interpreter can.

My guess ist that POSTPONE was not specified to handle numbers because
neither [COMPILE] nor COMPILE could handle numbers, and programmers
could work around the lack of POSTPONE 123 with 123 POSTPONE LITERAL.

>A fig-style interpreter looks something like:
>
>: interpret ( -- )
> begin
> bl word dup c@
> while
> find ?dup if
> state @ if compile, else execute then
> else
> ( c-addr) number
> then ?stack
> repeat drop ;
>
>Given 'number', we can define:
>
>: POSTPONE
> ?comp bl word find ?dup if
> 0< if compile compile then compile,
> end number ; immediate

This NUMBER is pretty "smart", leaving the number on the stack in
interpret state, performing the compilation semantics of LITERAL or
2LITERAL in compile state, and also knowing that one more level of
LITERAL or 2LITERAL is needed when it's called from POSTPONE.

I guess complications like that also discouraged the Forth-94
committee from standardizing POSTPONEing numbers.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2020: https://euro.theforth.net/2020

[dxforth]

Not a problem since POSTPONE is required to postpone. The ?COMP
handles any attempt by a user to do otherwise.

>
> I guess complications like that also discouraged the Forth-94
> committee from standardizing POSTPONEing numbers.

If they considered it, surely they would have mentioned it. Their
'go to' reference was Hayes' Rochester Conference paper. Does anyone
have it? It would be interesting to know what it said.

>
> - anton
>

[Anton Ertl]

dxforth <dxf...@gmail.com> writes:
>On 28/04/2021 16:29, Anton Ertl wrote:
>> dxforth <dxf...@gmail.com> writes:
>>>Given 'number', we can define:
>>>
>>>: POSTPONE
>>> ?comp bl word find ?dup if
>>> 0< if compile compile then compile,
>>> end number ; immediate
>>
>> This NUMBER is pretty "smart", leaving the number on the stack in
>> interpret state, performing the compilation semantics of LITERAL or
>> 2LITERAL in compile state, and also knowing that one more level of
>> LITERAL or 2LITERAL is needed when it's called from POSTPONE.
>
>Not a problem since POSTPONE is required to postpone.

How does that make NUMBER perform the extra LITERAL level?

Test case:

: foo postpone 123 ; immediate
: bar foo ;
bar . \ prints 123

>> I guess complications like that also discouraged the Forth-94
>> committee from standardizing POSTPONEing numbers.
>
>If they considered it, surely they would have mentioned it. Their
>'go to' reference was Hayes' Rochester Conference paper. Does anyone
>have it? It would be interesting to know what it said.

A number of Rochester Forth Conference Proceedings have appeared in
JFAR <http://soton.mpeforth.com/flag/jfar/index.html>. There are two
papers by Hayes listed in the author index:
<http://soton.mpeforth.com/flag/jfar/authors-H.html>, but they are
unlikely to be about POSTPONE.

[dxforth]

On 28/04/2021 17:37, Anton Ertl wrote:
> dxforth <dxf...@gmail.com> writes:
>>On 28/04/2021 16:29, Anton Ertl wrote:
>>> dxforth <dxf...@gmail.com> writes:
>>>>Given 'number', we can define:
>>>>
>>>>: POSTPONE
>>>> ?comp bl word find ?dup if
>>>> 0< if compile compile then compile,
>>>> end number ; immediate
>>>
>>> This NUMBER is pretty "smart", leaving the number on the stack in
>>> interpret state, performing the compilation semantics of LITERAL or
>>> 2LITERAL in compile state, and also knowing that one more level of
>>> LITERAL or 2LITERAL is needed when it's called from POSTPONE.
>>
>>Not a problem since POSTPONE is required to postpone.
>
> How does that make NUMBER perform the extra LITERAL level?
>
> Test case:
>
> : foo postpone 123 ; immediate
> : bar foo ;
> bar . \ prints 123
>

Omit immediate :)

POSTPONE 123 is akin to postponing an immediate word; it
will execute when later encountered, rather than compile.

123 POSTPONE LITERAL isn't postponing 123; it's postponing
LITERAL and 123 happens to be on the stack when it does.
IOW you're setting up a scenario to get a desired effect.

[dxforth]

On 28/04/2021 19:02, dxforth wrote:
> On 28/04/2021 17:37, Anton Ertl wrote:
>>
>> How does that make NUMBER perform the extra LITERAL level?
>>
>> Test case:
>>
>> : foo postpone 123 ; immediate
>> : bar foo ;
>> bar . \ prints 123
>>
>
> Omit immediate :)
>
> POSTPONE 123 is akin to postponing an immediate word; it
> will execute when later encountered, rather than compile.

I'll have to concede this isn't useful. I had an example
but it turns out I had used a constant rather than a literal!

[Ruvim]

On 2021-04-28 12:02, dxforth wrote:
> On 28/04/2021 17:37, Anton Ertl wrote:
>> dxforth <dxf...@gmail.com> writes:
>>> On 28/04/2021 16:29, Anton Ertl wrote:
>>>> dxforth <dxf...@gmail.com> writes:
>>>>> Given 'number', we can define:
>>>>>
>>>>> : POSTPONE
>>>>>   ?comp bl word find ?dup if
>>>>>     0< if compile compile then compile,
>>>>>   end  number ; immediate
>>>>
>>>> This NUMBER is pretty "smart", leaving the number on the stack in
>>>> interpret state, performing the compilation semantics of LITERAL or
>>>> 2LITERAL in compile state, and also knowing that one more level of
>>>> LITERAL or 2LITERAL is needed when it's called from POSTPONE.
>>>
>>> Not a problem since POSTPONE is required to postpone.
>>
>> How does that make NUMBER perform the extra LITERAL level?
>>
>> Test case:
>>
>> : foo postpone 123 ; immediate
>> : bar foo ;
>> bar . \ prints 123
>>
>
> Omit immediate :)
>
> POSTPONE 123 is akin to postponing an immediate word; it
> will execute when later encountered, rather than compile.

There is an approach that shows (or helps to understand) how POSTPONE
should work regardless of its argument kind.

: foo postpone 123 ; immediate

: bar foo ;

1. Substitute foo in bar by the body of bar in square brackets.
<==>
: bar [ postpone 123 ] ;

2. Suppose that interpretation semantics of POSTPONE X is to *perform*
compilation semantics of X. Remember that performing compilation
semantics of X is what the text interpreter does when it encounters X in
compilation state.
<==>
: bar 123 ;


Hence,

: bar foo ;

shall be equivalent to

: bar 123 ;


--
Ruvim

[Anton Ertl]

dxforth <dxf...@gmail.com> writes:
>On 28/04/2021 17:37, Anton Ertl wrote:
>> Test case:
>>
>> : foo postpone 123 ; immediate
>> : bar foo ;
>> bar . \ prints 123
>>
>
>Omit immediate :)
>
>POSTPONE 123 is akin to postponing an immediate word;

What would be the use of that? If we would want that, we could just
leave the POSTPONE away. Also, 123 does not behave like an immediate
word in normal compilation, so why would you want that POSTPONE 123
treats it as immediate? In normal text interpretation (interpret and
compile state), the number 123 behaves like

123 constant 123 \ no immediate

BTW, the example above works in the development version of Gforth, and
that's not a bug.

>123 POSTPONE LITERAL isn't postponing 123

If POSTPONE 123 does not work, I have to write 123 POSTPONE LITERAL,
and in Gforth the result after running the code is the same:

: foo1 postpone 123 ; immediate ok
: foo2 123 postpone literal ; immediate ok
: bar1 foo1 ; ok
: bar2 foo2 ; ok
simple-see bar1
$7F447B532DA0 lit
$7F447B532DA8 #123
$7F447B532DB0 ;s ok
simple-see bar2
$7F447B532DE0 lit
$7F447B532DE8 #123
$7F447B532DF0 ;s ok

[dxforth]

On 28/04/2021 15:24, dxforth wrote:
> ...

> A fig-style interpreter looks something like:
>
> : interpret ( -- )
> begin
> bl word dup c@
> while
> find ?dup if
> state @ if compile, else execute then
> else
> ( c-addr) number
> then ?stack
> repeat drop ;
>
> Given 'number', we can define:
>
> : POSTPONE
> ?comp bl word find ?dup if
> 0< if compile compile then compile,
> end number ; immediate

That won't work per Anton's comment. POSTPONE needs to know the compiling
literal used by 'number'. Here are the tweaks:

0 value nlit \ xt of the literal used by NUMBER

: number ( c-addr -- | d|n|r )
...
2dup number? if
2nip dpl @ 0< if
drop ['] literal
else
['] 2literal
then to nlit true
else
fnumber \ defer for floats
then
...
( flag) ?defined \ 0 = fail & abort
state @ if nlit execute then ;

: POSTPONE
?comp bl word find ?dup if
0< if compile compile then

else number nlit then compile, ; immediate

\\ test

DX-Forth 4.45 2021-04-17

: foo postpone 123 : immediate LITERAL is system ok
: bar foo ; ok
bar . 123 ok
: foo2 postpone 123e : immediate FLITERAL is system ok
: bar2 foo2 ; ok
bar2 f. 123. ok

[dxforth]

Result is ambiguous (POSTPONE inside brackets).

[Ruvim]

It isn't a real code, it's a pseudo code, a conceptual model. And in any
case, the result is not ambiguous since the item 2 describes semantics
for POSTPONE inside brackets (i.e., the interpretation semantics for
POSTPONE). If it would be a real code, POSTPONE should be just redefined
to have these interpretation semantics. In one my framework (standard
compliant) this looks like the following:

: postpone
state @ if postpone postpone exit then
parse-lexeme ['] translate-lexeme execute-compiling ?nf
; immediate

\ If you want to go deeper:
\ translate-lexeme ( i*x sd-lexeme -- j*x flag )
\ -- https://git.io/J3Tkf
\ execute-compiling ( i*x xt -- j*x )
\ -- https://git.io/J3Tk5

[dxforth]

On 28/04/2021 22:38, dxforth wrote:
> ...

> POSTPONE needs to know the compiling
> literal used by 'number'. Here are the tweaks:

We can improve on that as nlit wasn't needed after all.
'number' now returns the xt directly.

: number ( c-addr -- [d|n|r] xt )

...
2dup number? if
2nip dpl @ 0< if
drop ['] literal
else
['] 2literal

then true

else
fnumber \ defer for floats
then
...
( flag) ?defined \ 0 = fail & abort

>r state @ if r@ execute then r> ;

: POSTPONE
?comp bl word find ?dup if
0< if compile compile then

else number then compile, ; immediate

'number' in INTERPRET will need to drop the xt:

: interpret ( -- )
begin
bl word dup c@
while
find ?dup if
state @ if compile, else execute then
else

( c-addr) number DROP
then ?stack
repeat drop ;

What was the cost of making POSTPONE handle numeric literals?
Comparing the old and new DX-Forth executables, it was an
extra 16 bytes.

\ tests

[azathot...@gmail.com]

Hugh do you have an online forth tutorial?
ARe you able to do dynamic web sites in forth?

[dxforth]

This guy can:

https://www.45office.com/

[Anton Ertl]

Or, instead of making NUMBER state-smart, you put the STATE handling
in INTERPRET:

Replace the last line of NUMBER with ";"

Define INTERPRET as:

: interpret ( -- )
begin
bl word dup c@
while
find ?dup if
state @ if compile, else execute then
else

( c-addr) number state @ if execute else drop then
then ?stack
repeat drop ;

And POSTPONE as:

: POSTPONE

bl word find ?dup if
0< if compile compile then

else number dup >r execute r> then compile, ; immediate

This POSTPONE is not STATE-"smart".

[dxforth]

Would you have an example of use where it would matter? POSTPONE
COMPILE LITERAL etc are either legally or technically compile-only.

[none albert]

In article <2021Apr2...@mips.complang.tuwien.ac.at>,
Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
<SNIP>

>
>Or, instead of making NUMBER state-smart, you put the STATE handling
>in INTERPRET:
>
>Replace the last line of NUMBER with ";"
>
>Define INTERPRET as:
>
>: interpret ( -- )
> begin
> bl word dup c@
> while
> find ?dup if
> state @ if compile, else execute then
> else
> ( c-addr) number state @ if execute else drop then
> then ?stack
> repeat drop ;

Or, if `` number '' is handled by immediate prefixes you can just
leave the else part.
Note that prefixes like $ # % ' presort the different cases of numbers
to handle. ( $1234 #200 %10011100 'A' )

Or if INTERPRET monitors stack changes, all cases of `` number ''
would be simplified.

>
>And POSTPONE as:
>
>: POSTPONE
> bl word find ?dup if
> 0< if compile compile then
> else number dup >r execute r> then compile, ; immediate
>
>This POSTPONE is not STATE-"smart".

It is IMMEDIATE-"smart" , but that is less objectionable.

>- anton

Groetjes Albert
--
"in our communism country Viet Nam, people are forced to be
alive and in the western country like US, people are free to
die from Covid 19 lol" duc ha
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[Anton Ertl]

dxforth <dxf...@gmail.com> writes:
>> This POSTPONE is not STATE-"smart".
>
>Would you have an example of use where it would matter?

: postpone postpone postpone ; immediate
: foo [ postpone . ] ; immediate

>POSTPONE
>COMPILE LITERAL etc are either legally or technically compile-only.

"Compile-only" is used in two different meanings:

1) Has undefined interpretation semantics.

2) Errors out when being run in interpret state.

The fixed POSTPONE is technically fine wrt either meaning: it has
interpretation semantics, and it does not produce error when it runs
in interpret state

Concerning "legally", the standard only specifies 1) for POSTPONE, and
that is easily fixed:

: postpone postpone postpone ; immediate

Apart from that, given all the slavery and freedom rethoric you have
been using wrt standard compliance, don't you want to provide the
dxforth programmers with the freedom to just write

: foo [ postpone . ] ;

Gforth, SwiftForth, and VFX provide this freedom.

[Ruvim]

On 2021-04-29 11:54, Anton Ertl wrote:
> dxforth <dxf...@gmail.com> writes:
>>> This POSTPONE is not STATE-"smart".
>>
>> Would you have an example of use where it would matter?
>
> : postpone postpone postpone ; immediate
> : foo [ postpone . ] ; immediate

It's ambiguous. It should be admitted, at the moment the standard
doesn't allow to perform compilation semantics in interpretation state.

But in any standard system, POSTPONE can be redefined in such a way that
your test will work as expected:

: postpone ( -- ) \ "name"
bl word find dup 0= if -13 throw then
swap postpone literal
1 = if ['] execute-compiling else ['] compile, then compile,
; immediate



--
Ruvim

[dxforth]

On 29/04/2021 18:54, Anton Ertl wrote:
>
>>> This POSTPONE is not STATE-"smart".

I agree your mods correct the oddity of NUMBER doing work that is
properly the function of INTERPRET and POSTPONE. I'll be retaining
?COMP which in my actual code is positioned just before NUMBER in
POSTPONE. That's because my COMPILE already has ?COMP built-in for
crash prevention.

>>
>>Would you have an example of use where it would matter?
>
> : postpone postpone postpone ; immediate
> : foo [ postpone . ] ; immediate
>
>>POSTPONE
>>COMPILE LITERAL etc are either legally or technically compile-only.
>
> "Compile-only" is used in two different meanings:
>
> 1) Has undefined interpretation semantics.
>
> 2) Errors out when being run in interpret state.
>
> The fixed POSTPONE is technically fine wrt either meaning: it has
> interpretation semantics, and it does not produce error when it runs
> in interpret state
>
> Concerning "legally", the standard only specifies 1) for POSTPONE, and
> that is easily fixed:
>
> : postpone postpone postpone ; immediate
>
> Apart from that, given all the slavery and freedom rethoric you have
> been using wrt standard compliance, don't you want to provide the
> dxforth programmers with the freedom to just write
>
> : foo [ postpone . ] ;
>
> Gforth, SwiftForth, and VFX provide this freedom.
>

200x doesn't and the principals of those systems have been sitting on
its board since inception. If anyone is being disenfranchised, it
would appear to be 200x users.

[dxforth]

On 29/04/2021 21:09, Ruvim wrote:
> On 2021-04-29 11:54, Anton Ertl wrote:
>> dxforth <dxf...@gmail.com> writes:
>>>> This POSTPONE is not STATE-"smart".
>>>
>>> Would you have an example of use where it would matter?
>>
>> : postpone postpone postpone ; immediate
>> : foo [ postpone . ] ; immediate
>
> It's ambiguous. It should be admitted, at the moment the standard
> doesn't allow to perform compilation semantics in interpretation state.

Has the case been made that it should? That something now works
that previously didn't is a curiosity - not a demonstrated need.
The functionality described above can almost certainly be duplicated
by other means. I just haven't been able to find the many examples
of use that would justify changing POSTPONE to do it. Why I asked.
The same for POSTPONEing literals. I've just spent 21 bytes on
something I don't know I'll ever use. When forth's hacks start
thinking like Moore, the language may gain some credibility but
not until then.

[Ruvim]

So the question is why do we have (or why do we need to provide) some
features that are very rarely used and have the alternatives?

I see this problem as a problem of integrality and consistency.

A user expects that the system/language is (or should be) integral and
consistent. A user mentally builds a model of the system, and this model
allows to predict the system's behavior in edge case, or to extrapolate
behavior from one set of situations into another set.

If some prediction fails, the reason should be sound, convincing, and
clear to correct the model. Otherwise it will look like a bug or a flaw.



If POSTPONE works for words, why it cannot work for numbers? What is a
sound reason for that? Does it have some intrinsic uncertainty? Is it
difficult to implement?

If nothing of that, it's better to not break a more general model and
support applying POSTPONE to numbers.




Ditto about performing compilation semantics in interpretation state.
If a user (a program) explicitly directs a system to perform some
compilation semantics, why he should be concerned to also set
compilation state? The system can set compilation state by itself. It's
not difficult. And all required information is available for the system.

A small nuance is that the system should not revert the state if it was
explicitly directed to change the state. But it's also a simple thing,
see my the same example https://git.io/J3331


So there is no any sound reason to break (i.e. to not support) a more
general model.


--
Ruvim

[none albert]

Your reasoning is sound. My conclusion is that with the kind of
abstraction POSTPONE offers we took a wrong turn. Instead of supporting
POSTPONE for numbers, I want to get rid of POSTPONE altogether.

>--
>Ruvim

[Ruvim]

What makes you think so?

> Instead of supporting POSTPONE for numbers,
> I want to get rid of POSTPONE altogether.

But then you have to get rid something from FIND [ ] and COMPILE,
since otherwise POSTPONE can be implemented using these words.

So, do you want to lock the metaprogramming capabilities, or just
suggest a better alternative to POSTPONE like like "c{ ... }c" construct?


--
Ruvim

[none albert]

In article <s6h94c$1s7$1...@dont-email.me>, Ruvim <ruvim...@gmail.com> wrote:
>On 2021-04-30 14:43, albert wrote:
>> In article <s6gos1$7jf$1...@dont-email.me>, Ruvim <ruvim...@gmail.com> wrote:

<SNIP>

>> Your reasoning is sound. My conclusion is that with the kind of
>> abstraction POSTPONE offers we took a wrong turn.
>
>What makes you think so?

I need more time reading all the discussions about POSTPONE than I
have ever spent in using it.

>
>
>> Instead of supporting POSTPONE for numbers,
>> I want to get rid of POSTPONE altogether.
>
>But then you have to get rid something from FIND [ ] and COMPILE,
>since otherwise POSTPONE can be implemented using these words.
>
>So, do you want to lock the metaprogramming capabilities, or just
>suggest a better alternative to POSTPONE like like "c{ ... }c" construct?

There is one meta programming facility in lucky thus far.

{ _does ! _build ! '{ RUN 'NAME , 'DEA , 'dodoe , 'SWAP , '>CFA , '! ,
_does @ 'LITERAL RUN '>DFA , '@ , ', , _build @ , '} RUN : } : META

An immediate word is "postpone"d 1) by e.g.
'{ RUN
and a word is compiled by e.g.
'SWAP ,

-All words have one behaviour.
-If you meta compile you have to look up immediacy
- , is approximately COMPILE, and RUN is approximately EXECUTE
- For literals just use good old LITERAL

POSTPONE saves you from looking up immediacy, that is not worth it
in my book.

>--
>Ruvim

1) Actually it is not postponed, it is run immediately.

[dxforth]

I imagine a new user wants a concise, easily comprehended, language that
lets them do what needs to be done. Fooling compilers into doing weird
and wonderful things is one thing, but arguing entitlement another.

I wouldn't have a clue what the postpone/foo example shown above is meant
to do (and I'm supposed to know forth). In extending POSTPONE to handle
literals one might argue it's simple to understand, less clumsy than
POSTPONE LITERAL et al and costs little to implement. It's something that
could - possibly should - have been incorporated into POSTPONE from the
beginning, had someone thought of it. Can Standard Forth and users get by
without it? I imagine they can.

[Ruvim]

On 2021-04-30 23:17, albert wrote:
> In article <s6h94c$1s7$1...@dont-email.me>, Ruvim <ruvim...@gmail.com> wrote:
>> On 2021-04-30 14:43, albert wrote:
>>> In article <s6gos1$7jf$1...@dont-email.me>, Ruvim <ruvim...@gmail.com> wrote:
> <SNIP>
>>> Your reasoning is sound. My conclusion is that with the kind of
>>> abstraction POSTPONE offers we took a wrong turn.
>>
>> What makes you think so?
>
> I need more time reading all the discussions about POSTPONE than I
> have ever spent in using it.
>
>>
>>
>>> Instead of supporting POSTPONE for numbers,
>>> I want to get rid of POSTPONE altogether.
>>
>> But then you have to get rid something from FIND [ ] and COMPILE,
>> since otherwise POSTPONE can be implemented using these words.
>>
>> So, do you want to lock the metaprogramming capabilities, or just
>> suggest a better alternative to POSTPONE like like "c{ ... }c" construct?
>
> There is one meta programming facility in lucky thus far.
>
> { _does ! _build ! '{ RUN 'NAME , 'DEA , 'dodoe , 'SWAP , '>CFA , '! ,
> _does @ 'LITERAL RUN '>DFA , '@ , ', , _build @ , '} RUN : } : META

I see. It seems, it can be refactored as the following:

{ 'LITERAL RUN } : LIT,

{ _does ! _build ! '{ RUN { NAME DEA dodoe SWAP >CFA ! } ,
_does @ LIT, { >DFA @ , } , _build @ , '} RUN : } : META:


Or an even better variant:

\ If the control flow stack is separated from the data stack
{ ( xt1 -- xt2 ) '{ RUN RUN '} RUN } : DEF-BY

\ If the control flow stack is not separated
{ ( xt1 -- xt2 ) '{ RUN-EFFECT N>R RUN NR> DROP '} RUN } : DEF-BY


{ ( xt-build xt-does )
{ { NAME DEA dodoe SWAP >CFA ! } , LIT, { >DFA @ , } , , } DEF-BY
} : META

\ Usage:
\ { ...build } { ...does } META : X:

> An immediate word is "postpone"d 1) by e.g.
> '{ RUN
> and a word is compiled by e.g.
> 'SWAP ,
>
> -All words have one behaviour.
> -If you meta compile you have to look up immediacy
> - , is approximately COMPILE, and RUN is approximately EXECUTE
> - For literals just use good old LITERAL
>
> POSTPONE saves you from looking up immediacy, that is not worth it
> in my book.

Is it possible to define the word P in such a way that

'{ P 'NAME P

is equivalent to

'{ RUN 'NAME ,

?

Or maybe the prefix P: that

P:{ P:NAME

is equivalent to

'{ RUN 'NAME ,

?

> 1) Actually it is not postponed, it is run immediately.

But it's running is postponed!


--
Ruvim

[Ruvim]

Agreed.

> I wouldn't have a clue what the postpone/foo example shown above is meant
> to do (and I'm supposed to know forth).

By common expectations, the following definition of foo

: p postpone postpone ; immediate \ (1)
: foo [ p . ] ; immediate \ (2)

should be equivalent to

: foo p . ; immediate \ (3)

that is equivalent to

: foo postpone . ; immediate \ (4)

That is equivalent to

: foo ['] . compile, ; immediate \ (5)

since "." (Dot) is an ordinary word.


By definition (1), ES(p) is to perform CS(postpone).

The problem is that in the point (2) the program performs ES(p) in
interpretation state, that means it performs CS(postpone) in
interpretation state, and it is ambiguous. A standard program is not
allowed to perform CS in interpretation state.

Nevertheless, the expectation that a system can properly handle this
case is very sound. The system knows that it's directed to performs CS
(since they were appended via POSTPONE), and thus the system can set
compilation state by themself to perform these semantics.

> In extending POSTPONE to handle
> literals one might argue it's simple to understand, less clumsy than
> POSTPONE LITERAL et al and costs little to implement.  It's something that
> could - possibly should - have been incorporated into POSTPONE from the
> beginning, had someone thought of it.  Can Standard Forth and users get by
> without it?  I imagine they can.

Of course they can. Moreover, it's possible to redefine POSTPONE to
handle literals. Moreover, the major part of the standard words can be
implemented via the rest part, so users can get by without the former part.

The question is in a criterion — what is worth standardizing, and what
isn't.



--
Ruvim

[none albert]

I invented this system to simplify things.
So { } is supposed to be used in pairs, even if it is
'{ RUN ..... '} RUN
If you don't comply, the warranty stops.

>
>Or maybe the prefix P: that
>
> P:{ P:NAME
>
>is equivalent to
>
> '{ RUN 'NAME ,

A prefix that postpones? It tries to be clever like postpone
and I'm against it. It was the cleverness (IMMEDIATE-smartness)
that I'm railing against.

>
>?
>
>
>
>> 1) Actually it is not postponed, it is run immediately.
>
>But it's running is postponed!
>
>
>--
>Ruvim

[dxforth]

On 2/05/2021 17:02, Ruvim wrote:
> ...

> Nevertheless, the expectation that a system can properly handle this
> case is very sound. The system knows that it's directed to performs CS
> (since they were appended via POSTPONE), and thus the system can set
> compilation state by themself to perform these semantics.

It may be your expectation. (2) fails on my system. I get certain
advantages continuing to use a POSTPONE that's based on COMPILE and
thus far none using POSTPONE in immediate mode - which has always
been counter-intuitive.

>
>> In extending POSTPONE to handle
>> literals one might argue it's simple to understand, less clumsy than
>> POSTPONE LITERAL et al and costs little to implement.  It's something that
>> could - possibly should - have been incorporated into POSTPONE from the
>> beginning, had someone thought of it.  Can Standard Forth and users get by
>> without it?  I imagine they can.
>
> Of course they can. Moreover, it's possible to redefine POSTPONE to
> handle literals.

Not easily I presume?

> Moreover, the major part of the standard words can be
> implemented via the rest part, so users can get by without the former part.
>
> The question is in a criterion — what is worth standardizing, and what
> isn't.

The committee decides that. You make the choice whether you can use
it or not.

[Ruvim]

On 2021-05-03 07:58, dxforth wrote:
> On 2/05/2021 17:02, Ruvim wrote:
>> ...
>> Nevertheless, the expectation that a system can properly handle this
>> case is very sound. The system knows that it's directed to performs CS
>> (since they were appended via POSTPONE), and thus the system can set
>> compilation state by themself to perform these semantics.
>
> It may be your expectation.

Yes, such behavior from a Forth system is expected by me. But not only
me. It seems to me the majority of the users have the same expectation.
And one of the reason that people have so many discussion concerning
POSTPONE is that this expectation is not met, I believe.

> (2) fails on my system.

At the moment such fail is compliant with the standard.

BTW, in the Anton's edition you system will pass (2) in this particular
case (i.e., it will meet the expectation when POSTPONE is argument of
POSTPONE). But it doesn't have much sense, since in many other cases the
system will still not meet this expectation.

> I get certain advantages continuing to use a POSTPONE that's based
> on COMPILE and thus far none using POSTPONE in immediate mode - which
> has always been counter-intuitive.

Please take a note that in (2) POSTPONE is not used in immediate mode!
But only its compilation semantics are indirectly performed. It means
that even if interpretation semantics would be specified/defined for
POSTPONE, they are not taken into account in (2), according the
mentioned expectation.

>>> In extending POSTPONE to handle
>>> literals one might argue it's simple to understand, less clumsy than
>>> POSTPONE LITERAL et al and costs little to implement.  It's something
>>> that could - possibly should - have been incorporated into POSTPONE
>>> from the beginning, had someone thought of it.  Can Standard Forth and
>>> users get by without it? I imagine they can.
>> Of course they can. Moreover, it's possible to redefine POSTPONE to
>> handle literals.
>
> Not easily I presume?

If you have recognizers for numbers — it's very easy.

Even implementing standard POSTPONE via FIND from the scratch (in a
portable manner) is possible and quite easy.

Could you test this implementation in your system: https://git.io/J30D6


It's surprising how many systems provide broken FIND.

>>
>> The question is in a criterion — what is worth standardizing, and what
>> isn't.
>
> The committee decides that.  You make the choice whether you can use
> it or not.

Well, then even if users can get by without something, it isn't an
argument to not standardize this thing.

And the committee should also have some criteria or at least grounds.


--
Ruvim

[Hugh Aguilar]

On Monday, May 3, 2021 at 1:52:11 AM UTC-7, Ruvim wrote:
> Even implementing standard POSTPONE via FIND from the scratch (in a
> portable manner) is possible and quite easy.

This is not possible in ANS-Forth without the disambiguifiers already available
because FIND is ambiguous (sometimes fails completely) for about 51 words
in ANS-Forth. This is described here:
https://groups.google.com/g/comp.lang.forth/c/T-yYkpVwYew/m/C6uvd8djAgAJ
Ironically however, disambiguifiers are defined using POSTPONE --- so you can't
define POSTPONE on the assumption that the disambiguifiers are already available.

I don't think Ruvim knows any more about ANS-Forth than Stephen Pelc does,
which is nothing.

[dxforth]

On 3/05/2021 18:52, Ruvim wrote:
> On 2021-05-03 07:58, dxforth wrote:
>
>>>> In extending POSTPONE to handle
>>>> literals one might argue it's simple to understand, less clumsy than
>>>> POSTPONE LITERAL et al and costs little to implement.  It's something
>>>> that could - possibly should - have been incorporated into POSTPONE
>>>> from the beginning, had someone thought of it.  Can Standard Forth and
>>>> users get by without it? I imagine they can.
>>> Of course they can. Moreover, it's possible to redefine POSTPONE to
>>> handle literals.
>>
>> Not easily I presume?
>
> If you have recognizers for numbers — it's very easy.
>
> Even implementing standard POSTPONE via FIND from the scratch (in a
> portable manner) is possible and quite easy.
>
> Could you test this implementation in your system: https://git.io/J30D6

Compiling your definition of POSTPONE under VFX causes it to fails
when attempting:

: OF POSTPONE OVER POSTPONE = POSTPONE IF POSTPONE DROP ; IMMEDIATE

: TEST 2 OF ." two " ELSE . THEN ;

Which is the problem I encountered attempting a 'portable' POSTPONE
based on my expectations.

[dxforth]

There's certainly limits to what one can expect from a Standard.

[Ruvim]

My implementation is only portable among the standard systems that
provide the words that are used.

FVX provides the broken (non standard compliant) "COMPILE," word.

: test-if
c" if" find dup 0= if ." unfound" 2drop exit then
1 = if ." executed" execute else ." compiled" compile, then
; immediate

] test-if [ .s

This test prints "compiled" and leave two numbers on the stack.

It's non standard. "COMPILE," shall have the stack effect ( xt -- )


--
Ruvim

[Ruvim]

Regardless of this issue, "compile," doesn't return the consistent
values during compilation "if" (as an example).

The two following definitions shall be equivalent to "if" word when this
word is encountered by the Forth text interpreter.

: if-via-eval s" if" evaluate ; immediate

: if-via-find c" if" find dup 0= -13 and throw
state @ if dup -1 = if drop compile, exit then then drop execute
; immediate

It means that in any program we can replace "if" (that is not an
argument of any other word, like "postpone") by "if-via-eval" or
"if-via-find" and the results shall be the same.

Thus the following definitions for "foo" and "bar" should produce the
same results too.

: foo if-via-eval -1 else 0 then ;
t{ 1 foo 0 foo -> -1 0 }t

: bar if-via-find -1 else 0 then ;
t{ 1 bar 0 bar -> -1 0 }t

But compilation of "bar" produces error -22 "Control structure
mismatch", since "compile," leaves an excessive value that is not
consumed by else/then.

If we remove this excessive value, the code works correctly:

: bar if-via-find [ .s cr nip ] -1 else 0 then ;
1 bar . \ prints "-1"
0 bar . \ prints "0"



--
Ruvim

[Stephen Pelc]

On Tue, 4 May 2021 14:20:20 +1000, dxforth <dxf...@gmail.com> wrote:

>Compiling your definition of POSTPONE under VFX causes it to fails
>when attempting:
>
>: OF POSTPONE OVER POSTPONE = POSTPONE IF POSTPONE DROP ; IMMEDIATE
>
>: TEST 2 OF ." two " ELSE . THEN ;

Compiles fine on both 32 bit and 64 bit versions of the current v5.11.

dis test
TEST
( 000E5F70 4883FB02 ) CMP RBX, # 02
( 000E5F74 0F851B000000 ) JNZ/NE 000E5F95
( 000E5F7A 488B5D00 ) MOV RBX, [RBP]
( 000E5F7E 488D6D08 ) LEA RBP, [RBP+08]
( 000E5F82 E8E180F3FF ) CALL 0001E068 (.") "two "
( 000E5F90 E905000000 ) JMP 000E5F9A
( 000E5F95 E8DE76F3FF ) CALL 0001D678 .
( 000E5F9A C3 ) RET/NEXT
( 43 bytes, 8 instructions )
ok


Stephen

--
Stephen Pelc, ste...@vfxforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, +44 (0)78 0390 3612, +34 649 662 974
web: http://www.mpeforth.com - free VFX Forth downloads

[Stephen Pelc]

On Tue, 4 May 2021 08:33:46 +0300, Ruvim <ruvim...@gmail.com>
wrote:

>My implementation is only portable among the standard systems that
>provide the words that are used.
>
>FVX provides the broken (non standard compliant) "COMPILE," word.
>
> : test-if
> c" if" find dup 0= if ." unfound" 2drop exit then
> 1 = if ." executed" execute else ." compiled" compile, then
> ; immediate
>
> ] test-if [ .s
>
>This test prints "compiled" and leave two numbers on the stack.
>
>It's non standard. "COMPILE," shall have the stack effect ( xt -- )

It's perfectly fine. Under VFX, IF is NDCS and non-immediate. COMPILE,
executes the compilation behaviour, which as for many versions of IF
returns an address to patch during compilation.

If you want these tests to behave correctly, you actually have
to test an IF ... THEN pair and check stack actions a bit more
carefully.

[Anton Ertl]

ste...@mpeforth.com (Stephen Pelc) writes:
>On Tue, 4 May 2021 08:33:46 +0300, Ruvim <ruvim...@gmail.com>
>wrote:

>>FVX provides the broken (non standard compliant) "COMPILE," word.
>>
>> : test-if
>> c" if" find dup 0= if ." unfound" 2drop exit then
>> 1 = if ." executed" execute else ." compiled" compile, then
>> ; immediate
>>
>> ] test-if [ .s
>>
>>This test prints "compiled" and leave two numbers on the stack.
>>
>>It's non standard. "COMPILE," shall have the stack effect ( xt -- )
>
>It's perfectly fine. Under VFX, IF is NDCS and non-immediate. COMPILE,
>executes the compilation behaviour, which as for many versions of IF
>returns an address to patch during compilation.

The standard stack effect of COMPILE, is ( xt -- ). If COMPILE,
behaves differently on your system, it's non-standard. I reported
this bug to you in 2015
<2015100907...@a4.complang.tuwien.ac.at>. I had the
impression that you would fix it in VFX 5, but given your reaction,
this is now questionable; I work at a university, so I cannot test VFX
5 myself.

[Ruvim]

On 2021-05-05 13:14, Stephen Pelc wrote:
> On Tue, 4 May 2021 08:33:46 +0300, Ruvim <ruvim...@gmail.com>
> wrote:
>
>> My implementation is only portable among the standard systems that
>> provide the words that are used.
>>
>> FVX provides the broken (non standard compliant) "COMPILE," word.
>>
>> : test-if
>> c" if" find dup 0= if ." unfound" 2drop exit then
>> 1 = if ." executed" execute else ." compiled" compile, then
>> ; immediate
>>
>> ] test-if [ .s
>>
>> This test prints "compiled" and leave two numbers on the stack.
>>
>> It's non standard. "COMPILE," shall have the stack effect ( xt -- )
>
> It's perfectly fine. Under VFX, IF is NDCS and non-immediate. COMPILE,
> executes the compilation behaviour, which as for many versions of IF
> returns an address to patch during compilation.

Perhaps it's fine for VFX, but I can't agree it's standard compliant.

| COMPILE,
| Interpretation: Interpretation semantics for this word are undefined.
| Execution: ( xt -- )
| Append the execution semantics of the definition represented
| by xt to the execution semantics of the current definition.

Appending execution semantics cannot have any other side effects, except
specified here.

"COMPILE," is not allowed to perform compilation semantics for IF.



It's similar to POSTPONE

| Compilation: ( "<spaces>name" -- )
| Skip leading space delimiters. Parse name delimited by a space.
| Find name. Append the compilation semantics of name to the
| current definition. An ambiguous condition exists if name
| is not found.

POSTPONE just appends compilation semantics, and doesn't have any other
side effects on the stacks.

> If you want these tests to behave correctly, you actually have
> to test an IF ... THEN pair and check stack actions a bit more
> carefully.

I suppose you already notice, I did that.


--
Ruvim

[dxforth]

A less problematic test might be:

: test test-if ." non-zero" then ;

This fails on VFX. But was it reasonable? I doubt it - just as
redefining POSTPONE and expecting it to work on every system wasn't
reasonable. Only a TC can declare what is standard - so ask them.

[Stephen Pelc]

On Wed, 05 May 2021 11:46:26 GMT, an...@mips.complang.tuwien.ac.at
(Anton Ertl) wrote:

>The standard stack effect of COMPILE, is ( xt -- ). If COMPILE,
>behaves differently on your system, it's non-standard. I reported
>this bug to you in 2015
><2015100907...@a4.complang.tuwien.ac.at>.

NDCS = non default compilation semantics, e.g. IF

And that's probably what triggered my NDCS esploration. The base
problem comes from the standard's definitions of words. There are
three cases:
1) normal words
2) immediate words
3) NDCS words

The standard only says how to handle cases 1) and 2). So the answer
to Ruvim's problem is "Don't do that".

Until the standard encompasses NDCS words, the standard is incomplete.
The standard deliberately does not mandate implementation techniques
and VFX uses a technique for NDCS words that is useful, but breaks
careless tests. The EuroForth papers are there to read, and nobody has
said that my analysis is wrong.

My conclusion from all this is that the standard has a bug in it.
Unfortunately, fixing the bug will be a real pain.

> I had the
>impression that you would fix it in VFX 5, but given your reaction,
>this is now questionable; I work at a university, so I cannot test VFX
>5 myself.

It is fixed. The standard is broken. You can test it, I hereby give
you and other educators an exemption for university use. The licence
will change for university educators in due course. As you well know,
you only had to ask.

[Anton Ertl]

dxforth <dxf...@gmail.com> writes:
>Only a TC can declare what is standard - so ask them.

Standardization committees declare what is standard through the
standard document. There may be cases where the standard document is
unclear or contradictory, in which case clarification may be needed,
but the stack effect or COMPILE, is not one of these cases.

[Ruvim]

On 2021-05-05 17:37, Stephen Pelc wrote:
> On Wed, 05 May 2021 11:46:26 GMT, an...@mips.complang.tuwien.ac.at
> (Anton Ertl) wrote:
>
>> The standard stack effect of COMPILE, is ( xt -- ). If COMPILE,
>> behaves differently on your system, it's non-standard. I reported
>> this bug to you in 2015
>> <2015100907...@a4.complang.tuwien.ac.at>.
>
> NDCS = non default compilation semantics, e.g. IF
>
> And that's probably what triggered my NDCS esploration. The base
> problem comes from the standard's definitions of words. There are
> three cases:
> 1) normal words
> 2) immediate words
> 3) NDCS words

It's unclear do you mean the glossary entries, or the implementations in
particular Forth system?

In which group does the EXIT word fall?


The immediate words have non default compilation semantics. Probably you
wanted to place in the group 3 the NDCS words that are not implemented
as immediate words. And then the distribution of the words per these
three groups is system/implementation dependent. In some systems the
group 3 is empty.

> The standard only says how to handle cases 1) and 2).

You are right, the standard doesn't say how to perform compilation
semantics for NDCS words that are not immediate words.

The standard also doesn't say how to perform interpretation semantics
for the words with non default interpretation semantics.

It's the internal matters of a Forth system. And a Forth system even is
not obligated to use "COMPILE," or "EXECUTE" in its Forth text
interpreter at all.

But whatever matters a system does under the hood, it should not affect
the standard interfaces.

> So the answer to Ruvim's problem is "Don't do that".

This claim requires another ground.

> Until the standard encompasses NDCS words, the standard is incomplete.

Actually the standard does provide a mechanism to implement NDCS words —
it's the mechanism of immediacy — and this mechanism is capable to
implement any non default compilation semantics, and even default
compilation semantics in some exotic cases.

You also admit this in your "Special Words in Forth" paper, with
pointing to the problems in the popular implementations.

In the same time a system is allowed to use any other suitable mechanism
for that as well as the standard immediacy mechanism.

> The standard deliberately does not mandate implementation techniques
> and VFX uses a technique for NDCS words that is useful, but breaks
> careless tests.

Nothing internal implementation details can excuse a system in breaking
the specified interface of the "COMPILE," word.

> The EuroForth papers are there to read, and nobody has
> said that my analysis is wrong.

https://www.mpeforth.com/arena/SpecialWords3.pdf

It says that "COMPILE," may do optimization. It's correct.

Also it says that "COMPILE," can be used to perform compilation
semantics for the words like "IF", but then "COMPILE," will be broken:

| This technique can also be used for other words such as "IF",
| with the deliberate intention that the interpretation
| and compilation actions of a word can be separated.
| However, "COMPILE," is then broken as far as current standards
| are concerned because structure words such as "IF" produce
| or consume stack items, and string words parse the input stream.

It's also correct.

Even "IF" may be implemented as an ordinary word. The only requirement
is that the unspecified side effects of its compilation via "COMPILE,"
should be undetectable by a standard program. For example, a system can
use special internal stack for "orig". Or it can resolve "orig" in the
run-time.

> My conclusion from all this is that the standard has a bug in it.

Hope I can convince you that there mentioned issue is not a bug, but
something like misunderstanding.

> Unfortunately, fixing the bug will be a real pain.

A bug should be provable by an example.



--
Ruvim

[dxforth]

On 6/05/2021 00:37, Anton Ertl wrote:
> dxforth <dxf...@gmail.com> writes:
>>Only a TC can declare what is standard - so ask them.
>
> Standardization committees declare what is standard through the
> standard document. There may be cases where the standard document is
> unclear or contradictory, in which case clarification may be needed,
> but the stack effect or COMPILE, is not one of these cases.

The behaviour of test-if was ambiguous before it got to COMPILE, .
IMO once an ambiguous condition is in process, ANS has no say what
should happen.

[Ruvim]

On 2021-05-06 04:49, dxforth wrote:
> On 6/05/2021 00:37, Anton Ertl wrote:
>> dxforth <dxf...@gmail.com> writes:
>>> Only a TC can declare what is standard - so ask them.
>>
>> Standardization committees declare what is standard through the
>> standard document.  There may be cases where the standard document is
>> unclear or contradictory, in which case clarification may be needed,
>> but the stack effect or COMPILE, is not one of these cases.
>
> The behaviour of test-if was ambiguous before it got to COMPILE, .

Here it is:

: test-if
c" if" find dup 0= if ." unfound" 2drop exit then
1 = if ." executed" execute else ." compiled" compile, then
; immediate

In what place is it ambiguous? In "find"?

> IMO once an ambiguous condition is in process, ANS has no say what
> should happen.

Correct.

"The response of a Standard System to an ambiguous condition is left to
the discretion of the implementor" (A.2.1)


--
Ruvim

[dxforth]

On 6/05/2021 16:19, Ruvim wrote:
> On 2021-05-06 04:49, dxforth wrote:
>> On 6/05/2021 00:37, Anton Ertl wrote:
>>> dxforth <dxf...@gmail.com> writes:
>>>> Only a TC can declare what is standard - so ask them.
>>>
>>> Standardization committees declare what is standard through the
>>> standard document.  There may be cases where the standard document is
>>> unclear or contradictory, in which case clarification may be needed,
>>> but the stack effect or COMPILE, is not one of these cases.
>>
>> The behaviour of test-if was ambiguous before it got to COMPILE, .
>
> Here it is:
>
> : test-if
> c" if" find dup 0= if ." unfound" 2drop exit then
> 1 = if ." executed" execute else ." compiled" compile, then
> ; immediate
>
>
> In what place is it ambiguous? In "find"?

Using FIND to invoke IF is ambiguous. I've a good idea what will
happen on single-xt systems but that's all. Same for POSTPONE S" .
Dual-xt systems have muddied the waters. Today users have no way
to know what is portable - only what works on their system.

[P Falth]

On Thursday, 6 May 2021 at 10:52:00 UTC+2, dxforth wrote:
> On 6/05/2021 16:19, Ruvim wrote:
> > On 2021-05-06 04:49, dxforth wrote:
> >> On 6/05/2021 00:37, Anton Ertl wrote:
> >>> dxforth <dxf...@gmail.com> writes:
> >>>> Only a TC can declare what is standard - so ask them.
> >>>
> >>> Standardization committees declare what is standard through the
> >>> standard document. There may be cases where the standard document is
> >>> unclear or contradictory, in which case clarification may be needed,
> >>> but the stack effect or COMPILE, is not one of these cases.
> >>
> >> The behaviour of test-if was ambiguous before it got to COMPILE, .
> >
> > Here it is:
> >
> > : test-if
> > c" if" find dup 0= if ." unfound" 2drop exit then
> > 1 = if ." executed" execute else ." compiled" compile, then
> > ; immediate
> >
> >
> > In what place is it ambiguous? In "find"?
> Using FIND to invoke IF is ambiguous. I've a good idea what will
> happen on single-xt systems but that's all. Same for POSTPONE S" .
> Dual-xt systems have muddied the waters. Today users have no way
> to know what is portable - only what works on their system.

Yes I agree to this. My LXF is dual-xt. FIND on IF will return the interpretation
xt. When executed this will print an error message and abort.

In my opinion FIND was left in pre F94 state and not modified to align with
the execution, interpretation and compilation semantics definitions.

Of course you can get my IF to work with FIND by doing
: IF postpone IF ; immediate

As an alternative to FIND I can suggest
FIND-INTERPRET ( addr u -- addr u 0 | xt2 xt1 -1 )
FIND-COMPILE ( addr u -- addr u 0 | xt2 xt1 -1 )
FIND-POSTPONE ( addr u -- addr u 0 | xt2 xt1 -1 )

The result if found just needs to be executed to perform the respective actions
It also hides the implementation details ( single-xt, dual-xt, dual headers etc)

BR
Peter

[Stephen Pelc]

On Wed, 5 May 2021 21:08:28 +0300, Ruvim <ruvim...@gmail.com>
wrote:

>> NDCS = non default compilation semantics, e.g. IF
>>
>> And that's probably what triggered my NDCS esploration. The base
>> problem comes from the standard's definitions of words. There are
>> three cases:
>> 1) normal words
>> 2) immediate words
>> 3) NDCS words
>
>It's unclear do you mean the glossary entries, or the implementations in
>particular Forth system?
>
>In which group does the EXIT word fall?

Let's restate the groups:
a) normal words,
b) NDCS words,
c) Immediate words are a subset of NDCS words.

In most Forth systems, groups b) and c) are the same because the
system only has IMMEDIATE as a way to express NDCS behaviour.

In VFX, some special words are NDCS but not IMMEDIATE. The equivalent
of COMPILE, for such words is NDCS,

: ndcs, \ ??? xt -- ???
\ *G Perform the compilation action of an NDCS word. This may have
\ ** a stack effect or parse the input stream.

: compile-word \ i*x xt -- j*x
\ *G Process an XT for compilation. May parse or have stack effect.
dup ndcs? \ NDCS or normal
if ndcs, else compile, then
;

In VFX, most uses of COMPILE, should use COMPILE-WORD instead.
The alternative is to redefine COMPILE, to take note of NDCS words
but ... politics, politic, politics.

[Ruvim]

On 2021-05-06 14:38, P Falth wrote:
> On Thursday, 6 May 2021 at 10:52:00 UTC+2, dxforth wrote:
>> On 6/05/2021 16:19, Ruvim wrote:
>>> On 2021-05-06 04:49, dxforth wrote:
>>>> On 6/05/2021 00:37, Anton Ertl wrote:
>>>>> dxforth <dxf...@gmail.com> writes:
>>>>>> Only a TC can declare what is standard - so ask them.
>>>>>
>>>>> Standardization committees declare what is standard through the
>>>>> standard document. There may be cases where the standard document is
>>>>> unclear or contradictory, in which case clarification may be needed,
>>>>> but the stack effect or COMPILE, is not one of these cases.
>>>>
>>>> The behaviour of test-if was ambiguous before it got to COMPILE, .
>>>
>>> Here it is:
>>>
>>> : test-if
>>> c" if" find dup 0= if ." unfound" 2drop exit then
>>> 1 = if ." executed" execute else ." compiled" compile, then
>>> ; immediate
>>>
>>>
>>> In what place is it ambiguous? In "find"?
>> Using FIND to invoke IF is ambiguous. I've a good idea what will
>> happen on single-xt systems but that's all. Same for POSTPONE S" .
>> Dual-xt systems have muddied the waters. Today users have no way
>> to know what is portable - only what works on their system.
>
> Yes I agree to this. My LXF is dual-xt. FIND on IF will return the interpretation
> xt. When executed this will print an error message and abort.

It's OK for interpretation state, but not OK for compilation state.

In dual-xt systems FIND should depend on STATE and return the different
results depending on STATE for some words.

My clarification for FIND (the next version)
https://forth-standard.org/proposals/clarify-find-more-classic-approach?hideDiff#reply-682

>
> In my opinion FIND was left in pre F94 state and not modified to align with
> the execution, interpretation and compilation semantics definitions.

It's specification is too lean. But the Rationale part shows the
intention clearly. And using this rationale we can make the
specification better.

>
> Of course you can get my IF to work with FIND by doing
> : IF postpone IF ; immediate
>
> As an alternative to FIND I can suggest
> FIND-INTERPRET ( addr u -- addr u 0 | xt2 xt1 -1 )
> FIND-COMPILE ( addr u -- addr u 0 | xt2 xt1 -1 )
> FIND-POSTPONE ( addr u -- addr u 0 | xt2 xt1 -1 )

If you can implement this, nothing prevent you to implement STATE
dependent FIND

But what for do you need FIND-POSTPONE ?

> The result if found just needs to be executed to perform the respective actions
> It also hides the implementation details ( single-xt, dual-xt, dual headers etc)

It doesn't cover sing-xt systems. In these systems the corresponding
action should be performed only in the corresponding STATE.


--
Ruvim

[Ruvim]

On 2021-05-06 17:07, Ruvim wrote:
> On 2021-05-06 14:38, P Falth wrote:

[...]

>> As an alternative to FIND I can suggest
>> FIND-INTERPRET  ( addr u -- addr u 0 | xt2 xt1 -1 )
>> FIND-COMPILE     ( addr u -- addr u 0 | xt2 xt1 -1 )
>> FIND-POSTPONE  ( addr u -- addr u 0 | xt2 xt1 -1 )

They are quite close to FIND-NAME NAME>COMPILE

>
> If you can implement this, nothing prevent you to implement STATE
> dependent FIND

But only if for the cases FIND-INTEPRET and FIND-COMPILE, xt1 identify
either "EXECUTE" or "COMPILE," and nothing else.

Otherwise FIND cannot be implemented via these words.

>
> But what for do you need FIND-POSTPONE ?
>
>
>> The result if found just needs to be executed to perform the
>> respective actions
>> It also hides the implementation details ( single-xt, dual-xt, dual
>> headers etc)
>
> It doesn't cover sing-xt systems. In these systems the corresponding
> action should be performed only in the corresponding STATE.

I missed that they return two xt-s. Then yes, they cover single-xt
systems too.

But then what is a way to detect whether a word is an ordinary word?

And what is a way to get xt of an ordinary word, that identifies the
execution semantics for this word?


--
Ruvim

[P Falth]

The standard says that FIND may return different XTs depending on state.
It is not obliged to do so

> My clarification for FIND (the next version)
> https://forth-standard.org/proposals/clarify-find-more-classic-approach?hideDiff#reply-682

I have seen your efforts.
In my opinion it is better to start with a clean sheet and implement something
that can substitute FIND and make it obsolete.

Peter

[P Falth]

On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:
> On 2021-05-06 17:07, Ruvim wrote:
> > On 2021-05-06 14:38, P Falth wrote:
> [...]
> >> As an alternative to FIND I can suggest
> >> FIND-INTERPRET ( addr u -- addr u 0 | xt2 xt1 -1 )
> >> FIND-COMPILE ( addr u -- addr u 0 | xt2 xt1 -1 )
> >> FIND-POSTPONE ( addr u -- addr u 0 | xt2 xt1 -1 )
> They are quite close to FIND-NAME NAME>COMPILE

Yes they are. But they also hide the NT and can thus accomodate
also a system with dual NTs. Someone (Mark Williams?) described
such a system that was simple and clean.

> > If you can implement this, nothing prevent you to implement STATE
> > dependent FIND
> But only if for the cases FIND-INTEPRET and FIND-COMPILE, xt1 identify
> either "EXECUTE" or "COMPILE," and nothing else.

To require that XT1 be limited to "EXECUTE" or "COMPILE," would severely
limit the possibilities of the system. Take for example a constant.
XT2 could be the actual value of the constant and XT1 LITERAL for compiling.

My LXF64 compiles to tokenized code. XT1 of interpret could there be a call
to a JIT compiler that compiles the tokens to native code and then executes it

> Otherwise FIND cannot be implemented via these words.

There is no requirements to implement FIND with these words.

> >
> > But what for do you need FIND-POSTPONE ?
> >
> >
> >> The result if found just needs to be executed to perform the
> >> respective actions
> >> It also hides the implementation details ( single-xt, dual-xt, dual
> >> headers etc)
> >
> > It doesn't cover sing-xt systems. In these systems the corresponding
> > action should be performed only in the corresponding STATE.
> I missed that they return two xt-s. Then yes, they cover single-xt
> systems too.
>
> But then what is a way to detect whether a word is an ordinary word?

That is not needed!

>
> And what is a way to get xt of an ordinary word, that identifies the
> execution semantics for this word?

' and ['] are still available for this.

BR
Peter
>
>
> --
> Ruvim

[Anton Ertl]

P Falth <peter....@gmail.com> writes:
>On Thursday, 6 May 2021 at 16:07:26 UTC+2, Ruvim wrote:

[...]

>The standard says that FIND may return different XTs depending on state.
>It is not obliged to do so

True. The intention behind that is probably that classic
single-xt+immediate-flag systems return the same xt in either state
and cmForth returns, for some words, one xt in one state, and a
different xt in the other state.

Sure, someone who implements a cmForth-like implementation could say
that there is no requirement in the specification to do it that way,
and produce a find that only returns one of the xts, but that would
mean that user-defined text interpreters that use FIND do not work on
that system.

Likewise for a more modern dual-xt system like yours.

>> My clarification for FIND (the next version)
>> https://forth-standard.org/proposals/clarify-find-more-classic-approach?hideDiff#reply-682

Or another one:

https://forth-standard.org/proposals/clarify-find?hideDiff#reply-165

>I have seen your efforts.
>In my opinion it is better to start with a clean sheet and implement something
>that can substitute FIND and make it obsolete.

That has already been accepted:

https://forth-standard.org/proposals/find-name?hideDiff#reply-174

But FIND will continue to be in the standard for at least one more
release (or maybe more, it has not been declared obsolescent, and
nobody has proposed making it obsolescent, not even those who claim we
should not fix FIND, because it will be replaced).

Moreover, there are people who make an outrageous claims based on the
current specification of FIND (in particular, IIRC Stephen Pelc made
such a claim based on the usage of "immediate" in the current
specification that is at odds with the usage of "immediate" in the
rest of the document). So while FIND is still in the standard, it
will continue to be a sore point. That can be fixed by clarifying it.

[Anton Ertl]

P Falth <peter....@gmail.com> writes:
>To require that XT1 be limited to "EXECUTE" or "COMPILE," would severely
>limit the possibilities of the system. Take for example a constant.
>XT2 could be the actual value of the constant and XT1 LITERAL for compiling.

Yes, I also played with that idea. But it does not scale to doubles
and floats.

>My LXF64 compiles to tokenized code. XT1 of interpret could there be a call
>to a JIT compiler that compiles the tokens to native code and then executes it

If you implement EXECUTE, why go there?

Another idea I played with: for the compilation token xt1 xt2, let xt2
be a specialized compiler for xt1. I think you can find this idea in
my early writings about the compilation token. But we also have
COMPILE, and need to implement that. So the better way is to dispatch
the specialized compilers from "COMPILE,", and that's what we
implement now:

NAME>COMPILE always produces ' EXECUTE or ' COMPILE, as xt2. This
makes it straightforward to define FIND such that user-defined text
interpreters work. One can use SET-OPTIMIZER to tell the system what
COMPILE, should do for a certain word.

[P Falth]

On Thursday, 6 May 2021 at 19:30:49 UTC+2, Anton Ertl wrote:
> P Falth <peter....@gmail.com> writes:
> >To require that XT1 be limited to "EXECUTE" or "COMPILE," would severely
> >limit the possibilities of the system. Take for example a constant.
> >XT2 could be the actual value of the constant and XT1 LITERAL for compiling.
> Yes, I also played with that idea. But it does not scale to doubles
> and floats.

In fact my actual implementation is more compilated, XT2 is an address where
the constant, double or float is stored. XT1 is the appropriate fetch followed
by the appropriate literal

> >My LXF64 compiles to tokenized code. XT1 of interpret could there be a call
> >to a JIT compiler that compiles the tokens to native code and then executes it
> If you implement EXECUTE, why go there?

This is one area I have not decided yet. It could be that I will just execute
the tokenized code when interpreting. At least this would work well when the
tokenized code will be inlined while compiling. No need to waste space
compiling native code for it.

On the other hand words that for different reasons can not be inlined will need
to be compiled to native code so they later can be called directly from other
native code. In this case there is no need to keep the tokencode.

> Another idea I played with: for the compilation token xt1 xt2, let xt2
> be a specialized compiler for xt1. I think you can find this idea in
> my early writings about the compilation token. But we also have
> COMPILE, and need to implement that. So the better way is to dispatch
> the specialized compilers from "COMPILE,", and that's what we
> implement now:

My COMPILE, in both lxf and lxf64 are dumb. They will just compile a call
to the XT

> NAME>COMPILE always produces ' EXECUTE or ' COMPILE, as xt2. This
> makes it straightforward to define FIND such that user-defined text
> interpreters work. One can use SET-OPTIMIZER to tell the system what
> COMPILE, should do for a certain word.

None of my systems produces ' EXECUTE or ' COMPILE, as xt2.
I can see no requirement for limiting NAME>COMPILE in that way.

BR
Peter

[Stephen Pelc]

On Thu, 06 May 2021 16:26:01 GMT, an...@mips.complang.tuwien.ac.at
(Anton Ertl) wrote:

>Moreover, there are people who make an outrageous claims based on the
>current specification of FIND (in particular, IIRC Stephen Pelc made

>such a claim based on ...

I am proud to be thought to make outrageous claims, especially
when they are correct.

In particular, until we (standard committee and other interested
parties) come to terms with the impact of NDCS, arguing about
FIND and COMPILE, is just another silly discussion about how
many angels can dance on the head of a pin.

The easy thing to do is to declare non-immediate NDCS words
as non-compliant. Alternatively, you open the box and enable
new standards-compliant notations such as interpretive
BEGIN ... UNTIL and DO ... LOOP.

You have to start by understanding that the standards from
ANS-94 onwards have bugs. This is not to put the authors down,
but to realise that we are all imperfect.

[Ruvim]

Taking into account that "ndcs," just performs another xt:

: (ndcs,) \ i*x xt -- j*x
>code-len @ execute \ execute NDCS/IMMEDIATE word
;

Why not return this xt by "find" ?


: find ( c-addr -- c-addr 0 | xt -1 | xt 1 )
find dup 0= if exit then drop ( xt1 )
dup ndcs? if >code-len @ 1 else
dup immediate? if 1 else -1 then then ( xt1 xt2 n )
state @ if rot else swap then drop
;


After this fix all my tests work as expected.

To be on the safe side, I would redefine/fix some other words too, to
exclude situations when "compile," produces unexpected stack effect when
it's applied to some xt.

Nothing standard word should return such xt for which "compile,"
produces unexpected stack effect.



--
Ruvim

[Ruvim]

A typo correction:

[Ruvim]

On 2021-05-06 18:09, P Falth wrote:
> On Thursday, 6 May 2021 at 16:07:26 UTC+2, Ruvim wrote:
>> On 2021-05-06 14:38, P Falth wrote:
>>> On Thursday, 6 May 2021 at 10:52:00 UTC+2, dxforth wrote:
>>>> On 6/05/2021 16:19, Ruvim wrote:
>>>>> On 2021-05-06 04:49, dxforth wrote:
>>>>>>
>>>>>> The behaviour of test-if was ambiguous before it got to COMPILE, .
>>>>>
>>>>> Here it is:
>>>>>
>>>>> : test-if
>>>>> c" if" find dup 0= if ." unfound" 2drop exit then
>>>>> 1 = if ." executed" execute else ." compiled" compile, then
>>>>> ; immediate
>>>>>
>>>>>
>>>>> In what place is it ambiguous? In "find"?
>>>> Using FIND to invoke IF is ambiguous. I've a good idea what will
>>>> happen on single-xt systems but that's all. Same for POSTPONE S" .
>>>> Dual-xt systems have muddied the waters. Today users have no way
>>>> to know what is portable - only what works on their system.
>>>
>>> Yes I agree to this. My LXF is dual-xt. FIND on IF will return the interpretation
>>> xt. When executed this will print an error message and abort.
>> It's OK for interpretation state, but not OK for compilation state.
>>
>> In dual-xt systems FIND should depend on STATE and return the different
>> results depending on STATE for some words.
>
> The standard says that FIND may return different XTs depending on state.
> It is not obliged to do so

Yes, certainly. I mean, to make a user-defined text interpreter
implementable via FIND (as in "test-if" above), FIND should depend on
STATE in a dual-xt system.


It should be noted, among dual-xt and single-xt systems, only some
dual-xt systems provide FIND that is not capable to implement
user-defined text interpreter.

>> My clarification for FIND (the next version)
>> https://forth-standard.org/proposals/clarify-find-more-classic-approach?hideDiff#reply-682
>
> I have seen your efforts.
> In my opinion it is better to start with a clean sheet and implement something
> that can substitute FIND and make it obsolete.

Even when a consensus concerning this replacement will be reached, FIND
will remain a method of fallback in many years.

So FIND should be clarified regardless replacements.



--
Ruvim

[Anton Ertl]

ste...@mpeforth.com (Stephen Pelc) writes:
>On Wed, 05 May 2021 11:46:26 GMT, an...@mips.complang.tuwien.ac.at
>(Anton Ertl) wrote:

[...]

>And that's probably what triggered my NDCS esploration. The base
>problem comes from the standard's definitions of words. There are
>three cases:
>1) normal words
>2) immediate words
>3) NDCS words
>
>The standard only says how to handle cases 1) and 2).

Really? When I look at the standard, I find, e.g.

|3.4 The Forth text interpreter
|...
|
|b. Search the dictionary name space (see 3.4.2). If a definition name
|matching the string is found:
|
| 1. if interpreting, perform the interpretation semantics of the
| definition (see 3.4.3.2), and continue at a).
|
| 2. if compiling, perform the compilation semantics of the definition
| (see 3.4.3.3), and continue at a).

So here the standard does not talk about normal words, immediate
words, or NDCS words, but instead about the interpretation semantics
and compilation semantics of all words.

>The standard deliberately does not mandate implementation techniques
>and VFX uses a technique for NDCS words that is useful, but breaks
>careless tests.

The standard indeed does not mandate implementation techniques, but
instead specifies behaviour. Standard systems have to comply with
this specification; they can choose various implementation techniques
to achieve this, but if an implementation does not comply with the
specification, it is non-standard, and you can wave the "does not
mandate implementation techniques" card as much as you want, it won't
change that.

If a test shows that your system is non-standard, it's not the fault
of the test, but the fault of your system.

>The EuroForth papers are there to read, and nobody has
>said that my analysis is wrong.

I told you in <597E1567.25...@stephen.mpeforth.com> what was
wrong in the draft version of your 2017 paper. Looking at 4 places
where I pointed out that what you wrote is wrong, and comparing that
to the final paper, I see that in three places you kept the wrong
statements. You ammended the fourth place, but the result is still
wrong.

So: I wrote to you that your analysis is wrong before you published
the paper, and I maintain that it is still wrong.

As for complexity, we used to use an approach that uses IFs for
separating various cases (aliases, interpret/compile:, intelligent
"compile,", words that you can TO to); these exceptions were easy to
add step-by-step, but they resulted in a complex mess (in a way,
similar to a text interpreter with five (or was it seven)
special-purpose hooks, one for each extension that had become
necessary over the years), and they are inflexible. So Bernd Paysan
redesigned the Gforth header (with some input from me), and this
resulted in a much more flexible header design that we described in
detail in [paysan19].

@InProceedings{paysan19,
author = {Bernd Paysan and M. Anton Ertl},
title = {The new {Gforth} Header},
crossref = {euroforth19},
pages = {5--20},
url = {http://www.euroforth.org/ef19/papers/paysan.pdf},
url-slides = {http://www.euroforth.org/ef19/papers/paysan-slides.pdf},
video = {https://wiki.forth-ev.de/doku.php/events:ef2019:header},
OPTnote = {refereed},
abstract = {The new Gforth header is designed to directly
implement the requirements of Forth-94 and
Forth-2012. Every header is an object with a fixed
set of fields (code, parameter, count, name, link)
and methods (\texttt{execute}, \texttt{compile,},
\texttt{(to)}, \texttt{defer@}, \texttt{does},
\texttt{name>interpret}, \texttt{name>compile},
\texttt{name>string}, \texttt{name>link}). The
implementation of each method can be changed
per-word (prototype-based object-oriented
programming). We demonstrate how to use these
features to implement optimization of constants,
\texttt{fvalue}, \texttt{defer}, \texttt{immediate},
\texttt{to} and other dual-semantics words, and
\texttt{synonym}.}

}

>My conclusion from all this is that the standard has a bug in it.
>Unfortunately, fixing the bug will be a real pain.

I think the standard is mostly fine in this area. There are proposals
for specifying FIND in a way that would make user-defined text
interpreters standard
<https://forth-standard.org/proposals/clarify-find?hideDiff#reply-165>
<https://forth-standard.org/proposals/clarify-find-more-classic-approach?hideDiff#reply-682>
and for making the definition of the term "execution token" match it's
usage in FIND, ', ['], NAME>INTERPRET, NAME>COMPILE
<https://forth-standard.org/proposals/reword-the-term-execution-token-?hideDiff#reply-486>.

Fixing these issues is therefore as painless as accepting one of the
Clarify FIND proposals, and accepting the rewording of execution
token.

There is also a case for disallowing ticking and POSTPONEing S" and
S\" (to allow STATE-smart implementations of these words), but I leave
it to those who are interested in such implementations to make such
proposals.

>> I had the
>>impression that you would fix it in VFX 5, but given your reaction,
>>this is now questionable; I work at a university, so I cannot test VFX
>>5 myself.
>
>It is fixed.

Good! I have now tested the test case I sent you in 2015 and VFX 5.11 passed.

Let's see about Ruvim's test from <s6qmbs$ki6$1...@dont-email.me>:

: test-if
c" if" find dup 0= if ." unfound" 2drop exit then
1 = if ." executed" execute else ." compiled" compile, then
; immediate

] test-if [ .s

This fails on VFX 5.11. My guess is that you did not fix your broken
COMPILE, but instead rewrote your text interpreter such that my 2015
test does not invoke COMPILE,. So let's combine my scaffolded
COMPILE, from the 2015 test with Ruvim's test:

: defers
\ "defers X" produces a call to the current content of the deferred word X
' defer@ compile, ; immediate

:noname ( xt -- )
>in @ >r
depth 1- >r
defers compile,
depth r> <> abort" compile, stack effect wrong"
>in @ r> <> abort" compile, input stream effect wrong" ;
is compile,

: test-if
c" if" find dup 0= if ." unfound" 2drop exit then
1 = if ." executed" execute else ." compiled" compile, then
; immediate

] test-if [ .s

The last line outputs:

|compiledcompile, stack effect wrong

(why would it say "compiled" and not execute the .s after the [ ?)


Let's try a simple user-defined text interpreter based on the one
shown in
<https://forth-standard.org/standard/rationale#rat:core:COMPILE,>

: int-line ( ... "rest of line" -- ... )
cr source type cr
begin
bl word dup c@ while
FIND ?DUP IF ( xt +-1 )
STATE @ IF ( xt +-1 )
0> IF EXECUTE ELSE COMPILE, THEN ( ??? )
ELSE ( xt +-1 )
DROP EXECUTE ( ??? )
THEN
ELSE ( c-addr )
-13 throw \ numbers are left as an exercise
THEN
repeat
drop ;

[upper case stuff from
<https://forth-standard.org/standard/rationale#rat:core:COMPILE,>]

Note that Forth-94/2012 does not guarantee that this text-interprets,
e.g., IF (not even in compile state), because FINDing IF is ambiguous.
Anyway, let's see how various systems fare, on the following input:

int-line state @ .
int-line : foo if dup . then ;
int-line : bar s" bla" ;
int-line state @ true foo drop bar type

The outputs are, for various Forth systems:

SwiftForth 3.11.0, Gforth 0.7.9_20201015, iforth, and vfxlin-4.71 print:
int-line state @ .
0
int-line : foo if dup . then ;

int-line : bar s" bla" ;

int-line state @ true foo bar type
0 bla

vfx64 5.11 prints:
int-line state @ .
0
int-line : foo if dup . then ;

Err# -22 ERR: Control structure mismatch.
Source: "forth/text-interpreter.4th" on line 36
-> int-line : foo if dup . then ;
^sh: geany: command not found

ERROR on command line

The VFX 5.11 result was unexpected. VFX 4.71 works here because this
text interpreter does not keep any text-interpreter data on the data
stack during the call to COMPILE, (except of course the xt itself).
But let's change the user-defined text interpreter in a way that
should not change it's behaviour:

: int-line ( ... "rest of line" -- ... )
cr source type cr
begin
bl word dup c@ while
FIND ?DUP IF ( xt +-1 )
STATE @ IF ( xt +-1 )
0> IF EXECUTE ELSE >in @ swap COMPILE, >in ! THEN ( ??? )
ELSE ( xt +-1 )
DROP EXECUTE ( ??? )
THEN
ELSE ( c-addr )
-13 throw \ numbers are left as an exercise
THEN
repeat
drop ;

This saves >IN on the stack across the COMPILE,. Given that COMPILE,
is specified to only access the xt and not change >IN (by omitting any
mention of such an effect), this is equivalent to the INT-LINE above.
Let's see how the systems fare:

SwiftForth 3.11.0, Gforth 0.7.9_20201015 and iforth give the same
output as above (as they should).

vfxlin-4.71 now gives:
int-line state @ .
0
int-line : foo if dup . then ;

CS=0023 DS=002B ES=002B SS=002B FS=0000 GS=0063
EAX=6E65:683B EBX=110C:CBE8 ECX=F7F3:F626 EDX=0000:0020
ESI=08FF:9000 EDI=0804:A000 EBP=08FF:7FB0 ESP=08FE:7F1C
EIP=0804:B836 EFLAGS=0001:0206
--- RS top ---
$0805:7002 PARSE-WORD
$0000:0020
$0805:70E5 WORD
$080C:0938 INT-LINE
$0805:9C8B DEFER
$0805:97F3 (INTERPRET)
$0805:9C8B DEFER
$0805:EE90 ((INCLUDE-FILE))
Signal number SIGSEGV
at address 0804:B836, probably in SKIP

and VFX 5.11 produces similar output.

As mentioned, you can say that FINDing IF is ambiguous, so VFX is fine
as far as this test is concerned. Or alternatively, you can design
your system such that code like this works.

>You can test it, I hereby give
>you and other educators an exemption for university use.

Thank you.

[Ruvim]

On 2021-05-08 13:34, Anton Ertl wrote:
> ste...@mpeforth.com (Stephen Pelc) writes:
>> On Wed, 05 May 2021 11:46:26 GMT, an...@mips.complang.tuwien.ac.at
>> (Anton Ertl) wrote:
> [...]
>> And that's probably what triggered my NDCS esploration. The base
>> problem comes from the standard's definitions of words. There are
>> three cases:
>> 1) normal words
>> 2) immediate words
>> 3) NDCS words
>>
>> The standard only says how to handle cases 1) and 2).
>
> Really? When I look at the standard, I find, e.g.
>
> |3.4 The Forth text interpreter
> |...
> |
> |b. Search the dictionary name space (see 3.4.2). If a definition name
> |matching the string is found:
> |
> | 1. if interpreting, perform the interpretation semantics of the
> | definition (see 3.4.3.2), and continue at a).
> |
> | 2. if compiling, perform the compilation semantics of the definition
> | (see 3.4.3.3), and continue at a).
>
> So here the standard does not talk about normal words, immediate
> words, or NDCS words, but instead about the interpretation semantics
> and compilation semantics of all words.

As I understand, the Stephen's point was that the standard describes how
to perform default compilation semantics (that's to append ES), and how
to perform compilation semantics for immediate words (that's to perform
ES), but doesn't describe how to perform compilation semantics for other
words.

Actually, it's a fictional problem. The standard just considers and
describes behavior of a system on some level of abstraction, and doesn't
consider by what means this behavior is accomplished, and how the system
behaves on the underlying levels.

I.g., it also doesn't say how compilation semantics are appended to the
current definition.

If the standard doesn't say how to accomplish some results or behavior,
then it's up to an implementer how to to it, and the implementer is free
to use any suitable means.


--
Ruvim

[Marcel Hendrix]

On Saturday, May 8, 2021 at 3:10:49 PM UTC+2, Anton Ertl wrote:
[..]

> So let's combine my scaffolded
> COMPILE, from the 2015 test with Ruvim's test:

I have to dig through a lot of sneaky text (test of a test in 2015?)
to even approach what you have done here.
Maybe just insert it explicitly?

> : defers
> \ "defers X" produces a call to the current content of the deferred word X
> ' defer@ compile, ; immediate

I am not fond of these discussions but must point out that DEFER@
only works for words created by DEFER , a word in the CORE-EXT set.
It is therefore unlikely that DEFER@ is going to work on non-user-defined
words like COMPILE,. Or is there a guarantee now that existence
of DEFER means that all deferred words are created by DEFER ?

> :noname ( xt -- )
> >in @ >r
> depth 1- >r
> defers compile,

Here you assume COMPILE, is a deferred word. It would really help
to show the complete example.

> depth r> <> abort" compile, stack effect wrong"
> >in @ r> <> abort" compile, input stream effect wrong" ;
> is compile,

Again assuming COMPILE, is DEFERred, consisted with the above.

I couldn't (but wanted to) read past the above line.

-marcel

[Ruvim]

Nevertheless, we should admit that the standard better describes
single-xt systems than dual-nt systems (or dual-xt systems) in this regard.

A single-xt system doesn't provide "other words" at all, so a
program/user knows how to perform CS for any word. But due to FIND is
insufficiently specified, a user formally doesn't know how to perform CS
for other words in a dual-nt system (or dual-xt system). Certainly, the
system knows that, but a user doesn't know.

One sound expectation is that a user-defined text interpreter from a
single-xt system should work without changes in a dual-nt system. And
from this assumption a user can conclude how to perform CS for other
words in a dual-nt system (but conclusions concerning immediacy, if any,
will be wrong).


A problem is that this expectation is based not on the normative part,
but on the rationales. The purpose of clarification for FIND is to make
the normative part a formal basis for this expectation.

[Anton Ertl]

I don't know if this was his point, but anyway, the standard certainly
specifies the compilation semantics of all words:

|3.4.3.3 Compilation semantics
|
|Unless otherwise specified in a "Compilation:" section of the glossary
|entry, the compilation semantics of a Forth definition shall be to
|append its execution semantics to the execution semantics of the
|current definition.

So either a word specification has a "Compilation:" section, and that
gives its compilation semantics (then this is an "other word"), or its
compilation semantics is the default compilation semantics specified
in the last half-sentence of 3.4.3.3 (for both standard-specified and
user-defined words), or (for user-defined immediate words) its
compilation semantics is the execution semantics as per 2.1:

|immediate word:
|
|A Forth word whose compilation semantics are to perform its execution
|semantics.

[Anton Ertl]

dxforth <dxf...@gmail.com> writes:
>I get certain
>advantages continuing to use a POSTPONE that's based on COMPILE and
>thus far none using POSTPONE in immediate mode - which has always
>been counter-intuitive.

The way you use COMPILE in POSTPONE is compatible with POSTPONEing in
interpret state:

: POSTPONE
?comp bl word find ?dup if
0< if compile compile then
else number then compile, ; immediate

You do not rely on the text interpreter to "," the xt of the postponed
word; instead POSTPONE ","s the xt with COMPILE,. Of course, your
STATE-smart NUMBER remains a problem, but
<2021Apr2...@mips.complang.tuwien.ac.at> showed how to fix that.

Your use of COMPILE, raises the question of whether this will work in,
say, a native-code system. It will not, but if we assume that the
system puts code and data in the same section, a slight variation
could work; I'll start with the variant from
<2021Apr2...@mips.complang.tuwien.ac.at>:

: POSTPONE
bl word find ?dup if
0< if compile compile then
else number dup >r execute r> then compile, ; immediate

First, we have to define COMPILE and we define it in the almost
classic way:

: compile ( -- ) r> dup @ compile, cell+ >r ;

[inlining must be prevented for this word]

Note that this COMPILE expects the xt (not native code) of the word to
be compiled in the next cell, but then compiles native code with
COMPILE,. So instead of COMPILE COMPILE we have to write something
different:

: POSTPONE
bl word find ?dup if
0< if compile [ ' compile , ] , exit then
else number dup >r execute r> then compile, ; immediate

So in the COMPILE case we have to use ",", while in the other cases we
have to use COMPILE,.

The version above tries to stay close to the COMPILE COMPILE idea, but
it's simpler to write it as:

: POSTPONE
bl word find ?dup if
0< if [ ' compile compile, ] , exit then
else number dup >r execute r> then compile, ; immediate

>> Of course they can. Moreover, it's possible to redefine POSTPONE to
>> handle literals.
>
>Not easily I presume?

If we assume a NUMBER like as used above, and sticking close to your
retro style (because your NUMBER is designed for it):

: postpone ( "name" -- )
>in @ bl word find if
drop >in ! postpone postpone exit then
nip number dup >r execute r> compile, ;

Alternatively, if you have recognizers:

: POSTPONE ( "name" -- )
PARSE-NAME FORTH-RECOGNIZER RECOGNIZE DUP >R
RECTYPE>POST EXECUTE R> RECTYPE>COMP COMPILE, ;

[source: <https://forth-standard.org/proposals/recognizer>]

[dxforth]

On 10/05/2021 01:21, Anton Ertl wrote:
> dxforth <dxf...@gmail.com> writes:
>>I get certain
>>advantages continuing to use a POSTPONE that's based on COMPILE and
>>thus far none using POSTPONE in immediate mode - which has always
>>been counter-intuitive.
>
> The way you use COMPILE in POSTPONE is compatible with POSTPONEing in
> interpret state:
>
> : POSTPONE
> ?comp bl word find ?dup if
> 0< if compile compile then
> else number then compile, ; immediate
>
> You do not rely on the text interpreter to "," the xt of the postponed
> word; instead POSTPONE ","s the xt with COMPILE,. Of course, your
> STATE-smart NUMBER remains a problem, but
> <2021Apr2...@mips.complang.tuwien.ac.at> showed how to fix that.

I've since shifted the state-smartness from NUMBER to INTERPRET where
it belongs. What hasn't changed is the classic behaviour that executing
COMPILE when STATE=0 will crash the interpreter due to IP having been
advanced. COMPILE isn't in ANS for the reason they state, however it
remains useful for threaded-code systems where memory is a premium.

[Stephen Pelc]

On Sat, 8 May 2021 22:17:52 +0300, Ruvim <ruvim...@gmail.com>
wrote:

>As I understand, the Stephen's point was that the standard describes how
>to perform default compilation semantics (that's to append ES), and how
>to perform compilation semantics for immediate words (that's to perform
>ES), but doesn't describe how to perform compilation semantics for other
>words.

Thanks.

>Actually, it's a fictional problem. The standard just considers and
>describes behavior of a system on some level of abstraction, and doesn't
>consider by what means this behavior is accomplished, and how the system
>behaves on the underlying levels.

But it's not a fictional problem in the presence of testing.
If we do not have standard ways (words) to handle NDCS words, we
cannot have standard tests. Now that VFX Forth handles IF and other
words as NDCS words, we need to add NDCS handling words to the
standard.

>I.g., it also doesn't say how compilation semantics are appended to the
>current definition.
>
>If the standard doesn't say how to accomplish some results or behavior,
>then it's up to an implementer how to to it, and the implementer is free
>to use any suitable means.

Yes. Now the Forth words in the standard need to catch up with the
text. As the phrase warns us "Be careful what you wish for", we first
need to decide what we want in the standard.

[Ruvim]

On 2021-05-10 16:58, Stephen Pelc wrote:
> On Sat, 8 May 2021 22:17:52 +0300, Ruvim <ruvim...@gmail.com>
> wrote:
>
>> As I understand, the Stephen's point was that the standard describes how
>> to perform default compilation semantics (that's to append ES), and how
>> to perform compilation semantics for immediate words (that's to perform
>> ES), but doesn't describe how to perform compilation semantics for other
>> words.
>
> Thanks.
>
>> Actually, it's a fictional problem. The standard just considers and
>> describes behavior of a system on some level of abstraction, and doesn't
>> consider by what means this behavior is accomplished, and how the system
>> behaves on the underlying levels.
>
> But it's not a fictional problem in the presence of testing.

True. So, it isn't a problem for system, but it's a problem for a program.


Actually, a program has some indirect methods:

- perform EVALUATE in compilation state (it's unhygienic, it's
difficult to use for some dual-semantics parsing words, and cannot be
used for other).

- create a wrapper by means of POSTPONE and perform these wrapper in
compilation state (cannot be used during compilation).


But I'm convinced that FIND with EXECUTE should be capable to perform
compilation semantics for any NDCS word (NDCS by implementation),
regardless is it an immediate word or not.

This point is not specified in the normative parts of the standard at
the moment, but the rationales show that it was intended, without any doubt.

And I don't see any reason why this point should not be included in the
specification for FIND. (see my proposal re FIND clarification).

> If we do not have standard ways (words) to handle NDCS words, we
> cannot have standard tests.

The tests should not depend on how NDCS words are implemented, and
whether they are implemented via mechanism of immediacy, or not.

> Now that VFX Forth handles IF and other
> words as NDCS words, we need to add NDCS handling words to the
> standard.

I shown how FIND can be implemented in VFX without any additional words
for handling NDCS words.
See "Fix VFX FIND" at news:s73ea4$nt9$1...@dont-email.me

Why not?

>> I.g., it also doesn't say how compilation semantics are appended to the
>> current definition.
>>
>> If the standard doesn't say how to accomplish some results or behavior,
>> then it's up to an implementer how to to it, and the implementer is free
>> to use any suitable means.
>
> Yes. Now the Forth words in the standard need to catch up with the
> text. As the phrase warns us "Be careful what you wish for", we first
> need to decide what we want in the standard.
>

--
Ruvim

[Anton Ertl]

dxforth <dxf...@gmail.com> writes:
>I've since shifted the state-smartness from NUMBER to INTERPRET where
>it belongs.

Good.

>What hasn't changed is the classic behaviour that executing
>COMPILE when STATE=0 will crash the interpreter due to IP having been
>advanced.

This has nothing to do with STATE; you identify the dependency
correctly: IP. Here's an example how COMPILE can run when STATE=0:

\ First, define COMPILE, and show STATE when it runs:
: compile state ? r> dup cell+ >r @ , ;

: foo compile . ;
: bar [ foo ] ; \ prints "0" (output of COMPILE)
1 bar \ prints "1"

Tested with gforth-itc 0.7.9_20210415. As you can see, running
COMPILE when STATE=0 works.

[Ruvim]

On 2021-05-09 10:35, Marcel Hendrix wrote:
> On Saturday, May 8, 2021 at 3:10:49 PM UTC+2, Anton Ertl wrote:
> [..]
>> So let's combine my scaffolded
>> COMPILE, from the 2015 test with Ruvim's test:
>
> I have to dig through a lot of sneaky text (test of a test in 2015?)
> to even approach what you have done here.
> Maybe just insert it explicitly?
>
>> : defers
>> \ "defers X" produces a call to the current content of the deferred word X
>> ' defer@ compile, ; immediate
>
> I am not fond of these discussions but must point out that DEFER@
> only works for words created by DEFER , a word in the CORE-EXT set.
> It is therefore unlikely that DEFER@ is going to work on non-user-defined
> words like COMPILE,.

True.

These tests are not standard, they use carnal knowledge of the
particular Forth systems. And these tests were created to test these
particular systems.


--
Ruvim

[Ruvim]

On 2021-05-06 18:23, P Falth wrote:
> On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:
>> On 2021-05-06 17:07, Ruvim wrote:
>>> On 2021-05-06 14:38, P Falth wrote:
>> [...]
>>>> As an alternative to FIND I can suggest
>>>> FIND-INTERPRET ( addr u -- addr u 0 | xt2 xt1 -1 )
>>>> FIND-COMPILE ( addr u -- addr u 0 | xt2 xt1 -1 )
>>>> FIND-POSTPONE ( addr u -- addr u 0 | xt2 xt1 -1 )
>> They are quite close to FIND-NAME NAME>COMPILE
>
> Yes they are. But they also hide the NT and can thus accomodate
> also a system with dual NTs. Someone (Mark Williams?) described
> such a system that was simple and clean.

But why they are better than "FIND"? (except (c-addr u) string).

In my recent iteration I introduced the word FIND-SEM

FIND-SEM ( c-addr u -- xt flag-special true | c-addr u false )

It is closer to classic "FIND", with only difference that it accepts a
string as the pair (c-addr u), and returns more convenient flag-special,
rather than -1|1 code.


And why do you need "FIND-POSTPONE"?

>>> If you can implement this, nothing prevent you to implement STATE
>>> dependent FIND
>> But only if for the cases FIND-INTEPRET and FIND-COMPILE, xt1 identify
>> either "EXECUTE" or "COMPILE," and nothing else.
>
> To require that XT1 be limited to "EXECUTE" or "COMPILE," would severely
> limit the possibilities of the system. Take for example a constant.
> XT2 could be the actual value of the constant and XT1 LITERAL for compiling.

It looks like you reinvent recognizers under the different name.

> My LXF64 compiles to tokenized code. XT1 of interpret could there be a call
> to a JIT compiler that compiles the tokens to native code and then executes it

Why cannot EXECUTE be who calls a JIT compiler (if a better code is not
generated yet) and then executes this better code?

>> Otherwise FIND cannot be implemented via these words.
>
> There is no requirements to implement FIND with these words.

For back-compatibility, the system should still provide FIND for a
while. And it should be clear how to implement FIND via new methods in
the case when a system doesn't provide FIND (to use programs/libraries
that rely on FIND).


A one shortcoming of NAME>* methods is that they don't guarantee that
FIND can be implementable via them. Moreover, FIND is not implementable
via these methods in a single-xt system, if this system allows to
perform compilation semantics in interpretation state.

>>>
>>> But what for do you need FIND-POSTPONE ?
>>>
>>>
>>>> The result if found just needs to be executed to perform the
>>>> respective actions
>>>> It also hides the implementation details ( single-xt, dual-xt, dual
>>>> headers etc)
>>>
>>> It doesn't cover sing-xt systems. In these systems the corresponding
>>> action should be performed only in the corresponding STATE.
>> I missed that they return two xt-s. Then yes, they cover single-xt
>> systems too.
>>
>> But then what is a way to detect whether a word is an ordinary word?
>
> That is not needed!

I need that.

See for exampe: https://forth-standard.org/standard/tools#reply-445

Another example. A kind of portable SYNONYM should be able to detected
whether a word is an ordinary word, or not.

>> And what is a way to get xt of an ordinary word, that identifies the
>> execution semantics for this word?
>
> ' and ['] are still available for this.

They cannot be applied to any word. They cannot be used to words of
unknown kind.


--
Ruvim

[P Falth]

On Monday, 10 May 2021 at 20:34:52 UTC+2, Ruvim wrote:
> On 2021-05-06 18:23, P Falth wrote:
> > On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:
> >> On 2021-05-06 17:07, Ruvim wrote:
> >>> On 2021-05-06 14:38, P Falth wrote:
> >> [...]
> >>>> As an alternative to FIND I can suggest
> >>>> FIND-INTERPRET ( addr u -- addr u 0 | xt2 xt1 -1 )
> >>>> FIND-COMPILE ( addr u -- addr u 0 | xt2 xt1 -1 )
> >>>> FIND-POSTPONE ( addr u -- addr u 0 | xt2 xt1 -1 )
> >> They are quite close to FIND-NAME NAME>COMPILE
> >
> > Yes they are. But they also hide the NT and can thus accomodate
> > also a system with dual NTs. Someone (Mark Williams?) described
> > such a system that was simple and clean.
> But why they are better than "FIND"? (except (c-addr u) string).

When I developed the code generator for my forth (about 20 years ago)
I needed a way to execute the generator for a word when it was compiled
into a definition. I started with the CMFORTH style dual headers in different
wordlists. This worked fine until I also needed wordlists for its intended
purpose. That is when I added a compilation xt to the header. For some
words ( normal words, created words) at compiletime was also needed
the interpretation xt. FIND could not give me this. From this was born
FIND-INTERPRET and FIND-COMPILE. They do what I needed!

> In my recent iteration I introduced the word FIND-SEM
>
> FIND-SEM ( c-addr u -- xt flag-special true | c-addr u false )
>
> It is closer to classic "FIND", with only difference that it accepts a
> string as the pair (c-addr u), and returns more convenient flag-special,
> rather than -1|1 code.

But that would not let me find both xts at the same time

> And why do you need "FIND-POSTPONE"?

I do not at the moment and it is not implemented. But might be
useful for recognizers. It might serve to do words with non default
postpone semantics (NDPS words!)

That was a joke!

> >>> If you can implement this, nothing prevent you to implement STATE
> >>> dependent FIND
> >> But only if for the cases FIND-INTEPRET and FIND-COMPILE, xt1 identify
> >> either "EXECUTE" or "COMPILE," and nothing else.
> >
> > To require that XT1 be limited to "EXECUTE" or "COMPILE," would severely
> > limit the possibilities of the system. Take for example a constant.
> > XT2 could be the actual value of the constant and XT1 LITERAL for compiling.
> It looks like you reinvent recognizers under the different name.

Yes, just 20 years ago!

> > My LXF64 compiles to tokenized code. XT1 of interpret could there be a call
> > to a JIT compiler that compiles the tokens to native code and then executes it
> Why cannot EXECUTE be who calls a JIT compiler (if a better code is not
> generated yet) and then executes this better code?

Yes that is how it can work. ON the stack will be

NT JIT-compiler, coming from find-interpret

I am not there yet. Now I can call native code from the token code. But I also want to
do the reverse.

> >> Otherwise FIND cannot be implemented via these words.
> >
> > There is no requirements to implement FIND with these words.
> For back-compatibility, the system should still provide FIND for a
> while. And it should be clear how to implement FIND via new methods in
> the case when a system doesn't provide FIND (to use programs/libraries
> that rely on FIND).

I do provide a FIND in my systems. In lxf it is not possible to do a user defined
interpreter with it, for 2 reasons. It is not state-smart. It only returns the
interpretation XT. Words like IF with no defined interpretation semantics
return an xt that print an error message and aborts. If needed i could change this
but so far has not needed it.

> A one shortcoming of NAME>* methods is that they don't guarantee that
> FIND can be implementable via them. Moreover, FIND is not implementable
> via these methods in a single-xt system, if this system allows to
> perform compilation semantics in interpretation state.

My lxf64 is still single-xt, FIND works just fine there. There is no requirement
that find must be implemented by NAME>* methods

> >>>
> >>> But what for do you need FIND-POSTPONE ?
> >>>
> >>>
> >>>> The result if found just needs to be executed to perform the
> >>>> respective actions
> >>>> It also hides the implementation details ( single-xt, dual-xt, dual
> >>>> headers etc)
> >>>
> >>> It doesn't cover sing-xt systems. In these systems the corresponding
> >>> action should be performed only in the corresponding STATE.
> >> I missed that they return two xt-s. Then yes, they cover single-xt
> >> systems too.
> >>
> >> But then what is a way to detect whether a word is an ordinary word?
> >
> > That is not needed!
> I need that.
>
> See for exampe: https://forth-standard.org/standard/tools#reply-445
>
> Another example. A kind of portable SYNONYM should be able to detected
> whether a word is an ordinary word, or not.

I provide a completely compliant SYNONYM. Different NTs but same XTs
There is no need to load another synonym.

> >> And what is a way to get xt of an ordinary word, that identifies the
> >> execution semantics for this word?
> >
> > ' and ['] are still available for this.
> They cannot be applied to any word. They cannot be used to words of
> unknown kind.

I do not understand this. Any word that has an XT could be ticked, or?

BR
Peter

> --
> Ruvim

[minf...@arcor.de]

You can't be serious.

NDPS can be deadly when you POSTPONE TO when in interpretation mode
between square brackets, and while EVALUATEing that thing !!

It is absolutely necessary that someone rewrites the standard !!

[dxforth]

On 10/05/2021 23:58, Stephen Pelc wrote:
> On Sat, 8 May 2021 22:17:52 +0300, Ruvim <ruvim...@gmail.com>
> wrote:
>
>>As I understand, the Stephen's point was that the standard describes how
>>to perform default compilation semantics (that's to append ES), and how
>>to perform compilation semantics for immediate words (that's to perform
>>ES), but doesn't describe how to perform compilation semantics for other
>>words.
>
> Thanks.
>
>>Actually, it's a fictional problem. The standard just considers and
>>describes behavior of a system on some level of abstraction, and doesn't
>>consider by what means this behavior is accomplished, and how the system
>>behaves on the underlying levels.
>
> But it's not a fictional problem in the presence of testing.
> If we do not have standard ways (words) to handle NDCS words, we
> cannot have standard tests. Now that VFX Forth handles IF and other
> words as NDCS words, we need to add NDCS handling words to the
> standard.

Standard tests for what purpose? First one must define the
standard need i.e. why does FIND used in situation X need to
produce the same result on every system? ANS' premise in
permitting implementers to do as they pleased was that not
every situation required standardizing. Were they wrong?
I couldn't find a single app I'd written that used FIND :)

[dxforth]

On 11/05/2021 02:09, Anton Ertl wrote:
> dxforth <dxf...@gmail.com> writes:
>>I've since shifted the state-smartness from NUMBER to INTERPRET where
>>it belongs.
>
> Good.
>
>>What hasn't changed is the classic behaviour that executing
>>COMPILE when STATE=0 will crash the interpreter due to IP having been
>>advanced.
>
> This has nothing to do with STATE; you identify the dependency
> correctly: IP. Here's an example how COMPILE can run when STATE=0:
>
> \ First, define COMPILE, and show STATE when it runs:
> : compile state ? r> dup cell+ >r @ , ;
>
> : foo compile . ;
> : bar [ foo ] ; \ prints "0" (output of COMPILE)
> 1 bar \ prints "1"
>
> Tested with gforth-itc 0.7.9_20210415. As you can see, running
> COMPILE when STATE=0 works.

AFAIK Forth-83 didn't require that example to work. Same for
ANS and POSTPONE. Is there a demonstrated need that it should
work that would justify amending or clarifying ANS' ruling?

Change in my case would mean removing ?COMP from COMPILE. The
former hasn't proven an issue thus far. I'd like to know what
I was getting in return for removing compiler security.

[Anton Ertl]

Marcel Hendrix <m...@iae.nl> writes:
>On Saturday, May 8, 2021 at 3:10:49 PM UTC+2, Anton Ertl wrote:

[...]

>I am not fond of these discussions but must point out that DEFER@
>only works for words created by DEFER , a word in the CORE-EXT set.
>It is therefore unlikely that DEFER@ is going to work on non-user-defined
>words like COMPILE,.

As it happens, COMPILE, is defined with DEFER in VFX 4.71 and VFX 5.11:

VFX Forth for Linux IA32 Version: 4.71 [build 0537]
see compile,
COMPILE, is DEFERred. Action is COMPILE-REF
COMPILE-REF
...

VFX Forth 64 5.11 RC2 [build 0112] 2021-05-02 for Linux x64
© MicroProcessor Engineering Ltd, 1998-2021

see compile,
COMPILE, is DEFERred. Action is COMPILE-REF
COMPILE-REF
( 0046FC80 483B1D6659FAFF ) CMP RBX, FFFA5966 [RIP] @004155ED
...

COMPILE, is also defined with DEFER in Gforth, SwiftForth and lxf.

This is useful for my original test, because it allows to test the
COMPILE, embedded in the text interpreter of the system rather than
having to write a new text interpreter. For Ruvim's test that I
combined my test with, it would be good enough to just redefine
"COMPILE,", but the version with IS also shows what happens in the
rest of the system.

>Or is there a guarantee now that existence
>of DEFER means that all deferred words are created by DEFER ?

If your question is whether all non-user-defined words are created by
DEFER: No. IIRC such a guarantee exists for no standard-specified
word. In this case I just happened to know that
COMPILE, is deferred on a few systems.

>> :noname ( xt -- )
>> >in @ >r
>> depth 1- >r
>> defers compile,
>
>Here you assume COMPILE, is a deferred word. It would really help
>to show the complete example.

The test is complete. Paste it in VFX, Gforth, SwiftForth, or lxf and
see what happens. But just to have the test and results here to
discuss, I present them all:

The test:

: defers
\ "defers X" produces a call to the current content of the deferred word X
' defer@ compile, ; immediate

:noname ( xt -- )
>in @ >r
depth 1- >r
defers compile,

depth r> <> abort" compile, stack effect wrong"
>in @ r> <> abort" compile, input stream effect wrong" ;
is compile,

: test-if
c" if" find dup 0= if ." unfound" 2drop exit then
1 = if ." executed" execute else ." compiled" compile, then
; immediate

] test-if [ .s

The results:
[~:121903] gforth /tmp/xxx.fs
redefined Defers with defers executed<3> 0 140166832538920 2

[~:121904] sf "include /tmp/xxx.fs"
executed
134761828 <-Top ok

VFX Forth for Linux IA32 Version: 4.71 [build 0537]
Including /tmp/xxx.fs

CS=0023 DS=002B ES=002B SS=002B FS=0000 GS=0063

EAX=0000:0000 EBX=FFFF:FFFF ECX=090A:0E25 EDX=0000:0000
ESI=090B:2000 EDI=0804:A000 EBP=090B:0FC0 ESP=090A:0F1C
EIP=080C:091A EFLAGS=0001:0297
--- RS top ---
$FFFF:FFFF
$0000:0013
$0805:9C8B DEFER
$080C:0906 DEFERS

$0805:9C8B DEFER
$0805:97F3 (INTERPRET)
$0805:9C8B DEFER
$0805:EE90 ((INCLUDE-FILE))
Signal number SIGSEGV

at address 080C:091A, probably in DEFERS

VFX Forth for Linux IA32 Version: 4.72 [build 0555]
Including /tmp/xxx.fscompile, input stream effect wrong
Source: "/tmp/xxx.fs" on line 14
-> c" if" find dup 0= if ." unfound" 2drop exit then
^
ERROR on command line


VFX Forth 64 5.11 RC2 [build 0112] 2021-05-02 for Linux x64
© MicroProcessor Engineering Ltd, 1998-2021

Including /tmp/xxx.fscompiledcompile, stack effect wrong
Source: "/tmp/xxx.fs" on line 18
-> ] test-if [ .s
^
ERROR on command line

Linux/FORTH (C) 2005 Peter Fälth Version 1.6-982-823 Compiled on 2017-12-03
Running on Linux 4.5.0-0.bpo.2-amd64 on x86_64
Current directory is /home/anton no block file
include /tmp/xxx.fs compiled

Gforth and SwiftForth behave as expected: FIND reports 1 for IF, so
TEST-IF EXECUTEs the xt, and this pushes an orig on the stack.

vfx 4.71 fails due to a bug in DEFER@. This bug was fixed in 4.72,
and there scaffolded COMPILE, shows that VFX's COMPILE, has a
non-standard effect on the input stream when text-interpreting C".
VFX does not show that, but we see a non-standard stack effect of
COMPILE, in Ruvim's test. My guess is that VFX 5.11 defines the text
interpreter differently so the problem with COMPILE, does not show up
there, but it still shows up in a user-defined text interpreter.

lxf has a working COMPILE, but the FIND does not support user-defined
text interpreters.

[none albert]

In article <s7d1v6$jgs$1...@gioia.aioe.org>, dxforth <dxf...@gmail.com> wrote:
<SNIP>

>Standard tests for what purpose? First one must define the
>standard need i.e. why does FIND used in situation X need to
>produce the same result on every system? ANS' premise in
>permitting implementers to do as they pleased was that not
>every situation required standardizing. Were they wrong?
>I couldn't find a single app I'd written that used FIND :)
>

I once had the sophisticated ciasdis assembler to use a wordlist
to contain the objects associated with each assembler mnemonic.
Use of the wordlist came on top of the normal use of the assembler,
they are just words after all.
All other uses I now consider a design fault, and I made the assembler
mnemonic objects a linked list, for that.
A case in point, the objects are usable in my optimiser.
I need a few object of that type in my optimiser that would not
trigger in normal assembly use. That would have been been
a terrible pain, without the separate link chain.

>
Groetjes Albert
>
--
"in our communism country Viet Nam, people are forced to be
alive and in the western country like US, people are free to
die from Covid 19 lol" duc ha
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[Ruvim]

On 2021-05-11 07:42, dxforth wrote:
> On 10/05/2021 23:58, Stephen Pelc wrote:

[...]

>> If we do not have standard ways (words) to handle NDCS words, we
>> cannot have standard tests. Now that VFX Forth handles IF and other
>> words as NDCS words, we need to add NDCS handling words to the
>> standard.
>
> Standard tests for what purpose?  First one must define the
> standard need i.e. why does FIND used in situation X need to
> produce the same result on every system?  ANS' premise in
> permitting implementers to do as they pleased was that not
> every situation required standardizing.

Yes. But every standard word should be enough specified to be useful.


> Were they wrong?

The specification for the FIND word is not sufficient.

A.6.2.0945 says:
The intention is that COMPILE, can be used as follows
to write the classic interpreter/compiler loop:

... ( c-addr )

FIND ?DUP IF ( xt +-1 )
STATE @ IF ( xt +-1 )
0> IF EXECUTE ELSE COMPILE, THEN ( ??? )
ELSE ( xt +-1 )
DROP EXECUTE ( ??? )
THEN
ELSE ( c-addr )

( whatever you do for an undefined word )
THEN
...

Some systems implement FIND in such a way that this loop doesn't work as
expected for the words with non default interpretation semantics.

> I couldn't find a single app I'd written that used FIND :)

Probably you just didn't ever need a user-defined text interpreter for
some advanced DSL (domain specific language), or some advanced extension
of a Forth system.


--
Ruvim

[Anton Ertl]

dxforth <dxf...@gmail.com> writes:
>On 11/05/2021 02:09, Anton Ertl wrote:
>> dxforth <dxf...@gmail.com> writes:
>>>What hasn't changed is the classic behaviour that executing
>>>COMPILE when STATE=0 will crash the interpreter due to IP having been
>>>advanced.
>>
>> This has nothing to do with STATE; you identify the dependency
>> correctly: IP. Here's an example how COMPILE can run when STATE=0:
>>
>> \ First, define COMPILE, and show STATE when it runs:
>> : compile state ? r> dup cell+ >r @ , ;
>>
>> : foo compile . ;
>> : bar [ foo ] ; \ prints "0" (output of COMPILE)
>> 1 bar \ prints "1"
>>
>> Tested with gforth-itc 0.7.9_20210415. As you can see, running
>> COMPILE when STATE=0 works.
>
>AFAIK Forth-83 didn't require that example to work. Same for
>ANS and POSTPONE.

I guess that changing the goalposts is an admission that the statement

"executing COMPILE when STATE=0 will crash the interpreter due to IP

having been advanced." is wrong.

Let's consider the new goalposts:

1) Return-address manipulation is non-standard (I think also in
Forth-83). But let's assume the system-defined COMPILE rather than
the COMPILE defined above.

2) Forth-83

COMPILE is marked with the C attribute in Forth-83. In Section 11.4,
Forth-83 states:

| C The word may only be used during compilation of a colon
| definition.

It's not clear what "use" means here. Does it mean the compilation of
COMPILE during the definition of FOO? Does it mean the running of
COMPILE during the definition of BAR? Does it mean the result of the
COMPILE when running BAR in the final line of the example? I guess
that it's not the latter, because then COMPILE would be pretty
unusable.

The other unclear thing is what "during compilation of a colon
definition" means. My take is that FOO is run during the compilation
of BAR, so the definition of BAR is Forth-83-conforming in my book.
Fans of ?COMP will probably take a different position, claiming that
"during compilation" means STATE=-1, and ignoring the "of a colon
definition" part.

3) We have discussed POSTPONE repeatedly over the last
quarter-century, and I have established repeatedly that according to
the text of Forth-94 and Forth-2012

: foo postpone . ;
: bar [ foo ] ;
1 bar \ prints "1"

conforms to these standards.

>Is there a demonstrated need that it should
>work that would justify amending or clarifying ANS' ruling?

There is no need to amend or clarify. The text of the standard is
clear. If anyone wants to change it in the direction of disallowing
to append something to the current definition when STATE=0, they have
to propose it and should give a good reason for such a change. I
demonstrated above that basing POSTPONE on COMPILE is not such a
reason.

>Change in my case would mean removing ?COMP from COMPILE. The
>former hasn't proven an issue thus far. I'd like to know what
>I was getting in return for removing compiler security.

You get at least two bytes. And ?COMP is not compiler security.

[none albert]

tmanx2 (my interpreter for music, that plays music on the metallophones,
and maybe midi) has elements corresponding to notes, notes-duration, scores,
parts, bars, chords, articulation, transposition, tempo, note-binding.
It certainly can count as an example of an advanced DSL.
That is all handled by the Forth text interpreter itself.
Then there is my syntax checker for Pascal.
IMO DSL and advanced extensions can normally avoid using FIND explicitly,
probably resulting in better portability.
The only example I can think of is if you want to emulate the weird syntax
of lisp related to their idea of macro's, that arbitrarily defines part
of the input evaluated immediately. I tried it, and it made me conclude
that lisp is badly designed.

>--
>Ruvim

[Ruvim]

On 2021-05-11 00:16, P Falth wrote:
> On Monday, 10 May 2021 at 20:34:52 UTC+2, Ruvim wrote:
>> On 2021-05-06 18:23, P Falth wrote:
>>> On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:
>>>> On 2021-05-06 17:07, Ruvim wrote:
>>>>> On 2021-05-06 14:38, P Falth wrote:
>>>> [...]
>>>>>> As an alternative to FIND I can suggest
>>>>>> FIND-INTERPRET ( addr u -- addr u 0 | xt2 xt1 -1 )
>>>>>> FIND-COMPILE ( addr u -- addr u 0 | xt2 xt1 -1 )
>>>>>> FIND-POSTPONE ( addr u -- addr u 0 | xt2 xt1 -1 )
>>>> They are quite close to FIND-NAME NAME>COMPILE
>>>
>>> Yes they are. But they also hide the NT and can thus accomodate
>>> also a system with dual NTs. Someone (Mark Williams?) described
>>> such a system that was simple and clean.
>> But why they are better than "FIND"? (except (c-addr u) string).
>
> When I developed the code generator for my forth (about 20 years ago)
> I needed a way to execute the generator for a word when it was compiled
> into a definition. I started with the CMFORTH style dual headers in different
> wordlists. This worked fine until I also needed wordlists for its intended
> purpose. That is when I added a compilation xt to the header.

Another option is to introduce different flags into the header and a
functionality to find words with given flags only. But if you need both
semantics in the same time, FIND should be performed twice in this approach.

> For some
> words ( normal words, created words) at compiletime was also needed
> the interpretation xt. FIND could not give me this. From this was born
> FIND-INTERPRET and FIND-COMPILE. They do what I needed!

I see. It's OK for internal or system-specific purposes.

But then for compatibility you could make xt=nt (and then it will be a
sing-xt system from the user's point of view).

Or at least FIND should return something like xt of
[: true abort" not implemented" ;]
and n value 1, or just n value 0, when FIND is applied to the "IF"
string in compilation state, and ['] IF with n value 1 — in
interpretation state.

>> In my recent iteration I introduced the word FIND-SEM
>>
>> FIND-SEM ( c-addr u -- xt flag-special true | c-addr u false )
>>
>> It is closer to classic "FIND", with only difference that it accepts a
>> string as the pair (c-addr u), and returns more convenient flag-special,
>> rather than -1|1 code.
>
> But that would not let me find both xts at the same time

Do you talk about your system or a program?

In your system you are free to use system-specific methods.

A standard program have to call FIND-SEM twice (in two different states)
for that — the same as with FIND.

>> And why do you need "FIND-POSTPONE"?
>
> I do not at the moment and it is not implemented. But might be
> useful for recognizers. It might serve to do words with non default
> postpone semantics (NDPS words!)
>
> That was a joke!

Oh, I see. After all, NDPS is non standard.

>>>>> If you can implement this, nothing prevent you to implement STATE
>>>>> dependent FIND
>>>> But only if for the cases FIND-INTEPRET and FIND-COMPILE, xt1 identify
>>>> either "EXECUTE" or "COMPILE," and nothing else.
>>>
>>> To require that XT1 be limited to "EXECUTE" or "COMPILE," would severely
>>> limit the possibilities of the system. Take for example a constant.
>>> XT2 could be the actual value of the constant and XT1 LITERAL for compiling.
>> It looks like you reinvent recognizers under the different name.
>
> Yes, just 20 years ago!

Then why don't use recognizers for such things?

As Anton pointed out, above approach is possible for single-cell numbers
only, when recognizers are suitable for any literals.

>>> My LXF64 compiles to tokenized code. XT1 of interpret could there be a call
>>> to a JIT compiler that compiles the tokens to native code and then executes it
>> Why cannot EXECUTE be who calls a JIT compiler (if a better code is not
>> generated yet) and then executes this better code?
>
> Yes that is how it can work. ON the stack will be
>
> NT JIT-compiler, coming from find-interpret

I mean that EXECUTE behaves in the following manner:

: execute ( i*x xt -- j*x )
xt>oxt if ( oxt ) oexecute exit then ( xt )
ocompiled ( oxt ) oexecute
;

> I am not there yet. Now I can call native code from the token code. But I also want to
> do the reverse.



>
>>>> Otherwise FIND cannot be implemented via these words.
>>>
>>> There is no requirements to implement FIND with these words.
>> For back-compatibility, the system should still provide FIND for a
>> while. And it should be clear how to implement FIND via new methods in
>> the case when a system doesn't provide FIND (to use programs/libraries
>> that rely on FIND).
>
> I do provide a FIND in my systems.

When you propose an alternative to FIND, it's not about your system.
It's about the standard and the Forth ecosystem.

> In lxf it is not possible to do a user defined
> interpreter with it, for 2 reasons. It is not state-smart. It only returns the
> interpretation XT. Words like IF with no defined interpretation semantics
> return an xt that print an error message and aborts. If needed i could change this
> but so far has not needed it.

As I understand, the second reason is that lxf cannot provide an xt that
identifies the following execution semantics:

[: postpone if ;]

>> A one shortcoming of NAME>* methods is that they don't guarantee that
>> FIND can be implementable via them. Moreover, FIND is not implementable
>> via these methods in a single-xt system, if this system allows to
>> perform compilation semantics in interpretation state.
>
> My lxf64 is still single-xt, FIND works just fine there. There is no requirement
> that find must be implemented by NAME>* methods
>
>>>>>
>>>>> But what for do you need FIND-POSTPONE ?
>>>>>
>>>>>
>>>>>> The result if found just needs to be executed to perform the
>>>>>> respective actions
>>>>>> It also hides the implementation details ( single-xt, dual-xt, dual
>>>>>> headers etc)
>>>>>
>>>>> It doesn't cover sing-xt systems. In these systems the corresponding
>>>>> action should be performed only in the corresponding STATE.
>>>> I missed that they return two xt-s. Then yes, they cover single-xt
>>>> systems too.
>>>>
>>>> But then what is a way to detect whether a word is an ordinary word?
>>>
>>> That is not needed!
>> I need that.
>>
>> See for exampe: https://forth-standard.org/standard/tools#reply-445
>>
>> Another example. A kind of portable SYNONYM should be able to detected
>> whether a word is an ordinary word, or not.
>
> I provide a completely compliant SYNONYM. Different NTs but same XTs
> There is no need to load another synonym.

Again, you talk about your system. But I talk about Forth ecosystem, and
broad variety of Forth systems.

If a standard system doesn't provide SYNONYM but provide a full-fledged
FIND, then SYNONYM (that preserves execution semantics as well as IS and
CS) can be implemented.

>>>> And what is a way to get xt of an ordinary word, that identifies the
>>>> execution semantics for this word?
>>>
>>> ' and ['] are still available for this.
>> They cannot be applied to any word. They cannot be used to words of
>> unknown kind.
>
> I do not understand this. Any word that has an XT could be ticked, or?

Tick returns xt that identifies execution semantics (ES) for the name in
argument, but ES are not specified for many standard words.

Therefore, 4.1.2 says that an ambiguous condition exists if a program
attempts to obtain the execution token of a definition with undefined
interpretation semantics (for all such cases ES are either unspecified,
or cannot be provided in many implementations).

Concerning other words with unspecified ES, it says that applying Tick
to the word is ambiguous, individually for each such word except FILE S"
and FILE S\".

My proposal is to remove individual cases and declare the single rule:
an ambiguous condition exists if a program attempts to obtaining the
execution token of a definition with non default interpretation semantics.

NB: my proposal of FIND clarification explicitly specifies in what case
the returned xt identifies ES for the name/word in argument.


--
Ruvim

[Ruvim]

On 2021-05-11 17:00, albert wrote:
> In article <s7dnf0$fjm$1...@dont-email.me>, Ruvim <ruvim...@gmail.com> wrote:
>> On 2021-05-11 07:42, dxforth wrote:
>>> I couldn't find a single app I'd written that used FIND :)
>>
>> Probably you just didn't ever need a user-defined text interpreter for
>> some advanced DSL (domain specific language), or some advanced extension
>> of a Forth system.
>
> tmanx2 (my interpreter for music, that plays music on the metallophones,
> and maybe midi) has elements corresponding to notes, notes-duration, scores,
> parts, bars, chords, articulation, transposition, tempo, note-binding.
> It certainly can count as an example of an advanced DSL.
> That is all handled by the Forth text interpreter itself.
> Then there is my syntax checker for Pascal.
> IMO DSL and advanced extensions can normally avoid using FIND explicitly,
> probably resulting in better portability.

We have two options to implement control structures (or other lexical
blocks in source code):
- non parsing words (like "IF")
- parsing words (like "[IF]")

If we want to make it work in interpretation state, only parsing variant
is suitable. Also, parsing variant is easier in implementation since
control parameters can be kept on the r-stack.

But then you have to use a user-defined text interpreter, and rely on
FIND (or it's alternatives).



--
Ruvim

[Stephen Pelc]

On Fri, 7 May 2021 14:12:24 +0300, Ruvim <ruvim...@gmail.com>
wrote:

>> In VFX, some special words are NDCS but not IMMEDIATE. The equivalent
>> of COMPILE, for such words is NDCS,
>>
>> : ndcs, \ ??? xt -- ???
>> \ *G Perform the compilation action of an NDCS word. This may have
>> \ ** a stack effect or parse the input stream.
>>
>> : compile-word \ i*x xt -- j*x
>> \ *G Process an XT for compilation. May parse or have stack effect.
>> dup ndcs? \ NDCS or normal
>> if ndcs, else compile, then
>> ;
>>
>> In VFX, most uses of COMPILE, should use COMPILE-WORD instead.
>> The alternative is to redefine COMPILE, to take note of NDCS words
>> but ... politics, politic, politics.
>
>Taking into account that "ndcs," just performs another xt:
>
> : (ndcs,) \ i*x xt -- j*x
> >code-len @ execute \ execute NDCS/IMMEDIATE word
> ;
>
>Why not return this xt by "find" ?
>
> : find ( c-addr -- c-addr 0 | xt -1 | xt 1 )
> find dup 0= if exit then drop ( xt1 )
> dup ndcs? if >code-len @ 1 else
> dup immediate? if 1 else -1 then then ( xt1 xt2 n )
> state @ if rot else swap then drop
> ;
>

>After this fix all my tests work as expected.

The fix is the sensible one, which is to throw away IMMEDIATE and
let FIND indicate NDCS and normal. However, on some systems this
will be a code breaker.

Personally, and for MPE, I love the original idea of an xt that it
be the single point of contact between the user and the word. An
xt can be used for many purposes other than compilation and
interpretation, e.g. LOCATE XREF and so on.

Use of xt in this way requires that all words have a primary or
interpretation action, which the xt represents. Your fix breaks
this and I suppose makes it non-standard.

Because of SYNONYM and friends, there is an argument that the
unique token for a word is the name token.

Regardless of nt or xt, I see no way of dealing with NDCS words
without upgrading the standard.

>
>To be on the safe side, I would redefine/fix some other words too, to
>exclude situations when "compile," produces unexpected stack effect when
>it's applied to some xt.
>
>Nothing standard word should return such xt for which "compile,"
>produces unexpected stack effect.

This cannot be done without tools to handle NDCS words.

[Stephen Pelc]

On Mon, 10 May 2021 14:16:25 -0700 (PDT), P Falth
<peter....@gmail.com> wrote:

>I do provide a FIND in my systems. In lxf it is not possible to do a user defined
>interpreter with it, for 2 reasons. It is not state-smart. It only returns the
>interpretation XT. Words like IF with no defined interpretation semantics
>return an xt that print an error message and aborts. If needed i could change this
>but so far has not needed it.

VFX works the same way.

dis if
IF
( 0001A0D0 E803F2FFFF ) CALL 000192D8 NOINTERP
( 0001A0D5 C3 ) RET/NEXT
( 6 bytes, 2 instructions )
ok

>My lxf64 is still single-xt, FIND works just fine there. There is no requirement
>that find must be implemented by NAME>* methods

Same for VFX.

>I do not understand this. Any word that has an XT could be ticked, or?

For VFX, all words have xts which are accessible by the user.

I am coming round to replacing FIND with a word such as:
: Search-Context \ c-addr len -- 0 | xt 1 | xt -1
\ *G Perform the *\fo{SEARCH-WORDLIST} operation on all wordlists
\ ** within the search order. Returns -1 for a "normal" word,
\ ** +1 for an *\fo{IMMEDIATE} word and +2 for an NDCS word.

If the world changes yet again we could modify SEARCH-CONTEXT to
: Search-Context \ c-addr len -- 0 | xt class

where a word's class is
0 unknown
-1 normal
+1 IMMEDIATE
+2 NDCS

[P Falth]

in my system code, data and header lives in different memory regions.
For this reason nt will always be different from xt.

For me it is also important that execute just makes a call to the address
represented by the xt. I want to be able to execute code in my forth image as
well as in a loaded library or directly in the kernel.

> Or at least FIND should return something like xt of
> [: true abort" not implemented" ;]
> and n value 1, or just n value 0, when FIND is applied to the "IF"
> string in compilation state, and ['] IF with n value 1 — in
> interpretation state.

Today find "IF" will return the the xt of [: true abort" compile only" ;]
in both interpret and compile state.

I could of course make IF reverse immediate, that is have as execution
semantics its compilation semantics and return the immediate flag.
That would make at least IF and friends work.

If I chose to implement a state-smart FIND as you suggest can it also return
different flags in interpretation and compilation state?

To efficiently compile a constant I need the xt of the constant and the xt of literal
Could I return in compile state for a constant a temporary xt that points
to code that calls the constant and then calls literal?

AS mentioned above I value an execute that can be applied to any address of code

> > I am not there yet. Now I can call native code from the token code. But I also want to
> > do the reverse.
>
>
>
> >
> >>>> Otherwise FIND cannot be implemented via these words.
> >>>
> >>> There is no requirements to implement FIND with these words.
> >> For back-compatibility, the system should still provide FIND for a
> >> while. And it should be clear how to implement FIND via new methods in
> >> the case when a system doesn't provide FIND (to use programs/libraries
> >> that rely on FIND).
> >
> > I do provide a FIND in my systems.
> When you propose an alternative to FIND, it's not about your system.
> It's about the standard and the Forth ecosystem.
> > In lxf it is not possible to do a user defined
> > interpreter with it, for 2 reasons. It is not state-smart. It only returns the
> > interpretation XT. Words like IF with no defined interpretation semantics
> > return an xt that print an error message and aborts. If needed i could change this
> > but so far has not needed it.
> As I understand, the second reason is that lxf cannot provide an xt that
> identifies the following execution semantics:
> [: postpone if ;]

That is right via FIND it can not do that. As explained above I can change IF to
provide that xt from FIND

Another proposal could be to specify the ES for these words to be the CS

BR
Peter

> --
> Ruvim

[Ruvim]

On 2021-05-11 18:50, Stephen Pelc wrote:
> On Fri, 7 May 2021 14:12:24 +0300, Ruvim <ruvim...@gmail.com>
> wrote:
>
>>> In VFX, some special words are NDCS but not IMMEDIATE. The equivalent
>>> of COMPILE, for such words is NDCS,
>>>

[...]

>> Taking into account that "ndcs," just performs another xt:
>>
>> : (ndcs,) \ i*x xt -- j*x
>> >code-len @ execute \ execute NDCS/IMMEDIATE word
>> ;
>>
>> Why not return this xt by "find" ?
>>
>> : find ( c-addr -- c-addr 0 | xt -1 | xt 1 )
>> find dup 0= if exit then drop ( xt1 ) dup
>> dup ndcs? if >code-len @ 1 else
>> dup immediate? if 1 else -1 then then ( xt1 xt2 n )
>> state @ if rot else swap then drop
>> ;
>>
>> After this fix all my tests work as expected.
>
> The fix is the sensible one, which is to throw away IMMEDIATE and
> let FIND indicate NDCS and normal.

Yes. It just doesn't distinguish immediate and other NDCS words.
But a program can still distinguish them in some cases.

Perhaps a better variant for VFX

: find ( c-addr -- c-addr 0 | xt -1 | xt 1 )
find dup 0= if exit then drop ( xt1 )

dup immediate? if 1 exit then
dup ndcs? 0= if -1 exit then
state @ if >code-len @ then 1
;

(to explicitly prefer primary xt in the case of an immediate word)

> However, on some systems this will be a code breaker.

A system is free to use any system-specific things.

If you mean that some programs can be incompatible with such FIND —
probably such a program is not standard since it doesn't take into
account that FIND may return the different xt in the different STATE.

> Personally, and for MPE, I love the original idea of an xt that it
> be the single point of contact between the user and the word. An
> xt can be used for many purposes other than compilation and
> interpretation, e.g. LOCATE XREF and so on.

I don't see how can a user use an xt with LOCATE or XREF

Nevertheless, it looks like in my edition above FIND still returns usual
system xts.

And the following program

: ]execute[ ] execute postpone [ ;

bl word if ' find ]execute[ . ( xt-if-comp )

also returns a usual xt.

You can say that ">name" does not work for this xt. But it isn't
obligated. ">name" also doesn't work for anonymous definitions, e.g.:

:noname 123 . ; >name .name

> Use of xt in this way requires that all words have a primary or
> interpretation action, which the xt represents.
>
> Your fix breaks this and I suppose makes it non-standard.

How does my fix break this?

All words in VFX still have a primary action that an xt represents.

> Because of SYNONYM and friends, there is an argument that the
> unique token for a word is the name token.

nt=xt is also a possible approach.

> Regardless of nt or xt, I see no way of dealing with NDCS words
> without upgrading the standard.

Could you please clarify, what cannot a standard program do without
special words to handle NDCS words?



--
Ruvim

[Doug Hoffman]

On 5/11/21 12:42 AM, dxforth wrote:

> I couldn't find a single app I'd written that used FIND

I have an OOP extension that relies on FIND for knowing if a message
(name) is already defined. The user of the extension needs this
functionality as he/she writes new classes with their message names.

-Doug

[Hugh Aguilar]

On Tuesday, May 11, 2021 at 9:16:23 AM UTC-7, Stephen Pelc wrote:
> For VFX, all words have xts which are accessible by the user.

This is not compatible with gForth that has words such as semicolon that
do not have XTs and for tick will abort with a bizarre error message
(FIND also fails, IIRC).

Also, your FIND is not compatible with SwiftForth because it says that words
such as IF are non-immediate and SwiftForth says that these words are immediate.

All of this ambiguity in ANS-Forth is fixed by the disambiguifiers.
Why don't you just tell the truth and admit this?
You lied about this previously:
https://groups.google.com/g/comp.lang.forth/c/T-yYkpVwYew/m/C6uvd8djAgAJ
Do you intend to continue to lie about this forever?
Eventually the truth catches up to you!

[Ruvim]

On 2021-05-11 13:51, Anton Ertl wrote:
[...]

> 3) We have discussed POSTPONE repeatedly over the last
> quarter-century, and I have established repeatedly that according to
> the text of Forth-94 and Forth-2012
>
> : foo postpone . ;
> : bar [ foo ] ;
> 1 bar \ prints "1"
>
> conforms to these standards.

According to the only wording in the specification for POSTPONE this
code conforms the standard, and it should print "1".

But according to the official TC reply to RFI Q99-027, it's not
guaranteed that performing "foo" in interpretation state performs
compilation semantics for "." (dot), and then this code isn't compliant.

>> Is there a demonstrated need that it should
>> work that would justify amending or clarifying ANS' ruling?
>
> There is no need to amend or clarify.
> The text of the standard is clear.

But RFI Q0009 and RFI Q99-027 demonstrate that clarifications were
needed in regard to POSTPONE. It means that the text of the standard is
not enough clear.



--
Ruvim

[Ruvim]

On 2021-05-11 19:46, P Falth wrote:
> On Tuesday, 11 May 2021 at 16:05:10 UTC+2, Ruvim wrote:
>> On 2021-05-11 00:16, P Falth wrote:
>>> On Monday, 10 May 2021 at 20:34:52 UTC+2, Ruvim wrote:
>>>> On 2021-05-06 18:23, P Falth wrote:
>>>>> On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:

[...]

>>> For some
>>> words ( normal words, created words) at compiletime was also needed
>>> the interpretation xt. FIND could not give me this. From this was born
>>> FIND-INTERPRET and FIND-COMPILE. They do what I needed!
>>
>> I see. It's OK for internal or system-specific purposes.
>>
>> But then for compatibility you could make xt=nt (and then it will be a
>> sing-xt system from the user's point of view).
>
> in my system code, data and header lives in different memory regions.
> For this reason nt will always be different from xt.

I mean, from the user's point of view.

In this approach a system itself uses raw xts (machine code references)
under the hood. But "FIND", "TRAVERSE-WORDLIST", "FIND-NAME", Tick —
any of them returns the same value for given word (i.e., the xt from
Tick is equal to the nt from FIND-NAME for any given word). Then for
anonymous definitions a header should be created too.

> For me it is also important that execute just makes a call to the address
> represented by the xt. I want to be able to execute code in my forth image as
> well as in a loaded library or directly in the kernel.

Since this code is not portable anyway, it could use special set of
words, i.e., special CALL-RAW (or alike), etc.

>> Or at least FIND should return something like xt of
>> [: true abort" not implemented" ;]
>> and n value 1, or just n value 0, when FIND is applied to the "IF"
>> string in compilation state, and ['] IF with n value 1 — in
>> interpretation state.
>
> Today find "IF" will return the the xt of [: true abort" compile only" ;]
> in both interpret and compile state.

Well, this message is just quite confusing when "IF" is actually
processed in compilation state.

> I could of course make IF reverse immediate, that is have as execution
> semantics its compilation semantics and return the immediate flag.

"FIND" is not obligated to return in compilation state an xt that
identifies execution semantics for the word in argument.

But if you do that, then let (1)
' IF EXECUTE
be equivalent to
IF
in interpretation state, to meet the common expectations.

And then, either IS for "IF" will be to perform CS for "IF", or the ES
will be STATE-dependent — any variant is possible.


(1) This code is not standard since ticking "IF" is ambiguous, and an
implementer is free to provide any behavior. But it's better if it meets
the common pattern.

> That would make at least IF and friends work.

Yes, it makes the system compliant with A.6.2.0945 and possibly updated
specification for FIND.

> If I chose to implement a state-smart FIND as you suggest can it also return
> different flags in interpretation and compilation state?

Yes, it may.

For example, for "DUP" it may return -1 in interpretation state (since
the returned xt is the execution token for "DUP"), and 1 in compilation
state (since the returned xt is for a definition that compiles optimized
code for "DUP").

For "EXIT" it may return 1 in interpretation state (to report an error),
and -1 in compilation state (since it's compiled by "COMPILE,").

For "IF" it may return 0 in interpretation state (not found), and 1 in
compilation state (since the returned xt is for a definition that
performs compilation semantics for "IF").

> To efficiently compile a constant I need the xt of the constant and the xt of literal
> Could I return in compile state for a constant a temporary xt that points
> to code that calls the constant and then calls literal?

In all the cases, when some data objects are transient, the standard
explicitly declares this fact, and specifies lifetime duration.

For example, see FILE 'S"', sections 11.3.4, 3.3.3.6.

So, if the lifetime for an object is not specified, then it's not less
than the program (or system) running. Or at least, a program should not
be able to detect that an object is invalid (i.e., a system may use
garbage collection under the hood).



An object code for a constant (regardless whether it's a machine code or
a threaded code) is very simple, and usually it can be easily detected.
I.e., having an xt, you can detect whether it's a constant, and if yes,
get it's value (a number). Having the value, you can efficiently compile it.

Actually, it's a peephole optimization that "COMPILE," can perform.



So, I see three options for your case that are compliant to the standard:

A. Introduce peephole optimization inside "COMPILE,".

B. Make a high-level xt be equal to an nt (then "COMPILE," will know
from the header that it's a constant).

C. Generate quotations (also known as "thunk") on the fly (with or
without garbage collection).



[...]

>>>>>> And what is a way to get xt of an ordinary word, that identifies the
>>>>>> execution semantics for this word?
>>>>>
>>>>> ' and ['] are still available for this.
>>>> They cannot be applied to any word. They cannot be used to words of
>>>> unknown kind.
>>>
>>> I do not understand this. Any word that has an XT could be ticked, or?
>>>
>> Tick returns xt that identifies execution semantics (ES) for the name in
>> argument, but ES are not specified for many standard words.
>>
>> Therefore, 4.1.2 says that an ambiguous condition exists if a program
>> attempts to obtain the execution token of a definition with undefined
>> interpretation semantics (for all such cases ES are either unspecified,
>> or cannot be provided in many implementations).
>>
>> Concerning other words with unspecified ES, it says that applying Tick
>> to the word is ambiguous, individually for each such word except FILE S"
>> and FILE S\".
>>
>> My proposal is to remove individual cases and declare the single rule:
>> an ambiguous condition exists if a program attempts to obtaining the
>> execution token of a definition with non default interpretation semantics.
>>
>> NB: my proposal of FIND clarification explicitly specifies in what case
>> the returned xt identifies ES for the name/word in argument.

>
> Another proposal could be to specify the ES for these words to be the CS

I can guess, it unnecessarily constrains implementations.

Also, taking into account that a system is allowed to provide IS for
such words (see A.3.4.3.2),

" ' X EXECUTE " will *not* be equivalent to " X " in interpretation
state for such word X, that is quite unexpected.


--
Ruvim

[Ruvim]

On 2021-05-12 12:10, Ruvim wrote:
> On 2021-05-11 19:46, P Falth wrote:

[...]

>> To efficiently compile a constant I need the xt of the constant and
>> the xt of literal
>> Could I return in compile state for a constant a temporary xt that points
>> to code that calls the constant and then calls literal?
>
> In all the cases, when some data objects are transient, the standard
> explicitly declares this fact, and specifies lifetime duration.
>
> For example, see FILE 'S"', sections 11.3.4, 3.3.3.6.
>
> So, if the lifetime for an object is not specified, then it's not less
> than the program (or system) running. Or at least, a program should not
> be able to detect that an object is invalid (i.e., a system may use
> garbage collection under the hood).
>
>
>
> An object code for a constant (regardless whether it's a machine code or
> a threaded code) is very simple, and usually it can be easily detected.
> I.e., having an xt, you can detect whether it's a constant, and if yes,
> get it's value (a number). Having the value, you can efficiently compile
> it.
>
> Actually, it's a peephole optimization that "COMPILE," can perform.
>
>
>
> So, I see three options for your case that are compliant to the standard:
>
>   A. Introduce peephole optimization inside "COMPILE,".
>
>   B. Make a high-level xt be equal to an nt (then "COMPILE," will know
> from the header that it's a constant).
>
>   C. Generate quotations (also known as "thunk") on the fly (with or
> without garbage collection).

D. Generate the static quotation when a constant is created.


--
Ruvim

[Anton Ertl]

Actually neither of these RFIs states that the text of the standard is
unclear. On the contrary, Q0009 states that it is clear:

<http://forth.sourceforge.net/std/dpans/q0009.htm>:
|Nothing in the standard prevents you executing words with compilation
|semantics from POSTPONE in interpretation state (it does not say that
|it is ambiguous, so it looks like it is defined through the standard).

The reason for these RFIs is that enough people made claims that did
not agree with the text of the standard that the people who asked the
question wanted an official statement by the TC:

<http://forth.sourceforge.net/std/dpans/q0009.htm>:
|IMHO the confusion about this topic has reached a point where I need
|an official statement.
[...]
|Opinions: Elizabeth Rather says that compilation semantics requires
|to be in compilation state.

<http://forth.sourceforge.net/std/dpans/a99-027.txt>:
|Recent discussions on comp.lang.forth indicate there is a body of opinion
|which holds that one or both the following are implied by the ANSI Forth
|Standard:
|
| 1. It is an ambiguous condition for a program to perform compilation
|semantics in the interpretation state.
|
| 2. It is an ambiguous condition for a program to append semantics to the
|current definition in the interpretation state.

The TC never answered Q0009. The TC answered Q99-027 without
referring to the document at all, only to the TC's intent; instead of
referring to the document, they stated that they have "plans to modify
its terminology in an effort to avoid future misinterpretations of
this sort [i.e., of TC intent]." They never did. Nobody has proposed
such a modification for Forth-200x (not before the release of
Forth-2012 and not since). With every year of inaction the relevance
of a99-027 became less, and with the release of Forth-2012 its
relevance vanished completely.

[P Falth]

On Wednesday, 12 May 2021 at 11:10:40 UTC+2, Ruvim wrote:
> On 2021-05-11 19:46, P Falth wrote:
> > On Tuesday, 11 May 2021 at 16:05:10 UTC+2, Ruvim wrote:
> >> On 2021-05-11 00:16, P Falth wrote:
> >>> On Monday, 10 May 2021 at 20:34:52 UTC+2, Ruvim wrote:
> >>>> On 2021-05-06 18:23, P Falth wrote:
> >>>>> On Thursday, 6 May 2021 at 16:30:54 UTC+2, Ruvim wrote:
> [...]
> >>> For some
> >>> words ( normal words, created words) at compiletime was also needed
> >>> the interpretation xt. FIND could not give me this. From this was born
> >>> FIND-INTERPRET and FIND-COMPILE. They do what I needed!
> >>
> >> I see. It's OK for internal or system-specific purposes.
> >>
> >> But then for compatibility you could make xt=nt (and then it will be a
> >> sing-xt system from the user's point of view).
> >
> > in my system code, data and header lives in different memory regions.
> > For this reason nt will always be different from xt.
> I mean, from the user's point of view.
>
> In this approach a system itself uses raw xts (machine code references)
> under the hood. But "FIND", "TRAVERSE-WORDLIST", "FIND-NAME", Tick —
> any of them returns the same value for given word (i.e., the xt from
> Tick is equal to the nt from FIND-NAME for any given word). Then for
> anonymous definitions a header should be created too.

I know that approach and I do not like it. A header for a noname definition
just sound like the wrong way to go

There is also the option to just compile a call to the constant.
Find provides the xt and -1 this xt can then be compiled with compile,

The code generated will be less efficient but it will work.

I have looked thru all my sources. FIND is never used. in other peoples
sources I have I see it used some time to define [DEFINED] . For
this purpose my state dump find will work just fine.
This also means that I can test a state-smart find without breaking my system!

But what will I actually gain from it?

BR
Peter

[Ruvim]

I agree with you.
But this approach is standard-compliant too, so it should be mentioned.





[...]

>> So, I see three options for your case that are compliant to the standard:
>>
>> A. Introduce peephole optimization inside "COMPILE,".
>>
>> B. Make a high-level xt be equal to an nt (then "COMPILE," will know
>> from the header that it's a constant).
>>
>> C. Generate quotations (also known as "thunk") on the fly (with or
>> without garbage collection).
>
> There is also the option to just compile a call to the constant.

(I meant the only options that support optimization)

> Find provides the xt and -1 this xt can then be compiled with compile,
>
> The code generated will be less efficient but it will work.

Yes, it's also the simplest variant.

Something similar is described in A.6.2.0945 in length, and it also says:

| "COMPILE," might not generate code that is
| as efficient as normally compiled code.

> I have looked thru all my sources. FIND is never used. in other peoples
> sources I have I see it used some time to define [DEFINED] . For
> this purpose my state dump find will work just fine.
> This also means that I can test a state-smart find without breaking my system!
>
> But what will I actually gain from it?

As an implementer — nothing direct gain.

The users of your system can use some portable libraries and programs
that rely on FIND. Or the Forth users in general can choose your system
in more cases.

For example, at the moment I cannot use your system, and one of the
issues is that a user-defined text interpreter doesn't work.



--
Ruvim

[P Falth]

There is another problem waiting for a user defined text interpreter and that
is locals. They will not be found by FIND ( or find-name, search-wordlist etc)
and they do not have an xt. They are stored in a specific table that is setup
and destroyed at : and ;

BR
Peter

>
> --
> Ruvim

[dxforth]

On 11/05/2021 20:49, Ruvim wrote:

Probably those within Standard Forth didn't need it either, because if
FIND wasn't fit for purpose it would surely have been fixed by now.

[Stephen Pelc]

On Thu, 13 May 2021 00:08:37 +1000, dxforth <dxf...@gmail.com> wrote:

>Probably those within Standard Forth didn't need it either, because if
>FIND wasn't fit for purpose it would surely have been fixed by now.

Forth has changed since the 1990s and flwas in the 1990s documents
are being revealed.