?EXEC

[Albert van der Horst]

I find places where ?EXEC is used. This gives an error because a word
if a word is executed while it never should.

Examples are CODE and END-CODE in an assembler, but does this make
sense?

: CODE ?EXEC .... ;
: END-CODE ?EXEC .... ;

By analogy I see I defined my classes like this:
: class ?EXEC ... ;
: end-class ?EXEC ... ;

I find it hard to wrap my head around this. In what kind of situation
would this error trigger?
I would be equally happy with some other realistic example.

Groetjes Albert


--
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

[Ed]

Albert van der Horst wrote:
> I find places where ?EXEC is used. This gives an error because a word
> if a word is executed while it never should.
>
> Examples are CODE and END-CODE in an assembler, but does this make
> sense?
>
> : CODE ?EXEC .... ;
> : END-CODE ?EXEC .... ;
>
> By analogy I see I defined my classes like this:
> : class ?EXEC ... ;
> : end-class ?EXEC ... ;
>
> I find it hard to wrap my head around this. In what kind of situation
> would this error trigger?
> I would be equally happy with some other realistic example.

Possibly there is none.

?EXEC was a companion function to ?COMP. These were part
of the "compiler security" package introduced by Fig-Forth and
copied by just about everyone (except notably Forth Inc.)

Whether Forth needed the level compiler security provided by
Fig-Forth was questioned in the article "Compiler Security"
G.Shaw, FD 3/1.

I note in my own sources I use ?EXEC in CODE END-CODE
LABEL - which were all IMMEDIATE. I can't recall ever using or
needing this feature.

[BruceMcF]

On Jun 15, 4:40 pm, Albert van der Horst <alb...@spenarnc.xs4all.nl>
wrote:

> I find places where ?EXEC is used. This gives an error because a word
> if a word is executed while it never should.

"because a word if a word is executed while it never should." ?

... seems to be mangled. What is the actual condition that is supposed
to trigger the error? Ed notes that its a fig-Forthism, and its been
several decades since I knew the fig Forth model in any detail.

[Albert van der Horst]

In article <b445b94c-64c5-4773...@h10g2000yqn.googlegroups.com>,
BruceMcF <agi...@netscape.net> wrote:
>On Jun 15, 4:40=A0pm, Albert van der Horst <alb...@spenarnc.xs4all.nl>

>wrote:
>> I find places where ?EXEC is used. This gives an error because a word
>> if a word is executed while it never should.
>
>"because a word if a word is executed while it never should." ?
>
>... seems to be mangled. What is the actual condition that is supposed
>to trigger the error? Ed notes that its a fig-Forthism, and its been
>several decades since I knew the fig Forth model in any detail.

Sorry. I put too much trust in my spell checker ;-)
Maybe just this:
"
SEE ?EXEC

: ?EXEC
STATE @ 12 ?ERROR
;
"
?ERROR is THROW with some decoration.

>
>>
>> Examples are CODE and END-CODE in an assembler, but does this make
>> sense?
>>
>> : CODE ?EXEC .... ;
>> : END-CODE ?EXEC .... ;
>>
>> By analogy I see I defined my classes like this:
>> : class ?EXEC ... ;
>> : end-class ?EXEC ... ;
>>
>> I find it hard to wrap my head around this. In what kind of situation
>> would this error trigger?
>> I would be equally happy with some other realistic example.
>
>

I just discovered that having ?COMP in IF and DO is useful.

I had a bug in a class definition because I forgot that the build
code is run in interpret mode. Took me an hour to realize what
was going on. Maybe I should load the interpreted IF's and DO's
standard with the class tool, but certainly I will add the ?COMP.

[Ed]

Albert van der Horst wrote:

> ...

> I just discovered that having ?COMP in IF and DO is useful.
>
> I had a bug in a class definition because I forgot that the build
> code is run in interpret mode. Took me an hour to realize what
> was going on. Maybe I should load the interpreted IF's and DO's
> standard with the class tool, but certainly I will add the ?COMP.

How often do you make such errors :)

?COMP is a convenience that can alert one to silly errors
sooner, but like ?EXEC the times it is useful will be rare.
To this day SwiftForth has no ?COMP.

OTOH providing ?COMP is cheap and it turns out only 4
are necessary to cover all the standard structures IF ELSE
THEN AHEAD BEGIN WHILE REPEAT UNTIL DO LOOP
CASE etc.

[Anton Ertl]

Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>I just discovered that having ?COMP in IF and DO is useful.

No, it's the plague. It turns nicely working words into STATE-smart
crap. ?COMP (like all STATE-smart words) tests STATE at run-time,
whereas what is wanted is to have a check at parsing time (and that's
the same for all STATE-smart words).

If you want to protect against using IF and DO interpretively, add a
compile-only flag to the header, and let the text interpreter check
it. For more complicated cases (words that can be used
interpretively, but behave differently), there are a number of good
implementation approaches (checking STATE at run-time is not one of
them).

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2012: http://www.euroforth.org/ef12/

[Anton Ertl]

Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:

>I find places where ?EXEC is used. This gives an error because a word
>if a word is executed while it never should.
>
>Examples are CODE and END-CODE in an assembler, but does this make
>sense?
>
>: CODE ?EXEC .... ;
>: END-CODE ?EXEC .... ;

Hardly. It makes a little more sense if CODE and END-CODE are
immediate. I guess the idea is to protect the porgrammer against

: foo
... \ missing ";"

code bar
...

And of course in fig-Forth also ":" was immediate and contained a
?EXEC.

For defining words we usually get an error when the name of the word
to be defined is seen; ok, the error is not as descriptive as the one
produced by ?EXEC, but programmers learn to interpret it pretty fast.

And the disadvantages are pretty bad: First, if you want to include
CODE (or ":") in another word, you have to use [COMPILE] or POSTPONE
(note that standard CODE and : don't need this). Worse, that word
becomes STATE-smart; if you tick it and then happen to EXECUTE the xt
in compile STATE, you get an error, pretty removed from the cause of
the whole thing.

Overall, ?EXEC is a bad idea, like ?COMP and STATE-smart words in
general.

[BruceMcF]

On Jun 16, 11:01 pm, Albert van der Horst <alb...@spenarnc.xs4all.nl>
wrote:
> In article <b445b94c-64c5-4773-be56-ee8c75f24...@h10g2000yqn.googlegroups.com>,

>
> BruceMcF  <agil...@netscape.net> wrote:
> >On Jun 15, 4:40=A0pm, Albert van der Horst <alb...@spenarnc.xs4all.nl>
> >wrote:
> >> I find places where ?EXEC is used. This gives an error because a word
> >> if a word is executed while it never should.
>
> >"because a word if a word is executed while it never should." ?
>
> >... seems to be mangled. What is the actual condition that is supposed
> >to trigger the error? Ed notes that its a fig-Forthism, and its been
> >several decades since I knew the fig Forth model in any detail.
>
> Sorry. I put too much trust in my spell checker ;-)
> Maybe just this:
> "
> SEE ?EXEC
>
> : ?EXEC
> STATE   @   12 ?ERROR
> ;
> "
> ?ERROR is THROW with some decoration.

Aha, so its just the opposite of ?COMP with its own distinctive error
return.

I think I'm with Anton. If I was trying to implement a small Forth in
some retro system or other, rather than ?COMP I'd have a compile-only
bit and blind the interpreter to compile-only words in interpret mode.

[Elizabeth D. Rather]

On 6/17/12 2:52 AM, Anton Ertl wrote:
> Albert van der Horst<alb...@spenarnc.xs4all.nl> writes:
>> I find places where ?EXEC is used. This gives an error because a word
>> if a word is executed while it never should.
>>
>> Examples are CODE and END-CODE in an assembler, but does this make
>> sense?
>>
>> : CODE ?EXEC .... ;
>> : END-CODE ?EXEC .... ;
>
> Hardly. It makes a little more sense if CODE and END-CODE are
> immediate. I guess the idea is to protect the porgrammer against
>
> : foo
> ... \ missing ";"
>
> code bar
> ...
>
> And of course in fig-Forth also ":" was immediate and contained a
> ?EXEC.

Baffled. By what logic should CODE, END-CODE, and : be immediate?

...

> Overall, ?EXEC is a bad idea, like ?COMP and STATE-smart words in
> general.

Agreed. All of these fall in the category of cute tricks that you *can*
do in Forth, but are really a poor idea in serious production code.
Newbies in Forth like to explore "uncharted waters". That's ok, but it's
not how serious code is written.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[Albert van der Horst]

In article <2012Jun1...@mips.complang.tuwien.ac.at>,

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>>I just discovered that having ?COMP in IF and DO is useful.
>
>No, it's the plague. It turns nicely working words into STATE-smart
>crap. ?COMP (like all STATE-smart words) tests STATE at run-time,
>whereas what is wanted is to have a check at parsing time (and that's
>the same for all STATE-smart words).

In this case the problem happens at parsing time, so the check is
done at the right time, i.e. while interpreting. Think about it.
?EXEC can never be a proper error message. ?COMP can never hinder
postponing something. Has nothing to do with state smartness.

When was the last time you did
: aaap .... [ IF ] .... ;
while you could have done
: aaap .... IF .... ;

>
>If you want to protect against using IF and DO interpretively, add a
>compile-only flag to the header, and let the text interpreter check
>it. For more complicated cases (words that can be used
>interpretively, but behave differently), there are a number of good
>implementation approaches (checking STATE at run-time is not one of
>them).

I have a better solution: I can use IF and DO interpretively,
after I say WANT NEW-IF .

Making the text interpreter smarter is the opposite way from where
I want to go.
Remember, ciforth's interpreter doesn't know about numbers, only
prefixes. It is simpler and more flexible that way.

But ?LOADING and ?EXEC I have removed. So I have the luxury that
2 words may go in the kernel to replace them. Maybe I'll finally add
NIP . 68 of my 199 euler Forth files contain WANT NIP.


>
>- anton

[Ed]

Elizabeth D. Rather wrote:
> On 6/17/12 2:52 AM, Anton Ertl wrote:
> > Albert van der Horst<alb...@spenarnc.xs4all.nl> writes:
> >> I find places where ?EXEC is used. This gives an error because a word
> >> if a word is executed while it never should.
> >>
> >> Examples are CODE and END-CODE in an assembler, but does this make
> >> sense?
> >>
> >> : CODE ?EXEC .... ;
> >> : END-CODE ?EXEC .... ;
> >
> > Hardly. It makes a little more sense if CODE and END-CODE are
> > immediate. I guess the idea is to protect the porgrammer against
> >
> > : foo
> > ... \ missing ";"
> >
> > code bar
> > ...
> >
> > And of course in fig-Forth also ":" was immediate and contained a
> > ?EXEC.
>
> Baffled. By what logic should CODE, END-CODE, and : be immediate?

Presumably so that you don't attempt:

: foo ... code foo2 ... end-code ... ;

> ...
> > Overall, ?EXEC is a bad idea, like ?COMP and STATE-smart words in
> > general.
>
> Agreed. All of these fall in the category of cute tricks that you *can*
> do in Forth, but are really a poor idea in serious production code.
> Newbies in Forth like to explore "uncharted waters". That's ok, but it's
> not how serious code is written.

VFX is loaded with ?COMPs. Not "serious production code" ?

Until such time as someone can show an issue with using ?COMP
in IF ELSE THEN etc then how is it a problem? Similarly CODE is
non-portable so whether it's immediate or contains a ?EXEC affects
no-one but the user.

Most tools can be mis-used. We don't ban them because someone
has managed to cut their finger.

[Elizabeth D. Rather]

On 6/17/12 7:06 PM, Ed wrote:
> Elizabeth D. Rather wrote:
>> On 6/17/12 2:52 AM, Anton Ertl wrote:
>>> Albert van der Horst<alb...@spenarnc.xs4all.nl> writes:
>>>> I find places where ?EXEC is used. This gives an error because a word
>>>> if a word is executed while it never should.
>>>>
>>>> Examples are CODE and END-CODE in an assembler, but does this make
>>>> sense?
>>>>
>>>> : CODE ?EXEC .... ;
>>>> : END-CODE ?EXEC .... ;
>>>
>>> Hardly. It makes a little more sense if CODE and END-CODE are
>>> immediate. I guess the idea is to protect the porgrammer against
>>>
>>> : foo
>>> ... \ missing ";"
>>>
>>> code bar
>>> ...
>>>
>>> And of course in fig-Forth also ":" was immediate and contained a
>>> ?EXEC.
>>
>> Baffled. By what logic should CODE, END-CODE, and : be immediate?
>
> Presumably so that you don't attempt:
>
> : foo ... code foo2 ... end-code ... ;

What's to prevent...

code foo ... : poo ... ; end-code ;

Since CODE isn't specified to set STATE, it's legal. Equally silly,
equally unlikely.

>> ...
>>> Overall, ?EXEC is a bad idea, like ?COMP and STATE-smart words in
>>> general.
>>
>> Agreed. All of these fall in the category of cute tricks that you *can*
>> do in Forth, but are really a poor idea in serious production code.
>> Newbies in Forth like to explore "uncharted waters". That's ok, but it's
>> not how serious code is written.
>
> VFX is loaded with ?COMPs. Not "serious production code" ?

Everybody gets to draw his/her own line as to what's too foolish to warn
against.

> Until such time as someone can show an issue with using ?COMP
> in IF ELSE THEN etc then how is it a problem? Similarly CODE is
> non-portable so whether it's immediate or contains a ?EXEC affects
> no-one but the user.
>
> Most tools can be mis-used. We don't ban them because someone
> has managed to cut their finger.

I don't advocate banning ?COMP. I just think it's silly. And making CODE
etc. IMMEDIATE just so you can issue an error message in case of blatant
silliness is doubly silly.

As I've said elsewhere, I support the "intelligent programmer,
permissive compiler" theory of programming. I can think of several
reasons for using : and END-CODE in definitions, somewhat fewer for
CODE, but I see no advantage to making them immediate unless you fancy
yourself a programming nanny, in which case maybe Forth is not for you.

[Albert van der Horst]

In article <VqCdnXuTWa6dgUPS...@supernews.com>,

After the explanation of Anton Ertl I finally see how ?EXEC
could be -- marginally -- useful.

: test dothis dothat
X SWAP IF
qqq
THEN
DROP ;

: test2 ....

In this situation if you change the last line of test to
DROP
you will have `` : test2 '' compiled and this gives an
error of test2 not found/ Mysterious, but you learn to interpret
that message.
By making : IMMEDIATE it can signal this condition.
CODE aap ... \ END-CODE
CODE noot ... END-CODE
Same situation.

?COMP saves me from the following hazard:
_ _ \ #scores, #sides
class C
M: sides M; ,
M: maxscores M; DUP ,
M: scores SWAP CELLS + M;
HERE DUP >R OVER CELLS ALLOT R> SWAP CELLS ERASE
endclass

This defines a class with an integer field and an array of
integers field that is initialised to zero.
The building of the structure amounts to
: BUILD-C HERE >R , DUP ,
HERE DUP >R OVER CELLS ALLOT R> SWAP CELLS ERASE
;
This is compiled, but also interpreted immediately to
calculate the offsets.
I tend to forget that the following doesn't work:
_ _
class C
M: sides M; ,
M: scores SWAP CELLS + M;
0 DO 0 , LOOP
endclass

and it leads to mysterious errors.

If DO warned at compile time that would have been prevented.

Loading an interpreting DO facility is a solution.
>
>Cheers,
>Elizabeth

[Anton Ertl]

Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>In article <2012Jun1...@mips.complang.tuwien.ac.at>,
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>>>I just discovered that having ?COMP in IF and DO is useful.
>>
>>No, it's the plague. It turns nicely working words into STATE-smart
>>crap. ?COMP (like all STATE-smart words) tests STATE at run-time,
>>whereas what is wanted is to have a check at parsing time (and that's
>>the same for all STATE-smart words).
>
>In this case the problem happens at parsing time, so the check is
>done at the right time, i.e. while interpreting. Think about it.
>?EXEC can never be a proper error message. ?COMP can never hinder
>postponing something. Has nothing to do with state smartness.

Try <http://www.complang.tuwien.ac.at/forth/postponetest.fs> and see
how IF and DO with ?COMP are broken.

>When was the last time you did
>: aaap .... [ IF ] .... ;
>while you could have done
>: aaap .... IF .... ;

In a compiler where the text interpreter checks for compile-only words
(like Gforth) the first version would fail because IF is compile-only
and thus can only be compiled.

However, the issue is not

: aaap ... [ IF ] ... ;

but

: baz ... [ 3 5 gen-foo s" bla" gen-bar ] ... ;

where GEN-FOO contains, e.g., a "POSTPONE IF".

Yes, the ?COMP does not hinder POSTPONEing the IF, it just fails when
GEN-FOO runs. And it has everything to do with STATE-smartness: It
checks STATE at run-time, like any other STATE-smart word.

And yes, I have written production code where a code generator is
invoked in interpret state.

>>If you want to protect against using IF and DO interpretively, add a
>>compile-only flag to the header, and let the text interpreter check
>>it. For more complicated cases (words that can be used
>>interpretively, but behave differently), there are a number of good
>>implementation approaches (checking STATE at run-time is not one of
>>them).

...

>Making the text interpreter smarter is the opposite way from where
>I want to go.

If you don't want to go there, then don't. Just live without
recognizing interpretive uses of IF and DO. But if you want to
recognize them, do it properly. STATE-smartness (including ?COMP) is
both broken and more complicated than not checking at all, i.e., it's
a lose-lose solution.

[Coos Haak]

Op 15 Jun 2012 20:40:13 GMT schreef Albert van der Horst:

> I find places where ?EXEC is used. This gives an error because a word
> if a word is executed while it never should.
>
> Examples are CODE and END-CODE in an assembler, but does this make
> sense?
>
>: CODE ?EXEC .... ;
>: END-CODE ?EXEC .... ;
>
> By analogy I see I defined my classes like this:
>: class ?EXEC ... ;
>: end-class ?EXEC ... ;
>
> I find it hard to wrap my head around this. In what kind of situation
> would this error trigger?
> I would be equally happy with some other realistic example.
>
> Groetjes Albert
>
>
> --

I've looked at the Forth assemblers in volume 3 of Forth Dimensions.
William F. Ragdale's for the 6502 in issue 5 has immediate CODE END-CODE
and even for IF, ELSE, BEGIN, etc. and uses ?EXEC
John J. Cassady's for the 8080 in issue 6 has only CODE and END-CODE made
immediate with ?EXEC

I don't use immediate words in assembler and just removed ?EXEC from the
MetaForth. I sense no difference.

--
Coos

CHForth, 16 bit DOS applications
http://home.hccnet.nl/j.j.haak/forth.html

[Coos Haak]

Op Mon, 18 Jun 2012 17:47:07 +0200 schreef Coos Haak:

> I've looked at the Forth assemblers in volume 3 of Forth Dimensions.
> William F. Ragdale's for the 6502 in issue 5 has immediate CODE END-CODE
> and even for IF, ELSE, BEGIN, etc. and uses ?EXEC
> John J. Cassady's for the 8080 in issue 6 has only CODE and END-CODE made
> immediate with ?EXEC

immediate and uses ?EXEC

[Albert van der Horst]

That would actually be nice, because one concludes that we have
an advanced tool at our hands and I would insert
WANT NO-SECURITY NO-SECURITY
before including the file. This is a similar situation with
BEGIN .. WHILE .. WHILE .. REPEAT ..THEN
that triggers an error too.

Helas, that is not what happens.
What is much worse that I *do* have an interpreting IF and DO
that are state smart and can e.g. generate a table interpretively.
A cut and paste of my CRC screen:

5 \ Auxiliary table with values for single bytes.
6 CREATE CRCTable
7 100 0 DO I 8 0 DO
8 DUP >R 1 RSHIFT R> 1 AND IF CRC32_POLYNOMIAL XOR THEN
9 LOOP , LOOP

>
>And yes, I have written production code where a code generator is
>invoked in interpret state.

That sets me thinking.

>
>>>If you want to protect against using IF and DO interpretively, add a
>>>compile-only flag to the header, and let the text interpreter check
>>>it. For more complicated cases (words that can be used
>>>interpretively, but behave differently), there are a number of good
>>>implementation approaches (checking STATE at run-time is not one of
>>>them).
>...
>>Making the text interpreter smarter is the opposite way from where
>>I want to go.
>
>If you don't want to go there, then don't. Just live without
>recognizing interpretive uses of IF and DO. But if you want to
>recognize them, do it properly. STATE-smartness (including ?COMP) is
>both broken and more complicated than not checking at all, i.e., it's
>a lose-lose solution.

I feel that adding interpreted use of IF and DO is much more useful
then other smartness. I use &A instead of CHAR/[CHAR] A
and "AAAP" TYPE instead of ." AAAP" and .( AAAP)
but looking up whether to use CHAR or [CHAR] is an inconvenience
whereas IF and DO I find harder to dispense with.

Maybe I'm just an ISO-heretic. (What would Chuck Moore say?)

>
>- anton
>--
>M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
>comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
> New standard: http://www.forth200x.org/forth200x.html
> EuroForth 2012: http://www.euroforth.org/ef12/

[Elizabeth D. Rather]

On 6/18/12 8:29 AM, Albert van der Horst wrote:
> In article<2012Jun1...@mips.complang.tuwien.ac.at>,
...

>> If you don't want to go there, then don't. Just live without
>> recognizing interpretive uses of IF and DO. But if you want to
>> recognize them, do it properly. STATE-smartness (including ?COMP) is
>> both broken and more complicated than not checking at all, i.e., it's
>> a lose-lose solution.
>
> I feel that adding interpreted use of IF and DO is much more useful

> then other smartness. I use&A instead of CHAR/[CHAR] A

> and "AAAP" TYPE instead of ." AAAP" and .( AAAP)
> but looking up whether to use CHAR or [CHAR] is an inconvenience
> whereas IF and DO I find harder to dispense with.
>
> Maybe I'm just an ISO-heretic. (What would Chuck Moore say?)

Chuck Moore would say do whatever the job at hand requires and don't
bother with nagging nanny compilers.

[Bernd Paysan]

Anton Ertl wrote:
> Try <http://www.complang.tuwien.ac.at/forth/postponetest.fs> and see
> how IF and DO with ?COMP are broken.

I think this is a misunderstanding what "compilation semantics" could
possibly mean - it's a literal reading of the standard. Compilation
semantics is what the text interpreter does with a word it encounters,
which is

FIND 0< IF COMPILE, ELSE EXECUTE THEN

So what POSTPONE does is essentially (minus the self-reference, but I
assume this is the cross compiler's POSTPONE):

FIND SWAP POSTPONE LITERAL 0< IF POSTPONE COMPILE, ELSE POSTPONE EXECUTE
THEN

with some opportunity to optimize (lit+execute is call). POSTPONE was
introduced so that the implementer can choose to make some words
immediate or not.

For an immediate word, the compilation semantics is to EXECUTE that
word, for an non-immediate word, it is to COMPILE, this word. POSTPONE
embeds this into the current definition. Unfortunately, for an
immediate word, EXECUTE is also used to get the interpretation semantics
fo that word. The interpretation semantics of a word like IF is
undefined, and the only standard way IF has to find out if it's
interpretation or compilation semantics are requested is STATE.

postponetest.fs tests if the compiler uses non-standard features to
implement these things. You always argued that the standard would
mandate postponetest.fs to pass. It reads that way, but our recent
discussion about semantics has revealed that things are a bit more
complicated (or maybe let's say a bit more simplistic).

ANS Forth mandates that you implement CORE, but it also intents that
CORE is a good minimal set of words which can be used as tools for the
implementer himself. If you have dual-xt name tokens, a compile-only
flag, or an intelligent COMPILE,, you have no problems to implement
?COMP in a way that postponetest.fs passes. But the standard does not
mandate these extensions, they are quality-of-implementation issues (and
not having too many features is considered as quality of implementation
by some people in the Forth community).

--
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[Elizabeth D. Rather]

On 6/18/12 2:11 PM, Bernd Paysan wrote:
> Anton Ertl wrote:
>> Try<http://www.complang.tuwien.ac.at/forth/postponetest.fs> and see
>> how IF and DO with ?COMP are broken.
>

The question means, what is "broken"? Interpretive versions of IF and DO
are not illegal, those words are just not *guaranteed* to work in
interpret mode. "Interpretation semantics are undefined" means exactly
that: a standard program may not depend on any specif behavior in that
case. It's ok if it works. It's ok if it aborts. It's ok if it fails
silently.

...

>
> postponetest.fs tests if the compiler uses non-standard features to
> implement these things. You always argued that the standard would
> mandate postponetest.fs to pass. It reads that way, but our recent
> discussion about semantics has revealed that things are a bit more
> complicated (or maybe let's say a bit more simplistic).
>
> ANS Forth mandates that you implement CORE, but it also intents that
> CORE is a good minimal set of words which can be used as tools for the
> implementer himself. If you have dual-xt name tokens, a compile-only
> flag, or an intelligent COMPILE,, you have no problems to implement
> ?COMP in a way that postponetest.fs passes. But the standard does not
> mandate these extensions, they are quality-of-implementation issues (and
> not having too many features is considered as quality of implementation
> by some people in the Forth community).

In my view, the intent of the standard is to be very laissez-faire about
"undefined" features and "ambiguous conditions". It's ok to provide
certain features, but they aren't required. It's ok to write programs
that depend upon "undefined" features if you're willing to shoulder the
responsibility for using only systems that provide them. It's also ok to
issue error messages for options that you choose not to provide (or
not). So, things like ?COMP are neither required nor illegal, they are
matters of taste (even though some people think they're silly).

[Albert van der Horst]

In article <Z82dnTsD6awiVkLS...@supernews.com>,

Elizabeth D. Rather <era...@forth.com> wrote:

>On 6/18/12 2:11 PM, Bernd Paysan wrote:
>> Anton Ertl wrote:
>>> Try<http://www.complang.tuwien.ac.at/forth/postponetest.fs> and see
>>> how IF and DO with ?COMP are broken.
>>
>
>The question means, what is "broken"? Interpretive versions of IF and DO
>are not illegal, those words are just not *guaranteed* to work in
>interpret mode. "Interpretation semantics are undefined" means exactly
>that: a standard program may not depend on any specif behavior in that
>case. It's ok if it works. It's ok if it aborts. It's ok if it fails
>silently.

So if I was wary about a program being standard w.r.t. to IF
I could load a preamble.
-----------
: IF STATE @ 0=
ABORT" NONPORTABLE: interpretation semantics of IF is undefined"
POSTPONE IF ; IMMEDIATE
-----------

and it would only sift out nonportable programs.

If it triggers,I could try to find out whether I expect this particular
program to run on this particular implementation or modify the program
to behave as intended.

>Cheers,
>Elizabeth

[Anton Ertl]

Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>I feel that adding interpreted use of IF and DO is much more useful
>then other smartness.

If you want to add interpretation semantics for IF and DO, then go
ahead. But don't produce STATE-smart IF and DO; for simple cases they
work as desired, but for more complex cases they sometimes fail.
Instead, if you want that stuff, bite the bullet and implement one of
the schemes for doing that properly.

My feeling, though, is that in the balance it's a bad idea to have
such words. Not only do you have to implement one of the schemes
above, but you also have repercussions elsewhere (like the meaning of
xts and how to reify the interpretation and compilation semantics).

Your approach of using prefixes appears to be a very good alternative
for the two standard words (S" and TO) that have independent
interpretation and compilation semantics (one might have a prefix like
"->" instead of "TO", i.e., instead of "TO zoing" have "->zoing"); and
also for most other cases where people want such things. Now you want
to ruin it by complicating IF and DO. Sigh.

>Maybe I'm just an ISO-heretic. (What would Chuck Moore say?)

Ask him. However, we can also look at his Forth systems: If he ever
used STATE-smartness, it was a very long time ago. cmForth (late
1980s) used the separate compilation wordlist wordlist approach, which
could implement independent interpretation and compilation semantics
properly, but AFAIK he did not use it for that (certainly not for IF
and DO).

I am not very familiar with colorForth, but AFAIK it works like this:
you can use colours to execute the word right away (interpretation and
what would be immediate words in a conventional system), compile it,
or postpone it. But AFAIK IF is a word that you use in a definition
and that you execute right away.

[Anton Ertl]

Bernd Paysan <bernd....@gmx.de> writes:
>Anton Ertl wrote:
>> Try <http://www.complang.tuwien.ac.at/forth/postponetest.fs> and see
>> how IF and DO with ?COMP are broken.
>
>I think this is a misunderstanding what "compilation semantics" could
>possibly mean - it's a literal reading of the standard.

And in what way is this a misunderstanding? What do you suggest to
replace "reading the standard" with? Divine inspiration?

Anyway, even if we agree that we don't want to put the onus of
implementing S" and TO properly on all systems (i.e., such that they
pass postponetest.fs), it's a good idea for implementors to pass as
much of postponetest.fs as possible, because there is production code
out there that will fail on systems where certain postponetest.fs
tests fail.

And putting ?COMP in an IF and DO that pass postponetest.fs will make
them fail postponetest.fs.

>ANS Forth mandates that you implement CORE, but it also intents that
>CORE is a good minimal set of words which can be used as tools for the
>implementer himself. If you have dual-xt name tokens, a compile-only
>flag, or an intelligent COMPILE,, you have no problems to implement
>?COMP in a way that postponetest.fs passes.

What do you mean by that? ?COMP is a check of STATE at run-time. It
has this problem by nature. If you mean that on some systems I have
ways to do properly what people try (and fail) to achieve with ?COMP,
that's true, but that's not ?COMP. It's, e.g., Gforth's COMPILE-ONLY.

Here's what happens if I add ?COMP on a system that supports a
compile-only flag:

First, here's how it works without ?COMP:

gforth test/tester.fs test/coretest.fs postponetest.fs -e 'cr ." passed" cr bye'

[...]
passed


And here is the version with ?COMP:

gforth -e ': ?comp state @ 0= abort" running in interpret state" ; : if ?comp postpone if ; immediate' test/tester.fs test/coretest.fs postponetest.fs -e 'cr ." passed" cr bye'
[...]
in file included from *the terminal*:0
postponetest.fs:202: running in interpret state
{ : PIF1 [ POSTPONE-IF ] 123 THEN ; -> }
^^^^^^^^^^^

>But the standard does not
>mandate these extensions, they are quality-of-implementation issues (and
>not having too many features is considered as quality of implementation
>by some people in the Forth community).

In particular, one should not include misfeatures like ?COMP. And
that's not just a quality-of-implementation issue.

[Andrew Haley]

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:

> However, we can also look at [Chuck Moore's] Forth systems: If he

> ever used STATE-smartness, it was a very long time ago. cmForth
> (late 1980s) used the separate compilation wordlist wordlist
> approach, which could implement independent interpretation and
> compilation semantics properly, but AFAIK he did not use it for that
> (certainly not for IF and DO).

cmFORTH did use the wordlists for independent interpretation and
compilation semantics, but indeed not for IF and DO. There were
compiling versions of a bunch of things, like this:

FORTH : BINARY ( n) CREATE , DOES R> @ ?CODE @ DUP
... lots ... ;

COMPILER A840 BINARY + AA40 BINARY XOR A640 BINARY OR
AC40 BINARY - A240 BINARY AND A440 BINARY SWAP-

FORTH

... much later ...

: SWAP SWAP ; : OVER OVER ;
: DUP DUP ; : DROP DROP ;
: XOR XOR ; : AND AND ;
: OR OR ;
: + + ; : - - ;

Andrew.

[Anton Ertl]

"Elizabeth D. Rather" <era...@forth.com> writes:
>On 6/18/12 2:11 PM, Bernd Paysan wrote:
>> Anton Ertl wrote:
>>> Try<http://www.complang.tuwien.ac.at/forth/postponetest.fs> and see
>>> how IF and DO with ?COMP are broken.
>>
>
>The question means, what is "broken"?

Very simple: If I have an IF that passes postponetest.fs, and then
redefine it to call ?COMP (": if ?comp postpone if ; immediate"), it
then fails postponetest.fs.

>Interpretive versions of IF and DO
>are not illegal

This is not about adding interpretation semantics to IF and DO. It's
about the compilation semantics. postponetest.fs only tests the
compilation semantics of IF and DO (it POSTPONEs IF, and POSTPONE
accesses the compilation semantics).

>In my view, the intent of the standard is to be very laissez-faire about
>"undefined" features and "ambiguous conditions".

The compilation semantics of IF and DO are defined and unambiguous.

>So, things like ?COMP are neither required nor illegal, they are
>matters of taste (even though some people think they're silly).

Putting ?COMP in an IF makes the IF non-standard.

[Anton Ertl]

Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:

>So if I was wary about a program being standard w.r.t. to IF
> I could load a preamble.
>-----------
>: IF STATE @ 0=
> ABORT" NONPORTABLE: interpretation semantics of IF is undefined"
> POSTPONE IF ; IMMEDIATE
>-----------
>
>and it would only sift out nonportable programs.

After this definition the system is non-standard and would fail for
some standard programs. That's because your redefined IF has
compilation semantics that fail in interpret state.

[Anton Ertl]

Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>cmFORTH did use the wordlists for independent interpretation and
>compilation semantics, but indeed not for IF and DO. There were
>compiling versions of a bunch of things, like this:
>
>FORTH : BINARY ( n) CREATE , DOES R> @ ?CODE @ DUP
> ... lots ... ;
>
>COMPILER A840 BINARY + AA40 BINARY XOR A640 BINARY OR
> AC40 BINARY - A240 BINARY AND A440 BINARY SWAP-
>
>FORTH
>
>... much later ...
>
>: SWAP SWAP ; : OVER OVER ;
>: DUP DUP ; : DROP DROP ;
>: XOR XOR ; : AND AND ;
>: OR OR ;
>: + + ; : - - ;

That's just an unusual implementation of default compilation
semantics, but it's not really independent interpretation and
compilation semantics.

Which reinforces my conclusion: Even if you have the capability to
implement independent interpretation and compilation semantics, it's
probably better to just have default and compile-only words; and
immediate words, for comments (what does cmForth do for comments?).

[Andrew Haley]

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>cmFORTH did use the wordlists for independent interpretation and
>>compilation semantics, but indeed not for IF and DO. There were
>>compiling versions of a bunch of things, like this:
>>
>>FORTH : BINARY ( n) CREATE , DOES R> @ ?CODE @ DUP
>> ... lots ... ;
>>
>>COMPILER A840 BINARY + AA40 BINARY XOR A640 BINARY OR
>> AC40 BINARY - A240 BINARY AND A440 BINARY SWAP-
>>
>>FORTH
>>
>>... much later ...
>>
>>: SWAP SWAP ; : OVER OVER ;
>>: DUP DUP ; : DROP DROP ;
>>: XOR XOR ; : AND AND ;
>>: OR OR ;
>>: + + ; : - - ;
>
> That's just an unusual implementation of default compilation
> semantics, but it's not really independent interpretation and
> compilation semantics.

I don't understand what you mean.

> Which reinforces my conclusion: Even if you have the capability to
> implement independent interpretation and compilation semantics, it's
> probably better to just have default and compile-only words; and
> immediate words, for comments (what does cmForth do for comments?).

COMPILER : ( 29 WORD DROP ;
FORTH : ( \ ( ;

\ means [COMPILE]

As you'd expect, surely.

Andrew.

[Albert van der Horst]

In article <2012Jun1...@mips.complang.tuwien.ac.at>,
Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:

>Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>>So if I was wary about a program being standard w.r.t. to IF
>> I could load a preamble.
>>-----------
>>: IF STATE @ 0=
>> ABORT" NONPORTABLE: interpretation semantics of IF is undefined"
>> POSTPONE IF ; IMMEDIATE
>>-----------
>>
>>and it would only sift out nonportable programs.
>
>After this definition the system is non-standard and would fail for
>some standard programs. That's because your redefined IF has
>compilation semantics that fail in interpret state.

Apparently I miss the point here. Can you give an example where
such a preamble would spoil the cake?

>
>- anton

[Albert van der Horst]

In article <2012Jun1...@mips.complang.tuwien.ac.at>,
Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:

>Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>>I feel that adding interpreted use of IF and DO is much more useful
>>then other smartness.
>
>If you want to add interpretation semantics for IF and DO, then go
>ahead. But don't produce STATE-smart IF and DO; for simple cases they
>work as desired, but for more complex cases they sometimes fail.
>Instead, if you want that stuff, bite the bullet and implement one of
>the schemes for doing that properly.
>
>My feeling, though, is that in the balance it's a bad idea to have
>such words. Not only do you have to implement one of the schemes
>above, but you also have repercussions elsewhere (like the meaning of
>xts and how to reify the interpretation and compilation semantics).
>
>Your approach of using prefixes appears to be a very good alternative
>for the two standard words (S" and TO) that have independent
>interpretation and compilation semantics (one might have a prefix like
>"->" instead of "TO", i.e., instead of "TO zoing" have "->zoing"); and
>also for most other cases where people want such things. Now you want
>to ruin it by complicating IF and DO. Sigh.

Remember interpreted-DO is just a loadable extension. I'm going to run
your testpostpone.fs and draw my conclusions. I will add some heavy or
less heavy warnings for those who use it. It is just that my class
extension doesn't work properly without it.

>
>
>- anton

Groetjes Albert

[Anton Ertl]

Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>cmFORTH did use the wordlists for independent interpretation and
>>>compilation semantics, but indeed not for IF and DO. There were
>>>compiling versions of a bunch of things, like this:
>>>
>>>FORTH : BINARY ( n) CREATE , DOES R> @ ?CODE @ DUP
>>> ... lots ... ;
>>>
>>>COMPILER A840 BINARY + AA40 BINARY XOR A640 BINARY OR
>>> AC40 BINARY - A240 BINARY AND A440 BINARY SWAP-
>>>
>>>FORTH
>>>
>>>... much later ...
>>>
>>>: SWAP SWAP ; : OVER OVER ;
>>>: DUP DUP ; : DROP DROP ;
>>>: XOR XOR ; : AND AND ;
>>>: OR OR ;
>>>: + + ; : - - ;
>>
>> That's just an unusual implementation of default compilation
>> semantics, but it's not really independent interpretation and
>> compilation semantics.
>
>I don't understand what you mean.

These are normal Forth words (words with default compilation
semantics). With "independent interpretation and compilation
semantics" I mean things like S", where the interpretation semantics
puts the string in a buffer, and the compilation semantics compiles it
as string literal.

Of course, the separate compilation wordlist is capable of
implementing both, but apparently Chuck Moore chose not to do the
"independent interpretation and compilation semantics" thing. And
given all the complications that follow from that, I think he was
right.

[Anton Ertl]

Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:

[interpretation semantics for DO]

>It is just that my class
>extension doesn't work properly without it.

Why not? What's so bad about defining a :NONAME word and running it
(maybe with some code-memory management wrapped around that)?
Especially given that you claim that even a simple compile-only flag
(and the check for it in the text interpreter) adds too much
complexity for your taste.

[Anton Ertl]

Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>In article <2012Jun1...@mips.complang.tuwien.ac.at>,
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>>>So if I was wary about a program being standard w.r.t. to IF
>>> I could load a preamble.
>>>-----------
>>>: IF STATE @ 0=
>>> ABORT" NONPORTABLE: interpretation semantics of IF is undefined"
>>> POSTPONE IF ; IMMEDIATE
>>>-----------
>>>
>>>and it would only sift out nonportable programs.
>>
>>After this definition the system is non-standard and would fail for
>>some standard programs. That's because your redefined IF has
>>compilation semantics that fail in interpret state.
>
>Apparently I miss the point here. Can you give an example where
>such a preamble would spoil the cake?

: gen-foo ... postpone if ... ;
: bla ... [ 5 gen-foo s" hi" gen-bar ] ... ;

[Andrew Haley]

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>>cmFORTH did use the wordlists for independent interpretation and
>>>>compilation semantics, but indeed not for IF and DO. There were
>>>>compiling versions of a bunch of things, like this:
>>>>
>>>>FORTH : BINARY ( n) CREATE , DOES R> @ ?CODE @ DUP
>>>> ... lots ... ;
>>>>
>>>>COMPILER A840 BINARY + AA40 BINARY XOR A640 BINARY OR
>>>> AC40 BINARY - A240 BINARY AND A440 BINARY SWAP-
>>>>
>>>>FORTH
>>>>
>>>>... much later ...
>>>>
>>>>: SWAP SWAP ; : OVER OVER ;
>>>>: DUP DUP ; : DROP DROP ;
>>>>: XOR XOR ; : AND AND ;
>>>>: OR OR ;
>>>>: + + ; : - - ;
>>>
>>> That's just an unusual implementation of default compilation
>>> semantics, but it's not really independent interpretation and
>>> compilation semantics.
>>
>>I don't understand what you mean.
>
> These are normal Forth words (words with default compilation
> semantics).

In what sense? The COMPILER versions lay down instructions, the FORTH
versions call them. They are different words, with different
semantics, i.e. they do different things.

> With "independent interpretation and compilation semantics" I mean
> things like S", where the interpretation semantics puts the string
> in a buffer, and the compilation semantics compiles it as string
> literal.

How is the above different? I really don't understand what point
you're making.

Andrew.

[Anton Ertl]

Sure, the default compilation semantics of a normal Forth word are
different from the default interpretation semantics (the default
compilation semantics compiles the execution semantics, the default
interpretation semantics performs it; in a more conventional
implementation the default compilation semantics COMPILE,s the xt of
the word, whereas the default interpretation semantics EXECUTEs the
xt.

The point is that these words implement the default semantics. +
behaves like a normal Forth word; no immediacy, no other non-default
compilation semantics. This normality is also reflected in the
definition

: + + ;

He doesn't define compile-only words like IF or immediate words like (
like this, nor would that work for S" and TO.

>> With "independent interpretation and compilation semantics" I mean
>> things like S", where the interpretation semantics puts the string
>> in a buffer, and the compilation semantics compiles it as string
>> literal.
>
>How is the above different? I really don't understand what point
>you're making.

The definition of + etc. above is just an unusual *implementation* of
the default semantics. + behaves as specified in the standard (even
though cmForth does not aspire to being a standard system), and + in
the standard is a normal word (i.e., only default compilation and
interpretation semantics).

One difference is that

: S" S" ;

is not a valid implementation of the interpretation semantics of S".

If I still have not made myself clear: Implementation is not
semantics. Semantics is what is visible from the outside (black-box
view). Implementation is how it is done (white-box view). The same
semantics can (and often does) have several different implementations.
E.g.:

: 2* 2 * ;
: 2* 1 lshift ;
: 2* dup + ;

On a 2s-complement system all these definitions implement the same
semantics (those specified in the standard for 2*).

[BruceMcF]

On Jun 20, 7:09 am, an...@mips.complang.tuwien.ac.at (Anton Ertl)

wrote:
> Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>
>
>
>
>
>
>
>
>

> >In article <2012Jun19.175...@mips.complang.tuwien.ac.at>,

> >Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> >>Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
> >>>So if I was wary about a program being standard w.r.t. to IF
> >>> I could load a  preamble.
> >>>-----------
> >>>: IF    STATE @ 0=
> >>>    ABORT" NONPORTABLE: interpretation semantics of IF is undefined"
> >>>    POSTPONE IF ; IMMEDIATE
> >>>-----------
>
> >>>and it would only sift out nonportable programs.
>
> >>After this definition the system is non-standard and would fail for
> >>some standard programs.  That's because your redefined IF has
> >>compilation semantics that fail in interpret state.
>
> >Apparently I miss the point here. Can you give an example where
> >such a preamble would spoil the cake?
>
> : gen-foo ... postpone if ... ;
> : bla ... [ 5 gen-foo s" hi" gen-bar ] ... ;

So if source is trying, while interpreting, to perform the compilation
semantics of a word that doesn't necessarily have interpretation
behavior, that will be problematic?

Or, IOW, its problematic for source code with an implementation
dependency on a word containing postponed compilation semantics
functioning in interpretation state.

[Anton Ertl]

BruceMcF <agi...@netscape.net> writes:
>On Jun 20, 7:09=A0am, an...@mips.complang.tuwien.ac.at (Anton Ertl)

>wrote:
>> Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>> >Apparently I miss the point here. Can you give an example where
>> >such a preamble would spoil the cake?
>>
>> : gen-foo ... postpone if ... ;
>> : bla ... [ 5 gen-foo s" hi" gen-bar ] ... ;
>
>So if source is trying, while interpreting, to perform the compilation
>semantics of a word that doesn't necessarily have interpretation
>behavior, that will be problematic?

I am not sure what your question is, but the issue is about performing
compilation semantics in interpret state. An implementation of IF
containing ?COMP (or equivalent, like Albert's preamble) will fail
then, while it will likely work if the ?COMP is removed.

That's the general problem with STATE-smart words: they check STATE at
run-time instead of at parse time. It works when the word is
text-interpreted directly (because then the run-time is performed
right after parsing), but it can fail if the run-time is delayed by
ticking or POSTPONEing (or [COMPILE]ing).

[Albert van der Horst]

In article <2012Jun2...@mips.complang.tuwien.ac.at>,

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>>In article <2012Jun1...@mips.complang.tuwien.ac.at>,
>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>>>>So if I was wary about a program being standard w.r.t. to IF
>>>> I could load a preamble.
>>>>-----------
>>>>: IF STATE @ 0=
>>>> ABORT" NONPORTABLE: interpretation semantics of IF is undefined"
>>>> POSTPONE IF ; IMMEDIATE
>>>>-----------
>>>>
>>>>and it would only sift out nonportable programs.
>>>
>>>After this definition the system is non-standard and would fail for
>>>some standard programs. That's because your redefined IF has
>>>compilation semantics that fail in interpret state.
>>
>>Apparently I miss the point here. Can you give an example where
>>such a preamble would spoil the cake?
>
>: gen-foo ... postpone if ... ;
>: bla ... [ 5 gen-foo s" hi" gen-bar ] ... ;

I see. For the moment I think the following is best.

I leave ?COMP in but in the rare case that some one would do the
above:

- program doesn't compile
- double check: "Hey it is intentional"
- WANT NO-SECURITY NO-SECURITY

As far as I remember even the GNU c-compiler is only fully
compliant with the -pedantic option.
In Forth the equivalent is
REQUIRE pedantic

To answer your question in the other thread:
I must have either ?COMP or an interpreted IF DO ,

otherwise
"
1 \ size
class ARRAY
M: limit @ M; DUP ,

0 DO 0 , LOOP
endclass
"

goes wrong without a message and without an indication.

[Ed]

Elizabeth D. Rather wrote:
> ...
> I don't advocate banning ?COMP. I just think it's silly. And making CODE
> etc. IMMEDIATE just so you can issue an error message in case of blatant
> silliness is doubly silly.
>
> As I've said elsewhere, I support the "intelligent programmer,
> permissive compiler" theory of programming. I can think of several
> reasons for using : and END-CODE in definitions, somewhat fewer for
> CODE, but I see no advantage to making them immediate unless you fancy
> yourself a programming nanny, in which case maybe Forth is not for you.

And presumably most forth implementors including MPE by
implication!

?COMP ?EXEC was part of Fig's compiler security package.
Given compiler security is all about catching unlikely things,
it was not unreasonable.

If making CODE END-CODE immediate was a bad move,
then I'm sure those affected would like to hear about it, even
after all this time. Do you have practical examples to make
the case?

As regards ?COMP it is the more useful and has been used
for two things - to prevent inadvertent execution of conditionals
from corrupting memory, and/or to pinpoint errors early.

In SwiftForth inadvertent execution of BEGIN in immediate
mode results in an OK message. Is it "Ok" ? Most forth
implementations including commercials don't agree it is.

No, I don't fancy myself as a "programming nanny". In fact my
system uses less syntax checking than most. I could remove
the four ?COMPs currently present and my system would
still be protected. I don't mind spending the 35 bytes plus
header it costs for the additional warning ?COMP provides.
Who knows, even SwiftForth users might like it :)

[Stephen Pelc]

On Mon, 18 Jun 2012 15:06:47 +1000, "Ed" <inv...@nospam.com> wrote:

>VFX is loaded with ?COMPs.

Not any more. It was a while ago.

Stephen

--
Stephen Pelc, steph...@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

[JennyB]

On Tuesday, 19 June 2012 15:09:44 UTC+1, Anton Ertl wrote:
> Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
> >I feel that adding interpreted use of IF and DO is much more useful
> >then other smartness.
>
> If you want to add interpretation semantics for IF and DO, then go
> ahead. But don't produce STATE-smart IF and DO; for simple cases they
> work as desired, but for more complex cases they sometimes fail.
> Instead, if you want that stuff, bite the bullet and implement one of
> the schemes for doing that properly.
>
> My feeling, though, is that in the balance it's a bad idea to have
> such words. Not only do you have to implement one of the schemes
> above, but you also have repercussions elsewhere (like the meaning of
> xts and how to reify the interpretation and compilation semantics).
>

It may be a bad idea, but it's even worse to have a bad idea, badly executed. Since people insist doing such things, shouldn't there be a clear way of showing their meaning without mixing up interpretation and compilation behaviour in the one definition?

The simplest way I can think of is:

COMPILES xt -- set most recent definition to perform xt
at compile-time

IMMEDIATE may (or may not) be defined as LATESTXT COMPILES

COMPILES may be implemented either by extending FIND when compiling to return the compiling xt and an immediate flag, or by a smart COMPILE, to perform the appropriate action for the given xt. In the latter case, IMMEDIATE should be defined so that immediate words return an immediate flag, but that is mainly of historical interest. Consider for example this definition of POSTPONE :

FIND DUP IF immediate? IF COMPILE, ELSE
postpone LITERAL ['] COMPILE, COMPILE, THEN
ELSE

will still work as expected with a smart compiler if IMMEDIATE is defined as above, whichever flag is returned.

When interpreting, the xt returned by FIND must represent interpretation behaviour, and the flag is irrelevant. When compiling, an immediate flag indicates that the xt represents compiling behaviour, otherwise it represents interpreting behaviour. ' and ['] always return the interpreting behaviour.

There is a simple, portable way to get an xt that represents the compiling behaviour of any particular word:

: 'COMP :NONAME POSTPONE POSTPONE POSTPONE ; ;

STATE-checking words return the appropriate behaviour, but only if the xt returned is finally executed in the appropriate state.


: THROWS CREATE , DOES> @ THROW ;

-14 THROWS COMP-ONLY
-29 THROWS CAN'T-NEST

: CODE interpretation behaviour ; compiles can't-nest

: compDO compilation behaviour of DO ;
: DO comp-only ; compiles compDO

Or, later

'comp DO
: DO interpretation-version ; compiles

[Albert van der Horst]

I think this is a good presentation of the trade offs.
It helps me to decide what I want to do with ciforth.

For ciforth I want ?COMP in. It catches lots of typos.

Using ?EXEC and making defining words IMMEDIATE can prevent
a situation that is extremely puzzling to a novice:

: def1 ....
DROP \ note missing semicolon
: def2 ...
;
\ Point A
....
much later
....
: def3 ...... def2 .....
^^^^
Not found.

The problem is that an error is signalled at a much later time.
If the novice doesn't panic he discovers that at Point A
`` def2 '' is not defined. Then the bug is found soon enough.
The second time the novice (and even I myself) finds the
mistake immediately^H^H^H^H^H^H^H much faster.

On the other side the consequence is to always having to do
POSTPONE : .
One tends to forget that : is IMMEDIATE , because there
seems no good reason to do so, so this too leads to errors
(but those are signalled at the spot and easily corrected.)

Given that ciforth is simplistic and not very user
friendly to begin with, I'm not going to make all defining
words immediate and will not used ?EXEC.

[BruceMcF]

On Jun 21, 10:00 am, Albert van der Horst <alb...@spenarnc.xs4all.nl>
wrote:

> On the other side the consequence is to always having to do
> POSTPONE : .

Which makes it impossible to run standard source that compiles : to
execute at runtime. Forth94 : *has* default compilation semantics, so
an implementation with ?EXEC in : is not a Forth94 implementation.

[Anton Ertl]

Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>Using ?EXEC and making defining words IMMEDIATE can prevent
>a situation that is extremely puzzling to a novice:
>
>: def1 ....
> DROP \ note missing semicolon
>: def2 ...

^^^ Not found

>;
>\ Point A
>....
>much later
>....
>: def3 ...... def2 .....
> ^^^^
> Not found.
>
>The problem is that an error is signalled at a much later time.

It's usually signaled right after the defining word. I have inserted
this "not found" message in the example above.

>On the other side the consequence is to always having to do
>POSTPONE : .

Better use [COMPILE] :. That works on standard systems, too.

[Anton Ertl]

JennyB <jenny...@googlemail.com> writes:
>On Tuesday, 19 June 2012 15:09:44 UTC+1, Anton Ertl wrote:
>> If you want to add interpretation semantics for IF and DO, then go
>> ahead. But don't produce STATE-smart IF and DO; for simple cases they
>> work as desired, but for more complex cases they sometimes fail.
>> Instead, if you want that stuff, bite the bullet and implement one of
>> the schemes for doing that properly.

>>=20

>> My feeling, though, is that in the balance it's a bad idea to have
>> such words. Not only do you have to implement one of the schemes
>> above, but you also have repercussions elsewhere (like the meaning of
>> xts and how to reify the interpretation and compilation semantics).

>>=20
>
>It may be a bad idea, but it's even worse to have a bad idea, badly execute=
>d. Since people insist doing such things, shouldn't there be a clear way of=
> showing their meaning without mixing up interpretation and compilation beh=

>aviour in the one definition?
>
>The simplest way I can think of is:
>

> COMPILES xt -- set most recent definition to perform xt=20
> at compile-time
>
>IMMEDIATE may (or may not) be defined as LATESTXT COMPILES=20
>
>COMPILES may be implemented either by extending FIND when compiling to retu=

>rn the compiling xt and an immediate flag

Yes, that's a possibility, but it's pretty ugly: basically FIND now
has STATE as additional parameter and may deliver completely different
results based in this parameter. I know that it's ugly because that's
what Gforth's FIND does.

It's better to have separate words for producing an xt for the
interpretation semantics and something (either an xt or xt1 xt2) for
the compilation semantics.

>or by a smart COMPILE, to perfor=

>m the appropriate action for the given xt.

That's a broken "COMPILE,", not a smart one. A properly working
"COMPILE," compiles the same semantics that EXECUTE performs. That
would not be the case if "COMPILE," performs the compilation semantics
(of, e.g., S") whereas EXECUTE performs the interpretation/execution
semantics.

[Anton Ertl]

"Ed" <inv...@nospam.com> writes:
>If making CODE END-CODE immediate was a bad move,
>then I'm sure those affected would like to hear about it, even
>after all this time. Do you have practical examples to make
>the case?

I don't use CODE, but fig-Forth also made : immediate with ?EXEC.
This means that you cannot just use : inside a colon definition to get
a defining word. Instead, you have to [COMPILE] :. And that's what I
did in the first release of Gray. A bit later Marcel Hendrix ported
Gray to his Forth (not sure if it was iForth at the time); he saw the
[COMPILE] and converted it to POSTPONE. Since his : was not
immediate, this did not achieve what was intended ([COMPILE] would
have been ok).

>As regards ?COMP it is the more useful and has been used
>for two things - to prevent inadvertent execution of conditionals
>from corrupting memory, and/or to pinpoint errors early.

There are much better approaches for achieving this.

>I don't mind spending the 35 bytes plus
>header it costs for the additional warning ?COMP provides.

I think you can have a proper solution for similar cost.

[Andrew Haley]

Ed <inv...@nospam.com> wrote:
> Elizabeth D. Rather wrote:
>> ...
>> I don't advocate banning ?COMP. I just think it's silly. And making CODE
>> etc. IMMEDIATE just so you can issue an error message in case of blatant
>> silliness is doubly silly.
>>
>> As I've said elsewhere, I support the "intelligent programmer,
>> permissive compiler" theory of programming. I can think of several
>> reasons for using : and END-CODE in definitions, somewhat fewer for
>> CODE, but I see no advantage to making them immediate unless you fancy
>> yourself a programming nanny, in which case maybe Forth is not for you.
>
> And presumably most forth implementors including MPE by
> implication!
>
> ?COMP ?EXEC was part of Fig's compiler security package.
> Given compiler security is all about catching unlikely things,
> it was not unreasonable.

I think the key to understanding this is that IMMEDIATE words are, by
definition, anomalous. That is to say, they don't (or probably don't)
have "default compilation semantics".

The default in Forth is that given

: z ( -) foo bar baz ;

you expect that Z calls FOO, BAR, then BAZ . You may be wrong, but
it's a good first guess. The principle of least surprise says that
you want to match, as far as possible, the readers' experience,
expectations, and mental models. From this one can directly derive
the principle that one should minimize the number of IMMEDIATE words.

"Compiler security" in the fig-FORTH sense is part of Forth's history.

Andrew.


http://en.wikipedia.org/wiki/Principle_of_least_astonishment

[BruceMcF]

On Jun 21, 12:51 pm, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:

> The default in Forth is that given
>
> : z   ( -) foo bar baz ;
>
> you expect that Z calls FOO, BAR, then BAZ .  You may be wrong, but
> it's a good first guess.  The principle of least surprise says that
> you want to match, as far as possible, the readers' experience,
> expectations, and mental models.  From this one can directly derive
> the principle that one should minimize the number of IMMEDIATE words.

This also suggests that naming immediate words to highlight their
distinctiveness is good practice, as with [CHAR] [IF] [ELSE] [THEN]
etc, and as might have been done with [TO] to avoid the functional
overloading of TO.

[BruceMcF]

On Jun 20, 11:32 am, an...@mips.complang.tuwien.ac.at (Anton Ertl)
wrote:

> BruceMcF <agil...@netscape.net> writes:
> >On Jun 20, 7:09=A0am, an...@mips.complang.tuwien.ac.at (Anton Ertl)
> >wrote:
> >> Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
> >> >Apparently I miss the point here. Can you give an example where
> >> >such a preamble would spoil the cake?

> >> : gen-foo ... postpone if ... ;
> >> : bla ... [ 5 gen-foo s" hi" gen-bar ] ... ;

> >So if source is trying, while interpreting, to perform the compilation
> >semantics of a word that doesn't necessarily have interpretation
> >behavior, that will be problematic?

> I am not sure what your question is, but the issue is about performing
> compilation semantics in interpret state.

Yes, which we are not necessarily guaranteed to be able to do with a
compliant implementation, but which some otherwise portable source may
have an implementation dependency on being able to do.

[Albert van der Horst]

In article <2012Jun2...@mips.complang.tuwien.ac.at>,
Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:

>Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>>Using ?EXEC and making defining words IMMEDIATE can prevent
>>a situation that is extremely puzzling to a novice:
>>
>>: def1 ....
>> DROP \ note missing semicolon
>>: def2 ...
> ^^^ Not found

OOF! you're right. This whole rant I sited from memory without trying
it. I'm pretty sure I have encountered situations that presented
problems, but at least they are rarer. It makes the case against
immediacy of defining words stronger.

>>;
>>\ Point A
>>....
>>much later
>>....
>>: def3 ...... def2 .....
>> ^^^^
>> Not found.
>>
>>The problem is that an error is signalled at a much later time.
>
>It's usually signaled right after the defining word. I have inserted
>this "not found" message in the example above.
>
>>On the other side the consequence is to always having to do
>>POSTPONE : .
>
>Better use [COMPILE] :. That works on standard systems, too.

Standard, i.e. where : is not IMMEDIATE.
That is an important point, and I'm glad I finally understand it.
Luckily it doesn't matter, because I don't go that way.

>
>- anton

Groetjes Albert

[Bernd Paysan]

Anton Ertl wrote:
>>Apparently I miss the point here. Can you give an example where
>>such a preamble would spoil the cake?
>
> : gen-foo ... postpone if ... ;
> : bla ... [ 5 gen-foo s" hi" gen-bar ] ... ;

Anton, you should really stop reading the standard as a divine work, but
as a work of Elizabeth Rather, Mitch Bradley, and others. And these
people simply think you are wrong with your interpretation.

There may be a compliation token within the system, but it is not
required to have one (when it does not have one, it uses STATE at run-
time to decide which mode it is in). All it is required to have is
STATE and IMMEDIATE, and this implies that the compilation semantics
(=the text interpreter result in compilation state) is only accessible
when STATE is set to true.

We should write that into Forth200x, so that this silly discussion
stops. If you want to get rid of STATE and IMMEDIATE, make an RFC that

a) declare them as obsolescent and moves them out of CORE
b) provides a standard way to implement special
compilation/interpretation behavior

RFI007 declares that "normal" words (i.e. those where only the execution
semantics is specified, and all other semantics are derived by implicit
rules) cannot be implemented as immediate state-smart words. This means
that systems like VFX, where each primitive contains a special compiler
for itself have to use a compilation-token approach like smart COMPILE,.

I think we all agree that a "normal" word is most useful: You can
interpret and compile it, it does the same thing in both cases. You can
' and EXECUTE it, and you can ' and COMPILE, it, and the result is also
the same. Its behavior does not depend on a global variable like STATE.
This is good. This however has its own set of limitations.

--
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

[Bernd Paysan]

Anton Ertl wrote:
> And in what way is this a misunderstanding? What do you suggest to
> replace "reading the standard" with? Divine inspiration?

Asking the people who wrote it. We don't need divine inspiration.

And: The way we standardize things is by looking at common practice. If
we find a feature in the standard which is *not* common practice, it's
time to de-standardize it.

> Anyway, even if we agree that we don't want to put the onus of
> implementing S" and TO properly on all systems (i.e., such that they
> pass postponetest.fs), it's a good idea for implementors to pass as
> much of postponetest.fs as possible, because there is production code
> out there that will fail on systems where certain postponetest.fs
> tests fail.

Maybe. There are however quite a lot of Forths out there which fail
postponetest.fs, and they have production code running on it. I had
quite some troubles with VFX Forth with Postpone, and I was "the only
one" who used strange things where Stephen's source inliner failed. The
tests required to break that weren't even part of test/postpone.fs, it
would be something like

: n+: ( n -- ) >r : r> postpone literal postpone + postpone ; ;
3 n+: 3+
: bar 3+ ;
{ 5 bar -> 8 }

With the source inliner, VFX Forth did not see any source after the
definition of 3+, so it inlined that "nothing" into bar. I should
really add this one to postponetest.fs

> And putting ?COMP in an IF and DO that pass postponetest.fs will make
> them fail postponetest.fs.

Yes. But the question is whether postponetest really tests something
that is worth testing. It tests whether postpone is able to preserve
the compilation semantics of its postponed words in interpretation
state.

> What do you mean by that? ?COMP is a check of STATE at run-time. It
> has this problem by nature. If you mean that on some systems I have
> ways to do properly what people try (and fail) to achieve with ?COMP,
> that's true, but that's not ?COMP. It's, e.g., Gforth's COMPILE-ONLY.

Yes, but COMPILE-ONLY (which sets a header flag) is not a standard word.
It's a nice way to implement this feature, but it has to be pretty deep
in the system. You can't add it later. It's something that (including
other features) makes Gforth EC 16kB large, while 4kB probably should be
enough on an embedded system (and that 4k will not check anything and
only implement core words).

>>But the standard does not
>>mandate these extensions, they are quality-of-implementation issues
>>(and not having too many features is considered as quality of
>>implementation by some people in the Forth community).
>
> In particular, one should not include misfeatures like ?COMP. And
> that's not just a quality-of-implementation issue.

IMHO, if the ANS Forth TC wanted to deliberately outlaw this misfeature
(which was pretty common back then), it should have said so explicitely,
and offer a way how to do it properly.

[Andrew Haley]

BruceMcF <agi...@netscape.net> wrote:
> On Jun 21, 12:51?pm, Andrew Haley <andre...@littlepinkcloud.invalid>

> wrote:
>
>> The default in Forth is that given
>>

>> : z ? ( -) foo bar baz ;

>>
>> you expect that Z calls FOO, BAR, then BAZ . You may be wrong, but
>> it's a good first guess. The principle of least surprise says that
>> you want to match, as far as possible, the readers' experience,
>> expectations, and mental models. From this one can directly derive
>> the principle that one should minimize the number of IMMEDIATE words.
>
> This also suggests that naming immediate words to highlight their
> distinctiveness is good practice, as with [CHAR] [IF] [ELSE] [THEN]
> etc, and as might have been done with [TO] to avoid the functional
> overloading of TO.

Or make them a different colour. However, I think that naming all
immediate words like that would make Forth harder to read. It's
always a compromise, I suppose.

TO and S" seem to be perpetually problematic.

Andrew.

[Andrew Haley]

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>>In article <2012Jun1...@mips.complang.tuwien.ac.at>,
>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>Albert van der Horst <alb...@spenarnc.xs4all.nl> writes:
>>>>So if I was wary about a program being standard w.r.t. to IF
>>>> I could load a preamble.
>>>>-----------
>>>>: IF STATE @ 0=
>>>> ABORT" NONPORTABLE: interpretation semantics of IF is undefined"
>>>> POSTPONE IF ; IMMEDIATE
>>>>-----------
>>>>
>>>>and it would only sift out nonportable programs.
>>>
>>>After this definition the system is non-standard and would fail for
>>>some standard programs. That's because your redefined IF has
>>>compilation semantics that fail in interpret state.
>>
>>Apparently I miss the point here. Can you give an example where
>>such a preamble would spoil the cake?
>
> : gen-foo ... postpone if ... ;
> : bla ... [ 5 gen-foo s" hi" gen-bar ] ... ;

But interpretation semantics of IF are undefined. They don't
magically get to be defined just because you POSTPONEd IF .

Andrew.

[Elizabeth D. Rather]

On 6/21/12 8:42 PM, Andrew Haley wrote:
> BruceMcF<agi...@netscape.net> wrote:
...

>> This also suggests that naming immediate words to highlight their
>> distinctiveness is good practice, as with [CHAR] [IF] [ELSE] [THEN]
>> etc, and as might have been done with [TO] to avoid the functional
>> overloading of TO.
>
> Or make them a different colour. However, I think that naming all
> immediate words like that would make Forth harder to read. It's
> always a compromise, I suppose.

The naming convention [...] is conflicted. For [CHAR] and ['] it
identifies a compile-only version of a normally interpretive word. For
[IF], [ELSE], and [THEN] it suggests an interpretive version of a
normally compile-only word. Bottom line: it means, "check your
documentation."

But they're now "common usage," so IMO any remedy is worse than the
disease at this point. We just have to be careful going forward.

> TO and S" seem to be perpetually problematic.

Yes.

Cheers,
Elizabeth


--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

[Stephen Pelc]

On Thu, 21 Jun 2012 22:15:08 +0200, Bernd Paysan <bernd....@gmx.de>
wrote:

>Maybe. There are however quite a lot of Forths out there which fail
>postponetest.fs, and they have production code running on it. I had
>quite some troubles with VFX Forth with Postpone, and I was "the only
>one" who used strange things where Stephen's source inliner failed. The
>tests required to break that weren't even part of test/postpone.fs, it
>would be something like

1) I did not write the source inliner. It was MPE's, not Stephen's.
2) The source inliner is dead! Long live the tokeniser!

>Yes. But the question is whether postponetest really tests something
>that is worth testing. It tests whether postpone is able to preserve
>the compilation semantics of its postponed words in interpretation
>state.

POSTPONE had no common practice at the time it was introduced.
18 years later we still have trouble with it.

By comparison, CATCH and THROW had plenty of common practice, albeit
in other languages. It has caused no trouble.

The lesson is not to standardise a solution until it has been
exhaustively evaluated. However, POSTPONE is a mess we have to
cope with until we have a proper model of xt's and Forth compilation
that allows for modern techniques. Once we have a model we can try
it and define a sensible word set. STATE POSTPONE and IMMEDIATE
may be casualties of this process in 20 years time.

[Andrew Haley]

Stephen Pelc <steph...@mpeforth.com> wrote:
> On Thu, 21 Jun 2012 22:15:08 +0200, Bernd Paysan <bernd....@gmx.de>
> wrote:
>
>>Maybe. There are however quite a lot of Forths out there which fail
>>postponetest.fs, and they have production code running on it. I had
>>quite some troubles with VFX Forth with Postpone, and I was "the only
>>one" who used strange things where Stephen's source inliner failed. The
>>tests required to break that weren't even part of test/postpone.fs, it
>>would be something like
>
> 1) I did not write the source inliner. It was MPE's, not Stephen's.
> 2) The source inliner is dead! Long live the tokeniser!

He'll never let it go. :-)

>>Yes. But the question is whether postponetest really tests something
>>that is worth testing. It tests whether postpone is able to preserve
>>the compilation semantics of its postponed words in interpretation
>>state.
>
> POSTPONE had no common practice at the time it was introduced.
> 18 years later we still have trouble with it.
>
> By comparison, CATCH and THROW had plenty of common practice, albeit
> in other languages. It has caused no trouble.
>
> The lesson is not to standardise a solution until it has been
> exhaustively evaluated. However, POSTPONE is a mess we have to
> cope with until we have a proper model of xt's and Forth compilation
> that allows for modern techniques. Once we have a model we can try
> it and define a sensible word set. STATE POSTPONE and IMMEDIATE
> may be casualties of this process in 20 years time.

I'm very nervous about suggesting this, but here goes:

I've been thinking the opposite. Maybe the problem with some of the
modern techniques is that they're exposed to the programmer; they're
too shallow. Perhaps the right solution is to let the Forth compiler
generate tokens (just like a very trad Forth) and then let the
optimizing compiler run over them. Let IMMEDIATE do what it used to
do. Get rid of other ways of creating words that execute at compile
time. Let FIND do what it used to do. Etc., etc...

This is probably worthy of a workshop at EuroFORTH.

Andrew.

[Anton Ertl]

There are no interpretation semantics of IF in play in this example.
POSTPONE IF compiles the compilation semantics of IF into GEN-FOO, and
GEN-FOO then performs the compilation semantics of IF.

[Anton Ertl]

"Elizabeth D. Rather" <era...@forth.com> writes:
>The naming convention [...] is conflicted. For [CHAR] and ['] it
>identifies a compile-only version of a normally interpretive word. For
>[IF], [ELSE], and [THEN] it suggests an interpretive version of a
>normally compile-only word.

Yes, you can use [IF] interpretively, but you can also use it inside
colon definitions for conditional compilation. It's a bit cumbersome,
because you then have to provide a value on the stack at compile time,
but Forth200x has [DEFINED] and [UNDEFINED] to the rescue.

: foo
... [DEFINED] bar [IF] 5 bar [ELSE] ... replacement code ... [THEN] ... ;

In that context the brackets make sense.

[Anton Ertl]

Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>I've been thinking the opposite. Maybe the problem with some of the
>modern techniques is that they're exposed to the programmer; they're
>too shallow. Perhaps the right solution is to let the Forth compiler
>generate tokens (just like a very trad Forth) and then let the
>optimizing compiler run over them.

That's an implementation issue, but it's the right approach.

> Let IMMEDIATE do what it used to
>do. Get rid of other ways of creating words that execute at compile
>time.

There is no other way in the standard. A problem may be that the
other ways are *not* exposed to the programmer (in standard programs),
so people still resort to STATE-smartness in standard programs.

Anyway, it's not totally clear what you mean by this.

One interpretation is that we should go back to implementing S" and TO
with STATE-smart words, and to make STATE-smart words an accepted
technique, and to hell with xts, POSTPONE, code generators and other
guru stuff that's troublesome in the presence of STATE-smart words.

Another take on your statement is that we should get rid of words like
S" and TO (and not design more of the kind) which some implementations
implement with STATE-smartness, for which other implementations
provide implementations with better properties, but with repercussions
that are far more trouble than these words are worth.

Instead, we would only have normal words (like +) and IMMEDIATE words
like ( and IF (the latter maybe compile-only), but no access to STATE
and no words that have independent interpretation and compilation
semantics.

We would have to replace S" and TO with something else. The classic
approach would be the ' ['] solution, but that's not very popular.

What is popular is the solution taken for numbers: The text
interpreter deals with them as appropriate and everyone is happy. We
don't use NUM 123 and [NUM] 123 for numbers, we let the text
interpreter deal with them.

Probably most of the parsing words (the main driver behind STATE-smart
words) can be eliminated by having a few additions to the text
interpreter. E.g., instead of writing ." bla" or .( bla) one could
write "bla" TYPE, and have code that works both interpretively and
compiled as intended.

There are a few parsing words where the replacement requires its own
treatment in the text interpreter, in particular TO. Instead of

TO v

we could write

->v

and the text interpreter would do the appropriate thing with V.

Albert van der Horst used to call this feature "denotations", but
lately I have seen him use "prefixes".

Many will cry out against this complication of the text interpreter.
Some may also find the idea bad because up to now the text interpreter
is a black box so any additions there are only done by the compiler
vendor. However:

1) I think that this feature adds relatively little complexity to the
text interpreter, and it gets rid of a huge amount of baggage
elsewhere (STATE-smart words and the restrictions that their defenders
want to put upon us, or the complexities of the alternatives and their
ramifications).

2) One could make it possible for "mere" users to add further
prefixes/denotations (or, as Gforth calls them, recognizers) to the
text interpreter, just like it is possible to add further words now.
Of course some people will fear giving this much power to the users,
and it's certainly a feature that should be used sparingly, but that's
a management issue.

>This is probably worthy of a workshop at EuroFORTH.

Yes.

[Marcel Hendrix]

an...@mips.complang.tuwien.ac.at (Anton Ertl) writes Re: Alternatives to S" and TO (was: ?EXEC)

> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>I've been thinking the opposite. Maybe the problem with some of the
>>modern techniques is that they're exposed to the programmer; they're
>>too shallow. Perhaps the right solution is to let the Forth compiler
>>generate tokens (just like a very trad Forth) and then let the
>>optimizing compiler run over them.

This is the approach taken by iForth. Almost every word has a
tokenized form, and another form intended for ' ['] or direct execution.

> That's an implementation issue, but it's the right approach.

>> Let IMMEDIATE do what it used to
>>do. Get rid of other ways of creating words that execute at compile
>>time.

I don't know the repercussions of all that. My approach would be to
make the language as simple as possible, without worrying about speed
or size. Alternatives can go. Hooks can go too.

[..]

> What is popular is the solution taken for numbers: The text
> interpreter deals with them as appropriate and everyone is happy. We
> don't use NUM 123 and [NUM] 123 for numbers, we let the text
> interpreter deal with them.

An interesting idea. Then we could maybe also have dotted notation
and such.

[..]

> There are a few parsing words where the replacement requires its own
> treatment in the text interpreter, in particular TO. Instead of

> TO v

> we could write

> ->v

Great.

> and the text interpreter would do the appropriate thing with V.

[..]

> Many will cry out against this complication of the text interpreter.
> Some may also find the idea bad because up to now the text interpreter
> is a black box so any additions there are only done by the compiler
> vendor. However:

> 1) I think that this feature adds relatively little complexity to the
> text interpreter, and it gets rid of a huge amount of baggage
> elsewhere (STATE-smart words and the restrictions that their defenders
> want to put upon us, or the complexities of the alternatives and their
> ramifications).

The iForth text interpreter is already hugely complicated by workarounds
that are of no real benefit to the user.

> 2) One could make it possible for "mere" users to add further
> prefixes/denotations (or, as Gforth calls them, recognizers) to the
> text interpreter, just like it is possible to add further words now.

There is that old idea to factor the text interpreter, that would fit here.

> Of course some people will fear giving this much power to the users,
> and it's certainly a feature that should be used sparingly, but that's
> a management issue.

Who are these people?

[..]

-marcel

[BruceMcF]

On Jun 22, 11:19 am, an...@mips.complang.tuwien.ac.at (Anton Ertl)
wrote:

> We would have to replace S" and TO with something else.  The classic
> approach would be the ' ['] solution, but that's not very popular.

Is this conflating? The problem with S" is entirely that the buffered
S" and the compiled S" were given the same name. If "parse into a
buffer" S" had been given a name such as T" in the first place, then
there would be no problem.

OTOH, would be multiply overloaded even with TO and [TO] ... as noted,
inventing a new Forth94-like language with prefix semantics *could*
eliminate that overloading, but it would be just as likely to
proliferate the overloading into the prefixes, if the prefixes are
taken beyond just one-shot base modifications.

[Bernd Paysan]

Anton Ertl wrote:
> There are no interpretation semantics of IF in play in this example.
> POSTPONE IF compiles the compilation semantics of IF into GEN-FOO, and
> GEN-FOO then performs the compilation semantics of IF.

So this works if there is a compilation token for IF, which only
represents the compilation semantics, and not the interpretation
semantis, which the implementer of a Forth system is free to decide what
to do with (parse for the next ELSE/THEN in the input stream like [IF]
does, or report an error, or just compile a forward branch without
warning).

So far, we have a few solutions to this problem, with words like TO and
S" being the hardest:

* State-smart words, which have the problem that they check for the
state at run-time rather than at parse-time.

* Dual-xt solutions (whether dual-wordlist or a special flag in the
header like in Gforth) result in a state-smart FIND, which just moves
the problem around.

* Intelligent COMPILE, solutions, which however "violate" the spec of
COMPILE, - that is specified to compile the "execution semantics". This
works in VFX Forth:

: if1 ['] if compile, ; immediate ok
: else1 ['] else compile, ; immediate ok
: then1 ['] then compile, ; immediate ok
: test if1 ." true" else1 ." false" then1 ; ok
true test true ok

This is a bit embarrasing, but as IF in VFX Forth 4.4 is not immediate,
it passes Mitch's "user written text interpreter" test.

[Bernd Paysan]

Stephen Pelc wrote:
> 1) I did not write the source inliner. It was MPE's, not Stephen's.
> 2) The source inliner is dead! Long live the tokeniser!

Yes. This was the consequence of that discussion, and we now know that
this is the right way to do it.

> POSTPONE had no common practice at the time it was introduced.
> 18 years later we still have trouble with it.

POSTPONE "solved" a problem created by COMPILE and [COMPILE], without
understanding that the problem comes from the immediate flag, which
apparently is the root of all evil.

> By comparison, CATCH and THROW had plenty of common practice, albeit
> in other languages. It has caused no trouble.

We actually went some way further with CATCH as originally envisioned.
E.g. take all these file-IO words which return an ior. This wasn't
specified as "you can just throw this, the system will produce a
meaningful error message". We now all do it that way, we expect an ior
is something to throw.

> The lesson is not to standardise a solution until it has been
> exhaustively evaluated. However, POSTPONE is a mess we have to
> cope with until we have a proper model of xt's and Forth compilation
> that allows for modern techniques. Once we have a model we can try
> it and define a sensible word set. STATE POSTPONE and IMMEDIATE
> may be casualties of this process in 20 years time.

If we kill IMMEDIATE, what remains of POSTPONE is what once was called
COMPILE, and it probably will be equal to ['] xxx COMPILE, .

[BruceMcF]

On Jun 22, 3:47 am, "Elizabeth D. Rather" <erat...@forth.com> wrote:
> On 6/21/12 8:42 PM, Andrew Haley wrote:
>

> > BruceMcF<agil...@netscape.net>  wrote:

> ...
> >> This also suggests that naming immediate words to highlight their
> >> distinctiveness is good practice, as with [CHAR] [IF] [ELSE] [THEN]
> >> etc, and as might have been done with [TO] to avoid the functional
> >> overloading of TO.
>
> > Or make them a different colour.  However, I think that naming all
> > immediate words like that would make Forth harder to read.  It's
> > always a compromise, I suppose.
>
> The naming convention [...] is conflicted. For [CHAR] and ['] it
> identifies a compile-only version of a normally interpretive word. For
> [IF], [ELSE], and [THEN] it suggests an interpretive version of a
> normally compile-only word. Bottom line: it means, "check your
> documentation."

CHAR when compiling finds the character at run-time, [CHAR] when
compiling finds the character when compiling. IF when compiling finds
the flag on the data stack at run-time, [IF] when compiling finds the
flag on the data stack when compiling. I do not see the conflict.

That's why implementation dependency constants are immediate, eg:

1 CHARS 1 = CONSTANT [char=au] IMMEDIATE

0 TEST ! TRUE TEST C!
TEST C@ 255 = CONSTANT [char=byte] IMMEDIATE

[BruceMcF]

On Jun 22, 2:42 am, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:

> Or make them a different colour.  However, I think that naming all
> immediate words like that would make Forth harder to read.  It's
> always a compromise, I suppose.

Well, [(] [)] would be going overboard.

> TO and S" seem to be perpetually problematic.

Defining a standard alternate name for buffered-S" would be handy.
Among the multiple problems with the mistake in overloading the name
for the compiled S" and the buffered S" is that it makes compiling the
buffered S" into a definition more intricate than it ought to be.

[Elizabeth D. Rather]

On 6/22/12 9:46 AM, Bernd Paysan wrote:
> Stephen Pelc wrote:
...

>> By comparison, CATCH and THROW had plenty of common practice, albeit
>> in other languages. It has caused no trouble.
>
> We actually went some way further with CATCH as originally envisioned.
> E.g. take all these file-IO words which return an ior. This wasn't
> specified as "you can just throw this, the system will produce a
> meaningful error message". We now all do it that way, we expect an ior
> is something to throw.

Speak for yourself. I assure you, the Forth94 TC was very keen on using
THROW on iors. It didn't seem either appropriate or necessary to specify
it (or any of the other potential uses of THROW); it seemed obvious.

[Albert van der Horst]

In article <2012Jun2...@mips.complang.tuwien.ac.at>,

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>"Elizabeth D. Rather" <era...@forth.com> writes:
>>The naming convention [...] is conflicted. For [CHAR] and ['] it
>>identifies a compile-only version of a normally interpretive word. For
>>[IF], [ELSE], and [THEN] it suggests an interpretive version of a
>>normally compile-only word.
>
>Yes, you can use [IF] interpretively, but you can also use it inside
>colon definitions for conditional compilation. It's a bit cumbersome,
>because you then have to provide a value on the stack at compile time,
>but Forth200x has [DEFINED] and [UNDEFINED] to the rescue.
>
>: foo
> ... [DEFINED] bar [IF] 5 bar [ELSE] ... replacement code ... [THEN] ... ;
>
>In that context the brackets make sense.

Having interpreted IF I have looked whether it made [IF] superfluous,
and concluded that it does a poor job for conditional compilation:

global @ IF
: aap onething ;
ELSE
: aap otherthing ;
THEN

just doesn't work. So one really needs [IF] .

[Albert van der Horst]

In article <97-dnWNi04BM0HnS...@supernews.com>,

This ressembles an old idea of mine. Start with a very simple
indirect threaded Forth, with headers that allow for extra
information. A single one cell flag field goes a long way in
32 bits and 64 bits Forth.
I've added stack information to a version of ciforth, based on
analysis of the lowest (code) words. Then one can optimise away dead
code and execute all code that results in compile time constants,
meanwhile doing inlining. Then inline the assembler code and peephole
that.
See
http://home.hccnet.nl/a.w.m.van.der.horst/forthlecture5.html

The main thing is that we want to exhaust all optimisation possibilities
on the higher level before we look at low level aspects.

(Related: an experiment with an editor that show stack effect.)

The most dramatic results I expect in the use of generic tools like
a quicksort that works with execution tokens:

: my-sort ['] my-swap ['] my-compare qsort ;

At a certain point this collapses to
<constant xt> execute
which can be replaced by xt, which can be inlined etc.
This all happens even before all words like DUP and >R are replaced
by their code sequences, where there will be a lot of POP's and
PUSHES that cancel.

>
>Andrew.

[Bernd Paysan]

Elizabeth D. Rather wrote:

> On 6/22/12 9:46 AM, Bernd Paysan wrote:
>> E.g. take all these file-IO words which return an ior. This wasn't
>> specified as "you can just throw this, the system will produce a
>> meaningful error message". We now all do it that way, we expect an
>> ior is something to throw.
>
> Speak for yourself. I assure you, the Forth94 TC was very keen on
> using THROW on iors. It didn't seem either appropriate or necessary to
> specify it (or any of the other potential uses of THROW); it seemed
> obvious.

Yes, but what seems obvious and what is specified are two things - a
standard has to be specific, not obvious. At least it seemed obvious to
me to implement it that way, and AFAIK there was an RFC in Forth200x
which made it specific.

[Elizabeth D. Rather]

On 6/22/12 10:43 AM, Bernd Paysan wrote:
> Elizabeth D. Rather wrote:
>
>> On 6/22/12 9:46 AM, Bernd Paysan wrote:
>>> E.g. take all these file-IO words which return an ior. This wasn't
>>> specified as "you can just throw this, the system will produce a
>>> meaningful error message". We now all do it that way, we expect an
>>> ior is something to throw.
>>
>> Speak for yourself. I assure you, the Forth94 TC was very keen on
>> using THROW on iors. It didn't seem either appropriate or necessary to
>> specify it (or any of the other potential uses of THROW); it seemed
>> obvious.
>
> Yes, but what seems obvious and what is specified are two things - a
> standard has to be specific, not obvious. At least it seemed obvious to
> me to implement it that way, and AFAIK there was an RFC in Forth200x
> which made it specific.
>

A standard has to be specific about required behavior, but not all the
possible uses to which things may be put. THROW is useful for handling
iors, but it is certainly not required that they be handled that way.

I note that several examples in the Appendix to the File Access wordset
(e.g. A.11.6.1.2080 READ-FILE) show the use of THROW with iors returned.

[jacko]

? Implies if. Nop implies exec without ?. Why test? ... To test before the jump, and to ignore the jumpd result... Maybe I'm drunk, maybe I'm just a cycle analyst....

On Friday, 15 June 2012 21:40:13 UTC+1, Albert van der Horst wrote:
> I find places where ?EXEC is used. This gives an error because a word
> if a word is executed while it never should.
>
> Examples are CODE and END-CODE in an assembler, but does this make
> sense?
>
> : CODE ?EXEC .... ;
> : END-CODE ?EXEC .... ;
>
> By analogy I see I defined my classes like this:
> : class ?EXEC ... ;
> : end-class ?EXEC ... ;
>
> I find it hard to wrap my head around this. In what kind of situation
> would this error trigger?
> I would be equally happy with some other realistic example.

>
> Groetjes Albert
>
>
> --
> --
> Albert van der Horst, UTRECHT,THE NETHERLANDS
> Economic growth -- being exponential -- ultimately falters.
> albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

On Friday, 15 June 2012 21:40:13 UTC+1, Albert van der Horst wrote:
> I find places where ?EXEC is used. This gives an error because a word
> if a word is executed while it never should.
>
> Examples are CODE and END-CODE in an assembler, but does this make
> sense?
>
> : CODE ?EXEC .... ;
> : END-CODE ?EXEC .... ;
>
> By analogy I see I defined my classes like this:
> : class ?EXEC ... ;
> : end-class ?EXEC ... ;
>
> I find it hard to wrap my head around this. In what kind of situation
> would this error trigger?
> I would be equally happy with some other realistic example.

[Ed]

Stephen Pelc wrote:
> On Mon, 18 Jun 2012 15:06:47 +1000, "Ed" <inv...@nospam.com> wrote:
>
> >VFX is loaded with ?COMPs.
>
> Not any more. It was a while ago.

For various reasons I can only run the older demos.

So what do you now use in place of ?COMP ?EXEC etc
(if anything). Does VFX still warn if an attempt is made to
use conditionals IF THEN etc interpretively?

[Andrew Haley]

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>I've been thinking the opposite. Maybe the problem with some of the
>>modern techniques is that they're exposed to the programmer; they're
>>too shallow. Perhaps the right solution is to let the Forth compiler
>>generate tokens (just like a very trad Forth) and then let the
>>optimizing compiler run over them.
>
> That's an implementation issue, but it's the right approach.
>
>>Let IMMEDIATE do what it used to do. Get rid of other ways of
>>creating words that execute at compile time.
>
> There is no other way in the standard.

True, but there are other ways that an implementer may use internally,
as we konw.

> A problem may be that the other ways are *not* exposed to the
> programmer (in standard programs), so people still resort to
> STATE-smartness in standard programs.
>
> Anyway, it's not totally clear what you mean by this.
>
> One interpretation is that we should go back to implementing S" and
> TO with STATE-smart words, and to make STATE-smart words an accepted
> technique, and to hell with xts, POSTPONE, code generators and other
> guru stuff that's troublesome in the presence of STATE-smart words.

There's no need to use state-smartness for any words except S" and TO .
I don't think this alone makes state-smartness an accepted technique;
just one to be tolerated.

Given no means other than IMMEDIATE to create words with compilation
semantics, FIND would have one unambigous defintion.

> Another take on your statement is that we should get rid of words like
> S" and TO (and not design more of the kind) which some implementations
> implement with STATE-smartness, for which other implementations
> provide implementations with better properties, but with repercussions
> that are far more trouble than these words are worth.

Probably, yes.

All of this is very interesting, but I wasn't proposing anything as
radical.

My solution is more "Back to the future": a Forth that looks to the
programmer like a traditional implementation, but does as much
optimization as you want under the hood. One that does its tricks,
but not in a way that confuses FIND with several different kinds of
token. We know from our discussions here that isn't going to work.

Andrew.

[Stephen Pelc]

On Sat, 23 Jun 2012 15:36:50 +1000, "Ed" <inv...@nospam.com> wrote:

>So what do you now use in place of ?COMP ?EXEC etc
>(if anything). Does VFX still warn if an attempt is made to
>use conditionals IF THEN etc interpretively?

We allow the compilation behaviour to be defined separately
from the interpretation behaviour.

dis if
IF
( 00409E20 E86FF5FFFF ) CALL 00409394 NOINTERP
( 00409E25 C3 ) NEXT,
( 6 bytes, 2 instructions )
ok

IF has a separate compilation behaviour. This allows all the
structure words to be non-IMMEDIATE. The problem that this approach
solves is that IMMEDIATE specifies that the interpretation and
compilation actions are the same. In this case, in order to
differentiate, e.g. when compiling native code, one has to be
STATE smart, which is a "bad thing".

This approach also makes it very easy to have both interpreted
and compiled behaviours in S" and friends.

[Anton Ertl]

Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:

>> One interpretation is that we should go back to implementing S" and
>> TO with STATE-smart words, and to make STATE-smart words an accepted
>> technique, and to hell with xts, POSTPONE, code generators and other
>> guru stuff that's troublesome in the presence of STATE-smart words.
>
>There's no need to use state-smartness for any words except S" and TO .

Yes, and with a more capable system it's not even needed for S" and TO.

However, there is a reason why we got S" and TO, and that reason does
not stop at these words: people want to be able to write code (that
includes parsing words) that works both interpretively and inside
colon definitions. And they will define such words, and if no other
options are available, they will define STATE-smart words.

>Given no means other than IMMEDIATE to create words with compilation
>semantics, FIND would have one unambigous defintion.

For user-defined words yes, but what about system-defined words like
S" and TO? Do you want to require them to be STATE-smart?

>My solution is more "Back to the future": a Forth that looks to the
>programmer like a traditional implementation, but does as much
>optimization as you want under the hood.

That is definitely a good idea. The smart COMPILE, goes in that
direction, but I guess what you have in mind is optimization at an
even later stage.

> One that does its tricks,
>but not in a way that confuses FIND with several different kinds of
>token. We know from our discussions here that isn't going to work.

Actually it does work (Gforth is one example), it's just pretty awful.

And it's certainly not optimal to put code generation/optimization at
a level visible through FIND; e.g., consider ['] + COMPILE, in
cmForth; it will compile a call to a colon definition that performs a
+ instead of just compiling +. An intellingent COMPILE, does not have
this issue.

[Anton Ertl]

Bernd Paysan <bernd....@gmx.de> writes:

>Elizabeth D. Rather wrote:
>> Speak for yourself. I assure you, the Forth94 TC was very keen on
>> using THROW on iors. It didn't seem either appropriate or necessary to
>> specify it (or any of the other potential uses of THROW); it seemed
>> obvious.
>
>Yes, but what seems obvious and what is specified are two things - a
>standard has to be specific, not obvious. At least it seemed obvious to
>me to implement it that way, and AFAIK there was an RFC in Forth200x
>which made it specific.

IIRC we talked about it, but there is no RfD/CfV yet.

[Anton Ertl]

m...@iae.nl (Marcel Hendrix) writes:
>an...@mips.complang.tuwien.ac.at (Anton Ertl) writes Re: Alternatives to S" and TO (was: ?EXEC)

>> 1) I think that this feature adds relatively little complexity to the
>> text interpreter, and it gets rid of a huge amount of baggage
>> elsewhere (STATE-smart words and the restrictions that their defenders
>> want to put upon us, or the complexities of the alternatives and their
>> ramifications).
>
>The iForth text interpreter is already hugely complicated by workarounds
>that are of no real benefit to the user.

Hmm, so would you like to have this feature because it would replace
some of the workarounds? Or would you like not to have it because
even if taken by itself, the complexity is not big, it adds to the
already considerable complexity?

>> 2) One could make it possible for "mere" users to add further
>> prefixes/denotations (or, as Gforth calls them, recognizers) to the
>> text interpreter, just like it is possible to add further words now.
>
>There is that old idea to factor the text interpreter, that would fit here.
>
>> Of course some people will fear giving this much power to the users,
>> and it's certainly a feature that should be used sparingly, but that's
>> a management issue.
>
>Who are these people?

There always tend to be such people, but it's hard to predict who
turns out to be in that camp on a specific issue. We will learn at
the latest when someone makes an RfC for this feature.

[Alex McDonald]

On Jun 22, 4:19 pm, an...@mips.complang.tuwien.ac.at (Anton Ertl)
wrote:

Agreed. Quoted string support is a long overdue enhancement. We
already have single quoted characters ( 'A' ).

>
> There are a few parsing words where the replacement requires its own
> treatment in the text interpreter, in particular TO.  Instead of
>
> TO v
>
> we could write
>
> ->v
>
> and the text interpreter would do the appropriate thing with V.

While we're at it, junk ACTION-OF and IS as well.

>
> Albert van der Horst used to call this feature "denotations", but
> lately I have seen him use "prefixes".
>
> Many will cry out against this complication of the text interpreter.
> Some may also find the idea bad because up to now the text interpreter
> is a black box so any additions there are only done by the compiler
> vendor.  However:
>
> 1) I think that this feature adds relatively little complexity to the
> text interpreter, and it gets rid of a huge amount of baggage
> elsewhere (STATE-smart words and the restrictions that their defenders
> want to put upon us, or the complexities of the alternatives and their
> ramifications).
>
> 2) One could make it possible for "mere" users to add further
> prefixes/denotations (or, as Gforth calls them, recognizers) to the
> text interpreter, just like it is possible to add further words now.
> Of course some people will fear giving this much power to the users,
> and it's certainly a feature that should be used sparingly, but that's
> a management issue.

WF32* has supported such a scheme for a number of years now,
specifically to support prefixed numbers; but it can support any other
token that's not found in the dictionary. See below.

*(my now so heavily modified Win32Forth that a name change is in
order)

>
> >This is probably worthy of a workshop at EuroFORTH.
>
> Yes.

I would like to be there in person, but can't attend.

If you or Andrew are interested in an implementation, I can supply the
WF32 ANS compliant code that supports it, along with examples. The
text interpreter remains simple, as each recogniser is plugged into a
chain of recognisers that are successively invoked with a CATCH and
the signature ( addr len -- double ) (augmented with a DOUBLE flag to
indicate if it's a single, double, float cell value). On failure, a
recogniser does a THROW and the next in the chain is tried until
success or the last fails, when a -13 THROW is percolated up to the
parser. There's a chain for compiling and a chain for interpreting.

Here's how it supports a quoted string given the above support
structure;

: interpret-qstring ( addr len -- d )
dup 2 < throw
over c@ >r 2dup + c@ r> '"' dup d= if
1 /string 1-
true to double ( flag for type of number)
else true throw then ;

: compile-qstring ( addr len -- d )
interpret-string
here over 2dup 2>r 2+ allot place 2r> ; ( counted str, null
terminated)

' interpret-qstring interpret-chain chain+ ( add in to interpret-chain
list)
' compile-qstring compile-chain chain+ ( add in to compile-chain
list)

The DOUBLE flag is a bit messy (FALSE indicates a single cell, and
there is another variable to indicate if it's to be interpreted as a
pointer to a float value), but that's a historical hangover.

[Bernd Paysan]

Alex McDonald wrote:
> Agreed. Quoted string support is a long overdue enhancement. We
> already have single quoted characters ( 'A' ).

Here's the implementation in Gforth, just written in a few minutes
(recognizers are really great :-):

: slit, postpone sliteral ;

' noop ' slit, dup recognizer: r:string

: string-recognizer ( addr u -- addr u' r:string | addr u r:fail )
2dup s\" \"" string-prefix?
IF drop source drop - 1+ >in ! \"-parse save-mem r:string
ELSE r:fail THEN ;

' string-recognizer
forth-recognizer get-recognizers
1+ forth-recognizer set-recognizers

We have only one recognizer chain, and instead of passing a flag, we
return a vtable, which contains three methods: how to interpret the
thing, how to compile the thing, and how to serialize the thing
(serializing is for POSTPONE,). "Thing" is the rest the recognizer
returns (token, number, float, whatever).

Of course, we also want ."Print quoted string":

: slit. slit, postpone type ;

' type ' slit. ' slit, recognizer: r:."

: ."-recognizer ( addr u -- addr u' r:." | addr u r:fail )
2dup ".\"" string-prefix?
IF drop source drop - 2 + >in ! \"-parse save-mem r:."
ELSE r:fail THEN ;

' ."-recognizer
forth-recognizer get-recognizers
1+ forth-recognizer set-recognizers

The recognizer actions do the parse-time stuff, and pass the stuff for
further processing to the text interpreter - or whoever uses the
recognizer (like ]] for postponing multiple words). Yes, you can now
postpone string literals:

: test ]] "This is " type ."a test." [[ ; immediate ok
: test2 test ; ok
test2 This is a test. ok

Note that ." something" will partly conflict with normal .", as this is
a Forth word, and will be recognized first. So until we get rid of .",
print something starting with a space is

."\ something with a space"

I'm really for recognizer to solve the parse-time problem. They do it
nicely, they *only* work at parse-time, and they just tokenize. I.e.
the text processing is done, but whether to interpret or to compile (or
postpone) is not yet decided.

[Anton Ertl]

Alex McDonald <bl...@rivadpm.com> writes:
>On Jun 22, 4:19=A0pm, an...@mips.complang.tuwien.ac.at (Anton Ertl)
>wrote:

>> There are a few parsing words where the replacement requires its own

>> treatment in the text interpreter, in particular TO. =A0Instead of

>>
>> TO v
>>
>> we could write
>>
>> ->v
>>
>> and the text interpreter would do the appropriate thing with V.
>
>While we're at it, junk ACTION-OF and IS as well.

Yes, of course. "IS d" might become "->d", too. "ACTION-OF d" could
be replaced with "' d defer@", and "' d", "['] d" might also be
replaced with some prefix-based syntax.

[Anton Ertl]

Bernd Paysan <bernd....@gmx.de> writes:
>Of course, we also want ."Print quoted string":

"print quoted string" TYPE

is good enough for me. And it does not introduce the ." bla" ambiguity.

[Andrew Haley]

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>> One interpretation is that we should go back to implementing S" and
>>> TO with STATE-smart words, and to make STATE-smart words an accepted
>>> technique, and to hell with xts, POSTPONE, code generators and other
>>> guru stuff that's troublesome in the presence of STATE-smart words.
>>
>>There's no need to use state-smartness for any words except S" and TO .
>
> Yes, and with a more capable system it's not even needed for S" and TO.

Not by my definition of state-smart, but OK. I think I'd better start
saying something else. We need a tidy phrase that refers to words
with compilation semantics and non-default interpretation seamntics.
"Frankensetein words", perhaps.

>>One that does its tricks, but not in a way that confuses FIND with
>>several different kinds of token. We know from our discussions here
>>that isn't going to work.

> Actually it does work (Gforth is one example)

I think you're overstating that a litle. FIND makes perfect sense
when all words with compilation semantics are created by IMMEDIATE; it
doesn't make as much sense if such words are created in another way.
It seems to me that we (not just you and me, everyone) tied ourselves
in knots trying to figure out what FIND means in that situation. The
only solutions that made sense were (IMO over-) complicated
replacements for FIND.

> , it's just pretty awful.

Andrew.

[Anton Ertl]

Bernd Paysan <bernd....@gmx.de> writes:
>Anton Ertl wrote:
>> There are no interpretation semantics of IF in play in this example.
>> POSTPONE IF compiles the compilation semantics of IF into GEN-FOO, and
>> GEN-FOO then performs the compilation semantics of IF.
>
>So this works if there is a compilation token for IF, which only
>represents the compilation semantics, and not the interpretation
>semantis, which the implementer of a Forth system is free to decide what
>to do with (parse for the next ELSE/THEN in the input stream like [IF]
>does, or report an error, or just compile a forward branch without
>warning).
>
>So far, we have a few solutions to this problem, with words like TO and
>S" being the hardest:

[...]

>* Dual-xt solutions (whether dual-wordlist or a special flag in the
>header like in Gforth) result in a state-smart FIND, which just moves
>the problem around.

Not in the least. With these solutions the GEN-FOO example and S" and
TO work as described in the standard, so the problem has not just
moved around, it has been solved.

Concerning FIND, it's not a good fit for such systems, true. That's
easy to solve by getting rid of FIND, which is also a good idea for
other reasons (counted string input). Replace it with something
cleaner; there are so many better alternatives to FIND possible that
we have not been able to find consensus on which one we want to
standardize on. But if you insist on keeping FIND, having FIND check
STATE is much less problematic in my experience than more typical
STATE-smartness.

Anyway, for stuff like the GEN-FOO example such a system is actually
not necessary. A traditional-style single-xt-with-immediate system
can work for that, too. But ?COMP will break it. So ?COMP is the
problem.

>* Intelligent COMPILE, solutions, which however "violate" the spec of
>COMPILE,

A COMPILE, that violates (not just "violates") the spec of COMPILE, is
not intelligent, but broken.

[Anton Ertl]

Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>> One interpretation is that we should go back to implementing S" and
>>>> TO with STATE-smart words, and to make STATE-smart words an accepted
>>>> technique, and to hell with xts, POSTPONE, code generators and other
>>>> guru stuff that's troublesome in the presence of STATE-smart words.
>>>
>>>There's no need to use state-smartness for any words except S" and TO .
>>
>> Yes, and with a more capable system it's not even needed for S" and TO.
>
>Not by my definition of state-smart, but OK. I think I'd better start
>saying something else.

Good idea.

> We need a tidy phrase that refers to words
>with compilation semantics and non-default interpretation seamntics.
>"Frankensetein words", perhaps.

Fine with me. Although I think you mean "words with interpretation
semantics and non-default non-immediate compilation semantics". I
used to call them "combined words", but it has not caught on. Maybe
Frankenstein words is catchy enough:-).

>>>One that does its tricks, but not in a way that confuses FIND with
>>>several different kinds of token. We know from our discussions here
>>>that isn't going to work.
>
>> Actually it does work (Gforth is one example)
>
>I think you're overstating that a litle. FIND makes perfect sense
>when all words with compilation semantics are created by IMMEDIATE; it
>doesn't make as much sense if such words are created in another way.
>It seems to me that we (not just you and me, everyone) tied ourselves
>in knots trying to figure out what FIND means in that situation.

Not quite. Pretty early (like 15 years ago) I got the idea from
discussions with TC members that FIND should work in a user-defined
interpreter, and implemented it that way, and that has not changed
since. So, as an implementor there were no knots, although the result
is pretty awful (about as awful as specifying that in the standard
would be).

The problem I see is that this is not specified in the standard. The
standard specifies FIND so unclearly that it cannot really be used for
anything useful.

In addition, FIND has other problems like it's counted-string-based
interface.

Another problem that I see is: What's the point of designing FIND (or
its replacement) for a user-written interpreter? Nobody uses such an
interpreter in standard programs, because there is no way to plug such
an interpreter into the existing infrastructure (INCLUDED EVALUATE
...).

>The
>only solutions that made sense were (IMO over-) complicated
>replacements for FIND.

If they make sense, maybe they are not overcomplicated, just
complicated enough.

Even if we just have normal, immediate ("("), and compile-only words
(IF), or maybe just normal and immediate ("(" and "IF") words, it
seems to me that using a FIND-style interface for extracting
interpretation semantics and compilation semantics is by no means
simpler than having a word for extracting interpretation semantics and
a word for extracting compilation semantics.

The user-written text interpreter would look like (something like):

parse-name state @ if
find-comp if execute else ( convert to a number ) postpone literal then
else
find-int if execute else ( convert to a number ) then
then

instead of

bl word find dup if
0< state @ state @ or if execute else compile, then
else
( convert a number) state @ if postpone literal then
then

(both simplified; the number case is further complicated by doubles
and floats).

The nesting is reversed, but there does not seem to be much of a
difference in complication level (especially if you also look at
doubles etc.).

[Albert van der Horst]

In article <3e9da4e8-0ce2-4758...@k5g2000vbf.googlegroups.com>,

Alex McDonald <bl...@rivadpm.com> wrote:
>
>If you or Andrew are interested in an implementation, I can supply the
>WF32 ANS compliant code that supports it, along with examples. The
>text interpreter remains simple, as each recogniser is plugged into a
>chain of recognisers that are successively invoked with a CATCH and
>the signature ( addr len -- double ) (augmented with a DOUBLE flag to
>indicate if it's a single, double, float cell value). On failure, a
>recogniser does a THROW and the next in the chain is tried until
>success or the last fails, when a -13 THROW is percolated up to the
>parser. There's a chain for compiling and a chain for interpreting.

The mechanism in ciforth is much simpler. A word from the input stream
is found in the dictionary if its first part matches the prefix.
So based on the prefix flag the comparison is done over the length
of the prefix instead of the length of the word by SEARCH-WORDLIST.

That's all. So "AAAP" strings are recognized by a word " in the
ONLY wordlist.
: " ... ; PREFIX IMMEDIATE

In particular prefixes have no relation to each other and can reside
in a wordlist where they belong. So the order is under user control
in a familiar way.

This is more robust, the code for " is only called with a "AAP"
string. If it is not implemented correctly, it still cannot interfere
with regular numbers.

[Albert van der Horst]

In article <js5l9b$vhr$1...@online.de>, Bernd Paysan <bernd....@gmx.de> wrote:
>
>Of course, we also want ."Print quoted string":

Isn't it time to want $. $? instead?

"Print quoted string" $.

"Print quoted string" PAD $!
PAD $?

>
>--
>Bernd Paysan

[Bernd Paysan]

Anton Ertl wrote:

> Bernd Paysan <bernd....@gmx.de> writes:
>>So far, we have a few solutions to this problem, with words like TO
>>and S" being the hardest:
> [...]
>>* Dual-xt solutions (whether dual-wordlist or a special flag in the
>>header like in Gforth) result in a state-smart FIND, which just moves
>>the problem around.
>
> Not in the least. With these solutions the GEN-FOO example and S" and
> TO work as described in the standard, so the problem has not just
> moved around, it has been solved.

By now creating a problem in FIND. And as the discussion about a
replacement of FIND have shown, moving the problem there does not make
the situation easier.

> But if you insist on keeping FIND, having FIND check
> STATE is much less problematic in my experience than more typical
> STATE-smartness.

I've no problems with the more typical STATE-smartness, either. I just
know that the state is not just there for fun, but that compilation
state means when I want to compile something, I have to have STATE=true.
I need it to find the right token (if I use FIND), and I need it to get
the right reaction (if I use POSTPONE).

You said, you don't use FIND in your code, but I do. I've considered
rewriting my OOF to use Gforth's double tokens, but immediately realized
that this solves exactly nothing, because my OOF first uses FIND and
then EXECUTE or COMPILE, (depending on the flag) - all on potentially
state smart words, because I need quite a bunch of them. Either I have
the problem in FIND, or in EXECUTing the word, but the problem remains
the same.

> Anyway, for stuff like the GEN-FOO example such a system is actually
> not necessary. A traditional-style single-xt-with-immediate system
> can work for that, too. But ?COMP will break it. So ?COMP is the
> problem.

?COMP is a very traditional way of doing this check. Header flags are
more modern. Header flags have their own set of problems (particularly
that the xt comes without them).

>>* Intelligent COMPILE, solutions, which however "violate" the spec of
>>COMPILE,
>
> A COMPILE, that violates (not just "violates") the spec of COMPILE, is
> not intelligent, but broken.

IMHO the spec of COMPILE, is broken. It doesn't really fit to what
Mitch Bradley said it should fit to - his user-written text interpreter.
It fits in a particular classical implementation, with the FIND we have
now.

[Ed]

Stephen Pelc wrote:
> On Sat, 23 Jun 2012 15:36:50 +1000, "Ed" <inv...@nospam.com> wrote:
>
> >So what do you now use in place of ?COMP ?EXEC etc
> >(if anything). Does VFX still warn if an attempt is made to
> >use conditionals IF THEN etc interpretively?
>
> We allow the compilation behaviour to be defined separately
> from the interpretation behaviour.
>
> dis if
> IF
> ( 00409E20 E86FF5FFFF ) CALL 00409394 NOINTERP
> ( 00409E25 C3 ) NEXT,
> ( 6 bytes, 2 instructions )
> ok
>
> IF has a separate compilation behaviour. This allows all the
> structure words to be non-IMMEDIATE. The problem that this approach
> solves is that IMMEDIATE specifies that the interpretation and
> compilation actions are the same. In this case, in order to
> differentiate, e.g. when compiling native code, one has to be
> STATE smart, which is a "bad thing".
>
> This approach also makes it very easy to have both interpreted
> and compiled behaviours in S" and friends.

Thanks. I take it from your disassembly that executing IF BEGIN
DO etc interpretively in VFX still results in an error and abort.

[Stephen Pelc]

On Mon, 25 Jun 2012 14:26:30 +1000, "Ed" <inv...@nospam.com> wrote:

>Thanks. I take it from your disassembly that executing IF BEGIN
>DO etc interpretively in VFX still results in an error and abort.

Yes

[Andrew Haley]

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>>> One interpretation is that we should go back to implementing S" and
>>>>> TO with STATE-smart words, and to make STATE-smart words an accepted
>>>>> technique, and to hell with xts, POSTPONE, code generators and other
>>>>> guru stuff that's troublesome in the presence of STATE-smart words.
>>>>
>>>>There's no need to use state-smartness for any words except S" and TO .
>>>
>>> Yes, and with a more capable system it's not even needed for S" and TO.
>>
>>Not by my definition of state-smart, but OK. I think I'd better start
>>saying something else.
>
> Good idea.
>
>> We need a tidy phrase that refers to words
>>with compilation semantics and non-default interpretation seamntics.
>>"Frankensetein words", perhaps.
>
> Fine with me. Although I think you mean "words with interpretation
> semantics and non-default non-immediate compilation semantics".

By state-smart I mean words with interpretation semantics and
non-default compilation semantics. As far as I'm concerned, it's not
really of any huge consquence how they're created, whether by
IMMEDIATE or some other means.

>>The only solutions that made sense were (IMO over-) complicated
>>replacements for FIND.
>
> If they make sense, maybe they are not overcomplicated, just
> complicated enough.

I don't agree with you about that: they were complicated because the
requirement was complicated. Forth tradition is to simplify
requirements as well as solutions.

> Even if we just have normal, immediate ("("), and compile-only words
> (IF), or maybe just normal and immediate ("(" and "IF") words, it
> seems to me that using a FIND-style interface for extracting
> interpretation semantics and compilation semantics is by no means
> simpler than having a word for extracting interpretation semantics and
> a word for extracting compilation semantics.
>
> The user-written text interpreter would look like (something like):
>
> parse-name state @ if
> find-comp if execute else ( convert to a number ) postpone literal then
> else
> find-int if execute else ( convert to a number ) then
> then
>
> instead of
>
> bl word find dup if
> 0< state @ state @ or if execute else compile, then
> else
> ( convert a number) state @ if postpone literal then
> then
>
> (both simplified; the number case is further complicated by doubles
> and floats).
>
> The nesting is reversed, but there does not seem to be much of a
> difference in complication level (especially if you also look at
> doubles etc.).

I agreee that makes the most sense of the various schemes that was
proposed. There is some hidden complexity, though. I presume this is
the scheme where FIND-COMP returns two XTs, one to execute and one to
compile.

Andrew.

[BruceMcF]

On Jun 25, 5:49 am, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:

> I agreee that makes the most sense of the various schemes that was
> proposed.  There is some hidden complexity, though.  I presume this is
> the scheme where FIND-COMP returns two XTs, one to execute and one to
> compile.

The flag free version of find-comp returns two xt's,

FIND-COMP ( xt1 xt2 -- )
... where the stack diagram for xt2 is ( xt -- )

... which allows simple implementations to implement it by just
checking an immediate flag and returning xt2 as either EXECUTE or
COMPILE, allows dual-xt systems on gforth's approach to just return
their pair of xt's and allows dual-xt systems with a stand-alone
compile xt to just return EXECUTE and the compile-xt.

[Andrew Haley]

BruceMcF <agi...@netscape.net> wrote:
> On Jun 25, 5:49?am, Andrew Haley <andre...@littlepinkcloud.invalid>

Yes, that's what I thought. The top XT is always executed, the second
compiled (or executed).

Thanks,
Andrew.

[Bernd Paysan]

Bernd Paysan wrote:

> Alex McDonald wrote:
>> Agreed. Quoted string support is a long overdue enhancement. We
>> already have single quoted characters ( 'A' ).
>
> Here's the implementation in Gforth, just written in a few minutes
> (recognizers are really great :-):

Ok, I also wrote the ->something recognizer, and had to factor the
nonames for TO a little bit, but then it was easy:

' (int-to) ' (comp-to) ' lit, recognizer: r:to

: to-recognizer ( addr u -- xt r:to | addr u r:fail )
2dup s" ->" string-prefix? 0= IF r:fail EXIT THEN
2dup 2 /string dup 0= IF 2drop r:fail EXIT THEN
find-name dup 0= IF drop r:fail EXIT THEN
name>comp drop nip nip r:to ;

' to-recognizer

forth-recognizer get-recognizers
1+ forth-recognizer set-recognizers

(int-to) and (comp-to) are the two factors that already go into TO. You
now can even postpone things like "update the first local defined" with

: >l0 { a } postpone ->a ; immediate

[Ed]

While IMMEDIATE and STATE can pose certain issues I don't see it as
the "evil" that's being made out.

Definers such as CONSTANT USER etc were classically not immediate
because there was no need. Compiler words (IF DO LITERAL etc) were
immediate in Forth-79, 83 in recognition that, unlike regular words, these
needed to execute when in compilation state.

Forth-94 relaxed the immediacy tag on these words simply stating they
had "no interpretation semantics". This doesn't stop one from keeping
them immediate, and of course it makes adding compiler security easy.

Compiler words are also differentiated from regular words in that 'Standard
programs' are not permitted to tick or FIND them. That immediate and
state-smart words must be handled with care should not be surprising
given one is dealing with a special class of words. Forth never claimed
to be user-proof - I doubt any language which allows the user to fool with
the compiler to the extent that Forth does, can be.

Forth has co-existed with immediacy and state-smartness for a very
long time and many (most?) systems currently employ it. It hasn't
stopped users from creating substantial applications for over four
decades. It can't have been too evil - whatever problems it may cause
for the Standard.

Most things come with a cost and removing immediacy/state-smartness
has its own. Small systems in particular may see the cost outweigh
any benefit.

[Andrew Haley]

Ed <inv...@nospam.com> wrote:
> Compiler words are also differentiated from regular words in that 'Standard
> programs' are not permitted to tick or FIND them.

No. The Forth programmer has an entitlement to write an interpreter,
so compiler words must work with FIND .

> Forth has co-existed with immediacy and state-smartness for a very
> long time and many (most?) systems currently employ it.

By definition, standard systems have to: it's a requirement.

> It hasn't stopped users from creating substantial applications for
> over four decades. It can't have been too evil

Yeah, it can. STATE is one of those features, like COBOL's notorious
ALTER verb, that have their uses but are best avoided if possible.

> - whatever problems it may cause for the Standard.

STATE doesn't cause any problems for the standard AFAIK .

Andrew.