State and the standard ...

Rob Sciuk

unread,

Feb 28, 2013, 11:35:32 AM2/28/13

to

I'm wondering if, as the arguments proffered in that other thread seem to
indicate, state-fulness is generally considered a bad thing, then perhaps
a future Forth standard should address the issue head on??? I must
confess that I have always found [ ] postpone and friends to be confusing.

Multiple CFA's is simplistic (and an implementation detail which should
not be enshrined in a standard), but given the umbilical nature of modern
Forth such an approach need not be wasteful, in the sense that code
generated need not retain the extra header fields in run time code
produced ... (or any headers at all for that matter).

To play Gavino's advocate for a second, and throw out a hypothetical --

How might one describe a standardized language similar to Forth in such a
way as to avoid the run-time/compile-time semantics, but still allow
create/does type extensibility? Am I missing something obvious?

Cheers,
Rob.

Rod Pemberton

unread,

Mar 1, 2013, 8:46:05 PM3/1/13

to

"Rob Sciuk" <r...@controlq.com> wrote in message
news:alpine.BSF.2.00.1...@yoko.controlq.com...

>
> I'm wondering if, as the arguments proffered in that other
> thread seem to indicate, state-fulness is generally considered a
> bad thing, then perhaps a future Forth standard should address
> the issue head on??? I must confess that I have always found
> [ ] postpone and friends to be confusing.
>

STATE seems to be critical to a number of words in ANS. I.e., it
seems impossible to be able to implement them without STATE.

STATE also seems to be required to implement a historical Forth
interpreter.

Despite the standard recommendation to use POSTPONE , I found that
using COMPILE and [COMPILE] keeps things clearly marked in my
system definitions. You need to know which is which in regards to
immediacy when coding a definition. But, otherwise, I don't like
having two words for the job of one. For user code, POSTPONE is
probably better. It works 98% of the time without the user
knowing what type of word is being compiled.

For an interpreted Forth, [ ] just switch a Forth interpreter's
STATE variable from interpret to compile. I.e., it tells the
inner/address interpreter to either compile or interpret/execute a
word.

For my interpreter, these set STATE:

: ; [ ] QUIT

These check STATE:

LITERAL S" ." C" QUIT TO
inner_interpreter_routine_which_compiles_words
inner_interpreter_routine_which_interprets_words

LITERAL is on my "To Do." list to have STATE removed. But, ANS
seems to require STATE for S" ." and C" . How does one implement
them without it? STATE also seems to required in Forth
interpreters for QUIT : ; [ ] and the inner/address interpreter.
How does a compiled Forth implement two states, compile and
interactive, without STATE? STATE or it's equivalent seems to be
required even for compiled Forths to implement two modes. STATE's
use might be minimized, but it seems to still be required.

> Multiple CFA's is simplistic (and an implementation detail which
> should not be enshrined in a standard), but given the umbilical
> nature of modern Forth such an approach need not be wasteful, in
> the sense that code generated need not retain the extra header
> fields in run time code produced ... (or any headers at all for
> that matter).
>

CFA's can be completely eliminated *if* you have STATE.

However, if you decide to eliminate STATE, then you either 1) need
to break STATE-aware words apart into separate words for each mode
of operation, or 2) need multiple CFA's.

> [...]

> How might one describe a standardized language similar to Forth
> in such a way as to avoid the run-time/compile-time semantics,

[...]

_Everyone_ here is always touting Forth because it's so
interactive. That's like one of the top ten if not the first
ranked ability of Forth. Well, without STATE-aware words, you
can't create and use the exact same code sequence both
interactively and in a colon-definition. That's only possible
with STATE-aware words. It's a waste of time if you attempt it.
So, how interactive is Forth without STATE-aware words? I.e., not
as interactive as it could be or was once... When the use of
STATE-aware words declined, Forth words for each mode of operation
appeared, such as '(tick) and ['] or COMPILE and [COMPILE] etc.
With mode based words, to me, it's as if Forth lost much of it's
interactive "character". Personally, I happen to like a Forth
where a Forth word works EXACTLY the same way whether used
interactively or compiled into a definition. AIUI, that means
some STATE-aware words. But, I don't like this because of the
interactiveness. I like it because of the consistency. I.e., it
works the same in both modes, and it's available in both modes.

> but still allow create/does type extensibility? Am I missing
> something obvious?
>

How often is "create/does type extensibility" _actually_ used? I
think that's what you're missing. Forget that CREATE .. DOES> is
"one of the top ten ranked features of Forth" for a moment. Let
the myth and hyperbole lie. How often is it truly and honestly
used? Can those situations be eliminated? In my mind, the
answers are "very infrequently", "very infrequently", "yes".

AFAICT, CREATE .. DOES> is use a few times for certain system
definitions, and it's used occasionally for data structures, ...
It doesn't seem to be required or needed.

Someone (Alex?) posted a word count from their system for other
words recently. Maybe, they can post the static frequency of
CREATE and DOES> also.

Rod Pemberton

Elizabeth D Rather

unread,

Mar 2, 2013, 12:33:02 AM3/2/13

to

On 3/1/2013 3:46 PM, Rod Pemberton wrote:
> "Rob Sciuk" <r...@controlq.com> wrote in message
> news:alpine.BSF.2.00.1...@yoko.controlq.com...
>>
>> I'm wondering if, as the arguments proffered in that other
>> thread seem to indicate, state-fulness is generally considered a
>> bad thing, then perhaps a future Forth standard should address
>> the issue head on??? I must confess that I have always found
>> [ ] postpone and friends to be confusing.
>>
>
> STATE seems to be critical to a number of words in ANS. I.e., it
> seems impossible to be able to implement them without STATE.

The concept is important. Defining it as a variable isn't, as folks here
have repeatedly demonstrated.

> STATE also seems to be required to implement a historical Forth
> interpreter.

Well, it was used is some old systems, but not all. FORTH, Inc. didn't
use it from the early 80's to mid-90's, and only went back to it because
it was in Forth94 and we wanted to be standard.

> Despite the standard recommendation to use POSTPONE , I found that
> using COMPILE and [COMPILE] keeps things clearly marked in my
> system definitions. You need to know which is which in regards to
> immediacy when coding a definition. But, otherwise, I don't like
> having two words for the job of one. For user code, POSTPONE is
> probably better. It works 98% of the time without the user
> knowing what type of word is being compiled.

Yes, POSTPONE is primarily a portability tool.

> For an interpreted Forth, [ ] just switch a Forth interpreter's
> STATE variable from interpret to compile. I.e., it tells the
> inner/address interpreter to either compile or interpret/execute a
> word.
>
> For my interpreter, these set STATE:
>
> : ; [ ] QUIT
>
> These check STATE:
>
> LITERAL S" ." C" QUIT TO
> inner_interpreter_routine_which_compiles_words
> inner_interpreter_routine_which_interprets_words

That's a common strategy.

> LITERAL is on my "To Do." list to have STATE removed. But, ANS
> seems to require STATE for S" ." and C" . How does one implement
> them without it? STATE also seems to required in Forth
> interpreters for QUIT : ; [ ] and the inner/address interpreter.
> How does a compiled Forth implement two states, compile and
> interactive, without STATE? STATE or it's equivalent seems to be
> required even for compiled Forths to implement two modes. STATE's
> use might be minimized, but it seems to still be required.

People here have described many alternatives, including different search
orders during compiling and interpreting, having two different INTERPRET
loops, many possible options.

>> Multiple CFA's is simplistic (and an implementation detail which
>> should not be enshrined in a standard), but given the umbilical
>> nature of modern Forth such an approach need not be wasteful, in
>> the sense that code generated need not retain the extra header
>> fields in run time code produced ... (or any headers at all for
>> that matter).
>>
>
> CFA's can be completely eliminated *if* you have STATE.
>
> However, if you decide to eliminate STATE, then you either 1) need
> to break STATE-aware words apart into separate words for each mode
> of operation, or 2) need multiple CFA's.

...or some other strategy.

>> [...]
>> How might one describe a standardized language similar to Forth
>> in such a way as to avoid the run-time/compile-time semantics,
> [...]

That is the important point. It's not as hard to write some code as it
is to write normative text that is both clear and precise in describing
this concept.

> _Everyone_ here is always touting Forth because it's so
> interactive. That's like one of the top ten if not the first
> ranked ability of Forth. Well, without STATE-aware words, you
> can't create and use the exact same code sequence both
> interactively and in a colon-definition. That's only possible
> with STATE-aware words. It's a waste of time if you attempt it.
> So, how interactive is Forth without STATE-aware words? I.e., not
> as interactive as it could be or was once... When the use of
> STATE-aware words declined, Forth words for each mode of operation
> appeared, such as '(tick) and ['] or COMPILE and [COMPILE] etc.
> With mode based words, to me, it's as if Forth lost much of it's
> interactive "character". Personally, I happen to like a Forth
> where a Forth word works EXACTLY the same way whether used
> interactively or compiled into a definition. AIUI, that means
> some STATE-aware words. But, I don't like this because of the
> interactiveness. I like it because of the consistency. I.e., it
> works the same in both modes, and it's available in both modes.

The thing is, an interpreted ' and a compiled ' *do* do exactly the same
thing, when they are executed. The point you're missing is that a
compiled reference to ' executes when the word it's in is executed, just
like + or DUP. Why should this come as a surprise? The word ['] is quite
different from ' in that it compiles something, which ' does not. If it
had the same name, *that* would be confusing.

>> but still allow create/does type extensibility? Am I missing
>> something obvious?
>>
>
> How often is "create/does type extensibility" _actually_ used? I
> think that's what you're missing. Forget that CREATE .. DOES> is
> "one of the top ten ranked features of Forth" for a moment. Let
> the myth and hyperbole lie. How often is it truly and honestly
> used? Can those situations be eliminated? In my mind, the
> answers are "very infrequently", "very infrequently", "yes".
>
> AFAICT, CREATE .. DOES> is use a few times for certain system
> definitions, and it's used occasionally for data structures, ...
> It doesn't seem to be required or needed.
>
> Someone (Alex?) posted a word count from their system for other
> words recently. Maybe, they can post the static frequency of
> CREATE and DOES> also.

Well, I just counted 38 uses of DOES> in SwiftForth (clean boot), but I
find it used much more often in applications. Still, the magic of DOES>
isn't in frequency of use, it's in the power that it gives you to make
application-specific classes of words. Just a few appropriate uses can
work wonders in cleaning up code and making it more readable.

Learning how to use DOES> effectively is part of learning to use Forth
as Forth, rather than RPN C or some other language.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

Andrew Haley

unread,

Mar 2, 2013, 4:33:54 AM3/2/13

to

Rod Pemberton <do_no...@notemailnotz.cnm> wrote:
> LITERAL is on my "To Do." list to have STATE removed. But, ANS
> seems to require STATE for S" ." and C" . How does one implement
> them without it? STATE also seems to required in Forth interpreters
> for QUIT : ; [ ] and the inner/address interpreter. How does a
> compiled Forth implement two states, compile and interactive,
> without STATE?

You have two separate loops, a compiler loop and an interpreter loop,
rather than a single loop that checks STATE on every word. You don't
need STATE at all, but you might choose to set it in case anybody
wants to make a STATE-smart word.

> _Everyone_ here is always touting Forth because it's so interactive.
> That's like one of the top ten if not the first ranked ability of
> Forth. Well, without STATE-aware words, you can't create and use
> the exact same code sequence both interactively and in a
> colon-definition.

No, you can't. You can't completely do that with STATE-smart words
either. It's of little consequence to practical Forth use.

> That's only possible with STATE-aware words. It's a waste of time
> if you attempt it. So, how interactive is Forth without STATE-aware
> words? I.e., not as interactive as it could be or was once... When
> the use of STATE-aware words declined, Forth words for each mode of
> operation appeared, such as '(tick) and ['] or COMPILE and [COMPILE]
> etc. With mode based words, to me, it's as if Forth lost much of
> it's interactive "character". Personally, I happen to like a Forth
> where a Forth word works EXACTLY the same way whether used
> interactively or compiled into a definition. AIUI, that means some
> STATE-aware words. But, I don't like this because of the
> interactiveness. I like it because of the consistency. I.e., it
> works the same in both modes, and it's available in both modes.

That's a mistake because STATE-smart words are very difficult to use
correctly when metaprogramming. And metaprogramming is far more
important in application development than such niceties as having a
'(tick) that seems to work consistently in compile and interpret mode.

> How often is "create/does type extensibility" _actually_ used?

Eh? All the time. That's one of the key tools for defining
application data structures. It's not much used in Forth itself, but
it's for applications.

> I think that's what you're missing. Forget that CREATE .. DOES> is
> "one of the top ten ranked features of Forth" for a moment. Let the
> myth and hyperbole lie. How often is it truly and honestly used?

Like I said, all the time.

> Can those situations be eliminated?

Well, yes, if you want to dumb down Forth to reverse polish C. But
what would be the point of that? If you want C, use C.

Andrew.

Stephen Pelc

unread,

Mar 2, 2013, 6:34:41 AM3/2/13

to

On Thu, 28 Feb 2013 11:35:32 -0500, Rob Sciuk <r...@controlq.com>
wrote:

>I'm wondering if, as the arguments proffered in that other thread seem to
>indicate, state-fulness is generally considered a bad thing, then perhaps
>a future Forth standard should address the issue head on??? I must
>confess that I have always found [ ] postpone and friends to be confusing.

The problem is that implementing state-smart behaviour using the only
standard form available is bad. Implementing it in other ways is not.
IMHO the standard after Forth2012 should consider how to standardise
a notation to do this.

The only available standard template is

: foo
state @ if ... else ... then
; immediate

Anton's analysis of this is correct. However, there's no standard
alternative. In addition, the proposition that there should be two
words such as CHAR and [CHAR] is not popular among application
programmers, who really like being able to use words such as
." in both compilation and interpretation modes.

Below is a slightly simplified version of ." in VFX Forth. It avoids
the problems of the classical approach above.

: (.") \ --
\ Runtime action of ."
r> count 2dup + aligned >r type
;

: ." \ "ccc<quote>" --
\ Output the text up to the closing double-quotes character.
[char] " word $.
;
comp: ( xt -- ) drop ['] (.") compile, ", ;

Note that ." is not IMMEDIATE - it just has separate interpretation
and compilation behaviours. The problem is just to standardise the
notation.

Stephen

--
Stephen Pelc, steph...@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads

A. K.

unread,

Mar 2, 2013, 7:47:54 AM3/2/13

to

Since long I use COMPILES> similar to DOES>

: <DUALWORD>
<interpretation-semantics> COMPILES> <compilation-semantics> ;

The dictionary headers have two corresponding execution tokens. And I
don't need extra flags for immediate words or compilation-only words.

Bernd Paysan

unread,

Mar 2, 2013, 8:07:42 AM3/2/13

to

Stephen Pelc wrote:
> The problem is that implementing state-smart behaviour using the only
> standard form available is bad. Implementing it in other ways is not.
> IMHO the standard after Forth2012 should consider how to standardise
> a notation to do this.

Yes; IMHO we are at about the point to start doing this; as some Forth
systems have converted to a smart compile, - an approach we didn't use
before.

> : ." \ "ccc<quote>" --
> \ Output the text up to the closing double-quotes character.
> [char] " word $.
> ;
> comp: ( xt -- ) drop ['] (.") compile, ", ;

Gforth's development version now has a smart compile,, too, and the syntax
to use it is pretty similar to your comp:, so well, I should better just
call it comp: to avoid unnecessary discussions about how it's named. The
semantics is identical, it gets an xt. You can have comp: (or compile> as
it is called now) also inside a definition, which then, like DOES>,
overrides the compile,-method of the last definition.

This however requires that we also need to "set the words right", because
the execution and compilation semantics as written in the standard now are
not clear enough for this approach. Anton at least opposes the idea that
COMPILE, is for the compilation semantics, because now it a) isn't, and b)
is not defined that way.

BTW: I'm *not* happy with using something like compile>/comp: inside a
definition. Like DOES>, it tears the definition into two parts, ending the
first, and starting a second one, and thus means that you can't go on and
adding e.g. a DOES>, or a TO-behavior (which we don't need to discuss here,
but is available in Gforth-current). Quotations are more handy for that,
the "assing an xt to the compile, method of the last defined definition" in
Gforth it is now !COMPILE, (preliminary), in VFX ist is SET-COMPILER (and
there is a GET-COMPILER, which deems to be not very useful - the more
generic, to get the compiler method of any XT, would be sufficient).

--
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

Rob Sciuk

unread,

Mar 2, 2013, 12:38:43 PM3/2/13

to

On Sat, 2 Mar 2013, Stephen Pelc wrote:

[snip]

> Note that ." is not IMMEDIATE - it just has separate interpretation
> and compilation behaviours. The problem is just to standardise the
> notation.
>
> Stephen

Stephen, thanks for your reply, in fact thanks to all, I appreciate the
views expressed thus far.

You bring up the concept of immediacy, and it occurs that it is the
combination of STATE and IMMEDIACY which allows the language extension
facility.

Immediate words seem to inherently require state awareness to properly lay
down the correct run time behaviours during compilation.

Take the word ascii, which looks ahead to the next word on the input
stream and pushes the value of the ascii character to the stack. To work
properly in a compilation, it must be immediate, and aware of state, and
in addition to its runtime semantics, must also compile the (literal)
runtime semantics, along with the constant. My implementation looks like
this:

void ascii(){
Str_t p ;

word() ;
p = (Str_t) pop() ;
push( (Cell_t) *p ) ;
if( state == state_Compiling ){
push( (Cell_t) lookup( "(literal)" ) ) ;
comma();
comma();
}
}

And it behaves as expected ...

ok ' ascii see
-- ascii (5058e0) flg: 1 is coded in C (403060).
ok ascii x .
120 ok : z ascii x . cr ;
ok z
120
ok ' z see
-- z (507c00) word flg: 0.
50bc08 (literal) = 120
50bc18 .
50bc20 cr
50bc28 next
ok

I suppose I'm wondering if there is a more elegant mechanism to reconcile
compiling words such that *ALL* words are equivalent, and have for lack of
a better term, BOTH run time and compile time forks, rather than the
current situation where some words are "special" by means of having
immediacy (or not), and state awareness (or not).

Currently, immediacy and state awareness accrue only to certain "special"
words. I'm not saying that this is either good or bad, I'm just wondering
if there is an alternative implementation where such differentiation is
unnecessary and all words simply do the right thing in any context???

Cheers,
Rob.

Anton Ertl

unread,

Mar 2, 2013, 12:37:09 PM3/2/13

to

Rob Sciuk <r...@controlq.com> writes:
>How might one describe a standardized language similar to Forth in such a
>way as to avoid the run-time/compile-time semantics, but still allow
>create/does type extensibility? Am I missing something obvious?

CREATE/DOES> has very little to do with that.

The issue is that you write

1 2 +

and can then wrap it in a colon def

: foo 1 2 + ;

and you write exactly the same sequence "1 2 +". You can also extract
sequences from colon definitions and run them in the text interpreter.

But that does not work for all code. One part where it breaks down is
parsing words like "'". If you put

' + execute

in a colon definition, you don't get the same behaviour. Some people,
like Andrew Haley and Elizabeth Rather, just accept that and say that
everybody should accept that. Other people try to keep this property
even for parsing and control-flow words; one of the techniques used is
STATE-smart words, but they lead to problems, because they do not
always check STATE when the text interpreter sees them. There are
three approaches to that

1) Some take it as a confirmation of their view that we should just
forget about that desired property and be happy with ' ['] and .( .".

2) Some say that the problems of STATE-smart words do not happen in
(their) practice, so we should not worry about them.

3) Some look for (and have found) better solutions that allow
implementing words like S" without getting the problems of STATE-smart
words.

- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2013: http://www.euroforth.org/ef13/

Anton Ertl

unread,

Mar 2, 2013, 12:55:55 PM3/2/13

to

steph...@mpeforth.com (Stephen Pelc) writes:
>: (.") \ --
>\ Runtime action of ."
> r> count 2dup + aligned >r type
>;
>
>: ." \ "ccc<quote>" --
>\ Output the text up to the closing double-quotes character.
> [char] " word $.
>;
>comp: ( xt -- ) drop ['] (.") compile, ", ;
>
>Note that ." is not IMMEDIATE - it just has separate interpretation
>and compilation behaviours. The problem is just to standardise the
>notation.

A notation for defining combined words is certainly an improvement
over STATE-smart words, and if we cannot reach consensus on anything
better, we should go for that.

However, I wonder whether the problem should not be attacked at a
different level, by providing an attractive alternative to parsting
words (e.g., through recognizers); admittedly that would only solve
the problem for parsing words, not for control-flow words. But
parsing words seems to be the most common cause for STATE-smartness,
people seem to be more ready to accept the difference between
interpretation and compilation for control-flow (or maybe it's just
because that's harder to implement).

Bernd Paysan

unread,

Mar 2, 2013, 2:24:03 PM3/2/13

to

Anton Ertl wrote:
> A notation for defining combined words is certainly an improvement
> over STATE-smart words, and if we cannot reach consensus on anything
> better, we should go for that.
>
> However, I wonder whether the problem should not be attacked at a
> different level, by providing an attractive alternative to parsting
> words (e.g., through recognizers); admittedly that would only solve
> the problem for parsing words, not for control-flow words. But
> parsing words seems to be the most common cause for STATE-smartness,
> people seem to be more ready to accept the difference between
> interpretation and compilation for control-flow (or maybe it's just
> because that's harder to implement).

Is it really harder to implement? We have a whole bunch of interactive
control flow structures in Gforth, all with [] around their names.

They would work just as well if they were "smart" IFs and THENs.

IMHO, state+immediate was an ad hoc solution for this kind of problem, and
it turned out to be a bit too simple, and even Chuck realized that. The
next approach at solving this problems was cmForth with different
vocabularies for compilation and interpretation, and it again is too simple.
Your approach was dual-Xts for these special compilation semantics, and it
had some similar deficits. The next approach is smart compile, and the only
issue it has are compatibility problems with the state+immediate approach
(it can't model the deficiencies, and for some applications, bugs are
actually features ;-), so a Forth like VFX or Gforth 0.8 system needs to
support both.

Chucks pretty much sidestepped the problem in ColorForth, where the colors
are the only tokens which parse, and everything else has a more passive
role.

Andrew Haley

unread,

Mar 2, 2013, 4:50:18 PM3/2/13

to

Rob Sciuk <r...@controlq.com> wrote:

>
> Immediate words seem to inherently require state awareness to properly lay
> down the correct run time behaviours during compilation.

That's not true.

> Take the word ascii, which looks ahead to the next word on the input
> stream and pushes the value of the ascii character to the stack. To work
> properly in a compilation, it must be immediate, and aware of state, and
> in addition to its runtime semantics, must also compile the (literal)
> runtime semantics, along with the constant.

No. The right way (IMO, YMMV, etc.) to do this is to have two words,
one immediate and one not. The standard names for these words are
[CHAR] and CHAR. Like this:

: char ( "name" -- char ) bl word 1+ c@ ;
: [char] ( -- ) char postpone literal ; immediate

It's important to separate CHAR and [CHAR] because sometimes you want
to use CHAR (the non-immediate form that parses a char and pushes it
onto the stack) inside a definition. POSTPONE CHAR doesn't get you
the behaviour you need because when CHAR eventually executes it's
STATE-smart: whether it pushes a character onto the stack or compiles
a character depends on STATE. You can't control it. If you really
need the CHAR behaviour regardless of STATE, you're stuck.

Andrew.

Albert van der Horst

unread,

Mar 2, 2013, 5:11:14 PM3/2/13

to

In article <P-qdnRKtTO2H6a_M...@supernews.com>,

Or use &C for a number-like thing. Nobody is tempted to do
POSTPONE &C and if you can separate the & at all, you know you're
extending a very carnal part of the system.
Use &C as a number. No more headaches.

>
>Andrew.
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

Albert van der Horst

unread,

Mar 2, 2013, 5:22:29 PM3/2/13

to

In article <2013Mar...@mips.complang.tuwien.ac.at>,

Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>steph...@mpeforth.com (Stephen Pelc) writes:
>>: (.") \ --
>>\ Runtime action of ."
>> r> count 2dup + aligned >r type
>>;
>>
>>: ." \ "ccc<quote>" --
>>\ Output the text up to the closing double-quotes character.
>> [char] " word $.
>>;
>>comp: ( xt -- ) drop ['] (.") compile, ", ;
>>
>>Note that ." is not IMMEDIATE - it just has separate interpretation
>>and compilation behaviours. The problem is just to standardise the
>>notation.
>
>A notation for defining combined words is certainly an improvement
>over STATE-smart words, and if we cannot reach consensus on anything
>better, we should go for that.
>
>However, I wonder whether the problem should not be attacked at a
>different level, by providing an attractive alternative to parsting
>words (e.g., through recognizers); admittedly that would only solve
>the problem for parsing words, not for control-flow words. But
>parsing words seems to be the most common cause for STATE-smartness,
>people seem to be more ready to accept the difference between
>interpretation and compilation for control-flow (or maybe it's just
>because that's harder to implement).

[That is not hard, as few things in Forth are. It is totally
system-dependant of course. You must not try to implement things
like this as standard ISO-FORTH code.]

Assuming you can do SWAP-DP to go to a separate compilation area:

>R in this simple Forth can be user to begin interpretation form
a starting point

8 \ While compiling, T[ just throws away the state pushed by T].
9 \ Interpreting:
10 \ Start compiling at temporary place : return START and STATE.
11 : T] STATE @ 0= IF SWAP-DP HERE THEN STATE @ ] ;
12 \ Execute code at START dropping STATE, restore dictionary.
13 : T[ 0= IF POSTPONE (;) SWAP-DP POSTPONE [ >R THEN ; IMMEDIATE

3 : IF T] POSTPONE IF ; IMMEDIATE
7 : THEN POSTPONE THEN POSTPONE T[ ; IMMEDIATE

>
>- anton
>--

Groetjes Albert

Howerd

unread,

Mar 2, 2013, 5:50:14 PM3/2/13

to r...@controlq.com

Hi Rob,

> ... an alternative implementation where such differentiation is

> unnecessary and all words simply do the right thing in any context???

I think colorForth is just such an implementation :-)

Best regards,
Howerd

Bernd Paysan

unread,

Mar 2, 2013, 8:51:20 PM3/2/13

to

Albert van der Horst wrote:

> In article <P-qdnRKtTO2H6a_M...@supernews.com>,
> Andrew Haley <andr...@littlepinkcloud.invalid> wrote:
>>It's important to separate CHAR and [CHAR] because sometimes you want
>>to use CHAR (the non-immediate form that parses a char and pushes it
>>onto the stack) inside a definition. POSTPONE CHAR doesn't get you
>>the behaviour you need because when CHAR eventually executes it's
>>STATE-smart: whether it pushes a character onto the stack or compiles
>>a character depends on STATE. You can't control it. If you really
>>need the CHAR behaviour regardless of STATE, you're stuck.
>
> Or use &C for a number-like thing. Nobody is tempted to do
> POSTPONE &C

Works fine with current Gforth, the recognizers do the magic:

: test postpone &5 ; immediate ok
: test1 test ; ok
see test1
: test1
5 ; ok

If you have a special-compilation-word, you can access both the interpreter
and the compiler part without troubles, too:

: foo ." Interpreting" ; comp: ( xt -- ) drop ." Compiling" ; ok
: test1 ['] foo execute ; ok
: test2 postpone foo ; ok
test1 Interpreting ok
test2 Compiling ok

Note that by how the standard is written, both ['] foo execute and postpone
foo do what they should - one is providing the interpretation semantics, the
other the compilation semantics. What' *not* according to the standard is
['] foo compile, since that should add the execution semantics to the
current definition (default compile, behavior).

Now, we are mostly fine in so far as comp: is not a standard word, and
therefore words modified by comp: don't have to behave like standard words.
But what about standard words that have been defined using comp:?

Anton Ertl

unread,

Mar 3, 2013, 9:17:09 AM3/3/13

to

Rob Sciuk <r...@controlq.com> writes:
>Take the word ascii, which looks ahead to the next word on the input
>stream and pushes the value of the ascii character to the stack. To work
>properly in a compilation, it must be immediate, and aware of state, and
>in addition to its runtime semantics, must also compile the (literal)
>runtime semantics, along with the constant. My implementation looks like
>this:
>
>void ascii(){
> Str_t p ;
>
> word() ;
> p = (Str_t) pop() ;
> push( (Cell_t) *p ) ;
> if( state == state_Compiling ){
> push( (Cell_t) lookup( "(literal)" ) ) ;
> comma();
> comma();
> }
>}
>
>And it behaves as expected ...

Try:

: execute1 ] execute postpone [ ;

' char execute a .
' char execute1 a .

' ascii execute a .
' ascii execute1 a .

Does ASCII really behave as expected?

More importantly, can it be used without too many worries in a
reliable program? I don't think so.

Anton Ertl

unread,

Mar 3, 2013, 9:25:02 AM3/3/13

to

Bernd Paysan <bernd....@gmx.de> writes:
>Anton Ertl wrote:
>> A notation for defining combined words is certainly an improvement
>> over STATE-smart words, and if we cannot reach consensus on anything
>> better, we should go for that.
>>
>> However, I wonder whether the problem should not be attacked at a
>> different level, by providing an attractive alternative to parsting
>> words (e.g., through recognizers); admittedly that would only solve
>> the problem for parsing words, not for control-flow words. But
>> parsing words seems to be the most common cause for STATE-smartness,
>> people seem to be more ready to accept the difference between
>> interpretation and compilation for control-flow (or maybe it's just
>> because that's harder to implement).
>
>Is it really harder to implement? We have a whole bunch of interactive
>control flow structures in Gforth, all with [] around their names.

Take a look how they are implemented (and what would be necessary for
portability) and then compare it to something like ASCII. Yes, it's
harder.

>The
>next approach at solving this problems was cmForth with different
>vocabularies for compilation and interpretation, and it again is too simple.

I think it's ok. It just does not combine well with the search order.
Why do you think it's too simple?

>Your approach was dual-Xts for these special compilation semantics, and it
>had some similar deficits.

It combines well with the search order. What "similar deficits" do
you have in mind?

Anton Ertl

unread,

Mar 3, 2013, 9:56:47 AM3/3/13

to

Bernd Paysan <bernd....@gmx.de> writes:
>Note that by how the standard is written, both ['] foo execute and postpone
>foo do what they should - one is providing the interpretation semantics, the
>other the compilation semantics. What' *not* according to the standard is
>['] foo compile, since that should add the execution semantics to the
>current definition (default compile, behavior).
>
>Now, we are mostly fine in so far as comp: is not a standard word, and
>therefore words modified by comp: don't have to behave like standard words.
>But what about standard words that have been defined using comp:?

Given that these words were intended to be implementable as
STATE-smart words, and such words don't always behave as specified
when ticked and then compile,d, one might weasel out of that, too.

However, what I am worried about is that your approach breaks the
currently existing relation between EXECUTE and COMPILE,. Currently I
know that, when I write some tool that processes xts in some way, I
can COMPILE, these xts, and when they run, they will do what they
would have done if I had EXECUTEd them right away.

Say, if I know that I want to EXECUTE a sequence of xts several times,
I can define a colon definition, COMPILE, the xts into it, and then
call the colon definition several times. With your approach, I could
no lonmger do that; you suggested that I should replace COMPILE, with
POSTPONE LITERAL POSTPONE EXECUTE, but that's neither nice nor fast.

Lars Brinkhoff

unread,

Mar 3, 2013, 10:27:59 AM3/3/13

to

unread,

Mar 3, 2013, 4:41:39 PM3/3/13

to

Hi Andrew,

Assuming cmForth is like colorForth in his respect, it easy to know which 'X' you are using - it depends on whether you are in "macro" or "forth" wordlist.

The "macro" wordlist is used to compile new defining words, which are then used to compile new words in the "forth" wordlist.

Just to add to the confusion by way of clarification, "macro" words are cyan, "immediate" are yellow and "compilish" words are green.

I get the feeling that "immediate" and "compilation" states in conventional Forth, and the use of the "macro" and "forth" wordlists don't map well to each other.

Also to add to the mix, colorForth programs tend to be compiled from EMPTY every time they are used - the definitions of "dump" for example is "42 load" .
This removes a lot of complexity about which version of a word you are using, because it will be the one on the screen you are loading ;-)

Best regards,
Howerd

Bernd Paysan

unread,

Mar 3, 2013, 4:42:31 PM3/3/13

to

Anton Ertl wrote:
> However, what I am worried about is that your approach breaks the
> currently existing relation between EXECUTE and COMPILE,. Currently I
> know that, when I write some tool that processes xts in some way, I
> can COMPILE, these xts, and when they run, they will do what they
> would have done if I had EXECUTEd them right away.

Not really. Consider you start with the state-smart world, and you compile
in compilation state into a word that might later run in interpretation
state. If you EXECUTEd them right away, they would show their compilation
semantics, if you COMPILE,d them, they would later show their interpretation
semantics. Or their compilation semantics, if the assumption above is
wrong.

> Say, if I know that I want to EXECUTE a sequence of xts several times,
> I can define a colon definition, COMPILE, the xts into it, and then
> call the colon definition several times. With your approach, I could
> no lonmger do that; you suggested that I should replace COMPILE, with
> POSTPONE LITERAL POSTPONE EXECUTE, but that's neither nice nor fast.

If you actually need that property. For me, this looks like an academic
exercise. I've built my fair share of hand-written compilers. They usually
use the fragment Mitch Bradley posted to clarify what COMPILE, was for, i.e.
that FIND 0> IF EXECUTE ELSE COMPILE, THEN, because they wanted the
compilation semantics of the words found, not the interpretation semantics.
This is the common case. The fact that we need to look at a result from
FIND, which is not passed on with the xt to decide whether to use EXECUTE or
COMPILE, shows that this solution will have problems (and the problems are
in the xt, which is why we can't find a good replacement for FIND).

The uncommon case, where you need this ['] foo execute style is where you
deliberately want the interpretation semantics. We don't actually have a
POSTPONE compagnion for doing that, so I doubt it is of much use. Rather
the contrary: We do have POSTPONE, because this is what we want, and we
don't want to distinguish between immediate and non-immediate words: We want
compilation semantics. POSTPONE is for named words, COMPILE, for xts, but
because the immediate flag is lost when going to the xt, COMPILE, can't do
its job properly. IMHO that part of COMPILE,'s functionality you want is
the one that is only there by accident - since STATE + IMMEDIATE are not a
good way to have compilation macros.

BTW: for the state-smart implementations, it won't work either way.
COMPILE, will just preserve the state-smart nature of the word, just as well
as ]] LITERAL EXEUCTE [[ will (no difference there).

A typical hand-written compiler is e.g. the scoping and super call in an OOP
package, which forces early binding, either to the particular class you
selected, or to the super class. You first look up the actual xt in the
vtable (do the binding), and then what do you want with the token? Compile
it. If it has special compilation semantics, you probably want it.

The only place in Gforth where the new semantics might case problems is in
DEFERS. DEFERS takes the action of a deferred word, and compiles it. Note
that a deferred word itself can only expose the execution semantics of the
word bound to it, so the definition

: defers ( "name" -- ) ' defer@ compile, ; immediate compile-only

is actually not correct. It should use that ]] literal execute [[ thing.
The same thing for the OOP early binding, because the vtable also can only
do EXECUTE, and therefore, COMPILE, would be wrong.

However, despite defers is used not so infrequently, this didn't cause any
problem.

The most interesting piece here is probably porting BerndOOF to this
approach, because in BerndOOF, you have methods with special compilation
semantics. But they aren't polymorph... though with the state-smart
approach, that would have worked.

Bernd Paysan

unread,

Mar 3, 2013, 4:49:14 PM3/3/13

to

Lars Brinkhoff wrote:

> Bernd Paysan wrote:
>> IMHO, state+immediate was an ad hoc solution [...], and it turned
>> out to be a bit too simple, and even Chuck realized that. The next
>> approach at solving this problems was cmForth with different
>> vocabularies for compilation and interpretation, and it again is too
>> simple.
>
> What is the problem with the cmForth approach?

Andrew has described one problem, the problem of redefinitions. We assume
that new definitions override old ones; in cmForth, this isn't guaranteed.

The other problem cmForth has is the common problem of all dual-xt Forths
(where words can have more than one xt): It depends on the state you are in
what FIND will return. This can also be nasty, as FIND turns into a state-
smart word, and now may return a different xt as '.

>> [...] The next approach is smart compile, and the only issue it has
>> are compatibility problems with the state+immediate approach
>
> How does smart compile (or compile,) work?

"smart compile," means that each word has a field somewhere in the header or
alike, which describes what the word should do when COMPILE, is used.

> (I'm trying to find the answers by searching previous comp.lang.forth
> discussions, but it's not easy.)

Yeah, it is hard to google for stuff with ','. We promise to invent more
words which can't be googled. You better look for "header" discussions.

Stephen Pelc

unread,

Mar 3, 2013, 5:02:43 PM3/3/13

to

On Sun, 03 Mar 2013 22:42:31 +0100, Bernd Paysan <bernd....@gmx.de>
wrote:

>The uncommon case, where you need this ['] foo execute style is where you
>deliberately want the interpretation semantics. We don't actually have a
>POSTPONE compagnion for doing that, so I doubt it is of much use.

VFX provides
[INTERP] <name
for this because we needed it once or twice.

Rod Pemberton

unread,

Mar 3, 2013, 6:08:47 PM3/3/13

to

"Anton Ertl" <an...@mips.complang.tuwien.ac.at> wrote in message
news:2013Mar...@mips.complang.tuwien.ac.at...

> The issue is that you write
>
> 1 2 +
>
> and can then wrap it in a colon def
>
> : foo 1 2 + ;
>
> and you write exactly the same sequence "1 2 +". You can also
> extract sequences from colon definitions and run them in the
> text interpreter.
>
> But that does not work for all code. One part where it breaks
> down is parsing words like "'". If you put
>
> ' + execute
>
> in a colon definition, you don't get the same behaviour. Some
> people, like Andrew Haley and Elizabeth Rather, just accept that
> and say that everybody should accept that. Other people try to
> keep this property even for parsing and control-flow words; one
> of the techniques used is STATE-smart words, but they lead to
> problems, because they do not always check STATE when the text

> interpreter sees them. [...]

What's wrong with a set of words that can select the needed
behavior prior to the word being used?

E.g., .I .C .P

.I CHAR \ immediate behavior
.C CHAR \ compile behavior
.P CHAR \ etc.

Of course, each mode would be the default for that mode. Who
wants to type them all the time?

Rod Pemberton

Rod Pemberton

unread,

Mar 3, 2013, 6:09:36 PM3/3/13

to

"Andrew Haley" <andr...@littlepinkcloud.invalid> wrote in message
news:z5-dnZku3IgfWqzM...@supernews.com...
> Rod Pemberton <do_no...@notemailnotz.cnm> wrote:

> > LITERAL is on my "To Do." list to have STATE removed. But,
> > ANS seems to require STATE for S" ." and C" . How does one
> > implement them without it? STATE also seems to required in
> > Forth interpreters for QUIT : ; [ ] and the inner/address
> > interpreter. How does a compiled Forth implement two states,
> > compile and interactive, without STATE?
>
> You have two separate loops, a compiler loop and an interpreter
> loop, rather than a single loop that checks STATE on every word.
> You don't need STATE at all, but you might choose to set it in
> case anybody wants to make a STATE-smart word.
>

1) First, my reply.

If the code is single threaded, then there is a mechanism to
selectively switch from one loop to the other, or there is a
mechanism that decides when to exit each loop and jump to the
other, etc. That mechanism is effectively "STATE".

If the code is executed in parallel, then each loop must have
methods to both start execution and halt execution which work
together, since both can't operate on the input stream at the same
time. That mechanism is effectively "STATE".

As long as there is a compiler loop and an interpreter loop,
I don't see how you can eliminate STATE or it's equivalent.

2) Second, I think you might be insterested.

Originally, my code was based on fig-Forth, but I structured the
inner interpreter based on your 2009 post. You later updated that
code, which I didn't use. I was going to restructure it to use
the exact fig-Forth method. It seems to have fewer state
changes... Well, I haven't done that yet or maybe never.
However, I've since re-written my code to stay in the
'interpreter' or 'compiler' until the STATE changes. That
increased the speed of my interpreter.

Originally, I had three words like in your post: 'interpret',
'interpreter', 'compiler', basically the same. Now, I have just
two. My code for these is internal and I haven't converted them
to high-level Forth yet. It uses 'branch' and 'zbranch' instead
of high-level control-flow. Converting absolute and conditional
branches to high-level IF-ELSE-THEN and BEGIN AGAIN loops isn't
easy since they're unstructured. However, I basically just
patched in "STATE @ IF" sections into 'interpreter' and 'compiler'
which call each other, ping-pong style... Both become loops,
i.e., stay in that mode until a STATE change occurs. That allowed
'interpret' to be removed ('inter-loop' in your second post).
QUIT now calls the 'interpreter' word directly. The visible Forth
word INTERPRET in the dictionary was redirected from 'interpret'
to 'interpreter'.

So, since my code isn't high-level and slightly different, I'll
attempt to correctly place the "STATE @ IF" sequences and looping
into your sample code from 2009. That way you can get an idea of
what I did.

(untested, constructed)
: QUIT ... interpreter ... ;

: interpreter
begin
find if execute state @ if compiler then else >number then
again
;

\ execute drops back to QUIT if the word is not found
\ in the dictionary by executing a special null word...

: compiler
begin
find if immediate? if execute else compile, then
else >number ,literal then state @ if 0= exit then
again
;

\ If I didn't do that correctly in high-level Forth, then the
\ "state @ if 0= exit then" is supposed to exit 'compiler'
\ if STATE is for interpret mode and return to 'interpreter'
\ where 'compiler' was called

Your posts:
https://groups.google.com/group/comp.lang.forth/msg/68915cdba3b07323?dmode=source
https://groups.google.com/group/comp.lang.forth/msg/dd2d8e7342aa17c6?dmode=source

HTH,

Rod Pemberton

Elizabeth D. Rather

unread,

Mar 3, 2013, 8:01:16 PM3/3/13

to

The concept of state is essential. A variable with that name isn't.

Doing it with two loops rather than a variable really quite simple: the
"compile" loop is launched by :, :NONAME, or ] and runs until exited by
; or [ . The "interpret" loop is the outer loop, which parsed and
executed : etc., so when the "compile" loop is finished, you drop back
into the "interpret" loop.

The only reason a variable STATE is required is to provide portability
for code written using a system that depended on it.

The concept of words that "must only be used inside a colon definition"
isn't difficult. All languages have similar concepts, and in my
experience Forth newbies accept it with no confusion. Understanding the
correct use of words with two behaviors depending in compilation state
is a lot more difficult, which is why it's nice to avoid them.

Implementation note: if ] is really the word that runs the compile loop,
then : and :NONAME can simply call it when they're ready to start compiling.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

Mark Wills

unread,

Mar 4, 2013, 3:19:52 AM3/4/13

to

Can we not think of the problem 'the other way around' for a minute?
Maybe it would be useful.

We're pointing the finger at "state-smart" (using the conventional
meaning ;-) words because we cannot differentiate interpretation
semantics from compilation sematics via tick.

So, rotating the problem by 180 degrees, the problem is tick, not the
state-smart words themselves.

Therefore, simply standardise a new word, something like 'I (which
means 'tick the *interpretation* behaviour/address/cfa/whatever' of
the word.

Of course, this has implications on the underlying architecture, and
raises further questions. Now the dictionary (presumably) must be able
to store pointers to both the compilation (for regular ') and
interpretation semantics (for 'I ). Furthermore, what happens if the
word has no interpretation sematics? Do you return the compilation
sematics as a default, or return a 0 to expressly indicate that there
are no interpretation semantics? All open for argument.

Discuss.

Andrew Haley

unread,

Mar 4, 2013, 4:06:23 AM3/4/13

to

Rod Pemberton <do_no...@notemailnotz.cnm> wrote:
> "Andrew Haley" <andr...@littlepinkcloud.invalid> wrote in message
> news:z5-dnZku3IgfWqzM...@supernews.com...
>> Rod Pemberton <do_no...@notemailnotz.cnm> wrote:
>
>> > LITERAL is on my "To Do." list to have STATE removed. But,
>> > ANS seems to require STATE for S" ." and C" . How does one
>> > implement them without it? STATE also seems to required in
>> > Forth interpreters for QUIT : ; [ ] and the inner/address
>> > interpreter. How does a compiled Forth implement two states,
>> > compile and interactive, without STATE?
>>
>> You have two separate loops, a compiler loop and an interpreter
>> loop, rather than a single loop that checks STATE on every word.
>> You don't need STATE at all, but you might choose to set it in
>> case anybody wants to make a STATE-smart word.
>
> 1) First, my reply.
>
> If the code is single threaded, then there is a mechanism to
> selectively switch from one loop to the other, or there is a
> mechanism that decides when to exit each loop and jump to the
> other, etc. That mechanism is effectively "STATE".

No, it's just a call: the compiler loop is called by the interpreter
loop. The compiler loop is just a word.

> If the code is executed in parallel, then each loop must have
> methods to both start execution and halt execution which work
> together, since both can't operate on the input stream at the same
> time. That mechanism is effectively "STATE".
>
> As long as there is a compiler loop and an interpreter loop,
> I don't see how you can eliminate STATE or it's equivalent.

It's really very simple: there is not STATE, no trickery. ] is a word
that enters the compilation loop, and [ or ; exits it.

> 2) Second, I think you might be insterested.
>
> Originally, my code was based on fig-Forth, but I structured the
> inner interpreter based on your 2009 post.

I see. Sure, that post more or less described the STATEful way that
Standard Forth does it.

Andrew.

Lars Brinkhoff

unread,

Mar 4, 2013, 5:38:59 AM3/4/13

to

"Rod Pemberton" <do_no...@notemailnotz.cnm> writes:
> (untested, constructed)
> : QUIT ... interpreter ... ;
>
> : interpreter
> begin
> find if execute state @ if compiler then else >number then
> again
> ;
>
> \ execute drops back to QUIT if the word is not found
> \ in the dictionary by executing a special null word...
>
> : compiler
> begin
> find if immediate? if execute else compile, then
> else >number ,literal then state @ if 0= exit then
> again
> ;

My suggestion for a simplistic STATE-less interpreter would be
(also untested, but based on working code):

: finders: create ' , ' , ' , does> swap 1+ cells + @ execute ;
: literal, number postpone literal ; ( number from fig-Forth )

finders: execute-xt execute number execute
finders: compile-xt compile, literal, execute
defer interpret-xt

: interpret
begin
refill
while
begin source? while bl word find interpret-xt repeat
repeat ;

: quit ( reset return stack )
( reset input source )
execute-xt is interpret-xt
begin interpret ." ok" cr again ;

: [ ( 0 state ! ) quit ; immediate

: ] ( 1 state ! )
compile-xt is interpret-xt
begin interpret again ;

Anton Ertl

unread,

Mar 4, 2013, 12:52:14 PM3/4/13

to

Lars Brinkhoff <lars...@nocrew.org> writes:
>How does smart compile (or compile,) work?

I called the original idea "intelligent compile,", and that idea was
to implement optimization knowledge through "COMPILE,". E.g., consider the following code

5 constant foo
: bar foo ;

In a classic threaded-code system the compiler just takes the xt of
FOO and ,s it into BAR. Now with more sophisticated code generation,
say you want the system to generate code equivalent to

: bar [ 5 ] literal ;

(and what gets generated may be some native code, but the following
discussion is equally applicable to threaded code.)

Now some people have done horrible things to achieve that, like making
all constants STATE-smart (which is non-standard and breaks standard
code). The idea of the "intelligent compile," is to access the
optimization smarts through COMPILE, instead of at some other level.
This is conceptually cleaner, because all compilation goes through
COMPILE, but not necessarily through other parts of the Forth system
where one might consider putting this stuff (e.g., not all compilation
goes through the text interpreter in compilation state).

There can be different implementations of this concept. One not very
elegant one is implemented in Gforth 0.7.0: COMPILE, is a big case
that checks the word type of the xt and generates code accordingly
(e.g., it generates a literal for a constant).

It's more elegant if each word has a field that contains the xt of the
COMPILE, action; each constant would have the xt of CONST-COMPILE, in
that field:

: const-compile, ( xt -- )
execute \ get the value of the constant
POSTPONE literal ; \ and compile it

Up to now the intelligent COMPILE, is just an implementation technique
for "COMPILE,". I.e., the system implementor is responsible for
implementing COMPILE, correctly, i.e., in a way that (apart from
performance) is equivalent to

: compile, ( xt -- )
postpone literal postpone execute ;

Onwards to Bernd Paysan's "smart COMPILE,":

With the smart COMPILE, the equivalence above no longer holds true for
all words. With that, if you COMPILE, the xt of a word, what happens
is that the compilation semantics of the word are performed.

The difference between the intelligent and the smart COMPILE, is in
the xt that the system designer has put in the comp field: with the
"intelligent COMPILE," it compiles the interpretation/execution
semantics represented by the xt. With the "smart COMPILE," it
performs the compilation semantics. For many words, there is no
difference, but for words like S" and TO, there is.

Coos Haak

unread,

Mar 4, 2013, 2:55:31 PM3/4/13

to

Op Sun, 03 Mar 2013 15:13:18 -0600 schreef Andrew Haley:

> Rob Sciuk <r...@controlq.com> wrote:
>> I guess I would prefer to avoid the use of [ and ] to control STATE,
>> or worse, having two words x and [x], which presupposes subtle
>> knowledge of the behaviour of the word at execute and compile time,
>> thus cluttering up vocabulary with the implemenation un-niceties of
>> differentiating between STATEs.
>
> It's not really to do with implementation: they really are different.
>
>> Forth puts some serious demands upon its adherents
>
> Eh? Adherents? It's a programming language, not a religion!
>
>> to understand a great deal about the language. Arguably, better
>> programmers will by definition understand such arcanery, but
>> elegance shouldn't demand it. Things like [, ], postpone, compile
>> and other adverbs are taxing beyond the point of where a pleasant
>> script language should require of the user.
>
> Perhaps, but standard Forth doesn't have COMPILE , [COMPILE], and so
> on. These are obsolete.

Since when is [COMPILE] obsolete? I nearly never use it, most instances can
be dealt with POSTPONE. COMPILE is surely obsolete.

>
> There is a small cost in learning in order to make the full language
> better and more powerful and maintainable for reasonablty skilled
> programmers. Evey programming language has to make that choice
> somewhere.

Yes, Forth is not hard to learn.

--
Coos

CHForth, 16 bit DOS applications
http://home.hccnet.nl/j.j.haak/forth.html

Andrew Haley

unread,

Mar 4, 2013, 4:17:01 PM3/4/13

to

Coos Haak <chf...@hccnet.nl> wrote:
> Op Sun, 03 Mar 2013 15:13:18 -0600 schreef Andrew Haley:
>
>> Rob Sciuk <r...@controlq.com> wrote:
>>> to understand a great deal about the language. Arguably, better
>>> programmers will by definition understand such arcanery, but
>>> elegance shouldn't demand it. Things like [, ], postpone, compile
>>> and other adverbs are taxing beyond the point of where a pleasant
>>> script language should require of the user.
>>
>> Perhaps, but standard Forth doesn't have COMPILE , [COMPILE], and so
>> on. These are obsolete.
>
> Since when is [COMPILE] obsolete?

Since POSTPONE.

Andrew.

Bernd Paysan

unread,

Mar 4, 2013, 6:14:31 PM3/4/13

to

Mark Wills wrote:
> Can we not think of the problem 'the other way around' for a minute?
> Maybe it would be useful.
>
> We're pointing the finger at "state-smart" (using the conventional
> meaning ;-) words because we cannot differentiate interpretation
> semantics from compilation sematics via tick.

Actually, the problem goes a bit deeper, it is the xt returned by ' and
FIND. This xt is the problem, it does not allow straight-forward access to
its interpretation and compilation semantics. Especially since the
immediate flag is lost, the xt does not know if it is immediate or not.

Smart COMPILE, fixes that. But then, it somewhat changes the meaning of
COMPILE,.

> Therefore, simply standardise a new word, something like 'I (which
> means 'tick the *interpretation* behaviour/address/cfa/whatever' of
> the word.

Or COMP' for ticking the compilation behavior, as ' always did return the xt
for the interpretation semantics.

> Of course, this has implications on the underlying architecture, and
> raises further questions. Now the dictionary (presumably) must be able
> to store pointers to both the compilation (for regular ') and
> interpretation semantics (for 'I ). Furthermore, what happens if the
> word has no interpretation sematics? Do you return the compilation
> sematics as a default, or return a 0 to expressly indicate that there
> are no interpretation semantics? All open for argument.

The dual-token approach with ' for interpretation semantics and COMP' for
compilation semantics has already been tried out, it's Anton's approach at
solving the problem; it is in Gforth since quite some time.

It wasn't a full solution, since it required adding another type, the "name
token", which still had all relevant informations, and converting that to an
xt, these informations were lost.

A. K.

unread,

Mar 4, 2013, 6:18:36 PM3/4/13

to

On 03.03.2013 22:49, Bernd Paysan wrote:
> Lars Brinkhoff wrote:
>
>> Bernd Paysan wrote:
>>> IMHO, state+immediate was an ad hoc solution [...], and it turned
>>> out to be a bit too simple, and even Chuck realized that. The next
>>> approach at solving this problems was cmForth with different
>>> vocabularies for compilation and interpretation, and it again is too
>>> simple.
>>
>> What is the problem with the cmForth approach?
>
> Andrew has described one problem, the problem of redefinitions. We assume
> that new definitions override old ones; in cmForth, this isn't guaranteed.
>
> The other problem cmForth has is the common problem of all dual-xt Forths
> (where words can have more than one xt): It depends on the state you are in
> what FIND will return. This can also be nasty, as FIND turns into a state-
> smart word, and now may return a different xt as '.

Not necessarily. In our dual-xt system FIND is not affected by STATE,
only the INTERPRET loop is.

And INTERPRET has become just slightly more complicated as it now uses a
cousin to FIND that returns both xts.

Coos Haak

unread,

Mar 4, 2013, 6:27:22 PM3/4/13

to

Op Mon, 04 Mar 2013 15:17:01 -0600 schreef Andrew Haley:

In www.forth200x.org/documents/forth-rc1.pdf (2012) is no mention of
obsolescense of [COMPILE]

Bernd Paysan

unread,

Mar 4, 2013, 6:49:15 PM3/4/13

to

Anton Ertl wrote:
> The difference between the intelligent and the smart COMPILE, is in
> the xt that the system designer has put in the comp field: with the
> "intelligent COMPILE," it compiles the interpretation/execution
> semantics represented by the xt. With the "smart COMPILE," it
> performs the compilation semantics. For many words, there is no
> difference, but for words like S" and TO, there is.

Yes, and the argument I have is that words like S" and TO haven't been
considered when writing the glossary for COMPILE,. There is a reason why
most of these words "can't be ticked" in ANS Forth (and those who can are
just an oversight).

The idea to make COMPILE, smart is not my idea, it's Stephen Pelc's idea -
MPE at one point in time didn't have a single immediate word in VFX Forth,
because they liked comp: so much. But actually, some ANS Forth words are
deliberately specified as "this is an immediate word", so you better set the
immediate flag on them when you implement a Forth system.

And for immediate words, the comp field is pointing to the default
compilation method, so COMPILE, does just what Anton wants it to do.

If we really need Anton's thing dearly, we should have INTERP, and [INTERP]
<name> (maybe better without those brackets, we don't have them on POSTPONE)
as compagnion for COMPILE, and POSTPONE (POSTPONE should have been called
COMPILE - as COMPILE did only work for non-immediate words, and extending it
for immediate words wouldn't have required a name-change).

Andrew Haley

unread,

Mar 4, 2013, 7:41:50 PM3/4/13

to

Coos Haak <chf...@hccnet.nl> wrote:
> Op Mon, 04 Mar 2013 15:17:01 -0600 schreef Andrew Haley:
>
>> Coos Haak <chf...@hccnet.nl> wrote:
>>> Op Sun, 03 Mar 2013 15:13:18 -0600 schreef Andrew Haley:
>>>
>>>> Rob Sciuk <r...@controlq.com> wrote:
>>>>> to understand a great deal about the language. Arguably, better
>>>>> programmers will by definition understand such arcanery, but
>>>>> elegance shouldn't demand it. Things like [, ], postpone, compile
>>>>> and other adverbs are taxing beyond the point of where a pleasant
>>>>> script language should require of the user.
>>>>
>>>> Perhaps, but standard Forth doesn't have COMPILE , [COMPILE], and so
>>>> on. These are obsolete.
>>>
>>> Since when is [COMPILE] obsolete?
>>
>> Since POSTPONE.
>

> In www.forth200x.org/documents/forth-rc1.pdf (2012) is no mention of
> obsolescense of [COMPILE]

It's in the rationale, A.6.1.2033 POSTPONE.

Andrew.

Andrew Haley

unread,

Mar 4, 2013, 7:44:12 PM3/4/13

to

A. K. <a...@nospam.org> wrote:
> On 03.03.2013 22:49, Bernd Paysan wrote:
>> The other problem cmForth has is the common problem of all dual-xt Forths
>> (where words can have more than one xt): It depends on the state you are in
>> what FIND will return. This can also be nasty, as FIND turns into a state-
>> smart word, and now may return a different xt as '.
>
> Not necessarily. In our dual-xt system FIND is not affected by STATE,
> only the INTERPRET loop is.

Right, but that's just as bad if not worse: the interpreter now has a
different lookup behaviour from a simple FIND . Either way it's
nasty.

Andrew.

Rod Pemberton

unread,

Mar 4, 2013, 8:49:12 PM3/4/13

to

"Andrew Haley" <andr...@littlepinkcloud.invalid> wrote in message

news:oridnYhak5PToqjM...@supernews.com...

If it's obsoleted, why is it still a part of CORE EXT (6.2.2530)?

Shouldn't it be a part of CORE "OBS" ... ?

If it's _truly_ obsoleted, why does A.6.1.2033 say POSTPONE only
replaces MOST of the functionality of COMPILE and [COMPILE]?

Wouldn't POSTPONE need to replace ALL the functionality to
obsolete [COMPILE]?

A.6.1.2033 only states "... [COMPILE] has been moved in favor of
POSTPONE". That doesn't say it's obsolete. Why the word "moved"
is misused is a mystery, unless it originally said "removed".

ANS 94 had section D.6.7 on "Immediacy" which demonstrates *three*
situations where COMPILE and [COMPILE] can't be replaced
completely by use of POSTPONE. Two are in the body and one is at
the end. However, it only demonstrates a solution for only one of
the situations. I.e., there seems to still be two situations
which need [COMPILE].

Rod Pemberton

Michael L Gassanenko

unread,

Mar 5, 2013, 2:08:12 AM3/5/13

to r...@controlq.com

On Thursday, February 28, 2013 8:35:32 PM UTC+4, Rob Sciuk wrote:
> How might one describe a standardized language similar to Forth in such a
> way as to avoid the run-time/compile-time semantics, but still allow
> create/does type extensibility? Am I missing something obvious?

You can get rid of the variable STATE , but you will introduce something
that 1) will not be less complex 2) will be incompatible with the language
the other people use. (I mean, unless you invent something completely new,
in which case you probably will have only '2').

Multiple approaches have been made, one of them was getting rid of the interpretation mode. Finally, they had two compilation modes, compilation to a definition and compilation to a temporary buffer, which is also a state. (But a variation of this approach was very good for umbilical targets: compile-send-execute and no need for an interpreter.)

There have been attempts to eliminate STATE by using two text interpreters and/or two search orders (one for compilation and one for interpretation) but having the current interpreter or the current search order is still a state.

Hmmm... in fact, STATE is not needed at all:

: _: : postpone [ ;
: _; ] postpone ; ;
: _ ] ' compile, postpone [ ;
: n, ] postpone literal postpone [ ;

\ the following definition is created in *interpretation* state
_: test 2 n, 3 n, _ + _ . _; ok
.s <0> ok
test 5 ok
.s <0> ok

( The only disadvantage of this approach is that the code is not human-readable :)

Andrew Haley

unread,

Mar 5, 2013, 3:59:47 AM3/5/13

to

Rod Pemberton <do_no...@notemailnotz.cnm> wrote:
> "Andrew Haley" <andr...@littlepinkcloud.invalid> wrote in message
> news:oridnYhak5PToqjM...@supernews.com...
>> Coos Haak <chf...@hccnet.nl> wrote:
>> > Op Mon, 04 Mar 2013 15:17:01 -0600 schreef Andrew Haley:
>> >> Coos Haak <chf...@hccnet.nl> wrote:
>> >>> Op Sun, 03 Mar 2013 15:13:18 -0600 schreef Andrew Haley:
>> >>>> Rob Sciuk <r...@controlq.com> wrote:
>
>> >>>>> to understand a great deal about the language. Arguably,
>> >>>>> better programmers will by definition understand such
>> >>>>> arcanery, but elegance shouldn't demand it. Things like
>> >>>>> [, ], postpone, compile and other adverbs are taxing
>> >>>>> beyond the point of where a pleasant script language
>> >>>>> should require of the user.
>> >>>>
>> >>>> Perhaps, but standard Forth doesn't have COMPILE ,
>> >>>> [COMPILE], and so on. These are obsolete.
>> >>>
>> >>> Since when is [COMPILE] obsolete?
>> >>
>> >> Since POSTPONE.
>> >
>> > In www.forth200x.org/documents/forth-rc1.pdf (2012) is no
>> > mention of obsolescense of [COMPILE]
>>
>> It's in the rationale, A.6.1.2033 POSTPONE.
>
> If it's obsoleted, why is it still a part of CORE EXT (6.2.2530)?
>
> Shouldn't it be a part of CORE "OBS" ... ?

Probably.

> If it's _truly_ obsoleted, why does A.6.1.2033 say POSTPONE only
> replaces MOST of the functionality of COMPILE and [COMPILE]?
>
> Wouldn't POSTPONE need to replace ALL the functionality to
> obsolete [COMPILE]?

No. That's not what obsolete means. Programming practices change,
and in applications it makes far more sense to use POSTPONE, even if
that requires small changes. POSTPONE isn't a direct replacement but
a functional one.

> A.6.1.2033 only states "... [COMPILE] has been moved in favor of
> POSTPONE". That doesn't say it's obsolete. Why the word "moved"
> is misused is a mystery, unless it originally said "removed".
>
> ANS 94 had section D.6.7 on "Immediacy" which demonstrates *three*
> situations where COMPILE and [COMPILE] can't be replaced
> completely by use of POSTPONE. Two are in the body and one is at
> the end. However, it only demonstrates a solution for only one of
> the situations. I.e., there seems to still be two situations
> which need [COMPILE].

It's all explained pretty thoroughly there: the use of [COMPILE] with
non-immediate words is never needed, and COMPILE [COMPILE] isn't
really necessary, as also explained there.

[COMPILE] isn't necessary for anything, and can't be used portably in
most cases anyway. It's obsolete.

Andrew.

Stephen Pelc

unread,

Mar 5, 2013, 5:55:19 AM3/5/13

to

On Tue, 05 Mar 2013 00:49:15 +0100, Bernd Paysan <bernd....@gmx.de>
wrote:

>The idea to make COMPILE, smart is not my idea, it's Stephen Pelc's idea -

I don't think that it's my idea at all. The original VFX kernel was
not written by me, and the separate compilation xt and use of it by
COMPILE, was in the 1998 version of VFX. The VFX kernel was a group
effort and took inspiration from a wide range of sources.

>MPE at one point in time didn't have a single immediate word in VFX Forth,
>because they liked comp: so much. But actually, some ANS Forth words are
>deliberately specified as "this is an immediate word", so you better set the
>immediate flag on them when you implement a Forth system.

True. We made some words IMMEDIATE in order to be standards compliant.
IMHO, the standard's relationship with IMMEDIATE is just a set of
weasel words as a historical legacy. All it actually means is that
the interpretation and compilation actions are the same.

In modern Forth compilers, especially those with optimisers, many
words have different interpretation and compilation actions.

>If we really need Anton's thing dearly, we should have INTERP, and [INTERP]
><name> (maybe better without those brackets, we don't have them on POSTPONE)

[INTERP] is just the parallel to [COMPILE].

Alex McDonald

unread,

Mar 5, 2013, 9:58:40 AM3/5/13

to

On Mar 5, 8:59 am, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:
> Rod Pemberton <do_not_h...@notemailnotz.cnm> wrote:
> > "Andrew Haley" <andre...@littlepinkcloud.invalid> wrote in message
> >news:oridnYhak5PToqjM...@supernews.com...

> >> Coos Haak <chfo...@hccnet.nl> wrote:
> >> > Op Mon, 04 Mar 2013 15:17:01 -0600 schreef Andrew Haley:

> >> >> Coos Haak <chfo...@hccnet.nl> wrote:
> >> >>> Op Sun, 03 Mar 2013 15:13:18 -0600 schreef Andrew Haley:
> >> >>>> Rob Sciuk <r...@controlq.com> wrote:
>
> >> >>>>> to understand a great deal about the language. Arguably,
> >> >>>>> better programmers will by definition understand such
> >> >>>>> arcanery, but elegance shouldn't demand it. Things like
> >> >>>>> [, ], postpone, compile and other adverbs are taxing
> >> >>>>> beyond the point of where a pleasant script language
> >> >>>>> should require of the user.
>
> >> >>>> Perhaps, but standard Forth doesn't have COMPILE ,
> >> >>>> [COMPILE], and so on. These are obsolete.
>
> >> >>> Since when is [COMPILE] obsolete?
>
> >> >> Since POSTPONE.
>

> >> > Inwww.forth200x.org/documents/forth-rc1.pdf(2012) is no

My compiler doesn't have it; I've not noticed the loss. With smart (or
is it intelligent? I find the difference hard to grasp) COMPILE,
POSTPONE can deal with parsing words such as TO S" and so on. It's
worth it for that reason alone; there are no exceptions to POSTPONE
and just the one word.

(With possibly the issue of POSTPONE EXIT. Anton discusses in "State-
smartness; Why it is Evil and How to Exorcise it" IIRC.)

Rob Sciuk

unread,

Mar 5, 2013, 10:47:18 AM3/5/13

to

On Mon, 4 Mar 2013, Michael L Gassanenko wrote:

> Date: Mon, 4 Mar 2013 23:08:12 -0800 (PST)
> From: Michael L Gassanenko <m_l...@yahoo.com>
> To: comp.la...@googlegroups.com
> Cc: r...@controlq.com
> Newsgroups: comp.lang.forth
> Subject: Re: State and the standard ...

>
> On Thursday, February 28, 2013 8:35:32 PM UTC+4, Rob Sciuk wrote:
>> How might one describe a standardized language similar to Forth in such a
>> way as to avoid the run-time/compile-time semantics, but still allow
>> create/does type extensibility? Am I missing something obvious?
>
> You can get rid of the variable STATE , but you will introduce something
> that 1) will not be less complex 2) will be incompatible with the language
> the other people use. (I mean, unless you invent something completely new,
> in which case you probably will have only '2').

State, as indicated by a state variable is the simplest approach to the
problem, and in that regard, was the first solution hit upon. Not
necessarily a bad one, but one which introduced the necessity of
recognizing state, and manipulating it, which requires an understanding of
multiple (compile/run time) semantics.

>
> Multiple approaches have been made, one of them was getting rid of the
> interpretation mode. Finally, they had two compilation modes,
> compilation to a definition and compilation to a temporary buffer, which
> is also a state. (But a variation of this approach was very good for
> umbilical targets: compile-send-execute and no need for an interpreter.)
>
> There have been attempts to eliminate STATE by using two text
> interpreters and/or two search orders (one for compilation and one for
> interpretation) but having the current interpreter or the current search
> order is still a state.

Interpretation is a boon, IMHO, and ridding Forth proper of interpretation
is retrograde, notwithstanding that X-Development (umbilical) systems
don't necessarily require it.

> Hmmm... in fact, STATE is not needed at all:
>
> : _: : postpone [ ;
> : _; ] postpone ; ;
> : _ ] ' compile, postpone [ ;
> : n, ] postpone literal postpone [ ;
>
> \ the following definition is created in *interpretation* state
> _: test 2 n, 3 n, _ + _ . _; ok
> .s <0> ok
> test 5 ok
> .s <0> ok
>
> ( The only disadvantage of this approach is that the code is not human-readable :)

My question was intended to open a discussion (which seems to have worked)
to question some long held truths, and in hopes of improving Forth rather
than to damage it. I believe that the approach above is exactly the kind
of thing I'd hoped to avoid -- confuscation is never good.

Just as Esparanto was created as a "normalized" human language, with no
"irregular" verb conjugations (as compared to English, or any of the Latin
based languages), I was wondering if Forth could be made entirely
consistent, without "classes" of defining words.

I don't think this thread has caused any harm, but I have yet to see a
conclusive way that Forth might be improved (at least not without some
inelegant tradeoffs). Perhaps we just leave it alone, and accept these
minor "blemishes", or come to understand that the exceptions simply add to
the CHARM of the language 8-).

Cheers,
Rob.

Elizabeth D. Rather

unread,

Mar 5, 2013, 1:11:56 PM3/5/13

to

On 3/5/13 5:47 AM, Rob Sciuk wrote:
...

> Just as Esparanto was created as a "normalized" human language, with no
> "irregular" verb conjugations (as compared to English, or any of the
> Latin based languages), I was wondering if Forth could be made entirely
> consistent, without "classes" of defining words.

And we can all see how popular it's been, as a result, yes? Right up
there with Klingon and Elvish.

> I don't think this thread has caused any harm, but I have yet to see a
> conclusive way that Forth might be improved (at least not without some
> inelegant tradeoffs). Perhaps we just leave it alone, and accept these
> minor "blemishes", or come to understand that the exceptions simply add
> to the CHARM of the language 8-).

A lot of very challenging projects have been successfully completed with
this language, blemishes and all. There are actually reasons for most of
its "quirks". Beware lest the cure be worse than the disease.

Rob Sciuk

unread,

Mar 5, 2013, 1:44:41 PM3/5/13

to

On Tue, 5 Mar 2013, Elizabeth D. Rather wrote:

> Date: Tue, 05 Mar 2013 08:11:56 -1000
> From: Elizabeth D. Rather <era...@forth.com>

> Newsgroups: comp.lang.forth
> Subject: Re: State and the standard ...
>

> On 3/5/13 5:47 AM, Rob Sciuk wrote:
> ...
>> Just as Esparanto was created as a "normalized" human language, with no
>> "irregular" verb conjugations (as compared to English, or any of the
>> Latin based languages), I was wondering if Forth could be made entirely
>> consistent, without "classes" of defining words.
>
> And we can all see how popular it's been, as a result, yes? Right up there
> with Klingon and Elvish.

Touche' 8-)

Still, an orthagonal vocabulary might be considered a useful trait,
helpful mostly in the uptake (removing adoption impediments for newbies)
rather than for the black-belt Forth-Fu artists (such as yourself).

>
>> I don't think this thread has caused any harm, but I have yet to see a
>> conclusive way that Forth might be improved (at least not without some
>> inelegant tradeoffs). Perhaps we just leave it alone, and accept these
>> minor "blemishes", or come to understand that the exceptions simply add
>> to the CHARM of the language 8-).
>
> A lot of very challenging projects have been successfully completed with this
> language, blemishes and all. There are actually reasons for most of its
> "quirks". Beware lest the cure be worse than the disease.
>

My intent was not to bash Forth, but rather to ask a question, to which
the answers were very informative, and yet unsatisfactory. State and
immediacy appear to be fundamental, and apparently anything else would not
be Forth.

I'm ok with that, and again, I think the resulting introspection was
healthy ... if only to shake loose some historical approaches, and
proposals which as you say, are worse than the disease. I understand your
defense of the status quo, but I hope that such discussions are not
unwelcome.

Cheers,
Rob.

Elizabeth D. Rather

unread,

Mar 5, 2013, 2:26:16 PM3/5/13

to

On 3/5/13 8:44 AM, Rob Sciuk wrote:
> On Tue, 5 Mar 2013, Elizabeth D. Rather wrote:

...

>> A lot of very challenging projects have been successfully completed
>> with this language, blemishes and all. There are actually reasons for
>> most of its "quirks". Beware lest the cure be worse than the disease.
>>
>
> My intent was not to bash Forth, but rather to ask a question, to which
> the answers were very informative, and yet unsatisfactory. State and
> immediacy appear to be fundamental, and apparently anything else would
> not be Forth.
>
> I'm ok with that, and again, I think the resulting introspection was
> healthy ... if only to shake loose some historical approaches, and
> proposals which as you say, are worse than the disease. I understand
> your defense of the status quo, but I hope that such discussions are not
> unwelcome.

I'm not blindly defending the status quo... there have been many
improvements to Forth over the years, mostly along the lines of added
capabilities, improvements in performance, etc. I'm just highly
suspicious of "improvements" intended to make Forth more like other
languages simply because people are resistant to accepting Forth's
inherent style, in which lies much of its strength. I also resist adding
features whose utility in Forth hasn't been demonstrated. If someone can
show why a feature is valuable beyond "Language X has it, and it's
really cool!" I'm all for it.

This particular discussion about STATE and interpreter loops is annoying
to me because making state-smart (note lower case) words is really
rarely something of value to users -- it's something that people want to
do because their "expectations" of how things work are incorrect. Rod
asked (legitimately, IMO) how often DOES> is actually used, and the
answer is that it's incredibly valuable, particularly in application
code (which is difficult to appreciate if you're only interested in
writing yet another Forth kernel). But I would argue that writing
state-smart words is *rarely* useful in application code, and therefor
not something worth fussing about.

I share Stephen's view, that Forth is not about writing kernels, it's
about writing applications to get work done. Forth is to *use*. I have
no problem with people writing their own kernels for fun, but resist
changing the language to facilitate kernel writing instead of
emphasizing useful application capabilities.

Andrew Haley

unread,

Mar 5, 2013, 7:18:55 PM3/5/13

to

Rob Sciuk <r...@controlq.com> wrote:
> On Mon, 4 Mar 2013, Michael L Gassanenko wrote:
>
>> From: Michael L Gassanenko <m_l...@yahoo.com>
>>

>> On Thursday, February 28, 2013 8:35:32 PM UTC+4, Rob Sciuk wrote:
>>> How might one describe a standardized language similar to Forth in such a
>>> way as to avoid the run-time/compile-time semantics, but still allow
>>> create/does type extensibility? Am I missing something obvious?
>>
>> You can get rid of the variable STATE , but you will introduce something
>> that 1) will not be less complex 2) will be incompatible with the language
>> the other people use. (I mean, unless you invent something completely new,
>> in which case you probably will have only '2').
>
> State, as indicated by a state variable is the simplest approach to the
> problem, and in that regard, was the first solution hit upon.

It's not the simplest by any means. STATEless Forth is cleaner and
simpler, but it's not a big deal.

> Interpretation is a boon, IMHO, and ridding Forth proper of
> interpretation is retrograde,

Indeed.

Andrew.

unread,

Mar 6, 2013, 6:12:24 PM3/6/13

to

On 3/6/13 12:51 PM, Rod Pemberton wrote:
> "Elizabeth D. Rather" <era...@forth.com> wrote in message

...

> Forth apparently is "a way", "a path", "a faith", "a religion", a
> "set of beliefs accepted as truth without basis". Right? ...

Forth is a programming language that has enabled significant projects to
be done significantly faster than estimates by C experts; we have the
estimates and the actual billings to prove it. Many people have found it
useful in their careers. Nobody is forcing you to use it.

Cheers,
elizabeth

Elizabeth D. Rather

unread,

Mar 6, 2013, 6:13:43 PM3/6/13

to

On 3/6/13 12:51 PM, Rod Pemberton wrote:

Correct. Forth is naturally interactive; that's why most of us like it.

Rob Sciuk

unread,

Mar 6, 2013, 6:17:38 PM3/6/13

to

On Wed, 6 Mar 2013, Rod Pemberton wrote:

> Date: Wed, 6 Mar 2013 17:49:39 -0500
> From: Rod Pemberton <do_no...@notemailnotq.cpm>

> Newsgroups: comp.lang.forth
> Subject: Re: State and the standard ...
>

On further reflection, I declare this to be a new word. Henceforth, the
following will have meaning:

CONFUSCATION: A political act.
Verb: to CONFUSCATE

You may use it without royalties.

Rob.

unread,

Mar 7, 2013, 1:16:56 PM3/7/13

to

On 3/7/13 2:24 AM, Michael L Gassanenko wrote:
> On Tuesday, March 5, 2013 11:26:16 PM UTC+4, Elizabeth D. Rather wrote:
>> On 3/5/13 8:44 AM, Rob Sciuk wrote:
>> I'm not blindly defending the status quo... there have been many
>> improvements to Forth over the years, mostly along the lines of added
>> capabilities, improvements in performance, etc. I'm just highly
>> suspicious of "improvements" intended to make Forth more like other
>> languages simply because people are resistant to accepting Forth's
>> inherent style, in which lies much of its strength.
>
> It should be noted that many (if not most if not all) books about
> object-oriented design state in the introduction that object-oriented
> decomposition is better than functional decomposition.
>
> OO design answers the question:
> what class is responsible for what functionality?

As Doug points out, OOP is readily implemented in Forth, and a number of
Forths have extensive OOP support, although an implementation consensus
has not emerged.

However, althugh OOP is applicable to many application areas, it is not
a panacea for all programming. Many projects are inherently procedural
(especially in embedded systems), and forcing an OOP paradigm just adds
a layer of complexity and obfuscation.

> And the problems for the programmer look like:
> "We want you to reuse this (or this) and implement this".
>
> There is not much room for Forth programming in this world.
>
> Probably, because Forth is not very good at reusing libraries and concepts created for other languages.

Forth is good at reusing libraries, but "concepts created for other
languages" are usually naturally designed for those languages and
inappropriate for Forth, while concepts in Forth may be equally
inappropriate for other languages.

The library issue in Forth has two aspects: first, the most extensive
Forth programming is done in commercial projects where the organization
paying for the project owns the code, so it is not sharable.
Nonetheless, companies doing such work, such as FORTH, Inc., usually
offer generic versions of capabilities developed for such projects in
libraries with their systems.

Second, many academic and free-lance Forth programmers devote their
energies to developing kernels, rather than writing sharable application
code.

The current Forth standards make it possible to share and reuse code.
It's the choice of the programmers whether they want to make use of this
capability.

Anton Ertl

unread,

Mar 7, 2013, 1:04:15 PM3/7/13

to

Bernd Paysan <bernd....@gmx.de> writes:
>Anton Ertl wrote:
>> However, what I am worried about is that your approach breaks the
>> currently existing relation between EXECUTE and COMPILE,. Currently I
>> know that, when I write some tool that processes xts in some way, I
>> can COMPILE, these xts, and when they run, they will do what they
>> would have done if I had EXECUTEd them right away.
>
>Not really. Consider you start with the state-smart world, and you compile
>in compilation state into a word that might later run in interpretation
>state. If you EXECUTEd them right away, they would show their compilation
>semantics, if you COMPILE,d them, they would later show their interpretation
>semantics. Or their compilation semantics, if the assumption above is
>wrong.

You don't need STATE for that. Any variable that the code reads will
do. So, of course, in the presence of state variables, these
variables all need to have the same state. But for the kind of stuff
I am thinking of, that is the case.

>> Say, if I know that I want to EXECUTE a sequence of xts several times,
>> I can define a colon definition, COMPILE, the xts into it, and then
>> call the colon definition several times. With your approach, I could
>> no lonmger do that; you suggested that I should replace COMPILE, with
>> POSTPONE LITERAL POSTPONE EXECUTE, but that's neither nice nor fast.
>
>If you actually need that property. For me, this looks like an academic
>exercise.

Maybe, after all, I am an academic. But I find such equivalences
useful. I find it useful to know that > is equivalent to SWAP <, that
it is always equivalent and not just nearly always. And I think that
industrial people profit from that, too. Maybe after a few decades of
academic mulling over it, but eventually they do.

Currently I give an example of how MAP-ARRAY
<http://www.complang.tuwien.ac.at/forth/gforth/Docs-html/Execution-Tokens-Tutorial.html>
relates to COMPILE-MAP-ARRAY
<http://www.complang.tuwien.ac.at/forth/gforth/Docs-html/Advanced-macros-Tutorial.html>.
If that equivalence no longer holds, I either have to say that this
may or may not work, but more likely I would not have written it at
all if that equivalence had not held.

> I've built my fair share of hand-written compilers. They usually
>use the fragment Mitch Bradley posted to clarify what COMPILE, was for, i.e.
>that FIND 0> IF EXECUTE ELSE COMPILE, THEN, because they wanted the
>compilation semantics of the words found, not the interpretation semantics.
>This is the common case. The fact that we need to look at a result from
>FIND, which is not passed on with the xt to decide whether to use EXECUTE or
>COMPILE, shows that this solution will have problems (and the problems are
>in the xt, which is why we can't find a good replacement for FIND).

FIND is a dinosaur that deserves to die. But if you want to keep FIND
and keep that interpreter loop, no problem; This kind of interpreter
loop works with Gforth's FIND in all the versions of Gforth that had
combined words and where the equivalence held, so don't use that as an
excuse for breaking the equivalence!

Sure this FIND is ugly, but what do you expect? It's a relic from
another world.

>BTW: for the state-smart implementations, it won't work either way.
>COMPILE, will just preserve the state-smart nature of the word, just as well
>as ]] LITERAL EXEUCTE [[ will (no difference there).

Yes, that's as it should be.

>The only place in Gforth where the new semantics might case problems is in
>DEFERS. DEFERS takes the action of a deferred word, and compiles it. Note
>that a deferred word itself can only expose the execution semantics of the
>word bound to it, so the definition
>
>: defers ( "name" -- ) ' defer@ compile, ; immediate compile-only
>
>is actually not correct. It should use that ]] literal execute [[ thing.

So you found a place where the equivalence matters, and it was not
even written by me. Maybe it is not so academic after all. The
definition of DEFERS used to be correct. By making COMPILE, incorrect
DEFERS becaome incorrect. The mistake is not in DEFERS, but in the
change to COMPILE,.

>The same thing for the OOP early binding, because the vtable also can only
>do EXECUTE, and therefore, COMPILE, would be wrong.

And another example.

>However, despite defers is used not so infrequently, this didn't cause any
>problem.

Sure, most of the time the equivalence still holds. But STATE-smart
words also work most of the time.

Bernd Paysan

unread,

Mar 7, 2013, 7:31:00 PM3/7/13

to

Anton Ertl wrote:
>>: defers ( "name" -- ) ' defer@ compile, ; immediate compile-only
>>
>>is actually not correct. It should use that ]] literal execute [[ thing.
>
> So you found a place where the equivalence matters, and it was not
> even written by me. Maybe it is not so academic after all. The
> definition of DEFERS used to be correct. By making COMPILE, incorrect
> DEFERS becaome incorrect. The mistake is not in DEFERS, but in the
> change to COMPILE,.

No, the thing does not cause any problems at all. It shows that the
equivalence for all practical purposes is correct, and your concern is
academical. You wouldn't assign a word with special compilation semantics
to a deferred word, because it already would be broken that way. Think
about this:

defer " immediate
\ we want our single quote back, but the user can select c" or s"
' s" is " \ a reasonable default

: foo " blabla" ;

Oops, this doesn't work, it would have worked if s" is state-smart.
Actually, we can make it work with

: [defer] ( "name" -- )
defer [: >body @ compile, ;] set-compiler ;

Now if we use [defer] to define a deferred word, it can take a special
compilation semantics word, and perform their special compilation semantics
in compilation state.

>>The same thing for the OOP early binding, because the vtable also can only
>>do EXECUTE, and therefore, COMPILE, would be wrong.
>
> And another example.

But still, it does not cause any problems. This is because the methods are
actually just :noname definitions which don't have any special compilation
semantics. Only if they had, it would be a potential problem. You might
want something like [method] to create such a selector, which then would
allow you to define compiler macro methods (there's a place for them,
BerndOOF has several).

If you want to extend such a compiler macro method, and use super on them,
you then want to carefully select the interpretation and the compilation
semantics, so you actually need to versions of super.

>>However, despite defers is used not so infrequently, this didn't cause any
>>problem.
>
> Sure, most of the time the equivalence still holds. But STATE-smart
> words also work most of the time.

Yes. But on the other hand, IMHO this equivalence is an artificial one, and
comes out of a mis-specification of compile,, as you actually don't have
access to the compilation semantics of an xt in ANS Forth.

The fix for that IMHO is to provide interp, for the purpose you want. That
word makes pretty clear that you append the interpretation/execution
semantics, and are not actually attempting to really compile the token.

BTW, in VFX, [INTERP] is really just

: [interp] ( "name" -- )
' postpone Literal postpone execute ; immediate

The compiler does some optimizations, so it ends up as call, but no inlining
magic is performed. An optimizted version in Gforth could use an INTERP,
which does the original case statement for COMPILE, (looking at the doer to
decide what to compile).

This is Forth, we are striving for simplicity. One indication about
simplicity is orthogonality in the description. The ANS Forth standard has
three clearly defined semantics: interpretation, compilation, and run-time
semantics. By default, run-time=interpretation, and compilation=add
interpretation semantics. Now, COMPILE, is specified to perform the
default, this is breaking the symmetry, and making things more complicated.
The reason is that at that time, non-default compilation semantics was
specified by IMMEDIATE, and the xt doesn't know if it's immediate or not.

Let's assume we do what you suggest: add another method to the vtable, and
add another word to do "the compiling", which would be something different
than compile,. Now, you might have set-compiler and set-compile, to set
them. How to choose which is the appropriate one? Most of the time, it
doesn't make a difference, so chances are high that people will not
understand why there are two, and treat them interchangeable. It will
sometimes result in non-optimal code (if the optimizing version ends up in
compiler, but should end up in compile,), sometimes in bugs like the above,
where you can't use COMPILE, to get at the interpretation semantics.

If you have two things that do almost the same, but differ in sublte things
that happen only once in a blue moon, you better do just one of them.

--
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/

Rod Pemberton

unread,

Mar 8, 2013, 4:22:16 AM3/8/13

to

"Rob Sciuk" <r...@controlq.com> wrote in message
news:alpine.BSF.2.00.1...@yoko.controlq.com...

> On Wed, 6 Mar 2013, Rod Pemberton wrote:

> >> My question was intended to open a discussion (which seems to
> >> have worked) to question some long held truths, and in hopes
> >> of improving Forth rather than to damage it. I believe that
> >> the approach above is exactly the kind of thing I'd hoped to
> >> avoid -- confuscation is never good.
> >>
> >
> > Q: non-word "confuscation" --> obfuscation + confiscate ... ?
> >

> On further reflection, I declare this to be a new word.
> Henceforth, the following will have meaning:
>
> CONFUSCATION: A political act.
> Verb: to CONFUSCATE
>
> You may use it without royalties.
>

You're too late:
http://www.urbandictionary.com/define.php?term=confuscate
http://s1ngularity.blogspot.com/2004/01/confuscation.html

:-)

RP

unread,

Mar 8, 2013, 7:58:26 AM3/8/13

to

Mark Wills <markrob...@yahoo.co.uk> wrote:
> On Mar 8, 10:25?am, Andrew Haley <andre...@littlepinkcloud.invalid>

> wrote:
>> Mark Wills <markrobertwi...@yahoo.co.uk> wrote:
>> > I've been thinking about this, and it really is a tricky problem.
>>
>> > Separating out compilation and interpretation semantics, such that a
>> > system can excplicitly determine the difference, seems trvial to this
>> > novice: The problem is that FIND, ' and ['] can only return a single
>> > XT. That's the problem. FIND can tell you if a word is immediate, but
>> > that's only half of the information. Like you say, if it's state
>> > smart, how do you get the compilation (run-time) semantics? (I know
>> > this is obvious to you guys, please humour me as I endevour to keep
>> > up!)
>>
>> > What to do? Can we not invent new words that supercede the old ones
>> > (the old ones could remain for a while) that return both XT's?
>>
>> Before you get deep into this, have you read the previous threads
>> where we discussed it at length?
>>
>> Andrew.
>
> Good morning, Andrew.

Morning.

> Well, this has come up probably more than 10 times in the 3 or 4 years
> that I've been frequenting C.L.F. Which particular threads were you
> referring to? :-)

We discussed, at considerable length, the way gforth's FIND works, and
why it works with the classic interpreter loop. One of the
suggestions was IIRC very similar to what you're saying. You seem to
be talking like this idea came out of a clear blue sky, but it has
been done to death and then some.

Andrew.

https://groups.google.com/group/comp.lang.forth/browse_frm/thread/b5ada700d07a276a/

Anton Ertl

unread,

Mar 8, 2013, 9:41:37 AM3/8/13

to

Bernd Paysan <bernd....@gmx.de> writes:
>Anton Ertl wrote:
>>>: defers ( "name" -- ) ' defer@ compile, ; immediate compile-only
>>>
>>>is actually not correct. It should use that ]] literal execute [[ thing.
>>
>> So you found a place where the equivalence matters, and it was not
>> even written by me. Maybe it is not so academic after all. The
>> definition of DEFERS used to be correct. By making COMPILE, incorrect
>> DEFERS becaome incorrect. The mistake is not in DEFERS, but in the
>> change to COMPILE,.
>
>No, the thing does not cause any problems at all.

Yet.

> It shows that the
>equivalence for all practical purposes is correct, and your concern is
>academical.

Well, then why do you want to break that equivalence?

>You wouldn't assign a word with special compilation semantics
>to a deferred word, because it already would be broken that way.

Depends on what you expect. I guess you are thinking of a changeable
SYNONYM. I would hav thought that your universal xt and broken
"COMPILE," are intended to achieve that with an unchanged DEFER (just
like for the interpreter loop). If it does not achieve that, maybe
your concepts are not as good as you think.

OTOH, I don't expect DEFER to be a changeable SYNONYM, just like I
don't expect ALIAS to be a SYNONYM. I would just define a new word
for changeable synonyms, and that may or may not be tied to your ideas
about xts (preferably it would not depend on them).

Here you suggest a version that depends on them:

>: [defer] ( "name" -- )
> defer [: >body @ compile, ;] set-compiler ;

But it would be just as possible to write a [defer] without breaking
"COMPILE,", or (with a different interface) to write it in Gforth
0.7.0 (you would either pass the name, the nt or the xt and ct to the
IS replacement).

However, looking at the example again, I expect a deferred word to
bind at run-time, not at deferred-word compilation like your [DEFER]
does. I have not seen an example where such a changeable synonym that
binds at compile time when it is compiled, and at run-time when it is
interpreted, is useful.

The " example is not convincing, because you cannot use S" and C"
interchangeably (different stack effects), so switching them around
dynamically makes no sense. What you try to achieve with that can be
achieved already with SYNONYM and search order.

[...]

>>>The same thing for the OOP early binding, because the vtable also can only
>>>do EXECUTE, and therefore, COMPILE, would be wrong.
>>
>> And another example.
>
>But still, it does not cause any problems. This is because the methods are
>actually just :noname definitions which don't have any special compilation
>semantics. Only if they had, it would be a potential problem. You might
>want something like [method] to create such a selector, which then would
>allow you to define compiler macro methods (there's a place for them,
>BerndOOF has several).

So you can think of potential problems. If combined words are used
more commonly than now (and I guess you would be among the frequent
users), the problems will eventually show up in reality.

>Yes. But on the other hand, IMHO this equivalence is an artificial one

Why do you think so?

Looking at the DEFERS and [BIND]/SUPER/:: examples, it seems pretty
natural; did you even consider for one second that you should write
"POSTPONE literal POSTPONE execute" instead of "COMPILE," when you
originally wrote that code?

> and
>comes out of a mis-specification of compile,, as you actually don't have
>access to the compilation semantics of an xt in ANS Forth.

In ANS Forth compilation semantics is bound to named words, not xts.
There are two places where it occurs: When the text interpreter
processes a (named) word in compilation state, and in POSTPONE (which
also takes a name).

I think that COMPILE, is well specified. Why do you think it is
mis-specified? And please post your "improved" specification again.

>The fix for that IMHO is to provide interp, for the purpose you want. That
>word makes pretty clear that you append the interpretation/execution
>semantics, and are not actually attempting to really compile the token.

You mean that we should change "COMPILE," to do something different
from what it does now, and introduce "INTERP," to do what "COMPILE,"
does now? Memories of Forth-83 arise.

An alternative would be to just leave "COMPILE," as it is and
introduce a new word (maybe "COMPSEM,", but you sure can come up with
a better name) that does what you want. You would have to use
"COMPSEM," in the interpreter loop, but the FIND would be cleaner.

Or, if you want to keep the Bradley interpreter loop unchanged,
implement FIND to return different xts depending on STATE (like Gforth
is doing now). You can still introduce "COMPSEM," if you want.

>This is Forth, we are striving for simplicity. One indication about
>simplicity is orthogonality in the description. The ANS Forth standard has
>three clearly defined semantics: interpretation, compilation, and run-time
>semantics. By default, run-time=interpretation, and compilation=add
>interpretation semantics.

Yes, except that what you call "run-time semantics" is called
"execution semantics" in ANS Forth, and in ANS Forth the run-time
semantics of some words is a helper for defining the compilation
semantics of that word.

> Now, COMPILE, is specified to perform the
>default, this is breaking the symmetry, and making things more complicated.

What symmetry?

What makes things complicated is combined words.

>The reason is that at that time, non-default compilation semantics was
>specified by IMMEDIATE, and the xt doesn't know if it's immediate or not.

And? Why should it be necessary to know that?

>Let's assume we do what you suggest: add another method to the vtable, and
>add another word to do "the compiling", which would be something different
>than compile,. Now, you might have set-compiler and set-compile, to set
>them. How to choose which is the appropriate one? Most of the time, it
>doesn't make a difference, so chances are high that people will not
>understand why there are two, and treat them interchangeable. It will
>sometimes result in non-optimal code (if the optimizing version ends up in
>compiler, but should end up in compile,), sometimes in bugs like the above,
>where you can't use COMPILE, to get at the interpretation semantics.

These are not bugs. These use COMPILE, as specified.

OTOH, if you are right, you should be able to point to lots of uses of
COMPILE, where the current usage of COMPILE, is wrong, because the
programmer actually intended to perform the compilation semantics
instead of appending the intepretation/execution semantics. Until now
we only know several cases that would be broken by the change you
suggest for "COMPILE,". How many cases do you have that would be
fixed by that change?

>If you have two things that do almost the same, but differ in sublte things
>that happen only once in a blue moon, you better do just one of them.

Sounds like an argument against your "INTERP," (as well as against
"COMPSEM,").

You can also look at how many uses of POSTPONE, are wrong. Of course
POSTPONE, is sufficiently different (different stack effect) that it's
not easy to use it instead of COMPILE,. If so, and you are right
about confusing "COMPILE," with the other words, that's an argument
against going for a universal xt.

Anton Ertl

unread,

Mar 8, 2013, 11:26:00 AM3/8/13

to

"Rod Pemberton" <do_no...@notemailnotz.cnm> writes:
>What's wrong with a set of words that can select the needed
>behavior prior to the word being used?
>
>E.g., .I .C .P
>
> .I CHAR \ immediate behavior
> .C CHAR \ compile behavior
> .P CHAR \ etc.
>
>Of course, each mode would be the default for that mode. Who
>wants to type them all the time?

I once did an experiment with prefixes "[", "_" and "]" for
interpretation compilation, and postponeing, inspired by colorForth:

[5 [constant row-size
[row-size [cells [constant row-byte-size

[: gen-innerproduct [( a[row][*] -- xt )
[\ xt is of type ( b[*][column] -- n )
[\ this would be a candidate for using ]] ... [[
_>r _:noname _r>
]0 ]SWAP
_row-size _0 [do
]dup ]@
_dup _@ _literal ]* ]under+
]cell+ _row-byte-size _+
[loop
_drop
]drop _;
[;

It was surprisingly easy to change Gforth to accept that.

Disadvantage: You cannot just copy and paste between interpreted and
compiled code.

Anton Ertl

unread,

Mar 8, 2013, 11:34:52 AM3/8/13

to

Mark Wills <markrob...@yahoo.co.uk> writes:
>We're pointing the finger at "state-smart" (using the conventional
>meaning ;-) words because we cannot differentiate interpretation
>semantics from compilation sematics via tick.
>
>So, rotating the problem by 180 degrees, the problem is tick, not the
>state-smart words themselves.
>
>Therefore, simply standardise a new word, something like 'I (which
>means 'tick the *interpretation* behaviour/address/cfa/whatever' of
>the word.
>
>Of course, this has implications on the underlying architecture, and
>raises further questions. Now the dictionary (presumably) must be able
>to store pointers to both the compilation (for regular ') and
>interpretation semantics (for 'I ).

Since ' is defined to produce an xt for the interpretation semantics,
it would be a synonym to 'I, no need to standarize 'I. What you have
described is pretty close to what has been in Gforth from 1996 to
2012.

Of course these words where ' knows how to access the interpretation
semantics (and POSTPONE the compilation semantics) do not behave like
STATE-smart words, and therefore are not STATE-smart words; if you
write a conventional STATE-smart word in Gforth, you will get a
STATE-smart word: ticking it will give you the xt that, when executed,
depends on STATE.

> Furthermore, what happens if the
>word has no interpretation sematics? Do you return the compilation
>sematics as a default, or return a 0 to expressly indicate that there
>are no interpretation semantics? All open for argument.

Gforth heeds the "fail early" principle here and reports an error:

' if
*the terminal*:1: Cannot tick compile-only word (try COMP' ... DROP)
' if
^^

Anton Ertl

unread,

Mar 8, 2013, 11:49:55 AM3/8/13

to

Bernd Paysan <bernd....@gmx.de> writes:
>Anton Ertl wrote:

>> The difference between the intelligent and the smart COMPILE, is in
>> the xt that the system designer has put in the comp field: with the
>> "intelligent COMPILE," it compiles the interpretation/execution
>> semantics represented by the xt. With the "smart COMPILE," it
>> performs the compilation semantics. For many words, there is no
>> difference, but for words like S" and TO, there is.
...
>The idea to make COMPILE, smart is not my idea, it's Stephen Pelc's idea -
>MPE at one point in time didn't have a single immediate word in VFX Forth,
>because they liked comp: so much.

But actually, some ANS Forth words are
>deliberately specified as "this is an immediate word", so you better set the
>immediate flag on them when you implement a Forth system.
>
>And for immediate words, the comp field is pointing to the default
>compilation method, so COMPILE, does just what Anton wants it to do.
>
>If we really need Anton's thing dearly, we should have INTERP, and [INTERP]
><name> (maybe better without those brackets, we don't have them on POSTPONE)
>as compagnion for COMPILE, and POSTPONE (POSTPONE should have been called
>COMPILE - as COMPILE did only work for non-immediate words, and extending it
>for immediate words wouldn't have required a name-change).

>
>--
>Bernd Paysan
>"If you want it done right, you have to do it yourself"
>http://bernd-paysan.de/
>

Anton Ertl

unread,

Mar 8, 2013, 12:14:01 PM3/8/13

to

"Elizabeth D. Rather" <era...@forth.com> writes:
>Second, many academic and free-lance Forth programmers devote their
>energies to developing kernels, rather than writing sharable application
>code.
>
>The current Forth standards make it possible to share and reuse code.
>It's the choice of the programmers whether they want to make use of this
>capability.

Now why is that? Why do other languages have a thriving culture of
sharing reusable code and Forth doesn't?

One of the reasons is that the idea of code reuse is despised by a
significant part of the Forth community, according to lore starting
with Charles Moore.

Another reason is that there are still things missing in the standard
that would make sharing and reusing code easy. Some suggestions for
features in that area have been met with indifference or hostility,
and that probably comes from the culture mentioned above.

If someone proposes a feature that is useful for writing or combining
several reusable libraries, that feature is often opposed with the
argument that it just takes a few seconds to write or change the three
lines that having this feature would automate.

There are also people writing that the standard is not about portable
programs, but portable programmers, implying that features that allow
porting programs are unnecessary. The whole concept of taking a
program and running it unchanged on another system on another platform
appears alien and not desirable to a significant part of the Forth
community.

- anton

Elizabeth D. Rather

unread,

Mar 8, 2013, 1:57:59 PM3/8/13

to

On 3/8/13 7:14 AM, Anton Ertl wrote:
> "Elizabeth D. Rather" <era...@forth.com> writes:
>> Second, many academic and free-lance Forth programmers devote their
>> energies to developing kernels, rather than writing sharable application
>> code.
>>
>> The current Forth standards make it possible to share and reuse code.
>> It's the choice of the programmers whether they want to make use of this
>> capability.
>
> Now why is that? Why do other languages have a thriving culture of
> sharing reusable code and Forth doesn't?

Several reasons, IMO:

* Other languages produce precompiled binary libraries intended to be
processed by a linker, so compiler compatibility isn't much of an issue.
The absence of a linker requirement in Forth is a major convenience to
most of us, but there are consequences.

* There are larger communities of programmers in other languages,
because universities embraced them more, and academia is a major source
of distributed code.

* Since so many people in the Forth community insist on writing their
own kernels, and further insist on incorporating their own quirks, it's
been hard to achieve compatibility at high levels. People complain that
Forth standards are pretty low-level, even though very high level work
is being done in Forth. This is why. Languages for which it's hard to
write compilers suffer from this much less.

* Since it's so easy to write common procedures in Forth, they tend to
be tightly adapted to the application for which they're developed, which
makes them less amenable to sharing.

> One of the reasons is that the idea of code reuse is despised by a
> significant part of the Forth community, according to lore starting
> with Charles Moore.

That is not entirely true. Chuck carried around with him a large library
of things he had written in the past (in the 70's, a notebook full of
microfiches and a reader!), and always started with past code when he
could. Of course, then he re-wrote it for the application at hand (see
my last point above).

And everyone I know who is spending a career writing Forth applications
has a bulging "back pocket" of reusable code. FORTH, Inc., MPE, and
probably most other widely used systems have such libraries.

> Another reason is that there are still things missing in the standard
> that would make sharing and reusing code easy. Some suggestions for
> features in that area have been met with indifference or hostility,
> and that probably comes from the culture mentioned above.

Probably because the perceived need for additional library tools just
isn't all that strong.

> If someone proposes a feature that is useful for writing or combining
> several reusable libraries, that feature is often opposed with the
> argument that it just takes a few seconds to write or change the three
> lines that having this feature would automate.

Yep. See above.

> There are also people writing that the standard is not about portable
> programs, but portable programmers, implying that features that allow
> porting programs are unnecessary. The whole concept of taking a
> program and running it unchanged on another system on another platform
> appears alien and not desirable to a significant part of the Forth
> community.

What's a portable *program*? If it's a program of any significance, it
has a user interface that's dependent on an underlying host OS and the
language interface to it. Or it's an application that supports special
hardware, proprietary protocols, etc. Tell me about all the whole C
programs that run on any platform with any C compiler without change.

Portable capabilities are possible in Forth and exist, about as much as
there is a demand for them.

Coos Haak

unread,

Mar 8, 2013, 2:57:38 PM3/8/13

to

Op Fri, 8 Mar 2013 02:19:07 -0800 (PST) schreef Mark Wills:

<snip>

>: ASCII
> ( interpretation: "string" -- c )
> ( run-time: -- c )
> STATE @ IF
> POSTPONE CHAR
> ELSE
> POSTPONE CHAR POSTPONE LIT ,
> THEN ; STATE-SMART

Why POSTPONE CHAR ? CHAR is not immediate, do you want to compile it in a
definition?
: CHAR BL WORD 1+ C@ ;

: ASCII
CHAR STATE @ IF POSTPONE LITERAL THEN ;
( STATE-SMART )

--
Coos

CHForth, 16 bit DOS applications
http://home.hccnet.nl/j.j.haak/forth.html

Bernd Paysan

unread,

Mar 9, 2013, 11:04:30 AM3/9/13

to

Anton Ertl wrote:
>>No, the thing does not cause any problems at all.
>
> Yet.
>
>> It shows that the
>>equivalence for all practical purposes is correct, and your concern is
>>academical.
>
> Well, then why do you want to break that equivalence?

Because it was an unintended consequence of a misformulation of what
COMPILE, is supposed to do. It should *compile* something. It should not
add interpretation semantics.

I think our main difference is that I consider the current COMPILE, as
broken and misspecified, and you think it's god-given. It's not. It's
Mitch-Bradley-given, and Mitch Bradley had explained what he wanted it to
do. And following that explanation, the spec of COMPILE, in ANS Forth is
simply wrong.

>>You wouldn't assign a word with special compilation semantics
>>to a deferred word, because it already would be broken that way.
>
> Depends on what you expect. I guess you are thinking of a changeable
> SYNONYM. I would hav thought that your universal xt and broken
> "COMPILE," are intended to achieve that with an unchanged DEFER (just
> like for the interpreter loop). If it does not achieve that, maybe
> your concepts are not as good as you think.

How would that be possible? The unchanged DEFER uses EXECUTE on the token,
never COMPILE,. So how would that work without other changes? The token
knows how to compile, but the unchanged DEFER can't access this.

> OTOH, I don't expect DEFER to be a changeable SYNONYM, just like I
> don't expect ALIAS to be a SYNONYM. I would just define a new word
> for changeable synonyms, and that may or may not be tied to your ideas
> about xts (preferably it would not depend on them).

You probably would use your nt for that. This is something I absolutely
hate about typical Java code: Tons of types which are slightly different,
and do almost the same, but are incompatible, you often can only do forward
conversions (but not backward to where this thing came from), and it all
wraps onion layers around the real thing. Don't add things like this to a
language. Try to reduce the number of types to what you actually need.

> Here you suggest a version that depends on them:
>
>>: [defer] ( "name" -- )
>> defer [: >body @ compile, ;] set-compiler ;
>
> But it would be just as possible to write a [defer] without breaking
> "COMPILE,", or (with a different interface) to write it in Gforth
> 0.7.0 (you would either pass the name, the nt or the xt and ct to the
> IS replacement).

You see your inflation of types, which do almost, but certainly not exactly
the same thing, which lead to confusion and again subtle bugs.

> However, looking at the example again, I expect a deferred word to
> bind at run-time, not at deferred-word compilation like your [DEFER]
> does. I have not seen an example where such a changeable synonym that
> binds at compile time when it is compiled, and at run-time when it is
> interpreted, is useful.

That's probably why we don't have such a thing. BTW: The special-
compilation semantics OOP methods in BerndOOF are also all early bound, so
they actually don't go through a method table. But that might change if we
think of them as part of a Meta-OOP package, where different OOP systems
might want to implement them slightly differently.

> The " example is not convincing, because you cannot use S" and C"
> interchangeably (different stack effects), so switching them around
> dynamically makes no sense. What you try to achieve with that can be
> achieved already with SYNONYM and search order.

Maybe. The " example would be introduced as harness code for code that uses
" instead of C" or S".

> [...]

>>Only if they had, it would be a potential problem. You might
>>want something like [method] to create such a selector, which then would
>>allow you to define compiler macro methods (there's a place for them,
>>BerndOOF has several).
>
> So you can think of potential problems. If combined words are used
> more commonly than now (and I guess you would be among the frequent
> users), the problems will eventually show up in reality.

Which means go forward, and implement BerndOOF using monotokens, and see
when it starts to cause problems.

>>Yes. But on the other hand, IMHO this equivalence is an artificial one
>
> Why do you think so?
>
> Looking at the DEFERS and [BIND]/SUPER/:: examples, it seems pretty
> natural; did you even consider for one second that you should write
> "POSTPONE literal POSTPONE execute" instead of "COMPILE," when you
> originally wrote that code?

Let's go back again, and let's assume we have a POSTPONE method (there is),
which we want to override (which we can't ATM), say for debugging purposes.
So we do

: postpone ( "name" -- ) compile-only-error ;
comp: ( xt -- ) drop ." Print some debugging stuff" super postpone postpone
;

super as literal+execute would be clearly wrong here. Note that we need
this double "postpone", because the super postpone just performs the
compilation semantics in the super context, and there, it just postpones the
next word. Which is postpone in the super context - that's what we want.

BTW: Most of those state-smart words in BerndOOF are state-smart for
optimization, not as compilation macros. We want use COMPILE, on them,
definitely, because we want to optimize the code.

>> and
>>comes out of a mis-specification of compile,, as you actually don't have
>>access to the compilation semantics of an xt in ANS Forth.
>
> In ANS Forth compilation semantics is bound to named words, not xts.

Yes, this comes from the immediate flag, which is a header flag, and not
available through the xt.

> There are two places where it occurs: When the text interpreter
> processes a (named) word in compilation state, and in POSTPONE (which
> also takes a name).

Yes, but if you think the concept further, and we do that, we need
compilation semantics on tokens that are detached from their names, and that
detaching happens before we know if we want to compile them or just
interpret them.

> I think that COMPILE, is well specified. Why do you think it is
> mis-specified?

You still don't want to accept the rationale Mitch Bradley has given for
COMPILE,, and the fact that COMPILE, was added late in the process and
didn't get much review.

> And please post your "improved" specification again.

COMPILE, performs the compilation semantics of non-immediate words. The
compilation semantics of immediate words is performed with EXECUTE.
immediate words are a legacy of the STATE-smart universe. We can't get rid
of them quickly, but we know we need something better.

>>The fix for that IMHO is to provide interp, for the purpose you want.
>>That word makes pretty clear that you append the interpretation/execution
>>semantics, and are not actually attempting to really compile the token.
>
> You mean that we should change "COMPILE," to do something different
> from what it does now, and introduce "INTERP," to do what "COMPILE,"
> does now? Memories of Forth-83 arise.

Well, VFX has already changed that 15 years ago. If there was a serious
problem with that approach, I would have found it. I found quite a number
of problems with VFX (for that kind of "guru code" I write). COMPILE,
wasn't one. And I do write my own interpreters which use COMPILE, a lot,
especially in BerndOOF.

Stephen Pelc accused me that my attitude of porting programs to other
systems (by improving their implementation quality rather than lowering my
expectations) would not make the next port easier, and he's right. It will
not make *my* next port easier, but it will make the next port of all others
easier, because the systems behave more alike.

> An alternative would be to just leave "COMPILE," as it is and
> introduce a new word (maybe "COMPSEM,", but you sure can come up with
> a better name) that does what you want. You would have to use
> "COMPSEM," in the interpreter loop, but the FIND would be cleaner.

IMHO we just can't have both Mitch's interpreter loop and what you want.
From my point of view, Mitch's interpreter loop is more important, because
this really does exist (e.g. in BerndOOF - if VFX's COMPILE, would do what
you want it to do, BerndOOF quite likely wouldn't work).

MPE introduced [INTERP] <name>, because they had two use cases for that,
both with names. Given that it just expands to ['] <name> EXECUTE, it's not
sure if that was actually necessary.

> Or, if you want to keep the Bradley interpreter loop unchanged,
> implement FIND to return different xts depending on STATE (like Gforth
> is doing now). You can still introduce "COMPSEM," if you want.

I can't let FIND return different xts for that, because an executable
"compilation token" always is a pair of xts (as in Gforth). This won't
work.

There's some place for a NAME>COMP EXECUTE replacement, which handles both
the immediate case and the separate compilation semantics case.

> Yes, except that what you call "run-time semantics" is called
> "execution semantics" in ANS Forth, and in ANS Forth the run-time
> semantics of some words is a helper for defining the compilation
> semantics of that word.

Nah. The execution semantics of an immediate word is its compilation
semantics, and then it can add some other run-time semantics.

>> Now, COMPILE, is specified to perform the
>>default, this is breaking the symmetry, and making things more
>>complicated.
>
> What symmetry?

As described in the ANS Forth standard, words have interpretation and
compilation semantics. ANS Forth's COMPILE, performs default compilation
semantics even on those words which don't have default compilation
semantics. This is a part of COMPILE,'s behavior, which should have better
not specified. And in fact, IMHO VFX shows that leaving that open is not a
problem at all - other parts of VFX were problems, and have been fixed
since, but the non-standard COMPILE, is not one.

> What makes things complicated is combined words.

No, what makes things complicated is this mess of immediate and state-smart
to make words have non-default compilation semantics, and to detach the
compilation semantics from the xt.

>>The reason is that at that time, non-default compilation semantics was
>>specified by IMMEDIATE, and the xt doesn't know if it's immediate or not.
>
> And? Why should it be necessary to know that?

Because if it doesn't know that, you have to either introduce a new type
(the nt), or you have to pass around strings and maybe even contexts
(vocabulary stacks)... and then you end up with the bag of problems you
have with MPE's source inliner. You don't want to have that, you want to
have tokens. And I don't want to have yet another token, which is not
directly executable, because that did cause significant problems in Gforth,
when I started with the recognizers.

>>Let's assume we do what you suggest: add another method to the vtable, and
>>add another word to do "the compiling", which would be something different
>>than compile,. Now, you might have set-compiler and set-compile, to set
>>them. How to choose which is the appropriate one? Most of the time, it
>>doesn't make a difference, so chances are high that people will not
>>understand why there are two, and treat them interchangeable. It will
>>sometimes result in non-optimal code (if the optimizing version ends up in
>>compiler, but should end up in compile,), sometimes in bugs like the
>>above, where you can't use COMPILE, to get at the interpretation
>>semantics.
>
> These are not bugs. These use COMPILE, as specified.

The spec is buggy. As I said, you can't have both: The spec for COMPILE,
and Mitch's interpreter. Mitch's interpreter is the reason to specify
COMPILE,, and it went wrong. Spec bugs are the worst kinds, because some
people, like you, think the spec is always right. In this case, it is just
wrong, and we need to fix it.

> OTOH, if you are right, you should be able to point to lots of uses of
> COMPILE, where the current usage of COMPILE, is wrong, because the
> programmer actually intended to perform the compilation semantics
> instead of appending the intepretation/execution semantics.

Look at BerndOOF, it does that. It uses COMPILE, on all non-immediate
words, and wants compilation semantics. If you replace the immediate words
with special-compilation semantics words, this part of BerndOOF will still
work as it should. If you change the semantics of COMPILE, to what you
want, you have to keep the state-smart implementation.

Do you now tell me that state-smart is actually *not* evil, because it
harmonizes perfectly with your understanding of what COMPILE, should do?

> Until now
> we only know several cases that would be broken by the change you
> suggest for "COMPILE,". How many cases do you have that would be
> fixed by that change?

All the cases of Mitch's interpreter. As long as you stay with a immediate-
flag implementation, and have no special compile, words, you won't see the
difference.

>>If you have two things that do almost the same, but differ in sublte
>>things that happen only once in a blue moon, you better do just one of
>>them.
>
> Sounds like an argument against your "INTERP," (as well as against
> "COMPSEM,").

Yes, use ['] foo exeucte instead, don't give it a name. It's likely below
threshold of introducing a new name for it. If you deliberately want
interpretation semantics, this is how you get it. This is *not* the usual
case. Usually, when you want deliberate interpretation semantics, it's
because you reuse some word for defining a combined word. Then, you know
what you are doing.

> You can also look at how many uses of POSTPONE, are wrong.

You think POSTPONE where you really want [INTERP]? Works only in state-
smart systems. Yes, by removing state-smartness, you do break some
programs. You fix way more than you break, which is why this change is
accepted.

> Of course
> POSTPONE, is sufficiently different (different stack effect) that it's
> not easy to use it instead of COMPILE,. If so, and you are right
> about confusing "COMPILE," with the other words, that's an argument
> against going for a universal xt.

You can only reduce confusion about things like this if you *reduce* the
number of terms, states, and concepts. Not by increasing them, as you do.

The Forth approach at things is not through increasing complexity. It is
often by using ad-hoc approaches, which is why we have immediate and state.
But if we get at the limits, the Forth approach is to rethink, comprehend
the problem in total, and then remove the unnecessary cruft.

Bernd Paysan

unread,

Mar 9, 2013, 11:31:06 AM3/9/13

to

Elizabeth D. Rather wrote:
>> There are also people writing that the standard is not about portable
>> programs, but portable programmers, implying that features that allow
>> porting programs are unnecessary. The whole concept of taking a
>> program and running it unchanged on another system on another platform
>> appears alien and not desirable to a significant part of the Forth
>> community.
>
> What's a portable *program*? If it's a program of any significance, it
> has a user interface that's dependent on an underlying host OS and the
> language interface to it. Or it's an application that supports special
> hardware, proprietary protocols, etc. Tell me about all the whole C
> programs that run on any platform with any C compiler without change.

Yes, but let's consider reality. If I write a C program for Linux, I have
two free compilers to use, and two proprietary: clang or gcc for the free
ones, Portland or Intel's C compiler for proprietary. They are pretty
interchangeable, as they all support at least most GCC extensions (which is
also somewhat true for GCC itself, it sometimes doesn't support all GCC
extensions ;-).

I tried compiling Gforth on clang a few months ago, and there were only some
minor quirks, easy to fix. clang is still not as good as GCC, concerning
code quality, and it's also a lot slower to compile the engine.

Compare that to porting MINOS to VFX. Yes, we have the situation here that
everybody writes his own kernel and adds own quirks (like MPE's source code
tokenzier), and I'm definitely in Anton's camp here: I would rather advise
to fix these quirks instead of working around them. If we have less quirky
systems, our programs would be more portable. And when comparing Forth
systems, VFX is certainly one of the least quirky ones, and certainly also
not one of those which you can easily rewrite yourself in an afternoon - not
with the analytical compiler.

For sure, a common C interface syntax would do us all good, to write non-
trivial portable programs on hosted systems. I've recently done work with
stranger C libraries than I was used to, i.e not as neat and clean as
OpenGL, but something like the JNI interface. A C++-style library written
in C. Function pointers in structs, callbacks, structs passed as arguments
(*not* pointers to structs, structs as such)... and you absolutely need to
call Java if you have a serious Forth application on Android. The good
point: After you have managed to access the JNI library, the calls to Java
from Forth are actually easier than the calls to C from Forth. I use my
current-object principle, so it feels like a Forth-OOP system.

Andrew Haley

unread,

Mar 9, 2013, 1:28:10 PM3/9/13

to

Bernd Paysan <bernd....@gmx.de> wrote:
> Elizabeth D. Rather wrote:
>>> There are also people writing that the standard is not about portable
>>> programs, but portable programmers, implying that features that allow
>>> porting programs are unnecessary. The whole concept of taking a
>>> program and running it unchanged on another system on another platform
>>> appears alien and not desirable to a significant part of the Forth
>>> community.
>>
>> What's a portable *program*? If it's a program of any significance, it
>> has a user interface that's dependent on an underlying host OS and the
>> language interface to it. Or it's an application that supports special
>> hardware, proprietary protocols, etc. Tell me about all the whole C
>> programs that run on any platform with any C compiler without change.
>
> Yes, but let's consider reality. If I write a C program for Linux, I have
> two free compilers to use, and two proprietary: clang or gcc for the free
> ones, Portland or Intel's C compiler for proprietary. They are pretty
> interchangeable,

I presume you mean GUN/Linux, not some other OS running on a Linux
kernel (they do exist). That is is a very constrained and well-
defined environment, so this hypothetical C program is nobody's idea
of a portable program. It's portable only in the sense that it's not
written in assembly language.

Andrew.

Bernd Paysan

unread,

Mar 9, 2013, 3:41:17 PM3/9/13

to

Andrew Haley wrote:

> Bernd Paysan <bernd....@gmx.de> wrote:
>> Yes, but let's consider reality. If I write a C program for Linux, I
>> have two free compilers to use, and two proprietary: clang or gcc for the
>> free
>> ones, Portland or Intel's C compiler for proprietary. They are pretty
>> interchangeable,
>
> I presume you mean GUN/Linux, not some other OS running on a Linux
> kernel (they do exist).

And some are quite popular, like Android. The first port of Gforth to
Android was using the AndroidTerm as environment, and on that layer, there
is very little difference to a GNU/Linux. The libc lacks a few syscall
bindings, like ppoll. There's now even an Xserver app (beta) if you want
graphics ;-). Libraries have no version numbers, which means you have to
change four lines in libtool to make it work on Android.

Let's face it: To port Gforth to Android was a piece of cake - as text
console Forth. The hard work was to let Gforth actually *use* the features
the Android userland provides, and allow it to be installed as app. That's
because Android's userland is about as vendor-lock-in as Window's. Maybe
even more, because you are supposed to write your app in Java. In a
particular Java dialect, where many things are done quite differently...

> That is is a very constrained and well-
> defined environment, so this hypothetical C program is nobody's idea
> of a portable program. It's portable only in the sense that it's not
> written in assembly language.

If you go further, and say, you write a program for POSIX, you still have a
very similar situation. You might need tools like autoconf, since C is not
an interactive environment, and therefore adapting the source to what's
available doesn't work out of the box, but in Forth, something like autoconf
isn't necessary. If you want to know if a word is available, use [defined],
if you want to know whether it behaves to a particular spec, test it.

Popular options have what we call "Normative Kraft des Faktischen" (Kant,
translated as "normative power/force of the factual") in Germany, and gcc's
extensions proliferated, because it is/was the de-facto standard compiler
for many unixoid systems, clearly beyond GNU/Linux. This sort of things
make programs more portable, because the other vendors simply give up
insisting on the quirks in their system.

Paul Rubin

unread,

Mar 10, 2013, 4:51:33 AM3/10/13

to

Howerd <how...@yahoo.co.uk> writes:
>> >> [Andrew:] approach at solving this problems was cmForth with

>> >> different vocabularies for compilation and interpretation, and it

>> >> again is too simple....
> Assuming cmForth is like colorForth in his respect, it easy to know
> which 'X' you are using - it depends on whether you are in "macro" or
> "forth" wordlist.

I'm getting a maybe-wrong, halfway understanding of something from this,
namely that the separate vocabularies seems to be required to make
cmforth's beautifully simple and diabolically clever 2-line metacompiler
work. For someone like Chuck, I could imagine it being worth changing
almost everything else in Forth to be able to build like that. It might
even go some way toward explaining his rage against ANS Forth. A pure,
self-reproducing Forth with essentially no infrastructure for that
purpose is too perfect to let go. Anything like Jonesforth of course is
a near-abomination from that perspective. This makes me want to look
into Colorforth more closely. Wow! :-)

Andrew Haley

unread,

Mar 10, 2013, 5:35:21 AM3/10/13

to

Paul Rubin <no.e...@nospam.invalid> wrote:
> Howerd <how...@yahoo.co.uk> writes:
>>> >> [Andrew:] approach at solving this problems was cmForth with
>>> >> different vocabularies for compilation and interpretation, and it
>>> >> again is too simple....
>> Assuming cmForth is like colorForth in his respect, it easy to know
>> which 'X' you are using - it depends on whether you are in "macro" or
>> "forth" wordlist.
>
> I'm getting a maybe-wrong, halfway understanding of something from
> this, namely that the separate vocabularies seems to be required to
> make cmforth's beautifully simple and diabolically clever 2-line
> metacompiler work.

I don't think so: there's no reason a dual-xt-per-word scheme wouldn't
work just as well.

Which two lines, anyway? The compiler is much longer than that.

Andrew.

Howerd

unread,

Mar 10, 2013, 6:03:10 AM3/10/13

to

On Sunday, March 10, 2013 9:51:33 AM UTC+1, Paul Rubin wrote:

Hi Paul,

Yes, "wow" indeed :-)

My understanding of the way Chuck uses the macro wordlist is that you need somewhere to put words that you are building that create other words, so that the "macro" words do not get mixed up with the words being built by the "macros".

I hope someone will correct me if I am wrong.

Jonesforth is a noble attempt at creating a Forth, but some of the design decisions, aparently, were not in keeping with the latest Forth thinking.

Again, my understanding ( which is neccessarily limited ) is that Chuck always creates a "Problem Oriented Language". The original Forth, MicroForth, and polyForth suit the problem domain of creating a programming environment that has a serial terminal and text display, wheras colorForth suits a graphical environment that has no text stream as such.
Stepping one level higher, I think that Chuck advocates *always* solving the exact problem, rather than creating a generic tool to solve potential problems.
I have found this to be the best way to achieve results too.

Seen in this light, JonesForth solves the problem of "how do I write my own Forth that follows the Fig83 ( or whatever ) standard" - this has little to do with writing applications in Forth.

One of the other extremely clever things about colorForth is the way (almost) everything is a 32 bit "token". The token "colour" ( it doesn't have to be an actual colour ) indicates what different parts of the system should do with it. For example a "red" token is displayed in red by the editor, and acts like : in the compiler. This combined with the fact that each token has its own compressed name allows some interesting simplifications.

BTW GreenArrays' version of colorForth, arrayForth, has removed the continuous updating of the display by a background task - colorForth used this to display chip designs and simulations changing in "real time", wheras arrayForth just has to compile code - different problems, so different langauges.

> A pure, self-reproducing Forth with essentially no infrastructure
> for that purpose is too perfect to let go

AFAIK colorForth is not self-reproducing - Jeff Fox had some code to create a boot sector which could be extended, but the sources are in NASM or similar assembler. arrayForth as shipped by GreenArrays does not reproduce itself - I would interested to hear if GA have code to do this...

Best regards,
Howerd

Paul Rubin

unread,

Mar 10, 2013, 11:56:46 PM3/10/13

to

Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>> make cmforth's beautifully simple and diabolically clever 2-line
>> metacompiler

> I don't think so: there's no reason a dual-xt-per-word scheme wouldn't
> work just as well.

I'm not sure I see how to do this, but there's already some parts
of the picture that I'm missing.

> Which two lines, anyway? The compiler is much longer than that.

The { and } words that switch the metacompiler and target dictionary
pointers. The idea is you start with a fully loaded interpreter, then
set up the pointers so the compiled dictionary is an empty area of
memory, and then you can use all the interpreter features (including an
assembler or whatever) to make a new, possibly modified copy of the
whole system in the empty memory. All that without needing special
versions of CREATE, COMMA (,), etc.

There's a cool article called something like "The Demise of the
Metacompiler in Cmforth" by Jay Melvin, that describes it. I've only
briefly looked at cmforth itself and didn't find it very comprehensible,
unfortunately.

Andrew Haley

unread,

Mar 11, 2013, 4:47:05 AM3/11/13

to

Paul Rubin <no.e...@nospam.invalid> wrote:
> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>> make cmforth's beautifully simple and diabolically clever 2-line
>>> metacompiler
>> I don't think so: there's no reason a dual-xt-per-word scheme wouldn't
>> work just as well.
>
> I'm not sure I see how to do this, but there's already some parts
> of the picture that I'm missing.
>
>> Which two lines, anyway? The compiler is much longer than that.
>
> The { and } words that switch the metacompiler and target dictionary
> pointers. The idea is you start with a fully loaded interpreter,
> then set up the pointers so the compiled dictionary is an empty area
> of memory, and then you can use all the interpreter features
> (including an assembler or whatever) to make a new, possibly
> modified copy of the whole system in the empty memory. All that
> without needing special versions of CREATE, COMMA (,), etc.

Sure, but people were doing very similar thing before cmFORTH. I can
remember fig-FORTH being bootstrapped that way. But I don't see how
that actually requires separate wordlists for COMPILER words.

> There's a cool article called something like "The Demise of the
> Metacompiler in Cmforth" by Jay Melvin, that describes it. I've only
> briefly looked at cmforth itself and didn't find it very comprehensible,
> unfortunately.

cmFORTH isn't so bad. It's much easier to read with shadow blocks.

Andrew.

Mark Wills

unread,

Mar 11, 2013, 4:51:58 AM3/11/13

to

On Mar 11, 3:56 am, Paul Rubin <no.em...@nospam.invalid> wrote:

Also, this might be useful/interesting:

http://www.ultratechnology.com/mmeta.html

Paul Rubin

unread,

Mar 14, 2013, 2:48:11 AM3/14/13

to

>> The { and } words that switch the metacompiler and target dictionary

>> pointers...

>> without needing special versions of CREATE, COMMA (,), etc.
> Sure, but people were doing very similar thing before cmFORTH. I can
> remember fig-FORTH being bootstrapped that way.

The Figforths that I've seen had the primitives defined in assembler
(maybe MASM) but that code may have been produced somehow from
metacompiler output. I remember Jeff Fox complaining about that. But,
I thought that the usual approach to metacompilation was defining
special versions of words like COMMA to operate on the target space. I
did look briefly at the Eforth metacompiler and it's also rather big and
complicated. The Cmforth approach as I understand it starts with the
compilation dictionary completely empty, so you can populate it using
normal interpreter commands.

> But I don't see how that actually requires separate wordlists for
> COMPILER words.

I wonder if I misunderstand the terminology "COMPILER word". I just
mean when you say

: SQUARE DUP * ;

the word DUP gets looked up in a wordlist, that is not necessarily the
same wordlist that would be used if you typed DUP into the interpreter.

> cmFORTH isn't so bad. It's much easier to read with shadow blocks.

Any idea where a copy of that is available? I saw a mention that such a
thing existed, but the version I saw didn't have any comments to speak
of.

Elizabeth D. Rather

unread,

Mar 14, 2013, 3:01:53 AM3/14/13

to

On 3/13/13 8:48 PM, Paul Rubin wrote:
>>> The { and } words that switch the metacompiler and target dictionary
>>> pointers...
>>> without needing special versions of CREATE, COMMA (,), etc.
>> Sure, but people were doing very similar thing before cmFORTH. I can
>> remember fig-FORTH being bootstrapped that way.
>
> The Figforths that I've seen had the primitives defined in assembler
> (maybe MASM) but that code may have been produced somehow from
> metacompiler output. I remember Jeff Fox complaining about that. But,
> I thought that the usual approach to metacompilation was defining
> special versions of words like COMMA to operate on the target space. I
> did look briefly at the Eforth metacompiler and it's also rather big and
> complicated. The Cmforth approach as I understand it starts with the
> compilation dictionary completely empty, so you can populate it using
> normal interpreter commands.

That's characteristic of all Forth meta-compilers. I haven't read up on
cmForth, but from what folks are saying here it sounds as though it's
just swapping a bunch of pointers.

The current proposed cross-compiler strategy (which includes
meta-compilers) is to have different words building the target
dictionary, but one could do it by swapping a bunch of pointers *if* the
target is basically the same (e.g., cell-size, etc.) as the host. If
it's not, more intervention is necessary.

>> But I don't see how that actually requires separate wordlists for
>> COMPILER words.
>
> I wonder if I misunderstand the terminology "COMPILER word". I just
> mean when you say
>
> : SQUARE DUP * ;
>
> the word DUP gets looked up in a wordlist, that is not necessarily the
> same wordlist that would be used if you typed DUP into the interpreter.

In current terminology (per the proposed Cross Compiler wordset)
COMPILER refers to words that are only executed during compilation, such
as IF etc. TARGET is the term for a version of SQUARE that goes in the
target dictionary.

>> cmFORTH isn't so bad. It's much easier to read with shadow blocks.
>
> Any idea where a copy of that is available? I saw a mention that such a
> thing existed, but the version I saw didn't have any comments to speak
> of.

Paul Rubin

unread,

Mar 14, 2013, 3:23:29 AM3/14/13

to

> That's characteristic of all Forth meta-compilers. I haven't read up
> on cmForth, but from what folks are saying here it sounds as though
> it's just swapping a bunch of pointers.

It swaps just one pair of pointers, and one pair of offsets. I.e. two
cells get swapped with another two cells, in a one-liner. The "unswap"
operation is of course the same as the swap operation (it just swaps the
same things again).

> *if* the target is basically the same (e.g., cell-size, etc.) as the
> host. If it's not, more intervention is necessary.

Not an issue for cmforth which is not a cross-compiler afaik. It only
metacompiles itself.

I wonder why eforth's metacompiler is so messy. The interpreter itself
is (mostly) beautiful.

Lars Brinkhoff

unread,

Mar 14, 2013, 4:18:57 AM3/14/13

to

Elizabeth D. Rather wrote:
> I haven't read up on cmForth, but from what folks are saying here it
> sounds as though it's just swapping a bunch of pointers.

From "Demise of the metacompiler in cmForth":

: { dA @ HERE H' 2@ H ! dA ! H' 2! ;
: } { ;

So it loos like HERE and dA (I don't know what that is) is being swapped.

Andrew Haley

unread,

Mar 14, 2013, 4:34:02 AM3/14/13

to

Paul Rubin <no.e...@nospam.invalid> wrote:
>>> The { and } words that switch the metacompiler and target dictionary
>>> pointers...
>>> without needing special versions of CREATE, COMMA (,), etc.
>>
>> Sure, but people were doing very similar thing before cmFORTH. I can
>> remember fig-FORTH being bootstrapped that way.
>
> The Figforths that I've seen had the primitives defined in assembler
> (maybe MASM) but that code may have been produced somehow from
> metacompiler output.

It was originally produced by a Forth, Inc cross-compiler, but you
could recompile the sources without cross-compiling.

> I remember Jeff Fox complaining about that. But, I thought that the
> usual approach to metacompilation was defining special versions of
> words like COMMA to operate on the target space.

It was, yes.

> I did look briefly at the Eforth metacompiler and it's also rather
> big and complicated. The Cmforth approach as I understand it starts
> with the compilation dictionary completely empty, so you can
> populate it using normal interpreter commands.

That's right.

Andrew.

Albert van der Horst

unread,

Mar 14, 2013, 7:25:37 AM3/14/13

to

In article <7x1ubij...@ruckus.brouhaha.com>,

Paul Rubin <no.e...@nospam.invalid> wrote:
>>> The { and } words that switch the metacompiler and target dictionary
>>> pointers...
>>> without needing special versions of CREATE, COMMA (,), etc.
>> Sure, but people were doing very similar thing before cmFORTH. I can
>> remember fig-FORTH being bootstrapped that way.
>
>The Figforths that I've seen had the primitives defined in assembler
>(maybe MASM) but that code may have been produced somehow from
>metacompiler output. I remember Jeff Fox complaining about that. But,
>I thought that the usual approach to metacompilation was defining
>special versions of words like COMMA to operate on the target space. I
>did look briefly at the Eforth metacompiler and it's also rather big and
>complicated. The Cmforth approach as I understand it starts with the
>compilation dictionary completely empty, so you can populate it using
>normal interpreter commands.

What is metacompilaton? I have the source of lina 4.0.5 disassembled
with my ciasdis tool, then reassemble it with a Forth assembler
in the same tool. Not metacompilation as I understand it, but the
only tool to generate a new lina is lina itself.

Groetjes Albert
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

Lars Brinkhoff

unread,

Nov 21, 2013, 8:26:49 AM11/21/13

to

Elizabeth D. Rather wrote:

> Paul Rubin wrote:
> > Eforth metacompiler and it's also rather big and complicated. The
> > Cmforth approach as I understand it starts with the compilation
> > dictionary completely empty, so you can populate it using normal
> > interpreter commands.
> That's characteristic of all Forth meta-compilers. I haven't read up
> on cmForth, but from what folks are saying here it sounds as though
> it's just swapping a bunch of pointers.

Forth Dimensions volume 12 issue 6 has an article which describes the
Pygmy Forth metacompiler, which seems similar to the one in cmFORTH.

> > Any idea where a copy of that is available? I saw a mention that
> > such a thing existed, but the version I saw didn't have any
> > comments to speak of.

The pygmy17.zip file at http://pygmy.utoh.org/pygmyforth.html
seems to have a copy of cmFORTH. And also lots of other goodies.