RfD: !ME and @ME

65 views
Skip to first unread message

MarkWills

unread,
Mar 23, 2010, 11:07:36 AM3/23/10
to
Dear members of the Forth community,

This is my first RfD. I thought I would give it a try. Apologies if
this idea is old-hat or already provisioned in a different form.

Respectfully,

Mark Wills

---------------------------------------------------

RfD: Proposal for New Forth Words !ME and @ME
=============================================

Revision History
================
23rd March 2010 – First Draft

Change Log
==========


Rationale
=========

Problem
-------
A colon definition that needs to maintain state between calls, or
save pertinent associated data cannot do so without the use of
VARIABLEs or VALUEs, which are external to the colon definition.
A better approach is to store related data with the word itself,
promoting encapsulation at the word level.

This proposal presents an alternative data storage solution whereby
the data stored is directly associated with the colon definition
itself, and not reliant on an external VARIABLE or VALUE.

Current Practice
----------------
Current practice is to save stateful information in a variable.
Consider an example whereby in a multi-tasking environment, access
to an IO device shall only be granted to the current task if it is
not already in use by another task:

VARIABLE SerialOwnerID 0 SerialOwnerID !

: CanClaimSerialPort? ( taskid|0 – flag )
?DUP 0= IF 0 SerialOwnerID ! ResetSerialResources EXIT THEN
SerialOwnerID @ 2DUP 0= -ROT = OR
IF
DUP SerialOwnerID ! AllocateSerialResources TRUE ( granted )
ELSE
DROP FALSE ( denied )
THEN
;

Here, a variable is used to hold the ID of the task currently using
the serial port. A task wishing to claim the serial port pushes its
task ID onto the stack, and calls CanClaimSerialPort? If the
SerialOwnerID is 0 (not in use) or the same as the requesting task’s
ID, the request is granted by pushing TRUE. If any other task
requests the serial port, the request is denied by pushing FALSE.
The serial port can be freed by calling CanClaimSerialPort? with 0
on the stack.

Tyical Use
----------
The above code can be simplified, made more secure, and the
dependence on an external variable removed by the use of the
proposed words !ME and @ME pronounced 'store in me' and 'fetch from
me' respectively.

:!@ CanClaimSerialPort? ( taskid|0 – flag )
?DUP 0= IF 0 !ME ResetSerialResources EXIT THEN
@ME 2DUP 0= -ROT = OR
IF
DUP !ME AllocateSerialResources TRUE ( granted )
ELSE
DROP FALSE ( denied )
THEN CR
;

Here, the data is not stored in a variable. Rather, it is stored
'within' the word itself (in practice, the mechanics of how and
where the data is stored and retrieved is implementation specific,
but the Forth programmer can think of/visualise the data as being
stored 'inside' the word). Not only is the dependence on an external
variable removed, but the data can be considered 'protected' since,
unlike variables and values, which are global in scope, the data can
only be accessed by the word that 'contains' it.

It is proposed that any colon definition can use !ME and @ME. Each
colon definition can have exactly one cell sized 'me' storage area.

Remarks
-------
Note that in the above example, the word is built not with : but with
a modified : called :!@ (pronounced "build with storage").
This word is responsible for:

* building the dictionary entry in the normal way, as per :
* reserving a cell of memory;
* initialising the reserved cell to zero;
* making the address of the reserved cell available to the colon
definition in an implementation specific manner.

Alternative methods are of course available, and indeed welcomed for
discussion. It would probably be preferable if :!@ were not required.
One possible solution would be to place references to !ME and @ME on
the parameter stack or the return stack as they occur, and have ;
back-fill the references, in a similar way to IF...THEN and DO...LOOP
etc. This author welcomes and encourages discussion on this point.

Typical Use
-----------
Use of the above described facility would be desirable for:

* Generating unique IDs;
* Reference counting;
* Holding object instance references;
* Promoting secure/atomic data isolation/security.

Proposal
--------
13.6.1.xxxx :!@ LOCAL

Interpretation:
Interpretation semantics for this word are undefined.

Compilation:
Compilation semantics for this word are undefined.

Run-time: ( C: "(spaces)name" -- colon-sys )
Create a definition for name as per 6.1.0450, reserve a cell as
storage space, initialising the cell’s value to zero. Make the
address of the reserved cell available to the definition being
constructed in an implementation specific manner.

13.6.1.xxxx @ME LOCAL

Interpretation:
Interpretation semantics for this word are undefined.

Compilation:
Compilation semantics for this word are undefined.

Run-time: ( -- x )
Fetch the contents of the cell allocated by :!@ and place on
the parameter stack.

13.6.1.xxxx !ME LOCAL

Interpretation:
Interpretation semantics for this word are undefined.

Compilation:
Compilation semantics for this word are undefined.

Run-time: ( x -- )
Take the contents of the topmost stack entry and store in the
cell allocated by :!@

Reference Implementation:
-------------------------
The following reference implementation is written in a Forth-83
style, since this author’s knowledge is currently restricted to
Forth-83 type systems.

In the interests of simplicity for example purposes, the
implementation below assumes an ITC system and stores the data
in line with the word, coding a BRANCH instruction to jump over the
data. This example probably constitutes the simplest possible
solution on an ITC system (though rather in-efficient) and can be
coded purely in high level Forth. ANS Forth examples are welcomed.
With carnal knowledge, implementers can of course use much more
sophisticated solutions, reducing size and increasing efficiency.

:!@ FRED 1 2 @ME ;

Is compiled as:

+--------->--------+ +----@ME----+
| | | |
+------+--------+---+------+-----+---+-----+---+-----+-----+---+
| FRED | BRANCH | 4 | DATA | LIT | 1 | LIT | 2 | LIT | xxx | @ |
+------+--------+---+------+-----+---+-----+---+-----+-----+----
^ |
+----------------<---------------+

It can be seen that @ME has in fact been coded as LIT <address> @
where <address> points to the data cell reserved by :!@

High level code to implement the proposed behaviour is presented
thus:

: :!@ : COMPILE BRANCH 4 , 0 , ;
: ?ME LATEST @ >CFA 6 + [COMPILE] LITERAL ;
: @ME ?ME COMPILE @ ; IMMEDIATE
: !ME ?ME COMPILE ! ; IMMEDIATE

Since the above code is not ANS compliant, a discussion of the
code is presented here:

:!@
This word uses : to compile the dictionary entry, and then compiles
a BRANCH <offset> 0 sequence. The idea is that the 0 represents the
reserved storage cell, and the BRANCH simply branches over it.
Note that the requirement for :!@ could be removed by simply
re-defining : thus:
: : : COMPILE BRANCH 4 , 0 , ;
However, this would mean that a storage cell would be reserved for
every colon definition, regardless of whether !ME or @ME was used
or not. This could be un-desirable on memory constrained systems.


?ME
This word calculates the address of the reserved cell by invoking
LATEST to get the dictionary address of the dictionary header just
created by :!@ , converting to a CFA and adding (in this case, since
the above code was written on a legacy 16 bit machine) 6 to account
for the BRANCH <address> sequence. It then compiles this address
into the current colon definition as a literal. Since address
calculation and compilation is required by both !ME and @ME it is
factored out to become ?ME ('address of me').

@ME
Invokes ?ME to compile the address of the reserved cell into the
current definition, and then compiles @ into the definition. Note
this is an immediate word.

!ME
Invokes ?ME to compile the address of the reserved cell into the
current definition, and then compiles ! into the definition. Note
this is an immediate word.


Testing
-------

Test of the storage and retrieval of ME data:

:!@ TestME ( Storing: x true -- | Fetching: false -- x )
IF
ME! ." Stored " ME@ . ." in me."
ELSE
." My value is " ME@ .
THEN
CR
;

Test storage with:
99 TRUE TestME
Output:
Stored 99 in me
ok

Test retrieval with:
FALSE TestME
Output:
My value is 99
ok

Example UID generator:
:!@ GetUID ( -- uid )
@ME 1+ DUP !ME ;

Test with:
GetUID GetUID GetUID GetUID .S
Output:
1 2 3 4 ok

Example Reference Counter
:!@ RefCount ( true|false –- count+1|count-1 )
@ME SWAP IF 1+ ELSE 1- THEN DUP !ME ;
Test with:
TRUE RefCount TRUE RefCount TRUE RefCount
FALSE RefCount FALSE RefCount FALSE RefCount .S
Output:
1 2 3 2 1 0 ok

Author
------
Mark Wills
Overseas Consultants LTD
Parliament Street
Ramsey
Isle of Man
Email: MarkRob...@yahoo.co.uk
Website: http://www.OverseasConsultants.co.uk

[eof]

ar

unread,
Mar 23, 2010, 11:31:44 AM3/23/10
to
Hello

Sorry, I must be very obtuse but IMHO there is no problem in keeping
state in VALUES, VARIABLES or USER variables, as long as you have a
"module" mechanism which discourages unwanted access and use only the
exported interface for the module. VFX's Source Code Module is example
of this. Taking this to the extreme we have classes and object
oriented programmming.

Best regards
Rafael


> Output:
>         1 2 3 2 1 0 ok
>
> Author
> ------
> Mark Wills
> Overseas Consultants LTD
> Parliament Street
> Ramsey
> Isle of Man

> Email: MarkRobertWi...@yahoo.co.uk
> Website:http://www.OverseasConsultants.co.uk
>
> [eof]

Bernd Paysan

unread,
Mar 23, 2010, 11:43:27 AM3/23/10
to
MarkWills wrote:

> A colon definition that needs to maintain state between calls, or
> save pertinent associated data cannot do so without the use of
> VARIABLEs or VALUEs, which are external to the colon definition.

I'm sorry, but this is completely non-conclusive. *Of course* a colon
definition can store permanent state in a variable or value. This is
common usage, as you state yourself. The only "advantage" I can see
from your proposal is that it makes debugging of these words harder,
since you can't easily inspect their hidden state.

Alternative suggestion for combining storage and code just once:
Interpretative DOES> (combining different storage with the same code is
trivial: just normal DOES> is perfect ;-). Example:

Create CanClaimSerialPort? 0 ,
DOES> >r
?dup 0= IF 0 r> ! ResetSerialResources EXIT THEN
r@ @ 2dup 0= -rot = or IF
dup r@ ! AllocateSerialResources true
ELSE
drop false
THEN r> drop ;

Gforth and bigForth support interactive DOES>. So there is at least
some more common practice, and it's probably more general and more
useful than your suggestion.

--
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://www.jwdt.com/~paysan/

alberto pasquale

unread,
Mar 23, 2010, 1:46:04 PM3/23/10
to


On the system I use, two or more diferent tasks call the same words.
Specifically in your serial port sample, I have diferent tasks
invoking the same protocol driver with pointers to specific hardware
stored in user variables (each task has it's own user variables).
If the word is storing data in some private storage, how do you
propose to make that word re-entrant in a multitasker system ?

Alberto

Hugh Aguilar

unread,
Mar 23, 2010, 3:27:16 PM3/23/10
to
On Mar 23, 11:46 am, alberto pasquale <albe...@hal-pc.org> wrote:
> On the system I use, two or more diferent tasks call the same words.
> Specifically in your serial port sample, I have diferent tasks
> invoking the same protocol driver with pointers to specific hardware
> stored in user variables (each task has it's own user variables).
> If the word is storing data in some private storage, how do you
> propose to make that word re-entrant in a multitasker system ?
>
> Alberto

I think that Mark is trying to get something like ICON-style
generators. These aren't going to be reentrant. They actually get
reentered over and over again, but they take up where they left off;
the invocations aren't local to themselves. Reentrancy in a
multitasking system is only an issue in regard to a micro-controller.
By comparison, nobody is going to use generators in a micro-controller
--- generators are very much desktop software that does time-consuming
recursive-descent searches --- I don't see any conflict, as these are
completely different kinds of programming.

Realistically, Forth-200x is never going to support anything
innovative --- including generators. I am putting together a competing
standard that is intended to be innovative. Go to:
http://www.forthwiki.com/tiki-index.php and visit the Fixed-Forth
forum to discuss this further.

Andrew Haley

unread,
Mar 23, 2010, 4:19:48 PM3/23/10
to
Hugh Aguilar <hughag...@yahoo.com> wrote:

> Reentrancy in a multitasking system is only an issue in regard to a
> micro-controller.

Pardon?

Andrew.

Jerry Avins

unread,
Mar 23, 2010, 4:52:39 PM3/23/10
to

Discount that nonsense.

Jerry
--
Discovery consists of seeing what everybody has seen, and thinking what
nobody has thought. .. Albert Szent-Gyorgi
ŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻ

Hugh Aguilar

unread,
Mar 23, 2010, 6:56:06 PM3/23/10
to
On Mar 23, 2:19 pm, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:

> Hugh Aguilar <hughaguila...@yahoo.com> wrote:
> > Reentrancy in a multitasking system is only an issue in regard to a
> > micro-controller.
>
> Pardon?
>
> Andrew.

On desktop computers you have an OS that does your multitasking for
you --- you don't have to worry about reentrancy issues because your
tasks aren't sharing any code. On a micro-controller your Forth system
is your OS; your tasks are sharing code, and that code has to be
reentrant.

The Beez'

unread,
Mar 23, 2010, 6:56:09 PM3/23/10
to
On 23 mrt, 21:52, Jerry Avins <j...@ieee.org> wrote:

> Andrew Haley wrote:
> > Hugh Aguilar <hughaguila...@yahoo.com> wrote:
IMHO this is nothing more than a static variable.
- The first question is: why only one? Why not several?
- The second question is: a static variable is nothing but a global
one with its scope limited to one single routine (sorry, I don't feel
like doing a semantic discussion).

Consequently, the BIG question is: is this the right proposal. We
don't have many words standardized for limiting the scope of words or
data allocation. Is:

:private myword
lots of other words
;

Or even more wild:

variable myvar private

: mywords
static mystatic
lots of other words
; private

Not much more useful and generic?

Hans Bezemer

Hugh Aguilar

unread,
Mar 23, 2010, 7:16:17 PM3/23/10
to
On Mar 23, 4:56 pm, "The Beez'" <hans...@bigfoot.com> wrote:
> IMHO this is nothing more than a static variable.
> - The first question is: why only one? Why not several?
> - The second question is: a static variable is nothing but a global
> one with its scope limited to one single routine (sorry, I don't feel
> like doing a semantic discussion).

I agree that Mark's proposal is pretty much just syntactic sugar for
what could be done with a global variable. It is a step in the right
direction though, so I wouldn't put it down.

In order to implement generators, we need to be able to save
information that is not readily available. The problem is not *where*
we stash the data, but *how* we obtain the data. Specifically, we have
the data on the return stack such as DO LOOP information and a chain
of return addresses from a recursive descent. We also have the xt of
the address in the middle of the function where we will want to
reanimate the function at, which is what my XT function was supposed
to grab.

In regard to the return stack data, I would recommend that when this
generator is called, it is given its own return stack. The entire
return stack can then be saved somewhere (the heap). This way we grab
everything in one fell swoop.

MarkWills

unread,
Mar 24, 2010, 6:17:09 AM3/24/10
to
On 23 Mar, 22:56, "The Beez'" <hans...@bigfoot.com> wrote:
> On 23 mrt, 21:52, Jerry Avins <j...@ieee.org> wrote:> Andrew Haley wrote:
> > > Hugh Aguilar <hughaguila...@yahoo.com> wrote:
>
> IMHO this is nothing more than a static variable.
> - The first question is: why only one? Why not several?

It's a good question, and like I said in the proposal, I'm happy to
receive comments and questions/debate. Totally open to the idea being
improved upon.

> - The second question is: a static variable is nothing but a global
> one with its scope limited to one single routine (sorry, I don't feel
> like doing a semantic discussion).
>

Er, that's not a question. It's a statement ;-) (sorry, could not
resist!)


> Consequently, the BIG question is: is this the right proposal. We
> don't have many words standardized for limiting the scope of words or
> data allocation. Is:
>
> :private myword
>   lots of other words
> ;
>
> Or even more wild:
>
> variable myvar private
>
> : mywords
>   static mystatic
>   lots of other words
> ; private
>
> Not much more useful and generic?
>
> Hans Bezemer

It looks good to me. Why has something of this nature not been
proposed before?

It has been suggested that 'radical' (my choice of word) proposals do
not belong in the 200x, or to put it another way, radical, or
experimental ideas do not belong in the standard. I *can* see the
reasoning behind that, but at the same time, if that's the case, why
not just freeze the standard?

I'm sure Forth can benefit from borrowing ideas from other languages,
such as atomicity, OOP etc. I think it is a shame that currently there
seems to be some resistance to 'modernisation'. Again, I can see the
reasoning - the standards people have a responsibility to make sure
they don't introduce things that break existing code, for sure, no
argument. But is it not possible to introduce new ideas without
breaking existing systems? I would have thought so...

Perhaps it's just me... Perhaps I jumped the gun a bit. Perhaps an
idea needs to be proven in the field before it should be considered
for standardisation. It will either stand the test of time or it
won't. IIRC some paradigms that have been in use in VFX for many years
are only being proposed just now.

MarkWills

unread,
Mar 24, 2010, 6:31:32 AM3/24/10
to
On 23 Mar, 23:16, Hugh Aguilar <hughaguila...@yahoo.com> wrote:
> On Mar 23, 4:56 pm, "The Beez'" <hans...@bigfoot.com> wrote:
>
> > IMHO this is nothing more than a static variable.

It's a static local variable. People seemed to have missed that my
proposal proposed adding the words to the LOCAL word set.

> I agree that Mark's proposal is pretty much just syntactic sugar for
> what could be done with a global variable. It is a step in the right
> direction though, so I wouldn't put it down.
>

I think it's a bit more than that ;-)

Sure, anything can be done with a global:

VARIABLE Count
: UpDownCounter ( true|false -- +1|-1 )
Count @ SWAP IF 1+ ELSE 1- THEN DUP Count ! ;

Can be written as:
: UpDownCounter ( true|false -- +1|-1 )


@ME SWAP IF 1+ ELSE 1- THEN DUP !ME ;

No variable.

It permits and promotes 'safety' - it's kind of like a mini object
with a single (static) instance variable. Well, a "static local" is
the best description I can come up with. They differ from LOCALS in
that LOCALS have to be carried to the word in question via the stack.
Thus to persist LOCALS, they have to be stored somewhere else, or
carried on the stack. This proposal doesn't require use of the stack.
It's just a different flavour of a local.

I think for multi-tasking programming it could come in handy. For
object reference counting it is useful. Or, if you just want to make
sure that a value can ONLY be altered in ONE place, and one place only
(critical sections) it is also useful. I think in embedded systems
inparticular.

Anyway, it's sounding like the idea is too limited in it's current
form. Trouble is, if I expand the idea, it would be seen as even more
radical! What to do? ;-)

Mark

Albert van der Horst

unread,
Mar 24, 2010, 7:10:23 AM3/24/10
to
In article <108dcfb1-efb3-4e11...@q23g2000yqd.googlegroups.com>,

Or even the Python convention
_pad \ This starts with an underscore. Private!
\ Don't use outside of the word set 1]

A naming convention doesn't stay in the way of debugging.

>
>Hans Bezemer

1] In the Brody sense.

--
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

Andrew Haley

unread,
Mar 24, 2010, 7:20:57 AM3/24/10
to
MarkWills <markrob...@yahoo.co.uk> wrote:
>
> I think it's a bit more than that ;-)
>
> Sure, anything can be done with a global:
>
> VARIABLE Count
> : UpDownCounter ( true|false -- +1|-1 )
> Count @ SWAP IF 1+ ELSE 1- THEN DUP Count ! ;
>
> Can be written as:
> : UpDownCounter ( true|false -- +1|-1 )
> @ME SWAP IF 1+ ELSE 1- THEN DUP !ME ;
>
> No variable.
>
> It permits and promotes 'safety' - it's kind of like a mini object
> with a single (static) instance variable. Well, a "static local" is
> the best description I can come up with.

Sure, I think we all get that. The thing I'm having trouble with is
what advantage is gained from this 'safety'.

> They differ from LOCALS in that LOCALS have to be carried to the
> word in question via the stack. Thus to persist LOCALS, they have
> to be stored somewhere else, or carried on the stack. This proposal
> doesn't require use of the stack. It's just a different flavour of
> a local.
>
> I think for multi-tasking programming it could come in handy. For
> object reference counting it is useful. Or, if you just want to make
> sure that a value can ONLY be altered in ONE place, and one place only
> (critical sections) it is also useful. I think in embedded systems
> inparticular.
>
> Anyway, it's sounding like the idea is too limited in it's current
> form. Trouble is, if I expand the idea, it would be seen as even more
> radical! What to do? ;-)

The problem for me with this proposal is that it's not something that
isn't really easy to do in Forth already. Like this:

variable 'local-storage
: local-storage local-storage ! ;
: loc 'local-storage @ postpone literal ; immediate

... later ...

here 0 , local-storage


: UpDownCounter ( true|false -- +1|-1 )

loc @ SWAP IF 1+ ELSE 1- THEN DUP loc ! ;

Andrew.

alberto pasquale

unread,
Mar 24, 2010, 1:53:32 PM3/24/10
to
> Anyway, it's sounding like the idea is too limited in it's current
> form. Trouble is, if I expand the idea, it would be seen as even more
> radical! What to do? ;-)
>
> Mark

Mark,
I think is valid to explore new techniques and this is a good forum to
get feed-back.
You can continue to expand the idea and use in your work if you find
it beneficial.
What I don’t understand is the reason to propose as part of the
standard.

Alberto

Elizabeth D Rather

unread,
Mar 24, 2010, 3:45:33 PM3/24/10
to
MarkWills wrote:
...

> It has been suggested that 'radical' (my choice of word) proposals do
> not belong in the 200x, or to put it another way, radical, or
> experimental ideas do not belong in the standard. I *can* see the
> reasoning behind that, but at the same time, if that's the case, why
> not just freeze the standard?
>
> I'm sure Forth can benefit from borrowing ideas from other languages,
> such as atomicity, OOP etc. I think it is a shame that currently there
> seems to be some resistance to 'modernisation'. Again, I can see the
> reasoning - the standards people have a responsibility to make sure
> they don't introduce things that break existing code, for sure, no
> argument. But is it not possible to introduce new ideas without
> breaking existing systems? I would have thought so...
>
> Perhaps it's just me... Perhaps I jumped the gun a bit. Perhaps an
> idea needs to be proven in the field before it should be considered
> for standardisation. It will either stand the test of time or it
> won't. IIRC some paradigms that have been in use in VFX for many years
> are only being proposed just now.

All very reasonable questions. All languages evolve to meet changes
needs, Forth included. The issue here is, how can this evolution best
progress? I'll respond from the perspective of seeing Forth evolve for
nearly 40 years, having participated in Forth79, Forth83, and ANS Forth
standards processes.

In my view, evolution is most productive as programmers using the
language for challenging applications develop new tools. A tool or
approach in one application may evolve as it's used in others.
Different programmers evolve different strategies for similar
challenges, or similar strategies for different challenges.

Forth has benefitted significantly in that its development and evolution
has largely taken place in the cauldron of actual, practical use in
application problem-solving, rather than as theoretical or academic
exercises. Many concepts from other languages have found their way into
Forth, as they have been found useful to tackle practical challenges.

Forth has often been thought of as a province of solitary practitioners.
This has been far from the case. Starting in the 1970's, there have
been mechanisms for Forth programmers to share ideas, including FIG
groups in various places, the Rochester Conferences, and (most
productive of all, IMO), group projects. FORTH, Inc., MPE, and many
user organizations have supported teams of Forth programmers, benefiting
from the sharing of ideas and approaches while at the same time
practicing the kind of disciplines necessary to make group projects
successful: naming conventions, coding standards, formatting and
documentation disciplines, etc.

A Standards effort (regardless of the language) is a process in which
the needs presented by many experienced practitioners, and their
solutions to them, are evaluated and the best solutions incorporated in
a new Standard. It is not the place for experimentation or new ideas:
that needs to happen in application development over time, so that the
solutions presented are those that have been proven in use. The most
successful are solutions that have been developed independently by
different practitioners, with the Standards process serving as the forum
for comparing these solutions.

A Standard is valuable if, and only if, it is widely accepted. For that
to take place, it has to offer genuine positive improvements in the
sense of solutions to widely acknowledged needs, and be relatively free
of changes that will be expensive or disruptive for users to adopt.

In 1982 (when Forth83 was developed), many members of the Standards
group took the position that compatibility with past practice was
unnecessary, because there were so few users and in the future there
would be many more. At that time, Forth was getting a lot of publicity,
as the PC movement got well under way. Unfortunately, the disruption
caused by its discontinuities (e.g. change to NOT and DO-loops) and
limitations (e.g. requirement of a 16-bit, ITC model) in Forth83 slowed
that process, as many developers elected to stay with past practices
(either Forth79 or their own solutions to problems). Nonetheless, it
was influential, and many implementors followed it as much as possible
while meeting their needs (e.g. for 32-bit cell sizes & host OS
compatibility).

The ANS Forth process worked very hard to maintain consistency with
Forth83, while adapting to changes in the industry (source and data in
host OS files, different cell sizes, allowing for more efficient
implementation strategies, etc.). That plus the official cachet of ANSI
and ISO resulted in very widespread acceptance of ANS Forth. Virtually
all the widely-used versions of Forth, both public-domain or open-source
and commercial, follow that standard. And the Forth200x effort has
continued the philosophy of incremental steps to address
widely-perceived issues with solutions that have been proven in use.

There is definitely a place for experimentation and innovation. Right
now, the Forth community lacks good forums for exchanging ideas.
EuroFORML is somewhat successful (though difficult for many American
Forthers to participate in), and comp.lang.forth is also helpful. A lot
of people have been taking a good look at colorForth, for example,
although it doesn't appear to have penetrated serious application
development outside Chuck's companies. "Bright ideas" are always
interesting and welcome for discussion and evaluation, but they do not
belong in a Standard until they have been more widely tested and
evaluated in application use.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

The Beez'

unread,
Mar 25, 2010, 3:17:12 AM3/25/10
to
On 24 mrt, 12:10, Albert van der Horst <alb...@spenarnc.xs4all.nl>
wrote:

> >Not much more useful and generic?
>
> Or even the Python convention
>  _pad   \ This starts with an underscore. Private!
>         \ Don't use outside of the word set 1]
>
> A naming convention doesn't stay in the way of debugging.

No, but it is so unforth-like. To me it is always the amount of coding
I have to do to make those things work. E.g. I already have this
facility by having the word "HIDE":

variable me

: me-proposal
me ! lots of other words me @
;

hide me

variable me

: me-demo
me ! lots of other words me @
;

hide me

Problem solved. No things to "remember", no tagging, no flagging, very
flexible (since you determine how far the scope goes). And Forth-like
IMHO. Lots of words and few externals? Who said it was perfect?

Hans Bezemer

Andrew Haley

unread,
Mar 25, 2010, 5:36:02 AM3/25/10
to
Albert van der Horst <alb...@spenarnc.xs4all.nl> wrote:

I can't see any use for "private", but the Python convention of using
_ as a prefix for local static data seems to be very sensible. Given
that, one could tweak the "Redefined" warning message that most Forths
have so that it doesn't warn for locals. This is a clean, simple, and
very Forth solution.

Andrew.

Bruce McFarling

unread,
Mar 25, 2010, 11:26:41 AM3/25/10
to
On Mar 23, 11:07 am, MarkWills <markrobertwi...@yahoo.co.uk> wrote:
> Problem
> -------
> A colon definition that needs to maintain state between calls, or
> save pertinent associated data cannot do so without the use of
> VARIABLEs or VALUEs, which are external to the colon definition.
> A better approach is to store related data with the word itself,
> promoting encapsulation at the word level.

I agree with the tendency of discussion ... there is no problem in the
portable use of Forth scripts between implementations being solved
here, and there is indeed no problem *to* solve here.

A single colon definition *should not be* an entire process in and of
itself, it should be an item in a collection of definitions that
provide the process.

*That's factoring*.

To control scope given that a single definition *should not be*
atomic, but also given some words represent internal details to the
process that need not be visible, all that is required is a private
wordlist for the items that do not need to be accessible outside the
process, and there is no need for variables to be outside of the scope
of the *process*. The existing search order words provide the tools
required to provide for this. For a permanently hidden collection of
words, one can make a nameless wordlist, so that when the search order
is reset, it is sealed, or for a closed but available collection of
words, one can store the named wordlist in a constant or a vocabulary
word.

I'm not an implementer, so couldn't vote that I would not implement
the proposal, but as a sometime code author, I would never use the
proposal. It would be bad practice.

The Beez'

unread,
Mar 25, 2010, 2:44:20 PM3/25/10
to
On 25 mrt, 10:36, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:

> I can't see any use for "private", but the Python convention of using
> _ as a prefix for local static data seems to be very sensible.  Given
> that, one could tweak the "Redefined" warning message that most Forths
> have so that it doesn't warn for locals.  This is a clean, simple, and
> very Forth solution.
Violates the "no expectations" rule.

Hans

Andrew Haley

unread,
Mar 25, 2010, 3:21:33 PM3/25/10
to

What is the "no expectations" rule? Google turns up nothing.

What does this rule have to do with Forth?

Andrew.

Hugh Aguilar

unread,
Mar 25, 2010, 5:35:51 PM3/25/10
to
On Mar 24, 4:31 am, MarkWills <markrobertwi...@yahoo.co.uk> wrote:
> On 23 Mar, 23:16, Hugh Aguilar <hughaguila...@yahoo.com> wrote:
> > I agree that Mark's proposal is pretty much just syntactic sugar for
> > what could be done with a global variable. It is a step in the right
> > direction though, so I wouldn't put it down.
>
> I think it's a bit more than that ;-)
> ...

> It permits and promotes 'safety' - it's kind of like a mini object
> with a single (static) instance variable. Well, a "static local" is
> the best description I can come up with. They differ from LOCALS in
> that LOCALS have to be carried to the word in question via the stack.
> Thus to persist LOCALS, they have to be stored somewhere else, or
> carried on the stack. This proposal doesn't require use of the stack.
> It's just a different flavour of a local.

In the C language it is possible to make local variables STATIC. This
means that they are in static memory rather than being on the stack,
and they retain their values between invocations to the function. That
seems to be pretty much what you are trying to implement. The
advantage of using static locals rather than globals is that doing so
prevents other functions from accessing your variables. You can
accomplish this in Forth (as suggested by the Beez) by using a global
variable and then using HIDE after your function definition to remove
the global from the dictionary search so no other functions can access
it. One problem with this is that HIDE is not ANS-Forth standard; it
is one of those semi-standard words that shows up in a lot of
implementations. Another problem with the HIDE solution is that a
person reading the code might not notice the HIDE that comes afterward
and will be confused about what scope the variable has. HIDE is a bad
solution because it allows the programmer to write sloppy code and
then it provides a way to fix the problem afterward.

In my novice package I have the word { that defines a list of local
variables that are temporarily allocated during the duration of the
function invocation (usually on the return stack, but this is
implementation-dependent). Why don't you write a similar word
STATIC{ that defines a list of local variables that are allocated in
static memory? You could use my code for { as a guide for doing this.
Be sure to *not* initialize your static locals though. The whole point
of static locals is that they retain their values between invocations
of the function.

The Forth-200x committee is very opposed to standardizing code that
can be implemented using already-standard Forth. This is even true in
regard to something like STATIC{ that can only be implemented in a
convoluted and complicated manner that is beyond the ability of most
novice programmers. This is also why the committee killed my :NAME
proposal. It can be implemented in ANS-Forth --- I did so myself in my
novice package.

This is my major disagreement with the Forth-200x committee. I
consider the standardization of words such as :NAME and STATIC{ to be
a philosophical issue. By standardizing these things we give our
*approval* to their use. By refusing to standardize these words we
imply that they shouldn't be used. Also, as a practical matter, most
Forth programmers are not capable of implementing words like this
until they have significant experience in Forth. By this time they
have turned into zombies and they are no longer thinking creatively.
That is pretty much the point of Elizabeth Rather's novice classes at
Forth Inc. --- she turns people into zombies and discourages them from
thinking for themselves. She tells them that CREATE DOES> is the
*standard* way to write defining words, and makes no mention of :NAME
at all. The people become focused on writing idiomatic code because
this is the only way to graduate from the class. That is my definition
of a zombie --- a person who cares about being idiomatic. Most
corporations will only employ zombies and will *never* employ any
person that still has a spark of intelligence left within his withered
soul. That is not why I became a Forth programmer though --- if I had
wanted to become a zombie I would have joined the C community!

Bruce McFarling

unread,
Mar 25, 2010, 6:13:19 PM3/25/10
to
On Mar 23, 6:56 pm, "The Beez'" <hans...@bigfoot.com> wrote:
> Consequently, the BIG question is: is this the right proposal. We
> don't have many words standardized for limiting the scope of words or
> data allocation.

We don't?

What about WORDLIST GET-CURRENT SET-CURRENT GET-ORDER SET-ORDER?
Regulating the scope of words is exactly what they do.

alberto pasquale

unread,
Mar 25, 2010, 7:10:52 PM3/25/10
to
On Mar 25, 3:35 pm, Hugh Aguilar <hughaguila...@yahoo.com> wrote:
>
> of a zombie --- a person who cares about being idiomatic. Most
> corporations will only employ zombies and will *never* employ any
> person that still has a spark of intelligence left within his withered
> soul. That is not why I became a Forth programmer though --- if I had
> wanted to become a zombie I would have joined the C community!

I don’t know if you realize that you just characterized allot of
people you don’t know as “zombies” just because we happened to work
for corporations (I’m one of them)an others because they are part of
the C community.

I never sympathized with the philosophy of tools that try to protect
their users from their own stupidity.
I have to re-consider, you would be the perfect candidate to benefit
from such tools.
And I’m not referring to just programming tools…

Alberto

Hugh Aguilar

unread,
Mar 25, 2010, 7:24:22 PM3/25/10
to

One thing we don't have is a word that traverses through all of the
words in a wordlist and hits each of them with HIDE. This would be
very useful for implementing STATIC{ that I described above.
STATIC{ would define several static locals and put them all in a
wordlist (lets call it STATIC-LOCALS). When semicolon wraps up the
definition of the function, it would HIDE *everything* in STATIC-
LOCALS without having to know what words are in there --- it wouldn't
have to use HIDE explicitely on each variable name --- it would just
hide them all in one fell swoop.

Another problem with writing STATIC{ is that some Forth systems have
code and data in separate segments of memory, and other Forth systems
intermingle code and data. If code and data are separate, then
STATIC{ can just define the static locals using VARIABLE. The problem
is that, if code and data are intermingled, then STATIC{ can't use
VARIABLE because doing so would put the data right in the middle of
the function code. STATIC would have to first compile an absolute
branch over the data so the function execution doesn't crash into the
data.

Forth would be much improved if the standard required that code and
data *always* be in separate segments of memory. This would greatly
simplify constructs such as STATIC{. Most modern processors are
Harvard architecture anyway, so this is the usual circumstance. On old-
school processors that just have one big flat memory segment, it would
still be possible to segregate the code and data for the purpose of
conforming to this requirement. The whole business of intermingling
code and data goes back to the 1970s when there was so little memory
available that everything had to be scrunched together in order to
make use of every byte. That is not true anymore though.

Bruce McFarling

unread,
Mar 25, 2010, 9:08:37 PM3/25/10
to
On Mar 25, 7:24 pm, Hugh Aguilar <hughaguila...@yahoo.com> wrote:
> One thing we don't have is a word that traverses through all of the
> words in a wordlist and hits each of them with HIDE.

If you want to permanently hide the wordlist, leave it anonymous and
then once its out of the search order, its hidden. If you want to hide
the wordlist but make it possible to re-open it, make it a vocabulary.

The Beez'

unread,
Mar 26, 2010, 3:20:59 AM3/26/10
to
On Mar 25, 8:21 pm, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:

> What is the "no expectations" rule?  Google turns up nothing.
The "no expectations rule" is what makes Forth virtually syntax-less.
E.g. a missing semi-colon doesn't trigger a syntax error - may be an
error, but no syntax error. What you're introducing is some kind of
syntax. Note Chuck didn't really like the way you wrote a double
number (with a dot). He thought that was an ugly hack. This is even
worse. It simply pollutes a clean "syntax".

Hans Bezemer

The Beez'

unread,
Mar 26, 2010, 3:31:41 AM3/26/10
to

To me it is more like different namespaces. I'm talking about private
and public words. Although there are several clever schemes to achieve
just that, there is no easy equivalent for exposing only the public
words of (e.g.) a library to the outside world.

I'm not waiting for an actual standardization effort, because what
will probably come out is an ugly, bloated, C-like, overengineered
proposal like SYNONYM, S\" or BEGIN-STRUCTURE that I will need to
support somehow. HIDE works fine for me - and a few other tiny Forths
that have long given up the standard.

Hans Bezemer

The Beez'

unread,
Mar 26, 2010, 3:41:38 AM3/26/10
to

I agree - and I don't understand the rant. C gives you enough rope to
hang yourself, unlike Pascal and stuff. Note many good Forth
programmers know their C very well too - the author included. C is
just a different beast with a different philosophy to solve problems.
You can just write a tid bit more murky programs in C and get away
with it than you would with Forth. E.g. there are loads of C
programmers that can't balance their malloc() and free() calls - and
get away with it. Balancing a stack is much, much harder.

But also note there are loads of Forth programs with 1 or 2 page
(legal/A4) words. How people can get these to work - let alone
maintain is beyond me too.

There is a reason why blocks have only 16 lines.

Hans Bezemer

Andrew Haley

unread,
Mar 26, 2010, 4:30:50 AM3/26/10
to
The Beez' <han...@bigfoot.com> wrote:
> On Mar 25, 8:21?pm, Andrew Haley <andre...@littlepinkcloud.invalid>
> wrote:
>> What is the "no expectations" rule? ?Google turns up nothing.

> The "no expectations rule" is what makes Forth virtually syntax-less.
> E.g. a missing semi-colon doesn't trigger a syntax error - may be an
> error, but no syntax error.

"?" (i.e. word not found) isn't described as a syntax error, but it's
an error. It's not described as a syntax error, since in Forth there
is no separation of syntax and actions.

> What you're introducing is some kind of syntax.

It's a convention that seems to me to be pretty useful, far more
useful IMO than an apparently arbitrary rule. Besides, the
"redefined" warning isn't part of the language anyway, so can't have
any effect on it.

> Note Chuck didn't really like the way you wrote a double number
> (with a dot). He thought that was an ugly hack.

Well, yes.

> This is even worse. It simply pollutes a clean "syntax".

I don't understand why a prefix convention pollutes a clean "syntax".
Forth is chock-full of prefix conventions.

Andrew.

Albert van der Horst

unread,
Mar 26, 2010, 10:18:55 AM3/26/10
to
In article <2e969297-ce35-42b4...@w39g2000prd.googlegroups.com>,

Hugh Aguilar <hughag...@yahoo.com> wrote:
>On Mar 25, 4:13 pm, Bruce McFarling <agil...@netscape.net> wrote:
>> On Mar 23, 6:56 pm, "The Beez'" <hans...@bigfoot.com> wrote:
>>
>> > Consequently, the BIG question is: is this the right proposal. We
>> > don't have many words standardized for limiting the scope of words or
>> > data allocation.
>>
>> We don't?
>>
>> What about WORDLIST GET-CURRENT SET-CURRENT GET-ORDER SET-ORDER?
>> Regulating the scope of words is exactly what they do.
>
>One thing we don't have is a word that traverses through all of the
>words in a wordlist and hits each of them with HIDE. This would be
>very useful for implementing STATIC{ that I described above.
>STATIC{ would define several static locals and put them all in a
>wordlist (lets call it STATIC-LOCALS). When semicolon wraps up the
>definition of the function, it would HIDE *everything* in STATIC-
>LOCALS without having to know what words are in there --- it wouldn't
>have to use HIDE explicitely on each variable name --- it would just
>hide them all in one fell swoop.

This is very similar to the PRIVATES PRIVATE DEPRIVE wordset in
tforth. (1994)
All words not to be used outside a module are marked PRIVATE.
This sets a bit in the header.
DEPRIVE scans all words till the previous PRIVATES and hides all
private words.
During debugging PRIVATES and DEPRIVE are noops.

Groetjes Albert

Bruce McFarling

unread,
Mar 26, 2010, 10:23:03 AM3/26/10
to
On Mar 26, 3:31 am, "The Beez'" <hans...@bigfoot.com> wrote:
> To me it is more like different namespaces. I'm talking about private
> and public words. Although there are several clever schemes to achieve
> just that, there is no easy equivalent for exposing only the public
> words of (e.g.) a library to the outside world.

But different namespaces *is* the easy way to expose only the public
words of a library to the outside world. Put the public words in the
current namespace when the library is called, and the private words in
a namespace defined by the library.

The Beez'

unread,
Mar 26, 2010, 12:56:06 PM3/26/10
to
On 26 mrt, 09:30, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:

> I don't understand why a prefix convention pollutes a clean "syntax".
> Forth is chock-full of prefix conventions.
Most of 'em introduced since the Forth "corrosion point" Forth-83. I
do not and will not support these. Note words like S", .", ['],
[char], etc. aren't prefixes, but words.

- If it's a word, execute it.
- If not, try converting it to a number.
- If it's not a number, FAIL!

That is the golden rule - and avoid as many other "IF"s while doing
it.

Hans Bezemer

John Passaniti

unread,
Mar 26, 2010, 1:27:20 PM3/26/10
to
On Mar 26, 3:41 am, "The Beez'" <hans...@bigfoot.com> wrote:
> There is a reason why blocks have only 16 lines.

Yes. And that reason is that most paper and video terminals at the
time Forth was invented could do between 64 and 80 characters per
line. So they chose 64, which was the lowest common denominator
terminal width that divided the size of disk block evenly.

I know this is going to blow people's minds, but independent of the
language I program in, I do my very best that I very rarely write a
definition that extends past about 16 or so lines. Amazing-- and I do
this entirely without artificial limits being placed on me by my
editor! It's like I have some weird ability not to do things that are
bad for me!

Coos Haak

unread,
Mar 26, 2010, 2:01:43 PM3/26/10
to
Op Fri, 26 Mar 2010 09:56:06 -0700 (PDT) schreef The Beez':

My recipe:


- If it's a word, execute it.
- If not, try converting it to a number.

- If not, try converting it to a string.
- If it's not a word, number or string, fail.

--
Coos

CHForth, 16 bit DOS applications
http://home.hccnet.nl/j.j.haak/forth.html

Coos Haak

unread,
Mar 26, 2010, 2:06:20 PM3/26/10
to
Op Fri, 26 Mar 2010 00:31:41 -0700 (PDT) schreef The Beez':

WORDLIST and SET-CURRENT are all that is needed. Nothing
overengineered.

The Beez'

unread,
Mar 26, 2010, 2:43:49 PM3/26/10
to
On 26 mrt, 18:27, John Passaniti <john.passan...@gmail.com> wrote:
> Yes.  And that reason is that most paper and video terminals at the
> time Forth was invented could do between 64 and 80 characters per
> line.  So they chose 64, which was the lowest common denominator
> terminal width that divided the size of disk block evenly.
Wow, John! I didn't know that. It may have been that my very first
computer was a Sinclair Spectrum that was 32*22. I may be wrong here a
character or two but that may be because it's very hard to count all
the blanks.

> I know this is going to blow people's minds, but independent of the
> language I program in, I do my very best that I very rarely write a
> definition that extends past about 16 or so lines.  Amazing-- and I do
> this entirely without artificial limits being placed on me by my
> editor!  It's like I have some weird ability not to do things that are
> bad for me!

It's amazing! I really didn't know you were that goooooooood! You have
to tell me sometime how you do that, because I keep falling off the
screen.

Hans Bezemer

John Passaniti

unread,
Mar 26, 2010, 3:37:44 PM3/26/10
to
On Mar 26, 2:43 pm, "The Beez'" <hans...@bigfoot.com> wrote:
> Wow, John! I didn't know that. It may have been that my very first
> computer was a Sinclair Spectrum that was 32*22. I may be wrong here a
> character or two but that may be because it's very hard to count all
> the blanks.

Your first computer is irrelevant as was mine. What is relevant are
the paper and video terminals at the time Forth was created (the
statement was "there is a reason why blocks have only 16 lines"). The
implication of that statement is that blocks were *designed* to be 16
lines to prevent overly-long definitions. I seriously doubt that; it
is far more likely to be what I suggest-- the longest physical line
common to paper and video terminals at the time that would equally
divide into 1024.

> It's amazing! I really didn't know you were that goooooooood! You have
> to tell me sometime how you do that, because I keep falling off the
> screen.

It's called self-control mixed with professionalism. And it's why
whenever I see people advocate blocks for the reason that a
constrained size promotes smaller definitions, I just shake my head.
Forth-- at least classical Forth before optimized native compilation--
was always supposed to be about the programmer being smarter than the
compiler. Well if Forther's really believe that, then they should
have zero problems with files as the smarter programmer will
voluntarily restrict themselves to practices like short definitions.

Elizabeth D Rather

unread,
Mar 26, 2010, 4:25:10 PM3/26/10
to
John Passaniti wrote:
> On Mar 26, 2:43 pm, "The Beez'" <hans...@bigfoot.com> wrote:
>> Wow, John! I didn't know that. It may have been that my very first
>> computer was a Sinclair Spectrum that was 32*22. I may be wrong here a
>> character or two but that may be because it's very hard to count all
>> the blanks.
>
> Your first computer is irrelevant as was mine. What is relevant are
> the paper and video terminals at the time Forth was created (the
> statement was "there is a reason why blocks have only 16 lines"). The
> implication of that statement is that blocks were *designed* to be 16
> lines to prevent overly-long definitions. I seriously doubt that; it
> is far more likely to be what I suggest-- the longest physical line
> common to paper and video terminals at the time that would equally
> divide into 1024.

It was partly the physical dimensions of the average screen and partly a
function of the size of disk sectors, which tended to be 256 or 512
bytes on minicomputers (1970's). In the early days we did occasionally
run into odd-sized screens and sectors, and struggled to accommodate
them. Fortunately, the industry did, too, and they disappeared.

I guarantee that length of definition had *nothing* to do with it,
because very few definitions ran over about 3 lines (and that is still
true, on average, for FORTH, Inc. code).

In the very early days of microFORTH (~1977) we experimented with
256-byte blocks (to save buffer space), but it was just too
inconvenient. A block should contain a group of closely-related
definitions that you can see together on the screen. So we standardized
on 1024, and that worked out well.

>> It's amazing! I really didn't know you were that goooooooood! You have
>> to tell me sometime how you do that, because I keep falling off the
>> screen.
>
> It's called self-control mixed with professionalism. And it's why
> whenever I see people advocate blocks for the reason that a
> constrained size promotes smaller definitions, I just shake my head.
> Forth-- at least classical Forth before optimized native compilation--
> was always supposed to be about the programmer being smarter than the
> compiler. Well if Forther's really believe that, then they should
> have zero problems with files as the smarter programmer will
> voluntarily restrict themselves to practices like short definitions.

When FORTH, Inc. switched to file-based source in the 90's, we had a
couple of programmers who suddenly started writing long definitions,
with a lot of repeated code sequences as they had seen in other
file-based languages. We had a couple of meetings about that, and they
went back to good Forth style, as John describes.

Hugh Aguilar

unread,
Mar 26, 2010, 4:32:40 PM3/26/10
to

According to the ANS-Forth document (16.6.1.2460): "A system shall
allow the creation of at least 8 new word lists in addition to any
provided as part of the system."

If STATIC{ relied on this, and STATIC{ was used in more than eight
functions --- kerblooey!

Jerry Avins

unread,
Mar 26, 2010, 4:53:54 PM3/26/10
to
Elizabeth D Rather wrote:

...

> When FORTH, Inc. switched to file-based source in the 90's, we had a
> couple of programmers who suddenly started writing long definitions,
> with a lot of repeated code sequences as they had seen in other
> file-based languages. We had a couple of meetings about that, and they
> went back to good Forth style, as John describes.

I would love to have been a fly on the wall at that meeting! Are there
minutes? :-)

Jerry
--
Discovery consists of seeing what everybody has seen, and thinking what
nobody has thought. .. Albert Szent-Gyorgi
ŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻ

Hugh Aguilar

unread,
Mar 26, 2010, 5:12:15 PM3/26/10
to
On Mar 26, 8:18 am, Albert van der Horst <alb...@spenarnc.xs4all.nl>
wrote:
> In article <2e969297-ce35-42b4-b277-34231e609...@w39g2000prd.googlegroups.com>,

> Hugh Aguilar <hughaguila...@yahoo.com> wrote:
> >One thing we don't have is a word that traverses through all of the
> >words in a wordlist and hits each of them with HIDE. This would be
> >very useful for implementing STATIC{ that I described above.
> >STATIC{ would define several static locals and put them all in a
> >wordlist (lets call it STATIC-LOCALS). When semicolon wraps up the
> >definition of the function, it would HIDE *everything* in STATIC-
> >LOCALS without having to know what words are in there --- it wouldn't
> >have to use HIDE explicitely on each variable name --- it would just
> >hide them all in one fell swoop.
>
> This is very similar to the PRIVATES PRIVATE DEPRIVE wordset in
> tforth. (1994)
> All words not to be used outside a module are marked PRIVATE.
> This sets a bit in the header.
> DEPRIVE scans all words till the previous PRIVATES and hides all
> private words.
> During debugging PRIVATES and DEPRIVE are noops.

I never heard of Tforth, but I wrote something similar in UR/Forth.
They had EXCISE, which smudged a word given its name (the same as HIDE
in other Forths). This didn't work well for me. I would have a long
list of EXCISE calls at the end of each file to get rid of namespace
clutter. The problem was that this list of explicit calls to EXCISE
would often get out of sync with the code in the file. I would write
new functions and forget to add them to the EXCISE list. I wrote a
word called PRIVATE that I put after each word in the file that I
wanted to be smudged later on. I then wrote a word called END-MODULE
that I put at the end of the file and it would go back through the
words defined and smudge the ones that were private. The way this
worked is that I found that the name field had set of flags that
indicated things like if the word was immediate or not. I used one of
the unused bits to indicate if the word was private or not. END-MODULE
would traverse the entire dictionary and smudge everything that was
marked private. This involved disassembling UR/Forth and also delving
into assembly language (because the name field had to be accessed
through the ES register). The same would be true of ANS-Forth --- such
a tool can't be written without delving into the internals.

My complaint against ANS-Forth is that too much of the internal
workings are unavailable to the programmer. The whole point of Forth
is that the programmer can extend the compiler. This is different from
other languages in which the compiler is strictly off-limits. This is
what I meant about the corporate mentality of ANS-Forth. The
philosophy seems to have been that the programmers must be forced into
a procrustean bed of idiomatic code. The result is that a lot of the
low-level aspects of the compiler are not in the standard, but we have
only high-level constructs. I find it amazing that :NAME was not
allowed, although it is a more low-level construct than colon. It
would make more sense to put :NAME in the standard than colon, because
colon can trivially be written in terms of :NAME, but :NAME can only
be written in terms of colon by constructing a string to be given to
EVALUATE. The whole ANS-Forth philosophy is completely upside-down!
The standard should provide useful low-level words and let the
programmer write his own high-level words in terms of those low-level
words. Instead, the standard provides only high-level words and tells
the programmer that this is "idiomatic" style, so he has to just
forget about custom-extending the compiler. This corrupt philosophy is
the reason why Forth lost popularity after the ANS-Forth standard came
out in 1994.

Now we have words like HIDE that many programmers rely on, but which
are not ANS-Forth standard and can't be written in ANS-Forth. The
result is that programmers become married to a particular compiler
vendor. They end up with a lot of legacy code of their own that will
*only* compile under a particular proprietary Forth system, so they
have to continue paying that Forth vendor for upgrades. This is great
for the Forth vendors; it is called "customer lock-in" --- but it is a
complete corruption of the reason for why we have a standard. That is
corporate mentality!

Jerry Avins

unread,
Mar 26, 2010, 5:43:20 PM3/26/10
to
Hugh Aguilar wrote:

...

> My complaint against ANS-Forth is that too much of the internal
> workings are unavailable to the programmer. The whole point of Forth
> is that the programmer can extend the compiler. This is different from
> other languages in which the compiler is strictly off-limits. This is
> what I meant about the corporate mentality of ANS-Forth. The
> philosophy seems to have been that the programmers must be forced into
> a procrustean bed of idiomatic code. The result is that a lot of the
> low-level aspects of the compiler are not in the standard, but we have
> only high-level constructs.

...

You just don't get it. You can modify the internal workings if you have
the source, but you have to realize that every ANS Forth has different
guts, and modifications to one don't necessarily work on another. ANS is
intended to guarantee (or at least greatly ease) porting. You can build
a trailer hitch for just about any car, but it won't likely fit another.

Extensions to Forth are usually written in Forth. ANS ensures that those
extensions are portable to other ANS implementations.

...

Albert van der Horst

unread,
Mar 26, 2010, 5:36:44 PM3/26/10
to
In article <l9imah2r8t69$.memic0tm...@40tude.net>,

My recipe:
- If it's a word execute it.

(It may be a prefix, then it compiles a denotation:
a number, a string, or something you define yourself.)

By the way Forth has quite some conventions:
in particular prefixes like F and D to indicate the
operands of operators.
Words with ! store or initialise.
Words with . do output.
etc.

Groetjes Albert

>--
>Coos

Jerry Avins

unread,
Mar 26, 2010, 6:08:00 PM3/26/10
to

It's important to distinguish between conventional and functional
prefixes (prefices?). Deciding to use a leading underscore as an
indicator of word usage is the same as the dot in .S : a convention. On
the other hand the common numeric prefixes ( $ # etc.) are functional.

Elizabeth D Rather

unread,
Mar 26, 2010, 6:45:36 PM3/26/10
to
Albert van der Horst wrote:
> In article <l9imah2r8t69$.memic0tm...@40tude.net>,
> Coos Haak <chf...@hccnet.nl> wrote:
>> Op Fri, 26 Mar 2010 09:56:06 -0700 (PDT) schreef The Beez':
>>
>>> On 26 mrt, 09:30, Andrew Haley <andre...@littlepinkcloud.invalid>
>>> wrote:
>>>> I don't understand why a prefix convention pollutes a clean "syntax".
>>>> Forth is chock-full of prefix conventions.
>>> Most of 'em introduced since the Forth "corrosion point" Forth-83. I
>>> do not and will not support these. Note words like S", .", ['],
>>> [char], etc. aren't prefixes, but words.

Well, yes. Andrew said "conventions," and what he meant was practices
such as Albert describes below.

...


>> My recipe:
>> - If it's a word, execute it.
>> - If not, try converting it to a number.
>> - If not, try converting it to a string.
>> - If it's not a word, number or string, fail.

Insofar as what one is processing at this point *is* a string, I don't
understand "converting it to a string".

...


>
> By the way Forth has quite some conventions:
> in particular prefixes like F and D to indicate the
> operands of operators.
> Words with ! store or initialise.
> Words with . do output.
> etc.
>
> Groetjes Albert

I think this was the sort of thing Andrew meant.

J Thomas

unread,
Mar 26, 2010, 8:15:55 PM3/26/10
to
Hugh Aguilar wrote:

> .... This involved disassembling


> UR/Forth and also delving into assembly language (because the name field
> had to be accessed through the ES register). The same would be true of
> ANS-Forth --- such a tool can't be written without delving into the
> internals.
>
> My complaint against ANS-Forth is that too much of the internal workings
> are unavailable to the programmer. The whole point of Forth is that the
> programmer can extend the compiler. This is different from other
> languages in which the compiler is strictly off-limits. This is what I
> meant about the corporate mentality of ANS-Forth. The philosophy seems
> to have been that the programmers must be forced into a procrustean bed
> of idiomatic code. The result is that a lot of the low-level aspects of
> the compiler are not in the standard, but we have only high-level
> constructs.

If you are stuck with a single standard, then you mustn't too much
restrict how the low-level parts of the compiler can vary. Forth-83 did
that badly, it basicly required that Forth code had to be indirect-
threaded which on many systems (including PCs) required it to be somewhat
slow. There were some big advantages to that, but they were balanced that
it turned Forth into a scripting language instead of something that could
get within shouting distance of C.

> I find it amazing that :NAME was not allowed, although it is
> a more low-level construct than colon. It would make more sense to put
> :NAME in the standard than colon, because colon can trivially be written
> in terms of :NAME, but :NAME can only be written in terms of colon by

> constructing a string to be given to EVALUATE.The whole ANS-Forth
> philosophy is completely upside-down!

Look at places where ANS-Forth did do it that way. Like, say that you
care whether / is floored or symmetric. ANS-Forth gives you the tools to
define it for yourself so it will work as you want. But the result is
that you have to define it yourself in every file that might be compiled
separately! Lots of garbage. People have created workarounds -- [DEFINED]
etc. In the most complete solution to this problem you would have a giant
library that has everything you might want, and every time you use a word
that might not be defined you check and if it isn't already present you
define it from the library.

You could have a routine that checks a source file and lists all the
words that are not part of the Core wordset, and puts them at the
beginning of the file so it will be easy to check whether they're defined
and load the definitions if not.

All very baroque, but what can you do? If the standard defined only the
low-level constructs, we'd be re-inventing all the higher level stuff
every time we wanted to use it.

They made the best compromise they could. Even so, minimalists complained
that the Core wordset is far too large, and yet a whole lot of useful
things are left out.

> The standard should provide useful
> low-level words and let the programmer write his own high-level words in
> terms of those low-level words. Instead, the standard provides only
> high-level words and tells the programmer that this is "idiomatic"
> style, so he has to just forget about custom-extending the compiler.

You can extend the compiler in specific ways. You can create your own
control structures, there's no limit on that if you are willing to deal
with the complexity. They said that control-flow should behave as if the
items are stored on a stack, the control-flow stack. You can't use the
data stack in the middle of that because it might be used for the control-
flow stack. And with just two special words to manipulate that stack, you
can do a whole lot. No restrictions on how the low-level stuff works
except that these two stack commands have to work. An implementation
could put control structures into a linked list or whatever, and they can
be any size, just make those two words work and the code which uses them
is portable.

And you can do anything that can be done with CREATE DOES> . I'm pretty
sure that if CREATE DOES> was being created from scratch it would be
different, but the way it works now does allow a whole lot of
flexibility. You complain about the performance, but at least it does let
you do a whole lot of things. (And notice how standard Forth does not
provide any array words, and people keep re-inventing them over and over,
or doing it once and adding their own personal versions to each Forth
they use, etc.

> This corrupt philosophy is the reason why Forth lost popularity after
> the ANS-Forth standard came out in 1994.

It could be argued that Forth lost popularity at a slower rate after
1994. I saw a criticism that arguing about the standard took years from
many productive Forth users who might otherwise have done great things. I
couldn't prove that's wrong. It's very hard to say for sure why history
has gone the way it has because we do not have a control group. If only
we had 50 or 100 different worlds that were all the same except for
different choices about Forth! Then we could compare the results and
guess how important each difference was.



> Now we have words like HIDE that many programmers rely on, but which are
> not ANS-Forth standard and can't be written in ANS-Forth. The result is
> that programmers become married to a particular compiler vendor. They
> end up with a lot of legacy code of their own that will *only* compile
> under a particular proprietary Forth system, so they have to continue
> paying that Forth vendor for upgrades. This is great for the Forth
> vendors; it is called "customer lock-in" --- but it is a complete
> corruption of the reason for why we have a standard. That is corporate
> mentality!

Can't you look at each Forth you use and figure out how to write HIDE for
it?

How about this. To have one single standard it takes a whole lot of
compromise and jockeying around with implementers and so on, to get
something that does not perfectly fit anybody's needs, but that a lot of
people subscribe to. At least things that do fit the standard will be
common to a lot of systems. But what if you make your own standard,
independent of all that. Write up how it works. Aguilar's Low Level
Standard. Come up with a good name for it. Document it thoroughly. Then
for any code that fits your standard, people can get a good sense what
systems it will run on because of that documentation.

Then you or whoever wants to can implement your standard words on GForth
and SwiftForth and whatever they want.Your code will be portable to most
of the systems people care about. And people who write their own Forths
can have a sense of what they need to supply.

And you or whoever wants to can define high-level words in terms of your
standard. The result is that you provide one good way to build a Forth
from a low-level kernel. And if the Forth-200x words have been defined in
your system, then any ANS code can be portable to systems that fit your
standard, and any system that fits your standard can become a Forth-200x
system.

And if a number of Forth users do that, people can use any standard
that's documented well enough. We might even get somebody who looks at
the various standards and makes a sort of meta-standard -- he could point
at the special entitlements that each requires and develop a sort of
taxonomy of Forth standards.

It might be fun. It looks to me like a lot more fun than spending the
same effort arguing about what's wrong with Forth-94 and Forth-200x etc.

Coos Haak

unread,
Mar 26, 2010, 8:45:29 PM3/26/10
to
Op Fri, 26 Mar 2010 12:45:36 -1000 schreef Elizabeth D Rather:

<snip>


>> Coos Haak <chf...@hccnet.nl> wrote:
>>> My recipe:
>>> - If it's a word, execute it.
>>> - If not, try converting it to a number.
>>> - If not, try converting it to a string.
>>> - If it's not a word, number or string, fail.
>
> Insofar as what one is processing at this point *is* a string, I don't
> understand "converting it to a string".
>

A text between double quotes. There is some conversion, the pair "" in
a string gives a single " in the resulting string.
The difference to S" is that no space is needed after the quote, so I
had to modify the interpreter.
The usage is the same as Albert's "-prefix, but the implementation is
different.


> ...
>>
>> By the way Forth has quite some conventions:
>> in particular prefixes like F and D to indicate the
>> operands of operators.
>> Words with ! store or initialise.
>> Words with . do output.
>> etc.
>>
>> Groetjes Albert
>
> I think this was the sort of thing Andrew meant.
>

And the prefix 2 for manipulation of pairs on the stack.

> Cheers,
> Elizabeth

The Beez'

unread,
Mar 26, 2010, 8:57:54 PM3/26/10
to
On 26 mrt, 21:25, Elizabeth D Rather <erat...@forth.com> wrote:
> When FORTH, Inc. switched to file-based source in the 90's, we had a
> couple of programmers who suddenly started writing long definitions,
> with a lot of repeated code sequences as they had seen in other
> file-based languages.  We had a couple of meetings about that, and they
> went back to good Forth style, as John describes.
Well, I've written a few tiny - but non-trivial - programs in my time,
but I'm still in awe about a BASIC interpreter Chuck wrote - in a
couple of blocks! Well, that's coding.

BTW, John. The Spectrum didn't work out very well with blocks. Someone
had ported a vanilla FIG implementation to it, including the editor.
Kept scrolling all over the place. I used to edit Forth in Tasword II
which conveniently had a block layout AND 64 chars per line. Having
patched the compiler - by locating and overwriting the assembly
routines - so it worked with my add-on disk, made it quite workable.
Layout the stuff with Tasword and edit it with the blockeditor..

The fun part was: it worked as if it had been especially written for
disk instead of cassette tape. Even loading error messages from disk -
which was disabled for obvious reasons. That's when I had my first
"WOW!" Forth experience. Patching in assembly routines was quite
smooth as well - contrary to the other programming beasts (like
Pascal, Prolog) on that 40K machine.

Hans

Coos Haak

unread,
Mar 26, 2010, 9:05:55 PM3/26/10
to
Op Fri, 26 Mar 2010 17:57:54 -0700 (PDT) schreef The Beez':

I had a driver for 64 characters per line (4 bit wide, just readable).
So I could fare reasonable well with blocks, Figforth editor, and a
8080 assembler with Z80 extensions. Two-pass metacompiling (to resolve
relocations) with a Microdrive was done in about 12 minutes, but that
could have been the cassette version, I don't rememember.

The Beez'

unread,
Mar 27, 2010, 7:37:26 AM3/27/10
to
On Mar 27, 2:05 am, Coos Haak <chfo...@hccnet.nl> wrote:
> I had a driver for 64 characters per line (4 bit wide, just readable).
> So I could fare reasonable well with blocks, Figforth editor, and a
> 8080 assembler with Z80 extensions. Two-pass metacompiling (to resolve
> relocations) with a Microdrive was done in about 12 minutes, but that
> could have been the cassette version, I don't rememember.
Tasword used a similar 4 bit wide driver. I wanted to pursue the same
direction as you did, but then I lost interest since I got a Toshiba
portable with two 720KB drives and TurboC v2. I never really liked
FPC, although I did run it. Never wrote a non-trivial program in it.
After that Forth lay dormant for almost 10 years, until I developed
4tH. 4tH proved a smooth transition between C and Forth. Looking back,
I've never written a C program since 4tH ;-)

Strangely enough, 4tH was originally developed as a scripting engine
for another (C) program I was writing, but things got a bit out of
hand.. ;-)

Hans Bezemer

Andrew Haley

unread,
Mar 27, 2010, 2:15:18 PM3/27/10
to
The Beez' <han...@bigfoot.com> wrote:
> On 26 mrt, 09:30, Andrew Haley <andre...@littlepinkcloud.invalid>
> wrote:
>> I don't understand why a prefix convention pollutes a clean "syntax".
>> Forth is chock-full of prefix conventions.
> Most of 'em introduced since the Forth "corrosion point" Forth-83.

Oh, come one now. Common prefixes are 2 , ' , ? , and so on. These
go back to the dawn of Forth.

> - If it's a word, execute it.
> - If not, try converting it to a number.
> - If it's not a number, FAIL!
>
> That is the golden rule - and avoid as many other "IF"s while doing
> it.

No argument there: I wasn't suggesting that the interpreter or
compiler treat words begining with _ specially, it's just a
convention.

Andrew.

Bruce McFarling

unread,
Mar 27, 2010, 3:15:01 PM3/27/10
to
On Mar 27, 2:15 pm, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:

> No argument there: I wasn't suggesting that the interpreter or
> compiler treat words begining with _ specially, it's just a
> convention.

Referring to it as local static data, and the suggestion that the
redefinition warning be modified added to the impression that this is
precisely was being suggested.

As a naming convention, I used much the same for external assembler
labels when the Forth dictionary name violates the assemblers label
syntax. _Colon for ``:'' and _Variable_ for ``<Variable>''.

Andrew Haley

unread,
Mar 27, 2010, 6:09:47 PM3/27/10
to
Bruce McFarling <agi...@netscape.net> wrote:
> On Mar 27, 2:15?pm, Andrew Haley <andre...@littlepinkcloud.invalid>

> wrote:
>
>> No argument there: I wasn't suggesting that the interpreter or
>> compiler treat words begining with _ specially, it's just a
>> convention.
>
> Referring to it as local static data, and the suggestion that the
> redefinition warning be modified added to the impression that this is
> precisely was being suggested.

I don't see why: there's no need for the interpreter to treat local
static data any differently from other static data. As for
suppressing the redefinition warning, that's a convenience that
doesn't affect the language.

> As a naming convention, I used much the same for external assembler
> labels when the Forth dictionary name violates the assemblers label
> syntax. _Colon for ``:'' and _Variable_ for ``<Variable>''.

Mmm, you mean a non-Forth assembler, I guess.

Andrew.

Albert van der Horst

unread,
Mar 28, 2010, 7:11:44 AM3/28/10
to
In article <se-dna86d9cfszbW...@supernews.com>,
Andrew Haley <andr...@littlepinkcloud.invalid> wrote:
<SNIP>
>
>I can't see any use for "private", but the Python convention of using
>_ as a prefix for local static data seems to be very sensible. Given
>that, one could tweak the "Redefined" warning message that most Forths
>have so that it doesn't warn for locals. This is a clean, simple, and
>very Forth solution.

Be careful! Redefinition is a red flag.
Redefinition of a local more so, than of a regular word, because
it is always inadvertant.

>
>Andrew.

Albert van der Horst

unread,
Mar 28, 2010, 6:58:15 AM3/28/10
to
In article <f64a17cf-e7f2-43ac...@u19g2000prh.googlegroups.com>,

On the practical side, very few system restrict the creation of
wordlists (as opposed to the search order).
But the point of a private wordlist is that it is only used
during the creation of one particular application wordlist,
so it is not in the search order for user programs.

Groetjes Albert

Bruce McFarling

unread,
Mar 28, 2010, 1:02:59 PM3/28/10
to
On Mar 28, 7:11 am, Albert van der Horst

> Be careful! Redefinition is a red flag.
> Redefinition of a local more so, than of a regular word, because
> it is always inadvertant.

He doesn't mean a local when he says "local static", he means what
would be a local static variable in some other languages, but what is
just be a variable (possibly residing in a local toolkit vocabulary)
in normal Forth.

Bruce McFarling

unread,
Mar 28, 2010, 1:20:22 PM3/28/10
to
On Mar 26, 4:32 pm, Hugh Aguilar <hughaguila...@yahoo.com> wrote:
> If STATIC{ relied on this, and STATIC{ was used in more than eight
> functions --- kerblooey!

Yes, someone with an obsession for coverage of all cases allowed in
Forth-94 might include an alternate definition of STATIC{ that put all
of the static variables in a single wordlist, for the hypothetical
Forth-94 implementation which implements wordlists as a vector or
array and the wordlist id is the index into the vector.

Since the search order has to operate as a stack rather than as a
linked list, its likely simplest to just have each wordlist id point
to a head of list pointer in dataspace and implement the search order
as a stack of those pointers.

Andrew Haley

unread,
Mar 28, 2010, 3:32:18 PM3/28/10
to
Albert van der Horst <alb...@spenarnc.xs4all.nl> wrote:
> In article <se-dna86d9cfszbW...@supernews.com>,
> Andrew Haley <andr...@littlepinkcloud.invalid> wrote:
> <SNIP>
>>
>>I can't see any use for "private", but the Python convention of using
>>_ as a prefix for local static data seems to be very sensible. Given
>>that, one could tweak the "Redefined" warning message that most Forths
>>have so that it doesn't warn for locals. This is a clean, simple, and
>>very Forth solution.
>
> Be careful! Redefinition is a red flag.
> Redefinition of a local more so, than of a regular word, because
> it is always inadvertant.

I've always found the redefinition warning a bit dubious because it
gives people the idea that redefining a word used locally is a bad
thing, and this leads people to use complex and messy names. But
using words with real names like AWAIT and START, etc, is a good
thing, and there's no reason not to use them for the sake of a
"redefined" message.

Andrew.