RfD: !ME and @ME

69 views
Skip to first unread message

MarkWills

unread,
Mar 23, 2010, 11:07:36 AM3/23/10
to
Dear members of the Forth community,

This is my first RfD. I thought I would give it a try. Apologies if
this idea is old-hat or already provisioned in a different form.

Respectfully,

Mark Wills

---------------------------------------------------

RfD: Proposal for New Forth Words !ME and @ME
=============================================

Revision History
================
23rd March 2010 – First Draft

Change Log
==========


Rationale
=========

Problem
-------
A colon definition that needs to maintain state between calls, or
save pertinent associated data cannot do so without the use of
VARIABLEs or VALUEs, which are external to the colon definition.
A better approach is to store related data with the word itself,
promoting encapsulation at the word level.

This proposal presents an alternative data storage solution whereby
the data stored is directly associated with the colon definition
itself, and not reliant on an external VARIABLE or VALUE.

Current Practice
----------------
Current practice is to save stateful information in a variable.
Consider an example whereby in a multi-tasking environment, access
to an IO device shall only be granted to the current task if it is
not already in use by another task:

VARIABLE SerialOwnerID 0 SerialOwnerID !

: CanClaimSerialPort? ( taskid|0 – flag )
?DUP 0= IF 0 SerialOwnerID ! ResetSerialResources EXIT THEN
SerialOwnerID @ 2DUP 0= -ROT = OR
IF
DUP SerialOwnerID ! AllocateSerialResources TRUE ( granted )
ELSE
DROP FALSE ( denied )
THEN
;

Here, a variable is used to hold the ID of the task currently using
the serial port. A task wishing to claim the serial port pushes its
task ID onto the stack, and calls CanClaimSerialPort? If the
SerialOwnerID is 0 (not in use) or the same as the requesting task’s
ID, the request is granted by pushing TRUE. If any other task
requests the serial port, the request is denied by pushing FALSE.
The serial port can be freed by calling CanClaimSerialPort? with 0
on the stack.

Tyical Use
----------
The above code can be simplified, made more secure, and the
dependence on an external variable removed by the use of the
proposed words !ME and @ME pronounced 'store in me' and 'fetch from
me' respectively.

:!@ CanClaimSerialPort? ( taskid|0 – flag )
?DUP 0= IF 0 !ME ResetSerialResources EXIT THEN
@ME 2DUP 0= -ROT = OR
IF
DUP !ME AllocateSerialResources TRUE ( granted )
ELSE
DROP FALSE ( denied )
THEN CR
;

Here, the data is not stored in a variable. Rather, it is stored
'within' the word itself (in practice, the mechanics of how and
where the data is stored and retrieved is implementation specific,
but the Forth programmer can think of/visualise the data as being
stored 'inside' the word). Not only is the dependence on an external
variable removed, but the data can be considered 'protected' since,
unlike variables and values, which are global in scope, the data can
only be accessed by the word that 'contains' it.

It is proposed that any colon definition can use !ME and @ME. Each
colon definition can have exactly one cell sized 'me' storage area.

Remarks
-------
Note that in the above example, the word is built not with : but with
a modified : called :!@ (pronounced "build with storage").
This word is responsible for:

* building the dictionary entry in the normal way, as per :
* reserving a cell of memory;
* initialising the reserved cell to zero;
* making the address of the reserved cell available to the colon
definition in an implementation specific manner.

Alternative methods are of course available, and indeed welcomed for
discussion. It would probably be preferable if :!@ were not required.
One possible solution would be to place references to !ME and @ME on
the parameter stack or the return stack as they occur, and have ;
back-fill the references, in a similar way to IF...THEN and DO...LOOP
etc. This author welcomes and encourages discussion on this point.

Typical Use
-----------
Use of the above described facility would be desirable for:

* Generating unique IDs;
* Reference counting;
* Holding object instance references;
* Promoting secure/atomic data isolation/security.

Proposal
--------
13.6.1.xxxx :!@ LOCAL

Interpretation:
Interpretation semantics for this word are undefined.

Compilation:
Compilation semantics for this word are undefined.

Run-time: ( C: "(spaces)name" -- colon-sys )
Create a definition for name as per 6.1.0450, reserve a cell as
storage space, initialising the cell’s value to zero. Make the
address of the reserved cell available to the definition being
constructed in an implementation specific manner.

13.6.1.xxxx @ME LOCAL

Interpretation:
Interpretation semantics for this word are undefined.

Compilation:
Compilation semantics for this word are undefined.

Run-time: ( -- x )
Fetch the contents of the cell allocated by :!@ and place on
the parameter stack.

13.6.1.xxxx !ME LOCAL

Interpretation:
Interpretation semantics for this word are undefined.

Compilation:
Compilation semantics for this word are undefined.

Run-time: ( x -- )
Take the contents of the topmost stack entry and store in the
cell allocated by :!@

Reference Implementation:
-------------------------
The following reference implementation is written in a Forth-83
style, since this author’s knowledge is currently restricted to
Forth-83 type systems.

In the interests of simplicity for example purposes, the
implementation below assumes an ITC system and stores the data
in line with the word, coding a BRANCH instruction to jump over the
data. This example probably constitutes the simplest possible
solution on an ITC system (though rather in-efficient) and can be
coded purely in high level Forth. ANS Forth examples are welcomed.
With carnal knowledge, implementers can of course use much more
sophisticated solutions, reducing size and increasing efficiency.

:!@ FRED 1 2 @ME ;

Is compiled as:

+--------->--------+ +----@ME----+
| | | |
+------+--------+---+------+-----+---+-----+---+-----+-----+---+
| FRED | BRANCH | 4 | DATA | LIT | 1 | LIT | 2 | LIT | xxx | @ |
+------+--------+---+------+-----+---+-----+---+-----+-----+----
^ |
+----------------<---------------+

It can be seen that @ME has in fact been coded as LIT <address> @
where <address> points to the data cell reserved by :!@

High level code to implement the proposed behaviour is presented
thus:

: :!@ : COMPILE BRANCH 4 , 0 , ;
: ?ME LATEST @ >CFA 6 + [COMPILE] LITERAL ;
: @ME ?ME COMPILE @ ; IMMEDIATE
: !ME ?ME COMPILE ! ; IMMEDIATE

Since the above code is not ANS compliant, a discussion of the
code is presented here:

:!@
This word uses : to compile the dictionary entry, and then compiles
a BRANCH <offset> 0 sequence. The idea is that the 0 represents the
reserved storage cell, and the BRANCH simply branches over it.
Note that the requirement for :!@ could be removed by simply
re-defining : thus:
: : : COMPILE BRANCH 4 , 0 , ;
However, this would mean that a storage cell would be reserved for
every colon definition, regardless of whether !ME or @ME was used
or not. This could be un-desirable on memory constrained systems.


?ME
This word calculates the address of the reserved cell by invoking
LATEST to get the dictionary address of the dictionary header just
created by :!@ , converting to a CFA and adding (in this case, since
the above code was written on a legacy 16 bit machine) 6 to account
for the BRANCH <address> sequence. It then compiles this address
into the current colon definition as a literal. Since address
calculation and compilation is required by both !ME and @ME it is
factored out to become ?ME ('address of me').

@ME
Invokes ?ME to compile the address of the reserved cell into the
current definition, and then compiles @ into the definition. Note
this is an immediate word.

!ME
Invokes ?ME to compile the address of the reserved cell into the
current definition, and then compiles ! into the definition. Note
this is an immediate word.


Testing
-------

Test of the storage and retrieval of ME data:

:!@ TestME ( Storing: x true -- | Fetching: false -- x )
IF
ME! ." Stored " ME@ . ." in me."
ELSE
." My value is " ME@ .
THEN
CR
;

Test storage with:
99 TRUE TestME
Output:
Stored 99 in me
ok

Test retrieval with:
FALSE TestME
Output:
My value is 99
ok

Example UID generator:
:!@ GetUID ( -- uid )
@ME 1+ DUP !ME ;

Test with:
GetUID GetUID GetUID GetUID .S
Output:
1 2 3 4 ok

Example Reference Counter
:!@ RefCount ( true|false –- count+1|count-1 )
@ME SWAP IF 1+ ELSE 1- THEN DUP !ME ;
Test with:
TRUE RefCount TRUE RefCount TRUE RefCount
FALSE RefCount FALSE RefCount FALSE RefCount .S
Output:
1 2 3 2 1 0 ok

Author
------
Mark Wills
Overseas Consultants LTD
Parliament Street
Ramsey
Isle of Man
Email: MarkRob...@yahoo.co.uk
Website: http://www.OverseasConsultants.co.uk

[eof]

ar

unread,
Mar 23, 2010, 11:31:44 AM3/23/10
to
Hello

Sorry, I must be very obtuse but IMHO there is no problem in keeping
state in VALUES, VARIABLES or USER variables, as long as you have a
"module" mechanism which discourages unwanted access and use only the
exported interface for the module. VFX's Source Code Module is example
of this. Taking this to the extreme we have classes and object
oriented programmming.

Best regards
Rafael


> Output:
>         1 2 3 2 1 0 ok
>
> Author
> ------
> Mark Wills
> Overseas Consultants LTD
> Parliament Street
> Ramsey
> Isle of Man

> Email: MarkRobertWi...@yahoo.co.uk
> Website:http://www.OverseasConsultants.co.uk
>
> [eof]

Bernd Paysan

unread,
Mar 23, 2010, 11:43:27 AM3/23/10
to
MarkWills wrote:

> A colon definition that needs to maintain state between calls, or
> save pertinent associated data cannot do so without the use of
> VARIABLEs or VALUEs, which are external to the colon definition.

I'm sorry, but this is completely non-conclusive. *Of course* a colon
definition can store permanent state in a variable or value. This is
common usage, as you state yourself. The only "advantage" I can see
from your proposal is that it makes debugging of these words harder,
since you can't easily inspect their hidden state.

Alternative suggestion for combining storage and code just once:
Interpretative DOES> (combining different storage with the same code is
trivial: just normal DOES> is perfect ;-). Example:

Create CanClaimSerialPort? 0 ,
DOES> >r
?dup 0= IF 0 r> ! ResetSerialResources EXIT THEN
r@ @ 2dup 0= -rot = or IF
dup r@ ! AllocateSerialResources true
ELSE
drop false
THEN r> drop ;

Gforth and bigForth support interactive DOES>. So there is at least
some more common practice, and it's probably more general and more
useful than your suggestion.

--
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://www.jwdt.com/~paysan/

alberto pasquale

unread,
Mar 23, 2010, 1:46:04 PM3/23/10
to


On the system I use, two or more diferent tasks call the same words.
Specifically in your serial port sample, I have diferent tasks
invoking the same protocol driver with pointers to specific hardware
stored in user variables (each task has it's own user variables).
If the word is storing data in some private storage, how do you
propose to make that word re-entrant in a multitasker system ?

Alberto

Hugh Aguilar

unread,
Mar 23, 2010, 3:27:16 PM3/23/10
to
On Mar 23, 11:46 am, alberto pasquale <albe...@hal-pc.org> wrote:
> On the system I use, two or more diferent tasks call the same words.
> Specifically in your serial port sample, I have diferent tasks
> invoking the same protocol driver with pointers to specific hardware
> stored in user variables (each task has it's own user variables).
> If the word is storing data in some private storage, how do you
> propose to make that word re-entrant in a multitasker system ?
>
> Alberto

I think that Mark is trying to get something like ICON-style
generators. These aren't going to be reentrant. They actually get
reentered over and over again, but they take up where they left off;
the invocations aren't local to themselves. Reentrancy in a
multitasking system is only an issue in regard to a micro-controller.
By comparison, nobody is going to use generators in a micro-controller
--- generators are very much desktop software that does time-consuming
recursive-descent searches --- I don't see any conflict, as these are
completely different kinds of programming.

Realistically, Forth-200x is never going to support anything
innovative --- including generators. I am putting together a competing
standard that is intended to be innovative. Go to:
http://www.forthwiki.com/tiki-index.php and visit the Fixed-Forth
forum to discuss this further.

Andrew Haley

unread,
Mar 23, 2010, 4:19:48 PM3/23/10
to
Hugh Aguilar <hughag...@yahoo.com> wrote:

> Reentrancy in a multitasking system is only an issue in regard to a
> micro-controller.

Pardon?

Andrew.

Jerry Avins

unread,
Mar 23, 2010, 4:52:39 PM3/23/10
to

Discount that nonsense.

Jerry
--
Discovery consists of seeing what everybody has seen, and thinking what
nobody has thought. .. Albert Szent-Gyorgi
ŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻŻ

Hugh Aguilar

unread,
Mar 23, 2010, 6:56:06 PM3/23/10
to
On Mar 23, 2:19 pm, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:

> Hugh Aguilar <hughaguila...@yahoo.com> wrote:
> > Reentrancy in a multitasking system is only an issue in regard to a
> > micro-controller.
>
> Pardon?
>
> Andrew.

On desktop computers you have an OS that does your multitasking for
you --- you don't have to worry about reentrancy issues because your
tasks aren't sharing any code. On a micro-controller your Forth system
is your OS; your tasks are sharing code, and that code has to be
reentrant.

The Beez'

unread,
Mar 23, 2010, 6:56:09 PM3/23/10
to
On 23 mrt, 21:52, Jerry Avins <j...@ieee.org> wrote:

> Andrew Haley wrote:
> > Hugh Aguilar <hughaguila...@yahoo.com> wrote:
IMHO this is nothing more than a static variable.
- The first question is: why only one? Why not several?
- The second question is: a static variable is nothing but a global
one with its scope limited to one single routine (sorry, I don't feel
like doing a semantic discussion).

Consequently, the BIG question is: is this the right proposal. We
don't have many words standardized for limiting the scope of words or
data allocation. Is:

:private myword
lots of other words
;

Or even more wild:

variable myvar private

: mywords
static mystatic
lots of other words
; private

Not much more useful and generic?

Hans Bezemer

Hugh Aguilar

unread,
Mar 23, 2010, 7:16:17 PM3/23/10
to
On Mar 23, 4:56 pm, "The Beez'" <hans...@bigfoot.com> wrote:
> IMHO this is nothing more than a static variable.
> - The first question is: why only one? Why not several?
> - The second question is: a static variable is nothing but a global
> one with its scope limited to one single routine (sorry, I don't feel
> like doing a semantic discussion).

I agree that Mark's proposal is pretty much just syntactic sugar for
what could be done with a global variable. It is a step in the right
direction though, so I wouldn't put it down.

In order to implement generators, we need to be able to save
information that is not readily available. The problem is not *where*
we stash the data, but *how* we obtain the data. Specifically, we have
the data on the return stack such as DO LOOP information and a chain
of return addresses from a recursive descent. We also have the xt of
the address in the middle of the function where we will want to
reanimate the function at, which is what my XT function was supposed
to grab.

In regard to the return stack data, I would recommend that when this
generator is called, it is given its own return stack. The entire
return stack can then be saved somewhere (the heap). This way we grab
everything in one fell swoop.

MarkWills

unread,
Mar 24, 2010, 6:17:09 AM3/24/10
to
On 23 Mar, 22:56, "The Beez'" <hans...@bigfoot.com> wrote:
> On 23 mrt, 21:52, Jerry Avins <j...@ieee.org> wrote:> Andrew Haley wrote:
> > > Hugh Aguilar <hughaguila...@yahoo.com> wrote:
>
> IMHO this is nothing more than a static variable.
> - The first question is: why only one? Why not several?

It's a good question, and like I said in the proposal, I'm happy to
receive comments and questions/debate. Totally open to the idea being
improved upon.

> - The second question is: a static variable is nothing but a global
> one with its scope limited to one single routine (sorry, I don't feel
> like doing a semantic discussion).
>

Er, that's not a question. It's a statement ;-) (sorry, could not
resist!)


> Consequently, the BIG question is: is this the right proposal. We
> don't have many words standardized for limiting the scope of words or
> data allocation. Is:
>
> :private myword
>   lots of other words
> ;
>
> Or even more wild:
>
> variable myvar private
>
> : mywords
>   static mystatic
>   lots of other words
> ; private
>
> Not much more useful and generic?
>
> Hans Bezemer

It looks good to me. Why has something of this nature not been
proposed before?

It has been suggested that 'radical' (my choice of word) proposals do
not belong in the 200x, or to put it another way, radical, or
experimental ideas do not belong in the standard. I *can* see the
reasoning behind that, but at the same time, if that's the case, why
not just freeze the standard?

I'm sure Forth can benefit from borrowing ideas from other languages,
such as atomicity, OOP etc. I think it is a shame that currently there
seems to be some resistance to 'modernisation'. Again, I can see the
reasoning - the standards people have a responsibility to make sure
they don't introduce things that break existing code, for sure, no
argument. But is it not possible to introduce new ideas without
breaking existing systems? I would have thought so...

Perhaps it's just me... Perhaps I jumped the gun a bit. Perhaps an
idea needs to be proven in the field before it should be considered
for standardisation. It will either stand the test of time or it
won't. IIRC some paradigms that have been in use in VFX for many years
are only being proposed just now.

MarkWills

unread,
Mar 24, 2010, 6:31:32 AM3/24/10
to
On 23 Mar, 23:16, Hugh Aguilar <hughaguila...@yahoo.com> wrote:
> On Mar 23, 4:56 pm, "The Beez'" <hans...@bigfoot.com> wrote:
>
> > IMHO this is nothing more than a static variable.

It's a static local variable. People seemed to have missed that my
proposal proposed adding the words to the LOCAL word set.

> I agree that Mark's proposal is pretty much just syntactic sugar for
> what could be done with a global variable. It is a step in the right
> direction though, so I wouldn't put it down.
>

I think it's a bit more than that ;-)

Sure, anything can be done with a global:

VARIABLE Count
: UpDownCounter ( true|false -- +1|-1 )
Count @ SWAP IF 1+ ELSE 1- THEN DUP Count ! ;

Can be written as:
: UpDownCounter ( true|false -- +1|-1 )


@ME SWAP IF 1+ ELSE 1- THEN DUP !ME ;

No variable.

It permits and promotes 'safety' - it's kind of like a mini object
with a single (static) instance variable. Well, a "static local" is
the best description I can come up with. They differ from LOCALS in
that LOCALS have to be carried to the word in question via the stack.
Thus to persist LOCALS, they have to be stored somewhere else, or
carried on the stack. This proposal doesn't require use of the stack.
It's just a different flavour of a local.

I think for multi-tasking programming it could come in handy. For
object reference counting it is useful. Or, if you just want to make
sure that a value can ONLY be altered in ONE place, and one place only
(critical sections) it is also useful. I think in embedded systems
inparticular.

Anyway, it's sounding like the idea is too limited in it's current
form. Trouble is, if I expand the idea, it would be seen as even more
radical! What to do? ;-)

Mark

Albert van der Horst

unread,
Mar 24, 2010, 7:10:23 AM3/24/10
to
In article <108dcfb1-efb3-4e11...@q23g2000yqd.googlegroups.com>,

Or even the Python convention
_pad \ This starts with an underscore. Private!
\ Don't use outside of the word set 1]

A naming convention doesn't stay in the way of debugging.

>
>Hans Bezemer

1] In the Brody sense.

--
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst

Andrew Haley

unread,
Mar 24, 2010, 7:20:57 AM3/24/10
to
MarkWills <markrob...@yahoo.co.uk> wrote:
>
> I think it's a bit more than that ;-)
>
> Sure, anything can be done with a global:
>
> VARIABLE Count
> : UpDownCounter ( true|false -- +1|-1 )
> Count @ SWAP IF 1+ ELSE 1- THEN DUP Count ! ;
>
> Can be written as:
> : UpDownCounter ( true|false -- +1|-1 )
> @ME SWAP IF 1+ ELSE 1- THEN DUP !ME ;
>
> No variable.
>
> It permits and promotes 'safety' - it's kind of like a mini object
> with a single (static) instance variable. Well, a "static local" is
> the best description I can come up with.

Sure, I think we all get that. The thing I'm having trouble with is
what advantage is gained from this 'safety'.

> They differ from LOCALS in that LOCALS have to be carried to the
> word in question via the stack. Thus to persist LOCALS, they have
> to be stored somewhere else, or carried on the stack. This proposal
> doesn't require use of the stack. It's just a different flavour of
> a local.
>
> I think for multi-tasking programming it could come in handy. For
> object reference counting it is useful. Or, if you just want to make
> sure that a value can ONLY be altered in ONE place, and one place only
> (critical sections) it is also useful. I think in embedded systems
> inparticular.
>
> Anyway, it's sounding like the idea is too limited in it's current
> form. Trouble is, if I expand the idea, it would be seen as even more
> radical! What to do? ;-)

The problem for me with this proposal is that it's not something that
isn't really easy to do in Forth already. Like this:

variable 'local-storage
: local-storage local-storage ! ;
: loc 'local-storage @ postpone literal ; immediate

... later ...

here 0 , local-storage


: UpDownCounter ( true|false -- +1|-1 )

loc @ SWAP IF 1+ ELSE 1- THEN DUP loc ! ;

Andrew.

alberto pasquale

unread,
Mar 24, 2010, 1:53:32 PM3/24/10
to
> Anyway, it's sounding like the idea is too limited in it's current
> form. Trouble is, if I expand the idea, it would be seen as even more
> radical! What to do? ;-)
>
> Mark

Mark,
I think is valid to explore new techniques and this is a good forum to
get feed-back.
You can continue to expand the idea and use in your work if you find
it beneficial.
What I don’t understand is the reason to propose as part of the
standard.

Alberto

Elizabeth D Rather

unread,
Mar 24, 2010, 3:45:33 PM3/24/10
to
MarkWills wrote:
...

> It has been suggested that 'radical' (my choice of word) proposals do
> not belong in the 200x, or to put it another way, radical, or
> experimental ideas do not belong in the standard. I *can* see the
> reasoning behind that, but at the same time, if that's the case, why
> not just freeze the standard?
>
> I'm sure Forth can benefit from borrowing ideas from other languages,
> such as atomicity, OOP etc. I think it is a shame that currently there
> seems to be some resistance to 'modernisation'. Again, I can see the
> reasoning - the standards people have a responsibility to make sure
> they don't introduce things that break existing code, for sure, no
> argument. But is it not possible to introduce new ideas without
> breaking existing systems? I would have thought so...
>
> Perhaps it's just me... Perhaps I jumped the gun a bit. Perhaps an
> idea needs to be proven in the field before it should be considered
> for standardisation. It will either stand the test of time or it
> won't. IIRC some paradigms that have been in use in VFX for many years
> are only being proposed just now.

All very reasonable questions. All languages evolve to meet changes
needs, Forth included. The issue here is, how can this evolution best
progress? I'll respond from the perspective of seeing Forth evolve for
nearly 40 years, having participated in Forth79, Forth83, and ANS Forth
standards processes.

In my view, evolution is most productive as programmers using the
language for challenging applications develop new tools. A tool or
approach in one application may evolve as it's used in others.
Different programmers evolve different strategies for similar
challenges, or similar strategies for different challenges.

Forth has benefitted significantly in that its development and evolution
has largely taken place in the cauldron of actual, practical use in
application problem-solving, rather than as theoretical or academic
exercises. Many concepts from other languages have found their way into
Forth, as they have been found useful to tackle practical challenges.

Forth has often been thought of as a province of solitary practitioners.
This has been far from the case. Starting in the 1970's, there have
been mechanisms for Forth programmers to share ideas, including FIG
groups in various places, the Rochester Conferences, and (most
productive of all, IMO), group projects. FORTH, Inc., MPE, and many
user organizations have supported teams of Forth programmers, benefiting
from the sharing of ideas and approaches while at the same time
practicing the kind of disciplines necessary to make group projects
successful: naming conventions, coding standards, formatting and
documentation disciplines, etc.

A Standards effort (regardless of the language) is a process in which
the needs presented by many experienced practitioners, and their
solutions to them, are evaluated and the best solutions incorporated in
a new Standard. It is not the place for experimentation or new ideas:
that needs to happen in application development over time, so that the
solutions presented are those that have been proven in use. The most
successful are solutions that have been developed independently by
different practitioners, with the Standards process serving as the forum
for comparing these solutions.

A Standard is valuable if, and only if, it is widely accepted. For that
to take place, it has to offer genuine positive improvements in the
sense of solutions to widely acknowledged needs, and be relatively free
of changes that will be expensive or disruptive for users to adopt.

In 1982 (when Forth83 was developed), many members of the Standards
group took the position that compatibility with past practice was
unnecessary, because there were so few users and in the future there
would be many more. At that time, Forth was getting a lot of publicity,
as the PC movement got well under way. Unfortunately, the disruption
caused by its discontinuities (e.g. change to NOT and DO-loops) and
limitations (e.g. requirement of a 16-bit, ITC model) in Forth83 slowed
that process, as many developers elected to stay with past practices
(either Forth79 or their own solutions to problems). Nonetheless, it
was influential, and many implementors followed it as much as possible
while meeting their needs (e.g. for 32-bit cell sizes & host OS
compatibility).

The ANS Forth process worked very hard to maintain consistency with
Forth83, while adapting to changes in the industry (source and data in
host OS files, different cell sizes, allowing for more efficient
implementation strategies, etc.). That plus the official cachet of ANSI
and ISO resulted in very widespread acceptance of ANS Forth. Virtually
all the widely-used versions of Forth, both public-domain or open-source
and commercial, follow that standard. And the Forth200x effort has
continued the philosophy of incremental steps to address
widely-perceived issues with solutions that have been proven in use.

There is definitely a place for experimentation and innovation. Right
now, the Forth community lacks good forums for exchanging ideas.
EuroFORML is somewhat successful (though difficult for many American
Forthers to participate in), and comp.lang.forth is also helpful. A lot
of people have been taking a good look at colorForth, for example,
although it doesn't appear to have penetrated serious application
development outside Chuck's companies. "Bright ideas" are always
interesting and welcome for discussion and evaluation, but they do not
belong in a Standard until they have been more widely tested and
evaluated in application use.

Cheers,
Elizabeth

--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com

"Forth-based products and Services for real-time
applications since 1973."
==================================================

The Beez'

unread,
Mar 25, 2010, 3:17:12 AM3/25/10
to
On 24 mrt, 12:10, Albert van der Horst <alb...@spenarnc.xs4all.nl>
wrote:

> >Not much more useful and generic?
>
> Or even the Python convention
>  _pad   \ This starts with an underscore. Private!
>         \ Don't use outside of the word set 1]
>
> A naming convention doesn't stay in the way of debugging.

No, but it is so unforth-like. To me it is always the amount of coding
I have to do to make those things work. E.g. I already have this
facility by having the word "HIDE":

variable me

: me-proposal
me ! lots of other words me @
;

hide me

variable me

: me-demo
me ! lots of other words me @
;

hide me

Problem solved. No things to "remember", no tagging, no flagging, very
flexible (since you determine how far the scope goes). And Forth-like
IMHO. Lots of words and few externals? Who said it was perfect?

Hans Bezemer

Andrew Haley

unread,
Mar 25, 2010, 5:36:02 AM3/25/10
to
Albert van der Horst <alb...@spenarnc.xs4all.nl> wrote:

I can't see any use for "private", but the Python convention of using
_ as a prefix for local static data seems to be very sensible. Given
that, one could tweak the "Redefined" warning message that most Forths
have so that it doesn't warn for locals. This is a clean, simple, and
very Forth solution.

Andrew.

Bruce McFarling

unread,
Mar 25, 2010, 11:26:41 AM3/25/10
to
On Mar 23, 11:07 am, MarkWills <markrobertwi...@yahoo.co.uk> wrote:
> Problem
> -------
> A colon definition that needs to maintain state between calls, or
> save pertinent associated data cannot do so without the use of
> VARIABLEs or VALUEs, which are external to the colon definition.
> A better approach is to store related data with the word itself,
> promoting encapsulation at the word level.

I agree with the tendency of discussion ... there is no problem in the
portable use of Forth scripts between implementations being solved
here, and there is indeed no problem *to* solve here.

A single colon definition *should not be* an entire process in and of
itself, it should be an item in a collection of definitions that
provide the process.

*That's factoring*.

To control scope given that a single definition *should not be*
atomic, but also given some words represent internal details to the
process that need not be visible, all that is required is a private
wordlist for the items that do not need to be accessible outside the
process, and there is no need for variables to be outside of the scope
of the *process*. The existing search order words provide the tools
required to provide for this. For a permanently hidden collection of
words, one can make a nameless wordlist, so that when the search order
is reset, it is sealed, or for a closed but available collection of
words, one can store the named wordlist in a constant or a vocabulary
word.

I'm not an implementer, so couldn't vote that I would not implement
the proposal, but as a sometime code author, I would never use the
proposal. It would be bad practice.

The Beez'

unread,
Mar 25, 2010, 2:44:20 PM3/25/10
to
On 25 mrt, 10:36, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:

> I can't see any use for "private", but the Python convention of using
> _ as a prefix for local static data seems to be very sensible.  Given
> that, one could tweak the "Redefined" warning message that most Forths
> have so that it doesn't warn for locals.  This is a clean, simple, and
> very Forth solution.
Violates the "no expectations" rule.

Hans

Andrew Haley

unread,
Mar 25, 2010, 3:21:33 PM3/25/10
to

What is the "no expectations" rule? Google turns up nothing.

What does this rule have to do with Forth?

Andrew.

Hugh Aguilar

unread,
Mar 25, 2010, 5:35:51 PM3/25/10
to
On Mar 24, 4:31 am, MarkWills <markrobertwi...@yahoo.co.uk> wrote:
> On 23 Mar, 23:16, Hugh Aguilar <hughaguila...@yahoo.com> wrote:
> > I agree that Mark's proposal is pretty much just syntactic sugar for
> > what could be done with a global variable. It is a step in the right
> > direction though, so I wouldn't put it down.
>
> I think it's a bit more than that ;-)
> ...

> It permits and promotes 'safety' - it's kind of like a mini object
> with a single (static) instance variable. Well, a "static local" is
> the best description I can come up with. They differ from LOCALS in
> that LOCALS have to be carried to the word in question via the stack.
> Thus to persist LOCALS, they have to be stored somewhere else, or
> carried on the stack. This proposal doesn't require use of the stack.
> It's just a different flavour of a local.

In the C language it is possible to make local variables STATIC. This
means that they are in static memory rather than being on the stack,
and they retain their values between invocations to the function. That
seems to be pretty much what you are trying to implement. The
advantage of using static locals rather than globals is that doing so
prevents other functions from accessing your variables. You can
accomplish this in Forth (as suggested by the Beez) by using a global
variable and then using HIDE after your function definition to remove
the global from the dictionary search so no other functions can access
it. One problem with this is that HIDE is not ANS-Forth standard; it
is one of those semi-standard words that shows up in a lot of
implementations. Another problem with the HIDE solution is that a
person reading the code might not notice the HIDE that comes afterward
and will be confused about what scope the variable has. HIDE is a bad
solution because it allows the programmer to write sloppy code and
then it provides a way to fix the problem afterward.

In my novice package I have the word { that defines a list of local
variables that are temporarily allocated during the duration of the
function invocation (usually on the return stack, but this is
implementation-dependent). Why don't you write a similar word
STATIC{ that defines a list of local variables that are allocated in
static memory? You could use my code for { as a guide for doing this.
Be sure to *not* initialize your static locals though. The whole point
of static locals is that they retain their values between invocations
of the function.

The Forth-200x committee is very opposed to standardizing code that
can be implemented using already-standard Forth. This is even true in
regard to something like STATIC{ that can only be implemented in a
convoluted and complicated manner that is beyond the ability of most
novice programmers. This is also why the committee killed my :NAME
proposal. It can be implemented in ANS-Forth --- I did so myself in my
novice package.

This is my major disagreement with the Forth-200x committee. I
consider the standardization of words such as :NAME and STATIC{ to be
a philosophical issue. By standardizing these things we give our
*approval* to their use. By refusing to standardize these words we
imply that they shouldn't be used. Also, as a practical matter, most
Forth programmers are not capable of implementing words like this
until they have significant experience in Forth. By this time they
have turned into zombies and they are no longer thinking creatively.
That is pretty much the point of Elizabeth Rather's novice classes at
Forth Inc. --- she turns people into zombies and discourages them from
thinking for themselves. She tells them that CREATE DOES> is the
*standard* way to write defining words, and makes no mention of :NAME
at all. The people become focused on writing idiomatic code because
this is the only way to graduate from the class. That is my definition
of a zombie --- a person who cares about being idiomatic. Most
corporations will only employ zombies and will *never* employ any
person that still has a spark of intelligence left within his withered
soul. That is not why I became a Forth programmer though --- if I had
wanted to become a zombie I would have joined the C community!

Bruce McFarling

unread,
Mar 25, 2010, 6:13:19 PM3/25/10
to
On Mar 23, 6:56 pm, "The Beez'" <hans...@bigfoot.com> wrote:
> Consequently, the BIG question is: is this the right proposal. We
> don't have many words standardized for limiting the scope of words or
> data allocation.

We don't?

What about WORDLIST GET-CURRENT SET-CURRENT GET-ORDER SET-ORDER?
Regulating the scope of words is exactly what they do.

alberto pasquale

unread,
Mar 25, 2010, 7:10:52 PM3/25/10
to
On Mar 25, 3:35 pm, Hugh Aguilar <hughaguila...@yahoo.com> wrote:
>
> of a zombie --- a person who cares about being idiomatic. Most
> corporations will only employ zombies and will *never* employ any
> person that still has a spark of intelligence left within his withered
> soul. That is not why I became a Forth programmer though --- if I had
> wanted to become a zombie I would have joined the C community!

I don’t know if you realize that you just characterized allot of
people you don’t know as “zombies” just because we happened to work
for corporations (I’m one of them)an others because they are part of
the C community.

I never sympathized with the philosophy of tools that try to protect
their users from their own stupidity.
I have to re-consider, you would be the perfect candidate to benefit
from such tools.
And I’m not referring to just programming tools…

Alberto

Hugh Aguilar

unread,
Mar 25, 2010, 7:24:22 PM3/25/10
to

One thing we don't have is a word that traverses through all of the
words in a wordlist and hits each of them with HIDE. This would be
very useful for implementing STATIC{ that I described above.
STATIC{ would define several static locals and put them all in a
wordlist (lets call it STATIC-LOCALS). When semicolon wraps up the
definition of the function, it would HIDE *everything* in STATIC-
LOCALS without having to know what words are in there --- it wouldn't
have to use HIDE explicitely on each variable name --- it would just
hide them all in one fell swoop.

Another problem with writing STATIC{ is that some Forth systems have
code and data in separate segments of memory, and other Forth systems
intermingle code and data. If code and data are separate, then
STATIC{ can just define the static locals using VARIABLE. The problem
is that, if code and data are intermingled, then STATIC{ can't use
VARIABLE because doing so would put the data right in the middle of
the function code. STATIC would have to first compile an absolute
branch over the data so the function execution doesn't crash into the
data.

Forth would be much improved if the standard required that code and
data *always* be in separate segments of memory. This would greatly
simplify constructs such as STATIC{. Most modern processors are
Harvard architecture anyway, so this is the usual circumstance. On old-
school processors that just have one big flat memory segment, it would
still be possible to segregate the code and data for the purpose of
conforming to this requirement. The whole business of intermingling
code and data goes back to the 1970s when there was so little memory
available that everything had to be scrunched together in order to
make use of every byte. That is not true anymore though.

Bruce McFarling

unread,
Mar 25, 2010, 9:08:37 PM3/25/10
to
On Mar 25, 7:24 pm, Hugh Aguilar <hughaguila...@yahoo.com> wrote:
> One thing we don't have is a word that traverses through all of the
> words in a wordlist and hits each of them with HIDE.

If you want to permanently hide the wordlist, leave it anonymous and
then once its out of the search order, its hidden. If you want to hide
the wordlist but make it possible to re-open it, make it a vocabulary.

The Beez'

unread,
Mar 26, 2010, 3:20:59 AM3/26/10
to
On Mar 25, 8:21 pm, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:

> What is the "no expectations" rule?  Google turns up nothing.
The "no expectations rule" is what makes Forth virtually syntax-less.
E.g. a missing semi-colon doesn't trigger a syntax error - may be an
error, but no syntax error. What you're introducing is some kind of
syntax. Note Chuck didn't really like the way you wrote a double
number (with a dot). He thought that was an ugly hack. This is even
worse. It simply pollutes a clean "syntax".

Hans Bezemer

The Beez'

unread,
Mar 26, 2010, 3:31:41 AM3/26/10
to

To me it is more like different namespaces. I'm talking about private
and public words. Although there are several clever schemes to achieve
just that, there is no easy equivalent for exposing only the public
words of (e.g.) a library to the outside world.

I'm not waiting for an actual standardization effort, because what
will probably come out is an ugly, bloated, C-like, overengineered
proposal like SYNONYM, S\" or BEGIN-STRUCTURE that I will need to
support somehow. HIDE works fine for me - and a few other tiny Forths
that have long given up the standard.

Hans Bezemer

The Beez'

unread,
Mar 26, 2010, 3:41:38 AM3/26/10
to

I agree - and I don't understand the rant. C gives you enough rope to
hang yourself, unlike Pascal and stuff. Note many good Forth
programmers know their C very well too - the author included. C is
just a different beast with a different philosophy to solve problems.
You can just write a tid bit more murky programs in C and get away
with it than you would with Forth. E.g. there are loads of C
programmers that can't balance their malloc() and free() calls - and
get away with it. Balancing a stack is much, much harder.

But also note there are loads of Forth programs with 1 or 2 page
(legal/A4) words. How people can get these to work - let alone
maintain is beyond me too.

There is a reason why blocks have only 16 lines.

Hans Bezemer

Andrew Haley

unread,
Mar 26, 2010, 4:30:50 AM3/26/10
to
The Beez' <han...@bigfoot.com> wrote:
> On Mar 25, 8:21?pm, Andrew Haley <andre...@littlepinkcloud.invalid>
> wrote:
>> What is the "no expectations" rule? ?Google turns up nothing.

> The "no expectations rule" is what makes Forth virtually syntax-less.
> E.g. a missing semi-colon doesn't trigger a syntax error - may be an
> error, but no syntax error.

"?" (i.e. word not found) isn't described as a syntax error, but it's
an error. It's not described as a syntax error, since in Forth there
is no separation of syntax and actions.

> What you're introducing is some kind of syntax.

It's a convention that seems to me to be pretty useful, far more
useful IMO than an apparently arbitrary rule. Besides, the
"redefined" warning isn't part of the language anyway, so can't have
any effect on it.

> Note Chuck didn't really like the way you wrote a double number
> (with a dot). He thought that was an ugly hack.

Well, yes.

> This is even worse. It simply pollutes a clean "syntax".

I don't understand why a prefix convention pollutes a clean "syntax".
Forth is chock-full of prefix conventions.

Andrew.

Albert van der Horst

unread,
Mar 26, 2010, 10:18:55 AM3/26/10
to
In article <2e969297-ce35-42b4...@w39g2000prd.googlegroups.com>,

Hugh Aguilar <hughag...@yahoo.com> wrote:
>On Mar 25, 4:13 pm, Bruce McFarling <agil...@netscape.net> wrote:
>> On Mar 23, 6:56 pm, "The Beez'" <hans...@bigfoot.com> wrote:
>>
>> > Consequently, the BIG question is: is this the right proposal. We
>> > don't have many words standardized for limiting the scope of words or
>> > data allocation.
>>
>> We don't?
>>
>> What about WORDLIST GET-CURRENT SET-CURRENT GET-ORDER SET-ORDER?
>> Regulating the scope of words is exactly what they do.
>
>One thing we don't have is a word that traverses through all of the
>words in a wordlist and hits each of them with HIDE. This would be
>very useful for implementing STATIC{ that I described above.
>STATIC{ would define several static locals and put them all in a
>wordlist (lets call it STATIC-LOCALS). When semicolon wraps up the
>definition of the function, it would HIDE *everything* in STATIC-
>LOCALS without having to know what words are in there --- it wouldn't
>have to use HIDE explicitely on each variable name --- it would just
>hide them all in one fell swoop.

This is very similar to the PRIVATES PRIVATE DEPRIVE wordset in
tforth. (1994)
All words not to be used outside a module are marked PRIVATE.
This sets a bit in the header.
DEPRIVE scans all words till the previous PRIVATES and hides all
private words.
During debugging PRIVATES and DEPRIVE are noops.

Groetjes Albert

Bruce McFarling

unread,
Mar 26, 2010, 10:23:03 AM3/26/10
to
On Mar 26, 3:31 am, "The Beez'" <hans...@bigfoot.com> wrote:
> To me it is more like different namespaces. I'm talking about private
> and public words. Although there are several clever schemes to achieve
> just that, there is no easy equivalent for exposing only the public
> words of (e.g.) a library to the outside world.

But different namespaces *is* the easy way to expose only the public
words of a library to the outside world. Put the public words in the
current namespace when the library is called, and the private words in
a namespace defined by the library.

The Beez'

unread,
Mar 26, 2010, 12:56:06 PM3/26/10
to
On 26 mrt, 09:30, Andrew Haley <andre...@littlepinkcloud.invalid>
wrote:

> I don't understand why a prefix convention pollutes a clean "syntax".
> Forth is chock-full of prefix conventions.
Most of 'em introduced since the Forth "corrosion point" Forth-83. I
do not and will not support these. Note words like S", .", ['],
[char], etc. aren't prefixes, but words.

- If it's a word, execute it.
- If not, try converting it to a number.
- If it's not a number, FAIL!

That is the golden rule - and avoid as many other "IF"s while doing
it.

Hans Bezemer

John Passaniti

unread,
Mar 26, 2010, 1:27:20 PM3/26/10
to
On Mar 26, 3:41 am, "The Beez'" <hans...@bigfoot.com> wrote:
> There is a reason why blocks have only 16 lines.

Yes. And that reason is that most paper and video terminals at the
time Forth was invented could do between 64 and 80 characters per
line. So they chose 64, which was the lowest common denominator
terminal width that divided the size of disk block evenly.

I know this is going to blow people's minds, but independent of the
language I program in, I do my very best that I very rarely write a
definition that extends past about 16 or so lines. Amazing-- and I do
this entirely without artificial limits being placed on me by my
editor! It's like I have some weird ability not to do things that are
bad for me!

Coos Haak

unread,
Mar 26, 2010, 2:01:43 PM3/26/10
to
Op Fri, 26 Mar 2010 09:56:06 -0700 (PDT) schreef The Beez':

My recipe:


- If it's a word, execute it.
- If not, try converting it to a number.

- If not, try converting it to a string.
- If it's not a word, number or string, fail.

--
Coos

CHForth, 16 bit DOS applications
http://home.hccnet.nl/j.j.haak/forth.html

Coos Haak

unread,
Mar 26, 2010, 2:06:20 PM3/26/10