Why smudge?
Stephen J Bevan
found during a dictionary lookup? If not, then its only use would
seem to be to allow a word to be defined in terms of another (older)
word with the same name. Are there some useful things that can be
done with this ability that make up for the (minor) extra complication?
Chris Waters
No, SMUDGE (and its associated "smudge-bit") serves no other useful
purpose. No, there's no reason why you couldn't replace it with
something more efficient. Its not part of any Forth standard, and only
exists in a few, mostly very old, Forth systems.
The *functionality*, however, is required by all Forth standards. You
*must* me able to define a word in terms of an older word with the same
name in Forth. But this is more easily accomplished by simply not
linking the current definition into the dictionary until it is complete.
SMUDGE is a kludge, and not a particularly clever one, IMO. Its
presence does not bode well for the brand of Forth in question (whatever
that may be). If it weren't for the rather mediocre, though popular-
in-its-time, Fig-Forth model, SMUDGE would probably have died a well-
deserved death many years ago.
--
Chris Waters | I think, there- | "Never look a gift horse in the mouth"
xt...@netcom.COM| fore I thwim. | --Priam, King of Troy
Roger Bicknell
Being able to see previous versions of a word is important when RECURSING or
when updating a word by incorporating it into a new definition having the
same name. Of course, in order to recurse one wants to see the current definition;
while updating, one wants to see only previous def.
Stephen J Bevan
The *functionality* [ that "smudge" provided ], however, is
required by all Forth standards. You *must* me able to define a
word in terms of an older word with the same name in Forth.
Ok, so could you answer the second part of the question which is _why_
is it necessary to be able to define a word in this way i.e. what
practical benefit is it?
Chris Waters
Well, the reason that the standard requires it is because of existing
practice. Why is it existing practice? Well, consider the alternative.
A previous definition *cannot* be found. But what if you *want* to
access the previous definition? You're out of luck. IOW, the practical
benefit is the availablity of the feature.
Recursion is available in either case, so that's not an issue.
Another consideration is that an incomplete definition should *not* be
executed. For this reason, it is best not to make the word visible
until the definition is successfully compiled. If it is not visible, an
older version will be. This, in fact, is probably the real reason
behind the existing practice. The fact that it results in a more
flexible and powerful system is a fringe benefit. :-)
Stephen J Bevan
Well, the reason that the standard requires it is because of existing
practice. Why is it existing practice? Well, consider the alternative.
A previous definition *cannot* be found. But what if you *want* to
access the previous definition? You're out of luck.
Could you not alias the old definition (i.e. ": old-foo foo ;" or a
true alias) and use that to access the old definition?
Another consideration is that an incomplete definition should *not* be
executed. For this reason, it is best not to make the word visible
until the definition is successfully compiled.
This is something more tangible. However, is there no use that could
be made of executing a partially defined word? I can't think of one
off hand, but then I can't think of why I'd want to access an old
definition without renaming either :-)
Heribert Dahms
: In article <xtifrC9...@netcom.com> xt...@netcom.com (Chris Waters) writes:
: Well, the reason that the standard requires it is because of existing
: practice. Why is it existing practice? Well, consider the alternative.
: A previous definition *cannot* be found. But what if you *want* to
: access the previous definition? You're out of luck.
:
: Could you not alias the old definition (i.e. ": old-foo foo ;" or a
: true alias) and use that to access the old definition?
:
Good point. That's the way examples in dpANS are written since this Standard
says it's implementation defined wether incomplete words are FINDable or not.
And recursions are to be programmed using RECURSE, easy to remember 8-)
:
: Another consideration is that an incomplete definition should *not* be
: executed. For this reason, it is best not to make the word visible
: until the definition is successfully compiled.
:
: This is something more tangible. However, is there no use that could
: be made of executing a partially defined word? I can't think of one
: off hand, but then I can't think of why I'd want to access an old
: definition without renaming either :-)
Agreed.
I would never try to execute a partially defined word. It might hang, loop,
run wild, produce garbage, destroy important data on disk, crash the system...
If anybody thinks a partially definition might be useful, why not simply
cut the semantics of that word and properly end it? Let's factor out!
Bye, Heribert (da...@ifk20.mach.uni-karlsruhe.de)
Julian V. Noble
My Uniforth for a Definicon board has SMUDGE. It did not have RECURSE.
So I defined RECURSE as
: RECURSE SMUDGE ; IMMEDIATE
(this of course requires you to say the name of the word being recursed to
after saying SMUDGE; if I had known the dictionary structure better I would
probably have done it somewhat differently.
I think Uniforth is based directly on F83, so I presume that means Laxen-
Parry F83 had SMUDGE.
I agree SMUDGE is a KLUDGE.
--jvn
Chris Waters
>My Uniforth for a Definicon board has SMUDGE. It did not have RECURSE.
>So I defined RECURSE as
> : RECURSE SMUDGE ; IMMEDIATE
I would have called this word "RECURSIVE", since it merely marks a
definition as allowing recursion, rather than actually doing the
recursion. In dpANS, "RECURSE" recurses. Thus:
: HANG RECURSIVE HANG ;
is exactly the same as:
: HANG RECURSE ;
Of course, only RECURSE is found in dpANS, since having both is somewhat
redundant. :-)
>(this of course requires you to say the name of the word being recursed to
>after saying SMUDGE; if I had known the dictionary structure better I would
>probably have done it somewhat differently.
Also, since SMUDGE is a toggle (which makes it even *more* confusing,
IMO), you would have to execute SMUDGE one more time after you complete
the definition, to make the word visible again, since the SMUDGE in
semicolon, which is supposed to reveal the word, will hide it instead.
>I think Uniforth is based directly on F83, so I presume that means Laxen-
>Parry F83 had SMUDGE.
Actually, while I don't have it on-line at the moment, I'm fairly sure
that l&P used HIDE and REVEAL, rather than SMUDGE. Not only is it not a
confusing toggle, but the names actually make sense, unlike "SMUDGE"! :-)
>I agree SMUDGE is a KLUDGE.
There's also a minor bug in SMUDGE (which is also a bug in *some*
implementations of IMMEDIATE). If you change vocabularies, you'll end
up SMUDGing the wrong word.
I believe that L&P used a variable, LAST, which pointed to the most
recent definition, irrespective of vocabulary. HIDE would actually
*remove* the word from the current vocabulary, so that nothing in the
system *except* LAST was pointing to it. Note that with this scheme,
there is no smudge bit in the header. F-PC (another system derived from
L&P) works this way. With this scheme, IMMEDIATE will work correctly
because it uses LAST to find the word to make immediate.
And, of course, you can define:
: RECURSIVE REVEAL ; IMMEDIATE
Which doesn't require any tweaking afterwards, or (this is more system
dependent):
: RECURSE LAST @ NAME> , ; IMMEDIATE
OC, since the definition of RECURSE is system dependent, it should be
part of the system, so it's a good thing that it's in the standard! :-)
Roger Bicknell
>I think Uniforth is based directly on F83, so I presume that means Laxen-
>Parry F83 had SMUDGE.
>I agree SMUDGE is a KLUDGE.
Not so quick! SMUDGE is perhaps a lesser "elegant" solution for controlling the
visibility of a word as a normal Forth system would use... but it does work *and*
it is a very simple solution to control of visiblity of local definitions.
For example, one could (with a suitable segmented system) define other definitions
within the space of a colon word while it is still being defined. At this time
it is SMUDGEd out. At the end of the word one can then follow the chain of words back,
toggling SMUDGE bit on each word, until the colon word is toggled - thus hiding all
locally defined words and making visible the colon word in one fell swoop!
This is the simplest way to do this (as opposed to some kind of linked-list tree, etc).
You may say that SMUDGE is an unelegant hack. I look at it as a simple and unstructured
(read "uncluttered") minimalist solution.
PS. Personally, for *normal* Forth systems, I prefer the more modern solution - it is
faster to run and has less control structures (structure? no flag ;) .
Andrew Haley
A good implementation of Forth treats the smudge bit as the most
significant bit of the length of the name; that way a smudged word
won't be found as its name seems to be too long. There's no cost in
run time.
SMUDGE is a pretty simple solution to the problem which costs very
little.
Andrew.
Roger Bicknell
>In article <1993Jun29.1...@mdd.comm.mot.com> bick...@mdd.comm.mot.com writes:
>>In <C9CFn...@murdoch.acc.Virginia.EDU> j...@fermi.clas.Virginia.EDU (Julian V.
>> Noble) writes:
>>
>>You may say that SMUDGE is an unelegant hack. I look at it as a simple and
>> unstructured
>>(read "uncluttered") minimalist solution.
>>
>>
>>PS. Personally, for *normal* Forth systems, I prefer the more modern solution -
>> it is faster to run and has less control structures (structure? no flag ;) .
>A good implementation of Forth treats the smudge bit as the most
>significant bit of the length of the name; that way a smudged word
>won't be found as its name seems to be too long. There's no cost in
>run time.
"..as its name seems to be too long..." Seems to whom? This all
requires logic and time to process while compiling. While, normally,
no words are SMUDGEd out in the dic ('cepting the last partial def.)
is is costly to repeatedly check for such while FINDing a word. Having
a word removed from the search chain is more efficient.
I note that if one wanted a local def. functionality within the dic. that
using a linked-list solution can get expensive in time and complex in
implimentation (compared to just using a SMUDGE bit).
.roger.
Chris Waters
>In article <1993Jun29.1...@mdd.comm.mot.com> bick...@mdd.comm.mot.com writes:
>>You may say that SMUDGE is an unelegant hack. I look at it as a simple and
>> unstructured
>>(read "uncluttered") minimalist solution.
Seems like not linking the word into the dictionary is an even simpler
and less cluttered solution. :-)
I admit that bicknell's example of an exotic use for smudge (a *highly*
system dependent use, at that, which depends heavily on how the
dictionary is threaded) was interesting and thought provoking. I don't
necessarily think that it's the best approach to the problem, however.
Especially since it depends just as much on the dictionary threading
structure as any alternative would. Nevertheless, I haven't analyzed it
in detail, so I'm not going to condemn it out of hand. But I think that
it's a rare case in in case.
>>PS. Personally, for *normal* Forth systems, I prefer the more modern solution -
>> it is faster to run and has less control structures (structure? no flag ;) .
And is more readable to boot! :-)
>A good implementation of Forth treats the smudge bit as the most
Hrm, I'm not sure that I'd concede that a "good implementation" of Forth
would even *have* a smudge bit!! :-)
>significant bit of the length of the name; that way a smudged word
>won't be found as its name seems to be too long. There's no cost in
>run time.
*Ahem* It still has to determine that the length is out of range (a
test that it shouldn't even have to make in the first place!), and then
traverse to the next link and follow the link to the next header. That
may not be a *high* run-time cost, but it is most certainly a run-time
cost! And, on an 8-bit embedded machine, it may even be an
unacceptably high cost, small as it is.
>SMUDGE is a pretty simple solution to the problem which costs very
>little.
True, but hardly as simple as unlinking/relinking (the links have to be
built anyway, so the linking code is already in the system), and *far*
more confusing and obscure. SMUDGE offers no discernable benefits, and
has some obvious, though minor, disadvantages. Technical comparisons
aside (production code is unlikely to have any unfinished/smudged words
in the dictionary), SMUDGE is badly named and unpredictable (it either
hides or reveals a word--if you're lucky, it'll do the one you want).
Forth already has enough potential to be hard-to-read and confusing.
SMUDGE just makes this worse. HIDE and REVEAL, on the other hand, are
readable and predictable, and don't depend on exotica such as using
"unused bits in a byte".
Andrew Haley
>In <741392...@aph.demon.co.uk> a...@aph.demon.co.uk (Andrew Haley) writes:
>>A good implementation of Forth treats the smudge bit as the most
>
>Hrm, I'm not sure that I'd concede that a "good implementation" of Forth
>would even *have* a smudge bit!! :-)
>
>>significant bit of the length of the name; that way a smudged word
>>won't be found as its name seems to be too long. There's no cost in
>>run time.
>
>*Ahem* It still has to determine that the length is out of range (a
>test that it shouldn't even have to make in the first place!), and then
>traverse to the next link and follow the link to the next header.
^^^^^^^^^^^^^^^^^^^^^^^^^
Traversal of the name is only done on really bad Forth systems; in a
good system the links are chained together so that once a match on
the length has failed you can skip directly to the next word.
>That may not be a *high* run-time cost, but it is most certainly a run-time
>cost! And, on an 8-bit embedded machine, it may even be an
>unacceptably high cost, small as it is.
Oh, I see, you're just talking about the match attempt on the word
currently being defined. OK, not zero, just vanishingly small.
>>SMUDGE is a pretty simple solution to the problem which costs very
>>little.
>
>True, but hardly as simple as unlinking/relinking (the links have to be
>built anyway, so the linking code is already in the system), and *far*
>more confusing and obscure.
Taking the linking code out of CREATE requires an extra word (not too
bad) and an extra USER variable in _every_ terminal task to point to
the word currently being defined (pretty bad). Also, to prevent a
word being linked in twice (with catastrophic results) it's necessary
to check that it's not already linked. Also, the code to unlink a
word (which isn't needed at present) is pretty complex.
It's an increase in complexity for no apparent (to the user)
improvement in usage.
>SMUDGE offers no discernable benefits, and
>has some obvious, though minor, disadvantages. Technical comparisons
>aside (production code is unlikely to have any unfinished/smudged words
>in the dictionary), SMUDGE is badly named and unpredictable (it either
>hides or reveals a word--if you're lucky, it'll do the one you want).
True. That can be a real problem in some circumstances.
>Forth already has enough potential to be hard-to-read and confusing.
>SMUDGE just makes this worse. HIDE and REVEAL, on the other hand, are
>readable and predictable, and don't depend on exotica such as using
>"unused bits in a byte".
Oh, I don't mind HIDE and REVEAL as alternative names to SMUDGE; I've
had to do that myself whilst metacompiling. I just don't see what
the problem is with using a bit in the length byte to do the job.
Any "improvements" to Forth must either offer significant
improvements in performance or be simpler to implement. I don't
think this suggestion offers either.
Andrew.
Heribert Dahms
: In article <xtifrC9...@netcom.com> xt...@netcom.com writes:
:
: >In <741392...@aph.demon.co.uk> a...@aph.demon.co.uk (Andrew Haley) writes:
:
: >>A good implementation of Forth treats the smudge bit as the most
[...]
: >>significant bit of the length of the name; that way a smudged word
: >>won't be found as its name seems to be too long. There's no cost in
: >>run time.
: >
: >*Ahem* It still has to determine that the length is out of range (a
: >test that it shouldn't even have to make in the first place!), and then
: >traverse to the next link and follow the link to the next header.
: ^^^^^^^^^^^^^^^^^^^^^^^^^
: Traversal of the name is only done on really bad Forth systems; in a
: good system the links are chained together so that once a match on
: the length has failed you can skip directly to the next word.
:
: >That may not be a *high* run-time cost, but it is most certainly a run-time
: >cost! And, on an 8-bit embedded machine, it may even be an
: >unacceptably high cost, small as it is.
:
Checking the length first can be faster as it is some sort of hashing, even
on a 8bit machine.
On a 16bit machine, e.g. PDP-11, it is possible to check the length byte and
the name's first character at the same time with only one word-instruction.
Then, also doing a SMUDGE test by reserving a high-bit in the length byte
comes for free run-time cost. See PDP-11 FIG Forth.
The results may be even more dramatically on a 32bit or 64bit machine!
Remaining characters, if any, are only compared if necassary, of course.
[...]
: Oh, I don't mind HIDE and REVEAL as alternative names to SMUDGE; I've
: had to do that myself whilst metacompiling. I just don't see what
: the problem is with using a bit in the length byte to do the job.
:
Agreed.
: Any "improvements" to Forth must either offer significant
: improvements in performance or be simpler to implement. I don't
: think this suggestion offers either.
:
Or be more portable, but this may not be your criterion.
And if portability is the issue, the complexity problem is up the implementor,
not to the normal user, as portable programs can't make assumptions about the
dictionary structure or they are environmentally dependent.
Bye, Heribert (da...@ifk20.mach.uni-karlsruhe.de)
Anton Ertl
|> Well, the reason that the standard requires it is because of existing
|> practice. Why is it existing practice? Well, consider the alternative.
|> A previous definition *cannot* be found. But what if you *want* to
|> access the previous definition? You're out of luck.
How about
: bar foo ;
: foo ... bar ... ;
This is slow and does not look very nice, but then what you are doing
is not very nice either.
- anton
--
M. Anton Ertl Some things have to be seen to be believed
an...@mips.complang.tuwien.ac.at Most things have to be believed to be seen
Anton Ertl
|>
|> is exactly the same as:
|>
|> : HANG RECURSE ;
|>
|> Of course, only RECURSE is found in dpANS, since having both is somewhat
|> redundant. :-)
Yes, but there are cases where RECURSIVE works, but RECURSE does not.
Just yesterday I have written code that looks like
: foo ... ['] foo ... ;
Now I have to figure out how to run this. Using a deferred word is
easy, but ugly.
While we are at it, IMO RECURSIVE is also better from the viewpoint of
readability: When I write a recursive word (e.g. sort), I don't think
"Now it should recurse". I think "Now sort this part". And a reader
should think so, too.
Standardizing RECURSIVE, not RECURSE would also eliminate the
headaches people have with using RECURSE in DOES>-code and in
anonymous words (of course, these are the cases where RECURSE can
express things that RECURSIVE cannot).
Doug Philips
an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
+How about
+
+: bar foo ;
+
+: foo ... bar ... ;
+
+This is slow and does not look very nice [...]
And it isn't an exact replacement either. It fails to be an exact
replacement because it allows access to the original word (foo) via the
intermediate word (bar).
+but then what you are doing is not very nice either.
Oh? What about adding more control structure "security" than the
underlying system provides? Or adding more debugging information than the
underlying system provides? It is very elegant to be able to redefine a
word in terms of its old definition, and by doing it the way Forth does,
you can be sure that the old definition will not be accessed when new
words are added to the system.
-Doug
Doug Philips
DA...@ifk20.mach.uni-karlsruhe.de (Heribert Dahms) writes:
+Checking the length first can be faster as it is some sort of hashing, even
+on a 8bit machine.
+On a 16bit machine, e.g. PDP-11, it is possible to check the length byte and
+the name's first character at the same time with only one word-instruction.
+Then, also doing a SMUDGE test by reserving a high-bit in the length byte
+comes for free run-time cost. See PDP-11 FIG Forth.
+The results may be even more dramatically on a 32bit or 64bit machine!
+Remaining characters, if any, are only compared if necassary, of course.
But the point is that you are still making a test that you _KNOW_, a
priori, will _ALWAYS_ fail. Why bother making such a test at all? I
think the "don't link the word into the dictionary until its definition
is complete" is a much more elegant solution.
-Doug
Chris Waters
>In article <xtifrC9...@netcom.com> xt...@netcom.com writes:
>>In <741392...@aph.demon.co.uk> a...@aph.demon.co.uk (Andrew Haley) writes:
>>*Ahem* It still has to determine that the length is out of range (a
>>test that it shouldn't even have to make in the first place!), and then
>>traverse to the next link and follow the link to the next header.
> ^^^^^^^^^^^^^^^^^^^^^^^^^
>Traversal of the name is only done on really bad Forth systems; in a
So you're saying that the name field doubles as the link? :-)
You have to do *something* to get to the link field, even if it's merely
a "1 CELLS -".
>>That may not be a *high* run-time cost, but it is most certainly a run-time
>>cost! And, on an 8-bit embedded machine, it may even be an
>>unacceptably high cost, small as it is.
>>>SMUDGE is a pretty simple solution to the problem which costs very
>>>little.
>>
>>True, but hardly as simple as unlinking/relinking (the links have to be
>>built anyway, so the linking code is already in the system), and *far*
>>more confusing and obscure.
>Taking the linking code out of CREATE requires an extra word (not too
>bad)
To replace SMUDGE, you mean? I'd hardly call that *extra*. :-)
>and an extra USER variable in _every_ terminal task to point to
>the word currently being defined (pretty bad).
Maybe. Most people seem to think it's worth it.
>Also, to prevent a
>word being linked in twice (with catastrophic results) it's necessary
>to check that it's not already linked.
It is? That's funny, I've never seen that done. "LAST @ CURRENT @ !"
will not link twice no matter how many times you call it.
> Also, the code to unlink a
>word (which isn't needed at present) is pretty complex.
Sure didn't seem complex in any of the myriad systems I've seen it in!
"LAST @ @ CURRENT @ !" is pretty typical.
>It's an increase in complexity for no apparent (to the user)
>improvement in usage.
Then why is it found in every Forth system I've seen in the last decade?
It's no more complex than trying to juggle individual bits (which
someone has arbitrarily decided are "unused", a decision I'm *not* sure
I like!) As for improvement in usage, I would argue that that is the
whole *point* of the scheme!
>Oh, I don't mind HIDE and REVEAL as alternative names to SMUDGE; I've
>had to do that myself whilst metacompiling. I just don't see what
>the problem is with using a bit in the length byte to do the job.
Munging bits is always awkward at best, and can be a downright PITA on
some machines. Furthermore, it mangles the length byte! Why have the
length byte not equal the length? That only adds to the confusion and
the difficulty in understanding Forth.
>Any "improvements" to Forth must either offer significant
>improvements in performance or be simpler to implement. I don't
>think this suggestion offers either.
Well, it's hard to argue with someone who has already made his mind up.
But I think it's a trifle arrogant to talk about an "improvement" to
Forth when discussing something that is more common than the alternative
that you're advocating. I haven't seen a Forth system that uses SMUDGE
in nearly a decade.
I might turn the question around, and say, what possible advantages can
accrue from this strange SMUDGE mechanism you're proposing? The way
that Forth works now, with HIDE and REVEAL linking and unlinking the
word, works fine, and doesn't rely on odd concepts like "unused bits"
and bit toggles and stranges hacks to (FIND).
Andrew Haley
>In <741520...@aph.demon.co.uk> a...@aph.demon.co.uk (Andrew Haley) writes:
>
>>Also, to prevent a
>>word being linked in twice (with catastrophic results) it's necessary
>>to check that it's not already linked.
>
>It is? That's funny, I've never seen that done. "LAST @ CURRENT @ !"
>will not link twice no matter how many times you call it.
That doesn't work too well if you want to hide a word that's not the
top of the dictionary. Maybe that's not too important.
>> Also, the code to unlink a
>>word (which isn't needed at present) is pretty complex.
>
>Sure didn't seem complex in any of the myriad systems I've seen it in!
>"LAST @ @ CURRENT @ !" is pretty typical.
Um. Okay, I guess that you're using a rather simpler dictionary
structure than I'm used to where this thing is somewhat easier.
>>It's an increase in complexity for no apparent (to the user)
>>improvement in usage.
>
>Then why is it found in every Forth system I've seen in the last decade?
I've no idea. That's what I'm trying to discover!
>>Any "improvements" to Forth must either offer significant
>>improvements in performance or be simpler to implement. I don't
>>think this suggestion offers either.
>
>Well, it's hard to argue with someone who has already made his mind up.
I haven't. Really. In fact, I'm going to try it to see if it makes
the system any faster or better. The whole point of my line of
argument is to try to get the proponents of this scheme fully to
describe its advantages. I'm not opposed to changing things; when it
comes to dictionary structure, there are some great ways to improve
performance: the best that I know of is hashing the dictionary,
rather than using a linked list.
>But I think it's a trifle arrogant to talk about an "improvement" to
>Forth when discussing something that is more common than the alternative
>that you're advocating. I haven't seen a Forth system that uses SMUDGE
>in nearly a decade.
Ah. That's something I really didn't know. Almost all of my Forth
experience is with polyFORTH, and I assumed that the deletion of the
smudge bit was something that was being proposed rather than
something that was common usage. Mea culpa.
How is the code organized? In the systems I know, CREATE creates a
word and links it in. CREATE is called by : . Presumably, you need
two versions of CREATE, one which does and one which doesn't link in
the word. Or do you have a flag so that CREATE knows what kind of
thing it's creating?
If you have a vectored CREATE, do you need to vector both versions?
Andrew.
Peter Churchyard
14 years ago when I was new to forth, I decided that to reference the
current verb being defined need a simple verb which I called
recursive
This was a simple thing to implement and didn't have to fiddle with the
dictionary.
Pete.
Stephen J Bevan
In article <20uk4l$k...@nz12.rz.uni-karlsruhe.de>,
DA...@ifk20.mach.uni-karlsruhe.de (Heribert Dahms) writes:
+Checking the length first can be faster as it is some sort of
+... it is possible to check the length byte and
+the name's first character at the same time with only one word-instruction.
+Then, also doing a SMUDGE test by reserving a high-bit in the length byte
+comes for free run-time cost. See PDP-11 FIG Forth.
But the point is that you are still making a test that you _KNOW_, a
priori, will _ALWAYS_ fail. Why bother making such a test at all? I
think the "don't link the word into the dictionary until its definition
is complete" is a much more elegant solution.
Well after asking what I thought was a simple question I'll have to
admit to being confused by most of the replies :-(. So at the risk of
showing even more naivette: Assuming a linear linked dictionary, why
not link word being searched for to the _end_ of the dictionary, so
that it will always be found? The purpose of this is to avoid the "is
end of dictionary" test after following the link from the previous
header. Once the word is found, it is only necessary to check if the
address is the same the one at the end of the dictionary to determine
if it is an attempted recursive definition or if a previous definition
has been found. Using this technique, there isn't much point to a
SMUDGE bit, the required functionality is tacked on after the address
comparision. Comments?
Chris Waters
>In article <xtifrC9...@netcom.com> xt...@netcom.com writes:
>>In <741520...@aph.demon.co.uk> a...@aph.demon.co.uk (Andrew Haley) writes:
>>
>>>Also, to prevent a
>>>word being linked in twice (with catastrophic results) it's necessary
>>>to check that it's not already linked.
>>
>>It is? That's funny, I've never seen that done. "LAST @ CURRENT @ !"
>>will not link twice no matter how many times you call it.
>That doesn't work too well if you want to hide a word that's not the
>top of the dictionary. Maybe that's not too important.
No more than it is with "CURRENT @ @ DUP C@ $20 XOR SWAP !" (aka
SMUDGE). :-)
>>> Also, the code to unlink a
>>>word (which isn't needed at present) is pretty complex.
>>
>>Sure didn't seem complex in any of the myriad systems I've seen it in!
>>"LAST @ @ CURRENT @ !" is pretty typical.
>Um. Okay, I guess that you're using a rather simpler dictionary
>structure than I'm used to where this thing is somewhat easier.
It's a little more complex if you hash the vocabulary into multiple
threads, but not greatly. OC, the complexity of SMUDGE is also
increased when you hash the dictionary. It's a matter of finding the
thread that the latest word has been added to. And here, the user
variable LAST gives you a big advantage, since you can just use the
variable to find the thread, instead of comparing the tops of all the
threads to see which one has a higher address (a test that may not be
valid if your threads go through different areas of memory, i.e. you
haven't built your dictionary in a strictly linear fashion, which is a
fairly common thing to do when creating temporary definitions above the
normal dictionary).
For comparison, here's the definitions of HIDE and REVEAL from F-PC:
: HIDE ( -- )
LAST @ DUP N>LINK Y@ SWAP CURRENT @ YHASH ! ;
I think that the "Y@" has to do with the segmented architecture, but
I'm not sure. Note how the presence of the LAST variable makes it easy
to do the hashing to find the appropriate thread. N>LINK is an alias
for 2-, btw.
: REVEAL ( -- )
LAST @ DUP N>LINK SWAP CURRENT @ YHASH ! ;
Now, if we use the same linking scheme, but assume no LAST variable, and
try to implement SMUDGE, we end up with:
: LATEST ( -- alf )
0 CURRENT @ DUP #THREADS CELLS +
SWAP DO I @ UMAX 1 CELLS +LOOP ;
and
: SMUDGE ( -- )
LATEST L>NAME DUP C@ $20 XOR SWAP C! ;
Now, which scheme seems more complex? Note that we can't use the YHASH
function, since that requires that we have the name to hash, which we
don't unless we have a variable like LAST pointing to the most recent
definition. Thus we have to sort through all the threads by hand with a
loop. Granted, all the overhead of LATEST will only be stored in the
dictionary once, while the user variable LAST has one occurance per
terminal task. And I admit that it hadn't occured to me that this might
be a big deal (I rarely have more than three or four tasks). But there
are obvious speed and simplicity gains from the variable.
Note that this will also break down completely if I try one of the
temporary definition schemes that basically involve pointing HERE to
another place in memory (usually up high somewhere). Without LAST to
point to the most recent definition, I *cannot* temporarily move the
dictionary pointer up into high memory somewhere without breaking this
definition of LATEST.
So there's the argument for the LAST variable. And once you have that,
the differences between using SMUDGE or HIDE/REVEAL are extremely
minimal (as you can see above). And HIDE/REVEAL don't (as I have
mentioned before) rely on obscure and dubious features like unused (sez
who?) bits in the length byte.
>>>It's an increase in complexity for no apparent (to the user)
>>>improvement in usage.
>>
>>Then why is it found in every Forth system I've seen in the last decade?
>I've no idea. That's what I'm trying to discover!
Well, I'm not the originator of the scheme, so I'm having to do a little
guesswork and analysis to see just what's going on here. And I've never
used a Forth system that had hashed dictionary threads that *didn't* use
the LAST variable. I'm interested in how PolyFORTH defines SMUDGE, and
how it goes about finding the latest definition from the various threads
if it doesn't use LAST.
>I haven't. Really. In fact, I'm going to try it to see if it makes
>the system any faster or better. The whole point of my line of
>argument is to try to get the proponents of this scheme fully to
>describe its advantages.
Hmm, well, I can offer the data above. I have to admit that it never
even occured to me that there might be systems around that used hashing
but that *didn't* use HIDE and REVEAL. Maybe there's some tricks that
haven't occured to me. But I'm not so much a proponent of HIDE and
REVEAL as I am a happy user of systems that have them.
BTW, doesn't PolyFORTH use a compination of hashing and threading, much
like the scheme outlined above? I've only ever seen one Forth system
that used hashing *instead* of threading, and that was an experimental
system by Michael McNeil which he gave up on as too rigid and limited
(although it sure did compile fast!) :-)
>>But I think it's a trifle arrogant to talk about an "improvement" to
>>Forth when discussing something that is more common than the alternative
>>that you're advocating. I haven't seen a Forth system that uses SMUDGE
>>in nearly a decade.
>Ah. That's something I really didn't know. Almost all of my Forth
>experience is with polyFORTH, and I assumed that the deletion of the
>smudge bit was something that was being proposed rather than
>something that was common usage. Mea culpa.
Ah, well I didn't know that PolyFORTH still used the SMUDGE bit. As I
say, I didn't think that SMUDGE was compatible with hashing, and I knew
that PolyFORTH used some hashing. So mea culpa to me too. I admit that
I have a lot of respect for Forth Inc., so I'm curious how they
implemented SMUDGE with hashing, and without LAST. (I also thought that
LAST was a PolyFORTH concept, which just shows ta go how much I've kept
up with PolyFORTH, I guess.) :-)
>How is the code organized? In the systems I know, CREATE creates a
>word and links it in. CREATE is called by : . Presumably, you need
>two versions of CREATE, one which does and one which doesn't link in
>the word. Or do you have a flag so that CREATE knows what kind of
>thing it's creating?
Nothing so complex. CREATE creates a word, points LAST to it, then
calls REVEAL. : just calls HIDE to unlink it again.
Ray Duncan
>that used hashing *instead* of threading, and that was an experimental
>system by Michael McNeil which he gave up on as too rigid and limited
>(although it sure did compile fast!) :-)
The LMI UR/FORTH implementations use a totally hashed dictionary.
This is a solid commercial system that has been shipping since 1986
and is available for DOS, OS/2, 80386 32-bit protected mode (using
the Phar Lap DOS Extender), and Windows 3.1. The hashed symbol
table approach actually simplifies many things and dramatically
improves compilation speed.
There is a "header segment" which is just all the symbols and
pointers to the corresponding code and data fields along with
some flags, but it is never searched directly. FIND reaches the
header segment through a hash table, then does a string compare.
Hash code collisions are handled by chaining the headers with
the same hash code together. Even with several thousand words
in the dictionary, FIND does on the average <2 string comparisons.
There are some interesting implications of this approach. For
example FORGET is easy - nothing to unlink and relink, just chop
the header segment back to the desired header, then scan the
header segment from the start and rebuild the hash table. Since
FORGETs are very rare, the overhead involved here is irrelevant.
Similarly, to make one word or some contiguous words headerless,
you just chop them out of the header segment, collapse the
header segment by moving later definitions up over the excised
ones as necessary, then rebuild the hash table as for FORGET.
Vocabularies are implemented by appending a vocabulary number byte
to the symbol string before it is hashed (thus, up to 255 vocabularies
are possible in the system). As I said before, hash code collisions
are chained together. As it happens, the most recent definition
in such a chain will be at the front of the chain, so redefinitions
of a name in the same vocabulary will have the expected behavior
(the most recent will be used). Words with the same name in different
vocabularies will never have the same hash code so will always be
findable according to the search order.
Incidentally, per someone's comment that he hadn't seen SMUDGE
used in a decade... I never did like the fig-Forth SMUDGE because
it was toggling a bit -- when you executed SMUDGE you weren't bringing
a header to a known state, you were bringing it to the opposite of
its previous state. So we changed LMI systems a long time ago to
use SMUDGE (which is like HIDE) and UNSMUDGE (which is like REVEAL).
Chris Waters
>In article <xtifrC9...@netcom.com> xt...@netcom.com (Chris Waters) writes:
>>... I've only ever seen one Forth system
>>that used hashing *instead* of threading, and that was an experimental
>>system by Michael McNeil which he gave up on as too rigid and limited
>>(although it sure did compile fast!) :-)
>The LMI UR/FORTH implementations use a totally hashed dictionary.
[...]
>There is a "header segment" which is just all the symbols and
>pointers to the corresponding code and data fields along with
>some flags, but it is never searched directly. FIND reaches the
>header segment through a hash table, then does a string compare.
>Hash code collisions are handled by chaining the headers with
>the same hash code together. Even with several thousand words
>in the dictionary, FIND does on the average <2 string comparisons.
Yup, sounds exactly like McNeil's system (which was described in one of
the early ForML conference proceedings. Interesting. I didn't know
that anyone was using it. The rest of the details mentioned (ease of
coding FORGET, how to handle vocabularies, etc.) are all exactly the
same as McNeil described in his paper.
Live and learn. I'm glad to hear that this scheme is successful. I
guess that with newer machines, with more memory, the overhead of the
hash tables isn't as much of a problem as it was back when McNeil first
tried it (back when 64k was all the CPUs could address). Memory was the
reason that he abandoned the scheme.
>Incidentally, per someone's comment that he hadn't seen SMUDGE
>used in a decade...
That was I.
>I never did like the fig-Forth SMUDGE because
>it was toggling a bit -- when you executed SMUDGE you weren't bringing
>a header to a known state, you were bringing it to the opposite of
>its previous state.
Exactly! :-)
>So we changed LMI systems a long time ago to
>use SMUDGE (which is like HIDE) and UNSMUDGE (which is like REVEAL).
That's better, though I still dislike the concept of fiddling with bits.
I realize that standard(s) define 31 characters as the maximum header
length, but I prefer to be able to define much longer names in my system
if I feel like it. But then, I suppose it might be harder to actually
unlink and relink in a fully hashed system. Oh well.
I'm a little dubious about the names as well (something more descriptive
seems like it would be preferable), but again, oh well. :-)
cheers, and thanks for the info, Ray.
Doug Philips
+The LMI UR/FORTH implementations use a totally hashed dictionary.
I am curious to know what your hashing function is. Is the hash table
fixed size or does it grow/shrink?
+Vocabularies are implemented by appending a vocabulary number byte
+to the symbol string before it is hashed (thus, up to 255 vocabularies
+are possible in the system).
Are you saying then that if there are n vocabularies in the search order
that up to n separate probes of the hash table must be done before
deciding that maybe the word you are trying to find is a number?
-Doug
Ray Duncan
>Yup, sounds exactly like McNeil's system (which was described in one of
>the early ForML conference proceedings. Interesting. I didn't know
>that anyone was using it. The rest of the details mentioned (ease of
>coding FORGET, how to handle vocabularies, etc.) are all exactly the
>same as McNeil described in his paper.
That's news to me. I don't remember ever seeing McNeil's paper.
But it's possible that Rick Wilton (who did much of the original
implementation of UR/FORTH on DOS) had read it. Or perhaps it's
just convergent evolution!
>>So we changed LMI systems a long time ago to
>>use SMUDGE (which is like HIDE) and UNSMUDGE (which is like REVEAL).
>
>That's better, though I still dislike the concept of fiddling with bits.
>I realize that standard(s) define 31 characters as the maximum header
>length, but I prefer to be able to define much longer names in my system
>if I feel like it. But then, I suppose it might be harder to actually
>unlink and relink in a fully hashed system. Oh well.
But if you have SMUDGE and UNSMUDGE (or HIDE and REVEAL), the
implementation is really irrelevant. There's no reason why the
bit (if it is implemented in a bit flag) has to be embedded in
the symbol string. In UR/FORTH, each header has a separate
"flags byte" which includes the "smudged/unsmudged" bit along
with several others. The full extended character set can
be used in definition names, and names can be as long as 255
characters. (although such names wouldn't be Standard)
>I'm a little dubious about the names as well (something more
>descriptive seems like it would be preferable)...
Yeah, in retrospect, I agree. At the time, however, we felt
that we were presenting our users with enough new concepts to
cope with in UR/FORTH that we didn't want to invent new names
for functionality they were already relatively familiar with.
Live & learn, I guess.
Ray Duncan
>In article <212tps$e...@news.cerf.net> dun...@nic.cerf.net (Ray Duncan) writes:
>+The LMI UR/FORTH implementations use a totally hashed dictionary.
>
>I am curious to know what your hashing function is. Is the hash table
>fixed size or does it grow/shrink?
Fixed size, although we make it large enough that there will
be relatively few collisions in a typical application (the hash
table's in its own segment, so memory is not a constraint).
The hash function itself is somewhat ad-hoc -- we tried a
number of different hash functions, using large applications
of our own as testbeds, and picked one that was efficient yet
scattered the name space fairly evenly across the hash codes.
>+Vocabularies are implemented by appending a vocabulary number byte
>+to the symbol string before it is hashed (thus, up to 255 vocabularies
>+are possible in the system).
>
>Are you saying then that if there are n vocabularies in the search order
>that up to n separate probes of the hash table must be done before
>deciding that maybe the word you are trying to find is a number?
Yes. However, we are trying to optimize the common case, and
as far as we have seen search orders deeper than 2 or 3 vocabularies
are quite uncommon. FIND also collapses the search order
if possible before doing anything else, e.g. if CONTEXT and
CURRENT are both the same vocabulary, it will only probe the
hash table once.
In any event, the hashed symbol table approach is always
going to be better than the traditional linked dictionary.
You are indirectly raising the spectre of the pathological
case, an application that contains (say) 255 vocabularies,
and then somehow during compilation sets up a search order
that contains most or all of these vocabularies. (Never
mind the difficulties of documenting and maintaining such
an application!) Well, such an application would undoubtedly
contain many thousands of definitions -- at least, I can't
visualize an application where it would make sense to have
the memory and maintenance overhead of a vocabulary only to
put one or two words in the vocabulary. Then,
probing the hash table 255 times would still be preferable
to performing a linear search down thousands of words
partitioned among 255 vocabulary threads before trying
to convert the string to a number.
Chris Waters
>Yes. However, we are trying to optimize the common case, and
>as far as we have seen search orders deeper than 2 or 3 vocabularies
>are quite uncommon. FIND also collapses the search order
>if possible before doing anything else, e.g. if CONTEXT and
>CURRENT are both the same vocabulary, it will only probe the
>hash table once.
I thought that searching CURRENT was removed from both the 83-standard
and dpANS! In fact, I'm not even sure that it was in the 79-standard.
(Although collapsing any duplicate vocabularies pointed to by CONTEXT is
certainly a good idea in any case.)
>In any event, the hashed symbol table approach is always
>going to be better than the traditional linked dictionary.
Yes, substantially, if you're talking about compile speed. It does,
however, require giving up some flexibility. Probably a worthwhile
tradeoff in 97% of the cases however.
I presume that there are also regular threads maintained in the
dictionary? (Used to rebuild the hash-table when needed, as well as to
find the most recent definition in a given thread.)
Definitely sounds like a groovy scheme. If you ever decide to bring
that OS/2 version up-to-date, let me know. (My interest in a 16-bit
OS/2 version is lower than my interest in JCL, and almost as low as my
interest in a Windows version. A 32-bit OS/2 version, OTOH, has at
least one guaranteed sale right off the bat.) :-)
Ray Duncan
>
>>Yes. However, we are trying to optimize the common case, and
>>as far as we have seen search orders deeper than 2 or 3 vocabularies
>>are quite uncommon. FIND also collapses the search order
>>if possible before doing anything else, e.g. if CONTEXT and
>>CURRENT are both the same vocabulary, it will only probe the
>>hash table once.
>
>I thought that searching CURRENT was removed from both the 83-standard
>and dpANS! In fact, I'm not even sure that it was in the 79-standard.
>(Although collapsing any duplicate vocabularies pointed to by CONTEXT is
>certainly a good idea in any case.)
I am using CONTEXT and CURRENT as shorthand for the "first vocabulary
in the search order" and "the compilation vocabulary." VARIABLEs
with the names CONTEXT and CURRENT actually exist in our system, but
they just contain a vocabulary number, not a pointer to a linked
list of headers as in a traditional system. The FIND in our
system is driven by a byte array of vocabulary numbers that is set
up by execution of a vocabulary name and DEFINITIONS and so on.
This byte array defines the search order and can theoretically
be of any length up to 255 bytes, although in our systems the
default search order has 3 tiers: CONTEXT, CURRENT, and lastly
FORTH.
As for Forth-83 and dpANS -- Forth-83 was singularly vague on
the subject of vocabulary implementation, and (the last time
I looked) so is dpANS. In the Forth-83 standard there is nothing
that says the use of system variables named CONTEXT and CURRENT
makes the system nonstandard. Nor is there anything that says
the compilation vocabulary CAN'T be part of the search order.
Perhaps you are confusing the infamous Ragsdale ALSO/ONLY
experimental vocabulary proposal with the Forth-83 standard.
It's true it was included in the Forth-83 Standard document,
but it is not part of the standard and is not required.
We considered, and still consider, the ALSO/ONLY scheme to
be severely brain-damaged and we have never used it, although
it is straightforward to layer an ALSO/ONLY compatibility layer
on top of any of our systems.
Doug Philips
+The hash function itself is somewhat ad-hoc -- we tried a
+number of different hash functions, using large applications
+of our own as testbeds, and picked one that was efficient yet
+scattered the name space fairly evenly across the hash codes.
Thanks for the info.
+In any event, the hashed symbol table approach is always
+going to be better than the traditional linked dictionary.
+You are indirectly raising the spectre of the pathological
+case, an application that contains (say) 255 vocabularies,
+and then somehow during compilation sets up a search order
+that contains most or all of these vocabularies.
I was not trying to say that hashing wasn't better. I wanted to be
sure I understood the way that you kept the name spaces of the
vocabularies separate. Since your vocabulary byte is "appended" to the
word being looked up, I don't suppose that you compute the hash of the
word itself, then incrementally compute the hash for the vocabulary <n>
version of the word name?
-Doug
Julian V. Noble
In article <xtifrC9...@netcom.com> xt...@netcom.com (Chris Waters) writes:
>... I've only ever seen one Forth system
>that used hashing *instead* of threading, and that was an experimental
>system by Michael McNeil which he gave up on as too rigid and limited
>(although it sure did compile fast!) :-)
linked-list dictionary structure. The hashed version compiles 10x faster at
least!
--jvn
Ray Duncan
I see that I was somewhat vague about where the vocabulary number
resides. The vocabulary number for a given definition is actually
part of the header and follows directly after the symbol string.
The overall structure of a header is something like this:
symbol length
symbol string (n bytes)
vocabulary number
flags byte (smudge, precedence, etc.)
pointer to code field
pointer to parameter field ("body")
pointer to next header with same hash code, or NULL
The helper word that computes a hash code accepts an address and
length on the stack. So the main definition that builds the
hash table only has to add one to the symbol string length to
have the hash function include the vocabulary number into the
hash code.
For lookups, FIND accepts the address of a counted string.
If necessary, it copies the string to a scratch buffer. It
then loops across the byte array that defines the search order.
For each vocabulary number in the search order, it copies the
vocabulary number to the end of the string being looked up,
calls the hash function, then probes the hash table. Sounds
complicated but the amount of code involved is actually very
small.
Michael Hore
j...@fermi.clas.Virginia.EDU (Julian V. Noble) writes:
>I think Uniforth is based directly on F83, so I presume that means Laxen-
>Parry F83 had SMUDGE.
No - F83 used HIDE and REVEAL.
-- Mike.
------------------------------------------------------------------------
Mike Hore mi...@kralizec.zeta.org.au (best) _--_|\
CompuServe: 100033,3164 / \
\_.--._x
Insanity is hereditary. You get it from your kids. v
------------------------------------------------------------------------
hughag...@yahoo.com
"Transform the search order consisting of widn, ... wid2, wid1 (where wid1 is searched first) into widn, ... wid2, wid1, wid1." I said to myself: "WTF? Why would I want wid1 searched twice?" Then after some more reading, I discovered the definition for FORTH (16.6.2.1590):
"Transform the search order consisting of widn, ... wid2, wid1 (where wid1 is searched first) into widn, ... wid2, widFORTH-WORDLIST."
So, apparently the idea is to use FORTH after ALSO to fix the top value of the search-order. ALSO makes no sense except in the context of being followed by a word such as FORTH that fixes the problem that it creates. But there is no word provided to allow the user to do this with his own word-lists. The user has to write code for this himself. I wrote the following, that will go into the next novice package upgrade:
: <set-context> { wid anchor -- widN... N }
get-order dup 1 < abort" *** SET-CONTEXT given empty search-order ***"
over anchor = if
wid anchor = if exit then
wid swap 1+ exit then
nip wid swap ;
\ WID will replace whatever was on top, unless ANCHOR was on top
\ in which case WID will be pushed above it
: set-context ( wid -- )
forth-wordlist <set-context> set-order ;
\ like in Forth-83, the WID becomes the context
\ with FORTH-WORDLIST as the anchor below it
\ FORTH-WORDLIST SET-CONTEXT would do the same thing that FORTH does.
: <push-context> { wid -- }
get-order
over wid = if exit then
wid swap 1+ ;
\ the WID becomes the context, with whatever was below it remaining
: push-context ( wid -- )
<push-context> set-order ;
\ I don't recommend using ALSO with <SET-CONTEXT> because problems will
\ result if the anchor is the top wid. Use PUSH-CONTEXT instead.
\ All in all, ALSO is a screw-ball idea, as it involves making a duplicate
\ in the search-order, which makes no sense.
novembe...@gmail.com
>
> \ in the search-order, which makes no sense.
- "only" empties the stack
- "also" behaves like dup
- "previous" behaves like drop
So why would you want to duplicate the top of the wordlist stack ?
Any word created using "vocabulary" acts directly with whatever is on top of the wordlist stack.
When you type say 10 at a console...
The forth data stack is => ( -- 10 20 30 )
you dont get ( -- 10 20 10 )
but you get ( 10 20 30 10 )
But on the wordlist stack,
Say the wordlist stack is => ( -- 10 20 30 )
you dont get ( 10 20 30 10 )
but you get ( -- 10 20 10 )
Usually when you are trying to change the context, you dont want to lose whatever is at the top already there. That's why "also" is important.
That how I think of it, but lets see if others agree or disagree.
NN
novembe...@gmail.com
: >context ( wid -- ) >r get-order r> swap 1+ set-order ;
: context> ( -- wid ) get-order swap >r 1- set-order r> ;
: context@ ( -- wid ) get-order over >r set-order r> ;
If I understand your code correctly then it translates to :-
\ dont duplicate
\ wid will replace whatever was on top unless anchor was on top
over = if drop else >context then
else drop context> drop >context then ;
Hope its useful.
Albert van der Horst
the system and the application. The application changed all the time.
colorforth is much like this still.
In this time and age, now we have the idea of several libraries used
to run an application (apart from the system), it makes more sense to
push a namespace (as everybody else calls it) to the search order.
Also wordlists suffer from the same problem as locals: semantics without
a syntax, such that everybody has to invent their own. This makes it legal
for me to in ciforth to replace VOCABULARY with NAMESPACE and push
instead of replace when a namespace is invoked.
>
>NN
Groetjes Albert
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst
hughag...@yahoo.com
Note that I have <SET-CONTEXT> that allows any wid to be used as the anchor, not just FORTH-WORDLIST. Eventually, I will want to be able to write my cross-compiler in itself, so this will be necessary. There may be other cases in which it is necessary --- why should FORTH-WORDLIST be so special that only it can be used as an anchor?
hughag...@yahoo.com
I am surprised that nobody pointed out the bug in my SET-CONTEXT --- that would have been an opportunity slam me for being a bad ANS-Forth programmer --- but too late now, as I have found the bug myself (although I don't know how to fix the bug).
Michael Barry
>
> This code that I presented here is a bug-ridden mess, so I'm retracting it.
> It won't go into the novice package; SET-CONTEXT doesn't work. I now
> believe that it is impossible to write SET-CONTEXT in ANS-Forth, and I have
> given up on the effort.
>
>
> I am surprised that nobody pointed out the bug in my SET-CONTEXT --- that
> would have been an opportunity slam me for being a bad ANS-Forth programmer
> --- but too late now, as I have found the bug myself (although I don't know
> how to fix the bug).
>
Forth nearly well enough to find it myself, but I would like to learn, both
about its nature and your doubts in your ability to fix it. I'm one of those
guys who is writing his own interpreter, but has never programmed anything of
note in actual Forth. The language intrigues me, and it would be nice to
gather some insights interactively from a live person with experience, rather
than just from on-line texts.
If it helps, I certify that I am not homosexual, and I have no vested interest
in any particular standard or implementation (although I have been using an
assembly listing of Dr. Brad Rodriguez' ITC Camel Forth for the MSP430 as a
general guide. My target is a vapor-ware micro-processor of my own design,
and I thought that the best way to shape and optimize the instruction set
was to write something non-trivial, like a Forth interpreter.
Thanks,
Mike
hughag...@yahoo.com
> Hugh, could you be so kind as to explain the nature of the bug? I don't know
>
> Forth nearly well enough to find it myself, but I would like to learn, both
> about its nature and your doubts in your ability to fix it.
In UR/Forth, this problem never arose. CONTEXT and CURRENT were only changed by the programmer, and never by the internal workings of the system. UR/Forth didn't have local variables though, so maybe if Ray Duncan had continued upgrading UR/Forth he would have started mucking around with CONTEXT when he implemented local variables, and gotten into the same mess that ANS-Forth is in --- but Ray Duncan was pretty smart, so he might have foreseen the problem and dodged it --- we will never know, as he gave up Forth programming when ANS-Forth came out.
The best solution would have been for the ANS-Forth document to state emphatically that the search-order and CURRENT are never changed by the internal workings of the system, but are for exclusive use by the user. Pelc would have had to use SEARCH-WORDLIST to find his local variables --- but he would have managed --- and this would have allowed users such as myself to write SET-CONTEXT and be confident in it always working the same way.
P.S. --- Thanks for your politeness! :-)
Michael Barry
>
> P.S. --- Thanks for your politeness! :-)
>
but I will try to do so, and will post any further questions that may pop
up. My spare time is limited, but this type of activity is my preferred
method of unwinding after a long day at work, fixing old worn-out cars in
a poor neighborhood.
Take care,
Mike
hughag...@yahoo.com
Gerry Jackson
> On Thursday, October 2, 2014 4:35:30 PM UTC-7, Michael Barry wrote:
>> Hugh, could you be so kind as to explain the nature of the bug? I don't know
>>
>> Forth nearly well enough to find it myself, but I would like to learn, both
>> about its nature and your doubts in your ability to fix it.
>
> SET-CONTEXT assumes that the 1st wid in the search order is the wid that you want to replace. Sometimes this is true, and sometimes it is not true, and this varies from one ANS-Forth-compliant Forth system to another. In VFX, colon words with local variables will have a wid called LOCALVARS in 1st place in the search order during the compilation of the colon word, but semicolon removes it. If you execute SET-CONTEXT during the execution of a colon word that has locals, it will replace LOCALVARS rather than the wid that you expected to replace (the wid you thought was 1st is actually 2nd) --- chaos will result. I have only tested VFX so far
using the current free version of VFX Forth (August 2014) [1]
: foo { bar } bar . [ previous ] bar . ;
Err# -13 ERR: Undefined word.
-> : foo { bar } bar . [ previous ] bar . ;
This is a greatly simplified version of a problem I had a few years ago
which I reported to MPE but, presumably, was not considered important
enough to fix.
Investigating further reveals inconsistent behaviour:
: x { y } [ order previous order ] ;
LOCALVARS FORTH EXTERNALS ROOT
Current: FORTH
FORTH EXTERNALS ROOT
Current: FORTH
ok
shows that the LOCALSVAR wordlist is removed by PREVIOUS.
However going back to the first definition and looking at the search
order after it fails to compile:
: foo { bar } bar . [ previous ] bar . ;
Err# -13 ERR: Undefined word.
-> : foo { bar } bar . [ previous ] bar . ;
^
order
EXTERNALS ROOT
Current: FORTH
ok
It seems to have removed the Forth wordlist as well as the LOCALSVAR
wordlist.
ALSO leaves LOCALSVAR in the search order after a definition is complete
: x { y } [ also ] ; order
LOCALVARS FORTH EXTERNALS ROOT
Current: FORTH
ok
GET-ORDER seems to return the LOCALSVAR wid. The following is a clear
non-compliance
1234 constant y ok
ok
: x { y } [ get-order forth-wordlist swap 1+ set-order ] y . ; ok
9999 x 1234 ok
See ANS Forth 13.3.3 1) " ... none of the Search-Order words shall
change the locals' privileged position in the search order. ..."
As I said - less than perfect behaviour. I don't think that any of the
other major systems have these problems. I agree that system wids should
not be placed in the search order.
[1] I know that using { for locals is not ANS Forth but the behaviour is
the same using LOCALS| and the Forth 200X {:
--
Gerry
hughag...@yahoo.com
> You are using VFX Forth in an area where it is less than perfect e.g.
> using the current free version of VFX Forth (August 2014) [1]
> : foo { bar } bar . [ previous ] bar . ;
> Err# -13 ERR: Undefined word.
> -> : foo { bar } bar . [ previous ] bar . ;
>
> This is a greatly simplified version of a problem I had a few years ago
> which I reported to MPE but, presumably, was not considered important
> enough to fix.
I doubt that Pelc thought that this "was not considered important enough to fix." More likely, he purposely strives to undermine anybody's independent effort to write a cross-compiler, because cross-compilers are how he makes his money primarily. This is also why he made sure that SMUDGE was not in ANS-Forth --- the only people who need SMUDGE are those writing cross-compilers --- people writing Forth application programs are happy so long as : and ; behave as if they are internally using SMUDGE and UNSMUDGE (aka HIDE and REVEAL) --- weirdly enough, the ANS-Forth document requires : and ; to behave as if they are using SMUDGE and UNSMUDGE, but makes no mention at all of SMUDGE (not even in the section on porting programs from old standards to ANS-Forth).
Are you writing a cross-compiler? I can't think of any other reason why you would be doing word-list manipulations at compile-time inside of a colon word. I'm writing one for the ARM Cortex-M3 right now. If you are interested in getting involved, contact me by private email. I liked your quotation code that you sent me, btw (you are one of only a tiny handful of people on C.L.F. who actually writes Forth code, and are not just blowing hot air) --- my Forth will have quotations, but this will be implemented completely differently than your approach --- the compiler is for micro-controllers, so speed is of the essence!
> Investigating further reveals inconsistent behaviour:
> : x { y } [ order previous order ] ;
> LOCALVARS FORTH EXTERNALS ROOT
> Current: FORTH
>
> FORTH EXTERNALS ROOT
> Current: FORTH
> ok
>
> shows that the LOCALSVAR wordlist is removed by PREVIOUS.
> However going back to the first definition and looking at the search
> order after it fails to compile:
>
> : foo { bar } bar . [ previous ] bar . ;
> Err# -13 ERR: Undefined word.
> -> : foo { bar } bar . [ previous ] bar . ;
>
> EXTERNALS ROOT
> Current: FORTH
> ok
>
> It seems to have removed the Forth wordlist as well as the LOCALSVAR
> wordlist.
> ALSO leaves LOCALSVAR in the search order after a definition is complete
>
> : x { y } [ also ] ; order
> LOCALVARS FORTH EXTERNALS ROOT
> Current: FORTH
> ok
>
> GET-ORDER seems to return the LOCALSVAR wid. The following is a clear
> non-compliance
> 1234 constant y ok
>
> 9999 x 1234 ok
> I agree that system wids should
> not be placed in the search order.
It would take maybe one day of work to fix this problem, but I doubt that Pelc will do it --- I think he purposely strives to pull the rug out from under cross-compiler writers --- these are bugs that will never get fixed.
Over on this thread (https://groups.google.com/forum/#!topic/comp.lang.forth/-FDxsdLIIkU), everybody is bragging about making hundreds of dollars per hour programming in Forth --- so maybe Pelc doesn't fix the bug because his one-day of work required to fix the bug is worth thousands of dollars, and he is waiting for us to send him a check. ;-)
Of course, I haven't actually bought a copy of VFX (as mentioned above), so I can't really expect him to spring into action to fix bugs that I find in VFX (I did expect Forth Inc. to fix bugs that I found in SwiftForth, but that was different because I had paid over $400 for SwiftForth). It is a bug though, as it clearly violates 13.3.3-1 --- so even though I'm not paying him, he ought to fix it anyway. On a positive note though, this is the first bug I have found in VFX --- by comparison, every other Forth I tested my novice package on (SwiftForth, Gforth and Win32Forth) turned up bugs --- but I haven't yet tested all of my example programs in the novice package on VFX, so I can't say for sure that no bugs will turn up.
BTW: Gerry --- did you know Ray Duncan? He didn't fix two bugs in UR/Forth that I told him about. For the most part though, he was a good guy and his UR/Forth was a good system --- it would have been difficult for me to write MFX in any of the other Forth systems available at that time.
Raimond Dragomir
>
> > ALSO leaves LOCALSVAR in the search order after a definition is complete
>
> >
>
> > : x { y } [ also ] ; order
>
> > LOCALVARS FORTH EXTERNALS ROOT
>
> > Current: FORTH
>
> > ok
>
> >
>
> > GET-ORDER seems to return the LOCALSVAR wid. The following is a clear
>
> > non-compliance
>
>
>
> > 1234 constant y ok
>
> > : x { y } [ get-order forth-wordlist swap 1+ set-order ] y . ; ok
>
> >
>
> > 9999 x 1234 ok
>
>
VFX seems to be quickly patched to have local support without further testing or investigation.
hughag...@yahoo.com
> This simple code is a clear evidence of a bad design regarding locals implementation (not wordlists). For me it is more and more clear that local variables are really not welcome in the forth community.
Locals are a bad design too, mostly because LOCALS| takes its parameters backwards. In Forth-200x locals are a bad design because the locals to the right of the | are not zeroed out. But this has nothing to do with what we are discussing here, in regard to how locals relate to word-list juggling. ANS-Forth actually did something intelligent in 13.3.3-1 when it required that the word-list jugglers would not affect LOCALVARS --- but, sadly, that is the rule that Pelc violated in VFX --- so blame Pelc on this one, not ANS-Forth.
Also, I don't think that locals are not welcome in the Forth community --- this was true of Charles Moore who hates locals, and it was true up through the era of Forth-83 (UR/Forth was Forth-83 and it did not have locals, although MFX written in UR/Forth did have locals) --- but people realized that locals were a good thing and they were clamoring for them in the early 1990s, which is why ANS-Forth provided them. I used to be a purist in that I didn't use locals at all (see my LowDraw.4th program that was written about 10 years prior to the novice package), but I came to realize that they are useful --- so did everybody else.
> VFX seems to be quickly patched to have local support without further testing or investigation.
I think Pelc is not going to fix the bug, because he wants to make life as difficult as possible for cross-compiler writers. But I have written a work-around for the bug in VFX! I said earlier that I had abandoned SET-CONTEXT --- but now I have figured out how to write SET-CONTEXT. This will go in the next novice package upgrade. Here it is (with the comments removed for brevity):
: set-forth ( -- )
only forth definitions ;
: get-context ( -- wid )
get-order dup 1 < abort" *** GET-CONTEXT given empty search-order ***"
over >r \ r: -- top-wid
0 do drop loop \ --
r> ;
: <push-context> { wid -- }
get-order
wid swap 1+
set-order ;
: <pop-context> ( -- wid )
get-order dup 1 < abort" *** POP-CONTEXT given empty search-order ***"
set-order
r> ;
<pop-context> drop ;
false value localvars#
false value expected#
: fill-localvars# ( -- )
[ get-context to expected# ]
1 2 3 locals| x y z |
[
get-context expected# <> [if]
<pop-context> to localvars#
get-context expected# <> [if] .( *** FILL-LOCALVARS# can't find EXPECTED# ***) abort [then]
localvars# <push-context> [then]
] ;
: push-underneath { wid -- }
<pop-context>
get-context wid <> if
wid <push-context> then
<push-context> ;
: replace-1st ( wid -- )
push-underneath <drop-context> ;
: <set-context> { wid anchor -- }
get-context anchor = if
exit then
get-context localvars# = if
<pop-context>
get-context anchor = if
wid <push-context>
else
wid replace-1st then
<push-context>
exit then
wid replace-1st ;
: set-context ( wid -- )
: push-context { wid -- }
get-context localvars# = if
<pop-context> wid <push-context> <push-context>
else
wid <push-context> then ;
: pop-context { -- wid }
<pop-context>
dup localvars# = if
<pop-context> swap <push-context> then ;
: drop-context ( -- )
pop-context drop ;
char & comment \ this is for testing the word-list stuff
wordlist constant test1#
wordlist constant test2#
: test ( -- )
[ .( TEST1# = ) test1# u. ]
[ .( TEST2# = ) test2# u. ]
[ set-forth order ]
[ forth-wordlist set-context order ]
[ test1# set-context order ]
1 2 3 locals| x y z | [ order ]
[ test1# push-context order ]
[ drop-context order ]
[ test1# set-context order ]
[ test2# set-context order ]
[ test1# push-context order ]
[ drop-context order ]
;
order
set-forth
&
Andrew Haley
>
> Wordlists are a bad design ("severely brain-damaged" in Ray Duncan's
> words). This is because ALSO duplicates the 1st wid of the
> search-order, which makes no sense.
an inconvenience.
> Forth is supposed to be context-free,
never been a "pure" language in that sense.
> Locals are a bad design too, mostly because LOCALS| takes its
> parameters backwards.
like. Given that there was insufficient common practice in this area
at the time there was nothing else that ANS could have done.
> In Forth-200x locals are a bad design because the locals to the
> right of the | are not zeroed out.
Forth provides the freedom you need.
> Also, I don't think that locals are not welcome in the Forth
> community --- this was true of Charles Moore who hates locals, and
> it was true up through the era of Forth-83 (UR/Forth was Forth-83
> and it did not have locals, although MFX written in UR/Forth did
> have locals) --- but people realized that locals were a good thing
> and they were clamoring for them in the early 1990s, which is why
> ANS-Forth provided them. I used to be a purist in that I didn't use
> locals at all (see my LowDraw.4th program that was written about 10
> years prior to the novice package), but I came to realize that they
> are useful --- so did everybody else.
Andrew.
Gerry Jackson
> On Friday, October 3, 2014 4:30:47 PM UTC-7, Gerry wrote:
> I doubt that Pelc thought that this "was not considered important enough to fix." More likely, he purposely strives to undermine anybody's independent effort to write a cross-compiler, because cross-compilers are how he makes his money primarily. This is also why he made sure that SMUDGE was not in ANS-Forth --- the only people who need SMUDGE are those writing cross-compilers --- people writing Forth application programs are happy so long as : and ; behave as if they are internally using SMUDGE and UNSMUDGE (aka HIDE and REVEAL) --- weirdly enough, the ANS-Forth document requires : and ; to behave as if they are using SMUDGE and UNSMUDGE, but makes no mention at all of SMUDGE (not even in the section on porting programs from old standards to ANS-Forth).
>
does. After all Stephen Pelc is not listed as a (presumably voting)
member of the ANS Forth Technical Committee, only as a contributor. I
prefer the cockup side of the cockup vs conspiracy theory (look up
Hanlon's Razor that I hadn't previously heard of before. Off topic: as
an aside that led me to Sturgeon's Law which I like).
> Are you writing a cross-compiler?
the ideas given in the Forth cross-compiler paper regularly mentioned on
c.l.f. That uses wordlists. However other aspects of my cross-compiler
are now over-complicated and it needs a simplifying re-write.
> I can't think of any other reason why you would be doing word-list manipulations at compile-time inside of a colon word.
worked around the problem so the details are a bit hazy now. I think it
is during the scanning phase where words like ; [: ;] S" etc need to do
something and they are (re)defined in a separate wordlist. I believe a
similar technique is used in systems that have different wordlists for
interpreter and compiler states. Wordlists are handy for a domain
specific language that has words overlapping standard Forth word names.
Also some people use wordlists to define private words in modules.
> I'm writing one for the ARM Cortex-M3 right now. If you are interested in getting involved, contact me by private email.
foreseeable future.
> I liked your quotation code that you sent me
[...]
> If your "it" referred to PREVIOUS, that is unlikely --- I don't think this is inconsistent --- more likely, it was QUIT that was called due to the crash, and it did an ONLY. I think that it is okay for QUIT to do ONLY because we want to get back to a rational state.
can investigate the state of the system when it went wrong (if it did).
> I do wonder at why ONLY doesn't leave FORTH on the top of the search-order though --- putting us back in the REPL without FORTH in the search-order is not exactly putting us in a "rational state."
the same thing as -1 SET-ORDER so it does seem redundant in a standard.
[...]
> all of the word-list jugglers have access to LOCALSVAR just like any other wid.
>
>> I agree that system wids should
>> not be placed in the search order.
>
> It would have actually be quite easy to implement this correctly. Pelc could have put his locals in LOCALVARS but not put LOCALVARS in the search order. He could have just written FIND so that it does a SEARCH-WORDLIST of LOCALVARS first (giving LOCALVARS the "privileged position" that ANS-Forth requires), and then done a normal FIND of the search-chain.
[...]
> BTW: Gerry --- did you know Ray Duncan?
I have no idea what he did.
> He didn't fix two bugs in UR/Forth that I told him about. For the most part though, he was a good guy and his UR/Forth was a good system --- it would have been difficult for me to write MFX in any of the other Forth systems available at that time.
Gerry
Enoch
> hughag...@yahoo.com wrote:
>
> >
>
> > Wordlists are a bad design ("severely brain-damaged" in Ray Duncan's
>
> > words). This is because ALSO duplicates the 1st wid of the
>
> > search-order, which makes no sense.
>
>
>
> I don't like it much, but it doesn't seem like a show-stopper; it's
>
> an inconvenience.
>
AmForth <http://amforth.sourceforge.net/TG/Implementation.html> has introduced a deferred system word, called WLSCOPE, that selects which wordlist to add the created word to based upon its name.
For example, words beginning with underscore ('_'), can be put on a PRIVATE word list with no need for get-current / set-current wrapping.
Does any of the big name Forth implementations have anything similar?
Thanks, Enoch.
hughag...@yahoo.com
> ...I have written a work-around for the bug in VFX!
Holey buckets! I found another bug in VFX --- DEFINITIONS doesn't work either (meta-compiled words end up in LOCALVARS rather than the expected word-list) --- this isn't really another bug, it is just another manifestation of the bug we have been discussing regarding LOCALVARS getting in the way
Here is a new version of my code with DEFINITIONS and GET-CONTEXT fixed so they will work inside of colon definitions at compile-time. This is only a slight upgrade from what I posted previously (once again, comments are removed for brevity).
dup localvars# = if drop
<pop-context> <get-context>
swap <push-context> then ;
: definitions ( -- )
get-context set-current ;
: push-context { wid -- }
<get-context> localvars# = if
<pop-context> wid <push-context>
<push-context>
else
wid <push-context> then ;
: pop-context { -- wid }
<pop-context>
dup localvars# = if
<pop-context>
swap <push-context> then ;
: drop-context ( -- )
pop-context drop ;
char & comment \ this is for testing the word-list stuff
wordlist constant test1#
wordlist constant test2#
[ cr .( TEST1# = ) test1# u. ]
[ cr .( TEST2# = ) test2# u. ]
[ cr .( FORTH-WORDLIST = ) forth-wordlist u. ]
[ set-forth order ]
[ get-context cr .( context = ) u. order ]
[ test1# set-context order ]
1 2 3 locals| x y z | [ order ]
[ drop-context order ]
[ test1# set-context order ]
[ test2# set-context order ]
[ test1# push-context order ]
[ drop-context order ]
;
order
set-forth
[ set-forth cr .( test-forth) order ]
[ forth order ]
;
order
set-forth
&
hughag...@yahoo.com
> On 04/10/2014 01:40, hughag...@yahoo.com wrote:
> > I doubt that Pelc thought that this "was not considered important enough to fix." More likely, he purposely strives to undermine anybody's independent effort to write a cross-compiler, because cross-compilers are how he makes his money primarily. This is also why he made sure that SMUDGE was not in ANS-Forth --- the only people who need SMUDGE are those writing cross-compilers --- people writing Forth application programs are happy so long as : and ; behave as if they are internally using SMUDGE and UNSMUDGE (aka HIDE and REVEAL) --- weirdly enough, the ANS-Forth document requires : and ; to behave as if they are using SMUDGE and UNSMUDGE, but makes no mention at all of SMUDGE (not even in the section on porting programs from old standards to ANS-Forth).
>
> You shouldn't ascribe machiavellian motives to everything that someone
> does. After all Stephen Pelc is not listed as a (presumably voting)
> member of the ANS Forth Technical Committee, only as a contributor. I
> prefer the cockup side of the cockup vs conspiracy theory (look up
> Hanlon's Razor that I hadn't previously heard of before. Off topic: as
> an aside that led me to Sturgeon's Law which I like).
I got out of Forth-200x primarily so I wouldn't have my name listed in the peanut-gallery section of the Forth-200x document --- the whole mailing-list is just a charade to make Forth-200x seem to be a product of the entire Forth community, rather than just SwiftForth with pretensions of being the standard --- why should I lend my name to a Forth Inc. marketing scheme?
Pelc does make his money selling cross-compilers, so I doubt that he wants competition. This is my first-ever VFX program, and I have only been programming in VFX for less than a month --- yet I am aware of the bug --- it seems difficult to believe that Pelc has remained unaware of the bug for all of these years. I predict that Pelc will not thank me for spending a week of my time writing a work-around for the bug in VFX so people can now use VFX to write cross-compilers. He is going to continue to say that I know nothing about Forth, haven't read the manual, etc..
After the 9-11-2001 attack, I became interested in conspiracy theory, as I could readily see some gaping holes in the official story. Actually, I was already aware that there were gaping holes in the Oklahoma City story, so it didn't take long to notice the parallels. All in all, I'm not too impressed by Hanlon's Razor --- historically speaking, Machiavellian conspiracies have been the norm not the exception. Also, there are a huge number of people in prison for conspiracy to commit a crime, without having actually committed a crime --- so why is conspiracy considered to be real enough to put people in prison for it, but not real when common citizens rather than the police are the ones to point it out?
It seems like a remarkable coincidence that this bug in VFX, that can be found by novices on their first day, is also a bug that largely prevents people from using VFX to compete against Pelc in the cross-compiler field where he makes his money. Well, you can call me a "conspiracy theorist" if I get to call you a "coincidence theorist." ;-)
I have found bugs in these ANS-Forth systems: SwiftForth, Gforth, Win32Forth, and now VFX. Doing paid work in ANS-Forth is problematic for me because I might run into a bug in the compiler and then find myself in the awkward position of explaining to my boss that I spent the entire week writing a work-around for a bug in the compiler, and have accomplished zilch on the program that I'm getting paid to write (I would likely get fired). I have written work-arounds for the bugs in all of these Forth systems, and provided the work-arounds for free in the novice package, but I just get attacked for doing this --- nobody wants a light shined on the bugs in their compiler --- they prefer to present an image of a having a bug-free compiler in order to attract more users.
> > Are you writing a cross-compiler?
>
> I wrote one in about 2005/6 which I have been using ever since.
hughag...@yahoo.com
> > right of the | are not zeroed out.
>
> Eh? If you need a local to be zeroed, zero it. If you don't, don't:
> Forth provides the freedom you need.
It is important for the system to behave consistently. If you don't initialize locals and globals to a known state, then your program will behave differently from one execution to another, or from one compilation to another. If the programmer forgets to explicitly initialize a variable before he uses it, the program may run correctly because by luck there was a value in the random garbage that it got initialized to that worked --- but this luck may run out at an awkward moment, like when the customer tries to run the program.
This is programming-101 --- I would expect anybody with 6 months experience programming to know basic concepts such as the need to initialize to a known and consistent state.
Over two decades ago you were on said in this thread? "Almost all of my Forth experience is with polyFORTH...." Now, in 2014, you still don't know basic programming ideas, such as the need for programs to be consistent from one run to the next and from one compilation to the next. Also, when I said that putting control-flow data on the parameter-stack was a bad idea, you were oblivious to why this would be, and you even continued to argue the point after I explained it. You have apparently never written a meta-compiling word, as you haven't noticed that when you write one you don't have access to the data on the parameter stack because there may or may not be control-flow data on top of it. You're an idiot! This is an indictment of Elizabeth Rather --- anybody who follows Elizabeth Rather must be an idiot, and if they aren't an idiot when they start, they will be pretty soon.
In 1993, one year prior to ANS-Forth getting written in stone, Ray Duncan who is not an idiot described a crucial aspect of ANS-Forth as being "severely brain-damaged." After ANS-Forth came out, Ray Duncan gave up on Forth and moved on to other pursuits. Obviously, he had as much influence on the ANS-Forth committee as I have on the Forth-200x committee, which is zilch --- so he walked off. Two decades later, you are still here on C.L.F. and still just as ignorant of programming-101 concepts as you were in 1993. This is why I consider ANS-Forth to be a cult. Everybody who knew anything about programming was driven out, and those who knew nothing stayed for decades.
Now here I am with one month of experience in VFX, and one week of that was spent writing a work-around for a bug in a crucial aspect of VFX (word-lists) that I need for my cross-compiler. I didn't have to waste time dinking around with stuff like this when I wrote MFX for Testra in UR/Forth (UR/Forth comes from Ray Duncan's company: LMI) --- LMI started producing Forth compilers about 30 years ago --- I would really expect a modern Forth system such as VFX to have fewer bugs, not more bugs, especially in regard to a fundamental aspect of Forth such as word-lists.
Gerry Jackson
> I hadn't actually read the top of the ANS-Forth document where the names are listed. I had just assumed that Pelc was a voting member. I see now that he is listed in the "peanut gallery" along with Ray Duncan, Charles Moore, etc., who had exactly zero influence on ANS-Forth and who ignored ANS-Forth completely after it came out. So maybe I'm giving him too much credit (actually blame) for ANS-Forth, and he was just an innocent by-stander. Since you seem to be familiar with the list of names at the top, can you tell me how many were employees of Forth Inc., or at least had some financial reason to support Forth Inc.?
and, apart from the two obvious ones, I haven't a clue whether they have
any connection to Forth Inc or not.
>
> For what target?
>
Same as the host - Windows desktop PC. I had intentions to re-target it
once the system was proven but never got round to it.
--
Gerry
Elizabeth D. Rather
...
> I am not familiar with people in the list, they are just names to me
> and, apart from the two obvious ones, I haven't a clue whether they have
> any connection to Forth Inc or not.
were typically 14 members at each meeting.
Cheers,
Elizabeth
--
==================================================
Elizabeth D. Rather (US & Canada) 800-55-FORTH
FORTH Inc. +1 310.999.6784
5959 West Century Blvd. Suite 700
Los Angeles, CA 90045
http://www.forth.com
"Forth-based products and Services for real-time
applications since 1973."
==================================================
Andrew Haley
> On Sunday, October 5, 2014 1:42:59 AM UTC-7, Andrew Haley wrote:
>> hughag...@yahoo.com wrote:
>> > In Forth-200x locals are a bad design because the locals to the
>> > right of the | are not zeroed out.
>>
>> Eh? If you need a local to be zeroed, zero it. If you don't, don't:
>> Forth provides the freedom you need.
>
> I have gone over this repeatedly several times already --- like
> Sisyphus pushing that boulder up the hill.
>
> It is important for the system to behave consistently. If you don't
> initialize locals and globals to a known state, then your program
> will behave differently from one execution to another, or from one
> compilation to another.
> If the programmer forgets to explicitly initialize a variable before
> he uses it, the program may run correctly because by luck there was
> a value in the random garbage that it got initialized to that worked
> --- but this luck may run out at an awkward moment, like when the
> customer tries to run the program.
initialization on programmers: they have the freedom to shoot
themselves in the foot if they wish.
Anyway, I don't believe it. If a programmer does not expect local
temporaries to be zeroed, why would they make this mistake? And why
zero, anyway? Why not DEADBEEF or somesuch?
> Over two decades ago you were on said in this thread?
> "Almost all of my Forth experience is with polyFORTH...." Now, in
> 2014, you still don't know basic programming ideas, such as the need
> for programs to be consistent from one run to the next and from one
> compilation to the next. Also, when I said that putting control-flow
> data on the parameter-stack was a bad idea, you were oblivious to
> why this would be, and you even continued to argue the point after I
> explained it.
your metacompiling words was not that the control flow stack was in
your way, but that they were badly factored and far too long. The
control flow stack did not force you to do that.
> You have apparently never written a meta-compiling word, as you
> haven't noticed that when you write one you don't have access to the
> data on the parameter stack because there may or may not be
> control-flow data on top of it. You're an idiot! This is an
> indictment of Elizabeth Rather --- anybody who follows Elizabeth
> Rather must be an idiot, and if they aren't an idiot when they
> start, they will be pretty soon.
But why bother actually arguing the technical points when you can
instead attack the person you're talking to? That would take some
effort, I suppose.
Andrew.
Michael Barry
>
> Holey buckets! I found another bug in VFX --- DEFINITIONS doesn't
> work either (meta-compiled words end up in LOCALVARS rather than the
> expected word-list) --- this isn't really another bug, it is just
> another manifestation of the bug we have been discussing regarding
> LOCALVARS getting in the way
>
> Here is a new version of my code with DEFINITIONS and GET-CONTEXT
> fixed so they will work inside of colon definitions at compile-time.
> This is only a slight upgrade from what I posted previously (once
> again, comments are removed for brevity).
>
the sake of brevity. It would make learning much easier for me.
In the course of my part-time studies, I have seen references to
numerous Forth stacks: parameter, return, control, leave, float,
etc. It is my impression that only the first two are omni-present,
but that the others can provide useful work-arounds for more
advanced coding techniques. Do you (or anyone else here) have
any opinions on the use of one of the others for implementing
locals, or perhaps even a dedicated local stack, if one doesn't
already exist?
Thanks,
Mike
Gerry Jackson
> On Saturday, October 4, 2014 9:14:07 PM UTC-7, hughag...@yahoo.com wrote:
>> ...I have written a work-around for the bug in VFX!
>
> false value expected#
>
> : fill-localvars# ( -- )
> [ <get-context> to expected# ]
> 1 2 3 locals| x y z |
> [
> <get-context> expected# <> [if]
> <pop-context> to localvars#
> <get-context> expected# <> [if] .( *** FILL-LOCALVARS# can't find EXPECTED# ***) abort [then]
> localvars# <push-context> [then]
> ] ;
round it and forget it e.g. (untested)
marker forget-this
<get-context> value expected#
: fill-localvars#
1 2 3 locals| a b c |
... etc ;
fill-localvars#
forget-this
etc
>
> wordlist constant test1#
> wordlist constant test2#
>
> : test ( -- ) \ this tests SET-CONTEXT
> [ cr .( TEST1# = ) test1# u. ]
> [ cr .( TEST2# = ) test2# u. ]
> [ cr .( FORTH-WORDLIST = ) forth-wordlist u. ]
> [ set-forth order ]
> [ get-context cr .( context = ) u. order ]
> [ forth-wordlist set-context order ]
> [ test1# set-context order ]
> 1 2 3 locals| x y z | [ order ]
> [ get-context cr .( context = ) u. order ]
> [ test1# push-context order ]
> [ drop-context order ]
> [ test1# set-context order ]
> [ test2# set-context order ]
> [ test1# push-context order ]
> [ drop-context order ]
> ;
>
--
Gerry
hughag...@yahoo.com
> On 06/10/2014 01:09, hughag...@yahoo.com wrote:
> A couple of fairly minor comments.
>
> > false value expected#
>
> > : fill-localvars# ( -- )
> > [ <get-context> to expected# ]
> > 1 2 3 locals| x y z |
> > [
> > <get-context> expected# <> [if]
> > <pop-context> to localvars#
> > <get-context> expected# <> [if] .( *** FILL-LOCALVARS# can't find EXPECTED# ***) abort [then]
> > localvars# <push-context> [then]
> > ] ;
>
> Presumably the above is only executed once so you could put a marker
> round it and forget it e.g. (untested)
> > wordlist constant test1#
> > wordlist constant test2#
>
> > : test ( -- ) \ this tests SET-CONTEXT
> > [ cr .( TEST1# = ) test1# u. ]
> > [ cr .( TEST2# = ) test2# u. ]
> > [ cr .( FORTH-WORDLIST = ) forth-wordlist u. ]
> > [ set-forth order ]
> > [ get-context cr .( context = ) u. order ]
> > [ forth-wordlist set-context order ]
> > [ test1# set-context order ]
> > 1 2 3 locals| x y z | [ order ]
> > [ get-context cr .( context = ) u. order ]
> > [ test1# push-context order ]
> > [ drop-context order ]
> > [ test1# set-context order ]
> > [ test2# set-context order ]
> > [ test1# push-context order ]
> > [ drop-context order ]
> > ;
> This doesn't execute fill-localvars# and hence not all of <set-context>
I think that I will include the following line in FILL-LOCALVARS# to make this point clear:
TRUE ABORT" *** FILL-LOCALVARS# not supposed to be executed ***"
Notice that I test SET-CONTEXT both prior to and after defining some local variables in TEST, so I am testing SET-CONTEXT for both those words that do not have local variables and those that do.
Thanks for taking the time to read my code. :-) Look at it again to see how LOCAL-VARS# gets set, as I think you didn't grok that part. The rest of the code is pretty straight-forward --- each word just does what it did in the original naive implementations, but also checks to see if the LOCALVARS# wid is on top of the search-order, and if so removes it temporarily while doing its thing, and then puts it back on top unaffected. The original naive implementation is mostly still there (the words with the <xxx> names) and is used by the more sophisticated words that work-around the VFX bug.
This code has been tested under VFX and it works. It should work on any ANS-Forth system that does not have this bug --- in this case, FILL-LOCALVARS# will fill LOCALVARS# with a FALSE, and all the words that look for LOCALVARS# in the search-order will not find it (there is presumably no such thing as a wid equal to zero), so they will just do their thing in a straight-forward manner.
I can't guarantee that this code will work under any ANS-Forth implementation. It will have to be tested on a case-by-case basis. It will work on any system that has a bug similar to the VFX bug, in that a wid is pushed onto the search-order for the local variables inside of colon words with local variables, and removed by semi-colon. If the system has any other unexpected use of the search-order that gets in the way of the programmer's use of the search-order, then this code won't catch it. It will work on any system that does not have any bug at all.
hughag...@yahoo.com
> > Here is a new version of my code with DEFINITIONS and GET-CONTEXT
> > fixed so they will work inside of colon definitions at compile-time.
> > This is only a slight upgrade from what I posted previously (once
> > again, comments are removed for brevity).
>
> Hugh, may I politely request that you _not_ remove comments for
> the sake of brevity. It would make learning much easier for me.
The code is just ANS-Forth except that I use { to define locals. This is available in the novice package, and there are several other implementations that are compatible. I also have COMMENT to comment out the test code, and COMMENT is available in the novice package. If anybody wants to use this without the novice package, they just need to have { available plus they need to delete the COMMENT and & lines.
Note that Anton Ertl wrote a word {: for defining locals that works like my { except that it does not zero out the locals to the right of the | sign. This doesn't affect code at all --- zeroing out those locals is done to help debugging, so that if the programmer forgets to explicitly initialize them they will be initialized to a known state and the program will behave the same every time that it is run (not behave differently depending upon what garbage values the locals were initialized to) --- consistency is a huge help in debugging --- getting different results with each program run, with no apparent reason, can be mind-blowing! Andrew Haley is most likely aware of this --- he is just playing dumb because he knows that Anton Ertl's code will automatically go into the Forth-200x standard without review, so he doesn't want to criticize it and then have to back down when Anton Ertl ignores him (as Anton Ertl ignored me when he first posted {: and I pointed out the problem with it).
Another strange thing about Anton Ertl's code, is that he named it {: rather than { that has been used in several implementations including mine (all derived from the John Hopkins implementation). He did this because SwiftForth uses { for multi-line comments, and the Forth-200x mandate is that all SwiftForth code has to be Forth-200x compliant without any modification whatsoever --- so, rather than switch to my COMMENT he sticks with { as the multi-line comment word and uses the ugly {: for local definitions --- and to heck with everybody who has been using { for local definitions for many years.
> In the course of my part-time studies, I have seen references to
> numerous Forth stacks: parameter, return, control, leave, float,
> etc. It is my impression that only the first two are omni-present,
> but that the others can provide useful work-arounds for more
> advanced coding techniques. Do you (or anyone else here) have
> any opinions on the use of one of the others for implementing
> locals, or perhaps even a dedicated local stack, if one doesn't
> already exist?
Some examples of this problem in ANS-Forth (just a few):
1.) Local variables "may" be held on the return-stack, rather than have their own stack. The result is that the standard document gives us weird restrictions as the two uses of the return-stack clash (13.3.3):
- Locals shall not be declared until values previously placed on the return stack within the definition have been removed;
- After a definition's locals have been declared, a program may place data on the return stack. However, if this is done, locals shall not be accessed until those values have been removed from the return stack;
2.) Control-flow data "may" be held on the parameter-stack, rather than have their own stack. The result is that, when you write a meta-compiling word (see 1ARRAY etc. in the novice package), you don't have access to the parameter-stack in your compile-time code because there may or may not be control-flow data on top of your data (even colon itself sometimes pushes control-flow data), and you don't know if it is there or not, or how big it is.
3.) Float data "may" be held on the parameter-stack, but we don't know how big the float data is.
4.) Double-size data is held on the parameter-stack along with single-size data. This can be worked with, because you know how big it is (two cells), but it results in a lot of complicated stack-juggling that could be gotten rid of if the doubles had their own stack distinct from the singles.
In my language, I will have all of these things separated so they don't clash.
When Pelc uses the word-list search-order stack for internal uses, he is following this same bad pattern --- he is using a resource that is supposed to be for the programmer's exclusive use, for internal system use. His attitude seems to be: "Well, I will just push LOCALVARS on top of the user's wids, and the user will just have to notice this and work around it, and VFX still gets to be called ANS-Forth compliant even though VFX will no longer run ANS-Forth programs that don't work around VFX's internal use of this public resource."
Andrew Haley
>
> The code is just ANS-Forth except that I use { to define
> locals. This is available in the novice package, and there are
> several other implementations that are compatible. I also have
> COMMENT to comment out the test code, and COMMENT is available in
> the novice package. If anybody wants to use this without the novice
> package, they just need to have { available plus they need to delete
> the COMMENT and & lines.
>
> Note that Anton Ertl wrote a word {: for defining locals that works
> like my { except that it does not zero out the locals to the right
> of the | sign. This doesn't affect code at all --- zeroing out those
> locals is done to help debugging,
> so that if the programmer forgets to explicitly initialize them they
> will be initialized to a known state and the program will behave the
> same every time that it is run (not behave differently depending
> upon what garbage values the locals were initialized to) ---
> consistency is a huge help in debugging --- getting different
> results with each program run, with no apparent reason, can be
> mind-blowing! Andrew Haley is most likely aware of this --- he is
> just playing dumb because he knows that Anton Ertl's code will
> automatically go into the Forth-200x standard without review, so he
> doesn't want to criticize it and then have to back down when Anton
> Ertl ignores him (as Anton Ertl ignored me when he first posted {:
> and I pointed out the problem with it).
declaring locals:
https://groups.google.com/d/msg/comp.lang.forth/5FM16LcHvo8/Lv4hw3DB8_IJ
It was a long time ago, but IIRC I argued in the Forth 200x meeting
against it going in. However, it was explained to me that it was in
use in many projects, with no problems, so I gave up because I was in
a small minority and could not win.
> Another strange thing about Anton Ertl's code, is that he named it
> {: rather than { that has been used in several implementations
> including mine (all derived from the John Hopkins implementation).
existing code.
Andrew.
Albert van der Horst
>hughag...@yahoo.com wrote:
>>
>> The code is just ANS-Forth except that I use { to define
>> locals. This is available in the novice package, and there are
>> several other implementations that are compatible. I also have
>> COMMENT to comment out the test code, and COMMENT is
>available in
>> the novice package. If anybody wants to use this
>without the novice
>> package, they just need to have { available plus they
>need to delete
>> the COMMENT and & lines.
>>
>> Note that Anton Ertl wrote a word {: for defining
>locals that works
>> like my { except that it does not zero out the locals
>to the right
>> of the | sign. This doesn't affect code at all ---
>zeroing out those
>> locals is done to help debugging,
>
>That won't help debugging at all.
I've done more than one complicated debugging job.
The last thing you need is that you can't compare the
results of your small change, because there are other
random gratitious changes resulting from uninitialised
memory.
It may be an exaggeration to say that it *helps* debugging,
but it removes one obstacle that's for sure.
>Andrew.
Bernd Paysan
>>That won't help debugging at all.
>
> As a general principle: Hugh is right about this.
you want "implicitely initialized", that's fine with me, but then initialize
to zero (for values and pointers). Programs simply behave better if you
make sure everything is deterministic. If zero isn't the right value to
initialize, you'll quickly figure out, and explicitely initialize; but if
it's uninitialized, changes are high that it is 95% zero (especially on the
first run in the clean system where you enter the definition on the command
line to try it), and your program crashes unexpectedly in the 5% where it's
"some leftover" instead.
--
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://bernd-paysan.de/
Anton Ertl
based on (LOCAL)
<http://www.forth200x.org/reference-implementations/extended-locals.fs>,
and that implementation uses a fixed value (12345) for initializing
the locals after "|". I did not use 0, because that is more likely to
hide a bug that might surface on an implementation that really does
not initialize. The fixed value was necessitated by the fundament on
which I built {: (there is no uninitialized local with (LOCAL)), but
maybe I would have done it even if I had, say (UNINITIALIZED-LOCAL),
at my disposal.
Indeed I think that both uninitialized and implicitly initialized
locals are a bad idea in most cases. The way to go is to define the
local when you have the initial value for it, and with Gforth's locals
that is possible in all but very few cases.
The desire for uninitialized locals was vehemently expressed by a
vendor whose locals implementation imposes restrictions that make it
impossible in many cases to define a local when you have the initial
value for it. So his idea is to define locals uninitialized in a
place where his locals implementation allows you to define locals, and
then initialize it later, with TO.
Hmm, given that he still implements all locals on the in-memory return
stack, it should not be that hard to extend his implementation to
accept something close to the Gforth-supported usage, with one
restriction still remaining: The first locals definition must happen
outside any control structure. When you see another locals
definition, you backpatch the return-stack reservation produced by the
first locals definition to increase the number of reserved locals, and
store the newly defined locals in the new slots.
He would also need to implement the scoping of locals, and that can
also be done in a single pass (and is done so by Gforth), but my guess
is that he did not like the define-locals-where-needed approach
because of code generation issues.
- anton
--
M. Anton Ertl http://www.complang.tuwien.ac.at/anton/home.html
comp.lang.forth FAQs: http://www.complang.tuwien.ac.at/forth/faq/toc.html
New standard: http://www.forth200x.org/forth200x.html
EuroForth 2014: http://www.euroforth.org/ef14/
Bernd Paysan
> Hmm, given that he still implements all locals on the in-memory return
> stack, it should not be that hard to extend his implementation to
> accept something close to the Gforth-supported usage, with one
> restriction still remaining: The first locals definition must happen
> outside any control structure. When you see another locals
> definition, you backpatch the return-stack reservation produced by the
> first locals definition to increase the number of reserved locals, and
> store the newly defined locals in the new slots.
the return stack for locals is when your code contains >r and r>, and
declares the locals in between.
So let's see how VFX does cope with that today:
: test >r { a b } a b + r> . . ; ok
1 2 3 test 134520832 3
CS=0023 DS=002B ES=002B SS=002B FS=0000 GS=0063
EAX=0000:0001 EBX=091A:F000 ECX=091A:EF78 EDX=0000:0001
ESI=091C:0000 EDI=080B:F345 EBP=091B:EFC4 ESP=091A:EFB0
EIP=0000:0003 EFLAGS=0001:0207
--- RS top ---
Oops. So I would declare that possible problem a non-issue, because it
doesn't work right now, either, and it fails, because declaring locals
pushes a "clean up the locals" address on the return stack.
If your >r and r> are balanced without declaring new locals in between (and
VFX can handle that quite well), you can just allocate locals on the return
stack using exactly the same algorithm we use in Gforth to allocate locals
on the locals stack.
> He would also need to implement the scoping of locals, and that can
> also be done in a single pass (and is done so by Gforth), but my guess
> is that he did not like the define-locals-where-needed approach
> because of code generation issues.
i.e. it is just a virtual register, assigned to a value on the stack, given
a place on a virtual locals stack (which may actually live on the return
stack, but allocation of the maximum space needed in a word is moved to the
beginning, and deallocation is attached to every compiled EXIT) to be saved
when the register allocater runs out of registers (or needs to caller-save
registers when calling other words), and that's it - only those locals which
need saving in between need to be allocated on that stack, all locals that
can be kept in registers and are dead before they need saving don't. That's
what an analytical compiler is for, to take the whole word, and analyze the
total dataflow within. The sequence above { a b } a b + should compile to
the same code as a simple +.
It's just that in VFX, locals are second class to the optimizer, because
they don't happen enough to be worth the effort. That's a tradeoff
decision, MPE can't do everything.
BTW, that's what iForth 4.06 (64 bit Linux version, the last trial version I
have) does on the code above (properly Marcelified):
FORTH> : test >r params| a b | a b + r> . . ; ok
FORTH> 1 2 3 test 3 3 ok
So others have done some more effort ;-). The code it generates doesn't
fully convince me, but it is at least doing what you might expect.
Andrew Haley
> Indeed I think that both uninitialized and implicitly initialized
> locals are a bad idea in most cases. The way to go is to define the
> local when you have the initial value for it, and with Gforth's locals
> that is possible in all but very few cases.
some value. I don't much see the point of locals initialized to some
canonical "magic" value because the initial value is necesarily
application-dependent; maybe 0, maybe 1, maybe -1. On the other hand,
if someone really wants to create an uninitialized local I can't see
much harm in it. Even where the specification allows a local to be
uninitialized, an implementation could use DEADBEEF, and that's a good
idea.
Andrew.
m...@iae.nl
> BTW, that's what iForth 4.06 (64 bit Linux version, the last trial version I
> have) does on the code above (properly Marcelified):
> FORTH> : test >r params| a b | a b + r> . . ; ok
> FORTH> 1 2 3 test 3 3 ok
> So others have done some more effort ;-). The code it generates doesn't
> fully convince me, but it is at least doing what you might expect.
and locals :-)
FORTH> : (test) params| a b c | a b + c ; : test 1 2 3 (test) . . ; ok
FORTH> see test
Flags: ANSI
$01223200 : test
$0122320A push 3 b#
$0122320C push 3 b#
$0122320E lea rbp, [rbp -8 +] qword
$01223212 mov [rbp 0 +] qword, $0122321F d#
$0122321A jmp .+10 ( $0113A97A ) offset NEAR
$0122321F jmp .+10 ( $0113A97A ) offset NEAR
$01223224 ;
FORTH> : test2 1 2 3 (test) + drop ; ok
FORTH> ' test2 idis
$01223240 : test2
$0122324A ;
-marcel
Bernd Paysan
> On Tuesday, October 7, 2014 10:15:43 PM UTC+2, Bernd Paysan wrote:
>> BTW, that's what iForth 4.06 (64 bit Linux version, the last trial
>> version I have) does on the code above (properly Marcelified):
>
>> FORTH> : test >r params| a b | a b + r> . . ; ok
>> FORTH> 1 2 3 test 3 3 ok
>
>> So others have done some more effort ;-). The code it generates doesn't
>> fully convince me, but it is at least doing what you might expect.
>
> Well, the generator does not expect illogical combinations of R-stack use
> and locals :-)
probably good enough for this sort of "being cruel to the compiler" code
;-).
> FORTH> : (test) params| a b c | a b + c ; : test 1 2 3 (test) . . ; ok
> FORTH> see test
> Flags: ANSI
> $01223200 : test
> $0122320A push 3 b#
> $0122320C push 3 b#
> $0122320E lea rbp, [rbp -8 +] qword
> $01223212 mov [rbp 0 +] qword, $0122321F d#
> $0122321A jmp .+10 ( $0113A97A ) offset NEAR
> $0122321F jmp .+10 ( $0113A97A ) offset NEAR
> $01223224 ;
and rbp as return stack. The consequence is that you can't use call, but
you have to push return addresses pretty much manually on the return stack.
That comes at a cost: the CPU actually has an internal return stack used to
predict rets (to a limited level of call depth, which is probably enough for
Forth to work almost perfectly). The costs of using something other than
esp as data stack comes at a cost, too - you can't do push/pop. The way
push/pop are implemented now probably means just more code space used, but
no performance degradation if you do it manually - actually, IIRC, the
Pentium was the last x86 CPU which could do two pushs or pops in one cycle;
starting with the Pentium Pro, there's a sp update register dependency which
prevents the CPU to do more than one push/pop per cycle, while you certainly
can do more than one of those moves with offset per cycle (as long as you
merge all stack pointer updates to the start/end of one basic block).
Have you done some calculation on the tradeoffs? I'm pretty sure I would go
for rsp as return stack, and rbp or some other easily used pointer register
as data stack, even though then push/pop sequences are out of question. But
that's because Intel has stopped making them run real fast a long time ago.
> FORTH> : test2 1 2 3 (test) + drop ; ok
> FORTH> ' test2 idis
> $01223240 : test2
> $0122324A ;
to-compiler variant.
Anton Ertl
>Anton Ertl wrote:
>
>> Hmm, given that he still implements all locals on the in-memory return
>> stack, it should not be that hard to extend his implementation to
>> accept something close to the Gforth-supported usage, with one
>> restriction still remaining: The first locals definition must happen
>> outside any control structure. When you see another locals
>> definition, you backpatch the return-stack reservation produced by the
>> first locals definition to increase the number of reserved locals, and
>> store the newly defined locals in the new slots.
>
>I don't see any need for this sort of restriction; the only problem of using
>the return stack for locals is when your code contains >r and r>, and
>declares the locals in between.
such an implementation with a separate locals stack (indeed I started
out without specifying on which stack the locals were stored, and then
made it more concrete for easier unterstanding; but apparently also
easier misunderstanding).
This is about words like (from the Gforth tutorial):
: strcmp {: addr1 u1 addr2 u2 -- n :}
addr1 addr2
u1 u2 min 0
?do {: s1 s2 :}
s1 c@ s2 c@ -
?dup-if
unloop exit
then
s1 char+ s2 char+
loop
2drop
u1 u2 - ;
Because some systems cannot do this, we got the uninitialized locals,
so that code could be rewritten as:
: strcmp {: addr1 u1 addr2 u2 | s1 s2 -- n :}
addr1 addr2
u1 u2 min 0
?do to s2 to s1
s1 c@ s2 c@ -
?dup-if
unloop exit
then
s1 char+ s2 char+
loop
2drop
u1 u2 - ;
I am claiming that the compiler can do that for the programmer with
relatively little effort. The restriction I mentioned is to disallow
having the first locals definition inside a control structure, like
here:
: foo ( ... ) \ no locals definition here:
... begin {: a b :}
... while
... repeat
... ;
If you allow that, either the backpatching does not just need to
update the space used by locals, but has to insert the locals
allocation in the code before the control structure, or you have to
generate matched allocation and deallocation on every iteration the
way Gforth does it; but the latter option is probably seen as too
inefficient by the implementor (one argument for uninitialized locals
was efficiency).
>Actually, with an analytical compiler, locals could be implemented "right",
>i.e. it is just a virtual register, assigned to a value on the stack, given
>a place on a virtual locals stack (which may actually live on the return
>stack, but allocation of the maximum space needed in a word is moved to the
>beginning, and deallocation is attached to every compiled EXIT) to be saved
>when the register allocater runs out of registers (or needs to caller-save
>registers when calling other words), and that's it - only those locals which
>need saving in between need to be allocated on that stack, all locals that
>can be kept in registers and are dead before they need saving don't. That's
>what an analytical compiler is for, to take the whole word, and analyze the
>total dataflow within. The sequence above { a b } a b + should compile to
>the same code as a simple +.
I wanted to point out is that it does not take such a big effort to
remove the restrictions that led to uninitialized locals.
Anton Ertl
> On the other hand,
>if someone really wants to create an uninitialized local I can't see
>much harm in it.
that every read from the local produces a proper value. You cannot
read the uninitialized value.
Does someone really *want* to create an uninitialized local, or is it
just that the programming language does not support what the
programmer really wants, and the uninitialized local is a workaround
for that? E.g., in Forth-2012 the uninitialized local is a workaround
for disallowing to define locals everywhere in a definition.
So the harm of uninitialized locals to the programmer is that he may
make errors that would be caught (as "undefined word") if the locals
were defined later, where they are properly initialized. The harm to
the language is that it allows restrictions such as "only one locals
definition in a colon definition" and "no locals definition inside
control structures" to persist.
Andrew Haley
> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>> On the other hand, if someone really wants to create an
>>uninitialized local I can't see much harm in it.
>
> The benefit of defining a local where it is properly initialized is
> that every read from the local produces a proper value. You cannot
> read the uninitialized value.
if you're only allowed to define locals in one place.
> Does someone really *want* to create an uninitialized local, or is it
> just that the programming language does not support what the
> programmer really wants, and the uninitialized local is a workaround
> for that? E.g., in Forth-2012 the uninitialized local is a workaround
> for disallowing to define locals everywhere in a definition.
I'm not going to argue about the programmer convenience of being able
to declare locals as required, but it's not that easy for the
implementation and it brings up some tricky issues to do with
lifetimes. (Which I know you have an answer to in gforth.)
All other things being equal, simple implementations are better than
complex ones, and locals aren't important enough to justify making
implementations any more complex than they are at present. IMO, YMMV.
> So the harm of uninitialized locals to the programmer is that he may
> make errors that would be caught (as "undefined word") if the locals
> were defined later, where they are properly initialized.
gun, and foot. Initialized locals are already provided, and if you
prefer to initialize locals, add all of them to the initialized locals
list with whatever constant you prefer as the initial value. The
problem is already solved.
> The harm to the language is that it allows restrictions such as
> "only one locals definition in a colon definition" and "no locals
> definition inside control structures" to persist.
do extra work to support that. So, it's nice to have, but that's all.
Andrew.
Anton Ertl
>The way
>push/pop are implemented now probably means just more code space used, but
>no performance degradation if you do it manually - actually, IIRC, the
>Pentium was the last x86 CPU which could do two pushs or pops in one cycle;
>starting with the Pentium Pro, there's a sp update register dependency which
>prevents the CPU to do more than one push/pop per cycle, while you certainly
>can do more than one of those moves with offset per cycle (as long as you
>merge all stack pointer updates to the start/end of one basic block).
more recent than the Pentium (too dim to remember whether that was
done by Intel or by AMD). But certainly, if the compiler is not
totally naive, the data stack pointer updates should not be that
frequent (and should cost almost nothing, as there is probably some
ALU around that would otherwise go unused).
Anton Ertl
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>> On the other hand, if someone really wants to create an
>>>uninitialized local I can't see much harm in it.
>>
>> The benefit of defining a local where it is properly initialized is
>> that every read from the local produces a proper value. You cannot
>> read the uninitialized value.
>
>I'm not going to argue with that advantage, but it's hard to do that
>if you're only allowed to define locals in one place.
>> Does someone really *want* to create an uninitialized local, or is it
>> just that the programming language does not support what the
>> programmer really wants, and the uninitialized local is a workaround
>> for that? E.g., in Forth-2012 the uninitialized local is a workaround
>> for disallowing to define locals everywhere in a definition.
>
>So that's OK, then. Well, it's not OK, it's all we have at present.
>I'm not going to argue about the programmer convenience of being able
>to declare locals as required, but it's not that easy for the
>implementation and it brings up some tricky issues to do with
>lifetimes. (Which I know you have an answer to in gforth.)
>
>All other things being equal, simple implementations are better than
>complex ones, and locals aren't important enough to justify making
>implementations any more complex than they are at present. IMO, YMMV.
present, and they provide exactly zero benefit. Where was your
defense of simple implementations when the addition of uninitialized
locals was being discussed?
>> So the harm of uninitialized locals to the programmer is that he may
>> make errors that would be caught (as "undefined word") if the locals
>> were defined later, where they are properly initialized.
>
>Sure, but I do not have to remind you that this is Forth: your bullet,
>gun, and foot. Initialized locals are already provided, and if you
>prefer to initialize locals, add all of them to the initialized locals
>list with whatever constant you prefer as the initial value. The
>problem is already solved.
you know the proper initial value, you will not catch a to-early read
of the local. This problem is not at all solved in Forth-2012.
Anyway, if you think that the existance of initialized locals solves
the problem, what problem is solved by uninitialized locals?
>> The harm to the language is that it allows restrictions such as
>> "only one locals definition in a colon definition" and "no locals
>> definition inside control structures" to persist.
>
>Yes. But OTOH, I'm not in favour of asking Forth implementations to
>do extra work to support that.
work to support uninitialized locals? Why?
Anton Ertl
>Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>> The benefit of defining a local where it is properly initialized is
>>> that every read from the local produces a proper value. You cannot
>>> read the uninitialized value.
>>
>>I'm not going to argue with that advantage, but it's hard to do that
>>if you're only allowed to define locals in one place.
>
>Lifting that restriction is not that hard.
implementation of that idea in
<2014Oct...@mips.complang.tuwien.ac.at>. Concerning the
automatic scoping,
<http://www.complang.tuwien.ac.at/papers/ertl94l.ps.gz> shows a
somewhat elaborate and quite precise way of how to do this. Here I
present a simpler approach:
The local is visible from the definition of the local until the end of
the colon definition. That lets some locals live longer than
described in the paper above earlier proposal (and one could argue
that the local will then be visible in places where it should not be
visible), but compared to the current situation of defining all the
locals at the start and letting them all be visible during the whole
definition, it is still an improvement.
For standardizing, I would suggest allowing different visibility
variants, from something block-structure-like to the simple approach
above. Yes, a weak standard is not nice, but at least in this case it
is something that shows up in compilation.
Andrew Haley
> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>> On the other hand, if someone really wants to create an
>>>>uninitialized local I can't see much harm in it.
>>>
>>> The benefit of defining a local where it is properly initialized is
>>> that every read from the local produces a proper value. You cannot
>>> read the uninitialized value.
>>
>>I'm not going to argue with that advantage, but it's hard to do that
>>if you're only allowed to define locals in one place.
>
> Lifting that restriction is not that hard.
implementations.
>>> Does someone really *want* to create an uninitialized local, or is it
>>> just that the programming language does not support what the
>>> programmer really wants, and the uninitialized local is a workaround
>>> for that? E.g., in Forth-2012 the uninitialized local is a workaround
>>> for disallowing to define locals everywhere in a definition.
>>
>>So that's OK, then. Well, it's not OK, it's all we have at present.
>
> And before, all we had in the standard were initialized locals.
>
>>I'm not going to argue about the programmer convenience of being able
>>to declare locals as required, but it's not that easy for the
>>implementation and it brings up some tricky issues to do with
>>lifetimes. (Which I know you have an answer to in gforth.)
>>
>>All other things being equal, simple implementations are better than
>>complex ones, and locals aren't important enough to justify making
>>implementations any more complex than they are at present. IMO, YMMV.
>
> Well, uninitialized locals make Gforth more complex than it is at
> present, and they provide exactly zero benefit. Where was your
> defense of simple implementations when the addition of uninitialized
> locals was being discussed?
uninitialized locals. As for the rest, Lord knows I tried -- and you
were there -- but IMO the whole syntax is a disaster. IIRC I argued
very strongly against it one year, but when it came back a year later
I gave up because I was told that it was successfully used in many
Forth projects, and I didn't think I had the support of the committee.
It is wrong in many ways, of which uninitialized locals are a fairly
minor part.
>>> So the harm of uninitialized locals to the programmer is that he may
>>> make errors that would be caught (as "undefined word") if the locals
>>> were defined later, where they are properly initialized.
>>
>>Sure, but I do not have to remind you that this is Forth: your bullet,
>>gun, and foot. Initialized locals are already provided, and if you
>>prefer to initialize locals, add all of them to the initialized locals
>>list with whatever constant you prefer as the initial value. The
>>problem is already solved.
>
> No, by defining a local and initializing it to a dummy value before
> you know the proper initial value, you will not catch a to-early read
> of the local. This problem is not at all solved in Forth-2012.
local? (I'm assuming we're not talking about multiple locals blocks,
but the single one that we have.)
> Anyway, if you think that the existance of initialized locals solves
> the problem, what problem is solved by uninitialized locals?
there is only one locals block in a word, you have to declare them all
at the top, and you don't have initial values for them yet.
>>> The harm to the language is that it allows restrictions such as
>>> "only one locals definition in a colon definition" and "no locals
>>> definition inside control structures" to persist.
>>
>>Yes. But OTOH, I'm not in favour of asking Forth implementations to
>>do extra work to support that.
>
> But you were in favour of asking Forth implementations to do extra
> work to support uninitialized locals? Why?
not be saved. It was already so hopelessly baroque that it was not
worth the effort.
Andrew.
Andrew Haley
> Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>
>>>I'm not going to argue with that advantage, but it's hard to do
>>>that if you're only allowed to define locals in one place.
>>
>> Lifting that restriction is not that hard.
>
> Ummm. I don't think that you can really say so for all
> implementations.
have vanother stack you need yet another stack pointer (we already
have three!) and you have to be able to allocate that stack somehow
for every thread. Also, the compiler has to keep track of lifetimes
at control flow joins and splits.
>>>All other things being equal, simple implementations are better than
>>>complex ones, and locals aren't important enough to justify making
>>>implementations any more complex than they are at present. IMO, YMMV.
>>
>> Well, uninitialized locals make Gforth more complex than it is at
>> present, and they provide exactly zero benefit. Where was your
>> defense of simple implementations when the addition of uninitialized
>> locals was being discussed?
>
> Firstly, I was not aware that there was any problem with implementing
> uninitialized locals.
problem implementing uninitialized locals. In your notation, the
runtime action is 0DEADBEEF >L (or DUP >L, to taste) for each local,
which is more complex than just >L , but not very much more so.
Andrew.
Gerry Jackson
> Thanks for taking the time to read my code. :-) Look at it again to see how LOCAL-VARS# gets set, as I think you didn't grok that part.
--
Gerry
Anton Ertl
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>>> On the other hand, if someone really wants to create an
>>>>>uninitialized local I can't see much harm in it.
>>>>
>>>> The benefit of defining a local where it is properly initialized is
>>>> that every read from the local produces a proper value. You cannot
>>>> read the uninitialized value.
>>>
>>>I'm not going to argue with that advantage, but it's hard to do that
>>>if you're only allowed to define locals in one place.
>>
>> Lifting that restriction is not that hard.
>
>Ummm. I don't think that you can really say so for all
>implementations.
that hard.
To support that claim, I have outlined how to do it in
<2014Oct...@mips.complang.tuwien.ac.at> and
<2014Oct...@mips.complang.tuwien.ac.at>.
>It is wrong in many ways, of which uninitialized locals are a fairly
>minor part.
that I am aware of.
>>>> So the harm of uninitialized locals to the programmer is that he may
>>>> make errors that would be caught (as "undefined word") if the locals
>>>> were defined later, where they are properly initialized.
>>>
>>>Sure, but I do not have to remind you that this is Forth: your bullet,
>>>gun, and foot. Initialized locals are already provided, and if you
>>>prefer to initialize locals, add all of them to the initialized locals
>>>list with whatever constant you prefer as the initial value. The
>>>problem is already solved.
>>
>> No, by defining a local and initializing it to a dummy value before
>> you know the proper initial value, you will not catch a to-early read
>> of the local. This problem is not at all solved in Forth-2012.
>
>And in what way will an initialized local catch a too-early read of a
>local? (I'm assuming we're not talking about multiple locals blocks,
>but the single one that we have.)
arises from the restriction to one locals definition that you want to
uphold. I pointed out that it did not. So you have to answer your
question yourself.
>> Anyway, if you think that the existance of initialized locals solves
>> the problem, what problem is solved by uninitialized locals?
>
>I think it's for compatibility with existing usage, isn't it? Because
>there is only one locals block in a word, you have to declare them all
>at the top, and you don't have initial values for them yet.
restriction (actually, the Forth-94 TC created it, we just upheld it).
Better get rid of the restriction and therefore the problem, then we
don't need that solution.
Anton Ertl
>Andrew Haley <andr...@littlepinkcloud.invalid> wrote:
>> Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>>
>>>>I'm not going to argue with that advantage, but it's hard to do
>>>>that if you're only allowed to define locals in one place.
>>>
>>> Lifting that restriction is not that hard.
>>
>> Ummm. I don't think that you can really say so for all
>> implementations.
>
>I just read your paper, and it looks rather complex to me.
<2014Oct...@mips.complang.tuwien.ac.at> (and
<2014Oct...@mips.complang.tuwien.ac.at>).
For anybody who wants to know what's in the paper, it can be found at
<http://www.complang.tuwien.ac.at/papers/ertl94l.ps.gz>.
> If you
>have vanother stack you need yet another stack pointer (we already
>have three!) and you have to be able to allocate that stack somehow
>for every thread.
have a locals stack without automatic scoping, and you can have
automatic scoping without locals stack. There are two reasons for the
locals stack:
1) Alignment of FP locals. If we kept locals on the return stack, we
would have to FP-align the return stack in a lot of places. On 32-bit
machines this costs padding space on the return stack and, if we want
to minimize padding, extra complexity.
2) Lift the Forth-94 restructions on combining return stack usage and
locals usage without introducing a lot of complexity.
>Also, the compiler has to keep track of lifetimes
>at control flow joins and splits.
the visibility, so that comes automatic with the scoping.
In the approach described in
<2014Oct...@mips.complang.tuwien.ac.at>, all locals have space
reserved for them from the first local definition until the end of the
definition, and there is no need to keep track of the lifetimes.
>>> Well, uninitialized locals make Gforth more complex than it is at
>>> present, and they provide exactly zero benefit.
>problem implementing uninitialized locals. In your notation, the
>runtime action is 0DEADBEEF >L (or DUP >L, to taste) for each local,
>which is more complex than just >L , but not very much more so.
complexity only slightly once you are there (but it does increase it).
Supporting the additional syntax is more effort. Still nothing
unsurmountable, but given that it provides 0 benefit ...
Looking at code for { (and {:) without and with the "uninitialized
locals" part, that additional complexity is actually significant:
Hayes [hayes92] already mentioned the possibility of uninitialized
locals, but implemented only the part without uninitialized locals and
left uninitialized locals as an "exercise to the reader".
When I created a downloadable form of his work
<http://www.complang.tuwien.ac.at/forth/anslocal.fs>, I considered
implementing the | part as suggested, but did not pull it through.
The result takes 21 lines of code, including >IN manipulation (which
is not present in some of the variants below).
I also did not implement | in Gforth (which uses quite a different
implementation approach); I think the reason is that in both cases the
additional complexity appeared excessive.
Stephen Pelc's "reference implementation" of {: with |
<http://www.forth200x.org/locals-v5.txt> assumes a magic word BUILDLV
whose implementation is not provided, and then still needs 55 lines of
code.
My first attempt at a reference implementation based on (LOCAL)
<http://www.forth200x.org/reference-implementations/extended-locals.fs>
takes 44 lines of code, and would be even longer if we used WORD and
>IN like anslocal.fs (instead, we use PARSE-NAME).
I later forgot about that implementation and made another one, this
time finding a better approach, resulting in 23 lines of code
<http://www.forth200x.org/reference-implementations/locals.fs>. But
as discussed in <2013Oct3...@mips.complang.tuwien.ac.at>, this
version would be longer if it used WORD and >IN like anslocal.fs.
Maybe I should write a version of anslocal.fs with PARSE-NAME to show
the real difference.
@Article{hayes92,
author = "John R. Hayes",
title = "User-Defined Local Variable Syntax with {ANS Forth}",
journal = sigforth,
year = "1992",
volume = "4",
number = "2",
OPTpages = "19, 20, 26",
annote = "Shows how to define a nice locals syntax using the ANS
Forth locals wordset."
Andrew Haley
> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>>>> On the other hand, if someone really wants to create an
>>>>>>uninitialized local I can't see much harm in it.
>>>>>
>>>>> The benefit of defining a local where it is properly initialized is
>>>>> that every read from the local produces a proper value. You cannot
>>>>> read the uninitialized value.
>>>>
>>>>I'm not going to argue with that advantage, but it's hard to do that
>>>>if you're only allowed to define locals in one place.
>>>
>>> Lifting that restriction is not that hard.
>>
>>Ummm. I don't think that you can really say so for all
>>implementations.
>
> Sure I can: For all implementations lifting that restriction is not
> that hard.
>
> To support that claim, I have outlined how to do it in
> <2014Oct...@mips.complang.tuwien.ac.at> and
> <2014Oct...@mips.complang.tuwien.ac.at>.
Locals are not such a big deal that it's worth so much effort.
>>It is wrong in many ways, of which uninitialized locals are a fairly
>>minor part.
>
> Uninitialized locals are the only part that's wrong in the proposal
> that I am aware of.
comment) on the right hand side of the "--". They are misleading.
With those included, and the implication that the locals declaration
be used instead of a stack comment, the whole proposal is so ugly that
I no longer care. This is what I objected to, at length, when the
proposal was on the table.
>>>>> So the harm of uninitialized locals to the programmer is that he may
>>>>> make errors that would be caught (as "undefined word") if the locals
>>>>> were defined later, where they are properly initialized.
>>>>
>>>>Sure, but I do not have to remind you that this is Forth: your bullet,
>>>>gun, and foot. Initialized locals are already provided, and if you
>>>>prefer to initialize locals, add all of them to the initialized locals
>>>>list with whatever constant you prefer as the initial value. The
>>>>problem is already solved.
>>>
>>> No, by defining a local and initializing it to a dummy value before
>>> you know the proper initial value, you will not catch a to-early read
>>> of the local. This problem is not at all solved in Forth-2012.
>>
>>And in what way will an initialized local catch a too-early read of a
>>local? (I'm assuming we're not talking about multiple locals blocks,
>>but the single one that we have.)
>
> It was you who claimed that initialized locals solve the problem that
> arises from the restriction to one locals definition that you want to
> uphold.
That's the problem it solves. But that's nothing to do with a
too-early read of a local.
> I pointed out that it did not. So you have to answer your question
> yourself.
>>> Anyway, if you think that the existance of initialized locals solves
>>> the problem, what problem is solved by uninitialized locals?
>>
>>I think it's for compatibility with existing usage, isn't it? Because
>>there is only one locals block in a word, you have to declare them all
>>at the top, and you don't have initial values for them yet.
>
> It's a solution to a problem that we created ourselves with this
> restriction (actually, the Forth-94 TC created it, we just upheld
> it).
> Better get rid of the restriction and therefore the problem, then we
> don't need that solution.
: strcmp {: addr1 u1 addr2 u2 -- n :}
addr1 addr2
u1 u2 min 0
?do {: s1 s2 :}
s1 c@ s2 c@ -
?dup-if
unloop exit
then
s1 char+ s2 char+
loop
2drop
u1 u2 - ;
refactored, and with that refactoring the need for the inner locals
declaration goes away.
Given a word -STR ( a1 a2 u), which does the inner comparison, this
becomes:
: @pair ( a1 a2 - a1 a2 c1 c2) over c@ over c@ ;
: step ( a1 a2 - a1' a2') swap 1+ swap 1+ ;
: -str ( a1 a2 u - n)
0 do
@pair - ?dup if unloop exit then
step
loop
2drop 0 ;
: strcmp {: addr1 u1 addr2 u2 -- n :}
addr1 addr2
u1 u2 min
u1 u2 - ;
And with a bit more work the need for any locals goes away altogether.
(Untested, and probably buggy. But you get the idea.)
The question in my mind is whether there is ever a word not in need of
refactoring which needs multiple locals declarations.
Andrew.
hughag...@yahoo.com
> There are two reasons for the
> locals stack:
>
> 1) Alignment of FP locals. If we kept locals on the return stack, we
> would have to FP-align the return stack in a lot of places. On 32-bit
> machines this costs padding space on the return stack and, if we want
> to minimize padding, extra complexity.
>
> 2) Lift the Forth-94 restructions on combining return stack usage and
> locals usage without introducing a lot of complexity.
Also, in the previous post you said:
"Uninitialized locals are the only part that's wrong in the proposal
that I am aware of."
This should be our new motto on C.L.F:
memegenerator.net/instance/53401411
Strictly speaking though, uninitialized locals aren't the "only part that's wrong," because we also have #2 above: locals "may" be on the return stack, and hence we have that weird restriction about how all >R data has to be removed when any local variable is accessed.
hughag...@yahoo.com
> On 04/10/2014 01:40, hughag...@yahoo.com wrote:
> > I doubt that Pelc thought that this "was not considered important enough to fix." More likely, he purposely strives to undermine anybody's independent effort to write a cross-compiler, because cross-compilers are how he makes his money primarily. This is also why he made sure that SMUDGE was not in ANS-Forth --- the only people who need SMUDGE are those writing cross-compilers --- people writing Forth application programs are happy so long as : and ; behave as if they are internally using SMUDGE and UNSMUDGE (aka HIDE and REVEAL) --- weirdly enough, the ANS-Forth document requires : and ; to behave as if they are using SMUDGE and UNSMUDGE, but makes no mention at all of SMUDGE (not even in the section on porting programs from old standards to ANS-Forth).
>
> You shouldn't ascribe machiavellian motives to everything that someone
> does. After all Stephen Pelc is not listed as a (presumably voting)
> member of the ANS Forth Technical Committee, only as a contributor. I
> prefer the cockup side of the cockup vs conspiracy theory (look up
> Hanlon's Razor that I hadn't previously heard of before. Off topic: as
> an aside that led me to Sturgeon's Law which I like).
In my ongoing study of VFX, I cycled through the complete "Tip of the Day" file. He has Hanlon's Razor in there:
"Never attribute to malice that which can be explained by stupidity."
Then, in the next tip, he has Richard Bos's Corollary:
"Stupidity maintained long enough is a form of malice"
;-)
Well, I have already written a work-around for the bug, so there is no point in beating up Stephen Pelc forever on the subject; I might as well maintain cordial relations --- I think that it is inevitable that Pelc will eventually write a compiler for my language. ANS-Forth is a dead end; he is aware of this, and has been for 20 years (that is why VFX contains >NAME SMUDGE and various other goodies denied to ANS-Forth programmers). I'm the only person on C.L.F. who knows how to implement quotations, and quotations are the future, so it is inevitable that the entire Forth community will eventually be assimilated --- resistance is futile!
My aside about uninitialized locals was trivial and unimportant compared to this bug regarding word-lists that cost me a week's labor in fixing. I find it humorous that everybody jumped on this triviality and analyzed it to death --- a rather obvious effort to divert the discussion away from the bug in VFX --- afaik, you are the only person who has read my code and understood what it does. And Pelc's silence is deafening...
hughag...@yahoo.com
> Well, the generator does not expect illogical combinations of R-stack use
> and locals :-)
In VFX's "Tip of the Day" file we have this:
"Einstein argued that there must be simplified explanations of nature, because God is not capricious or arbitrary. No such faith comforts the software engineer."
Fred Brooks, Jr.
In 13.3.3 we have this rule that you apparently consider to be perfectly logical:
- Locals shall not be declared until values previously placed on the return stack within the definition have been removed;
- After a definition's locals have been declared, a program may place data on the return stack. However, if this is done, locals shall not be accessed until those values have been removed from the return stack;
Stephen Pelc
wrote:
... much ranting ...
>> VFX seems to be quickly patched to have local support without further tes=
>ting or investigation.
>
>This may be an explanation for why VFX has a bug --- but it is not an expla=
>nation for why Pelc didn't fix the bug after Gerry told him about it.
VFX has had the brace notation since well before Forth-94. LOCALS| was
added later for compatibility.
>I think Pelc is not going to fix the bug, because he wants to make life as =
>difficult as possible for cross-compiler writers. But I have written a work=
>-around for the bug in VFX!
What a waste of effort. Yours was the first request that included a
usage case. However, since you just ranted, it had to wait until I got
back from holiday and EuroForth. We have now done some testing on the
fix, compiled and tested the big app with 1.1 million lines of source
code, and rebuilt two cross compilers and four versions of VFX.
Note that our cross compilers have been running for 30+ years and
don't need your particular tricks.
v4.71 of VFX Forth for Windows, Mac OS X, ARM Linux and Intel Linux
are now available.
Stephen
--
Stephen Pelc, steph...@mpeforth.com
MicroProcessor Engineering Ltd - More Real, Less Time
133 Hill Lane, Southampton SO15 5AF, England
tel: +44 (0)23 8063 1441, fax: +44 (0)23 8033 9691
web: http://www.mpeforth.com - free VFX Forth downloads
hughag...@yahoo.com
> >I think Pelc is not going to fix the bug, because he wants to make life as =
> >difficult as possible for cross-compiler writers. But I have written a work=
> >-around for the bug in VFX!
> What a waste of effort. Yours was the first request that included a
> usage case. However, since you just ranted, it had to wait until I got
> back from holiday and EuroForth. We have now done some testing on the
> fix, compiled and tested the big app with 1.1 million lines of source
> code, and rebuilt two cross compilers and four versions of VFX.
> Note that our cross compilers have been running for 30+ years and
> don't need your particular tricks.
When I first encountered the problem with SET-CONTEXT not working, I tried to use PUSH-CONTEXT and POP-CONTEXT for temporarily switching the context. This is possible, but it made the cross-compiler more complicated than necessary, and it is already pretty complicated by nature. If this is the "trick" that you use, then your cross-compiler is more complicated than mine --- I pity the luckless devil that you hire for the job of maintaining your cross-compiler. Note that MFX has been in continuous use for 20 years and the luckless devil who was my coworker back then has been maintaining it all of this time --- when I visited the other day, I was told that he had "grumbled" and had said that it was "complicated," but that it was always less expensive to maintain it than to rewrite it.
> v4.71 of VFX Forth for Windows, Mac OS X, ARM Linux and Intel Linux
> are now available.
I'm only interested in the free evaluation version of VFX, as I'm way to cheap to pay for a Forth compiler. I won't buy a commercial VFX version until I'm sure that VFX is either bug-free or there are work-arounds for any bugs that it may have --- I have already been stung for over $400 on the bug-ridden generally-awful SwiftForth, so I'm a lot more cautious with my money now.
Testing the bug-fix version of VFX isn't a priority as I have already got the work-around written and can continue to use the version with the bug in it --- also, my work-around may still be necessary for other ANS-Forth implementations that have a similar bug (I haven't tested SwiftForth or Gforth or anything else). I want my cross-compiler to be ANS-Forth, and not vendor-specific, which is why I don't use your >NAME or SMUDGE etc., that you recommended, although they would help immensely. If you knew that this stuff was useful, why didn't you push it through into the ANS-Forth standard? Perhaps you brought up the issue and Elizabeth Rather said: "Quiet in the peanut-gallery!" I can tell from Ray Duncan's remarks above that he didn't like where ANS-Forth was going, but he had been relegated to the peanut-gallery along with you and Charles Moore etc., and the chair wasn't listening to him --- he seems to have just abandoned Forth altogether after ANS-Forth came out.
Anton Ertl
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>>>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>>>>> On the other hand, if someone really wants to create an
>>>>>>>uninitialized local I can't see much harm in it.
>>>>>>
>>>>>> The benefit of defining a local where it is properly initialized is
>>>>>> that every read from the local produces a proper value. You cannot
>>>>>> read the uninitialized value.
>>>>>
>>>>>I'm not going to argue with that advantage, but it's hard to do that
>>>>>if you're only allowed to define locals in one place.
>>>>
>>>> Lifting that restriction is not that hard.
>>>
>>>Ummm. I don't think that you can really say so for all
>>>implementations.
>>
>> Sure I can: For all implementations lifting that restriction is not
>> that hard.
>>
>> To support that claim, I have outlined how to do it in
>> <2014Oct...@mips.complang.tuwien.ac.at> and
>> <2014Oct...@mips.complang.tuwien.ac.at>.
>
>Hmmm. I suppose "not that hard" is in the eye of the beholder.
>Locals are not such a big deal that it's worth so much effort.
number for that. To me it seems that implementing this simple
approach is very little effort. Or do you mean to write that locals
are not worth any effort.
>The part I really object to is the dummy return values (effectively a
>comment) on the right hand side of the "--". They are misleading.
Do you really think that
: strcmp ( addr1 u1 addr2 u2 -- n )
{: addr1 u1 addr2 u2 :}
... ;
is an improvement over
: strcmp {: addr1 u1 addr2 u2 -- n :}
?
>>>>>> So the harm of uninitialized locals to the programmer is that he may
>>>>>> make errors that would be caught (as "undefined word") if the locals
>>>>>> were defined later, where they are properly initialized.
>>>>>
>>>>>Sure, but I do not have to remind you that this is Forth: your bullet,
>>>>>gun, and foot. Initialized locals are already provided, and if you
>>>>>prefer to initialize locals, add all of them to the initialized locals
>>>>>list with whatever constant you prefer as the initial value. The
>>>>>problem is already solved.
>>>>
>>>> No, by defining a local and initializing it to a dummy value before
>>>> you know the proper initial value, you will not catch a to-early read
>>>> of the local. This problem is not at all solved in Forth-2012.
>>>
>>>And in what way will an initialized local catch a too-early read of a
>>>local? (I'm assuming we're not talking about multiple locals blocks,
>>>but the single one that we have.)
>>
>> It was you who claimed that initialized locals solve the problem that
>> arises from the restriction to one locals definition that you want to
>> uphold.
>
>It does. It's a way of declaring locals that may be defined later.
>That's the problem it solves. But that's nothing to do with a
>too-early read of a local.
allowing only one locals definition are not a solution to that
problem.
>> I pointed out that it did not. So you have to answer your question
>> yourself.
>
>I no longer have any idea what you are talking about.
problem is already solved." (full quote above). Later you asked "And
I am not to lazy to trim quotes, these quotes are included because
they provide the context.
Anyway, concerning your factored solution, I find it pretty horrible
in many respects, but there's no accounting for tastes.
Yes, in this example one can do without the inner locals relatively
easily, because there are only two cells permanently alive in the
loop. No need for any factoring:
: strcmp {: addr1 u1 addr2 u2 -- n :}
addr1 addr2
u1 u2 min 0
over c@ over c@ -
?dup-if
nip nip unloop exit
then
over char+ over char+
2drop
u1 u2 - ;
single-assignment style over one that uses TO on locals:
: strcmp {: addr1 u1 addr2 u2 -- n :}
?do
?dup-if
unloop exit
then
addr1 char+ TO addr1
addr2 char+ TO addr2
loop
u1 u2 - ;
The original purpose was not to demonstrate the need for inner locals.
I could now search for one example where I used it in production code,
but examples from production code have the problem that they are
usually hard to understand without a ton of context.
>The question in my mind is whether there is ever a word not in need of
>refactoring which needs multiple locals declarations.
note that you used no uninitialized locals in your version of strcmp,
but you used the "--" feature:-).
I guess if I take the trouble to search for production code with
multiple locals definitions, and challenge you to refactor them, the
result will be equally deafening silence as for my repeated challanges
to refactor locals-containing code in a better way without locals (and
two of the examples are really much too big). Here's the challenge
from two years ago <2012Oct1...@mips.complang.tuwien.ac.at>:
|Concerning factoring, it helps in some cases, but not always. I put
|up some cases where I could not come up with a factoring that
|eliminates locals here as a challenge
|<2003Jan2...@a0.complang.tuwien.ac.at>
|<a3j4aa$a23$1...@news.tuwien.ac.at>
|<2009Sep1...@mips.complang.tuwien.ac.at>, but nobody has taken up
|the challenge.
Andrew Haley
> To me it seems that implementing this simple approach is very little
> effort. Or do you mean to write that locals are not worth any
> effort.
elaborate system as we have now is that some vendors want to keep some
customers who don't get it (and will probably never get it) happy.
IMO.
>>The part I really object to is the dummy return values (effectively a
>>comment) on the right hand side of the "--". They are misleading.
>
> When you know the notation, they are not misleading, but convenient.
>
> Do you really think that
>
> : strcmp ( addr1 u1 addr2 u2 -- n )
> {: addr1 u1 addr2 u2 :}
> ... ;
>
> is an improvement over
>
> : strcmp {: addr1 u1 addr2 u2 -- n :}
> ... ;
>
> ?
stack comment, and in this form it encourages people to put all args
in locals, even those which need no local. This is very bad.
> Anyway, concerning your factored solution, I find it pretty horrible
> in many respects, but there's no accounting for tastes.
>
> Yes, in this example one can do without the inner locals relatively
> easily, because there are only two cells permanently alive in the
> loop. No need for any factoring:
>>The question in my mind is whether there is ever a word not in need of
>>refactoring which needs multiple locals declarations.
>
> If there is not, there is no need for uninitialized locals, either.
> I note that you used no uninitialized locals in your version of
> strcmp, but you used the "--" feature:-).
myself.
> I guess if I take the trouble to search for production code with
> multiple locals definitions, and challenge you to refactor them, the
> result will be equally deafening silence as for my repeated challanges
> to refactor locals-containing code in a better way without locals (and
> two of the examples are really much too big)
> Here's the challenge from two years ago
> <2012Oct1...@mips.complang.tuwien.ac.at>:
>
> |Concerning factoring, it helps in some cases, but not always. I put
> |up some cases where I could not come up with a factoring that
> |eliminates locals here as a challenge
> |<2003Jan2...@a0.complang.tuwien.ac.at>
> |<a3j4aa$a23$1...@news.tuwien.ac.at>
> |<2009Sep1...@mips.complang.tuwien.ac.at>, but nobody has taken up
> |the challenge.
use multiple locals declarations.
MAP-ARRAY is an example of something that could easily and more
efficiently and more idiomatically be done with a compiling word, like
so:
: ` postpone postpone ; immediate
: map-array ( xt) >r
:
` cells ` over ` + ` swap ` do
` i ` @ r> compile,
` cell ` +loop ` ; ;
Or, perhaps better:
: <map-array ` cells ` over ` + ` swap ` do ` i ` @ ; immediate
: map-array> ` cell ` +loop ; immediate
Of course there will always be examples of (rather contrived?)
problems which resist nice factoring. The point with Forth, as
always, is to find a solution which is nicely factored, and if that
means taking a different approach from the one you first thought of,
so be it.
Incidentally, I have on at least one occasion here had to resort to
locals, but that's pretty rare, and not something which requires an
elaborate locals facility. So no, I am not saying that there is never
any need for locals.
Andrew.
Anton Ertl
>> To me it seems that implementing this simple approach is very little
>> effort. Or do you mean to write that locals are not worth any
>> effort.
>
>They're not worth any further effort.
effort at all, i.e., this is just a restatement of your dislike of
locals.
Anyway, my guess is that one could lift that restriction with about 10
lines of code, but of course that depends on what the original code
looks like.
>>>The part I really object to is the dummy return values (effectively a
>>>comment) on the right hand side of the "--". They are misleading.
>>
>> When you know the notation, they are not misleading, but convenient.
>>
>> Do you really think that
>>
>> : strcmp ( addr1 u1 addr2 u2 -- n )
>> {: addr1 u1 addr2 u2 :}
>> ... ;
>>
>> is an improvement over
>>
>> : strcmp {: addr1 u1 addr2 u2 -- n :}
>> ... ;
>>
>> ?
>
>Yes although I would not do either.
>A locals declaration is not a
>stack comment, and in this form it encourages people to put all args
>in locals, even those which need no local. This is very bad.
>> Yes, in this example one can do without the inner locals relatively
>> easily, because there are only two cells permanently alive in the
>> loop. No need for any factoring:
>
>Well-factored words are the goal, not something to be avoided!
>> I guess if I take the trouble to search for production code with
>> multiple locals definitions, and challenge you to refactor them, the
>> result will be equally deafening silence as for my repeated challanges
>> to refactor locals-containing code in a better way without locals (and
>> two of the examples are really much too big)
>
>Probably. Like you, I have at least one job already.
>> Here's the challenge from two years ago
>> <2012Oct1...@mips.complang.tuwien.ac.at>:
>>
>> |Concerning factoring, it helps in some cases, but not always. I put
>> |up some cases where I could not come up with a factoring that
>> |eliminates locals here as a challenge
>> |<2003Jan2...@a0.complang.tuwien.ac.at>
>> |<a3j4aa$a23$1...@news.tuwien.ac.at>
>> |<2009Sep1...@mips.complang.tuwien.ac.at>, but nobody has taken up
>> |the challenge.
>
>My news server can't find the first two.
<http://www.complang.tuwien.ac.at/anton/search.html> (but works only
with JavaScript).
>The last one doesn't seem to
>use multiple locals declarations.
of the people who write "factoring makes locals unnecessary" are just
hot air.
>MAP-ARRAY is an example of something that could easily and more
>efficiently and more idiomatically be done with a compiling word, like
>so:
>
>: ` postpone postpone ; immediate
>: map-array ( xt) >r
> :
> ` cells ` over ` + ` swap ` do
> ` i ` @ r> compile,
> ` cell ` +loop ` ; ;
: foo ... [ ' + ] map-array ... ;
instead of
: foo ... ['] + map-array ... ;
Not so nice.
>Or, perhaps better:
>
>: <map-array ` cells ` over ` + ` swap ` do ` i ` @ ; immediate
>: map-array> ` cell ` +loop ; immediate
: foo ... <map-array + map-array> ... ;
Neither is usable interpretively, unlike the original MAP-ARRAY.
What I find interesting in the original MAP-ARRAY, though, is that,
for such a simple word, the restrictions on combining the return stack
and DO loops make it really cumbersome to write such a word without
locals. You circumvent this by passing the xt at a different time
level in your first variant, and by not passing an xt at all in the
latter variant.
Marcel Hendrix
> BTW: I know you have made the design decision of using rsp as data stack,
> and rbp as return stack. The consequence is that you can't use call, but
> you have to push return addresses pretty much manually on the return stack.
this entry in iForth.prf:
[DEFINED] XCALL-used@
[IF] TRUE XCALL-used! ( jmp imm | call )
[THEN]
E.g.
FORTH> TRUE XCALL-used! ok
FORTH> see test
Flags: ANSI
$01220A8A push 3 b#
$01220A8C push 3 b#
$01220A8E lea rbp, [rbp -8 +] qword
$01220A92 mov [rbp 0 +] qword, $01220A9F d#
$01220A9A jmp .+10 ( $0113A97A ) offset NEAR
$01220A9F jmp .+10 ( $0113A97A ) offset NEAR
$01220AA4 ;
FORTH> FALSE XCALL-used! ok
Redefining test ok
Flags: ANSI
$01220B4A push 3 b#
$01220B4C push 3 b#
$01220B4E call . ( $0113A970 ) offset NEAR
$01220B53 jmp .+10 ( $0113A97A ) offset NEAR
$01220B58 ;
You may note that all iForth non-inlined words have a standard prologue
than can be skipped.
> That comes at a cost: the CPU actually has an internal return stack used to
> predict rets (to a limited level of call depth, which is probably enough for
> Forth to work almost perfectly). The costs of using something other than
> no performance degradation if you do it manually - actually, IIRC, the
> Pentium was the last x86 CPU which could do two pushs or pops in one cycle;
> starting with the Pentium Pro, there's a sp update register dependency which
> prevents the CPU to do more than one push/pop per cycle, while you certainly
> can do more than one of those moves with offset per cycle (as long as you
> merge all stack pointer updates to the start/end of one basic block).
practice, push and pop proved to be slow no matter what I did. TRUE XCALL-used!
is better than FALSE XCALL-used!. One possible explanation might be that the
internal return stack looked at ALL rsp-related operations (not just calls).
At the time, I thought having 16 registers would minimize the number of push/pops
to a very low number and that therefore the effect, if it existed at all, would
not be noticeable.
Maybe I should do these benchmarks again.
Experiments with mxforth (oh my, how long ago that is) indicated that
having rbp for the datastack is MUCH better than rsp. Also, caching the top of the
(rbp) data stack in a register is much better than letting the compiler
allocate it, especially if I want to win a particular standard (recursive)
benchmark.
> Have you done some calculation on the tradeoffs? I'm pretty sure I would go
> for rsp as return stack, and rbp or some other easily used pointer register
> as data stack, even though then push/pop sequences are out of question. But
> that's because Intel has stopped making them run real fast a long time ago.
directly.
Changing rbp<->rsp is on my list of things to do. (It is a long list and at
the moment I much prefer using Forth over 'improving' it).
-marcel
Andrew Haley
There's not much argument about how to do it.
>>> To me it seems that implementing this simple approach is very little
>>> effort. Or do you mean to write that locals are not worth any
>>> effort.
>>
>>They're not worth any further effort.
>
> Ah, ok. It's not that it is "so much effort", it's that it is any
> effort at all, i.e., this is just a restatement of your dislike of
> locals.
seem to me to be associated with unfactored code, and the more they
are used the worse that is. Therefore, any syntax which postively
encouarages an "all or nothing" approach to locals will, in the main,
make code worse.
> Anyway, my guess is that one could lift that restriction with about 10
> lines of code, but of course that depends on what the original code
> looks like.
>>Yes although I would not do either.
>
> Well, to anyone skilled in the art code redundancy in is bad.
>>A locals declaration is not a
>>stack comment, and in this form it encourages people to put all args
>>in locals, even those which need no local. This is very bad.
>
> Why?
items in locals, even when there is no need to do so.
>>> Yes, in this example one can do without the inner locals relatively
>>> easily, because there are only two cells permanently alive in the
>>> loop. No need for any factoring:
>>
>>Well-factored words are the goal, not something to be avoided!
>
> Sure, but your version is not *well*-factored code.
"there is no need for factoring", and though factoring is a scarce
resource that might be used up, or something to avoid. I don't know
why you are talking like that.
>>> Here's the challenge from two years ago
>>> <2012Oct1...@mips.complang.tuwien.ac.at>:
>>>
>>> |Concerning factoring, it helps in some cases, but not always. I put
>>> |up some cases where I could not come up with a factoring that
>>> |eliminates locals here as a challenge
>>> |<2003Jan2...@a0.complang.tuwien.ac.at>
>>> |<a3j4aa$a23$1...@news.tuwien.ac.at>
>>> |<2009Sep1...@mips.complang.tuwien.ac.at>, but nobody has taken up
>>> |the challenge.
>>
>>My news server can't find the first two.
>
> You can look them up on google through
> <http://www.complang.tuwien.ac.at/anton/search.html> (but works only
> with JavaScript).
it now.
>>The last one doesn't seem to use multiple locals declarations.
>
> The others do, although I mentioned them here to show that the claims
> of the people who write "factoring makes locals unnecessary" are just
> hot air.
until people give up from boredom or exhaustion. If I were to look at
something monstrous like F>BUF-RDP-TRY it'd take me half a day just to
figure out how it works.
But factoring surely at least *reduces* the need for multiple locals
declarations, because one can factor out an inner block. Well, modulo
things which make factoring problematic such as UNLOOP .
Why are the exponential and fixed-point notations not factored out of
F>BUF-RDP-TRY , anyway?
>>MAP-ARRAY is an example of something that could easily and more
>>efficiently and more idiomatically be done with a compiling word, like
>>so:
>>
>>: ` postpone postpone ; immediate
>>: map-array ( xt) >r
>> :
>> ` cells ` over ` + ` swap ` do
>> ` i ` @ r> compile,
>> ` cell ` +loop ` ; ;
>
> Used like
>
> : foo ... [ ' + ] map-array ... ;
>
> instead of
>
> : foo ... ['] + map-array ... ;
' + map-array +array
then
: foo ... +array ... ;
> Not so nice.
>
>>Or, perhaps better:
>>
>>: <map-array ` cells ` over ` + ` swap ` do ` i ` @ ; immediate
>>: map-array> ` cell ` +loop ; immediate
>
> Used like
>
> : foo ... <map-array + map-array> ... ;
>
> Neither is usable interpretively, unlike the original MAP-ARRAY.
use" excludes throwaway definitions, and I can't imagine why it would.
> What I find interesting in the original MAP-ARRAY, though, is that,
> for such a simple word, the restrictions on combining the return stack
> and DO loops make it really cumbersome to write such a word without
> locals.
allowed access to the third item on the return stack, but even if not
it's possible to add a primitive to do so. This is cumbersome for
*portable* Forth code, but for the kind of turnkey application Forth
has shined at it doesn't matter.
> You circumvent this by passing the xt at a different time level in
> your first variant, and by not passing an xt at all in the latter
> variant.
unless the XT really varies at runtime. So, I prefer
0 foo n <map-array + map-array>
to
0 foo n ['] + map-array
LISPers might like that kind of thing, I suppose.
Andrew.
Anton Ertl
the effort. E.g., if you think it takes changes in ten lines, but
that it's not worth that effort, your statement is more about what you
think the value of the change is, whereas if you think it takes
changes in 10000 lines, it's more a statement about what you think the
effort is.
>> Ah, ok. It's not that it is "so much effort", it's that it is any
>> effort at all, i.e., this is just a restatement of your dislike of
>> locals.
>
>Not exactly, no. It's not dislike per se: as I've said before, locals
>seem to me to be associated with unfactored code, and the more they
>are used the worse that is. Therefore, any syntax which postively
>encouarages an "all or nothing" approach to locals will, in the main,
>make code worse.
all-or-nothing approach? Requiring that all locals are defined in one
definition seems to be much more all-or-nothing to me.
>> Well, to anyone skilled in the art code redundancy in is bad.
>
>I absolutely agree, but it's the lesser of two evils. IMO.
such cases.
>>>A locals declaration is not a
>>>stack comment, and in this form it encourages people to put all args
>>>in locals, even those which need no local. This is very bad.
>>
>> Why?
>
>This should surely be obvious. People are being encouraged to put
>items in locals, even when there is no need to do so.
>>>> Yes, in this example one can do without the inner locals relatively
>>>> easily, because there are only two cells permanently alive in the
>>>> loop. No need for any factoring:
>>>
>>>Well-factored words are the goal, not something to be avoided!
>>
>> Sure, but your version is not *well*-factored code.
>
>Perhaps not, but you are rather avoiding the point. You are saying
>"there is no need for factoring", and though factoring is a scarce
>resource that might be used up, or something to avoid. I don't know
>why you are talking like that.
than the code with factoring (as in this case), then yes, this
factoring is something to avoid.
But actually, this was not a comment about the desirability of
factoring in general, but about the properties of the specific word at
hand.
>> The others do, although I mentioned them here to show that the claims
>> of the people who write "factoring makes locals unnecessary" are just
>> hot air.
>
>But we all know how this game works: more and more extreme examples
>until people give up from boredom or exhaustion.
had solved these examples and I would have come up with additional
ones, then your complaint would have a real basis. But as it is,
these examples have been around for a decade, and there has been very
little effort gone into factoring them, while claims about factoring
making locals unnecessary have been repeated pretty often during that
time.
Anyway, these are the most extreme examples I have from real-world
code, so I am unlikely to come up with new examples.
>If I were to look at
>something monstrous like F>BUF-RDP-TRY it'd take me half a day just to
>figure out how it works.
words would not be challenging, and prove nothing, would it?
>Why are the exponential and fixed-point notations not factored out of
>F>BUF-RDP-TRY , anyway?
had too many parameters and would not stand for an identifyable
concept (already F>BUF-RDP-TRY has that problem).
>>>MAP-ARRAY is an example of something that could easily and more
>>>efficiently and more idiomatically be done with a compiling word, like
>>>so:
>>>
>>>: ` postpone postpone ; immediate
>>>: map-array ( xt) >r
>>> :
>>> ` cells ` over ` + ` swap ` do
>>> ` i ` @ r> compile,
>>> ` cell ` +loop ` ; ;
>>
>> Used like
>>
>> : foo ... [ ' + ] map-array ... ;
>>
>> instead of
>>
>> : foo ... ['] + map-array ... ;
>
>Eh? No. Used like
>
>' + map-array +array
>
>then
>
>: foo ... +array ... ;
the code in other cases; but actually it's even cumbersome if the
defined word is used in several places.
>> What I find interesting in the original MAP-ARRAY, though, is that,
>> for such a simple word, the restrictions on combining the return stack
>> and DO loops make it really cumbersome to write such a word without
>> locals.
>
>That is true, and is a real disadvantage. Forths I have used have
>allowed access to the third item on the return stack, but even if not
>it's possible to add a primitive to do so.
system-specific and not easily comprehensible code elevates the
avoidance of locals to an end in and of itself.
I remember the horribly unfactored code that Wil Baden posted for
compression or encryption or such, all without locals. Maybe he did
it to show that locals-freedom does not guarantee good factoring, but
he actually seemed pretty serious to me.
>That is true. Passing XTs is, in general, something I would avoid
>unless the XT really varies at runtime. So, I prefer
>
> 0 foo n <map-array + map-array>
>
>to
>
> 0 foo n ['] + map-array
1) It cannot be used interpretively, which is not so nice.
2) Xt-passing has turned out to be a useful factoring technique.
E.g., consider SAVE-INPUT RESTORE-INPUT with all the many ways that
RESTORE-INPUT can fail, compared to properly nesting words like
EVALUATE. Now EVALUATE always INTERPRETs the string, so for doing
something else we have EXECUTE-PARSING ( ... xt -- ... ). And that
principle has been applied to other problems.
Albert van der Horst
>Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>That is true. Passing XTs is, in general, something I would avoid
>>unless the XT really varies at runtime. So, I prefer
>>
>> 0 foo n <map-array + map-array>
>>
>>to
>>
>> 0 foo n ['] + map-array
>
>Yes, in this case this works nicely, but
>
>1) It cannot be used interpretively, which is not so nice.
>
>2) Xt-passing has turned out to be a useful factoring technique.
>E.g., consider SAVE-INPUT RESTORE-INPUT with all the many ways that
>RESTORE-INPUT can fail, compared to properly nesting words like
>EVALUATE. Now EVALUATE always INTERPRETs the string, so for doing
>something else we have EXECUTE-PARSING ( ... xt -- ... ). And that
>principle has been applied to other problems.
Is it true that
EVALUATE
is equivalent to
' INTERPRET EXECUTE-PARSING
?
>
>-
Groetjes Albert
--
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst
Anton Ertl
>Suddenly I seem to understand this better.
>Is it true that
> EVALUATE
>is equivalent to
> ' INTERPRET EXECUTE-PARSING
>?
: evaluate ( ... addr u -- ... ) \ core,block
\G [... documentation ...]
['] interpret execute-parsing ;
- anton
Andrew Haley
> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>>> How much effort is it? With such a statement, you surely have a
>>>>> number for that.
>>>>
>>>>How many "not that hard"s are there in a "big deal effort", I wonder.
>>>
>>> Please answer the question.
>>
>>What for?
>
> So that others can come to their own judgement of whether it is worth
> the effort. E.g., if you think it takes changes in ten lines, but
> that it's not worth that effort, your statement is more about what you
> think the value of the change is, whereas if you think it takes
> changes in 10000 lines, it's more a statement about what you think the
> effort is.
something like
IF {: a b :} else a . THEN
compiling. i.e. it's ten more lines and it's still not really right.
>>> Ah, ok. It's not that it is "so much effort", it's that it is any
>>> effort at all, i.e., this is just a restatement of your dislike of
>>> locals.
>>
>>Not exactly, no. It's not dislike per se: as I've said before, locals
>>seem to me to be associated with unfactored code, and the more they
>>are used the worse that is. Therefore, any syntax which postively
>>encouarages an "all or nothing" approach to locals will, in the main,
>>make code worse.
>
> How does allowing defining locals in multiple places encourage an
> all-or-nothing approach? Requiring that all locals are defined in one
> definition seems to be much more all-or-nothing to me.
seems to encourage the use of locals where not needed. But any
further locals syntax is just more locals syntax, IMO. We've got too
much of it already and the Basic Principle of Forth is "Keep it
simple!":
"As the number of capabilities you add to a program increases, the
complexity of the program increases exponentially. The problem of
maintaining compatibility among these capabilities, to say nothing of
some sort of internal consistency in the program, can easily get out
of hand. You can avoid this if you apply the Basic Principle. You may
be acquainted with an operating system that ignored the Basic
Principle.
"It is very hard to apply. All the pressures, internal and external,
conspire to add features to your program. After all, it only takes a
half-dozen instructions, so why not? The only opposing pressure is the
Basic Principle, and if you ignore it, there is no opposing pressure."
[1]
The most important consequence of this princple is that the burden of
evidence is *on the person proposing an addition* to Forth to show
that there is a great advantage. But not every Forth user agrees with
the Basic Principle.
>>> Well, to anyone skilled in the art code redundancy in is bad.
>>
>>I absolutely agree, but it's the lesser of two evils. IMO.
>
> Anyone agreeing with you has the option of not using the "--" part in
> such cases.
code using more locals. And, horrors, people coming to Forth and
thinking that's what it's supposed to look like.
>>>>A locals declaration is not a
>>>>stack comment, and in this form it encourages people to put all args
>>>>in locals, even those which need no local. This is very bad.
>>>
>>> Why?
>>
>>This should surely be obvious. People are being encouraged to put
>>items in locals, even when there is no need to do so.
>
> Why is that "very bad"?
item into a local and back onto the stack, where that is only needed
because of the locals-as-stack-comment syntax, is a bad thing? Why
adding more work into a definition is a bad thing?
>>>>> Yes, in this example one can do without the inner locals relatively
>>>>> easily, because there are only two cells permanently alive in the
>>>>> loop. No need for any factoring:
>>>>
>>>>Well-factored words are the goal, not something to be avoided!
>>>
>>> Sure, but your version is not *well*-factored code.
>>
>>Perhaps not, but you are rather avoiding the point. You are saying
>>"there is no need for factoring", and though factoring is a scarce
>>resource that might be used up, or something to avoid. I don't know
>>why you are talking like that.
>
> If the factoring is so bad that the code without factoring is better
> than the code with factoring (as in this case), then yes, this
> factoring is something to avoid.
> But actually, this was not a comment about the desirability of
> factoring in general, but about the properties of the specific word at
> hand.
specific word is pointless.
But, IMO, any word which is much longer than than the magical "seven,
plus or minus two" words may well be in need of some attention. There
are exceptions to this, especially low-complexity words which are just
lists of items with no control structures.
>>> The others do, although I mentioned them here to show that the claims
>>> of the people who write "factoring makes locals unnecessary" are just
>>> hot air.
>>
>>But we all know how this game works: more and more extreme examples
>>until people give up from boredom or exhaustion.
>
> You should not attribute your discussion style to others.
points by challenging people to solve problems and then declaring
those people to be full of "hot air" when they failed to do so. (But
perhaps I have forgotten doing so; I've been here a long time, and my
behaviour has not always been what I aspire to.)
>>> What I find interesting in the original MAP-ARRAY, though, is that,
>>> for such a simple word, the restrictions on combining the return stack
>>> and DO loops make it really cumbersome to write such a word without
>>> locals.
>>
>>That is true, and is a real disadvantage. Forths I have used have
>>allowed access to the third item on the return stack, but even if not
>>it's possible to add a primitive to do so.
>
> In Gforth you could use J for that. But preferring to use such
> system-specific and not easily comprehensible code elevates the
> avoidance of locals to an end in and of itself.
>>That is true. Passing XTs is, in general, something I would avoid
>>unless the XT really varies at runtime. So, I prefer
>>
>> 0 foo n <map-array + map-array>
>>
>>to
>>
>> 0 foo n ['] + map-array
>
> Yes, in this case this works nicely, but
>
> 1) It cannot be used interpretively, which is not so nice.
do. Throwaway definitions cost nothing.
> 2) Xt-passing has turned out to be a useful factoring technique.
> E.g., consider SAVE-INPUT RESTORE-INPUT with all the many ways that
> RESTORE-INPUT can fail, compared to properly nesting words like
> EVALUATE. Now EVALUATE always INTERPRETs the string, so for doing
> something else we have EXECUTE-PARSING ( ... xt -- ... ). And that
> principle has been applied to other problems.
not appropriate for more general use.
Andrew.
[1] From _The Evolution Of Forth_
Rod Pemberton
wrote:
> If you want "implicitely initialized", [...]
>
> Bernd Paysan
> "If you want it done right, you have to do it yourself"
Well, you're still doing it wrong.
s/implicitely/implicitly
So, I guess that "you" excludes you and means "other people".
Rod Pemberton
Rod Pemberton
<ge...@jackson9000.fsnet.co.uk> wrote:
> You shouldn't ascribe machiavellian motives to everything that someone
> does. After all Stephen Pelc is not listed as a (presumably voting)
> member of the ANS Forth Technical Committee, only as a contributor. I
> prefer the cockup side of the cockup vs conspiracy theory (look up
> Hanlon's Razor that I hadn't previously heard of before. Off topic: as
> an aside that led me to Sturgeon's Law which I like).
>
and Dunning-Kruger effect are far more interesting.
Dunning-Kruger has come up here in the past.
Hopefully, Hugh and Bernd may read them some day:
http://en.wikipedia.org/wiki/Dunning–Kruger_effect
http://en.wikipedia.org/wiki/Peter_Principle
Rod Pemberton
Anton Ertl
>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>>>>>Anton Ertl <an...@mips.complang.tuwien.ac.at> wrote:
>>>>>> Andrew Haley <andr...@littlepinkcloud.invalid> writes:
>something like
>
> IF {: a b :} else a . THEN
>
>compiling. i.e. it's ten more lines and it's still not really right.
correct programs, and also allows you to shoot yourself in the foot.
Compared to the "one definition with uninitialized locals" approach it
does not miss anything; E.g., if I transcribe the example above for
this approach, I get:
{: | a b :} IF TO b TO a ELSE a . THEN
and that would also be accepted by the compiler, and the code would be
longer, and the programmer needs to reverse the local names for the
TOs.
If you reverse the branches of the IF structure above, then the
define-on-initialization approach would not compile, while the
define-uninitialized approach still would. But the error checking is
not the reason to go for such an implementation; the reason to go for
such an implementation is so that you can run programs written in that
style, and to allow easier reading and writing of programs: No
redundancy of having to mention the locals both in the definition and
in the initialization, and no reversal of the initialization.
If you want to error-check the program, you can do a more elaborate
implementation, or you can just compile the program on a Forth system
with such an implementation, like Gforth (where the scoping takes 65
SLOC, plus maybe some prerequisistes).
BTW, "changes in ten lines" does not mean "ten more lines".
>But any
>further locals syntax is just more locals syntax, IMO. We've got too
>much of it already and the Basic Principle of Forth is "Keep it
>simple!":
>
>"As the number of capabilities you add to a program increases, the
>complexity of the program increases exponentially. The problem of
>maintaining compatibility among these capabilities, to say nothing of
>some sort of internal consistency in the program, can easily get out
>of hand. You can avoid this if you apply the Basic Principle. You may
>be acquainted with an operating system that ignored the Basic
>Principle.
>
>"It is very hard to apply. All the pressures, internal and external,
>conspire to add features to your program. After all, it only takes a
>half-dozen instructions, so why not? The only opposing pressure is the
>Basic Principle, and if you ignore it, there is no opposing pressure."
>[1]
>
>The most important consequence of this princple is that the burden of
>evidence is *on the person proposing an addition* to Forth to show
>that there is a great advantage. But not every Forth user agrees with
>the Basic Principle.
arrive at colorForth in my Forth work, so I obviously don't agree with
it. In my view the real opposing forces are implementation and
maintenance cost, so I don't need to invent a Basic Principle to
strive for simplicity. I then have to balance functionality, ease of
use etc. against complexity (and its consequents implementation and
maintenance cost).
In the present case, an implementation with define-on-initialization
is easier to use and (without scoping) has comparable (probably even
less) complexity than an implementation with define-uninitialized
locals.
>What I'm afraid of, of course, is more badly-factored Forth
>code using more locals. And, horrors, people coming to Forth and
>thinking that's what it's supposed to look like.
factored code without locals, and some of them produce nicely factored
code with locals. I have not done a statistical evaluation, but from
what I saw, there does not seem to be a correlation between use of
locals and bad factoring; I guess that I did not see it because the
cases where the factoring is so hard that you give up if you have the
easy way out (locals) are relatively rare, whereas those people who
factor badly factor all their code badly.
>>>>>A locals declaration is not a
>>>>>stack comment, and in this form it encourages people to put all args
>>>>>in locals, even those which need no local. This is very bad.
>
>Because it's unnecessary. Do I have to explain why moving a stack
>item into a local and back onto the stack, where that is only needed
>because of the locals-as-stack-comment syntax, is a bad thing?
>Why
>adding more work into a definition is a bad thing?
A simple example is:
: bla {: a b c d -- :}
d . a . b . c . ;
Now one could write instead:
: bla ( a b c d -- )
. {: a b c :} a . b . c . ;
Is this an improvement? The source code certainly is longer and has
more tokens.
>> But actually, this was not a comment about the desirability of
>> factoring in general, but about the properties of the specific word at
>> hand.
>
>Right. Which is what I am trying to get away from; arguing about a
>specific word is pointless.
this case you pointed out that that word could be factored, and I
pointed out that the data flow in this example is not convoluted
enough to really need locals in the loop, and, I think, not anywhere;
which means that it is probably not a good example of locals use.
Of course, in the past you have often made claims in the abstract, and
any refutation of such a claim was answered by saying that I had
misrepresented your claim (of course without making your claims any
more concrete, just to keep them unrefutable). That's what I consider
>But, IMO, any word which is much longer than than the magical "seven,
>plus or minus two" words may well be in need of some attention.
As for longer definitions, your strcmp factoring example demonstrates
the folly of such a pure numbers-based factoring approach: The
resulting factors make no sense on their own, and you introduce an
additional IF structure that makes understanding the whole more
difficult.
>>>> The others do, although I mentioned them here to show that the claims
>>>> of the people who write "factoring makes locals unnecessary" are just
>>>> hot air.
>>>
>>>But we all know how this game works: more and more extreme examples
>>>until people give up from boredom or exhaustion.
>>
>> You should not attribute your discussion style to others.
>
>I think you're being unfair: I can't remember ever trying to score
>points by challenging people to solve problems and then declaring
>those people to be full of "hot air" when they failed to do so. (But
>perhaps I have forgotten doing so; I've been here a long time, and my
>behaviour has not always been what I aspire to.)
was one of moving the goalposts. That's not quite what you did;
instead, you made the goalposts sufficiently undefined that for any
shot at them, you could then claim that the goal was not there.
What you described here is reacting to a claim by challenging the
claimer to demonstrate the claim on an example. I don't see anything
unfair about that in general (although some may use it in unfair
ways).
>>>That is true. Passing XTs is, in general, something I would avoid
>>>unless the XT really varies at runtime. So, I prefer
>>>
>>> 0 foo n <map-array + map-array>
>>>
>>>to
>>>
>>> 0 foo n ['] + map-array
>>
>> Yes, in this case this works nicely, but
>>
>> 1) It cannot be used interpretively, which is not so nice.
>
>Shrug. I think that other people worry about that much more than I
>do. Throwaway definitions cost nothing.
significant for me. And I guess, for others, too, or we would not
have such a capable interpretation mode as we have (with all the
headaches about interpretation semantics or state-smartness, etc.).
If words such as <map-array ... map-array> were more frequent, maybe
we should have a way to put the rest of the line in a colon definition
and execute it and forget it right away, like some non-standard Forth
systems have. Say:
% 0 a 5 <map-array + map-array> .
Just Ian
> Well, you're still doing it wrong.
>
> s/implicitely/implicitly
Ian