Forth without threads (4tH V2.6c)
Bernd Paysan
hbez...@vsngroep.nl (Hans Bezemer) wrote:
> 8. Epilogue
> -----------
> I hope I've proved that the Forth-concept can be stretched beyond the
> traditional threaded implementation. I still have a few other ideas,
> allowing the Forth-concept to enter the realms of native code-generation
> without the hassle of a metacompiler. May be I'll work these out someday.
> For now, you've got to do with 4tH.
You are not the first. Your system is "token-threaded", it is not a
traditional threaded code (address-threaded). It does not use an interactive
compiler (which is IMHO one of the basic strength of FORTH). And your aims
at native code generation: There are a number of NC Forths out there, even
for Intel. There is TCOM (which does not contain an interactive compiler),
and for 32 bit iForth (Marcel Hendrix, nl), and bigFORTH (from me), both
full compliant ANS FORTH systems, not only metacompilers. The main reasons
for both being 32 bit is that both were implemented on another 32-bit
architecture before (iForth on a transputer, called tForth, and bigFORTH
on an 68k Atari ST), and because 16 bit is brain-dead (your 4tH is 32 bit,
too; good).
Threading is an implementation technique, and traditional Forth uses
threading. It is by no way bound to Forth itself (except by outdated
standards, as FORTH-83). As it is simple and powerful, it is still
first joice.
--
Bernd Paysan
"Late answers are wrong answers!"
http://www.informatik.tu-muenchen.de/~paysan/
Lutz Sammer
Johns
Hans Bezemer
>In article <DF5A7...@inter.NL.net>,
>hbez...@vsngroep.nl (Hans Bezemer) wrote:
>> 8. Epilogue
>> -----------
>> I hope I've proved that the Forth-concept can be stretched beyond the
>> traditional threaded implementation. I still have a few other ideas,
>> allowing the Forth-concept to enter the realms of native code-generation
>> without the hassle of a metacompiler. May be I'll work these out someday.
>> For now, you've got to do with 4tH.
>You are not the first. Your system is "token-threaded", it is not a
>traditional threaded code (address-threaded). It does not use an interactive
>compiler (which is IMHO one of the basic strength of FORTH). And your aims
>at native code generation: There are a number of NC Forths out there, even
>for Intel. There is TCOM (which does not contain an interactive compiler),
>and for 32 bit iForth (Marcel Hendrix, nl), and bigFORTH (from me), both
>full compliant ANS FORTH systems, not only metacompilers. The main reasons
>for both being 32 bit is that both were implemented on another 32-bit
>architecture before (iForth on a transputer, called tForth, and bigFORTH
>on an 68k Atari ST), and because 16 bit is brain-dead (your 4tH is 32 bit,
>too; good).
Lately I've done some more research and I found e.g. ForthCMP which seems to
work from the same model. I already wondered why nobody ever SEEMED to think
along the same lines. I know the iForth compiler pretty well (never knew it
worked from the same model als 4tH) and the performance is amazing. Although
Forth in its "classical" implementation is a very very interesting concept I
think the language Forth could benefit more from the more traditional compiler
approach as 4tH, ForthCMP and others you mentioned, since it produces blazingly
fast and compact programs.
>Threading is an implementation technique, and traditional Forth uses
>threading. It is by no way bound to Forth itself (except by outdated
>standards, as FORTH-83). As it is simple and powerful, it is still
>first joice.
OK, but I think it keeps Forth from entering the real world. In my view Forth
should use the traditional compiler concept and be very very portable. The
ANS-Forth concept of Forth could be useful here by supporting both
architectures and make a concept for address-threaded Forth and token-threaded
Forth. Core-words should be able to function in both architectures and most
compiler words in Address-Threaded Extensions and compiler directives in
Token-threaded Extensions. And please discard the Forth-83 concept all
together. The implementation of DO..LOOP is ghastly and not very transparent to
the programmer, for instance (IMHO).
Hans.
Bruce McFarling
[among other things]
> > Your system ... does not use an interactive compiler (which is IMHO
> > one of the basic strength of FORTH). ...
On Fri, 22 Sep 1995, Hans Bezemer wrote [among other things]:
> Although Forth in its "classical" implementation is a very very
> interesting concept I think the language Forth could benefit more
> from the more traditional compiler approach as 4tH, ForthCMP and others
> you mentioned, since it produces blazingly fast and compact programs.
It only produces blazingly fast and compact programs *if* an
appropriate algorythm is being used. How do we find out if an appropriate
algorythm is being used? On a general level, the answer is very simple:
*exploration*. I do fail to see the point of FORTH if I am going to be
chained to an edit - compile - compile error - edit - compile - runtime
error - debug - edit - compile - runtime error development cycle. The
whole process discourages developing in small, byte size pieces that you
can verify are in fact working as intended before you throw them together
in a system.
Now, *if* you add something to the language that facilitates
compiling collections of words after they have been well and truly tested,
used, played with, alternatives tried, etc., so that the result runs more
efficiently (smaller and/or faster) and enjoys the same interface with the
creative learning process, fine. But as a not-professional programmer,
who has sometimes spent weeks writing programs in C that only take a
couple of minutes to run, if you want to take the problem that drove me
from C back to Forth, and bring that problem from C into Forth, thanks,
but no thanks.
Virtually,
Bruce McFarling, Knoxville
br...@utkux1.utk.edu
Tom Almy
>Enjoy, and comments are still greatly appreciated!!
Just a note that I wrote a native code compiler for colon definitions within
an interpreted Forth environment back in 1982. Written for the Z80 at the
time, current 16 and 32 bit 80x86 versions are part of every Laboratory
Microsystems Forth.
I also have a Forth compiler without an interpreted environment, the Shareware
program ForthCMP, for the 83 and ANS standards. This is a one pass compiler
that is blazingly fast both in compilation and in execution. It has been
available for 11 years.
--
Tom Almy -- tom....@tek.com
Standard Disclaimers Apply, including:
"Any ideas or opinions expressed here do not necessarily
reflect the ideas or opinions of my employer."
Elizabeth D. Rather
>OK, but I think it keeps Forth from entering the real world. In my view Forth
>should use the traditional compiler concept and be very very portable. The
>ANS-Forth concept of Forth could be useful here by supporting both
>architectures and make a concept for address-threaded Forth and token-threaded
The TC tried very hard to create a standard that is implementation-independent.
Many members had direct experience with a variety of implementation techniques.
As much as we may debate what Forth _is_, the TC at least was clear that it is
not defined by its implementation!
Marcel Hendrix
(..)
> I know the iForth compiler pretty well
> (never knew it worked from the same model als 4tH)
Well, if "model" here means: like a Pascal or C-compiler, not having
a real interpreter, then it doesn't.
iForth's compiler-phase is different from the standard approach in
the sense that real machine code is produced, and that the result thereof
is optimized as good as a one-pass compiler can do it. The basic Forth
model is 'subroutine-threading', but most standard words are just compiled
inline (macros). On the macro edges the compiler eliminates parameter
pushes and pops. Forth flags are converted to machine status register flags.
Some common sequences of machine code bytes are recognized and converted to
more optimal (faster) sequences (most Forth words involving literals). The
top of the parameter and floating-point stacks are kept in registers. All
stackpointers (parameter, return, system, floating-point and local) are kept
in registers. Returns and calls are converted to jumps into word bodies where
possible.
iForth could do much better than it does now if it used a multi-pass compiler.
However, this conflicts with having the full power of the interpreter
available. (I disagree with Bernd Paysan that the interpreter is a basic
strength of Forth, I think that without the interpreter Forth is plainly not
useful). The innards of iForth are relatively complicated because of this
requirement. For instance, I need several copies of e.g. DUP around. Only
one of them is executable from the keyboard, the others are optimizations
for specific circumstances. (Compiling in a buffer and then executing the
result only works 80% of the time). I want the user to see iForth as just
a very, very fast ANS Forth, no special treatment should be necessary, no
exceptions, no limitations, no gotcha's (I can hope, can I :-)).
Of course some aspects of Forth lead to very bad code ( LOOP +LOOP LEAVE ),
or require the compiler writer to be very conservative ( : foo R> R> @ SWAP
>R ; ). Nevertheless, standard iForth is as fast as BP 7.0 (some 1.5 to 2
times slower than C). 4THCMP is faster than C. Adding some inline machinecode
to iForth speeds it up to C levels (I did this for NSPICE's vector * matrix
routine).
( .. )
> Although Forth in its "classical"
> implementation is a very very interesting concept I think the
> language Forth could benefit more from the more traditional
> compiler approach as 4tH, ForthCMP and others you mentioned,
> since it produces blazingly fast and compact programs.
No, I don't agree (if you mean excluding or severely limiting
the interpreter). Fast, small code is not enough (small code is even
irrelevant these days, considering the amounts of data that will be handled
by even simple programs). You need the interpreter. It's fabulous. Wait till
your small, fast program needs a custom interpreter to read in a configuration
file... The Forth interpreter enables you to debug and experiment online,
not only to get the code bugfree, but to find the best algorithm!
For the two bigger projects I did in the past months (MIDI-based music
language, a SPICE-like circuit emulator) the interpreter proved absolutely
essential.
( .. )
> Hans.
Please do not consider this a flame. I just wanted to add some fuel to an
interesting discussion.
-marcel {m...@iaehv.nl}
Hans Bezemer
>In article <DF5A7...@inter.NL.net> hbez...@vsngroep.nl (Hans Bezemer) writes:
>>Enjoy, and comments are still greatly appreciated!!
>Just a note that I wrote a native code compiler for colon definitions within
>an interpreted Forth environment back in 1982. Written for the Z80 at the
>time, current 16 and 32 bit 80x86 versions are part of every Laboratory
>Microsystems Forth.
>I also have a Forth compiler without an interpreted environment, the Shareware
>program ForthCMP, for the 83 and ANS standards. This is a one pass compiler
>that is blazingly fast both in compilation and in execution. It has been
>available for 11 years.
Yes, other people set me on the very same track. I already wondered why nobody
else seemed to think along the same lines I did. I was impressed by your
products, the sheer speed of it. Shame this kind of Forth never broke through.
It's so easy to develop in forth. Make the low level routines, test them, build
some more. I try to use the very same technique in C, but I always have to use
a pretty bad main() in order to test them. And then the abundance of variables
you need! But C has one single advantage: it is so portable. If only Forth
would have had the same standardazation... Shame. ANS-Forth is just too late I
guess..
H.
Hans Bezemer
>hbez...@vsngroep.nl (Hans Bezemer) writes:
>(..)
>> I know the iForth compiler pretty well
>> (never knew it worked from the same model als 4tH)
>Well, if "model" here means: like a Pascal or C-compiler, not having
>a real interpreter, then it doesn't.
Sorry, my error!
>iForth's compiler-phase is different from the standard approach in
>the sense that real machine code is produced, and that the result thereof
>is optimized as good as a one-pass compiler can do it. The basic Forth
>model is 'subroutine-threading', but most standard words are just compiled
>inline (macros). On the macro edges the compiler eliminates parameter
>pushes and pops. Forth flags are converted to machine status register flags.
>Some common sequences of machine code bytes are recognized and converted to
>more optimal (faster) sequences (most Forth words involving literals). The
>top of the parameter and floating-point stacks are kept in registers. All
>stackpointers (parameter, return, system, floating-point and local) are kept
>in registers. Returns and calls are converted to jumps into word bodies where
>possible.
iForth was developed with other goals in mind than mine, that's a fact. You
took the standard Forth and optimized it internally as far as I understand.
Kind of hybrid?
>iForth could do much better than it does now if it used a multi-pass compiler.
>However, this conflicts with having the full power of the interpreter
>available. (I disagree with Bernd Paysan that the interpreter is a basic
>strength of Forth, I think that without the interpreter Forth is plainly not
>useful). The innards of iForth are relatively complicated because of this
>requirement. For instance, I need several copies of e.g. DUP around. Only
>one of them is executable from the keyboard, the others are optimizations
>for specific circumstances. (Compiling in a buffer and then executing the
>result only works 80% of the time). I want the user to see iForth as just
>a very, very fast ANS Forth, no special treatment should be necessary, no
>exceptions, no limitations, no gotcha's (I can hope, can I :-)).
Like I said, iForth was designed with other goals in mind than mine. Speed
seems to be the utmost design-goal. Nothing wrong about that. I just want to
make the point that in some areas a "classical" design might fit programmers
better than in fact adding a completely new OS on top of a host-OS. People want
to run their programs on their own on their own OS. That may have been a key
factor why Forth was never popular and for that reason BASIC was never seen as
a professional tool. I'm not talking about Forth-converts, but the rest of the
world.
>Of course some aspects of Forth lead to very bad code ( LOOP +LOOP LEAVE ),
>or require the compiler writer to be very conservative ( : foo R> R> @ SWAP
>>R ; ). Nevertheless, standard iForth is as fast as BP 7.0 (some 1.5 to 2
>times slower than C). 4THCMP is faster than C. Adding some inline machinecode
>to iForth speeds it up to C levels (I did this for NSPICE's vector * matrix
>routine).
If you still needed the overhead of the complete BP 7.0 environment to run your
application, I don't think it would have been too popular. In Forths classical
design, you don't have a choice.
>( .. )
>> Although Forth in its "classical"
>> implementation is a very very interesting concept I think the
>> language Forth could benefit more from the more traditional
>> compiler approach as 4tH, ForthCMP and others you mentioned,
>> since it produces blazingly fast and compact programs.
>No, I don't agree (if you mean excluding or severely limiting
>the interpreter). Fast, small code is not enough (small code is even
>irrelevant these days, considering the amounts of data that will be handled
>by even simple programs). You need the interpreter. It's fabulous. Wait till
>your small, fast program needs a custom interpreter to read in a configuration
>file... The Forth interpreter enables you to debug and experiment online,
>not only to get the code bugfree, but to find the best algorithm!
Are these situations you encounter every time time again, or exceptions?
>For the two bigger projects I did in the past months (MIDI-based music
>language, a SPICE-like circuit emulator) the interpreter proved absolutely
>essential.
Like I said, exceptions or rules? Do you really need this configuration of
Forth all the time. Most people are happily developing and running programs
without it. Still, Forth easy to compile and parse language can be an approach
that is suitable in some circumstances. I really don't need the complete
environment to be around all the time..
>( .. )
>> Hans.
>Please do not consider this a flame. I just wanted to add some fuel to an
>interesting discussion.
You're comments are greatly appreciated and I'm honored. I used parts of your
Benchmarks (published in vijgeblad (1993/2)) to bench 4tH. Great to hear from
you! BTW, I'm HCC member too (Unix/C).
H.
Hans Bezemer
>> pay...@informatik.tu-muenchen.de (Bernd Paysan) wrote:
> [among other things]
>> > Your system ... does not use an interactive compiler (which is IMHO
>> > one of the basic strength of FORTH). ...
>On Fri, 22 Sep 1995, Hans Bezemer wrote [among other things]:
>> Although Forth in its "classical" implementation is a very very
>> interesting concept I think the language Forth could benefit more
>> from the more traditional compiler approach as 4tH, ForthCMP and others
>> you mentioned, since it produces blazingly fast and compact programs.
> It only produces blazingly fast and compact programs *if* an
>appropriate algorythm is being used. How do we find out if an appropriate
>algorythm is being used? On a general level, the answer is very simple:
>*exploration*. I do fail to see the point of FORTH if I am going to be
>chained to an edit - compile - compile error - edit - compile - runtime
>error - debug - edit - compile - runtime error development cycle. The
>whole process discourages developing in small, byte size pieces that you
>can verify are in fact working as intended before you throw them together
>in a system.
On a multiscreen Unix this is not much of a problem. One screen holds 'vi' and
an occasional 'w' writes the source to disk again. In another I have the
compiler and runtime. The 4tH compiler is fast enough to compile the code in a
couple of seconds at the most, so I don't think that's much of a problem. We
will see more and more windowed applications and with fast enough compilers (C
is not fast on a Unix machine (quality of the compiler?)), so I develop under
MS-DOS using an old TC V2.0 compiler. Bottom up development is the very same
technique I'm using when developing C. I learned it BTW by using Forth. I wrote
a file-converter once and started off with the low-level routines. After 7
releases those stayed the same. All other stuff has been heavily altered.
> Now, *if* you add something to the language that facilitates
>compiling collections of words after they have been well and truly tested,
>used, played with, alternatives tried, etc., so that the result runs more
>efficiently (smaller and/or faster) and enjoys the same interface with the
>creative learning process, fine. But as a not-professional programmer,
>who has sometimes spent weeks writing programs in C that only take a
>couple of minutes to run, if you want to take the problem that drove me
>from C back to Forth, and bring that problem from C into Forth, thanks,
>but no thanks.
When I write a program I'm always keen on routines that are so general that I
could use them again. So I generalize them and add them to my library, which is
quite extensive now, thus extending the language itself. Well, but I don't have
to forget them or load them before executing a program. If they're needed,
they're there. The linker took care of that. When you program for weeks on a
program that runs a few minutes, did it the job for a person that would
otherwise have been manually perfomed it? If it took you more days to program
the thing than it would have taken you to do the job yourself, you're not doing
something quite right!
H.
Hans Bezemer
I'm genuinely happy to hear that! Although it is not the language I use for
heavy work, I like it a lot!
H.
Marcel Hendrix
> iForth was developed with other goals in mind than mine, that's
> a fact. You took the standard Forth and optimized it internally
> as far as I understand. Kind of hybrid?
No, it's a full ANS Forth. The 'new' standard allows to do what I did
without the user having to worry about it. It's transparent.
> Like I said, iForth was designed with other goals in mind than
> mine. Speed seems to be the utmost design-goal.
My three goals were: speed, flat-model address space (32-bit), and the
largest array of non-trivial examples and tools found with any Forth
(I'd love to be disproved here).
> I just want to make the point that in some areas a "classical" design
> might fit programmers better than in fact adding a completely new OS on
> top of a host-OS. People want to run their programs on their own on their
> own OS. That may have been a key factor why Forth was never popular and
> for that reason BASIC was never seen as a professional tool. I'm not
> talking about Forth-converts, but the rest of the world.
Wow, a new OS :-) I don't do that with iForth (but when iForth finally runs
on top of MMURTL...). The programmer has full access to the OS with
macros, so s/he can type cls, dir, copy, xcopy echo etc. exactly like from
the DOS-prompt. Any utility can be run from within iForth (this is the
way the program editor, file manager, etc. are invoked). When running from
Windows this isn't even necessary as you can open as many windows (DOS
boxes?) as you want. iForth can produce executables, however, these are
very large. They serve their purpose: hassle-free starting of canned
applications which are still online debuggable by the experts.
Bernd Paysan's BigForth does the same, and more, as you get a nice character
graphics window interface with it that works great and looks professional. I'm
not much of a WIMPy guy myself, but maybe when Bernd throws this stuff in
the PD sometime...
> If you still needed the overhead of the complete BP 7.0 environment to
> run your application, I don't think it would have been too popular. In
> Forths classical design, you don't have a choice.
I don't get it. A "classical" Forth would never be larger than 64 KBytes.
And what does the typical user with a 16 MByte Pentium 150 MHz gain when
I shrink the 192 Kbytes of a minimal iForth application to the 50 Kbytes a
smart-linked BP 7.0 solution takes? (extrapolating to the end of '96 when
I might be able to deliver such a minimizing target compiler).
>> The Forth interpreter enables you to debug and experiment online,
>>not only to get the code bugfree, but to find the best algorithm!
>
> Are these situations you encounter every time time again, or exceptions?
Debugging: always. Experimenting: for any non-trivial program (For trivial
programs I use Pascal, TCOM or assembler). I can understand the question
as I developed the experimenting-with-algorithm approach when I became
somewhat better with Forth and acquired an extensive set of 32-bit tools.
In Algol/Pascal/BASIC/assembler I always gladly stop when my programs
finally work.
In Forth you don't build programs, but sets of words to solve a specific
problem. This generally means that a "program" (word set) can be used to
help solve other problems with _very minor_ changes. This is hard to explain
and maybe impossible to believe for somebody who has never experienced what
a customized monster Forth environment will do for its creator :-)
>>For the two bigger projects I did in the past months (MIDI-based music
>>language, a SPICE-like circuit emulator) the interpreter proved absolutely
>>essential.
>
> Like I said, exceptions or rules? Do you really need this configuration
> of Forth all the time. Most people are happily developing and running
> programs without it. Still, Forth easy to compile and parse language can
> be an approach that is suitable in some circumstances. I really don't
> need the complete environment to be around all the time..
The exception is when I _don't_ need it. Ten years ago I didn't need it at all.
So I suppose I adapted to the tools (when you have a hammer ..).
-marcel
Bernd Paysan
m...@iaehv.IAEhv.nl (Marcel Hendrix) wrote:
> However, this conflicts with having the full power of the interpreter
> available. (I disagree with Bernd Paysan that the interpreter is a basic
> strength of Forth, I think that without the interpreter Forth is plainly not
^^^^^^
That's what I wanted to say (so no need to disagree ;-). BTW without
interpreter, a Forth can't be ANS Forth.